LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 06-24-2004, 05:52 PM   #1
versaulis
Member
 
Registered: Sep 2003
Posts: 139

Rep: Reputation: 15
Please Help! How Do I Download A File Over 4 Gb !!!?


I can't find a Window OR EVEN LINUX program that will download files over 4 GB!!! HELP!!! They all download the first 300 MB and quit on me!!!
 
Old 06-24-2004, 05:55 PM   #2
david_ross
Moderator
 
Registered: Mar 2003
Location: Scotland
Distribution: Slackware, RedHat, Debian
Posts: 12,047

Rep: Reputation: 64
Try wget:
wget -c http://www.somesite.com/file.ext
 
Old 06-24-2004, 07:17 PM   #3
versaulis
Member
 
Registered: Sep 2003
Posts: 139

Original Poster
Rep: Reputation: 15
didn't work
 
Old 06-24-2004, 07:37 PM   #4
bzsleeping
LQ Newbie
 
Registered: Jun 2003
Location: at my desk.. sleeping
Distribution: Red Hat 9.0 with latest kernel.. i think :p
Posts: 27

Rep: Reputation: 15
how do u download the file? where's the file located?
 
Old 06-24-2004, 11:09 PM   #5
amosf
Senior Member
 
Registered: Jun 2004
Location: Australia
Distribution: Mandriva/Slack - KDE
Posts: 1,672

Rep: Reputation: 46
wget will at least get more than 300 meg :-) Must be something wrong with the connection???

If wget drops out you can restart it with wget -c and it will start from where it left off...

Also what filesystem are you putting it on. Some are limitted to 2gig file size if not created with large filesize in mind...
 
Old 06-25-2004, 03:32 AM   #6
J.W.
LQ Veteran
 
Registered: Mar 2003
Location: Milwaukee, WI
Distribution: Mint
Posts: 6,642

Rep: Reputation: 69
Dude - what the heck kind of file is over 4G, other than pirated movies and the like? Maybe I'm overlooking something obvious, but even with a fast connection, moving this much data would still take a great deal of time, and at least for me, the need to perform a 4G transfer sounds mighty unusual. -- J.W.
 
Old 06-25-2004, 07:06 AM   #7
versaulis
Member
 
Registered: Sep 2003
Posts: 139

Original Poster
Rep: Reputation: 15
Alright people, here is the info...

What I'm trying to download are two unofficial debian DVD disks for their Sid distro. I've got a link to download these disks and 5 mirrors. The first goes over the 4 GB threshold and therefore most/all programs will not download it if the server will tell the software what the size of the file is. I got away with downloading Fedora Core 1 DVD from an ftp server that wouldn't tell my client what the file's size was, but all my sources for this download tell my client what the size is.

Why does wget and other software only download about 300 MB? I know why --
When programmers make software, they use variables (much like in math) to store values. These variables have different types. Some store characters, some store numbers, etc. The point is that these variables have limits. For example, an "integer" can store values from *about* -32000 to 32000 or so. A "unsigned long integer" on the other hand, can store a number from 0 to 4294967295, which is 4 GB. Most programmers use this unsigned long integer to store the size of the files I'm trying to download. So what happens if I try to download a files that has size, 4609540096 bytes? Simple, overflow occurs. Basically, the computer goes from 4294967295 to zero and then starts counting up from there... so it basically does a (4609540096 - 4294967295) which equals 314572801 which is 300 MB. The download client then downloads 300 MB and quits thinking it did it's job.

There is nothing wrong with the connection. wget download the exact same number of bytes that all the other software did and quit. wget is the problem.

I'm downloading from a college LAN where I get over 1 MB per second download speed (and yes, these mirrors give me that speed).

The mirrors for this debian DVD download are as follows:

ftp://ftp.proxad.net/mirrors/ftp.fsn...ficial/sid-dvd
http://mirrors.sec.informatik.tu-dar...ficial/sid-dvd
http://mirror.switch.ch/ftp/mirror/d...ficial/sid-dvd
http://debian.linux.org.tw/debian-unofficial/sid-dvd
ftp://ftp.kfki.hu/pub/linux/cdimages...ficial/sid-dvd
ftp://ftp.fsn.hu/pub/CDROM-Images/de...ficial/sid-dvd


My problem is that I cannot find *any* software in windows or linux that will download that first dvd, sid-i386-1.iso. The second one will download, but not the first. I'm almost ready to go find some source code and change all the file size variables to long double.

Does anyone know of any software that will download this file? Under linux I tried wget, d4x, ftp (command line), and some other stuff I can't remember right now...
 
Old 06-25-2004, 08:43 AM   #8
linuxxed
Member
 
Registered: Feb 2004
Posts: 273

Rep: Reputation: 30
Re: Please Help! How Do I Download A File Over 4 Gb !!!?

Quote:
Originally posted by versaulis
I can't find a Window OR EVEN LINUX program that will download files over 4 GB!!! HELP!!! They all download the first 300 MB and quit on me!!!

Simple. Use a "range" header and progressively download. Or use a download manager that supports download-resume.

An example:

http://aspn.activestate.com/ASPN/Coo...n/Recipe/83208
 
Old 06-25-2004, 08:57 AM   #9
versaulis
Member
 
Registered: Sep 2003
Posts: 139

Original Poster
Rep: Reputation: 15
Great! I'll be able to download the first 4 GB but not the last 300 MB! WOOT!

I can't specify that last 300 MB even using the range method. And software that downloads files in parts do not read the file size correctly either. They download the first 300 MB like everything else.
 
Old 06-25-2004, 09:11 AM   #10
linuxxed
Member
 
Registered: Feb 2004
Posts: 273

Rep: Reputation: 30
Quote:
Originally posted by versaulis
Great! I'll be able to download the first 4 GB but not the last 300 MB! WOOT!

I can't specify that last 300 MB even using the range method. And software that downloads files in parts do not read the file size correctly either. They download the first 300 MB like everything else.

Some things for you to check. Are there any filesize limitations (max bytes/file) of your system? ulimit ?

If yes then you need to clear that. Second thing to check if you've got enough free space.

What's wrong with the mozilla download manager?
 
Old 06-25-2004, 09:57 AM   #11
versaulis
Member
 
Registered: Sep 2003
Posts: 139

Original Poster
Rep: Reputation: 15
Quote:
Originally posted by linuxxed
Some things for you to check. Are there any filesize limitations (max bytes/file) of your system? ulimit ?

If yes then you need to clear that. Second thing to check if you've got enough free space.

What's wrong with the mozilla download manager?


I have no file size limitations and mozilla doesn't know how to download files in excess of 4 GB either. I have about 40 GB of free space to download into.
 
Old 06-25-2004, 11:07 AM   #12
Ebel
Member
 
Registered: May 2004
Distribution: Slackware 10
Posts: 63

Rep: Reputation: 15
If many programmes have this problem then the debian people probably would have been affected, so maybe they might know a way around it. Have you asked them?
 
Old 06-25-2004, 11:27 AM   #13
versaulis
Member
 
Registered: Sep 2003
Posts: 139

Original Poster
Rep: Reputation: 15
Quote:
Originally posted by Ebel
If many programmes have this problem then the debian people probably would have been affected, so maybe they might know a way around it. Have you asked them?

I don't think debian officially distributes DVD images since there are not many people that have DVD burners yet.
 
Old 06-25-2004, 03:17 PM   #14
bulliver
Senior Member
 
Registered: Nov 2002
Location: Edmonton AB, Canada
Distribution: Gentoo x86_64; Gentoo PPC; FreeBSD; OS X 10.9.4
Posts: 3,760
Blog Entries: 4

Rep: Reputation: 77
Read this:
http://www.debian.org/distrib/cd

In particular:
Quote:
Note that jigdo is the only way to download Debian DVD images.
Jigbo splits the download into many (I guess we can assume 300MB) chunks and assembles them afterwards.
 
Old 06-25-2004, 09:25 PM   #15
versaulis
Member
 
Registered: Sep 2003
Posts: 139

Original Poster
Rep: Reputation: 15
Quote:
Originally posted by bulliver
Read this:
http://www.debian.org/distrib/cd

In particular:


Jigbo splits the download into many (I guess we can assume 300MB) chunks and assembles them afterwards.

Well, I guess that solves the riddle. Thank you very much! I'm running Jigdo right now.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Fast Download Site for Linux, enables >4GB Download of single file TigerLinux Linux - Distributions 9 10-29-2005 12:45 PM
Can I download a file after a pause? anindyanuri Linux - Software 8 03-31-2005 08:23 AM
Konqueror wont download download TAR,ZIP file but preview Velocide Linux - Newbie 0 03-22-2004 06:07 AM
*.h file download.. eye Linux - Software 1 06-03-2003 04:17 AM
how do i download this file? doublefailure Linux - General 4 09-01-2002 06:36 AM


All times are GMT -5. The time now is 10:12 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration