LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 03-13-2007, 07:44 AM   #1
Hacker X
Member
 
Registered: Jun 2006
Location: Urbana, Illinois, US
Distribution: Zenwalk, Vector
Posts: 76

Rep: Reputation: 15
Question Incomplete bz2 download


Trying to download a darcs.bz2 file over a modem,
the download keeps stopping before it's done.

The file is supposed to be 1.6M, but it came out
334K and 436K on two different tries.

The starting point is here:
http://darcs.net/DarcsWiki/CategoryBinaries

Click "Linux v1.08"
=> http://evan.martin.googlepages.com/home

Click "darcs v1.08, built ..." and it starts to download,
but just gives up and exits early.
bunzip2 says the file is corrupt.

??

Last edited by Hacker X; 03-13-2007 at 07:45 AM.
 
Old 03-13-2007, 07:54 AM   #2
nx5000
Senior Member
 
Registered: Sep 2005
Location: Out
Posts: 3,307

Rep: Reputation: 52
Here I managed to get the file.

Try a more robust downloader, maybe wget for command line:

wget http://evan.martin.googlepages.com/darcs.bz2
 
Old 03-13-2007, 07:54 AM   #3
Junior Hacker
Senior Member
 
Registered: Jan 2005
Location: North America
Distribution: Debian testing Mandriva Ubuntu
Posts: 2,687

Rep: Reputation: 59
I would'nt trust anything with the name "martin" in it.
 
Old 03-13-2007, 09:02 AM   #4
Hacker X
Member
 
Registered: Jun 2006
Location: Urbana, Illinois, US
Distribution: Zenwalk, Vector
Posts: 76

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by nx5000
Try a more robust downloader, maybe wget for command line
Holy heaps of halibut! 5.1 attempts and it finally finished.
WTF is going on there? I've never seen that before.

PS. Thanks, that worked great.

PPS. JH: Why not?
 
Old 03-13-2007, 08:38 PM   #5
Junior Hacker
Senior Member
 
Registered: Jan 2005
Location: North America
Distribution: Debian testing Mandriva Ubuntu
Posts: 2,687

Rep: Reputation: 59
Just a cheap shot

Elbow in the side of the ribs type comment.
Actually there are many many Martins around, very popular name, have not found an extremely bad egg in the bunch yet.

Glad you got your download.
 
Old 10-27-2007, 07:12 PM   #6
Hacker X
Member
 
Registered: Jun 2006
Location: Urbana, Illinois, US
Distribution: Zenwalk, Vector
Posts: 76

Original Poster
Rep: Reputation: 15
Question

Quote:
Originally Posted by Hacker X View Post
Originally Posted by nx5000
> Try a more robust downloader, maybe wget for command line
Thanks, that worked great.
Okay, here's a harder problem. This has happening a couple/few times over the past couple years. Downloading a file over the dialup connection, the download just stops, even with wget. I think wget always stops at the same spot for any given file as Firefox does. The process doesn't exit, it just sits there not doing anything visible. In this case, Firefox and wget both stopped just two SECONDS shy of completing another bz2 file

It's not necessarily the file format though. I think I once had the same problem with a pdf file.

Last edited by Hacker X; 10-27-2007 at 07:20 PM.
 
Old 10-27-2007, 07:27 PM   #7
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
As far as firefox when downloading, or wget is concerned it is just a sequence of bytes - you are quite correct that it doesn't have anything to do with the file type.

I remember with horror the days of dialup - always dropping connections at the worst possible time. I found that the biggest two factors governing reliability of dialup service were phone line quality and service provider... it takes a while to find a combination which works. I had a line from comcast and no where I dialled to, the connection would drop periodically. Make sure you have call waiting turned off, and any other service which interjects in a connected call.

With wget you can use the --continue option to complete a partial download, and you can use the --tries option to specify a number of times to retry.
 
Old 10-28-2007, 04:01 AM   #8
Hacker X
Member
 
Registered: Jun 2006
Location: Urbana, Illinois, US
Distribution: Zenwalk, Vector
Posts: 76

Original Poster
Rep: Reputation: 15
Thumbs up

Quote:
Originally Posted by matthewg42 View Post
As far as firefox when downloading, or wget is concerned it is just a sequence of bytes - you are quite correct that it doesn't have anything to do with the file type.
I've heard, I have no recollection where, that the dialup modem has a certain bit sequence that it interprets as a command to terminate the transmission. The only filetype issue I can think of is that maybe some formats don't contain that special code, if it exists, while others do. That's just desperate speculation, though. I have no idea for sure.

Quote:
I remember with horror the days of dialup - always dropping connections at the worst possible time.
The connection is fine, except for this download (and the poky speed).

Great galloping gefiltefish[1], a miracle seems to have occurred. There's a .tgz version of the file I'm trying to download, so I tried that one with wget. Firefox didn't appear to do anything with that file. It would just sit there with the Firefox tab "Loading" for possibly upwards of an an hour or more, until my ISP cut off my connection, without even displaying the download popup window.

wget at least displayed its progress, which was much more reassuring. At the 99% point, it also stopped, so for a while there, all hope appeared to be lost. However, after sitting there for another half hour or so, it eventually retried. Here's the transcript:

Code:
dan[python]$ wget http://www.python.org/ftp/python/2.5.1/Python-2.5.1.tgz
--02:41:04--  http://www.python.org/ftp/python/2.5.1/Python-2.5.1.tgz
           => `Python-2.5.1.tgz'
Resolving www.python.org... 82.94.237.218
Connecting to www.python.org|82.94.237.218|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11,060,830 (11M) [application/x-tar]

99% [========================================================> ] 11,046,419    --.--K/s    ETA 00:04

03:35:57 (3.28 KB/s) - Read error at byte 11046419/11060830 (Connection timed out). Retrying.

--03:35:58--  http://www.python.org/ftp/python/2.5.1/Python-2.5.1.tgz
  (try: 2) => `Python-2.5.1.tgz'
Connecting to www.python.org|82.94.237.218|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 11,060,830 (11M), 14,411 (14K) remaining [application/x-tar]

100%[+++++++++++++++++++++++++++++++++++++++++++++++++++++++++>] 11,060,830     4.93K/s

03:36:02 (4.92 KB/s) - `Python-2.5.1.tgz' saved [11060830/11060830]
wget wins again! (checksum is correct) So I guess this thread is now a matter of curiosity rather than panic. It would be nice for future reference to know where the culprit is, now that we know it applies to both .bz2 and .tgz formats.

--Hacker X (a.k.a. "Dan")
www.prairienet.org/~dsb/

[1] My 2nd gratuitous fish reference on this thread, apparently caused by wget's awesome performance.

Last edited by Hacker X; 10-28-2007 at 04:31 AM.
 
Old 10-28-2007, 04:19 AM   #9
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
I would imagine that if the modem traffic data can contain a hangup sequence, the modem protocol would be designed to escape it somehow... just like it is possible in C programs to put a double quote in double quotes by prefixing it with a backslash, for example: "like \" this". I would expect this to be automatic.

If you are able to retrieve the file on re-try this doesn't seem likely to be the case.

There does seem to be /something/ fishy going on if it always happens at 99%. If you start to download a different file which is twice as large, does it happen after the same number of bytes, or at 99% of the new file too? Curious!

Mackerel magic!
 
Old 10-28-2007, 04:56 AM   #10
Hacker X
Member
 
Registered: Jun 2006
Location: Urbana, Illinois, US
Distribution: Zenwalk, Vector
Posts: 76

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by matthewg42 View Post
I would imagine that if the modem traffic data can contain a hangup sequence, the modem protocol would be designed to escape it somehow... I would expect this to be automatic.
It certainly would be convenient.

Quote:
If you are able to retrieve the file on re-try this doesn't seem likely to be the case.
There does seem to be /something/ fishy going on if it always happens at 99%. If you start to download a different file which is twice as large, ...
This doesn't happen with just any random file. Historically, it's only happened a couple/few times out of scores of downloads over several years.

Quote:
... does it happen after the same number of bytes, or at 99% of the new file too? Curious!
It's 99% of the file in both cases, even though they differ by about 2MB (tgz 11MB, bz2 9MB). That might point away from file format or content, unless there's some content that bz2 can't compress any more than or differently from tgz and gets inserted at the end of the file in both formats. Other than that, it looks like some more insidious phenomenon. Maybe it's GvR's fault

Last edited by Hacker X; 10-28-2007 at 05:02 AM.
 
Old 10-30-2007, 09:32 AM   #11
nx5000
Senior Member
 
Registered: Sep 2005
Location: Out
Posts: 3,307

Rep: Reputation: 52
Humm no clue really..

I would say one of these reasons:
There is a stupid (pleonasm) antivirus somewhere blocking the traffic,
Bad dial-up quality,
Bad modem/router.

Can you try these 2 files:
wget http://kernel.org/pub/linux/kernel/v...h-2.6.23.1.bz2
wget http://kernel.org/pub/linux/kernel/v...geLog-2.6.23.1
then try with replacing http by https and see if commands finish.

Also there is this eicar test files. It is not a virus but is designed to test your antivirus so it is triggered as a "test virus"
Try to download them here (http version should do):
http://www.eicar.org/anti_virus_test_file.htm
 
Old 10-31-2007, 05:54 AM   #12
Hacker X
Member
 
Registered: Jun 2006
Location: Urbana, Illinois, US
Distribution: Zenwalk, Vector
Posts: 76

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by nx5000 View Post
I would say one of these reasons:
There is a stupid (pleonasm) antivirus somewhere blocking the traffic,
Bad dial-up quality,
Bad modem/router.
Or some kind of timeout issue? The files were ~10MB each, downloaded over a 56K modem. That takes at least about 1/2 an hour.

Worked fine:
Code:
Length: 1,412 (1.4K) [application/x-bzip2]
100%[=========================================================>] 1,412         --.--K/s
05:25:29 (21.18 KB/s) - `patch-2.6.23.1.bz2' saved [1412/1412]

Length: 756 [text/plain]
100%[=========================================================>] 756           --.--K/s
05:26:44 (10.30 MB/s) - `ChangeLog-2.6.23.1' saved [756/756]
Quote:
then try with replacing http by https and see if commands finish.
Excerpts:
Code:
...
Connecting to kernel.org|204.152.191.37|:443... connected.
ERROR: Certificate verification error for kernel.org: self signed certificate
ERROR: certificate common name `localhost.localdomain' doesn't match requested host name `kernel.org'.
...
Connecting to kernel.org|204.152.191.5|:443... connected.
ERROR: Certificate verification error for kernel.org: self signed certificate
ERROR: certificate common name `localhost.localdomain' doesn't match requested host name `kernel.org'.
...
Quote:
Also there is this eicar test files. It is not a virus but is designed to test your antivirus so it is triggered as a "test virus"
Try to download them here (http version should do):
http://www.eicar.org/anti_virus_test_file.htm
I've never knowingly installed any antivirus software. What am I supposed to do, just download the four files at the bottom of the page and look for warnings? And is it safe without my having installed any antivirus code?
 
Old 10-31-2007, 09:26 AM   #13
nx5000
Senior Member
 
Registered: Sep 2005
Location: Out
Posts: 3,307

Rep: Reputation: 52
The eicar stuff is a dummy program but its signature is known by antiviruses. It is a safe way to test an antivirus. I thought downloading these small files could show if there is an antivirus on the way.
But I guess it's a ppp problem. Eventhough I've worked on ppp I've never really understood everything to be honest

You should look at the /var/log/pppd.log <-- can't remember the exact logfile name
Change the IP adresses if you post them here.

For the unescape stuffs, it is handled by asyncmap (some caracters are translated), usually set to a000; but I guess you didn't touch the configuration.

This only happens for compressed files?

When I had problems with ppp, I usually went to comp.protocols.ppp groups, there are some very skilled persons overthere.

Last edited by nx5000; 10-31-2007 at 09:28 AM.
 
Old 10-31-2007, 11:16 AM   #14
Hacker X
Member
 
Registered: Jun 2006
Location: Urbana, Illinois, US
Distribution: Zenwalk, Vector
Posts: 76

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by nx5000 View Post
You should look at the /var/log/pppd.log <-- can't remember the exact logfile name
I don't see anything like that, and the man page says pppd uses syslog. There doesn't seem to be much in `grep pppd /var/log/syslog*`, or anything about pppd in /etc/syslog.conf.

Quote:
This only happens for compressed files?
I don't think so. I think it happened to a pdf file last year sometime.

Quote:
When I had problems with ppp, I usually went to comp.protocols.ppp groups, there are some very skilled persons overthere.
Sounds good. I probably won't ask until/unless I have the problem again, but it helps to know where to ask. Thanks.
 
  


Reply

Tags
bzip2, corrupt, download, modem


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Incomplete type ChemicalBurn Programming 4 02-24-2005 08:44 AM
Is my celestia incomplete? eantoranz Linux - Software 10 11-02-2004 07:42 PM
Knoppix is incomplete? Craigwd Debian 2 05-30-2004 01:01 PM
incomplete installation pirozzi Linux - Laptop and Netbook 1 10-25-2003 09:08 AM
incomplete programs ampex189 Linux - Software 3 10-11-2003 11:37 PM


All times are GMT -5. The time now is 01:43 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration