LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 06-17-2009, 10:39 PM   #1
hbar
Member
 
Registered: Dec 2007
Location: Canada
Distribution: Debian
Posts: 50

Rep: Reputation: 15
wget produces corrupt files??


Can someone try this and see if this happens? Download one of the .txt.gz archive files on this page: https://www.redhat.com/archives/amd64-list/ , and you should be able to open it. Now download one using wget, and the file can't be uncompressed because it is not a gzip file! What is happening!?
 
Old 06-17-2009, 11:20 PM   #2
billymayday
Guru
 
Registered: Mar 2006
Location: Sydney, Australia
Distribution: Fedora, CentOS, OpenSuse, Slack, Gentoo, Debian, Arch, PCBSD
Posts: 6,678

Rep: Reputation: 122Reputation: 122
Looks like someone forgot to gzip them. It looks like a mail file in raw for to me. Try renaming to .txt and see what I mean when you open it in an editor.
 
Old 06-17-2009, 11:31 PM   #3
mbostwick
Member
 
Registered: Feb 2009
Location: Where ever life takes me
Distribution: gentoo,opensuse,ubuntu, arch
Posts: 110

Rep: Reputation: 18
I agree. I am able to open it with nano and see it plain text.
 
Old 06-18-2009, 09:23 AM   #4
hbar
Member
 
Registered: Dec 2007
Location: Canada
Distribution: Debian
Posts: 50

Original Poster
Rep: Reputation: 15
The smaller files seem to be plain text but I can reproduce the problem with the larger ones. Often the connection is interrupted and the download resumes automatically, and this results in a bad file. Is there any way around this? Downloading the file in Firefox works fine but I need to use wget (or something similarly non-interactive).

Code:
$ wget https://www.redhat.com/archives/amd64-list/2008-January.txt.gz
--2009-06-18 10:20:52--  https://www.redhat.com/archives/amd64-list/2008-January.txt.gz
Resolving www.redhat.com... 69.192.64.112
Connecting to www.redhat.com|69.192.64.112|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5848 (5.7K) [application/x-gzip]
Saving to: `2008-January.txt.gz'

99% [=====================================> ] 5,847       --.-K/s   in 0s      

2009-06-18 10:20:53 (272 MB/s) - Connection closed at byte 5847. Retrying.

--2009-06-18 10:20:54--  (try: 2)  https://www.redhat.com/archives/amd64-list/2008-January.txt.gz
Connecting to www.redhat.com|69.192.64.112|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 26780 (26K), 20933 (20K) remaining [application/x-gzip]
Saving to: `2008-January.txt.gz'

100%[++++++++==============================>] 26,780      --.-K/s   in 0.1s    

2009-06-18 10:20:55 (139 KB/s) - `2008-January.txt.gz' saved [26780/26780]
 
Old 06-18-2009, 04:37 PM   #5
hbar
Member
 
Registered: Dec 2007
Location: Canada
Distribution: Debian
Posts: 50

Original Poster
Rep: Reputation: 15
Ah, I've got it. The server was doing something funny when it detected wget. By forging the useragent and referer, it works fine. I wish it didn't have to come to this, but they asked for it....
 
Old 06-18-2009, 04:42 PM   #6
billymayday
Guru
 
Registered: Mar 2006
Location: Sydney, Australia
Distribution: Fedora, CentOS, OpenSuse, Slack, Gentoo, Debian, Arch, PCBSD
Posts: 6,678

Rep: Reputation: 122Reputation: 122
What did they ask for? People not to copy all of their archives?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Corrupt CD, Trying to get files off, need help mloberg Linux - Hardware 9 09-22-2008 10:47 AM
Restore old corrupt files? General General 2 04-15-2007 05:56 PM
Command to copy files/folders but skip bad/corrupt files?? leemoreau Linux - Newbie 2 04-02-2007 02:27 PM
ffmpeg with -sameq form .mov produces huge files ziocicion Linux - Software 2 06-01-2006 10:07 AM
CVS produces read-only files on checkout gepesz Linux - Software 1 07-01-2004 03:50 AM


All times are GMT -5. The time now is 11:48 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration