LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 03-07-2012, 07:31 AM   #1
jnojr
Member
 
Registered: Sep 2007
Location: Chandler, AZ
Posts: 227

Rep: Reputation: 20
wget question


Let's say there's a directory structure you want to wget recursively. And let's say your download is occasionally interrupted for whatever reason. So, you have a whole bunch of files that have been successfully downloaded, and some that have not, but you do not know which ones.

Will using the "-c" option always successfully restart any file, anywhere in the directory structure? Is there any mechanism to validate that all files were obtained successfully? The man page is a little unclear on the issue, and gives several provisos that indicate that there are absolutely situations where you're going to wind up with a corrupted download.

And is there a way to tell wget to check the size of each existing file against the source, and if it's different to just delete the local copy and then re-download it from the beginning?
 
Old 03-08-2012, 03:40 AM   #2
TKH
Member
 
Registered: Jul 2011
Location: Milky Way
Distribution: Ubuntu, LFS, Slackware, Fedora
Posts: 223

Rep: Reputation: 20
Try using this:
Code:
for x in $(cat <your list>)
do
if [ !-e $x ]
wget <url>$x
fi
done
I know that the code above is far from perfection, so try fixing it a bit.
A little explanation:
  • The first line ("for" command) reads the list file per line (with "cat" command), putting the content of each line into the "x" variable.
  • The third line ("if" command) makes sure that the file does not exist.
Don't forget to replace the text inside brackets ("<....>").

Hope it helps
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
wget question diamond_D Linux - Newbie 5 12-09-2010 07:39 PM
wget question coubury Linux - Newbie 7 03-03-2008 07:54 AM
wget question kpachopoulos Linux - General 1 07-23-2005 10:00 AM
wget question Yohhan Linux - Software 1 05-05-2004 07:49 PM
wget question ziggie216 Linux - Software 0 10-12-2003 12:02 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 07:03 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration