LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
Reload this Page wget takes ages to rebuild after broken download and resume.
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 07-19-2015, 09:41 AM   #1
lucky7456969
LQ Newbie
 
Registered: Sep 2012
Posts: 26

Rep: Reputation: Disabled
wget takes ages to rebuild after broken download and resume.

I have a big site to mirror.
However, my download was somewhat broken
When I resume the download using
wget -c -mk -w 2 www.somesite.com
It takes me two weeks and still not yet finish
rebuilding the internal database.
How can I accelerate this process?
I don't want wget to restart from scratch.
Today, the rescan was broken again.
Very frustrated!
Any ideas?
Thanks
Jack
 
Old 07-19-2015, 04:56 PM   #2
Keith Hedger
Senior Member
 
Registered: Jun 2010
Location: Wiltshire, UK
Distribution: Linux From Scratch, Slackware64, Partedmagic
Posts: 3,097

Rep: Reputation: 834Reputation: 834Reputation: 834Reputation: 834Reputation: 834Reputation: 834Reputation: 834
There's a program called ( I think ) httrack that can mirror a complete site, it's more effecient than using wget and resumes OK, used to use it a few years ago when I was on dial-up, google for it its pretty cool.
 
Old 07-21-2015, 08:46 AM   #3
/dev/random
Member
 
Registered: Aug 2012
Location: Ontario, Canada
Distribution: Slackware 14.2, LFS-current, NetBSD 6.1.3, OpenIndiana
Posts: 319

Rep: Reputation: 112Reputation: 112
If you are trying to mirror your own site, why not just login to the host and do an rsync?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
YaST2 takes ages to load General SUSE / openSUSE 15 06-07-2007 09:08 AM
mounting takes ages! Steave Linux - General 3 02-23-2006 06:39 AM
clamscan takes ages to complete. gbowden Slackware 9 02-07-2006 06:05 PM
X takes ages and ages to load! dcdbutler Slackware 9 03-23-2005 07:53 AM
'Whatever' login takes ages... Worstje Slackware 7 10-15-2004 03:06 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 07:37 PM.

Contact Us - Advertising Info - Rules - Privacy - LQ Merchandise - Donations - Contributing Member - LQ Sitemap -
Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration