LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 05-14-2009, 07:11 AM   #1
Eddie1506
LQ Newbie
 
Registered: Oct 2005
Posts: 2

Rep: Reputation: 0
wget input file and directory structure


Hi all,

I have a file with few hundred links to some files on a webpage.

I would like to download them all, but at the same time keep the directory structure of the website.

So for example i have these links:

http://www.mysite.com/zips/big/big1.zip
http://www.mysite.com/zips/big/big2.zip
http://www.mysite.com/zips/small/small1.zip
http://www.mysite.com/zips/small/small2.zip
http://www.mysite.com/rars/big/big1.rar
http://www.mysite.com/rars/big/big2.rar
http://www.mysite.com/rars/small/small1.rar
http://www.mysite.com/rars/small/small2.rar

I would like wget to store downloaded files on my disk in the following way:

Code:
start folder
|--zips
|   |--big
|   |   |--big1.zip
|   |   |--big2.zip
|   |--small
|   |   |--small1.zip
|   |   |--small2.zip
|--rars
|   |--big
|   |   |--big1.rar
|   |   |--big2.rar
|   |--small
|   |   |--small1.rar
|   |   |--small2.rar
Is this possible with wget?

Thanks!
 
Old 05-14-2009, 07:25 AM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Code:
wget -x -nH $link
should do the trick, where -x forces creation of directories and -nH don't create host directory, as www.mysite.com/.
 
Old 05-14-2009, 08:17 AM   #3
Eddie1506
LQ Newbie
 
Registered: Oct 2005
Posts: 2

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by colucix View Post
Code:
wget -x -nH $link
should do the trick, where -x forces creation of directories and -nH don't create host directory, as www.mysite.com/.
Thanks! It works flawlessly.

I saw those options btw, but somehow I presumed they were only meant to be used with recursive downloading and not with an url list.
 
Old 05-14-2009, 08:26 AM   #4
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Quote:
Originally Posted by Eddie1506 View Post
I saw those options btw, but somehow I presumed they were only meant to be used with recursive downloading and not with an url list.
The -x option is not needed together with -r, since it is implicit in recursion, while the -nH option does its job even with recursion.
 
  


Reply

Tags
wget


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Bash script to download a directory structure and a file... JoeBleaux Programming 2 03-21-2009 11:21 AM
how many different ways we can remove the file from a directory structure. dina3e Linux - General 8 10-01-2008 03:06 AM
Home Jail Folder Structure like Gobolinux Directory Structure luispt Linux - General 3 07-26-2008 06:46 PM
wget options for directory structure davimint Linux - Server 3 05-18-2007 10:24 PM
[wget] Crawl through directory structure, get only certain files FnordPerfect Linux - Software 2 01-19-2007 12:06 PM


All times are GMT -5. The time now is 08:01 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration