LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-08-2013, 09:37 AM   #1
Ferodactyl
LQ Newbie
 
Registered: Aug 2013
Posts: 4

Rep: Reputation: Disabled
How do I download all files with the same name onto my computer?


I have about 1000 different files on my server all named "img001.jpg"

Is it possible for me to find and download all of them onto my computer automatically?

They are stored in different sub-folders. The directory structure is like this:

Fashion/1930s/Women's/4 - Separates/1 - Blazers/Lv1
Fashion/1930s/Women's/5 - Active Sportswear/3 -Swimwear/Lv1

etc.

There are about 1000 of these Lv1 folders.

There is one "img001.jpg" inside every Lv1 folder.

In a sense the URL is known since I know they are all inside the Lv1 folders. However, I don't know how I would enter all the URLS unless I did it manually by checking the folders? Sorry for this confusion.

There is access to SSH. I'm not sure what type of server it is.

Since they all have the same name, how do I maintain the folder structure so that my computer doesn't try to rewrite over them as they download?

Thank you for any advice you might have. Please let me know if I should add more details about this situation.

Last edited by Ferodactyl; 08-08-2013 at 10:50 AM. Reason: To add more detail to the question
 
Old 08-08-2013, 09:56 AM   #2
yancek
LQ Guru
 
Registered: Apr 2008
Distribution: PCLinux, Slackware
Posts: 6,872

Rep: Reputation: 1249Reputation: 1249Reputation: 1249Reputation: 1249Reputation: 1249Reputation: 1249Reputation: 1249Reputation: 1249Reputation: 1249
I think you will need to post a little more detail to get help. How do you define 'many', 3 or 3,000,000 or some approximate number between? Also the directory/folder structure. Are these images all in sub-directories of another directory? You should be able to write a script (probably bash would work) to first 'find' then 'cp' (copy) to wherever you want and you might want to rename them. If you post a little more detail someone who knows a little about programming might give you some specific pointers.
 
Old 08-08-2013, 09:57 AM   #3
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,029

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
i think this is something rsync could handle but i dont know the exact syntax.
 
Old 08-08-2013, 10:00 AM   #4
joe_2000
Member
 
Registered: Jul 2012
Location: Aachen, Germany
Distribution: Void, Debian
Posts: 808

Rep: Reputation: 216Reputation: 216Reputation: 216
Agree with Yancek, you should go with a bash script. Use

Code:
wget <url to img>
To download a file with a known url to your local drive. Use mv to rename it. E.g. append a running number. Cycle through all files like that.
Do you have all urls to the images? If you don't, we need more info on the server layout. What kind of server is it / what kind of access do you have to it? (FTP / SSH, etc...)
 
Old 08-08-2013, 10:05 AM   #5
joe_2000
Member
 
Registered: Jul 2012
Location: Aachen, Germany
Distribution: Void, Debian
Posts: 808

Rep: Reputation: 216Reputation: 216Reputation: 216
Quote:
Originally Posted by schneidz View Post
i think this is something rsync could handle but i dont know the exact syntax.
Good point, if there is any access like ssh or similar that can be used with rsync that would be a good way. It has a recurse option which could be used..., wouldn't even need regular expressions cause all files have the same name.

Again, need more info from OP :-)
 
Old 08-08-2013, 10:08 AM   #6
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,029

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
Quote:
Originally Posted by joe_2000 View Post
Agree with Yancek, you should go with a bash script. Use

Code:
wget <url to img>
To download a file with a known url to your local drive. Use mv to rename it. E.g. append a running number. Cycle through all files like that.
Do you have all urls to the images? If you don't, we need more info on the server layout. What kind of server is it / what kind of access do you have to it? (FTP / SSH, etc...)
^ assuming nfs or sshfs mount:
Code:
i=1; for jpg in `find /path/to/server -name "img001.jpg"`
do
 cp $jpg $i-`basename $jpg`
 i=`expr $i + 1`
done

Last edited by schneidz; 08-08-2013 at 10:10 AM.
 
Old 08-08-2013, 10:45 AM   #7
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
Quote:
Originally Posted by schneidz View Post
^ assuming nfs or sshfs mount:
Code:
i=1; for jpg in `find /path/to/server -name "img001.jpg"`
do
 cp $jpg $i-`basename $jpg`
 i=`expr $i + 1`
done
that's not going to work too well

look at the example
Quote:
Fashion/1930s/Women's/4 - Separates/1 - Blazers/Lv1
not only whitespace but a '

assuming mounted over network
Code:
find /path/to/server/ -name "img001.jpg" | while read jpg;do
   install -D "$jpg" "/path/to/Local/${jpg#/path/to/server/}"
done
if you want some noise, add -v to install
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Jigdo download of DVD-7 ALWAYS fails with just 5 files left to download IslandWolf Debian 3 03-12-2011 04:24 AM
Installing Ubuntu: On Which Computer & Which Download (Computer Type) Ntvu Linux - Newbie 2 06-09-2008 05:05 PM
Can't copy files from an SCO Unix System V computer to a Linux computer gnppapas Linux - General 2 11-27-2004 02:39 PM
Download Manager - download BIG files AndrewMSConvert Linux - General 3 11-09-2003 07:26 AM
Why would a windows computer smoke a linux computer for download speed ? lostboy Linux - General 4 10-21-2003 06:20 PM


All times are GMT -5. The time now is 12:56 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration