LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-11-2013, 10:24 PM   #1
Paul_Lee
Member
 
Registered: Jul 2001
Location: Cambridgeshire, England
Distribution: Ubuntu 13.10
Posts: 93

Rep: Reputation: 15
Recursive file operation in a directory


Hi,
I have a directory of jpg images and I'd like to run through the recursively so that the output file name is the same as the input, with only the file type suffix changed. A bit like doing
cuneiform 001.jpg -o 001.txt
cuneiform 002.jpg -o 002.txt
and so on (there are a lot of images)
I can't remember how to do this recursively in a shell script, could someone help please?
(Also, if I wanted to do this on certain files but miss out on others, how would I achieve this? For instance, omit processing all files that had "img" in the name, such as 002img.jpg, 010img.jpg and so on)

Many thanks!

Paul
 
Old 11-11-2013, 10:48 PM   #2
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Hu,

most people would use find for that. Somthing like:
Code:
find . -name '*.jpg' ! -name '*img*.jpg'
Then you need to pipe that output to your command. You could use a for loop, or you can do something with xargs. For example (using echo instead of cuneiform for testing purposes):
Code:
find . -name '*.jpg' ! -name '*img*.jpg' | xargs -I{} sh -c 'echo "$1 -o ${1%.*}.txt"' -- {}
If your filenames might have whitespaces in them then you might want to use the null separator.
Eg
Code:
find . -name '*.jpg' ! -name '*img*.jpg' -print0 | xargs -0 -I{} sh -c 'echo "$1 -o ${1%.*}.txt"' -- {}
If you are happy with that, then replace the "echo" with the command you actually want to run.


HTH,

Evo2.
 
1 members found this post helpful.
Old 11-11-2013, 10:57 PM   #3
Paul_Lee
Member
 
Registered: Jul 2001
Location: Cambridgeshire, England
Distribution: Ubuntu 13.10
Posts: 93

Original Poster
Rep: Reputation: 15
Hi,
Many thanks, I shall try, FWIW, the filenames don't have whitespace, they are just of the variety
002.jpg 003a.jpg 005img.jpg 010.jpg 124img.jpg and so on - all I'm trying to veto are the ones that have "img" in the filename.

Incidentally, what does "-- {}" mean?
 
Old 11-12-2013, 12:58 AM   #4
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Hi,

the "-- {}" allows the xarg variable to be put into the $1 shell variable. There are other ways to achieve the same thing with xargs, but like find, its a very powerful program and once I've found one way that works and can remember, I'm less likely to explore the other options.

Cheers,

Evo2.
 
Old 11-12-2013, 02:33 AM   #5
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
an alternative

Code:
#!/bin/bash
Path=$1
Exclude=$2
while read -d '' FileName;do
    [[ "${FileName}" =~ "${Exclude}" ]] || \
        mv -v "${FileName}" "${FileName%.jpg}.txt"
done < <(find "${Path}" -name "*.jpg" -print0)

takes two options
example , process files in current dir ( including sub directories ) excluding files that contain the string "img"
Code:
TheScripit.sh . img


with comments
Code:
#!/bin/bash
Path=$1
Exclude=$2
# start a read loop, using 'null' as record separator
while read -d '' FileName;do
    [[ "${FileName}" =~ "${Exclude}" ]] || \
        mv -v "${FileName}" "${FileName%.jpg}.txt"
# list construct http://www.tldp.org/LDP/abs/html/list-cons.html#LISTCONSREF
# ^^ test $FileName, if it does not include the $Exclude string, then 
# mv -v ( verbose, it prints what it does, nice for feedback )
# http://www.tldp.org/LDP/abs/html/refcards.html#AEN22664
# ^^ string sub. explains what ${FileName%.jpg}.txt is doing
done < <(find "${Path}" -name "*.jpg" -print0)
# < <() heredocs.. this is passing the output of the subshell () into the 'loop'
# http://mywiki.wooledge.org/BashGuide/InputAndOutput
# find , straight forward.. the -print0 ensures we use a 'null' instead of a new line to separate records
# my comments are awful ...
more reading

http://www.tldp.org/LDP/Bash-Beginners-Guide/html/
http://www.tldp.org/LDP/abs/html/
http://mywiki.wooledge.org/BashGuide
http://www.gnu.org/software/bash/manual/bashref.html

The tldp stuff is great, however there are some nasty bad habits in it,
The mywiki.wooledge does a very good job of 'fixing' those habits


a note on the heredocs use here

Code:
find "${Path}" -name "*.jpg" -print0 | while read -d '' FileName;do
   .. stuff ..
done
would work in this case,
However the downside is that any variables you set inside the loop are 'lost' as the | ( pipe ) starts a subshell

best example I can come up with here..

create a new dir, and 'touch' example files

Code:
touch 00{1..5}{,img}.jpg
Code:
#!/bin/bash
Path=$1
Exclude=$2
while read -d '' FileName;do
    [[ "${FileName}" =~ "${Exclude}" ]] \
        || mv -v "${FileName}" "${FileName%.jpg}.txt" \
        && skipped+=("${FileName}")
done < <(find "${Path}" -name "*.jpg" -print0)
echo "I skipped ${skipped[@]}"
Code:
#!/bin/bash
Path=$1
Exclude=$2
find "${Path}" -name "*.jpg" -print0 | while read -d '' FileName;do
    [[ "${FileName}" =~ "${Exclude}" ]] \
        || mv -v "${FileName}" "${FileName%.jpg}.txt" \
        && skipped+=("${FileName}")
done
echo "I skipped ${skipped[@]}"
 
Old 11-12-2013, 02:50 AM   #6
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Hi,

well, if we're doing loops and shell scripts... here's how I would actually do it myself. I use zsh, so I'd run the follow command:
Code:
for f in **/*.jpg~*img.jpg ; do echo "$f" -o "${f%.*}.txt"; done
Again with echo holding the place of the real command until I was happy with the output.

The '**' in zsh will recurse into the sub directories and the '~' in the glob excludes the *img.jpg files.

Cheers,

Evo2.

Last edited by evo2; 11-12-2013 at 03:00 AM. Reason: Don't forget to quote the variables!
 
Old 11-12-2013, 03:17 AM   #7
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
nice, I might have to try using zsh
 
Old 11-12-2013, 05:44 AM   #8
Paul_Lee
Member
 
Registered: Jul 2001
Location: Cambridgeshire, England
Distribution: Ubuntu 13.10
Posts: 93

Original Poster
Rep: Reputation: 15
I tried the one-line command of yours, evo2:

Code:
find . -name '*.jpg' ! -name '*img.jpg' | xargs -I{} sh -c 'cuneiform "$1 -o ${1%.*}.txt"' -- {}
But I got horrible errors, such as:
Magick: Unable to open file (./057.jpg -o ./057.txt) reported by magick/blob.c:2866 (OpenBlob)

But it works it I explicitly type into a shell:
cuneiform 057.jpg -o 057.txt
 
Old 11-12-2013, 05:58 AM   #9
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,110

Rep: Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268
try remove " :
find . -name '*.jpg' ! -name '*img.jpg' | xargs -I{} sh -c 'cuneiform $1 -o ${1%.*}.txt' -- {}
 
1 members found this post helpful.
Old 11-12-2013, 06:10 AM   #10
Paul_Lee
Member
 
Registered: Jul 2001
Location: Cambridgeshire, England
Distribution: Ubuntu 13.10
Posts: 93

Original Poster
Rep: Reputation: 15
Thanks, it worked like a champ!
 
Old 11-12-2013, 06:37 AM   #11
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,110

Rep: Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268Reputation: 2268
glad to help you.
If you really want to say thanks just press YES. Also please mark the thread SOLVED if your problem has been solved.
 
Old 11-12-2013, 06:41 AM   #12
Paul_Lee
Member
 
Registered: Jul 2001
Location: Cambridgeshire, England
Distribution: Ubuntu 13.10
Posts: 93

Original Poster
Rep: Reputation: 15
Great, thanks!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
recursive mass file comparison betw. 2 locations with different directory structures furryspider Linux - Software 15 08-26-2010 09:13 AM
How to delete directory/file (operation not permitted)? lvgamer1 Linux - Newbie 4 12-28-2004 11:53 AM
How to do recursive file copy of directory for specific files? Arodef Linux - Newbie 4 06-29-2004 06:35 PM
recursive file permissions does not change new files in same directory PAB Linux - Newbie 2 03-08-2004 01:27 PM
recursive directory traversal klfreese Linux - Newbie 2 08-20-2003 08:27 PM


All times are GMT -5. The time now is 09:01 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration