LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-31-2004, 06:13 PM   #1
NiallC
Member
 
Registered: Feb 2003
Distribution: Suse 8.1
Posts: 102

Rep: Reputation: 15
Need help with shell script - renaming multiple files


Hi there,

I have a folder full of image files named as follows:

dennp2.dd-94783.gif-dead-94783>
dennp2.dd-94784.gif-dead-94784>
dennp2.dd-94787.gif-dead-94787>
dennp2.dd-94788.gif-dead-94788>
dennp2.dd-94789.gif-dead-94789>
dennp2.dd-94795.jpg-dead-94795>
dennp2.dd-94798.gif-dead-94798>
dennp2.dd-94799.gif-dead-94799>
dennp2.dd-94800.gif-dead-94800>
dennp2.dd-94801.jpg-dead-94801>
dennp2.dd-94802.gif-dead-94802>
dennp2.dd-94804.jpg-dead-94804>
etc

There are over 16,000 of these files.

What I want to do is take each of these files in turn and rename it to say something like:

dennp294783.gif

I don't care if I have to use two separate scripts to carve out the jpgs and the gifs.

The files don't have to retain their original names so the following result is acceptable:

foo1.jpg
foo2.jpg
foo1.gif

The files must retain the .gif and .jpg naming in the original file. So to be clear the script should rename :

dennp2.dd-94801.jpg-dead-94801>

to

dennp294801.jpg (or whatever#.jpg)

I hope this is clear. I've been struggling with foreach and mv commands all afternoon and I've decided I'm just an idiot.

I know the good folks on here are probably laughing at this point, but I really need some help.

TIA

NIallC
 
Old 05-31-2004, 06:43 PM   #2
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
Take a look at the rename command (man rename).

You might have to issue the command more than once, but for starters:

Code:
rename ".dd-" "" *
That would eliminate the ".dd-" portion of the names (by replacing it with nothing). In fact, you would probably need to form the command like this:

Code:
find . -type f -maxdepth 1 -name "*.dd-*" -exec rename ".dd-" "" {} \;
Using the * in the first command will likely make your argument list too long. And this way, the find command will only execute the rename on filenames that match the pattern you want to get rid of.

For the last part, you could try this:

Code:
find . -type f -maxdepth 1 -name "*-dead*" -exec mv {} `echo {} | cut -f 1 -d "-"` \;
Since the fist rename operation removed the "-dd,", that means the first "-" in the filename is the beginning of the unwanted end of the filename; so just pretend the "-" is a delimiter, and keep only the first part of the filename using cut.

I'd suggest copying over a subset of the files and testing these commands on them before unleashing them on your full set of data.
 
Old 05-31-2004, 08:20 PM   #3
NiallC
Member
 
Registered: Feb 2003
Distribution: Suse 8.1
Posts: 102

Original Poster
Rep: Reputation: 15
Thanks !! the first rename command for the .dd worked beautifully.

However, since don't understand your find command, and since have an aversion to running anything I don't understand, I found another way around the problem

I did an

ls > names.txt

I opened names.txt with a spreadsheet (excel on windows in this case) and used the "-" as a delimiter.

I separated out the jpegs and gifs using the sort command.

I pasted in a coumn of "mv"'s at the start, and I pasted in a column of sequentially numbered *.jpg" s at the end.

I saved the spreadsheet as a CSV.

Then using kedit I did a simple Find & Replace on the commas themselves to structure a long list of mv commands which I then saved as an executable text file.

I ran the text file and it worked fine.

I'm listing this process for any other newbie that may be baffled by long script commands, and because I couldn't really find any other solution.

I do sincerely thank Dark_Helmet. It's a shame I don' t know enough to be able to use his obviously more elegant solution.

I'm off to study it though, to see if I can make any sense of it.

Thanks again.....

N
 
Old 05-31-2004, 08:56 PM   #4
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
Hehehe... you're quite welcome, and "elegant" is a subject of much debate. Just to be thorough, this is an explanation of what the last find command did (or is supposed to do):

-type f => return only files that are regular files (no directories, symbolic links, etc)
-maxdepth 1 => don't look for files in any subdirectories; just the current one
-name "*-dead" => just your regular wildcard for the filenames to match
-exec mv {} `echo {} | cut -f 1 -d "-"` \; => This will require a little break-up of its own, but the short answer is: the find command executes the command after "-exec" for each file it matches. The \; signifies the end of the command.

First thing to note is that in the execution of the command, every open/close curly brace pair ( {} ) is replaced with a matching filename (given by the wildcard and other parameters from before). So, as an example, we'll say the filename we're working on is dennp294783.gif-dead-94783>

The command that find executes becomes:
mv dennp294783.gif-dead-94783> `echo dennp294783.gif-dead-94783> | cut -f 1 -d "-"`

Also note, the use of the backticks (not apostrophes) for the part of the command from echo... using the backticks says to replace the command with its output. So what does the command inside the backticks do? The cut command works on files (or standard input), so we pipe the output of the echo command into cut. In effect, we're placing the filename as standard input to the cut command. So now, cut takes over. It's argument are:

-f 1 => retrieve the first "field" of the input
-d "-" => each field is separated from other fields by a dash character

And then cut's output is the field we selected. Again, using the filename we had before, the echo cause the cut command to see "dennp294783.gif-dead-94783>" as its input. Then the cut looks for dashes in the input. It sees 2 of them, but it is only concerned with the text leading up to the first dash (that text is the first field). So cut ignores everything but that one field, and spits out "dennp294783.gif"

So now, we finally know what is substituted for the backticks earlier. The mv command's final form becomes:
mv dennp294783.gif-dead-94783> dennp294783.gif

Nifty eh? I thought so, but only if it works I didn't try it myself, but the structure is sound. The only thing that would concern me is the ">" in the filenames. The shell might want to interpret it as an output redirection. I guess we may never know

Last edited by Dark_Helmet; 05-31-2004 at 08:57 PM.
 
Old 05-31-2004, 09:23 PM   #5
NiallC
Member
 
Registered: Feb 2003
Distribution: Suse 8.1
Posts: 102

Original Poster
Rep: Reputation: 15
Ok, so I decided to test your command. Here's what I got:

mv: `./dennp294830.gif-dead-94830>' and `./dennp294830.gif-dead-94830>' are the same file
mv: `./dennp294831.jpg-dead-94831>' and `./dennp294831.jpg-dead-94831>' are the same file
mv: `./dennp294832.gif-dead-94832>' and `./dennp294832.gif-dead-94832>' are the same file
mv: `./dennp294833.gif-dead-94833>' and `./dennp294833.gif-dead-94833>' are the same file

If my reading of your command is correct, it seems that you need to reassign a new name to the file in order for it to work, since what you're telling it to do is overwrite itself ? (that's my take, which is a newbie interpretation)

I wish I could suggest something, but I can't.

(I'm actually banking on your command now, sinceI found that there are many other file types and your command is not dependant on filetype, making it more suitable).

Any help would be greatly appreciated.

N
 
Old 05-31-2004, 10:18 PM   #6
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
I'll futz with it some... I'll recreate an environment to test it on real quick-like and let you know.
 
Old 05-31-2004, 10:46 PM   #7
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
Okey dokey. The problem is, the -exec portion of the find command doesn't like the backticks. That can be worked around though, but it will require a shell script. Here's what to do.

Open a text editor and create a file with the contents below:
Code:
#!/bin/bash

# Lines beginning with a "#" are ignored when the script is run...

#   $1 is a special variable that contains the first argument given to the script on the command line.
# in this case, it's the name of the file we want to change. So the if statement below makes sure that
# we were actually given a filename. If no filename was given, it does nothing.
if [ "$1x" != "x" ] ; then
  filename=$1

  # You'll notice that the backticks are back :). This is exactly the same contents of the backticks
  # from the original find command
  new_filename=`echo $filename | cut -f 1 -d '-'`

  # This simply displays the command the script would execute
  echo "mv ${filename} ${new_filename}"

  # The line below is the actual mv command to execute. It is commented out so you can give the
  # script a trial run. When you're satisfied that it will do what you want, enable the mv command
  # by removing the "#" in front of it.
  #mv $filename $new_filename
fi
Not having a creative bone in my body, I named it "mv_portion". So substitute the name and path of your file wherever you see me use it in the following commands.

Now, make the script executable by issuing this command:
chmod u+x mv_portion

Finally, modify the find command to use the script rather than the original -exec contents:
find . -type f -maxdepth 1 -name "*-dead*" -exec mv_portion {} \;

In case you missed it in the script's comments above, the script won't actually do anything as it is. You'll need to uncomment the mv command to actually perform the renaming. It's best to test it out before actually running it full-steam.

If you run into any problems, just lemme know.
 
Old 05-31-2004, 11:11 PM   #8
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
Side note:
I figured out why that find command didn't work. The problem was this:
When "enter" was pressed, the shell replaced the contents of the backticks with its result immediately instead of how I imagined it. So that echo -> cut sequence collapsed into a "mv {} {}" command which is why the errors were display. It was trying to move the file onto itself. My mistake was assuming the backticks would be preserved and evaluated once for each file found. So that was an oversight on my part. The find -> script routine works though, and is pretty much just as good.
 
Old 05-31-2004, 11:35 PM   #9
NiallC
Member
 
Registered: Feb 2003
Distribution: Suse 8.1
Posts: 102

Original Poster
Rep: Reputation: 15
gosh, I guess I need some sleep. I literally cut and pasted from above and got:

find: mv_portion: No such file or directory

(16K times....smile)

ls | grep dead

brought back a list of filenames with "dead" in them.

mv_portion is an exe, and I'm assuming the "fi" at the end of the script was a typo???

Thanks for all your help, you've gone way above and beyond. this is driving me batty.

Niall.
 
Old 05-31-2004, 11:37 PM   #10
NiallC
Member
 
Registered: Feb 2003
Distribution: Suse 8.1
Posts: 102

Original Poster
Rep: Reputation: 15
oh and the mv_portion file is located in the same directory as the actual files to be renamed, if that makes any difference....
 
Old 06-01-2004, 12:00 AM   #11
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
Ugh, my mistake. Ok, since the script is in the same directory as the files, issue this command:
find . -type f -maxdepth 1 -name "*-dead*" -exec ./mv_portion {} \;

The short answer for why it didn't work was that the system could not find the script (even though it was in the same directory); there's a specific list of directories it checks, and the current directory is not part of that list. The "./" prepended to the script name explicitly tells the system where to find the script (in this case: the current directory).

The "fi" at the end of the script is intentional It's tells the system where the end of the if-statement is; it's a marker.

Give that a go, and remember to uncomment the mv command in the script when you're satisfied everything is ok.
 
Old 06-02-2004, 09:52 PM   #12
NiallC
Member
 
Registered: Feb 2003
Distribution: Suse 8.1
Posts: 102

Original Poster
Rep: Reputation: 15
OK here's a good one for you.

It actually renamed all the files with "dead" in them to dennp2.dd

So it eliminated over 75% of the files......lol...Good job I ran it on a copy of the originals.

Any ideas?
 
Old 06-02-2004, 10:10 PM   #13
globeTrotter
Member
 
Registered: Feb 2004
Location: Townsville, Queensland, Oz
Distribution: Red Hat 9
Posts: 107

Rep: Reputation: 15
Hi all

intresting stuff i'm reading through. i'll leave you guys to it, but before i go. i have to say, dont' remove the "fi" from the end of the script, what i can see, it's not a typo!?!?!
regards
GT
 
Old 06-02-2004, 10:29 PM   #14
NiallC
Member
 
Registered: Feb 2003
Distribution: Suse 8.1
Posts: 102

Original Poster
Rep: Reputation: 15
nope, I had it omitted, but put it back in before I ran it that last time.

Here's another script I received that didn't work either....lol.,.,..but it might inspire:


#!/usr/bin/perl

$whatdir = shift;
opendir(DIR, $whatdir);

foreach $file (readdir(DIR)) {

# we want to rename files with '-dead-' in the name
if ($file =~ /-dead-/) {

# we only want the stuff leading up to '-dead...'
$nfile = substr($file, 0, index($file, "-dead"));

# we don't want the '.dd-'
$nfile =~ s/\.dd-//;

# show what we will do
print "'$file' -> '$nfile'\n";

# this line actually performs the renaming. Comment it out if you
# just want to see what WOULD happen without actually doing anything.
rename("$whatdir/$file", "$whatdir/$nfile");
}

}

closedir(DIR);
 
Old 06-02-2004, 10:36 PM   #15
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
No, the "fi" is definitely not a typo. The script essentially says this:

Do nothing if no argument is given on the command-line.
If there is an argument on the command line, then perform all the commands between the if and the fi.

I ran the rename on the files, and then the find with a set of files exactly as listed in the original post and things went smoothly on my end. So I would suspect something got mixed up in the transition. Here's the exact sequence of steps I performed:

1. cd into the directory with the data files
2. create a script named mv_portion
3. Fill the script with this content (feel free to copy and paste)
Code:
#!/bin/bash
if [ "$1x" != "x" ] ; then
  filename=$1
  new_filename=`echo $filename | cut -f 1 -d '-'`
  echo "mv ${filename} ${new_filename}"
  #mv $filename $new_filename
fi
NOTE: I removed all the comments from earlier. It is the exact same script, but with the "clutter" of the comments removed.

4. Make the script executable by issuing: chmod u+x mv_portion
5. Remove the ".dd-" portion of the filenames by issuing: find . -type f -maxdepth 1 -name "*.dd-*" -exec rename ".dd-" "" {} \;
6. Finish the renaming with this command: find . -type f -maxdepth 1 -name "*-dead*" -exec ./mv_portion {} \;

You must remove the ".dd-" portion first. Otherwise, the script gets confused. It sees the dash used in ".dd-" as a field separator. So when the cut command is issued in the script, it says "give back the text of the filename up to (but not including) the first dash I see".
 
  


Reply

Tags
md5sum, regex, rename, renamescript, script


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
renaming multiple files linuxhippy Slackware 10 01-21-2011 01:53 AM
shell renaming script paskari Programming 3 09-24-2005 02:15 PM
file renaming with shell script or ? XJNick Linux - General 5 07-29-2005 02:43 PM
directory renaming shell script? Apostasy Linux - General 15 02-22-2005 06:50 AM
Renaming multiple files with the same format Shr00mBoXx Slackware 7 06-20-2004 07:22 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:29 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration