LinuxQuestions.org - Batch Rename

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - Batch Rename (https://www.linuxquestions.org/questions/linux-newbie-8/batch-rename-412643/)

I have have a bunch of .img files and .hdr files that I need to rename (they are all in the same directory). Specifically, I'd like to rename them to include a subject identifier. For example, I'd like to rename both types of files such that:

con_0040.img (or .hdr) > 4001_con0040.img (or .hdr)

I'm a complete novice when it comes to Linux, and I am in the unenviable position of having to rename hundreds of files! I'd be really grateful for any assistance.

You should be able to do that by using a shell script - something similar to this:
http://aplawrence.com/Linux/rename.html

First of all make a backup copy of the entire folder!
If you only have .img or .hdr files in the dir:

It looks like you want this:
Rename all files with con_ to 4001_con so that should be:
/edit: i typed a mistake here before, instead try this:
rename 'con_' '4001_con' *

or
for i in *; do mv $i `echo $i | sed 's/con_/4001_con/g'`; done
(mind the `'s)
For all files in this dir; do move from old name ($i) to new name where we replace con_ with 4001_con
-------------
If you have other files with other extensions in the same folder:
rename 'con_' '4001_con' *.img
Or both extensions together:
rename 'con_' '4001_con' *.img *.hdr

for i in *.img; do mv $i `echo $i | sed 's/con_/4001_con/g'`; done
Or both extensions together:
for i in *.img *.hdr; do mv $i `echo $i | sed 's/con_/4001_con/g'`; done

I'm not really sure on your renaming scheme. Do all files get the same prefix 4001_con ?

Thanks for this! I will try it tomorrow when I'm back in the lab. Yes, all the files in the said directory will get the same numerical prefix, though the prefix will change as a function of the particular subject directory that I'm working with at the time, if that makes sense (so for any given directory, I will want to add the same identifying prefix to the .img and .hdr files, but it will be 4001 ... 4002 ... 4003 and so on).

Ah! If you want a counter you should look at the link posted by ksgill, and use the part with x=$((x+1))
If you provide a better example of the actual filenames you are dealing with i could give the proper command for it. Just put down three or four names of them and what the filenames should become. Anyways you can do it with a for-loop with a counter.

Here's what I want to do:

old name: con_0027.img
new name: 4001_con0027.img

old name: con_027.hdr
new name: 4001_con027.hdr

The directories consist of both .hdr and .img files and there will be 27-40 of each (i.e., con_027 ... con_028 ... etc.). All the files in a given directory will get the same numerical prefix. As I navigate to each new subject's directory (I'm dealing with fMRI data), I will have to do the same thing, only I will change the prefix to that particular subject's identification number. I need to do this because I have to create one directory for all the subject files, so there has to be something to distinguish between subjects since the files are named the same in each person's directory

I may not be making sense, but I hope I am. Thanks again for your help with this!

ok! This is getting interesting :) Now my question is: where does the prefix come from?
Maybe it's like the directory is called 4001 for subject 4001
and 4002 for subject 4002?

Also: con does not change per subject right? Meaning: all files from all directories will get the _con part in it?

Do you want to be able to do multiple directories in one go? Is that an option? I'd try that if i were you since it saves alot of browsing the directories.

/edit: i have a working command for you if you just answered yes the the questions above:
A couple things:
1. first make a backup of the directories yourself!
2. i called the directory where you put all the renamed files: outputdir
You can change it to any name.
3. To create the outputdir (called outputdir) type this command:

Code:

mkdir ./outputdir

4. when you execute this command i'm assuming you are one directory up of the subject-dirs.
So when you do a directory listing we should see this:

Code:

ls

4001  4002  4003  4004  outputdir

5. it will copy all *.img and *.hdr from all subdirectories, if want to exclude directories: let me know.
6. you could also put this command in a script. If you are interested in that, let me know.
7. for this 'problem' there are many possible solutions, i think this one works reasonably well.
8. in my earlier post i was using the move (mv) command, i think it is safer to use copy (cp), so i used it here.
9. I am assuming you named all the subject directories like so: 4001
And another patient directory could be called 4002

Anyways here is the command:

Code:

for i in `find . -name "*.img" -o -name "*.hdr"|grep -v outputdir`; do cp -i $i ./outputdir/`dirname $i |sed 's/^.\///g'``basename $i | sed 's/con_/_con/g'`; done

(make sure to COPY this command to your terminal since it contains backticks `` that can be easily missed when typing by hand.
To paste it in your terminal do a middle-mouse button click or paste with right-click)

Explanation:
for i in `find . -name "*.img" -o -name "*.hdr"|grep -v outputdir`;
find all files with extensions .hdr and .img in all subdirectories. Exclude the output directory (since it may also contain files from previous runs) with grep.
-------------------------------------------------------------------

do cp -i $i
copy the file from its original location to
(the -i option is to specify that we don't want to overwrite any files in the outputdir so
if a file already exists it will prompt you with a question whether to overwrite or not)

./outputdir/
(copy to) ./outputdir specifies the output directory that will contain all copied&renamed files.

`dirname $i |sed 's/^.\///g'`
take the original directory-name, it will output the patient prefixes like 4001. Strip the leading ./ with sed to give a name like 4001

`basename $i | sed 's/con_/_con/g'`;
specify the name of the file to copy to. Replace con_ with _con

done :)
-------------------------------------------------------------------

sources:
http://www.splike.com/howtos/bash_fa...sion%2C+etc%3F

Off-topic: mri rules! I was a subject for a study into gambling-addiction this year, i think it was mostly structural mri.
For the record: i'm was one of the negative-test-group :P

I'm interested whether this solution works for you, if have questions feel free to ask!

muha,

Wow, the time you're spending on this--your efforts are truly appreciated! Unfortunately, I haven't had any time to process data in the last few days (busy scanning), but once I do, I will certainly let you know how things worked out! Thanks, too for the explanations of each command--as a novice this is incredibly helpful!

Very best,
SF

no problem! I'm in the process of learning about this myself so i have usefull text lying around on how to do these things.
Also, i'm trying to score karma :D
cya,
Muha

Great, this is getting very close. I'm another newbie with a similar issue--actually a simpler version I think--but I need a couple more wrinkles ironed out.

I've migrated a bunch of files from my Mac to the new Linux box. I've been a Mac user all my life, and these files, going back to 1991 or so, are mostly named without filetypes and commonly with spaces in the filenames. I just want to go through them and turn them all into compliant filenames.

So two things I need to add to what's been explained already: first, how to I batch-append to filenames? In some cases I have a directory whose contents are all MSWord docs, so it's like the
rename .htm .html *.htm
referenced before, but without the .htm to hang my hat on. I'm sure there's some simple way to signify "at the end of the string" but I haven't found it. I thought it would be $ but no.

Secondly: to strip out the spaces, I'll probably need to use a loop, which I haven't tried yet; how can I make that loop recursive, and de-space the whole directory structure in one go?

Thanks!

RST

First to remove all spaces, and for instance to turn them into underscores _

Code:

rename ' ' '_' *

does the same as:

Code:

for i in *; do mv "$i" `echo $i | tr ' ' '_'`; done

Replace all spaces with underscores when moving files. You're 'moving' them to the same directory so basicly it's renaming spaces to underscores.

I'm not sure what you mean with the .htm part but ..
for renaming all files in a dir to *.htm:

Code:

for i in *; do mv "$i" `echo $i.htm`; done

as an example: finding the end of a string and append .htm to it:

Code:

for i in *; do echo $i|sed 's/$/&.htm/g'; done

$ does mean end of the string
s for subdstitute
g for global, so multiple times per line
& inserts the previously matched characters

/edit: i haven't found a one-liner that replaces spaces with underscores recusively.
My guess is your best option would be a rename bash-script.
Maybe this one: Space Replace 1.0.1: http://www.novell.com/coolsolutions/tools/15601.html

muha,

I should have checked in sooner to let you know how helpful your suggestions have been! I've been using parts of your code, and it has saved me and my colleagues a bunch of time!

Mostly we've used the for loop when working with individual directories, but now I'm getting greedy and want to try what you've provided to do many directories all at once. I just tested the sequence you provided:

for i in `find . -name "*.img" -o -name "*.hdr" | grep -v outputdir`; do cp -i $i ./outputdir/`dirname $i |sed 's/^.\///g'``basename $i | sed 's/con_/_con/g'`; done

It works brilliantly, but there is something I would like to tweak. The .img and .hdr files I need to rename are not directly in 4001 for example, but are in a subdirectory (i.e., 4001/task/faces). I need to get the files from this subdirectory, prepend the 4001 to them and copy them to an output directory. Is there an easy modification for this?

Hey Shelfitz, nice to hear things are working out over there :D
I wrote a small script which I think does what you want:

Code:

#!/bin/bash

# Copy all *.img and *.hdr files from subdirs and rename then to <outputdir><dirnumber>_<con number>.extension

# If the files were not present in the output directory, it gives feedback in the form of:

# 'old filename' -> 'new filename' (because of the -v verbose option in the copy)

# If the files were present in the outputdir it gives NO feedback.



# Rename script that copies files called old, into new:

# old ./4001/task/faces/con_0027.img

# new ./outputdir/4001_con0027.img

# old ./4001/task/faces/con_0027.hdr

# new ./outputdir/4001_con0027.img

# old ./4002/task/faces/con_028.img

# new ./outputdir/4002_con028.img

# etc ..



# usage:

# 1) save this script as rename_script.sh

#    Put it in the same dir as the ./4001 directories and the outputdir.

# 2) Make sure the outputdirectory is present. Set it as variable output_dir

# 3) chmod the script to make it executable: chmod +x rename_script.sh

# 4) When invoked like ./rename_script.sh it will output a test.

#    Look at the output and when that looks ok:

# 5) Invoke the script like so: ./rename_script |sh

#    in order to execute the output of the script (the actual copy commands)



# assumptions:

# 1) all subfolders of ./4001 must start with a letter. In the example t for task

# 2) none of the main folder can start with a letter but must be called as a number: 4001

# 3) It should never overwrite files in the outputdir. See variable copy_overwrite.



# set some variables

# the name for the output directory (set it to what you want)

output_dir="outputdir"

# overwrite file in outputdir when it is already present when copying

copy_overwrite="no"



# The find and copy command. A for loop for each file present.

# First find all files in subdirs called *.hdr and *.img (excluding subdir $output_dir).

# Variable $i is now in the form of ./4001/task/faces/con_0027.img

for i in `find . -name "*.img" -o -name "*.hdr" | grep -v ${output_dir}`; \

# The copy bit; -i --reply=no is not to overwrite files in $output_dir

# -n to delete the trailing endline to keep it on one line

# -v is for verbose mode when copying (not essential)

# What comes out of the echo is: cp -v -i --reply=no ./4001/task/faces/con_0027.img

do echo -n "cp -v -i --reply=${copy_overwrite} ${i} " ; \

# followed by

# ${i%%/[a-zA-Z]*} leaves only the ./4001 part

# In other words, it deletes anything coming after ./4001 which has the form: slash letters; so /task/faces

#

# sed 's#^\.#\.\/'"${output_dir}"'#g'

# This sed converts the output of the echo: ./4001

# First it looks for 'the begin of the sentence' followed immediately by a .

# it then replaces . by ./outputdir and echos out the rest, which is /4001

# So in total this first echo outputs: ./outputdir/4001

echo -n ${i%%/[a-zA-Z]*}| sed 's#^\.#\.\/'"${output_dir}"'#g'; \

# followed by a replace of ./4001/task/faces/con_0027.img into _con0027.img

# What sed does exactly: replace anything (.*) (here: ./4001/task/faces/) followed by con_ and the trailing bit,

# with _con followed by the trailing bit (0027.img)

echo ${i}|sed 's#.*con_#_con#g'; done

All the parts with \ at the end get strung together in one big line, like the previous for-loop you used.
/note: It would be nice to include some more checks to see if the output dir exists and all went well. Maybe some options to feed into the script from the commandline. You could also do a checksum to see if all went well: count the number of files in the input dirs and compare that to the ouput after the copy (or something like a proper hash-checksum). Also, I have not thouroughly tested it yet. Do that yourself in a testdir and try some weird situations to see if still works. I'd say always backup the input dirs before you start (if possible).
I'm sure it's not up to professional standards, but hey: it's free and it works for me! Anyways, have fun :D

The output I get from when testing and running it:

Code:

$ ls -R *

rename_script.sh



4001:

task/



4001/task:

faces/



4001/task/faces:

con_0027.hdr  con_0027.img



4002:

task/



4002/task:

faces/



4002/task/faces:

con_0028.hdr



outputdir:



$ ./rename_script.sh 

cp -v -i --reply=no ./4001/task/faces/con_0027.img ./outputdir/4001_con0027.img

cp -v -i --reply=no ./4001/task/faces/con_0027.hdr ./outputdir/4001_con0027.hdr

cp -v -i --reply=no ./4002/task/faces/con_0028.hdr ./outputdir/4002_con0028.hdr



$ ./rename_script.sh |sh

`./4001/task/faces/con_0027.img' -> `./outputdir/4001_con0027.img'

`./4001/task/faces/con_0027.hdr' -> `./outputdir/4001_con0027.hdr'

`./4002/task/faces/con_0028.hdr' -> `./outputdir/4002_con0028.hdr'



$ ls -R *

rename_script.sh*



4001:

task/



4001/task:

faces/



4001/task/faces:

con_0027.hdr  con_0027.img



4002:

task/



4002/task:

faces/



4002/task/faces:

con_0028.hdr



outputdir:

4001_con0027.hdr  4001_con0027.img  4002_con0028.hdr

In the last line you can see that the outputdir now contains the *.img and *.hdr
If you have questions, feel free to ask!

Muha,

Wow, you're incredible! Thanks for working on this. I did run into a problem though:

$ ./rename_script.sh
cp -v -i --reply=no ./4001/task/faces/con001.hdr ./outputdir/4001./4001/task/faces/con001.hdr
cp -v -i --reply=no ./4001/task/faces/con001.img ./outputdir/4001./4001/task/faces/con001.img
cp -v -i --reply=no ./4002/task/faces/con001.hdr ./outputdir/4002./4002/task/faces/con001.hdr
cp -v -i --reply=no ./4002/task/faces/con001.img ./outputdir/4002./4002/task/faces/con001.img
cp -v -i --reply=no ./4003/task/faces/con001.hdr ./outputdir/4003./4003/task/faces/con001.hdr
cp -v -i --reply=no ./4003/task/faces/con001.img ./outputdir/4003./4003/task/faces/con001.img

$ ./rename_script.sh | sh
`./4001/task/faces/con001.hdr' -> `./outputdir/4001./4001/task/faces/con001.hdr'
cp: cannot create regular file `./outputdir/4001./4001/task/faces/con001.hdr': No such file or directory
`./4001/task/faces/con001.img' -> `./outputdir/4001./4001/task/faces/con001.img'
cp: cannot create regular file `./outputdir/4001./4001/task/faces/con001.img': No such file or directory
`./4002/task/faces/con001.hdr' -> `./outputdir/4002./4002/task/faces/con001.hdr'
cp: cannot create regular file `./outputdir/4002./4002/task/faces/con001.hdr': No such file or directory
`./4002/task/faces/con001.img' -> `./outputdir/4002./4002/task/faces/con001.img'
cp: cannot create regular file `./outputdir/4002./4002/task/faces/con001.img': No such file or directory
`./4003/task/faces/con001.hdr' -> `./outputdir/4003./4003/task/faces/con001.hdr'
cp: cannot create regular file `./outputdir/4003./4003/task/faces/con001.hdr': No such file or directory
`./4003/task/faces/con001.img' -> `./outputdir/4003./4003/task/faces/con001.img'
cp: cannot create regular file `./outputdir/4003./4003/task/faces/con001.img': No such file or directory

I'm not quite sure what's happening here. Your assumptions were spot on (i.e., my directory contains individual subject directories (4001, 4002, etc.) and within each there are identically named subdirectories. My actual subdirectory pathway differs from the example I provided (it is 4001/stats/snod) but I tested things as if were named 4001/task/faces and it still produced an error. By the way, I just created an output directory named outputdir so I did not change the variable name in the script.

Also, one other thing. Is it possible to have the program look in a specific subdirectory in each subject directory (i.e., 4001/stats/snod)? There are other subdirectories that have .img and .hdr files that I do not want to rename at this time.

Many thanks for your help and time, Muha! Your documentation has really helped me understand what you are doing!

Cheers,
SF

/edit:
I see now. Your filenames are different than I thought.
./4001/task/faces/con001.hdr
instead of
./4001/task/faces/con_001.hdr
That's why my script does not work (yet).

Are these filenames consistent? Are they all named:
con<number>.<img or hdr>

- I'll look at /stats/snod
So you want to be able to specify per copy-action which subdirectory you want to use? /task/faces/ or /stats/snod ?