Mass rename and move

TwilightZone · 09-23-2011, 03:38 PM

I got a client I do work for that i need to clean up the files on to make it more coherent for searches. The hardware currently does a test to make sure the imagining hardware works then creates the required file. So in the directory I get a sample and the full output file. What I want to do is rename the final output file the same as the folder, kill all the white spaces and strip out all the hardware info that is after the date in the folder name. It creates files like most cameras do so I get something like the following
folder name: patient name 23Sept2011 09AQ_RE43
contents: sample.avi 09AQ_RE43.avi (imager tag)
all i really need is the full image file named like the folder name and then moved to make searches easier they are getting tired of the folder structure and if you email them for a consult the file name means nothing on it's own without changing it.
This is what i got so far... i rename based on parent folder and move to another directory. Unfortunately I need it to ignore the sample if there is one and just move the full file (egrep -vi sample)? Plus i'd like it to strip the hardware tagging info and put the year in brackets, output file should look like
patient.name.23Sept.(2011).avi rather then
patient.name.23Sept2011.09AQ_RE43.avi which is what i get now... little closer

find ${SOURCE_DIR} -type f -iname "*.avi" -print0 | while read -d $'\0' filename ; do
removeSpaces=$( echo "${filename}" | sed 's@ @.@g' )
parentDirectory=${removeSpaces%/*}
newFilename="${DEST_DIR}/${parentDirectory##*/}.avi"

echo mv "\"${filename}\"" "\"${newFilename}\""
mv "${filename}" "${newFilename}"
done

claudelepoisson · 09-23-2011, 05:26 PM

Code:

find ${SOURCE_DIR} -type f -iname "*.avi" -print0 | while read -d $'\0' filename ; do
removeSpaces=$( echo "${filename}" | sed 's@ @.@g' )
parentDirectory=${removeSpaces%/*}

# Add this line:
parentDirectory=$(sed "s@\(....\)\..*\.\(...\)$@(\1).\2@" <<< ${parentDirectory})
#                 match:   yyyy   .foo.  ext
newFilename="${DEST_DIR}/${parentDirectory##*/}.avi"

echo mv "\"${filename}\"" "\"${newFilename}\""
mv "${filename}" "${newFilename}"
done

TwilightZone · 09-23-2011, 06:12 PM

adding that line in moves the file but it just stripes the name out so it is just .avi
sed: -e expression #1, char 31: unterminated `s' command

also looking at that the sed s@ command it is for patterns which the only constant I have is the year being 4 digits and wanting to strip everything after it , put year in brackets and the correct extension ..(2011).avi First part is a patient name of which they use the entire name as the identifier. So it can be as simple and first and last but that isn't always the case as some cultures they can be much bigger. The more identifiers the better it seems.

claudelepoisson · 09-23-2011, 07:38 PM

Ok sorry for the botched reply,
This time I did actual work and testing.

Code:

[~]$ n="patient name 23Sept2011 09AQ_RE43/09AQ_RE43.avi"
[~]$ echo "$n" | sed "y/ /./;s@\(.*\)\([[:digit:]]\{4\}\)\.\([[:alnum:]_]\{1,\}\)/\3@mv \"$n\" \1(\2)@"
mv "patient name 23Sept2011 09AQ_RE43/09AQ_RE43.avi" patient.name.23Sept(2011).avi
[~]$ n="very long patient name 23Sept2011 d8s0s03_sd9/d8s0s03_sd9.avi"
[~]$ echo "$n" | sed "y/ /./;s@\(.*\)\([[:digit:]]\{4\}\)\.\([[:alnum:]_]\{1,\}\)/\3@mv \"$n\" \1(\2)@"
mv "very long patient name 23Sept2011 d8s0s03_sd9/d8s0s03_sd9.avi" very.long.patient.name.23Sept(2011).avi

[~]$ sed --version  # FYI
GNU sed version 4.2.1

So:

Code:

find ${SOURCE_DIR} -type f -iname "*.avi" -print0 | while read -d $'\0' filename ; do
sed "y/ /./;s@\(.*\)\([[:digit:]]\{4\}\)\.\([[:alnum:]_]\{1,\}\)/\3@mv \"$n\" ${DEST_DIR}/\1(\2)@" <<< "$filename"
done

When you are satisfied, you can append ' | sh' after '"$filename"'. sh will execute the mv command it receives from stdin.

Yours,

TwilightZone · 09-24-2011, 09:58 AM

oddly this works but only partial. Your 2 examples worked fine but on my end I found the output wasn't stripping after the year.
For example I got a HD full motion xray so when I manually do it
n="Bill W Stevens 2010 HD 720p-Noa.avi"
echo "$n" | sed "y/ /./;s@$.*$$[[:digit:]]\{4\}$\.$[[:alnum:]_]\{1,\}$/\3@mv \"$n\" \1(\2)@"
bill.stevens.2010.HD.720p-Noa.avi (is the output)

So it isn't stripping right. The name structure seems to be all over the place the only constant is patient name and a year, not even a full date but years are all 4 digit so at least their is a pattern.

I did find a hiccup on my end some of the folders are within a subfolder. So i need to keep the rename at a depth of 1 from what is declared as the SOURCE_DIR and if there are multiple directories below that rename with a number after the year, (2011).1.avi (2011).2.avi. Some are images as well as other video formats, jpg, avi, mov looks like some other camera formats all depends but the extension is easy to declare.

thx for the help as well m8 the thought on doing this manually was going to drive me to drink.

claudelepoisson · 09-24-2011, 10:51 AM

Ok, so with this regex, I rely on the year being 4 digits and the extension being a dot and three characters.

Code:

[~]$ n="Bill W Stevens 2010 HD 720p-Noa.avi
> patient name 23Sept2011 09AQ_RE43/09AQ_RE43.avi
> very long patient name 23Sept2011 d8s0s03_sd9/d8s0s03_sd9.avi"
[~]$ echo "$n" | sed "y/ /./;s@\(.*\)\([[:digit:]]\{4\}\).*\(\..\{3\}\)@\1(\2)\3@"
Bill.W.Stevens.(2010).avi
patient.name.23Sept(2011).avi
very.long.patient.name.23Sept(2011).avi

However, the first .* is greedy so:

Code:

[~]$ n="Bill W Stevens 2010 HD 1080p-Noa.avi"
[~]$                        #  ^^^^
[~]$ echo "$n" | sed "y/ /./;s@\(.*\)\([[:digit:]]\{4\}\).*\(\..\{3\}\)@mv \"$n\" \1(\2)\3@"
mv "Bill W Stevens 2010 HD 1080p-Noa.avi" Bill.W.Stevens.2010.HD.(1080).avi

But if we know that the "HD 1080p-Noa" part will repeat, it's an anchor we can use:

Code:

[~]$ n="Bill W Stevens 2010 HD 1080p-Noa/HD 1080p-Noa.avi"
[~]$ echo "$n" | sed "y/ /./;s@\(.*\)\([[:digit:]]\{4\}\)\.\(.\{1,\}\)/\3\(\..\{3\}\)@mv \"$n\" \1(\2)\4@"
mv "Bill W Stevens 2010 HD 1080p-Noa/HD 1080p-Noa.avi" Bill.W.Stevens.(2010).avi

[~]$ n="Bill W Stevens 23sept2010 HD 1080p-Noa/HD 1080p-Noa.avi"
[~]$ echo "$n" | sed "y/ /./;s@\(.*\)\([[:digit:]]\{4\}\)\.\(.\{1,\}\)/\3\(\..\{3\}\)@mv \"$n\" \1(\2)\4@"
mv "Bill W Stevens 23sept2010 HD 1080p-Noa/HD 1080p-Noa.avi" Bill.W.Stevens.23sept(2010).avi

How are your subfolders named?

TwilightZone · 09-24-2011, 11:14 AM

oddly enough they are MD1, MD2 etc, seen both upper and lower case but never more then a depth of 1. Not sure what camera is doing it but it seems to create the movies in a separate folder from the images it takes. Nothing I've ever seen on any of my cameras but some of these lab one hooked up to microscope are rather unique. Guess if i'm taking pics of a petri dish my evening has gone terribly wrong...

As to the HD 1080p-Noa repeating it doesn't, that specific tag looks like it was created by the intern taking the image / .mov file Noa is short for Noah the resolution info is accurate but you can get the metadata by checking properties. That is why I want to strip everything after the year most of the info is created by the camera and means nothing then some is tagged by whomever transferred it to the server but in most cases that info only has meaning to them and leaves everyone else guessing. That instance just stuck out as it was one of the few I could guess what it meant. Those however are rare. You think you can read a doctors handwriting it doesn't get better when tagging.

Where i'm using a find command to get the avi file is there a way to read the name and just delete everything after the year and put the year in brackets. The first 4 digit sequence is always the year looking at the files and I can't think of an instance where a patient name would be a numeric value. Also sample.avi have no date so just need to ignore those. I general purge those at a later date if there is no issues with the imager and they need to refer to it for calibration.

Question on the first example the dot and 3 characters it that defined as the end of the string or just a variable? reason I ask is I run a sed command to remove white spaces and replace with dots as a general rules just because white spaces and linux tend to create more work.

claudelepoisson · 09-24-2011, 11:41 AM

Code:

[~]$ n="Bill W Stevens 23sept2010 HD 1080p-Noa/HD 1080p-Noa.avi"
[~]$ echo "$n" | sed "y/ /./;s@\(.*\)\([[:digit:]]\{4\}\)\.\(.\{1,\}\)/\(\|.*\([[:digit:]]\)/\)\3\(\..\{3\}\)@mv \"$n\" \1(\2).\5\6@;s/\.\././"
mv "Bill W Stevens 23sept2010 HD 1080p-Noa/HD 1080p-Noa.avi" Bill.W.Stevens.23sept(2010).avi

[wip]$ n="Bill W Stevens 23sept2010 HD 1080p-Noa/foo1/HD 1080p-Noa.avi"
[~]$ echo "$n" | sed "y/ /./;s@\(.*\)\([[:digit:]]\{4\}\)\.\(.\{1,\}\)/\(\|.*\([[:digit:]]\)/\)\3\(\..\{3\}\)@mv \"$n\" \1(\2).\5\6@;s/\.\././"
mv "Bill W Stevens 23sept2010 HD 1080p-Noa/foo1/HD 1080p-Noa.avi" Bill.W.Stevens.23sept(2010).1.avi

How is that working?

grail · 09-24-2011, 01:04 PM

Here is another option:

Code:

#!/bin/bash

while read -r name
do
    new=${name// /.}
    new=${new,,}
    new=$(echo $new | sed -r 's/([0-9]{4}).*/(\1)/').avi

    echo mv "$name" "$new"
done< <(find -type f -name '*.avi' ! -name 'sample*' -printf "%P\n")

Just remove the echo from in front of mv and the command will be performed.

TwilightZone · 09-25-2011, 09:13 PM

wow ok that is very close to what I need m8. It renames all lower case though is the only hiccup. They are proper names so first letter is capitalized currently would like to retain that.

grail · 09-25-2011, 09:56 PM

So does that mean you do not want anything to have case changed?
Just remove:

Code:

new=${new,,}

TwilightZone · 09-25-2011, 10:10 PM

no being proper names they should all be first letter capitalized. That is the assumption anyway having patient data I isolated the server so I can't check until I'm there tomorrow. This all gets me a lot close though now i just need to deal with the folder depth issue for the odd ones that have a depth of 2. Master file is patient name, year. sub folder is MD1, MD2 etc. Would be nice to rename the file a combo of both folder names patient name, year, MD1.avi