[SOLVED] how do i remove ALL special characters in file names with one line command

atjurhs · 02-22-2017, 12:15 PM

hi guys,

daily i come across files that were not created by me but by windows users, who i have no control over their naming conventions, so they name files with white spaces and special characters.

as an example:

Code:

 1234_rev(a) draft [1].pdf

i have written a script that will remove any one of the special characters or white spaces and replace it with an underscore, below is my command that will remove just white spaces

Code:

find . -type f -name "* *" | while read src; do mv "$src" `echo $src | tr " " "_"`; done

and to remove [

Code:

find . -type f -name "*[*" | while read src; do mv "$src" `echo $src | tr "[" "_"`; done

etc. etc.

so i have to run the command once for white spaces and then once for each special character (this can add up to several commands to rename the files) so i'd like to give just one command to remove white spaces and special characters all at once.

Todd

JeremyBoden · 02-22-2017, 12:35 PM

Are '[' and ']' special characters?

I think you have to define what names are acceptable.

szboardstretcher · 02-22-2017, 12:43 PM

Finding names with whitespaces AND ['s and ]'s could be done with:

Code:

find . -type f  \( -name "*\ *" -o -name "*[*" -o -name "*]*" \)

kakistocrat · 02-22-2017, 01:21 PM

Look at the manual page for tr more closely, it can include a set of characters. And any of those characters can even be class of characters.

Code:

tr -s '[:space:][]()' '_____'

You can also replace those outside of the set you give:

Code:

tr -cs '[:alnum:]\.-_' '____'

r3sistance · 02-22-2017, 01:29 PM

I take it you are looking for something more a long the lines of this?

Code:

# touch Dnfs^754£.txt
# touch "Dnfs ^754£.txt"
# touch "Dnfs[ ^754£.txt"
# find . -iname "*[\[\] £#@~]*" ! -name "*mozilla*"
./Dnfs^754£.txt
./Dnfs ^754£.txt
./Dnfs[ ^754£.txt
# find . -iname "*[\[\] £#@~]*" ! -name "*mozilla*" | tr "[\[\] £#@~]" "_"
./Dnfs^754__.txt
./Dnfs_^754__.txt
./Dnfs__^754__.txt
# find . -iname "*[\[\]^ £#@~]*" ! -name "*mozilla*" | tr "[\[\]^ £#@~]" "_"
./Dnfs_754__.txt
./Dnfs__754__.txt
./Dnfs___754__.txt
# find . -iname "*[\[\]^ £#@~]*" ! -name "*mozilla*" | tr --delete "[\[\]^ £#@~]"
./Dnfs754.txt
./Dnfs754.txt
./Dnfs754.txt

Well this leads on to this

Code:

# cat rename.sh
#!/bin/bash
files=`find . -iname "*[\[\]^ £#@~]*" ! -name "*mozilla*"`

while read -r line; do
  newline=`echo "$line" | tr --delete "[\[\]^ £#@~]"`
  echo "mv \"$line\" $newline"
done <<< "$files"

Then to run it.

Code:

# sh rename.sh
mv "./Dnfs^754£.txt" ./Dnfs754.txt
mv "./Dnfs ^754£.txt" ./Dnfs754.txt
mv "./Dnfs[ ^754£.txt" ./Dnfs754.txt

You will note the echo is there to test the output is as you'd expect before actually running.

atjurhs · 02-22-2017, 01:41 PM

Turbo,

both of your commands run just fine, but both of them also leave a trailing underscore

Code:

  1234_rev_a_draft_1_.pdf_

i know that technically linux doesn't care, but for prettiness, how do i fix that?

r3 aaah, no

kakistocrat · 02-22-2017, 01:53 PM

Ok. The replacement was a bit aggressive, I was thinking along slightly different lines ( -print0 ) and the extra underscore would be because of the [:space:] class. Substitute it with a regular space in the first one or add a newline in the second one and it should stop replacing the trailing return.

Code:

tr -cs '\n[:alnum:]\.-_' '_'

tr works on what you actually give it and -exec separates the file names with a newline. Again see the manual page for both programs.

For what it's worth, there is a perl script rename on most systems that can process file names with s/// tr and more

dave@burn-it.co.uk · 02-22-2017, 02:28 PM

Your comment about Windows and not having any control over naming conventions is incorrect. In fact you would struggle to create those file names under Windows GUI.
They are far more likely to have been created by a piece of software that was written by someone without specific computer knowledge.

atjurhs · 02-22-2017, 02:44 PM

thanks Turbo! although they are my "friend" the man pages are often cryptic to me

dave, that's very possible, so i should shoot a programmer for this

dave@burn-it.co.uk · 02-22-2017, 03:06 PM

The large software package suppliers tend to use inexperienced people to write the boring bits and they like to use "meaningful names" even if those names contravene all sensible standards.

And talking of meaningful names:
I have been on call out most of my life for various corporations. Most of their serious computing processing is done in batch overnight so it was rare for me to get an uninterupted night with nothing going wrong.
One program that failed and required debugging was very cleverly written using very funny names and read almost like a story rather than a program. Not exactly what you want at 3:30 in the morning.

MadeInGermany · 02-22-2017, 03:23 PM

Code:

mv "$filename" `echo "$filename" | tr "[:punct:][:space:]" ":"`

That means replace all punctuation and space characters to colons.
Then send them back to your Windows guys and watch them fighting with the colons!

Honestly, in shell/commandline always have "quotes" around the filenames, and you don't need to rename them.

JeremyBoden · 02-22-2017, 03:38 PM

Are 'filename[[' and 'filename[[[[[' to be renamed to the same name?

Perhaps if 'filename' already exists, you need to generate a name such as 'filename(1)' or alternatively halt and ask for a bit of interactive intelligence!

dave@burn-it.co.uk · 02-22-2017, 06:14 PM

Windows file system would add the subscript if names ended up the same.

kakistocrat · 02-23-2017, 01:12 AM

Quote:

Originally Posted by MadeInGermany

Honestly, in shell/commandline always have "quotes" around the filenames, and you don't need to rename them.

atjurhs, perhaps we should ask why you wish to rename the files? If it's that the weird characters are causing difficulties renaming and copying using variables, then you can probably solve that by wrapping the variables in quotes.

atjurhs · 02-23-2017, 07:38 AM

Turbo,

for me it's far easier when dealing with many files a day to just change out the special characters of all the files once, then when i need to open whichever file with an application from the cmnd line, tab complete works fine

thanks for your help!

Todd