LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-22-2017, 12:15 PM   #1
atjurhs
Member
 
Registered: Aug 2012
Posts: 258

Rep: Reputation: Disabled
how do i remove ALL special characters in file names with one line command


hi guys,

daily i come across files that were not created by me but by windows users, who i have no control over their naming conventions, so they name files with white spaces and special characters.

as an example:

Code:
 1234_rev(a) draft [1].pdf
i have written a script that will remove any one of the special characters or white spaces and replace it with an underscore, below is my command that will remove just white spaces

Code:
find . -type f -name "* *" | while read src; do mv "$src" `echo $src | tr " " "_"`; done
and to remove [

Code:
find . -type f -name "*[*" | while read src; do mv "$src" `echo $src | tr "[" "_"`; done
etc. etc.

so i have to run the command once for white spaces and then once for each special character (this can add up to several commands to rename the files) so i'd like to give just one command to remove white spaces and special characters all at once.

Todd
 
Old 02-22-2017, 12:35 PM   #2
JeremyBoden
Senior Member
 
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,937

Rep: Reputation: 498Reputation: 498Reputation: 498Reputation: 498Reputation: 498
Are '[' and ']' special characters?

I think you have to define what names are acceptable.
 
Old 02-22-2017, 12:43 PM   #3
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,278

Rep: Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693
Finding names with whitespaces AND ['s and ]'s could be done with:

Code:
find . -type f  \( -name "*\ *" -o -name "*[*" -o -name "*]*" \)
 
Old 02-22-2017, 01:21 PM   #4
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 6,896
Blog Entries: 3

Rep: Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584
Look at the manual page for tr more closely, it can include a set of characters. And any of those characters can even be class of characters.

Code:
tr -s '[:space:][]()' '_____'
You can also replace those outside of the set you give:

Code:
tr -cs '[:alnum:]\.-_' '____'
 
Old 02-22-2017, 01:29 PM   #5
r3sistance
Senior Member
 
Registered: Mar 2004
Location: UK
Distribution: CentOS 6/7
Posts: 1,375

Rep: Reputation: 217Reputation: 217Reputation: 217
I take it you are looking for something more a long the lines of this?

Code:
# touch Dnfs^754£.txt
# touch "Dnfs ^754£.txt"
# touch "Dnfs[ ^754£.txt"
# find . -iname "*[\[\] £#@~]*" ! -name "*mozilla*"
./Dnfs^754£.txt
./Dnfs ^754£.txt
./Dnfs[ ^754£.txt
# find . -iname "*[\[\] £#@~]*" ! -name "*mozilla*" | tr "[\[\] £#@~]" "_"
./Dnfs^754__.txt
./Dnfs_^754__.txt
./Dnfs__^754__.txt
# find . -iname "*[\[\]^ £#@~]*" ! -name "*mozilla*" | tr "[\[\]^ £#@~]" "_"
./Dnfs_754__.txt
./Dnfs__754__.txt
./Dnfs___754__.txt
# find . -iname "*[\[\]^ £#@~]*" ! -name "*mozilla*" | tr --delete "[\[\]^ £#@~]"
./Dnfs754.txt
./Dnfs754.txt
./Dnfs754.txt
Well this leads on to this

Code:
# cat rename.sh
#!/bin/bash
files=`find . -iname "*[\[\]^ £#@~]*" ! -name "*mozilla*"`

while read -r line; do
  newline=`echo "$line" | tr --delete "[\[\]^ £#@~]"`
  echo "mv \"$line\" $newline"
done <<< "$files"
Then to run it.

Code:
# sh rename.sh
mv "./Dnfs^754£.txt" ./Dnfs754.txt
mv "./Dnfs ^754£.txt" ./Dnfs754.txt
mv "./Dnfs[ ^754£.txt" ./Dnfs754.txt
You will note the echo is there to test the output is as you'd expect before actually running.

Last edited by r3sistance; 02-22-2017 at 01:35 PM.
 
Old 02-22-2017, 01:41 PM   #6
atjurhs
Member
 
Registered: Aug 2012
Posts: 258

Original Poster
Rep: Reputation: Disabled
Turbo,

both of your commands run just fine, but both of them also leave a trailing underscore

Code:
  1234_rev_a_draft_1_.pdf_
i know that technically linux doesn't care, but for prettiness, how do i fix that?

r3 aaah, no

Last edited by atjurhs; 02-22-2017 at 01:43 PM. Reason: added reply to r3
 
Old 02-22-2017, 01:53 PM   #7
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 6,896
Blog Entries: 3

Rep: Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584
Ok. The replacement was a bit aggressive, I was thinking along slightly different lines ( -print0 ) and the extra underscore would be because of the [:space:] class. Substitute it with a regular space in the first one or add a newline in the second one and it should stop replacing the trailing return.

Code:
tr -cs '\n[:alnum:]\.-_' '_'
tr works on what you actually give it and -exec separates the file names with a newline. Again see the manual page for both programs.

For what it's worth, there is a perl script rename on most systems that can process file names with s/// tr and more

Last edited by Turbocapitalist; 02-22-2017 at 01:55 PM.
 
Old 02-22-2017, 02:28 PM   #8
dave@burn-it.co.uk
Member
 
Registered: Sep 2011
Distribution: Puppy
Posts: 601

Rep: Reputation: 172Reputation: 172
Your comment about Windows and not having any control over naming conventions is incorrect. In fact you would struggle to create those file names under Windows GUI.
They are far more likely to have been created by a piece of software that was written by someone without specific computer knowledge.
 
Old 02-22-2017, 02:44 PM   #9
atjurhs
Member
 
Registered: Aug 2012
Posts: 258

Original Poster
Rep: Reputation: Disabled
thanks Turbo! although they are my "friend" the man pages are often cryptic to me

dave, that's very possible, so i should shoot a programmer for this
 
Old 02-22-2017, 03:06 PM   #10
dave@burn-it.co.uk
Member
 
Registered: Sep 2011
Distribution: Puppy
Posts: 601

Rep: Reputation: 172Reputation: 172
The large software package suppliers tend to use inexperienced people to write the boring bits and they like to use "meaningful names" even if those names contravene all sensible standards.


And talking of meaningful names:
I have been on call out most of my life for various corporations. Most of their serious computing processing is done in batch overnight so it was rare for me to get an uninterupted night with nothing going wrong.
One program that failed and required debugging was very cleverly written using very funny names and read almost like a story rather than a program. Not exactly what you want at 3:30 in the morning.
 
Old 02-22-2017, 03:23 PM   #11
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,452

Rep: Reputation: 1061Reputation: 1061Reputation: 1061Reputation: 1061Reputation: 1061Reputation: 1061Reputation: 1061Reputation: 1061
Code:
mv "$filename" `echo "$filename" | tr "[:punct:][:space:]" ":"`
That means replace all punctuation and space characters to colons.
Then send them back to your Windows guys and watch them fighting with the colons!

Honestly, in shell/commandline always have "quotes" around the filenames, and you don't need to rename them.

Last edited by MadeInGermany; 02-22-2017 at 03:25 PM.
 
Old 02-22-2017, 03:38 PM   #12
JeremyBoden
Senior Member
 
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,937

Rep: Reputation: 498Reputation: 498Reputation: 498Reputation: 498Reputation: 498
Are 'filename[[' and 'filename[[[[[' to be renamed to the same name?

Perhaps if 'filename' already exists, you need to generate a name such as 'filename(1)' or alternatively halt and ask for a bit of interactive intelligence!
 
Old 02-22-2017, 06:14 PM   #13
dave@burn-it.co.uk
Member
 
Registered: Sep 2011
Distribution: Puppy
Posts: 601

Rep: Reputation: 172Reputation: 172
Windows file system would add the subscript if names ended up the same.
 
Old 02-23-2017, 01:12 AM   #14
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 6,896
Blog Entries: 3

Rep: Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584Reputation: 3584
Quote:
Originally Posted by MadeInGermany View Post
Honestly, in shell/commandline always have "quotes" around the filenames, and you don't need to rename them.
atjurhs, perhaps we should ask why you wish to rename the files? If it's that the weird characters are causing difficulties renaming and copying using variables, then you can probably solve that by wrapping the variables in quotes.
 
Old 02-23-2017, 07:38 AM   #15
atjurhs
Member
 
Registered: Aug 2012
Posts: 258

Original Poster
Rep: Reputation: Disabled
Turbo,

for me it's far easier when dealing with many files a day to just change out the special characters of all the files once, then when i need to open whichever file with an application from the cmnd line, tab complete works fine

thanks for your help!

Todd
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
KDE: File Names with special characters problem gookank Linux - General 2 02-23-2010 01:50 PM
Special characters in file names Jinouchi Linux - Newbie 7 07-25-2009 11:48 AM
converting linux (reiserfs) data to NTFS (file names contain special characters) jefffq Linux - General 2 03-08-2009 03:07 PM
How to remove file with name containing only special characters abhisheknayak Linux - Newbie 5 07-04-2008 10:53 AM
How to rename file while copying - and remove special characters corporal79 Linux - General 3 10-11-2007 04:16 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 09:34 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration