LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 02-05-2008, 01:53 PM   #1
TheFutonEng
LQ Newbie
 
Registered: Dec 2007
Posts: 2

Rep: Reputation: 0
Remove whitespace in file and directory names using bash


Hello, I am trying to write a very basic shell script to replace white space in file and directory names with underscores. I realize that there are probably several better ways to do this. Here is what I have come up with so far with two small scripts:

wsfind
Code:
#!/bin/bash

directory=$1
find $directory -name '* *' -exec wsrem \"{}\" \;
wsrem
Code:
#!/bin/bash

input=$1
echo $input
files=$(echo $input | tr ' ' _ | tr \" ' ') #replace white space with _ and remove quotes.
echo $files
mv -v $input $files
I have created some test files/directories and here is the output I get:

Code:
root@underwearless:~# wsrem test/
"test/this is a test"
test/this_is_a_test
mv: target `test/this_is_a_test' is not a directory
"test/this is a test/this is a test"
test/this_is_a_test/this_is_a_test
mv: target `test/this_is_a_test/this_is_a_test' is not a directory
"test/fjkd ls"
test/fjkd_ls
mv: target `test/fjkd_ls' is not a directory
Bash seems to be creating the string I want but they aren't be interpreted by mv correctly. Any thought on how to fix what I have or perhaps a better idea on how to implement this?
 
Old 02-05-2008, 03:29 PM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
rename 's/ /_/g' file name

Using the perl rename command.
 
1 members found this post helpful.
Old 02-05-2008, 04:25 PM   #3
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,004
Blog Entries: 11

Rep: Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903
Quote:
Originally Posted by TheFutonEng View Post
Hello, I am trying to write a very basic shell script to replace white space in file and directory names with underscores. I realize that there are probably several better ways to do this. Here is what I have come up with so far with two small scripts:

wsfind
Code:
#!/bin/bash

directory=$1
find $directory -name '* *' -exec wsrem \"{}\" \;
wsrem
Code:
#!/bin/bash

input=$1
echo $input
files=$(echo $input | tr ' ' _ | tr \" ' ') #replace white space with _ and remove quotes.
echo $files
mv -v $input $files
I have created some test files/directories and here is the output I get:

Code:
root@underwearless:~# wsrem test/
"test/this is a test"
test/this_is_a_test
mv: target `test/this_is_a_test' is not a directory
"test/this is a test/this is a test"
test/this_is_a_test/this_is_a_test
mv: target `test/this_is_a_test/this_is_a_test' is not a directory
"test/fjkd ls"
test/fjkd_ls
mv: target `test/fjkd_ls' is not a directory
Bash seems to be creating the string I want but they aren't be interpreted by mv correctly. Any thought on how to fix what I have or perhaps a better idea on how to implement this?

Without having spent too much thought or testing on
it I'd guess that the problem is that you're not
quoting the original string. Bash thinks that you're
trying to move several files (e.g. mv "test/this" "is"
"a" test" test/this_is_a_test) to some other directory
rather than renaming a file. And the target is not (an
existing) directory. If you tried mv -v "$input" "$files"
it should work?


Cheers,
Tink
 
Old 02-05-2008, 06:06 PM   #4
Telemachos
Member
 
Registered: May 2007
Distribution: Debian
Posts: 754

Rep: Reputation: 59
Quote:
Originally Posted by David the H. View Post
rename 's/ /_/g' file name

Using the perl rename command.
Maybe I'm wrong, but I don't think that what you wrote is functional Perl. Something like this would work though:
Code:
#!/usr/bin/perl
use strict;
use warnings;

foreach my $file (@ARGV) {
    (my $new_name = $file) =~ s/ /_/g;
    rename $file, $new_name;
}
Whatever files you specify on the command line (and you can use * to mean everything in the current directory that this rule applies to) would have single spaces changed to underscores.
 
Old 02-05-2008, 06:58 PM   #5
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,004
Blog Entries: 11

Rep: Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903
Quote:
Originally Posted by Telemachos View Post
Maybe I'm wrong, but I don't think that what you wrote is functional Perl.
You are wrong ...

rename is a perl-script included in debian based distros,
and the invocation "David the H" was using is perfectly
fine ... he wasn't using a perl-function in a wrong way.



Cheers,
Tink
 
Old 02-05-2008, 08:03 PM   #6
Telemachos
Member
 
Registered: May 2007
Distribution: Debian
Posts: 754

Rep: Reputation: 59
Quote:
Originally Posted by Tinkster View Post
You are wrong ...

rename is a perl-script included in debian based distros,
and the invocation "David the H" was using is perfectly
fine ... he wasn't using a perl-function in a wrong way.
Thanks for the info: I didn't know about "rename" as a command by itself. I took it for a very ugly piece of "real" perl. Thanks.
 
Old 02-05-2008, 08:32 PM   #7
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by TheFutonEng View Post
Hello, I am trying to write a very basic shell script to replace white space in file and directory names with underscores. I realize that there are probably several better ways to do this. Here is what I have come up with so far with two small scripts:

wsfind
Code:
#!/bin/bash

directory=$1
find $directory -name '* *' -exec wsrem \"{}\" \;
wsrem
Code:
#!/bin/bash

input=$1
echo $input
files=$(echo $input | tr ' ' _ | tr \" ' ') #replace white space with _ and remove quotes.
echo $files
mv -v $input $files
I have created some test files/directories and here is the output I get:

Code:
root@underwearless:~# wsrem test/
"test/this is a test"
test/this_is_a_test
mv: target `test/this_is_a_test' is not a directory
"test/this is a test/this is a test"
test/this_is_a_test/this_is_a_test
mv: target `test/this_is_a_test/this_is_a_test' is not a directory
"test/fjkd ls"
test/fjkd_ls
mv: target `test/fjkd_ls' is not a directory
Bash seems to be creating the string I want but they aren't be interpreted by mv correctly. Any thought on how to fix what I have or perhaps a better idea on how to implement this?
1) make sure you have created the "test" directory
2) are you running your script in "test" directory or somewhere else.
3) you can make use of the -p switch of mv command.
4) as tinkster has mentioned, use double quotes
 
Old 02-08-2008, 06:55 PM   #8
TheFutonEng
LQ Newbie
 
Registered: Dec 2007
Posts: 2

Original Poster
Rep: Reputation: 0
Using 'rename' worked out very well. Thanks for all of your help.
 
Old 03-27-2011, 02:06 PM   #9
rmcban
LQ Newbie
 
Registered: Apr 2010
Posts: 4

Rep: Reputation: 0
Quote:
Originally Posted by David the H. View Post
rename 's/ /_/g' file name
This worked perfectly for me.
Thx to the original poster.
 
Old 04-30-2011, 12:01 PM   #10
FrDarryl
LQ Newbie
 
Registered: Feb 2011
Posts: 2

Rep: Reputation: 0
Whitespace cleanup and more

The detox command is pretty useful, although not nearly as much fun as writing awk, sed and Perl scripts.
 
Old 02-21-2012, 09:05 PM   #11
edrubins
LQ Newbie
 
Registered: Feb 2012
Posts: 2

Rep: Reputation: Disabled
Quote:
Originally Posted by FrDarryl View Post
The detox command is pretty useful, although not nearly as much fun as writing awk, sed and Perl scripts.
Thanks for pointing me to the detox command. It sure saved a lot of work!

Best, Ed
 
Old 02-22-2012, 07:10 AM   #12
FrDarryl
LQ Newbie
 
Registered: Feb 2011
Posts: 2

Rep: Reputation: 0
Quote:
Originally Posted by edrubins View Post
Thanks for pointing me to the detox command. It sure saved a lot of work!

Best, Ed
Happy to help.

If you get in a bind with a filename you can't address with detox or any other command (like a failed inline bash command), you can mv or rm the file using its (file-system-unique) inode.

Here's an example.

669:02/22@11.54 ~/tmp/ ls -1
file1
file2
670:02/22@11.54 ~/tmp/ touch "file3 is a failed >>! UNIX redirection"
671:02/22@11.54 ~/tmp/ ls -1
file1
file2
file3 is a failed >>! UNIX redirection
672:02/22@11.54 ~/tmp/ ls -i -1
17 file1
43 file2
137 file3 is a failed >>! UNIX redirection
673:02/22@11.55 ~/tmp/ find . -inum 137 -exec mv {} file3 \;
674:02/22@11.55 ~/tmp/ ls -i -1
17 file1
43 file2
137 file3
675:02/22@11.56 ~/tmp/ find . -inum 137 -exec rm {} \;
676:02/22@11.56 ~/tmp/ ls -i -1
17 file1
43 file2
677:02/22@11.57 ~/tmp/

Last edited by FrDarryl; 02-22-2012 at 07:14 AM.
 
Old 02-25-2012, 07:44 PM   #13
edrubins
LQ Newbie
 
Registered: Feb 2012
Posts: 2

Rep: Reputation: Disabled
Inodes - it's been a long time.

Quote:
Originally Posted by FrDarryl View Post
Happy to help.

If you get in a bind with a filename you can't address with detox or any other command (like a failed inline bash command), you can mv or rm the file using its (file-system-unique) inode.

Here's an example.

669:02/22@11.54 ~/tmp/ ls -1
file1
file2
670:02/22@11.54 ~/tmp/ touch "file3 is a failed >>! UNIX redirection"
671:02/22@11.54 ~/tmp/ ls -1
file1
file2
file3 is a failed >>! UNIX redirection
672:02/22@11.54 ~/tmp/ ls -i -1
17 file1
43 file2
137 file3 is a failed >>! UNIX redirection
673:02/22@11.55 ~/tmp/ find . -inum 137 -exec mv {} file3 \;
674:02/22@11.55 ~/tmp/ ls -i -1
17 file1
43 file2
137 file3
675:02/22@11.56 ~/tmp/ find . -inum 137 -exec rm {} \;
676:02/22@11.56 ~/tmp/ ls -i -1
17 file1
43 file2
677:02/22@11.57 ~/tmp/
I hadn't thought about using iodes - in fact I haven't haven't thought about inodes in years.
 
Old 02-26-2012, 04:47 AM   #14
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269
Here's my script:

Code:
for i in *
do
  mv "$i" $(echo "$i" | tr [:upper:] [:lower:] | tr [:blank:] _ | sed 's|[^-._a-z0-9]||g')
done
This also makes everything lower case, but it can be modified not to do that:

Code:
for i in *
do
  mv "$i" $(echo "$i" | tr [:blank:] _ | sed 's|[^-._a-zA-Z0-9]||g')
done
 
Old 02-26-2012, 01:35 PM   #15
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
I'd recommend using a series of parameter substitutions, rather than a single ugly chain of tr/sed commands. You can simply add, modify, or remove substitutions as necessary to get the name pattern you want.

Code:
shopt -s extglob			#needed for the extended glob below

for i in *; do

	new=${i,,}			#lowercasing requires bash v.4+
	new=${new//+([[:blank:]])/_}	#see note below
	new=${new//[^-._a-z0-9]}

	mv "$i" "$new"

done
Note: The +() exended globbing pattern can be used to compress all sequential blank space into a single underscore. Remove it to have each character substituted individually. You might also consider using [:space:] instead, to catch potential newline characters as well.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
HELP! Script to RENAME/REMOVE ( ) From File NAMES!! xberetta21 Linux - Newbie 4 01-29-2008 02:10 PM
find command and file names with whitespace sir_woland Linux - General 8 10-04-2007 08:10 PM
remove * from file nameS zchoyt Linux - Newbie 1 02-13-2005 05:33 PM
remove whitespace at end of file FunkyRes Programming 2 10-05-2004 01:31 AM
Using sed in bash to remove whitespace jimieee Programming 3 01-28-2004 11:33 AM


All times are GMT -5. The time now is 04:28 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration