Remove whitespace in file and directory names using bash

TheFutonEng · 02-05-2008, 12:53 PM

Hello, I am trying to write a very basic shell script to replace white space in file and directory names with underscores. I realize that there are probably several better ways to do this. Here is what I have come up with so far with two small scripts:

wsfind

Code:

#!/bin/bash

directory=$1
find $directory -name '* *' -exec wsrem \"{}\" \;

wsrem

Code:

#!/bin/bash

input=$1
echo $input
files=$(echo $input | tr ' ' _ | tr \" ' ') #replace white space with _ and remove quotes.
echo $files
mv -v $input $files

I have created some test files/directories and here is the output I get:

Code:

root@underwearless:~# wsrem test/
"test/this is a test"
test/this_is_a_test
mv: target `test/this_is_a_test' is not a directory
"test/this is a test/this is a test"
test/this_is_a_test/this_is_a_test
mv: target `test/this_is_a_test/this_is_a_test' is not a directory
"test/fjkd ls"
test/fjkd_ls
mv: target `test/fjkd_ls' is not a directory

Bash seems to be creating the string I want but they aren't be interpreted by mv correctly. Any thought on how to fix what I have or perhaps a better idea on how to implement this?

David the H. · 02-05-2008, 02:29 PM

rename 's/ /_/g' file name

Using the perl rename command.

Tinkster · 02-05-2008, 03:25 PM

Quote:

Originally Posted by TheFutonEng

Hello, I am trying to write a very basic shell script to replace white space in file and directory names with underscores. I realize that there are probably several better ways to do this. Here is what I have come up with so far with two small scripts:

wsfind

Code:

#!/bin/bash

directory=$1
find $directory -name '* *' -exec wsrem \"{}\" \;

wsrem

Code:

#!/bin/bash

input=$1
echo $input
files=$(echo $input | tr ' ' _ | tr \" ' ') #replace white space with _ and remove quotes.
echo $files
mv -v $input $files

I have created some test files/directories and here is the output I get:

Code:

root@underwearless:~# wsrem test/
"test/this is a test"
test/this_is_a_test
mv: target `test/this_is_a_test' is not a directory
"test/this is a test/this is a test"
test/this_is_a_test/this_is_a_test
mv: target `test/this_is_a_test/this_is_a_test' is not a directory
"test/fjkd ls"
test/fjkd_ls
mv: target `test/fjkd_ls' is not a directory

Bash seems to be creating the string I want but they aren't be interpreted by mv correctly. Any thought on how to fix what I have or perhaps a better idea on how to implement this?

Without having spent too much thought or testing on
it I'd guess that the problem is that you're not
quoting the original string. Bash thinks that you're
trying to move several files (e.g. mv "test/this" "is"
"a" test" test/this_is_a_test) to some other directory
rather than renaming a file. And the target is not (an
existing) directory. If you tried mv -v "$input" "$files"
it should work?

Cheers,
Tink

Telemachos · 02-05-2008, 05:06 PM

Quote:

Originally Posted by David the H.

rename 's/ /_/g' file name

Using the perl rename command.

Maybe I'm wrong, but I don't think that what you wrote is functional Perl. Something like this would work though:

Code:

#!/usr/bin/perl
use strict;
use warnings;

foreach my $file (@ARGV) {
    (my $new_name = $file) =~ s/ /_/g;
    rename $file, $new_name;
}

Whatever files you specify on the command line (and you can use * to mean everything in the current directory that this rule applies to) would have single spaces changed to underscores.

Tinkster · 02-05-2008, 05:58 PM

Quote:

Originally Posted by Telemachos

Maybe I'm wrong, but I don't think that what you wrote is functional Perl.

You are wrong ...

rename is a perl-script included in debian based distros,
and the invocation "David the H" was using is perfectly
fine ... he wasn't using a perl-function in a wrong way.

Cheers,
Tink

Telemachos · 02-05-2008, 07:03 PM

Quote:

Originally Posted by Tinkster

You are wrong ...

rename is a perl-script included in debian based distros,
and the invocation "David the H" was using is perfectly
fine ... he wasn't using a perl-function in a wrong way.

Thanks for the info: I didn't know about "rename" as a command by itself. I took it for a very ugly piece of "real" perl. Thanks.

ghostdog74 · 02-05-2008, 07:32 PM

Quote:

Originally Posted by TheFutonEng

Hello, I am trying to write a very basic shell script to replace white space in file and directory names with underscores. I realize that there are probably several better ways to do this. Here is what I have come up with so far with two small scripts:

wsfind

Code:

#!/bin/bash

directory=$1
find $directory -name '* *' -exec wsrem \"{}\" \;

wsrem

Code:

#!/bin/bash

input=$1
echo $input
files=$(echo $input | tr ' ' _ | tr \" ' ') #replace white space with _ and remove quotes.
echo $files
mv -v $input $files

I have created some test files/directories and here is the output I get:

Code:

root@underwearless:~# wsrem test/
"test/this is a test"
test/this_is_a_test
mv: target `test/this_is_a_test' is not a directory
"test/this is a test/this is a test"
test/this_is_a_test/this_is_a_test
mv: target `test/this_is_a_test/this_is_a_test' is not a directory
"test/fjkd ls"
test/fjkd_ls
mv: target `test/fjkd_ls' is not a directory

Bash seems to be creating the string I want but they aren't be interpreted by mv correctly. Any thought on how to fix what I have or perhaps a better idea on how to implement this?

1) make sure you have created the "test" directory
2) are you running your script in "test" directory or somewhere else.
3) you can make use of the -p switch of mv command.
4) as tinkster has mentioned, use double quotes

TheFutonEng · 02-08-2008, 05:55 PM

Using 'rename' worked out very well. Thanks for all of your help.

rmcban · 03-27-2011, 01:06 PM

Quote:

Originally Posted by David the H.

rename 's/ /_/g' file name

This worked perfectly for me.
Thx to the original poster.

FrDarryl · 04-30-2011, 11:01 AM

The detox command is pretty useful, although not nearly as much fun as writing awk, sed and Perl scripts.

edrubins · 02-21-2012, 08:05 PM

Quote:

Originally Posted by FrDarryl

The detox command is pretty useful, although not nearly as much fun as writing awk, sed and Perl scripts.

Thanks for pointing me to the detox command. It sure saved a lot of work!

Best, Ed

FrDarryl · 02-22-2012, 06:10 AM

Quote:

Originally Posted by edrubins

Thanks for pointing me to the detox command. It sure saved a lot of work!

Best, Ed

Happy to help.

If you get in a bind with a filename you can't address with detox or any other command (like a failed inline bash command), you can mv or rm the file using its (file-system-unique) inode.

Here's an example.

669:02/22@11.54 ~/tmp/ ls -1
file1
file2
670:02/22@11.54 ~/tmp/ touch "file3 is a failed >>! UNIX redirection"
671:02/22@11.54 ~/tmp/ ls -1
file1
file2
file3 is a failed >>! UNIX redirection
672:02/22@11.54 ~/tmp/ ls -i -1
17 file1
43 file2
137 file3 is a failed >>! UNIX redirection
673:02/22@11.55 ~/tmp/ find . -inum 137 -exec mv {} file3 \;
674:02/22@11.55 ~/tmp/ ls -i -1
17 file1
43 file2
137 file3
675:02/22@11.56 ~/tmp/ find . -inum 137 -exec rm {} \;
676:02/22@11.56 ~/tmp/ ls -i -1
17 file1
43 file2
677:02/22@11.57 ~/tmp/

edrubins · 02-25-2012, 06:44 PM

Quote:

Originally Posted by FrDarryl

Happy to help.

If you get in a bind with a filename you can't address with detox or any other command (like a failed inline bash command), you can mv or rm the file using its (file-system-unique) inode.

Here's an example.

669:02/22@11.54 ~/tmp/ ls -1
file1
file2
670:02/22@11.54 ~/tmp/ touch "file3 is a failed >>! UNIX redirection"
671:02/22@11.54 ~/tmp/ ls -1
file1
file2
file3 is a failed >>! UNIX redirection
672:02/22@11.54 ~/tmp/ ls -i -1
17 file1
43 file2
137 file3 is a failed >>! UNIX redirection
673:02/22@11.55 ~/tmp/ find . -inum 137 -exec mv {} file3 \;
674:02/22@11.55 ~/tmp/ ls -i -1
17 file1
43 file2
137 file3
675:02/22@11.56 ~/tmp/ find . -inum 137 -exec rm {} \;
676:02/22@11.56 ~/tmp/ ls -i -1
17 file1
43 file2
677:02/22@11.57 ~/tmp/

I hadn't thought about using iodes - in fact I haven't haven't thought about inodes in years.

H_TeXMeX_H · 02-26-2012, 03:47 AM

Here's my script:

Code:

for i in *
do
  mv "$i" $(echo "$i" | tr [:upper:] [:lower:] | tr [:blank:] _ | sed 's|[^-._a-z0-9]||g')
done

This also makes everything lower case, but it can be modified not to do that:

Code:

for i in *
do
  mv "$i" $(echo "$i" | tr [:blank:] _ | sed 's|[^-._a-zA-Z0-9]||g')
done

David the H. · 02-26-2012, 12:35 PM

I'd recommend using a series of parameter substitutions, rather than a single ugly chain of tr/sed commands. You can simply add, modify, or remove substitutions as necessary to get the name pattern you want.

Code:

shopt -s extglob			#needed for the extended glob below

for i in *; do

	new=${i,,}			#lowercasing requires bash v.4+
	new=${new//+([[:blank:]])/_}	#see note below
	new=${new//[^-._a-z0-9]}

	mv "$i" "$new"

done

Note: The +() exended globbing pattern can be used to compress all sequential blank space into a single underscore. Remove it to have each character substituted individually. You might also consider using [:space:] instead, to catch potential newline characters as well.