LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-30-2006, 06:48 PM   #1
JimBass
Senior Member
 
Registered: Oct 2003
Location: New York City
Distribution: Debian Sid 2.6.32
Posts: 2,100

Rep: Reputation: 49
problems with mv in a bash script


As you'll quickly see, I have absolutely no skills with scripts!

I have combined all the mp3 files in my house into a directory in my home, so all the files reside in subdirectories of /home/jim/music/. Since they came from different computeres and OSes, I have some files/directories with and without capital letters and spaces or underscores in the names. Also, as some of the files were copied from certain garbage OSes, they didn't come with nice and neat file permissions, but rather the unholiest of unholy 777 permissions.

I tried coming up with a script that would turn all spaces to underscores, and make everything (directories and individual files) be all lowercase letters. I failed, pretty badly too. I scrounged up some script pieces of code from linuxquestions and other sites from google, but most wanted to be run in the individual directories. There are 90+ Gb of music in here, so there is no way I'm running a script hundrededs/thousands of times! I'm hoping the script can be run from the /home/jim/music directory (anywhere in home is fine though), and that it will recurse through all the subdirectories below /home/jim/music/ and modifies the names.

Also, to throw a small monkeywrench in things, some of the artist, album, or song names include special characters, like ?, ', and of of the album names is even tilda ~!

Here is what I have tried so far, and showed me this was well beyond my non-existant scripting ability.

Code:
#!/bin/bash

find . -name *mp3 -print > thelist
find . -name *jpg -print >> thelist
find . -name *txt -print >> thelist

for j in "$(cat thelist)"
do

       mv "$j" $(echo "$j" | tr " " "_"); done;
# tried this once the line above failed:       new=`echo "$j" | sed 's/ /_/g'`
       mv "$j" "$new"
       chmod 644 "$new"

done
Thanks for any help!

Peace,
JimBass
 
Old 09-30-2006, 07:16 PM   #2
makyo
Member
 
Registered: Aug 2006
Location: Saint Paul, MN, USA
Distribution: {Free,Open}BSD, CentOS, Debian, Fedora, Solaris, SuSE
Posts: 730

Rep: Reputation: 75
Hi, JimBass.

I think you are on the right track with tr, not for the spaces, but to rename files to lowercase.

To help with the odd-ball names that might include ?, *, ~, use set -o noglob. Example is below ... cheers, makyo
Code:
#!/bin/sh

# @(#) s1       Demonstrate inhibiting of ?, *, [], etc., in pathnames.

# Can use "set -o noglob", or "set -f", see man bash for details.

touch 't1?'
touch 't1*'
touch 't1~'

touch t1a t11 t1lemon
touch u1a u11 u1orange

echo
echo " Files with ls:"
ls

echo
echo  " Files with ls t*":
ls t*

echo
echo " Setting noglob."
set -o noglob

echo  " Files with ls t* again":
ls t*
Which produces:
Code:
% ./s1

 Files with ls:
s1  t1*  t11  t1?  t1a  t1lemon  t1~  u11  u1a  u1orange

 Files with ls t*:
t1*  t11  t1?  t1a  t1lemon  t1~

 Setting noglob.
 Files with ls t* again:
ls: t*: No such file or directory
 
Old 09-30-2006, 07:49 PM   #3
Caesar Tjalbo
Member
 
Registered: Aug 2006
Location: Ņuņoa
Distribution: KaOS
Posts: 93

Rep: Reputation: 16
JimBass: a n00b here: is that file 'thelist' being populated right? For me it works something like
Code:
find ~/ -name *.mp3 -print > thelist
to get all the mp3 filenames + paths in that file.

Something like this looks sensible to me
Code:
find ~/ -name *.mp3 -print  > thelist
for j in "$(cat thelist)"
do
 k=`echo "$j" | tr [:blank:] "_"`
 mv -v `"$j"` `"$k"`
done
edit: except it doesn't work, of course.

edit again: thanks everyone for contributing to this thread! Very instructive for me.

Last edited by Caesar Tjalbo; 10-02-2006 at 04:33 AM.
 
Old 09-30-2006, 07:59 PM   #4
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
You can use IFS=$'\n'.

Code:
#!/bin/bash

find . -name *mp3 -print > thelist
find . -name *jpg -print >> thelist
find . -name *txt -print >> thelist

IFS=$'\n'
for j in $(<thelist); do
	# choose only one from below
	new=$(echo "$j" | tr " " "_")
	new=$(echo "$j" | sed 's/ /_/g')
	echo "$j -> $new"
	# comment the next line if everything's already correct
        read
	mv "$j" "$new"
	chmod 644 "$new"
done

Last edited by konsolebox; 09-30-2006 at 08:00 PM.
 
Old 09-30-2006, 08:38 PM   #5
JimBass
Senior Member
 
Registered: Oct 2003
Location: New York City
Distribution: Debian Sid 2.6.32
Posts: 2,100

Original Poster
Rep: Reputation: 49
First off, thanks for the help guys!

Makyo, that looks like the set -noglob should do the trick for the "strange" chars. I'll give that a shot as I get further on.

Caesar Tjalbo, yes, thelist is being populated as I hoped it would, it looks like this when run dfrom a subdirectory I have been using for testing:

Code:
jim@JimsBeastie:~/music/owned/Zeckendorff$ pwd
/home/jim/music/owned/Zeckendorff
jim@JimsBeastie:~/music/owned/Zeckendorff$ ./checking
mv: target `./reduction/liner.jpg' is not a directory
jim@JimsBeastie:~/music/owned/Zeckendorff$ vim thelist

(leads to ->)  

./reduction/08-The Line Tune.mp3
./reduction/02-Scan.mp3
./reduction/05-Reduction.mp3
./reduction/03-The New Modernists.mp3
./reduction/01-Double U.mp3
./reduction/04-Engrave.mp3
./reduction/07-Anathema.mp3
./reduction/06-Manifold.mp3
./reduction/liner.jpg
Konsolebox, you gave me 99% of it! Thanks so much! The only thing that doesn't happen is changing the capital letters to lowercase. Here is the script as I use it, and the results:

Code:
#!/bin/bash

find . -name *mp3 -print > thelist
find . -name *jpg -print >> thelist
find . -name *txt -print >> thelist

IFS=$'\n'
for j in $(<thelist);
do

        new=$(echo "$j" | sed 's/ /_/g')
        echo "$j -> $new"
        read
        mv "$j" "$new"
        chmod 644 "$new"

done

and the output of files after allowing the script to run:

jim@JimsBeastie:~/music/owned/Zeckendorff$ ls -al reduction/
total 48718
drwxr-xr-x 2 jim jim      360 2006-09-30 20:32 .
drwxr-xr-x 3 jim jim      280 2006-09-30 20:35 ..
-rw-r--r-- 1 jim jim  7394514 2005-11-21 22:41 01-Double_U.mp3
-rw-r--r-- 1 jim jim  4802688 2004-03-11 20:49 02-Scan.mp3
-rw-r--r-- 1 jim jim   546944 2004-03-11 20:49 03-The_New_Modernists.mp3
-rw-r--r-- 1 jim jim  7682176 2004-03-11 20:50 04-Engrave.mp3
-rw-r--r-- 1 jim jim  8931456 2004-03-11 20:50 05-Reduction.mp3
-rw-r--r-- 1 jim jim  2435200 2004-03-11 20:51 06-Manifold.mp3
-rw-r--r-- 1 jim jim  3938432 2004-03-11 20:51 07-Anathema.mp3
-rw-r--r-- 1 jim jim 13729920 2004-03-11 20:52 08-The_Line_Tune.mp3
-rw-r--r-- 1 jim jim   359098 2004-03-14 15:47 liner.jpg
So again, thanks to all!

Peace,
JimBass
 
Old 10-01-2006, 03:34 PM   #6
JimBass
Senior Member
 
Registered: Oct 2003
Location: New York City
Distribution: Debian Sid 2.6.32
Posts: 2,100

Original Poster
Rep: Reputation: 49
I'm still having some trouble with the mv part of the script. I found a good perl script at http://blemished.net/chcase.html for changing capital letters to lowercase, and it works well.

I tried doing the next directory as a final test before I used the scripts on the entire music directory, and hit a snag that I can't understand. I didn't see this yesterday when I tried konsolebox's script. I used the cscase perl script to get everything lowercase, and that worked fine. Then when I ran konsolebox's script, I get this output:

Code:
jim@JimsBeastie:~/music/owned/Young, Neil$ ./konsolebox
./silver and gold/05-the great divide.mp3 -> ./silver_and_gold/05-the_great_divide.mp3

mv: cannot move `./silver and gold/05-the great divide.mp3' to `./silver_and_gold/05-the_great_divide.mp3': No such file or directory
chmod: cannot access `./silver_and_gold/05-the_great_divide.mp3': No such file or directory
./silver and gold/07-red sun.mp3 -> ./silver_and_gold/07-red_sun.mp3

mv: cannot move `./silver and gold/07-red sun.mp3' to `./silver_and_gold/07-red_sun.mp3': No such file or directory
chmod: cannot access `./silver_and_gold/07-red_sun.mp3': No such file or directory
./silver and gold/10-without rings?.mp3 -> ./silver_and_gold/10-without_rings?.mp3

mv: cannot move `./silver and gold/10-without rings?.mp3' to `./silver_and_gold/10-without_rings?.mp3': No such file or directory
chmod: cannot access `./silver_and_gold/10-without_rings?.mp3': No such file or directory
./silver and gold/02-silver  gold.mp3 -> ./silver_and_gold/02-silver__gold.mp3

mv: cannot move `./silver and gold/02-silver  gold.mp3' to `./silver_and_gold/02-silver__gold.mp3': No such file or directory
chmod: cannot access `./silver_and_gold/02-silver__gold.mp3': No such file or directory
./silver and gold/08-distant camera.mp3 -> ./silver_and_gold/08-distant_camera.mp3

mv: cannot move `./silver and gold/08-distant camera.mp3' to `./silver_and_gold/08-distant_camera.mp3': No such file or directory
chmod: cannot access `./silver_and_gold/08-distant_camera.mp3': No such file or directory
./silver and gold/09-razor love.mp3 -> ./silver_and_gold/09-razor_love.mp3

mv: cannot move `./silver and gold/09-razor love.mp3' to `./silver_and_gold/09-razor_love.mp3': No such file or directory
chmod: cannot access `./silver_and_gold/09-razor_love.mp3': No such file or directory
./silver and gold/01-good to see you.mp3 -> ./silver_and_gold/01-good_to_see_you.mp3

mv: cannot move `./silver and gold/01-good to see you.mp3' to `./silver_and_gold/01-good_to_see_you.mp3': No such file or directory
chmod: cannot access `./silver_and_gold/01-good_to_see_you.mp3': No such file or directory
./silver and gold/06-horseshoe man.mp3 -> ./silver_and_gold/06-horseshoe_man.mp3

mv: cannot move `./silver and gold/06-horseshoe man.mp3' to `./silver_and_gold/06-horseshoe_man.mp3': No such file or directory
chmod: cannot access `./silver_and_gold/06-horseshoe_man.mp3': No such file or directory
./silver and gold/04-buffalo springfield again.mp3 -> ./silver_and_gold/04-buffalo_springfield_again.mp3

mv: cannot move `./silver and gold/04-buffalo springfield again.mp3' to `./silver_and_gold/04-buffalo_springfield_again.mp3': No such file or directory
chmod: cannot access `./silver_and_gold/04-buffalo_springfield_again.mp3': No such file or directory
./silver and gold/03-daddy went walkin.mp3 -> ./silver_and_gold/03-daddy_went_walkin.mp3

mv: cannot move `./silver and gold/03-daddy went walkin.mp3' to `./silver_and_gold/03-daddy_went_walkin.mp3': No such file or directory
chmod: cannot access `./silver_and_gold/03-daddy_went_walkin.mp3': No such file or directory
./silver and gold/liner.jpg -> ./silver_and_gold/liner.jpg

mv: cannot move `./silver and gold/liner.jpg' to `./silver_and_gold/liner.jpg': No such file or directory
chmod: cannot access `./silver_and_gold/liner.jpg': No such file or directory
I tried both adding in the set -f line suggested by makyo. It should allow the funny chars, but with and without that line, I get the same output for this particular directory.

Peace,
JimBass
 
Old 10-01-2006, 04:36 PM   #7
makyo
Member
 
Registered: Aug 2006
Location: Saint Paul, MN, USA
Distribution: {Free,Open}BSD, CentOS, Debian, Fedora, Solaris, SuSE
Posts: 730

Rep: Reputation: 75
Hi.

Did you create the directory silver_and_gold?

Note also - you have more than one consecutive space in at least one item. If I were doing that, I think I'd collapse all spaces to a single underscore for consistency with one another (but obviously not with the original) ... cheers, makyo

( edit 1: typo )

Last edited by makyo; 10-01-2006 at 09:10 PM.
 
Old 10-01-2006, 04:37 PM   #8
spirit receiver
Member
 
Registered: May 2006
Location: Frankfurt, Germany
Distribution: SUSE 10.2
Posts: 424

Rep: Reputation: 33
You cannot move files into a non-existent directory, and I guess ./silver_and_gold doesn't exist. You can create the directory as follows:
Code:
ada@barnabas:~/tmp> echo "$name"
./Directory/File Name
ada@barnabas:~/tmp> echo "$(dirname "$name")"
./Directory
ada@barnabas:~/tmp> test -d "$(dirname "$name")" || mkdir "$(dirname "$name")"
The last line creates the directory if and only if it doesn't exist. You can use the following technique to modify the filename without doing any harm to the directory.
Code:
ada@barnabas:~/tmp> echo "$(dirname "$name")/$(basename "$name" | tr "A-Z " "a-z_")"
./Directory/file_name
 
Old 10-01-2006, 07:53 PM   #9
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
OK we need to make some mods to our script. Again please test this first.

Code:
#!/bin/bash

{
	find . -name *mp3 -print
	find . -name *jpg -print
	find . -name *txt -print
} | sort > thelist

cat /dev/null > trashdirs

is_empty() {
	for a in "$1"/*; do
		return 1
	done
	return 0
}

IFS=$'\n'
for OLD in $(<thelist); do
	NEW="$(echo "$OLD" | tr " " "_" | tr "A-Z" "a-z")"
	echo "$OLD -> $NEW"
	# comment the next line if everything's already correct
	read
	OLDDIR=$(dirnmae $OLD)
	NEWDIR=$(dirname $NEW)
	if [ "$NEWDIR" != "$OLDDIR" ]; then
		if [ -d "$NEWDIR" ]; then
			:
		elif [ -e "$NEWDIR" ]; then
			echo "cannot create directory $NEWDIR - a different file already exists."
			echo "breaking now.  please check the file after the deletion of empty old directories."
			break
		else
			# install -m 644 -d "$NEWDIR"
			mkdir -p "$NEWDIR"
			echo "$OLDDIR" >> trashdirs
		fi
	fi
	mv "$OLD" "$NEW"
	chmod 644 "$NEW"
done
echo

echo "removing empty old directories"
cat trashdirs | sort -r > tmp
mv tmp trashdirs
for TRASHDIR in $(<trashdirs); do
	if is_empty $TRASHDIR; then
		echo "$TRASHDIR"
		# comment the next line if everything's already correct
		read
		rm -fr $TRASHDIR
	fi
done
echo
 
Old 10-01-2006, 08:46 PM   #10
JimBass
Senior Member
 
Registered: Oct 2003
Location: New York City
Distribution: Debian Sid 2.6.32
Posts: 2,100

Original Poster
Rep: Reputation: 49
I see, and that means the reason it worked with the initial directory was because the artist/album_name/ format I used for storage didn't present any problems when the first directory tested in was all one word titles, /home/jim/music/Zeckendorff/Reduction.

The second test being /home/jim/music/owned/Young, Neil/Silver & Gold/ caused issues.

I see konsolebox posted while I was eating dinner and not posting this thread! I'll test out the suggestions now.

Thanks again everyone!

Peace,
JimBass
 
Old 10-01-2006, 09:21 PM   #11
JimBass
Senior Member
 
Registered: Oct 2003
Location: New York City
Distribution: Debian Sid 2.6.32
Posts: 2,100

Original Poster
Rep: Reputation: 49
That is close to being it as well! For some reason, the cleanup fails konsolebox. I just ran it in my Led Zeppelin directory, and this is what I ended up with:

Before:
Code:
jim@JimsBeastie:~/music/owned/Led Zeppelin$ ls
box  chcase  Led Zeppelin I  Led Zeppelin II  Led Zeppelin IV
jim@JimsBeastie:~/music/owned/Led Zeppelin$ ls Led\ Zeppelin\ I
01-Good Times Bad Times.mp3     06-Black Mountain Side.mp3
02-Babe Im Gonna Leave You.mp3  07-Communication Breakdown.mp3
03-You Shook Me.mp3             08-I Cant Quit You Baby.mp3
04-Dazed And Confused.mp3       09-How Many More Times.mp3
05-Your Time is Gonna Come.mp3  liner.jpg
After:
Code:
jim@JimsBeastie:~/music/owned/Led Zeppelin$ ls
box     led zeppelin i  led zeppelin ii  led zeppelin iv  thelist
chcase  led_zeppelin_i  led_zeppelin_ii  led_zeppelin_iv  trashdirs

jim@JimsBeastie:~/music/owned/Led Zeppelin$ cat trashdirs
./led zeppelin iv
./led zeppelin ii
./led zeppelin i
I'm off to hear some live music, so I'll continue with the cleanup activities tomorrow.

What happened to affero? I was all set to throw some money to the charity of Konsolebox's choice, when I realized the affero button is gone. Strange.

Peace,
JimBass
 
Old 10-02-2006, 04:33 AM   #12
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
sorry about the error. please try this again. i already tested it myself and it worked. i hope it works for you too.

Code:
#!/bin/bash

{
	find . -name *mp3 -print
	find . -name *jpg -print
	find . -name *txt -print
} | sort > list.tmp

cat /dev/null > trashdirs.tmp

is_empty() {
	if ls $1/* >/dev/null 2>&1; then
		return 1
	else
		return 0
	fi
}

IFS=$'\n'
for OLD in $(<list.tmp); do
	NEW="$(echo "$OLD" | tr " " "_" | tr "A-Z" "a-z")"
	DOTRASHOLDDIR=0
	if [ "$OLD" != "$NEW" ]; then
		echo "$OLD -> $NEW"
		# comment the next line if everything's already correct
		read
		OLDDIR=$(dirname $OLD)
		NEWDIR=$(dirname $NEW)
		if [ "$NEWDIR" != "$OLDDIR" ]; then
			if [ -d "$NEWDIR" ]; then
				DOTRASHOLDDIR=1
			elif [ -e "$NEWDIR" ]; then
				echo "cannot create directory $NEWDIR - a different file already exists."
				echo "breaking now.  please check the file after the deletion of empty old directories."
				break
			else
				# install -m 644 -d "$NEWDIR"
				mkdir -p "$NEWDIR"
				DOTRASHOLDDIR=1
			fi
		fi
		mv "$OLD" "$NEW"
		chmod 644 "$NEW"
		if [ $DOTRASHOLDDIR -eq 1 ]; then
			echo "$OLDDIR" >> trashdirs.tmp
		fi
	fi
done

cat trashdirs.tmp | sort -r > tmp
mv tmp trashdirs.tmp
for TRASHDIR in $(<trashdirs.tmp); do
	if [ -e $TRASHDIR ] && is_empty $TRASHDIR; then
		echo "removing $TRASHDIR"
		# comment the next line if everything's already correct
		read
		rm -fr $TRASHDIR
	fi
done

rm -f list.tmp
rm -f trashdirs.tmp
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Problems with a script (bash shell, Red Hat 7.2) illiniguy3043 Linux - Newbie 7 06-01-2004 02:49 AM
Bash script problems with "if" statement adz Programming 2 05-22-2004 04:34 AM
send automatic input to a script called by another script in bash programming jorgecab Programming 2 04-01-2004 01:20 AM
problems with bash script read wedgeworth Programming 3 02-16-2004 09:29 AM
Problems with bash script cowardnewbie Programming 0 09-30-2001 01:59 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:58 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration