Bash pipe question

Dalekk · 10-28-2009, 04:48 AM

Hi there
I have the script

Code:

find -iname "*.mp3" | while read n; do
	echo $n | sed 's/.wav././' | sed 's/[0-9]*\ //' | sed 's/-//' |
        sed 's/\ -\ /*/' | sed 's/.mp3//' | sed 's/.\///' | while read k; do
    	      find -iname "*$k*";
    done;
done;

My question is if I can avoid having the second loop? I would like to collect the output of the last sed command like j=`sed 's/.\///'`. Syntax doesn't allow this. If I put `` around the whole sequence of sed commands, I get something that looks like the accumulated output of all the seds.

Thank you

ghostdog74 · 10-28-2009, 05:17 AM

describe what you are doing properly. Provide examples of files you are searching for, what you want to do with them, and your final output. the above definitely can be coded more efficiently.

Dalekk · 10-28-2009, 05:33 AM

Well, I am in a situation where I have to rename a lot of files, and naturally I don't want to do this manually.

I have two groups of files. The first group is about 200 .mp3 files and the second group is about 100.000 files with lyrics from all sorts of music, including lyrics for the 200 .mp3 files. I want to make a third group containing the 200 .mp3 files and the 200 corresponding lyrics-files. The problem is that the naming of the .mp3 files follow different standards, with some being TRACK NUMBER - ARTIST - ALBUM - TITLE, some being TRACK NUMBER - TITLE and other combinations as well. The lyrics files are all ARTIST - ALBUM - TITLE.

Example:
I have the .mp3 file "01 - Dungen - Panda.mp3" and want to find the lyrics file
Dungen-Ta Det Lungt-2004-Panda.txt after which I want to rename the mp3 to Dungen-Ta Det Lungt-2004-Panda.mp3.

I have done this by first using sed to reform the .mp3 name to a string that can be used with find, and then use the string to search the lyrics directory.

My full code is:

Code:

#!/bin/bash
find -iname "*.mp3" | while read n; do
    echo $n | sed 's/.wav././' | sed 's/[0-9]*\ //' | sed 's/-//' | sed 's/\ -\ /*/' | sed 's/.mp3//' | sed 's/.\///' | while read k; do
        find ../emotions/ -iname "*$k*" | sed 's/..\/emotions\///' | sed 's/.txt/.mp3/' | while read j; do
            m=`echo $n | sed 's/.\///'`
            echo "Renaming" $m "to" $j
            cp "$m" "$j"
        done;
    done;
done;

Instead of using "while read" to snatch the output of the multi-piped sed command (I don't know what to call it), is it possible to assign the output of the last sed command to a variable?

Code:

j=`echo $n | sed 's/.wav././' | sed 's/[0-9]*\ //' | sed 's/-//' | sed 's/\ -\ /*/' | sed 's/.mp3//' | sed 's/.\///'`

does not work..

I would also like the user to be able to confirm the renaming of each individual file (e.g. y/n keyboard input). I tried to use read for this, but for some reason nothing happens.

bsat · 10-28-2009, 11:33 PM

I am not sure if this is what you want, but to assign the results of commands to variable you can use the following syntax

var=$(command)

i.e. in your case

j=$(`echo $n | sed 's/.wav././' | sed 's/[0-9]*\ //' | sed 's/-//' | sed 's/\ -\ /*/' | sed 's/.mp3//' | sed 's/.\///'`)

jschiwal · 10-29-2009, 12:57 AM

The main change I would recommend is to first prepare a file containing a list of the lyrics files. Running a directory list or find command on such a large number of files will take time. You don't want that inside a loop. Preparing a directory listing to work with before your loop will save a lot of time.

One thing you can do is use a single sed command instead of all of the pipe commands.
sed 'command1;command2;command3;command4'
or
sed -e 'command1' -e 'command2' -e 'command3' -e 'command4'

You can also create a file containing your sed commands and call the sed command like: "sed -f replacemp3s.sed". That would allow you to debug the sed commands as a separate entity.

If you do use sed to process the list of mp3 music files, you could use the memory registers to enable print the parts that match in the repacement string. This is commonly used to cut out parts of a line which you want to do.

Maybe this example will give you some ideas:

Code:

echo '01 - Dungen - Panda.mp3' | 
sed -n 's/^[[:digit:]][[:digit:]]* - \(.*\) - \(.*\).mp3/\1\t\2/p' | 
while read name artist; do 
   echo name=$name,artist=$artist; 
done
name=Dungen artist=Panda

or this one:

Code:

eval $(echo '01 - Dungen - Panda.mp3' | 
       sed -n 's/^[[:digit:]][[:digit:]]* - \(.*\) - \(.*\).mp3/name="\1" artist="\2\"/p')
~> echo $name
Dungen
~> echo $artist
Panda

For other ideas, look at the `-f' option for grep. You could construct regular expression strings from the list of mp3 files and use that in a search of a list of the lyric files. Or you could use variables in a grep or sed command to locate the matching lyrics file from the list.

The '-n' option I used in the sed example suppresses the output. The later `p' command after the replacement prints the line only if there was a match. This can eliminate problems if a filename doesn't match the pattern. I would recommend that you move the file to another directory after renaming it, since the pattern before and after are similar. Also using a text file of the mp3 directory for debugging until you get the script correct. You could print out the mv command line to create a script of mv commands. This would allow you to visually inspect it for problems before actually executing it.

At work I need to perform something similar, locating files with leading zero's and removing them from the filenames. There are thousands of files. I can access the directory from my workstation, but don't have permission to rename them. So I construct a windows batch file (using sed in a cygwin script), redirect the output to a batch file on the server, and then log in to the server and run the script.

Good Luck!

Dalekk · 11-01-2009, 02:17 AM

Thanks for the replies, I learned a lot from that

.