Another coding enigma? ;-) (for me)

mainstream · 03-28-2011, 02:27 PM

Ok here i am again with some question considering a script for managing my music collection.

Is it possible to see if
if [ -e "$file" ] but instead of checking file exist to check whether this file starts with [0-9]-* (i.e 00-wiz_khalifa-black_&_yellow-(remix)-(82_bpm).mp3) then...

And how do i remove this first part?
I now use

Code:

numbers=*-*
$(echo ${mp3#$numbers}

this has worked so far but this is not ideal...

The input may differ from 00- to 201_ so this is quite hard for me ;-)
(my current script sometimes screws this up -> 00_wiz_khalifa-black_&_yellow-(remix)-(82_bpm).mp3 becomes black_&_yellow-(remix)-(82_bpm).mp3 (which is not right)
But works with 00-

Thanks in regards !

Mainstream

toordog · 03-28-2011, 03:39 PM

Quote:

Originally Posted by mainstream

Ok here i am again with some question considering a script for managing my music collection.

Is it possible to see if
if [ -e "$file" ] but instead of checking file exist to check whether this file starts with [0-9]-* (i.e 00-wiz_khalifa-black_&_yellow-(remix)-(82_bpm).mp3) then...

And how do i remove this first part?
I now use

Code:

numbers=*-*
$(echo ${mp3#$numbers}

this has worked so far but this is not ideal...

The input may differ from 00- to 201_ so this is quite hard for me ;-)
(my current script sometimes screws this up -> 00_wiz_khalifa-black_&_yellow-(remix)-(82_bpm).mp3 becomes black_&_yellow-(remix)-(82_bpm).mp3 (which is not right)
But works with 00-

Thanks in regards !

Mainstream

You can try a loop and a regex like:

for i in `ls /somefolder/ | grep -E [0-9][0-9]-*`
do
mv $i $(cat $i | sed -e 's/[0-9][0-9]-//')
done

grail · 03-28-2011, 07:51 PM

Well one would say you still need to test that the file exists prior to testing it starts with numbers.
Assuming you are using a recent version of bash you could do:

Code:

if [[ -e "$file" && $file =~ ^[0-9] ]]

toordog · 03-28-2011, 11:30 PM

Quote:

Originally Posted by grail

Well one would say you still need to test that the file exists prior to testing it starts with numbers.
Assuming you are using a recent version of bash you could do:

Code:

if [[ -e "$file" && $file =~ ^[0-9] ]]

With its current if logic I agree he would need that and it's a neet if condition.

In a loop you would not need to test it, if it doesn't exist it will not show up. Like i'm doing a `ls`, if the file doesn't exist, it will not display. You save some pico seconds on the CPU and a few line of code. :P

You might want to test if you have write permission on the file but the worse is that the mv will return an error anyway and unless you need to automate a response to that error, it would be useless.

Is there a particular reason while you wanted to use a if instead of a loop? Most beginner in programmtion tend to use a if to test a condition which is not exactly a IF condition but more likely a loop condition. It is a bad logic to do a list of item and to test them individually for a condition. It is better to list only the object you want and to cycle an action on each of them. (My way to say my mind is not very clear because it's late. But if someone find a better way to explain it, you are more than welcome).

Regards

mainstream · 03-29-2011, 03:51 AM

Thanks everybody for the comments :-)

Quote:

##!/bin/sh
for file in *-*$*bpm$.mp3; do
if [ -e "$file" ]
then
echo \.---------------------------------------------------------------------------------\.
echo \| $file has BPM tags \-\> removing \|
echo \'---------------------------------------------------------------------------------\'
mv -i $file $(echo ${file/-([0-9]*_bpm)/} | sed 's/ /_/g' | sed 's/-(/_(/g' | sed 's/_-_/-/g' | tr '[A-Z]' '[a-z]')
else echo "No BPM tags found -> skipping"
fi
done

for file in `ls | grep -E [0-9]-* | egrep '\.mp3$'`; do
if [[ -e "$file" && $file =~ ^[0-9] ]]
then
echo \.---------------------------------------------------------------------------------\.
echo \| $file has numeric tags \-\> removing
echo \'---------------------------------------------------------------------------------\'
mv -i $file $(echo $file | sed -e 's/[0-9][0-9]-//')
else echo "No numeric tags found -> skipping"
fi
done

i know have this:
-except this doesn't work on 00_wiz_khalifa-sakdlaskd.mp3
if i change [0-9]_* it will detect
00_wiz_khalifa-sakdlaskd.mp3
10-wiz_khalifa-sakdlaskd.mp3
?

grail · 03-29-2011, 06:27 AM

@toordog - the question was how to test if a file exists and starts with a number and as the original question was an if, this is what I supplied.
Should you wish to get technical, I am happy to oblige with the issues with your code.

1. Any files that have spaces in them will fail due to word splitting in your for loop.
2. As your ls, generally a bad choice as outlined here, does not segregate files and directories,
any directory that starts with a digit will also be moved
3. The OP specifically asked what if there is an underscore instead of a dash and later pointed out there could be more than 2 digits at the front which your grep and sed will fail.
4. Your sed only allows for 2 digits followed by a dash so again the underscore will not be evaluated

@OP - The reason 00_wiz_khalifa-sakdlaskd.mp3 is not working is due to the following:

Code:

#first loop
for file in *-*\(*bpm\).mp3; do # This says the file MUST contain a dash

# second loop
mv -i $file $(echo $file | sed -e 's/[0-9][0-9]-//') # sed only considers files with a dash in them

See how you go solving with this information. If you get stuck just holler back

mainstream · 03-29-2011, 09:54 AM

Hmmz thanks for your reply; forgot about that!

Most files do not contain spaces as i replace them (sed 's/ /_/g') on the first run.

It'll properly be solved when the first underscore in [0-9]_ will be replaced with sed i guess?
Or create another loop (but that's to much?)

toordog · 03-29-2011, 10:45 AM

Quote:

Originally Posted by grail

@toordog - the question was how to test if a file exists and starts with a number and as the original question was an if, this is what I supplied.
Should you wish to get technical, I am happy to oblige with the issues with your code.

1. Any files that have spaces in them will fail due to word splitting in your for loop.
2. As your ls, generally a bad choice as outlined here, does not segregate files and directories,
any directory that starts with a digit will also be moved
3. The OP specifically asked what if there is an underscore instead of a dash and later pointed out there could be more than 2 digits at the front which your grep and sed will fail.
4. Your sed only allows for 2 digits followed by a dash so again the underscore will not be evaluated

@OP - The reason 00_wiz_khalifa-sakdlaskd.mp3 is not working is due to the following:

Code:

#first loop
for file in *-*\(*bpm\).mp3; do # This says the file MUST contain a dash

# second loop
mv -i $file $(echo $file | sed -e 's/[0-9][0-9]-//') # sed only considers files with a dash in them

See how you go solving with this information. If you get stuck just holler back

GRAIL thank you very much for pointing this out. It really gaves me some lead to read about the ls issue. I've been stuck in another context with the ls output to work under Linux and AIX and didn't find a solution at that time. I just found it with your reference. I should probably read much more about "find" as well.

I wish I had more feedback like this

mainstream · 03-29-2011, 11:01 AM

Ok i kinda changed the whole thing, but not fully working a.t.m.
So it's better to

Quote:

for f in *; do
...
done

then use ls?

Directory:

Quote:

~/Music $ ls *.mp3
nitrogenetics-hit_me-zzzz.mp3
wiz_khalifa-black_&_yellow-(remix)-(82_bpm)-zzzz.mp3
wiz_khalifa_ft._ne_yo-black_&_yellow_(rmx)-zzzz.mp3

Output:

Code:

~/Music $ ./bpm
mv: cannot stat `*-*(*bpm).mp3': No such file or directory
*-*(*bpm).mp3
*-*(*bpm).mp3
.----------------------------- - - -- -- -- --- - - - - - - - - - -
| File: "nitrogenetics-hit_me-zzzz.mp3"
| Number of characters 29
'----------------------------- - - -- -- -- --- - - - - - - - - - -
  + Matched: "-zzzz.", Substitute "."
.----------------------------- - - -- -- -- --- - - - - - - - - - -
| File: "wiz_khalifa-black_&_yellow_(remix)_(82_bpm)-zzzz.mp3"
| Number of characters 52
'----------------------------- - - -- -- -- --- - - - - - - - - - -
mv: cannot stat `wiz_khalifa-black_&_yellow-(remix)-(82_bpm)-zzzz.mp3': No such file or directory
  + Matched: "-zzzz.", Substitute "."
.----------------------------- - - -- -- -- --- - - - - - - - - - -
| File: "wiz_khalifa_ft._ne_yo-black_&_yellow_(rmx)-zzzz.mp3"
| Number of characters 51
'----------------------------- - - -- -- -- --- - - - - - - - - - -
  + Matched: "ne_yo", Substitute "ne-yo"
  + Matched: "rmx", Substitute "remix"
  + Matched: "-zzzz.", Substitute "."

Code:

##!/bin/sh
######################################
## clean-up numeric junk from filename
######################################
for file in `ls | grep -E [0-9]-* | egrep '\.mp3$'`; do
clean=$(echo $file | sed -e 's/[0-9][0-9]-//')
if [ "$file" != "$clean" ] && [[ -e "$file" && $file =~ ^[0-9] ]]
then
mv -i "$file" "$clean"
fi
done

##################################
## clean-up bpm tags from filename (something going wrong here?)
##################################
for file in *-*\(*bpm\).mp3; do
bpm=$(echo ${file/-([0-9]*_bpm)/})
if [ ! -e $file ] && [ "$file" != "$bpm" ]
then
mv -i "$file" "$bpm"
echo $file
echo $(echo ${file/-([0-9]*_bpm)/})
fi
done

##############################
## clean-up junk from filename
##############################
for file in *-*.mp3; do
clean=$(echo $file | sed 's/[\,\'\'']//g' | sed 's/ /_/g' | sed 's/-(/_(/g' | sed 's/_-_/-/g' | tr '[A-Z]' '[a-z]' | sed 's/(dirty)/(dirty_mix)/g' | sed 's/(clean)/(clean_mix)/g' | sed 's/(radio)\|(radio_edit)/(radio_mix)/g' | sed 's/(main)/(main_version)/g' | sed 's/_feat._\|_feat_\|_f._\|_ft_/_ft._/g')
echo .----------------------------- - - -- -- -- ---  -   - -    -   -    - - -    -    -
echo \| File: \"$clean\"
echo \| "Number of characters ${#clean}"
echo \'----------------------------- - - -- -- -- ---  -   - -    -   -    - - -    -    -
if [ ! -e $file ] && [ "$file" != "$clean" ]
then
mv -i "$file" $clean
fi
	####################################
	## search & replace tags in filename 
	####################################
	while read inputline
	do
	match="$(echo $inputline | cut -d: -f1)"
	substitute="$(echo $inputline | cut -d: -f2)"
	clean=$(echo $file | sed 's/'$match'/'$substitute'/g')
	if [ "$file" != "$clean" ] ; then
	echo \ \ \+ Matched: \"$match\", Substitute \"$substitute\"
	rename "s/${match}/$substitute/g" \E[ *.mp3 #how can i use this on 1 file (loop)?
	fi
  	done < .rsub
done

grail · 03-29-2011, 11:20 AM

Quote:

Most files do not contain spaces as i replace them (sed 's/ /_/g') on the first run.

Well I do not see this as necessary, but that is of course up to you.

Quote:

It'll properly be solved when the first underscore in [0-9]_ will be replaced with sed i guess?
Or create another loop (but that's to much?)

Personally I would probably do all your removals in a single while loop fed by find (as a bonus this will not be affected by word splitting).

Also you could easily do most of what you want without the use of sed, grep, tr and so on and just use bash ... assuming your version is recent enough.

For example:

Code:

FILE="This IS a FiLe WIth MIXeD case.mp3"
echo ${FILE,,}
echo ${FILE// /_}

#also
FILE='00_wiz_khalifa-sakdlaskd.mp3'
[[ $FILE =~ ^[0-9] ]] && echo ${FILE#*[-_]}

Just to give you some ideas

Quote:

Originally Posted by toordog

I wish I had more feedback like this

No problem

Happy to help

grail · 03-29-2011, 11:24 AM

Looks like we were posting at the same time.

Yes it is better to use 'for i in *' than ls as the star will glob entire name and ls values returned to the for loop will be subject to word splitting.

I see you have given us some input:

Quote:

nitrogenetics-hit_me-zzzz.mp3
wiz_khalifa-black_&_yellow-(remix)-(82_bpm)-zzzz.mp3
wiz_khalifa_ft._ne_yo-black_&_yellow_(rmx)-zzzz.mp3

Maybe you would like to provide what the output you would like to see is for these 3 files once your script has run?
Also I noted that none of these start with digits??

mainstream · 03-30-2011, 08:38 AM

Quote:

Originally Posted by grail

Well I do not see this as necessary, but that is of course up to you.

Personally I would probably do all your removals in a single while loop fed by find (as a bonus this will not be affected by word splitting).

Also you could easily do most of what you want without the use of sed, grep, tr and so on and just use bash ... assuming your version is recent enough.

For example:

Code:

FILE="This IS a FiLe WIth MIXeD case.mp3"
echo ${FILE,,}
echo ${FILE// /_}

#also
FILE='00_wiz_khalifa-sakdlaskd.mp3'
[[ $FILE =~ ^[0-9] ]] && echo ${FILE#*[-_]}

Just to give you some ideas

No problem

Happy to help

I would never come up with something like this, since:
1. I don't quite understand that (looks quite complex to me)
2. Can hardly find anything in literature (specific)

I used sed / tr because I'm quite new to coding, so still learning ;-)
So thanks for all the feedback and pointers you're giving -> really helpful.

I wanted to try this in 1 loop, but tried it so many times i eventually gave up hope.
Then a few weeks later i heard about this forum, and i must say: I'm very happy i did

.

There's some much more to learn from other people, then just browse the Internet.

Quote:

Originally Posted by grail

Looks like we were posting at the same time.

Yes it is better to use 'for i in *' than ls as the star will glob entire name and ls values returned to the for loop will be subject to word splitting.

I see you have given us some input:

Quote:

nitrogenetics-hit_me-zzzz.mp3
wiz_khalifa-black_&_yellow-(remix)-(82_bpm)-zzzz.mp3
wiz_khalifa_ft._ne_yo-black_&_yellow_(rmx)-zzzz.mp3

Maybe you would like to provide what the output you would like to see is for these 3 files once your script has run?
Also I noted that none of these start with digits??

Input would be
10-Nitrogenetics-hit me-zzzz.mp3 -> nitrogenetics-hit_me.mp3
202_wiz_khalifa-black_&_yellow-(remix)-(82_bpm)-zzzz.mp3 -> wiz_khalifa-black_&_yellow-(remix).mp3

And all the parameters from sed/tr in the script above.
I used this script quite a while, but a couple days ago, i wanted to make a new script, which is more compact and more functional, and learn something new.

toordog · 03-30-2011, 09:28 AM

If I may suggest you a great reading:

http://www.amazon.com/New-KornShell-.../dp/0131827006
This book is the best book out there to learn Korn Shell. It's been written by the creator of the language and is quite good to start and to use as a reference later.

New KornShell Command And Programming Language, The (2nd Edition)
Morris I. Bolsky (Author), David G. Korn (Author)

http://www.amazon.com/sed-awk-2nd-Da...d_bxgy_b_img_c
sed & awk (2nd Edition) [Paperback] By O'Reilly Edition
Dale Dougherty (Author), Arnold Robbins (Author)

With that, you can do 80% of everything in a Unix. For the rest, you have Perl or C

grail · 03-30-2011, 10:11 AM

Quote:

I used sed / tr because I'm quite new to coding, so still learning ;-)

Cool ... and doing a decent job too

Quote:

I would never come up with something like this, since:
1. I don't quite understand that (looks quite complex to me)
2. Can hardly find anything in literature (specific)

1. Not really complex once you understand the mechanism
2. Obviously the above comes from doing and reading so 2 pieces of literature I will direct you to are:

http://tldp.org/LDP/abs/html/ - I found this helpful for learning the basics (don't let the word 'Advanced' scare you off)
http://mywiki.wooledge.org/TitleIndex - Read through every single link, at your leisure. this is basically a list of all the things that people over think or do outright wrong (some of which are even
from the previous link I gave)

Now I realise there might be more combinations you may need to test, but the following works just fine on the given examples:

Code:

#!/bin/bash

while read -r DIR FILE
do
    TMP="${FILE,,}"
    TMP="${TMP// /_}"
    [[ $TMP =~ ^[0-9] ]] && TMP="${TMP#*[-_]}"
    [[ $TMP =~ bpm ]] && TMP="${TMP%-(*bpm*}.mp3"
    echo mv "$DIR/$FILE" "$DIR/$TMP"
done< <(find . -type f -name '*.mp3' -printf "%h %f\n")

Starting with the find which feeds the loop and then going from top to bottom:

1. Find all files in the current directory and below that are files (-type f), end in '.mp3' and once found return the path and the file name as separated items
2. Read path into DIR and filename into FILE
3. ,, - return the entire string, to variable TMP, represented by FILE in lowercase (Note: a single comma will only change the first letter)
4. // /_ - find all spaces in string and replace with an underscore (Note: Single / /_ will only replace the first space found)
5. If string starts with a digit, ^[0-9], then starting from the front of the string, #, remove the shortest match until first underscore or dash, [-_], is found (Note: ## would remove the longest match)
6. If string contains substring, bpm, then starting from the back, %, remove the shortest match to, -(*bpm*. As this will remove until the end of the string you need to replace the .mp3 (Note: %% would remove the longest match)
7. echo allows us to check that the mv command would move the previous file name to the correct new file name
8. Once happy and tested you can simply remove the echo and change will actually take place

Let me know if you have any questions? All the above can be found on the links I provided.

David the H. · 03-30-2011, 10:41 AM

Quote:

Originally Posted by mainstream

Code:

FILE="This IS a FiLe WIth MIXeD case.mp3"
echo ${FILE,,}
echo ${FILE// /_}

#also
FILE='00_wiz_khalifa-sakdlaskd.mp3'
[[ $FILE =~ ^[0-9] ]] && echo ${FILE#*[-_]}

I would never come up with something like this, since:
1. I don't quite understand that (looks quite complex to me)
2. Can hardly find anything in literature (specific)

These are all examples of parameter substitution, similar to the one you used in your opening post. ${var,,} is a form new to bash v.4 that converts the variable to lowercase, and ${var// /_} simply substitutes spaces for underscores, similar to sed's s///g command.

And as mentioned before, if the file name doesn't match the pattern then no change is made, so there's no need to test for the patterns first.

Quote:

Input would be
10-Nitrogenetics-hit me-zzzz.mp3 -> nitrogenetics-hit_me.mp3
202_wiz_khalifa-black_&_yellow-(remix)-(82_bpm)-zzzz.mp3 -> wiz_khalifa-black_&_yellow-(remix).mp3

I'd personally use bash's extended globbing feature to help with variable-length strings and simply do something like the following for files like the above:

Code:

IFS=$'\n'           #changing IFS to newline means spaces
                    #are ignored during word-breaking
shopt -s extglob    #enable extended globbing

for file in $@ ; do

     #break input filename into path and name
     #so that names with paths included are correctly handled
     filepath=${file%/*}
     newname=${file##*/}

     #change spaces (and tabs) to underscores in filename
     #extglob reduces multiple spaces to single underscore
     newname=${newname//+([[:space:]])/_}

     #lowercase filename
     newname=${newname,,}

     #remove all initial "digit+[-_]" combinations
     #again extglob handles variable numbers of digits
     newname=${newname#+([0-9])[_-]}

     #remove any final "-zzzz" strings
     #similar to the last operation
     #assumes the file ends in ".mp3", however
     newname=${newname/-+(z).mp3/.mp3}

     #remove any "-(nnn_bpm)" strings
     #remember, spaces have been changed to underscores already
     newname=${newname/-(+([0-9])_bpm)}

     #you can continue adding as many changes as you want here.

     #call mv to change the filename.
     #I used -iv to make it interactive and verbose, but that's up to you.
     echo "File \"$file\" will be changed to: \"$filepath/$newname\""
     mv -iv "$file" "$filepath/$newname"

done

Notice how it's all bash's built-in parameter expansion at work. There are no external commands involved except mv at the end. I also recommend changing the input from a simple * globbing pattern to $@, and feeding the filenames directly into the script as it's launched. In other words, run the script like this:

Code:

./cleanupscript.sh ./dir1/*.mp3 ./dir2/*.mp3

This is more flexible in the long run, allowing you to store the file somewhere like /usr/local/bin, and targeting any directory. It also allows you to feed it multiple files or directories at a time.

#Edit: noticed and corrected a minor coding error.