[SOLVED] issue with a for loop in a shell script i am working on

frieza · 12-22-2010, 01:25 PM

trying to loop through the results of a command such as

Code:

for file in `ls *.avi | awk 'NR>0&&NR<=1' | cut -d .  -f 1`;
 do 
echo $file; 
done

outputs

Code:

Tom
and
Jerry
-
001
-
Puss
Gets
the
Boot
[1939]

whereas simply outputing the command without the loop

Code:

ls *.avi | awk 'NR>0&&NR<=1' | cut -d .  -f 1

outputs

Code:

Tom and Jerry - 001 - Puss Gets the Boot [1939]

my assumption at this point is the spaces in the file names are getting in the way somehow, how can i surpress the spaces from getting in the way?
in short how can i get the above loop to actually work and display each file name on their own line not each word of each file name
and yes i intend on replacing the 'NR>0&&NR<=1' with "NR>$1&&NR<=$2" to control the start and end

David the H. · 12-22-2010, 01:49 PM

A for loop uses standard word-breaking techniques to decide what to iterate on. By default this is space/tab/newline. There are a couple of ways to work around this.

To start with, $(..) is highly recommended over the older `..` form of embedded command. Pretty much all modern shells support it.

This then leads to the first workaround. Assuming that the entries are newline-separated, you simply need to quote both the embedded command and the variable substitution to preserve spaces and tabs.

Code:

for file in "$(ls *.avi | awk 'NR>0&&NR<=1' | cut -d .  -f 1)"; do 
	echo "$file" 
done

The second, and "safer" way, if a bit more complex, is to change the default word-breaking characters by setting them in the IFS (internal field separator) shell variable.

Code:

IFS=$'\n'

for file in $(ls *.avi | awk 'NR>0&&NR<=1' | cut -d .  -f 1); do 
	echo $file
done

unset IFS

$'\x' is a bash form for embedding escaped characters such as \n (newline) and \t (tab).

Finally, you shouldn't even need that complicated embedded command. Parsing ls isn't recommended for one thing. Second, you shouldn't need to use cut and awk together, as awk is more than capable of handling it all itself. But all you want to do is remove the file ending, that can be done with a simple parameter substitution.

Code:

IFS=$'\n'

for file in *.avi; do 
	echo ${file%.*}
done

unset IFS

frieza · 12-22-2010, 02:05 PM

thanks, the second one appears to work
in case you are wondering, the end result is a modified version of a script i posted here a long time back for mass converting of video files to dvd video

Code:

#initialize the variable to increment for the jump statement in the xml file
let y=1
#remove the old xml file
rm -f ../mpgs/naruto.xml
#generate the static part at the begining of the xml file
echo "<dvdauthor dest=\"DVD\">" > ../mpgs/naruto.xml
echo "	<vmgm />" >> ../mpgs/naruto.xml
echo "	<titleset>" >> ../mpgs/naruto.xml
echo "		<titles>" >> ../mpgs/naruto.xml
echo "			<video aspect=\"16:9\" format=\"ntsc\" widescreen=\"nopanscan\" />" >> ../mpgs/naruto.xml
#set the upper bounds for the jump statement to one higher then the number of the last title
a=$1
b=$2
let c=$b-$a
let z=$c+2 
#repeat from the first number until the second in sequence
IFS=$'\n'

for file in $(ls *.$3 | awk 'NR>0&&NR<=1' | cut -d .  -f 1); do 

#increment the jump variable
let x=$y
let y=$y+1
#if on the last title set it to jump to title 1 (loop the dvd)
let end=$2+1;
if [ "$y" = "$end" ]
then 
let y=1
fi


#convert from divx to standard avi then remove the divx
mencoder -ovc lavc -oac lavc $file.$3 -o episode$x.avi
#rm -f *_"$x".avi
#convert from avi to mpeg2 for dvd then remove the avi
ffmpeg -i episode$x.avi -target dvd ../mpgs/episode$x.mpg
rm -f episode$x.avi
#generate the title part of the xml file
echo "			<pgc>" >> ../mpgs/naruto.xml
echo "			<vob file=\"episode$x.mpg\" chapters=\"0,5:00,10:00,15:00,20:00,25:00\"/>" >> ../mpgs/naruto.xml
echo "				<post> jump title $y; </post>" >> ../mpgs/naruto.xml
echo "			</pgc>" >> ../mpgs/naruto.xml
done
unset IFS
#add the closing tags to the divx
echo "		</titles>" >> ../mpgs/naruto.xml
echo "	</titleset>" >> ../mpgs/naruto.xml
echo "</dvdauthor>" >> ../mpgs/naruto.xml
#switch to the mpgs directory
cd ../mpgs
#generate the raw DVD image from the mpgs
dvdauthor -x naruto.xml
#burn the dvd
growisofs -Z /dev/dvd -dvd-video DVD/
#clean up
rm -rf *.mpg
rm -rf DVD
eject /dev/dvd

frieza · 12-22-2010, 02:41 PM

correction it's not quite working, back to the drawing board
any suggestions on how to improve the efficiency of the script would be appreciated

frieza · 12-22-2010, 03:20 PM

nvm
i figured out a couple errors i made
the correct script is

Code:

#initialize the variable to increment for the jump statement in the xml file
let y=$1
#remove the old xml file
rm -f ../mpgs/naruto.xml
#generate the static part at the begining of the xml file
echo "<dvdauthor dest=\"DVD\">" > ../mpgs/naruto.xml
echo "	<vmgm />" >> ../mpgs/naruto.xml
echo "	<titleset>" >> ../mpgs/naruto.xml
echo "		<titles>" >> ../mpgs/naruto.xml
echo "			<video aspect=\"16:9\" format=\"ntsc\" widescreen=\"nopanscan\" />" >> ../mpgs/naruto.xml
#set the upper bounds for the jump statement to one higher then the number of the last title
a=$1
b=$2
let c=$b-$a
let z=$c+2 
#repeat from the first number until the second in sequence
IFS=$'\n'

for file in $(ls *.$3 | awk "NR>$1&&NR<=$2" | cut -d .  -f 1); do 

#increment the jump variable
let x=$y
let y=$y+1
#if on the last title set it to jump to title 1 (loop the dvd)
let end=$2+1;
if [ "$y" = "$end" ]
then 
let y=1
fi

echo $file
#convert from divx to standard avi
mencoder -ovc lavc -oac lavc $file.$3 -o episode$y.avi
convert from avi to mpeg2 for dvd then remove the avi
ffmpeg -i episode$y.avi -target dvd ../mpgs/episode$y.mpg
rm -f episode$y.avi
#generate the title part of the xml file
echo "			<pgc>" >> ../mpgs/naruto.xml
echo "			<vob file=\"episode$y.mpg\" chapters=\"0,5:00,10:00,15:00,20:00,25:00\"/>" >> ../mpgs/naruto.xml
echo "				<post> jump title $y; </post>" >> ../mpgs/naruto.xml
echo "			</pgc>" >> ../mpgs/naruto.xml
done
unset IFS
#add the closing tags to the divx
echo "		</titles>" >> ../mpgs/naruto.xml
echo "	</titleset>" >> ../mpgs/naruto.xml
echo "</dvdauthor>" >> ../mpgs/naruto.xml
#switch to the mpgs directory
cd ../mpgs
#generate the raw DVD image from the mpgs
dvdauthor -x naruto.xml
#burn the dvd
growisofs -Z /dev/dvd -dvd-video DVD/
#clean up
rm -rf *.mpg
rm -rf DVD
eject /dev/dvd

colucix · 12-22-2010, 03:21 PM

Another option is a while loop using the read built-in:

Code:

while read file
do
  echo $file
done < <(ls *.avi)

The read statement assigns every line of input to the variable at each iteration and the input is not split into words. Also check process substitution, that prevent the shell from launching a subshell.

frieza · 12-22-2010, 03:29 PM

true but the goal as you can see by the script was to specify a start/end point to convert files between $1 and $2 out of a potentially much longer list of files

ntubski · 12-22-2010, 04:09 PM

Is there a reason for the 2-step conversion instead converting to MPEG with mencoder directly?

This script can be simplified some (untested):

Code:

#!/bin/bash

start="$1"
end="$2"
ext="$3"

#generate the static part at the begining of the xml file
cat <<'EOF' > ../mpgs/naruto.xml
<dvdauthor dest="DVD">

	<vmgm />
	<titleset>
		<titles>
			<video aspect="16:9" format="ntsc" widescreen="nopanscan" />
EOF

epnum=1
for file in *."$ext" ; do
    if ((epnum < start)) ; then continue; fi
    if ((epnum > end)) ; then break; fi

    #if on the last title set it to jump to title 1 (loop the dvd)
    let 'jumpto = (epnum == end)? 1 : epnum+1'

    episode=episode$epnum

    #convert from divx to standard avi
    mencoder -ovc lavc -oac lavc "$file" -o $episode.avi

    #convert from avi to mpeg2 for dvd then remove the avi
    ffmpeg -i $episode.avi -target dvd ../mpgs/$episode.mpg
    rm -f $episode.avi

    #generate the title part of the xml file
    cat >> ../mpgs/naruto.xml <<EOF
			<pgc>
			<vob file="$episode.mpg" chapters="0,5:00,10:00,15:00,20:00,25:00"/>
				<post> jump title $jumpto; </post>
			</pgc>
EOF
    let epnum++
done

#add the closing tags to the divx
cat >> ../mpgs/naruto.xml <<'EOF'
		</titles>
	</titleset>
</dvdauthor>
EOF

#switch to the mpgs directory
cd ../mpgs
#generate the raw DVD image from the mpgs
dvdauthor -x naruto.xml
#burn the dvd
growisofs -Z /dev/dvd -dvd-video DVD/
#clean up
rm -rf *.mpg
rm -rf DVD
eject /dev/dvd