bash script--variables have unexpected values on invoking mplayer

stairwayoflight · 06-15-2007, 05:13 PM

Hello,

I have a bash script I am working on, with which I am having trouble. It reads from a file that looks like:

01 item1 01
01 item1 02
01 item1 03
01 item1 04
02 item2 01
02 item2 02

etc etc.
Here is an approximation of the script:

Code:

#!/bin/bash
# Download stuff

misseditems="true"

while [ misseditems=="true" ]
do

   misseditems="false"

   cat allitems |
   while read attrib1 attrib2 attrib3
   do
      
      file=blah-"$attrib1"-"$attrib2"-"$attrib3".rm

      if [ -s "$file" ]
      then

         echo $file exists - skipped.

      else

         # download the items
         mplayer -dumpstream rtsp://some.domain/blah/blah-"$attrib1"-"$attrib2"-"$attrib3".rm
      
         if [ $? == 0 ] && [ -s stream.dump ]
         then

            echo "Hooray!"

            mv stream.dump blah-"$attrib1"-"$attrib2"-"$attrib3".rm

         else
      
            echo "$attrib2""$attrib3 >> failedlist
            misseditems="true"

         fi

         sleep 1

      fi

   done

done

It seems to go to about item 10, then there are errors with the output. Specifically the attributes (attrib1, attrib2, attrib3) for each item seem to get "eaten" by something else.

If I could conjecture, I would guess that 1 or 2 values are getting snapped up before the loop comes around again. My loop works if I simply do something like:
echo "$attrib1""$attrib2""$attrib3" >> itemlist
but I get weird results when I pass them to mplayer. (I am using mplayer to download content--yes free content.)

Because I can use the same loop and the correct values are cat'ed to "itemlist", the problem must be with the invocation of mplayer. The item skipping behavior is strange, and it is caught by the outer loop with the test for missed items, but it is a bit ugly.

Does anyone know what the problem is?

Thanks, Timothy

H_TeXMeX_H · 06-15-2007, 06:11 PM

You shouldn't use so many quotes, they can get messy, for example:

Code:

echo "$attrib2""$attrib3 >> failedlist

You're missing one here.

Just don't use quotes like that unless you have to. It leads to strange behavior from bash.

So this line:

Code:

mplayer -dumpstream rtsp://some.domain/blah/blah-"$attrib1"-"$attrib2"-"$attrib3".rm

would be better like:

Code:

mplayer -dumpstream rtsp://some.domain/blah/blah-$attrib1-$attrib2-$attrib3.rm

cfaj · 06-16-2007, 02:33 AM

Quote:

Originally Posted by stairwayoflight

Hello,

I have a bash script I am working on, with which I am having trouble. It reads from a file that looks like:

01 item1 01
01 item1 02
01 item1 03
01 item1 04
02 item2 01
02 item2 02

etc etc.
Here is an approximation of the script:

Code:

#!/bin/bash
# Download stuff

misseditems="true"

while [ misseditems=="true" ]

That will always evaluate to true; you need spaces around the operator,
and that should be =, not ==, and you need to specify a variable, not
a literal string:
Code:
while [ "$misseditems" = "true" ]

Quote:

Code:

do

   misseditems="false"

   cat allitems |

Unnecessary use of cat.

Quote:

Code:

   while read attrib1 attrib2 attrib3
   do

      file=blah-"$attrib1"-"$attrib2"-"$attrib3".rm

      if [ -s "$file" ]
      then

         echo $file exists - skipped.

      else

         # download the items
         mplayer -dumpstream rtsp://some.domain/blah/blah-"$attrib1"-"$attrib2"-"$attrib3".rm

         if [ $? == 0 ] && [ -s stream.dump ]
         then

            echo "Hooray!"

            mv stream.dump blah-"$attrib1"-"$attrib2"-"$attrib3".rm

Code:

mv stream.dump "blah-$attrib1-$attrib2-$attrib3.rm"

Quote:

Code:

         else

            echo "$attrib2""$attrib3 >> failedlist

You are missing a closing quote, and using unnecessary quotes.
Code:
echo "$attrib2$attrib3" >> failedlist

Quote:

Code:

            misseditems="true"

         fi

         sleep 1

      fi

   done

done

It seems to go to about item 10, then there are errors with the output. Specifically the attributes (attrib1, attrib2, attrib3) for each item seem to get "eaten" by something else.

What do you mean by "eaten"? If the variables are empty, it's because there was nothing on input stream.

Quote:

If I could conjecture, I would guess that 1 or 2 values are getting snapped up before the loop comes around again. My loop works if I simply do something like:
echo "$attrib1""$attrib2""$attrib3" >> itemlist
but I get weird results when I pass them to mplayer. (I am using mplayer to download content--yes free content.)

Because I can use the same loop and the correct values are cat'ed to "itemlist", the problem must be with the invocation of mplayer. The item skipping behavior is strange, and it is caught by the outer loop with the test for missed items, but it is a bit ugly.

Clean up the code, and post the actual code you are using.
An approximation tells us nothing.

stairwayoflight · 06-18-2007, 05:46 PM

Quote:

Originally Posted by cfaj

Unnecessary use of cat.

cfaj, I found removing 'cat' from the script breaks it completely, as I have nothing to read my variables from unless I do it another way--and I don't know another way.

For clarity, here is what allbooks2 looks like:

Code:

01 gen 01
01 gen 02
01 gen 03
<truncated for space>
02 exod 01
02 exod 02
02 exod 03
<etc etc>

When I said I thought some values were eaten, I thought perhaps "read var1 var2 var3" means the 3 variables need to be on the same line. If somehow the values ended up getting taken out of the queue, I thought it would explain how either one or two or all of them end up looking like a null. Eg. aside from the lines in 'failedlist' which resulted from an mplayer crash, I haven't seen any $book values in the file. That seems to be consistent behavior even through the process of tweaking the script as I have tried to eliminate errors.

Ok, here is the code, with some fixups suggested by people here. As I recall the first attempt at this script was without any variable quoting, and seemed to have the same kind of failure I mentioned above, but perhaps a greater percentage of failure. Anyways removing the quotes around variables didn't solve the problem.

Code:

#!/bin/bash

missedchapters="true"
while [ "$missedchapters" = "true" ]
do
   missedchapters="false"
   cat allbooks2 |
   while read prefix book chapter
   do
      file=esv-$prefix-$book-$chapter.rm
      if [ -s $file ]
      then
         echo $file exists - skipped.
      else
         mplayer -dumpstream rtsp://ra.gospelcom.net/bible/english/esv/max_mclean/rm_test/english-esv-$book-$chapter-mm.rm
         if [ $? == 0 ] && [ -s stream.dump ]
         then
            echo "Hooray!"
            mv stream.dump esv-$prefix-$book-$chapter.rm
         else
            echo "blech... i had a problem but i'll try and come back to this one.."
            echo "$book$chapter" >> failedlist
            missedchapters="true"
         fi
         sleep 1
      fi
   done
done

I have been trying a modified version with the first suggested change--removing all the quotes around variables. The "failedlist" output from the script is:

Code:

$ cat failedlist




11
deut13
josh20

(I did this before the script completed, it takes some time.)

I checked mplayer's output on the terminal and it apparently crashed on joshua 20. There's not much we can do about that maybe, and the perhaps the same thing happened with deuteronomy 13. But the 5 preceding errors look more related to the script itself.

cfaj · 06-18-2007, 09:44 PM

Quote:

Originally Posted by stairwayoflight

cfaj, I found removing 'cat' from the script breaks it completely, as I have nothing to read my variables from unless I do it another way--and I don't know another way.

You canot just remove it. You have to redirect input from the file:
Code:
while read VAR1 VAR2 VAR3
do
  : do whatever
done < FILENAME

Quote:

For clarity, here is what allbooks2 looks like:

Code:

01 gen 01
01 gen 02
01 gen 03
<truncated for space>
02 exod 01
02 exod 02
02 exod 03
<etc etc>

When I said I thought some values were eaten, I thought perhaps "read var1 var2 var3" means the 3 variables need to be on the same line. If somehow the values ended up getting taken out of the queue, I thought it would explain how either one or two or all of them end up looking like a null. Eg. aside from the lines in 'failedlist' which resulted from an mplayer crash, I haven't seen any $book values in the file. That seems to be consistent behavior even through the process of tweaking the script as I have tried to eliminate errors.

mplayer is also reading from stdin. Redirect the input to mplayer:
Code:
mplayer rtsp://ra....rm < /dev/null

Quote:

Ok, here is the code, with some fixups suggested by people here. As I recall the first attempt at this script was without any variable quoting, and seemed to have the same kind of failure I mentioned above, but perhaps a greater percentage of failure. Anyways removing the quotes around variables didn't solve the problem.

Always quote variables unless you want filename expansion and wordsplitting to be performed on them.

stairwayoflight · 07-08-2007, 12:17 PM

Thank you, cfaj.

I have updated the script but running it will take some time. I'll post back the results.

I am very busy with wedding planning and whatnot, but I'll try to not be too long.

Tim