Problem while writing Shell Script

Rahil Parikh · 11-28-2010, 01:55 AM

Hello,

I am new to writing shell scripts. So, please bare with me. I am currently trying to write a shell script which will read the directory path as input from user and will traverse the Dir tree to find all available audio and video files. I have tried to write as much as I could but I don't know where I am making mistake as I get some files to be audio file which are actully tar balls. On the second note there are some files which video but script shows them to be audio. And, some video files are completely skipped. I am giving the shell script below so that you can see. I am using two external files as source which I am attaching.

Code:

#!/bin/bash

#Let's load the extensions that we want to search for
vdExt=$(cat vdExtList)
adExt=$(cat adExtList)

#Clearing screen won't hurt
clear

#Now let' start with real stuff
echo "Enter starting directory : "
read initDir

#Get qualified paths for all files & dirs
dirTree=$(find $initDir -print)

for dirTreeEntry in $dirTree
do
	#Check if dirTreeEntry is a file
	timeToExit=0
	if [ -f $dirTreeEntry ]
	then
		#Get file name and convert it to lowercase for further string comparision
		fileName=$(basename $dirTreeEntry | tr [:upper:] [:lower:])
		
		#Ittrate through video extensions and if filename has
		#desired extension then we have found our file so now
		#it's time to skip to next file
		for ext in $vdExt
		do
			if [ $(echo $fileName | grep "$ext") ]
			then
				echo "$dirTreeEntry is a video file."
				timeToExit=1
				break;
			fi
				
			#We have already found out video file so we need not
			#search for audio file. It's safe to skip audio file
			#search.
			if [ $timeToExit -eq 1 ]
			then
				timeToExit=0
				break
			fi
		done
		
		#Ittrate through audio extensions and if filename has
		#desired extension then we have found our file so now
		#it's time to skip to next file
		for ext in $adExt
		do
			if [ $(echo $fileName | grep "$ext") ]
			then
				echo "$dirTreeEntry is an audio file."
				timeToExit=1
				break;
			fi
				
			if [ $timeToExit -eq 1 ]
			then
				timeToExit=0
				break
			fi
		done
	fi
done
exit 0

So, I would request someone to inspect my script and help me understand what is my mistake.

Thank you,
Rahil

unSpawn · 11-28-2010, 05:19 AM

First of all well done for providing your script. That really helps. You do not need to load up your ExtLists (just 'grep' them), you don't need to iterate through 'dirTreeEntry in $dirTree' (make 'find' find only files), iterating over extensions isn't efficient either ('grep') and you don't need lower case ('grep -i'):

Code:

#!/bin/bash
# ExtLists location:
LISTPATH=/dev/shm
echo "Enter starting directory : "
read initDir

find $initDir -type f -printf "%p\n" 2>/dev/null| while read ITEM; do
	EXT="${ITEM//*./}"
	if [ ${#EXT} -ne 0 ]; then
		grep -qim1 "^\.${EXT}$" ${LISTPATH}/vdExtList && echo "${ITEM} has video"
		grep -qim1 "^\.${EXT}$" ${LISTPATH}/adExtList && echo "${ITEM} has audio"
	else
		file "${ITEM}" 2>/dev/null|grep -qi video && echo "${ITEM} has video"
		file "${ITEM}" 2>/dev/null|grep -qi audio && echo "${ITEM} has audio"
	fi
done
exit 0

...however extensions are not a hard requirement and file has a finite database, so:

Code:

#!/bin/bash
echo "Enter starting directory : "
read initDir

find $initDir -type f -printf "%p\n" 2>/dev/null| while read ITEM; do
	RESULT=$(ffmpeg -vframes 1 -ss 00:01:00 -i "${ITEM:=item}" 2>&1| grep 'Stream.*[Au|Vi]')
	VIDEO="${RESULT//Video: /}"; AUDIO="${RESULT//Audio: /}"; 
	[ ${#VIDEO} -ne ${#RESULT} && echo "${ITEM} has video"
	[ ${#AUDIO} -ne ${#RESULT} && echo "${ITEM} has audio"
done
exit 0

may provide a more accurate method.

Rahil Parikh · 11-28-2010, 10:35 AM

Hey unSpawn,

Thank you very very much for such a quick reply. I saw what you wrote in the script but I am new to writing the scripts. So, the script you wrote seems to be pretty complex to me and I am unable to break it into pieces to understand. For that reason, I would like to request you to explain how the script works and I would be interested in learning the style/method behind it so that I can improve.

Thank you,
Rahil

Quote:

Originally Posted by unSpawn

First of all well done for providing your script. That really helps. You do not need to load up your ExtLists (just 'grep' them), you don't need to iterate through 'dirTreeEntry in $dirTree' (make 'find' find only files), iterating over extensions isn't efficient either ('grep') and you don't need lower case ('grep -i'):

Code:

#!/bin/bash
# ExtLists location:
LISTPATH=/dev/shm
echo "Enter starting directory : "
read initDir

find $initDir -type f -printf "\%p"\"\n" 2>/dev/null| while read ITEM; do
	EXT="${ITEM//*./}"
	if [ ${#EXT} -ne 0 ]; then
		grep -qim1 "^\.${EXT}$" ${LISTPATH}/vdExtList && echo "${ITEM} has video"
		grep -qim1 "^\.${EXT}$" ${LISTPATH}/adExtList && echo "${ITEM} has audio"
	else
		file "${ITEM}" 2>/dev/null|grep -qi video && echo "${ITEM} has video"
		file "${ITEM}" 2>/dev/null|grep -qi audio && echo "${ITEM} has audio"
	fi
done
exit 0

...however extensions are not a hard requirement and file has a finite database, so:

Code:

#!/bin/bash
echo "Enter starting directory : "
read initDir

find $initDir -type f -printf "\%p"\"\n" 2>/dev/null| while read ITEM; do
	RESULT=$(ffmpeg -vframes 1 -ss 00:01:00 -i "${ITEM:=item}" 2>&1| grep 'Stream.*[Au|Vi]')
	VIDEO="${RESULT//Video: /}"; AUDIO="${RESULT//Audio: /}"; 
	[ ${#VIDEO} -ne ${#RESULT} && echo "${ITEM} has video"
	[ ${#AUDIO} -ne ${#RESULT} && echo "${ITEM} has audio"
done
exit 0

may provide a more accurate method.

unSpawn · 11-28-2010, 02:05 PM

Quote:

Originally Posted by Rahil Parikh

I would be interested in learning the style/method behind it

None really. Reading and just doing mostly.

Quote:

Originally Posted by Rahil Parikh

explain how the script works

OK

Code:

# Use find to look for files only and always print the full path and item name with a newline and read it into variable ITEM
find $initDir -type f -printf "%p\n" 2>/dev/null| while read ITEM; do
        # Strip down ITEM to the last dot to get the "extension" in variable EXT
        # Try this on the CLI to see what I mean: '/bin/bash; set -vx; ITEM=/sbin/.metallica-jump_in_the_fire.aac; eval echo "${ITEM//*./}";'
	EXT="${ITEM//*./}"
        # Count amount of chars ('man test') to see if the variable holds something. String comparison is not needed.
	if [ ${#EXT} -ne 0 ]; then
                # If EXT isn't empty then
                # grep ${LISTPATH}/?dExtList for a line starting (^) with a literal dot (\.) and
                # the extension string quitly (-q) and finding only one match (speed).
		grep -qim1 "^\.${EXT}$" ${LISTPATH}/vdExtList && echo "${ITEM} has video"
		grep -qim1 "^\.${EXT}$" ${LISTPATH}/adExtList && echo "${ITEM} has audio"
	else
                # If EXT is empty then try running 'file' on ITEM and grep for a case-insensitive string
		file "${ITEM}" 2>/dev/null|grep -qi video && echo "${ITEM} has video"
		file "${ITEM}" 2>/dev/null|grep -qi audio && echo "${ITEM} has audio"
	fi
done
exit 0

Code:

# basically the same as above except here you determine the result using one tool: ffmpeg.
        # Store the result of running 'ffmpeg' on ITEM in variable RESULT and grep for a string that matches either
        # "Stream:.*Audio" or "Stream:.*Video". Speed up ffmpeg processing by extracting only one frame and limited time.
	RESULT=$(ffmpeg -vframes 1 -ss 00:01:00 -i "${ITEM:=item}" 2>&1| grep 'Stream.*[Au|Vi]')
        # Subtract the literal string "Video:" from RESULT into variable VIDEO and the same for Audio.
	VIDEO="${RESULT//Video: /}"; AUDIO="${RESULT//Audio: /}"; 
        # So if the amount of chars in the variable VIDEO does not match RESULT then ITEM must contain video stream.
	[ ${#VIDEO} -ne ${#RESULT} && echo "${ITEM} has video"
	[ ${#AUDIO} -ne ${#RESULT} && echo "${ITEM} has audio"

Rahil Parikh · 11-29-2010, 03:48 PM

Thanks unSpawn! That script worked like a gem! It gave me really hard time to understand but I got it almost. I have some doubts which I want to ask you.

Like,

Code:

find $initDir -type f -printf "%p\n" 2>/dev/null| while read ITEM; do

When does the printf gets executed?

Code:

grep -qim1 "^\.${EXT}$" ${LISTPATH}/vdExtList && echo "${ITEM} may be video."

How does echo know that now that the grep has found the extension and it's time to print to STDOUT?

unSpawn · 11-29-2010, 06:04 PM

Quote:

Originally Posted by Rahil Parikh

When does the printf gets executed?

Printing items is executed as part of the 'find' command.

Code:

grep -qim1 "^\.${EXT}$" ${LISTPATH}/vdExtList && echo "${ITEM} may be video."

How does echo know that now that the grep has found the extension and it's time to print to STDOUT?[/QUOTE]
The return value in variable "$?" (try '/bin/true; echo $?; /bin/false; echo $?;') tells the shell what to do.

Using "&&" could be translated as

Code:

if [ $? -eq 0 ]; then
 echo "${ITEM} may be video."
fi

or

Code:

case "$?" in
 0)  echo "${ITEM} may be video."
 ;;
esac

Here are some scripting guides that might come in handy:
http://www.tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
http://www.tldp.org/LDP/Bash-Beginne...tml/index.html
http://www.tldp.org/LDP/abs/html/