Using Mplayer to check a files integrity?

Hey guys

Trying to write a script that verifies a files integrity and pulls information with mplayer; however with a file that is not complete, say chopped half way through, mplayer -identify information will not complain.

Can anyone offer suggestions that will report this, and other errors? It needs to be relatively quick; but I'm happy with it parsing the file at an uncapped framerate or something. It also really needs to work on both music and video files, of all formats that work with mplayer but I guess I'll take whatever suggestions come my way

Cheers,
Fragged

the only way i can think is with a CRC or somthing similer
because if the media is clipped off you cant really tell because there is no footer

The plethora of media formats, containers, non-standard ways of encoding things, and other stuff like that makes it very difficult to analyze the media based on the contents. The only plausible way that I can think of is to use some checksum method, crc, md5sums, etc. But I am truly no specialist in media formats.

Originally Posted by i92guboj The plethora of media formats, containers, non-standard ways of encoding things, and other stuff like that makes it very difficult to analyze the media based on the contents. The only plausible way that I can think of is to use some checksum method, crc, md5sums, etc. But I am truly no specialist in media formats.

isnt that what i just said?

Originally Posted by fragged Hey guys Trying to write a script that verifies a files integrity and pulls information with mplayer; however with a file that is not complete, say chopped half way through, mplayer -identify information will not complain. Can anyone offer suggestions that will report this, and other errors? It needs to be relatively quick; but I'm happy with it parsing the file at an uncapped framerate or something. It also really needs to work on both music and video files, of all formats that work with mplayer but I guess I'll take whatever suggestions come my way Cheers, Fragged

Depending on what your definition of 'quick' is, I might have something that do what you want.

In fact, having had a similar problem with 220x20 minutes video clips, I wrote a BASH script which will play an arbitrary number of files at once (my system has trouble playing over 5 rmvb at the same time) and report files that didn't play for the correct time.

Therefore, if you must check a large number of video/audio files, this is a dirty and relatively fast way of achieving your goal.

The script is currently customized for the wmii window manager and uses the xine player rather than mplayer. Though, if you are familiar with BASH, reading the comments should help you customizing it to your needs within a few minutes.

Here's the script, if you are interested:

#! /bin/bash
#Author: Olivier Diotte
#Created on: 2009-08-30
#You are free to modify or use this script in any way you please as long as
#this note and the above is kept intact.

#function time2number
#Args:
#	$1: time of the form [[hh:]mm:]ss (required)
#Returns:
#	Total decimal number of seconds
function time2number () {
	local time="$1"
	local b_error=0
	local seconds=0
	local minutes=0
	local hours=0

	if [ $# -ne 1 ]; then
		b_error=1
	else
		hours=$( /usr/bin/date -d "$time" '+%H' ) && \
				minutes=$( /usr/bin/date -d "$time" '+%M' ) && \
				seconds=$( /usr/bin/date -d "$time" '+%S' )
		b_error=$?
	fi

	if [ $b_error -eq 0 ]; then
		echo $(( 10#$hours * 3600 + 10#$minutes * 60 + 10#$seconds ))
	else
		echo "Usage: time2number {[[hh:]mm:]ss}" >&2
	fi

	return $b_error
}

function abs () {
	if [ $1 -lt 0 ]; then
		echo $(( - $1 ))
	else
		echo $1
	fi
}

function checkVideo () {
	local file="$1"
	local length=$( time2number $( /usr/bin/ffmpeg -i "$file" 2>&1 | sed -rn \
			's/Duration:[[:blank:]]+(([[:digit:]]+:?)+).*$/\1/p' ) )
	local startTime=$( time2number $( date '+%H:%M:%S' ) )
	local startDate=$( date '+%Y%m%d' )
	local endTime=0
	local endDate=0
	local duration=0
	local tolerance='5' #seconds
	local graceTime='30' #seconds

 	#24 hours * 60 minutes/hour * 60 seconds/minute
	local -r NB_SECS_IN_A_DAY=86400

	timelimit "$length" $playerCmd "$file" 1>/dev/null 2>&1

	endTime=$( time2number $( date '+%H:%M:%S' ) )
	endDate=$( /usr/bin/date '+%Y%m%d' )

	#For now we'll assume it finished the day after it was started
	if [ "$startDate" -ne "$endDate" ]; then
		duration=$(( $NB_SECS_IN_A_DAY - $startTime + $endTime ))
	else
		duration=$(( $endTime - $startTime ))
	fi

	if [ "$duration" -eq "$length" ]; then
		echo "File $file length of \"$length\" perfectly matches runtime of "\
				"\"$duration\""
	elif [ "$( abs $(( $duration - $length )) )" -le "$tolerance" ]; then
		echo "File $file has length \"$length\" which roughly matches runtime "\
				"of \"$duration\""
	else
		echo "File $file has length: \"$length\" which doesn't match "\
				"runtime of: \"$duration\""
	fi
}

#function waitForVideo
#Waits for the player window to open (could roughly be replaced by a 'sleep 2s')
#Args
#	$1:	List of previous clients (as given by wmiir read /tag/$tag/index)
function waitForVideo () {
	local prevCli=$1

	if ! wmiir ls /tag/ | grep "$tag" 1>/dev/null; then
		until wmiir ls /tag/ | grep "$tag" 1>/dev/null; do
			sleep 1s
		done
 	else
		until wmiir read "/tag/$tag/index" 2>/dev/null | \
				grep -v "$prevCli" >/dev/null; do
			sleep 1s
		done
	fi
}

#Start Xine with sound muted (-a [huge number here]) quit upon end of playlist,
#don't show windows other than video
playerCmd="/usr/bin/xine -a 10 --auto-play=q -g"

#Max concurrent instances (8 seems to be the maximum my system accept without
#lagging)
concurrentVideos=${concurrentVideos:-8}

basePlayer=$( /usr/bin/basename $( echo $playerCmd | /usr/bin/cut -d\  -f1 ) )
tag="$( /usr/bin/basename $0 )"
prevTagRules=$( wmiir read /tagrules )

#echo "Usage: $0 [file] ..." >&2

#Wmii-specific command. Basically it adds a tag rule (if not already present)
#to send the checked files windows in a specific "tag" (roughly a virtual desktop)
wmiir read /tagrules | grep "/$basePlayer.*/ -> $tag" 1>/dev/null || \
		echo -e "/$basePlayer.*/ -> $tag\n$prevTagRules" | wmiir write /tagrules

for file in $@; do
	checkVideo $file &

	#Waits for the player window to open (could roughly be replaced by
	#a 'sleep 2s')
	waitForVideo "$( wmiir read "/tag/$tag/index" 2>/dev/null )"

	#Wmii-specific commands needed for the videos to start (strip it
	#if not using Wmii)
	prevView=$( wmiir read /ctl | grep view )
	wmiir xwrite /ctl "view $tag"
	/home/vhann/bin/orderWindows.bash $tag
	wmiir xwrite /ctl "$prevView"

	#This loop waits until one of the videos terminates (otherwise
	#this script would spawn all videos at once which would overflow
	#the system if a large number of videos must be tested)
	#The first part of the test could roughly be replaced by something like:
	#'$( pgrep -fl "$basePlayer" )'
	while [ $( wmiir read /tag/$tag/index | grep -v '^#' | wc -l ) \
			-ge "$concurrentVideos" ]; do
		/usr/bin/sleep 2s
	done
done

#Restore previous tag rules (Wmii-specific command, can be stripped)
echo "$prevTagRules" | wmiir write /tagrules

exit 0

Vhann thanks, should be what I want; I'll give it a look in my free time. I'm unfamiliar with the way that xine works; but the way you've suggested is probably a very good way; even if that means I end up writing my own script

As for the two posts above, unfortunately, seeing as the files may well be corrupt already, using checksums wouldnt work.

Originally Posted by fragged Vhann thanks, should be what I want; I'll give it a look in my free time. I'm unfamiliar with the way that xine works; but the way you've suggested is probably a very good way; even if that means I end up writing my own script As for the two posts above, unfortunately, seeing as the files may well be corrupt already, using checksums wouldnt work.

Basically, you just need to edit the lines regarding wmii: they are all situated after the function definitions (in the "main function") and edit the line that reads 'playerCmd="/usr/bin/xine ...."' to match the command you want to be issued when playing a file (i.e. a playerCmd of 'echo' would launch execute 'echo' (should it be a function or a file or whatever) with each filename as its first argument.

Being in a good mood, I've messed a bit with my script. I think this should do it for you:

#! /bin/bash
#Author: Olivier Diotte
#Created on: 2009-08-30
#You are free to modify or use this script in any way you please as long as
#this note and the above is kept intact.

#function time2number
#Args:
#	$1: time of the form [[hh:]mm:]ss (required)
#Returns:
#	Total decimal number of seconds
function time2number () {
	local time="$1"
	local b_error=0
	local seconds=0
	local minutes=0
	local hours=0

	if [ $# -ne 1 ]; then
		b_error=1
	else
		hours=$( /usr/bin/date -d "$time" '+%H' ) && \
				minutes=$( /usr/bin/date -d "$time" '+%M' ) && \
				seconds=$( /usr/bin/date -d "$time" '+%S' )
		b_error=$?
	fi

	if [ $b_error -eq 0 ]; then
		echo $(( 10#$hours * 3600 + 10#$minutes * 60 + 10#$seconds ))
	else
		echo "Usage: time2number {[[hh:]mm:]ss}" >&2
	fi

	return $b_error
}

function abs () {
	if [ $1 -lt 0 ]; then
		echo $(( - $1 ))
	else
		echo $1
	fi
}

function checkVideo () {
	local file="$1"
	local length=$( time2number $( /usr/bin/ffmpeg -i "$file" 2>&1 | sed -rn \
			's/Duration:[[:blank:]]+(([[:digit:]]+:?)+).*$/\1/p' ) )
	local startTime=$( time2number $( date '+%H:%M:%S' ) )
	local startDate=$( date '+%Y%m%d' )
	local endTime=0
	local endDate=0
	local duration=0
	local tolerance='5' #seconds
	local graceTime='30' #seconds

 	#24 hours * 60 minutes/hour * 60 seconds/minute
	local -r NB_SECS_IN_A_DAY=86400

	timelimit "$length" $playerCmd "$file" 1>/dev/null 2>&1

	endTime=$( time2number $( date '+%H:%M:%S' ) )
	endDate=$( /usr/bin/date '+%Y%m%d' )

	#For now we'll assume it finished the day after it was started
	if [ "$startDate" -ne "$endDate" ]; then
		duration=$(( $NB_SECS_IN_A_DAY - $startTime + $endTime ))
	else
		duration=$(( $endTime - $startTime ))
	fi

	if [ "$duration" -eq "$length" ]; then
		echo "File $file length of \"$length\" perfectly matches runtime of "\
				"\"$duration\""
	elif [ "$( abs $(( $duration - $length )) )" -le "$tolerance" ]; then
		echo "File $file has length \"$length\" which roughly matches runtime "\
				"of \"$duration\""
	else
		echo "File $file has length: \"$length\" which doesn't match "\
				"runtime of: \"$duration\""
	fi
}

playerCmd="mplayer"

#Max concurrent instances (8 seems to be the maximum my system accept without
#lagging)
concurrentVideos=${concurrentVideos:-8}

basePlayer=$( /usr/bin/basename $( echo $playerCmd | /usr/bin/cut -d\  -f1 ) )

for file in $@; do
	checkVideo $file &

	#Waits for the player window to open (could roughly be replaced by
	#a 'sleep 2s')
sleep 2s

	#This loop waits until one of the videos terminates (otherwise
	#this script would spawn all videos at once which would overflow
	#the system if a large number of videos must be tested)
	#The first part of the test could roughly be replaced by #something like:
	#'$( pgrep -fl "$basePlayer" )'
	while [ $( pgrep -fl "$basePlayer" | wc -l ) \
			-ge "$concurrentVideos" ]; do
		/usr/bin/sleep 2s
	done
done

exit 0

I think this should do it (but then you will probably want to mute sound or you'll hear 8 files playing at the same time).

Supposing the script is named testVideos.bash, you would invoke it like this:
/path/to/testVideos.bash file.mpg dir/to/files*.avi /home/my/videos/files*.flv [...] > fileToSendOutputTo.txt

EDIT: Forgot to mention, you will probably want to redirect the output of the script to a file (since the script will output one line per file). See the 'checkVideo' function for what output will ressemble.

I guess you get the idea.

Hope this helps

Note: You need ffmpeg for this script to work though

Regards,
Vhann

if you have a powerful system you could get 12 out of it

Originally Posted by smeezekitty if you have a powerful system you could get 12 out of it

Oh sure, 8 is what my Core 2 Duo 1.50 GHz allows me (if I put more, the videos will delay and the duration times won't be accurate, therefore making the script pretty useless).

But then, find the correct number for you, this is a number you can change ^^ .

EDIT: Forgot to mention, you may change that number permanently (concurrentVideos=) directly in the script or you can also change it on a per run basis like this:
concurrentVideos=[numberHere] /path/to/testVideos.bash {file} ...

Regards,
Vhann

i got 12 out of my 2GHZ 2 core

It depends on the videos. If you want accuracy, stay below the minimum, otherwise if a video is harder to decode for some reason or has an higher resolution it will crap out the measurements for all the rest.

I'd also stop any background daemon, overall cron, which can launch things like updatedb on the worst moment causing heavy i/o and enough cpu usage to screw your numbers.

The proposed method just verifies that the reported time matches the actual duration. There could be complete garbage (non-sense video) in the middle and this method would still validate the file.

The mplayer options '-benchmark -vo null -ao null -quiet' can be used to play a file as fast as possible. mplayer will report various errors in the stream to stderr. This may be a better metric than matching the header to the playing time. If the encoding software mis-reported the time in the header there may be nothing wrong with the video. Also, if there is intermediate junk (corrupt I-P-B frame) this is likely to be reported.

This will check that mplayer can process the file. mplayer is propably the best software for understanding various video formats. I would not recommend it for checking audio (unless it is with video). There are various packages such as mpck, mp3info, mpg123 with benchmark options, etc that will provide better data. If you are scripting anyways, you can detect the format and then process the will with the appropriate binary.

Originally Posted by bpringlemeir The proposed method just verifies that the reported time matches the actual duration. There could be complete garbage (non-sense video) in the middle and this method would still validate the file.

You are totally right, thanks for saying (so people won't take this for infallible software). That's what I meant by "dirty" method of checking files.

Regards,
Vhann

Cheers guys, I ended up using 'mplayer -ao null -vo null -fps 100000 FILE.avi' with python parsing the script; this *appears* to work for files with incorrect times; however I dont have any other corrupt files to test with it but I'll update the script as I do.

For reference here's the script, its a little dodgy but gets the job done quickly - at least on my quad .

def TestMovieIntegrity(file):
	#2>&1 returns stderr to stdout
	command = 'mplayer -ao null -vo null -fps 9999999
 -msglevel all=1 ' + file + ' 2>&1'
	output = os.popen(command).read()
	
	if output.find('Error at MB') != -1:
		return False
	return True