[SOLVED] Read bash script output and note iterations with errors

flackend · 08-31-2011, 10:56 PM

Hi,

I'm writing a bash script that iterates through a series of websites/records and saves screenshots. If the webpage loads I see this output:

Code:

Fetching http://www.google.com/ ...
 ... done

If there's an issue loading the page:

Code:

Fetching http://www.google.com/ ...
 ... something went wrong

How can I read the output (stdout?) and do something (like append the iteration number to a text file) when the " ... something went wrong" string appears?

I found that I could do this:

Code:

exec >> log.txt

And from there I could just do some sort of grep thing, but I'd like to see the output while the process is running too.

Code:

#!/bin/bash

noExtResrc="Records with no external resources: "

for ((record = 1; record <= 1642; record += 1))
do
	# grep record page for occurance of '&FF=' to determine number of external resources ($numResrcs)
	numResrcs=$(curl -s `printf "http://66.213.57.243:85/record=b100%04d" $record` | grep -c '&FF=')
	
	# if number of resources is > 0
	if [ $numResrcs -gt 0 ]
	then
		printf "RECORD %d\n\n" $record

		for ((i = 1; i <= $numResrcs; i+= 1))
		do
			resrcName=`printf "b100%04d-%02d-" $record $i`$(curl -s `printf "http://66.213.57.243:85/record=b100%04d" $record` | grep '&FF=' | awk -F\> '{print $2}' | awk -F\< '{print $1}' | head -n $i | tail -n 1 | sed 's/[ \t]*$//')
			echo "EXTERNAL RESOURCE ($i/$numResrcs)"
			python /usr/bin/webkit2png -F -W 1024 -H 800 -o "$resrcName" `printf "http://66.213.57.243:85/search~S0?/.b100%04d/.b100%04d/1,1,1,B/l856~b100%04d&FF=&1,0,,%d,0" $record $record $record $i`
			printf "\n"
		done
		printf "\n\n"
	elif [ $numResrcs -eq 0 ]
	then
		printf "RECORD %d HAS NO EXTERNAL RESOURCE\n\n\n\n" $record
		noExtResrc="$noExtResrc $record, "
	fi
done

echo $( echo $noExtResrc | sed 's/\(.*\)./\1/')

chrism01 · 09-01-2011, 12:09 AM

If you want to be able to log stuff and see it on the reminal at the same time, use tee http://linux.die.net/man/1/tee.
Note also the by default, normal out put goes to chan 1 (stdout), errors go to chan 2 (stderr).
FYI, stdin = chan 0

This is why you'll often see

Code:

cmd 1>cmd.log 2>&1

# OR
cmd >cmd.log 2>&1

Note 1st '1' is optional, as that's default if not specified.

Separate logs

Code:

cmd 1>cmd.log 2>cmd.err

Apologies if you know this stuff already

grail · 09-01-2011, 12:56 AM

I am a bit confused as I read it that you are trying to get a count of how many error?? Yes?

flackend · 09-01-2011, 01:53 AM

Thank you!