LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 08-31-2011, 10:56 PM   #1
flackend
LQ Newbie
 
Registered: Jun 2009
Location: Ohio
Distribution: Ubuntu 9.04
Posts: 10

Rep: Reputation: 0
Question Read bash script output and note iterations with errors


Hi,

I'm writing a bash script that iterates through a series of websites/records and saves screenshots. If the webpage loads I see this output:

Code:
Fetching http://www.google.com/ ...
 ... done
If there's an issue loading the page:

Code:
Fetching http://www.google.com/ ...
 ... something went wrong
How can I read the output (stdout?) and do something (like append the iteration number to a text file) when the " ... something went wrong" string appears?

I found that I could do this:

Code:
exec >> log.txt
And from there I could just do some sort of grep thing, but I'd like to see the output while the process is running too.

Code:
#!/bin/bash

noExtResrc="Records with no external resources: "

for ((record = 1; record <= 1642; record += 1))
do
	# grep record page for occurance of '&FF=' to determine number of external resources ($numResrcs)
	numResrcs=$(curl -s `printf "http://66.213.57.243:85/record=b100%04d" $record` | grep -c '&FF=')
	
	# if number of resources is > 0
	if [ $numResrcs -gt 0 ]
	then
		printf "RECORD %d\n\n" $record

		for ((i = 1; i <= $numResrcs; i+= 1))
		do
			resrcName=`printf "b100%04d-%02d-" $record $i`$(curl -s `printf "http://66.213.57.243:85/record=b100%04d" $record` | grep '&FF=' | awk -F\> '{print $2}' | awk -F\< '{print $1}' | head -n $i | tail -n 1 | sed 's/[ \t]*$//')
			echo "EXTERNAL RESOURCE ($i/$numResrcs)"
			python /usr/bin/webkit2png -F -W 1024 -H 800 -o "$resrcName" `printf "http://66.213.57.243:85/search~S0?/.b100%04d/.b100%04d/1,1,1,B/l856~b100%04d&FF=&1,0,,%d,0" $record $record $record $i`
			printf "\n"
		done
		printf "\n\n"
	elif [ $numResrcs -eq 0 ]
	then
		printf "RECORD %d HAS NO EXTERNAL RESOURCE\n\n\n\n" $record
		noExtResrc="$noExtResrc $record, "
	fi
done

echo $( echo $noExtResrc | sed 's/\(.*\)./\1/')
 
Old 09-01-2011, 12:09 AM   #2
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,242

Rep: Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024
If you want to be able to log stuff and see it on the reminal at the same time, use tee http://linux.die.net/man/1/tee.
Note also the by default, normal out put goes to chan 1 (stdout), errors go to chan 2 (stderr).
FYI, stdin = chan 0

This is why you'll often see
Code:
cmd 1>cmd.log 2>&1

# OR
cmd >cmd.log 2>&1
Note 1st '1' is optional, as that's default if not specified.

Separate logs
Code:
cmd 1>cmd.log 2>cmd.err
Apologies if you know this stuff already
 
1 members found this post helpful.
Old 09-01-2011, 12:56 AM   #3
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,437

Rep: Reputation: 1879Reputation: 1879Reputation: 1879Reputation: 1879Reputation: 1879Reputation: 1879Reputation: 1879Reputation: 1879Reputation: 1879Reputation: 1879Reputation: 1879
I am a bit confused as I read it that you are trying to get a count of how many error?? Yes?
 
1 members found this post helpful.
Old 09-01-2011, 01:53 AM   #4
flackend
LQ Newbie
 
Registered: Jun 2009
Location: Ohio
Distribution: Ubuntu 9.04
Posts: 10

Original Poster
Rep: Reputation: 0
Thank you!
 
  


Reply

Tags
bash, curl, grep, script


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Breaking iteration after a given number of iterations (bash). stf92 Linux - Newbie 8 06-14-2011 09:16 AM
(Bash) Redirect all output from script to all.log and copy of errors to err.log hmsdefender Programming 5 03-05-2010 01:52 PM
Trouble with making a bash script to read in different files and rename output files. rystke Linux - Software 1 05-07-2009 08:00 AM
python read output from seperate script? bendeco13 Programming 1 02-01-2005 10:38 PM
Read the output from a pipe with bash ? fluppi Linux - Software 3 01-13-2004 12:59 PM


All times are GMT -5. The time now is 08:09 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration