LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 07-26-2010, 06:55 AM   #1
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Rep: Reputation: 15
bash: redirect stderr to file including name of file with the error


First, my script:
Code:
meow="Wonderful Gentlemen"
for file in $(ls *jpg)
do
	query=$(exiv2 -g Iptc.Application2.FixtureId $file)
	jobid=$(echo $query | cut -d" " -f4-)
	echo -e "Read $file. Next!"
	if [ "$jobid" = "$meow" ]; then echo "$file">>frenchgents;fi
done
Now the command I've been using to execute it:
Code:
frenchgentsfinder.sh 2>>frencherrors.TX
Now the question:
It's kind of pointless imo for the types of errors that Exiv2 reports on to be written to a text file without some reference to the name of the file that prompted the error message in the first place. Is there a way to have bash identify the file that prompts the error and writes its name to the same file as the error (in my case, frencherrors.TX)?

I've tried a painfully simple syntax that does something identical to a 2>&1 'suffix", namely frenchgentsfinder.sh 2 $file>>frencherrors.TX. It makes sense to me as written, but I'd like to know why I'm getting nothing on screen and everything directed to the file when what I want to see in the latter are the filenames causing the errors along with the text of the errors.

Is there another level one has to bore down into before they can garner this kind of output? If so, what is it and how does one invoke it in bash?

BZT
 
Old 07-26-2010, 07:15 AM   #2
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,698

Rep: Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988
I would probably suggest looking at the exit code emitted from the offending command.
When you run all commands they generally have 2, can have more, exit types and in linux it is zero for success and anything else, often 1, for errors.

The exit value is stored in the special variable '?' and like any variable in bash it is retrieved, $?

Now the other problem you can sometimes face is if it is simply a warning which still does not give you the desired result of running the command
but as it is not an error it may still say it finished successfully. This one I am not so sure how to go about, but that should get you started.
 
1 members found this post helpful.
Old 07-26-2010, 07:23 AM   #3
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Original Poster
Rep: Reputation: 15
Cool No exit codes, maybe a seldom-used O or flag... see the author!

Quote:
Originally Posted by grail View Post
I would probably suggest looking at the exit code emitted from the offending command.
When you run all commands they generally have 2, can have more, exit types and in linux it is zero for success and anything else, often 1, for errors.

The exit value is stored in the special variable '?' and like any variable in bash it is retrieved, $?

Now the other problem you can sometimes face is if it is simply a warning which still does not give you the desired result of running the command
but as it is not an error it may still say it finished successfully. This one I am not so sure how to go about, but that should get you started.
I just checked the man page for Exiv2 -- no exit codes for the program or any of its commands/routines. I see what you mean, though: it's not bash taking offense at the style or condition of file geewiz.jpg internally or otherwise: it's Exiv2 that's finding the metadata-specific flaw in it and reporting the error. Maybe I can find out from the author or his Mac/Win code maintainer just what option or flag to use to get it to name the file causing the error and whether or not a redirect would get both those details into a text file.

Thanks for the nudge.

BZT
 
Old 07-26-2010, 07:43 AM   #4
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
[QUOTE=SilversleevesX;4045547]First, my script:
Code:
meow="Wonderful Gentlemen"
for file in $(ls *jpg)
do
	query=$(exiv2 -g Iptc.Application2.FixtureId $file)
	jobid=$(echo $query | cut -d" " -f4-)
	echo -e "Read $file. Next!"
	if [ "$jobid" = "$meow" ]; then echo "$file">>frenchgents;fi
done
[/code]

useless use of ls. Use shell expansion. Always quote your variables

Code:
for file in *.jpg
do 
  query=$(exiv2 -g Iptc.Application2.FixtureId "$file")
  ...
done
 
1 members found this post helpful.
Old 07-26-2010, 09:11 AM   #5
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,698

Rep: Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988
I am guessing here, but another direction, as exit code not an option, could be that you write your own little error catcher and if the returned info
is not to your liking then create your own output to include the file name
 
1 members found this post helpful.
Old 07-26-2010, 05:58 PM   #6
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Original Poster
Rep: Reputation: 15
I took one more look at what options were available in Exiv2, and found a -v (verbose) option. Using the command line and assigning the name of a random JPEG (downloaded in the last 24 hours) to a generic "file" variable, the return on an exiv2 -vg Iptc.Application2.FixtureId $file command I tried gave me this:
Code:
File 1/1: gae71-7147-152-006flas.jpg Iptc.Application2.FixtureId String 17 Asian Impressions
I thought, "That looks promising." A few simple "cut" commands later and I had
Code:
gae71-7147-152-006flas.jpg
which was the value of a variable I called nom.

So writing an error-catcher into the script, as I see it from here, means having a routine wait until some value of nom, as a file in the directory being looped through, causes Exiv2 to return an error; get the full text of that error and echo it to the text file in the form "$nom:<space>$appfileerror" or something very similar to that.

Am I right so far?

BZT

Last edited by SilversleevesX; 07-26-2010 at 06:00 PM.
 
Old 07-26-2010, 09:47 PM   #7
tonyfreeman
Member
 
Registered: Sep 2003
Location: Fort worth, TX
Distribution: Debian testing 64bit at home, EL5 32/64bit at work.
Posts: 187

Rep: Reputation: 30
wrap all output into single exit point with parentheticals

How about something like this:

Code:
meow="Wonderful Gentlemen"
for file in $(ls *jpg)
do
        (
	query=$(exiv2 -g Iptc.Application2.FixtureId $file)
	jobid=$(echo $query | cut -d" " -f4-)
	echo -e "Read $file. Next!"
	if [[ "$jobid" = "$meow" ]]
        then 
               echo "$file" >> frenchgents.txt
        fi
        ) 2>$file.error
        test -s $file.error || rm $file.error
done
This wraps all your output into a single exit point so that you can redirect ALL stderr and/or ALL stdout to separate files.

An empty $file.error will be created no matter what, so test if it has a size greater than zero and remove it if it does not.
 
1 members found this post helpful.
Old 07-26-2010, 10:40 PM   #8
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,698

Rep: Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988
Quote:
A few simple "cut" commands later
I am hoping this translates to only one cut command ... lol

I think you are on the right track but have a good look at tony's suggestion as this is a good way to go
 
1 members found this post helpful.
Old 07-27-2010, 03:15 AM   #9
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by tonyfreeman View Post
How about something like this:

Code:
meow="Wonderful Gentlemen"
for file in $(ls *jpg)
do
        (
    query=$(exiv2 -g Iptc.Application2.FixtureId $file)
    jobid=$(echo $query | cut -d" " -f4-)
    echo -e "Read $file. Next!"
    if [[ "$jobid" = "$meow" ]]
        then 
               echo "$file" >> frenchgents.txt
        fi
        ) 2>$file.error
        test -s $file.error || rm $file.error
done
This wraps all your output into a single exit point so that you can redirect ALL stderr and/or ALL stdout to separate files.

An empty $file.error will be created no matter what, so test if it has a size greater than zero and remove it if it does not.
I like, I like. Except for one thing. I think I may have vaguely hinted at it in a few previous posts in this thread: now I'll do everyone the favor of coming out and saying it.

So to not be sitting around waiting for a single or nested loop to finish executing, and so I can tell that a script doesn't have all that many errors (that it's hiding because, dummy me, I told it to), I often write a "Working on file X" echo line into the primary loop, like this one:
Quote:
Originally Posted by From 'fullcredit.sh'
richie=`basename "$joanie"`
echo -e "Working on file $richie now."
I did this with this one, too:
Code:
echo -e "Read $file. Next!"
So stout to a file, as I've seen with some of my "mis-fires" already, would create, not an empty $file.error document, but one with a corresponding "Read file getabashbook.jpg. Next!" kind of line unless I wedge that ech in between the "test" command and the "done" in that for loop. I'll try it and see what results I get.

BZT

Last edited by SilversleevesX; 07-27-2010 at 03:45 AM. Reason: Re-read tony's post -- understand it better. Mine needed to reflect that.
 
Old 07-27-2010, 04:05 AM   #10
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Original Poster
Rep: Reputation: 15
It runs a lot quicker than the more rudimentary for/do/done loop does. I watched as the .error files came and went in my file navigator, and then looked up the -s option on the man page for test. Deleting any that had a file size greater than zero contradicted my notion of seeing what error went with what file (can't do that if they're gone, can you?). So I tweaked tonyfreeman's code just a wee bit and dropped the rm, coming up with this (which I still have to test myself, speaking of tests):
Code:
if [[ "$jobid" = "$meow" ]]
        then 
               echo "$file" >> frenchgents.txt
        fi
        ) 2>error/$file.error
       
        echo -e "Read $file. Next!
Instead of identifiable .error files that are here one moment and gone the next (and if I'm looking at a terminal window popped out to max screen dimensions, never seen at all), now I should have a subfolder which, as tony pointed out, will have an .error file for every JPEG in the parent folder but will also show the size if I run a separate test command in another kind of loop, maybe starting off with an "ls *.error | wc -l" command to see how many there are and then testing for byte length and viewable content. Any that are indeed absolutely zero-length I can get rid of, no problem, and any that stick around will show errors I can fix, with the benefit of being named for the file that Exiv2 thinks is a "little off spec" for whatever reason.

BZT

Last edited by SilversleevesX; 07-27-2010 at 05:22 AM.
 
Old 07-27-2010, 04:15 AM   #11
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,698

Rep: Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988
Quote:
Deleting any that had a file size greater than zero pretty much goes against my idea of seeing what error goes with what file
Code:
test -s $file.error || rm $file.error
So I am not sure these two things are equal?? The code says, if the file does not exist or have a size greater than zero to remove the file.
This means that any file that does have an error recorded inside it will remain.
 
Old 07-27-2010, 04:06 PM   #12
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by grail View Post
Code:
test -s $file.error || rm $file.error
So I am not sure these two things are equal?? The code says, if the file does not exist or have a size greater than zero to remove the file.
This means that any file that does have an error recorded inside it will remain.
Suits me fine. I was able to fix one by reading the .error file, passing out all the meta tags of the original JPEG to a text file & going back & deleting the ones having to do with the camera with which the picture was taken one by one. After that, another "print all" command to Exiv2 showed the error was gone. The error, by the way, had the words "Directory" and "Sony" in it, which gave me a hint as to which EXIF tags had to go to "cure" the file.

BZT
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Cannot redirect all stdout and stderr into file hawkgao Programming 2 05-12-2009 10:14 AM
redirect stderr to file with a prefix? JPMallory Programming 2 12-26-2007 04:10 PM
bash stderr redirect question code-breaker Linux - Software 1 08-06-2006 05:44 AM
Redirecting error messages to stderr and file in Bash lowpro2k3 Programming 1 04-14-2005 07:47 PM
how do i redirect stderr to a file? nodger Programming 5 11-28-2004 12:22 PM


All times are GMT -5. The time now is 11:06 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration