bash: redirect stderr to file including name of file with the error

SilversleevesX · 07-26-2010, 05:55 AM

First, my script:

Code:

meow="Wonderful Gentlemen"
for file in $(ls *jpg)
do
	query=$(exiv2 -g Iptc.Application2.FixtureId $file)
	jobid=$(echo $query | cut -d" " -f4-)
	echo -e "Read $file. Next!"
	if [ "$jobid" = "$meow" ]; then echo "$file">>frenchgents;fi
done

Now the command I've been using to execute it:

Code:

frenchgentsfinder.sh 2>>frencherrors.TX

Now the question:
It's kind of pointless imo for the types of errors that Exiv2 reports on to be written to a text file without some reference to the name of the file that prompted the error message in the first place. Is there a way to have bash identify the file that prompts the error and writes its name to the same file as the error (in my case, frencherrors.TX)?

I've tried a painfully simple syntax that does something identical to a 2>&1 'suffix", namely frenchgentsfinder.sh 2 $file>>frencherrors.TX. It makes sense to me as written, but I'd like to know why I'm getting nothing on screen and everything directed to the file when what I want to see in the latter are the filenames causing the errors along with the text of the errors.

Is there another level one has to bore down into before they can garner this kind of output? If so, what is it and how does one invoke it in bash?

BZT

grail · 07-26-2010, 06:15 AM

I would probably suggest looking at the exit code emitted from the offending command.
When you run all commands they generally have 2, can have more, exit types and in linux it is zero for success and anything else, often 1, for errors.

The exit value is stored in the special variable '?' and like any variable in bash it is retrieved, $?

Now the other problem you can sometimes face is if it is simply a warning which still does not give you the desired result of running the command
but as it is not an error it may still say it finished successfully. This one I am not so sure how to go about, but that should get you started.

SilversleevesX · 07-26-2010, 06:23 AM

Quote:

Originally Posted by grail

I would probably suggest looking at the exit code emitted from the offending command.
When you run all commands they generally have 2, can have more, exit types and in linux it is zero for success and anything else, often 1, for errors.

The exit value is stored in the special variable '?' and like any variable in bash it is retrieved, $?

Now the other problem you can sometimes face is if it is simply a warning which still does not give you the desired result of running the command
but as it is not an error it may still say it finished successfully. This one I am not so sure how to go about, but that should get you started.

I just checked the man page for Exiv2 -- no exit codes for the program or any of its commands/routines. I see what you mean, though: it's not bash taking offense at the style or condition of file geewiz.jpg internally or otherwise: it's Exiv2 that's finding the metadata-specific flaw in it and reporting the error. Maybe I can find out from the author or his Mac/Win code maintainer just what option or flag to use to get it to name the file causing the error and whether or not a redirect would get both those details into a text file.

Thanks for the nudge.

BZT

ghostdog74 · 07-26-2010, 06:43 AM

[QUOTE=SilversleevesX;4045547]First, my script:

Code:

meow="Wonderful Gentlemen"
for file in $(ls *jpg)
do
	query=$(exiv2 -g Iptc.Application2.FixtureId $file)
	jobid=$(echo $query | cut -d" " -f4-)
	echo -e "Read $file. Next!"
	if [ "$jobid" = "$meow" ]; then echo "$file">>frenchgents;fi
done

[/code]

useless use of ls. Use shell expansion. Always quote your variables

Code:

for file in *.jpg
do 
  query=$(exiv2 -g Iptc.Application2.FixtureId "$file")
  ...
done

grail · 07-26-2010, 08:11 AM

I am guessing here, but another direction, as exit code not an option, could be that you write your own little error catcher and if the returned info
is not to your liking then create your own output to include the file name

SilversleevesX · 07-26-2010, 04:58 PM

I took one more look at what options were available in Exiv2, and found a -v (verbose) option. Using the command line and assigning the name of a random JPEG (downloaded in the last 24 hours) to a generic "file" variable, the return on an exiv2 -vg Iptc.Application2.FixtureId $file command I tried gave me this:

Code:

File 1/1: gae71-7147-152-006flas.jpg Iptc.Application2.FixtureId String 17 Asian Impressions

I thought, "That looks promising." A few simple "cut" commands later and I had

Code:

gae71-7147-152-006flas.jpg

which was the value of a variable I called nom.

So writing an error-catcher into the script, as I see it from here, means having a routine wait until some value of nom, as a file in the directory being looped through, causes Exiv2 to return an error; get the full text of that error and echo it to the text file in the form "$nom:<space>$appfileerror" or something very similar to that.

Am I right so far?

BZT

tonyfreeman · 07-26-2010, 08:47 PM

How about something like this:

Code:

meow="Wonderful Gentlemen"
for file in $(ls *jpg)
do
        (
	query=$(exiv2 -g Iptc.Application2.FixtureId $file)
	jobid=$(echo $query | cut -d" " -f4-)
	echo -e "Read $file. Next!"
	if [[ "$jobid" = "$meow" ]]
        then 
               echo "$file" >> frenchgents.txt
        fi
        ) 2>$file.error
        test -s $file.error || rm $file.error
done

This wraps all your output into a single exit point so that you can redirect ALL stderr and/or ALL stdout to separate files.

An empty $file.error will be created no matter what, so test if it has a size greater than zero and remove it if it does not.

grail · 07-26-2010, 09:40 PM

Quote:

A few simple "cut" commands later

I am hoping this translates to only one cut command ... lol

I think you are on the right track but have a good look at tony's suggestion as this is a good way to go

SilversleevesX · 07-27-2010, 02:15 AM

Quote:

Originally Posted by tonyfreeman

How about something like this:

Code:

meow="Wonderful Gentlemen"
for file in $(ls *jpg)
do
        (
    query=$(exiv2 -g Iptc.Application2.FixtureId $file)
    jobid=$(echo $query | cut -d" " -f4-)
    echo -e "Read $file. Next!"
    if [[ "$jobid" = "$meow" ]]
        then 
               echo "$file" >> frenchgents.txt
        fi
        ) 2>$file.error
        test -s $file.error || rm $file.error
done

This wraps all your output into a single exit point so that you can redirect ALL stderr and/or ALL stdout to separate files.

An empty $file.error will be created no matter what, so test if it has a size greater than zero and remove it if it does not.

I like, I like. Except for one thing. I think I may have vaguely hinted at it in a few previous posts in this thread: now I'll do everyone the favor of coming out and saying it.

So to not be sitting around waiting for a single or nested loop to finish executing, and so I can tell that a script doesn't have all that many errors (that it's hiding because, dummy me, I told it to), I often write a "Working on file X" echo line into the primary loop, like this one:

Quote:

Originally Posted by From 'fullcredit.sh'

richie=`basename "$joanie"`
echo -e "Working on file $richie now."

I did this with this one, too:

Code:

echo -e "Read $file. Next!"

So stout to a file, as I've seen with some of my "mis-fires" already, would create, not an empty $file.error document, but one with a corresponding "Read file getabashbook.jpg. Next!" kind of line unless I wedge that ech in between the "test" command and the "done" in that for loop. I'll try it and see what results I get.

BZT

SilversleevesX · 07-27-2010, 03:05 AM

It runs a lot quicker than the more rudimentary for/do/done loop does. I watched as the .error files came and went in my file navigator, and then looked up the -s option on the man page for test. Deleting any that had a file size greater than zero contradicted my notion of seeing what error went with what file (can't do that if they're gone, can you?). So I tweaked tonyfreeman's code just a wee bit and dropped the rm, coming up with this (which I still have to test myself, speaking of tests):

Code:

if [[ "$jobid" = "$meow" ]]
        then 
               echo "$file" >> frenchgents.txt
        fi
        ) 2>error/$file.error
       
        echo -e "Read $file. Next!

Instead of identifiable .error files that are here one moment and gone the next (and if I'm looking at a terminal window popped out to max screen dimensions, never seen at all), now I should have a subfolder which, as tony pointed out, will have an .error file for every JPEG in the parent folder but will also show the size if I run a separate test command in another kind of loop, maybe starting off with an "ls *.error | wc -l" command to see how many there are and then testing for byte length and viewable content. Any that are indeed absolutely zero-length I can get rid of, no problem, and any that stick around will show errors I can fix, with the benefit of being named for the file that Exiv2 thinks is a "little off spec" for whatever reason.

BZT

grail · 07-27-2010, 03:15 AM

Quote:

Deleting any that had a file size greater than zero pretty much goes against my idea of seeing what error goes with what file

Code:

test -s $file.error || rm $file.error

So I am not sure these two things are equal?? The code says, if the file does not exist or have a size greater than zero to remove the file.
This means that any file that does have an error recorded inside it will remain.

SilversleevesX · 07-27-2010, 03:06 PM

Quote:

Originally Posted by grail

Code:

test -s $file.error || rm $file.error

So I am not sure these two things are equal?? The code says, if the file does not exist or have a size greater than zero to remove the file.
This means that any file that does have an error recorded inside it will remain.

Suits me fine. I was able to fix one by reading the .error file, passing out all the meta tags of the original JPEG to a text file & going back & deleting the ones having to do with the camera with which the picture was taken one by one. After that, another "print all" command to Exiv2 showed the error was gone. The error, by the way, had the words "Directory" and "Sony" in it, which gave me a hint as to which EXIF tags had to go to "cure" the file.

BZT