stderr and piping (not as simple as 2>&1 - so what do I do?)
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
stderr and piping (not as simple as 2>&1 - so what do I do?)
OK, so I'm trying to clean up the output of my ZFS backup script a bit by piping the output through perl. Here's a rough example of what I've got:
Code:
... muchos code here ...
($LAST = name of the last snapshot, $CURR = the current. The rest speaks for itself if you know ZFS - if not,
doesn't really matter, except that it transfers data via stdout/stdin.)
zfs send -R -I $LAST tank@$CURR | zfs recv -Fvd slave 2>&1 | perl -ne 'if (/receiving \w+ stream of (.*?) into/) {
$dataset = $1; $dataset =~ s/@.*$//g;
print ">>> Backing up $dataset\n";
}
elsif (/received 312B stream/) {
print " no changes\n";
}
elsif (/received \w+ stream in/) {
s/^/ /; print;
}
### Here's the problem:
elsif (/WARNING: could not send ([^@]+)/) {
# print ">>> Skipping $1\n";
}
else {
print;
}'
So, the first three cases prettify the output; the last case does nothing... because zfs send (the *first* command in the pipe) sends it to stderr. I can't use 2>&1 or it becomes redirected into the backup stream, corrupting it.
Is there a simple way to delete that "WARNING:", or replace it with another message? Thanks in advance.
(BTW, the reason I want to replace the warning is that I do a pool-wide snapshot, then delete the ones for /var/tmp, /var/crash etc., and then "send" the entire pool; thus it warns me that the snapshots have been deleted, when in reality, that's exactly what I wanted. This seems to be the easiest way to deal with pool-wide backups and excludes, so what the heck.)
Here's what I get right now:
Code:
>> Importing pool
>> Taking snapshots
>> Cleaning out excluded snapshots
>> Starting backup...
WARNING: could not send tank/tmp@backup-20090910-2007: does not exist
>>> Backing up tank
no changes
>>> Backing up tank/media
no changes
>>> Backing up tank/var
WARNING: could not send tank/var/tmp@backup-20090910-2007: does not exist
received 138KB stream in 1 seconds (138KB/sec)
>>> Backing up tank/var/log
WARNING: could not send tank/var/crash@backup-20090910-2007: does not exist
received 194KB stream in 1 seconds (194KB/sec)
>>> Backing up tank/root
WARNING: could not send tank/usr/obj@backup-20090910-2007: does not exist
WARNING: could not send tank/usr/ports/distfiles@backup-20090910-2007: does not exist
received 144KB stream in 1 seconds (144KB/sec)
>>> Backing up tank/usr
received 42.1KB stream in 1 seconds (42.1KB/sec)
>>> Backing up tank/usr/src_r196905
no changes
>>> Backing up tank/usr/ports
no changes
>>> Backing up tank/export
no changes
>>> Backing up tank/usr/src
no changes
>> Exporting pool
To save someone else puzzling over this, although the tags include bash and not PERL, the question is about the PERL script embedded in the bash script.
To save someone else puzzling over this, although the tags include bash and not PERL, the question is about the PERL script embedded in the bash script.
Well, not really. The problem is in the shell piping, not in the script processing it - that is mostly irrelevant.
Hmm yes, could work, thanks. However... that'd also remove actual errors (ones I very much want to see). What I'd prefer is to be able to "grep -v" stderr or something to that effect, before it's printed and the stream piped on to zfs recv.
Gah! I feel like I'm being obtuse but I think I've finally got the requirement ...
The first command in the pipe can produce several messages on stderr but one (WARNING: could not send ...) is not an error message in this situation. You're OK that the others go to stderr but when this one appears you want to stop it going the same place as the others and instead to write an informative message to the log file.
Gah! I feel like I'm being obtuse but I think I've finally got the requirement ...
The first command in the pipe can produce several messages on stderr but one (WARNING: could not send ...) is not an error message in this situation. You're OK that the others go to stderr but when this one appears you want to stop it going the same place as the others and instead to write an informative message to the log file.
Is that right?
Yeah, except that after running it a few times I've decided to rid the warning one altogether, rather than rewrite it (it shows up out of order and I already know what it says every time, so it's useless). Shouldn't really make a difference in the solution - although it might be possible to redirect it to /dev/null somehow.
Mmm ... assuming there's a lot of data passing through stdout on the pipeline the obvious solution of filtering through sed or awk is not appealing. How about filtering stderr from the first command in the pipeline through a co-process? Are you OK with that general idea?
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
