LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 06-28-2005, 05:39 PM   #1
stefanlasiewski
Member
 
Registered: Aug 2003
Location: Berkeley, California, USA
Distribution: Red Hat Enterprise Linux, Debian & Ubuntu
Posts: 92

Rep: Reputation: 16
Shell scripting: Print output to logfile, error to logfile & screen


Hey Gang,

I'm looking for a KSH, Bourne or Bash way which will do the following:

- Print the standard output of a command to a logfile
- Print the error of the command to the logfile (So I can see the error in it's context) and to the screen (So I can see the error as it happens).

The unix command 'tee' will let me print content to both STDOUT and to a file, but it doesn't deal with STDERR.

This command will print STDOUT and STDERR to a logfile, and will tell me if there was an error:
Code:
make > make.log 2>&1 || print "There was an error"
But I'm looking for a way to print the actual error to the screen. I guess I'm looking for a 'tee'-like program that can deal with STDERR.

Anyone know how do this with a shell script, unix command, or any other method?

Thank you in advance.
 
Old 06-28-2005, 06:06 PM   #2
jailbait
Guru
 
Registered: Feb 2003
Location: Blue Ridge Mountain
Distribution: Debian Wheezy, Debian Jessie
Posts: 7,476

Rep: Reputation: 156Reputation: 156
"I'm looking for a KSH, Bourne or Bash way which will do the following:

- Print the standard output of a command to a logfile
- Print the error of the command to the logfile (So I can see the error in it's context) and to the screen (So I can see the error as it happens)."

This might work. I haven't tested it.

make 2>&1 | tee make.log

---------------------
Steve Stites
 
Old 06-28-2005, 06:19 PM   #3
stefanlasiewski
Member
 
Registered: Aug 2003
Location: Berkeley, California, USA
Distribution: Red Hat Enterprise Linux, Debian & Ubuntu
Posts: 92

Original Poster
Rep: Reputation: 16
It's close, but not quite what I'm looking for.

That prints the output & error to make.log and to the screen. It's a great way to run a command like this in a single line (very handy!)

make > make.log 2>&1 &
tail -f make.log

I'm actually looking for a way to hide the normal output of a make command (Thousands of good lines), and only print the errors.

Thanks Steve!
 
Old 06-28-2005, 08:51 PM   #4
carl.waldbieser
Member
 
Registered: Jun 2005
Location: Pennsylvania
Distribution: Kubuntu
Posts: 197

Rep: Reputation: 32
Is this what you are looking for?

$ command 2>&1 1> output | tee error

Example (test.py):
Code:
#!/bin/python

import sys

print >> sys.stdout, "Hello, output World!"
print >> sys.stderr, "Hello, error World!"
When run with the above pipeline:
Code:
$ python test.py 2>&1 1> output | tee error
Hello, error World!
$ cat output
Hello, output World!
$ cat error
Hello, error World!
 
Old 06-29-2005, 04:48 AM   #5
Bebo
Member
 
Registered: Jul 2003
Location: Göteborg
Distribution: Arch Linux (current)
Posts: 553

Rep: Reputation: 30
Hello stefanlasiewski,

This is an interesting problem. After searching around a bit I managed to print stderr to the screen as well as stderr together with stdout to a log file. However, the context was lost completely; either the stderr messages were printed on the bottom of the log file, or on top of it.

The solutions to similar problems have been about swapping stderr to stdout so that one is able to tee the stderr (which are now at stdout...). I have found a few people asking the same question as yours elsewhere, but those answering have posted solutions to a similar problem only... The best attempt at a solution is CASE 6 here, but it's not very nice...

I can really see the use of this, so I'll keep looking and trying
 
Old 06-30-2005, 02:35 PM   #6
Bebo
Member
 
Registered: Jul 2003
Location: Göteborg
Distribution: Arch Linux (current)
Posts: 553

Rep: Reputation: 30
Hm, I just noticed that stderr in the example on the website I posted above does ALSO loose its context. Not good. Odd that this (is|seems to be) so difficult.
 
Old 06-30-2005, 05:59 PM   #7
stefanlasiewski
Member
 
Registered: Aug 2003
Location: Berkeley, California, USA
Distribution: Red Hat Enterprise Linux, Debian & Ubuntu
Posts: 92

Original Poster
Rep: Reputation: 16
Yeah, isn't it wierd? I've never been able to do this, yet it seems like it would be a very desired feature. I often have commands which spit out thousands of lines of good output which I don't want to see. Burried in the output are some errors... I want to see those errors as they occur (To determine if I want to ignore the error or stop the process), but I also want to see the errors in context, next to the commands that produced the error.

'rsync --ignore-errors' and 'make --keep-going' being two examples that I see alot.

I think I also need to come up with a clearer example.
 
Old 06-30-2005, 06:08 PM   #8
stefanlasiewski
Member
 
Registered: Aug 2003
Location: Berkeley, California, USA
Distribution: Red Hat Enterprise Linux, Debian & Ubuntu
Posts: 92

Original Poster
Rep: Reputation: 16
Closer... much closer.

Quote:
Originally posted by carl.waldbieser
Code:
$ python test.py 2>&1 1> output | tee error
Hello, error World!
$ cat output
Hello, output World!
$ cat error
Hello, error World!
Interestingly, when I try this in ZSH, I got different output. The STDOUT & STDERR are printed to the screen and to the file 'error'. The file 'output' only contains the output, no error... odd.

Code:
./testerror.py 2>&1 1> output | tee error
Hello, error World!
Hello, output World!
% cat output
Hello, output World!
% cat error
Hello, error World!
Hello, output World!
Here's what happens with BASH. Now, if I could get the STDERR and STDOUT to both print into the file 'output', I'd be set!

Code:
$ ./testerror.py 2>&1 1> output | tee error
Hello, error World!
$ cat output
Hello, output World!
$ cat error
Hello, error World!
 
Old 06-30-2005, 08:39 PM   #9
carl.waldbieser
Member
 
Registered: Jun 2005
Location: Pennsylvania
Distribution: Kubuntu
Posts: 197

Rep: Reputation: 32
I think this will do what you want in bash:

Code:
$ python test.py 2>&1 1>> output | tee -a output
I don't know much about zsh, but the above works as follows:

1) The command generates the two streams (output and error streams).
2) File descriptor 2 (default points to /dev/stderr) is redirected to the same file as file descriptor 1 (by default points to /dev/stdout).
3) File descriptor 1 is redirected to append to the file, "output".
4) data on /dev/stdout (from descriptor 2) is piped to tee.
5) tee prints the stream to the console and (by using the -a option) appends to the file, "output".

Not sure how well this will work with a large sample. I don't know if there is any kind of timing issue with the pipeline.
 
Old 06-30-2005, 10:36 PM   #10
perfect_circle
Senior Member
 
Registered: Oct 2004
Location: Athens, Greece
Distribution: Slackware, arch
Posts: 1,783

Rep: Reputation: 52
Quote:
Originally posted by carl.waldbieser
I think this will do what you want in bash:

Code:
$ python test.py 2>&1 1>> output | tee -a output
I don't know much about zsh, but the above works as follows:

1) The command generates the two streams (output and error streams).
2) File descriptor 2 (default points to /dev/stderr) is redirected to the same file as file descriptor 1 (by default points to /dev/stdout).
3) File descriptor 1 is redirected to append to the file, "output".
4) data on /dev/stdout (from descriptor 2) is piped to tee.
5) tee prints the stream to the console and (by using the -a option) appends to the file, "output".

Not sure how well this will work with a large sample. I don't know if there is any kind of timing issue with the pipeline.
Code:
skalkoto@darkstar:~/src$ cat test.py
#!/usr/bin/python

import sys

print >> sys.stdout, "Hello, output World!1"
print >> sys.stderr, "Hello, error World!1"
print >> sys.stdout, "Hello, output World!2"
print >> sys.stdout, "Hello, output World!3"
print >> sys.stderr, "Hello, error World!2"
skalkoto@darkstar:~/src$ python test.py 2>&1 1>> output | tee -a output
Hello, error World!1
Hello, error World!2
skalkoto@darkstar:~/src$ cat output
Hello, output World!1
Hello, output World!2
Hello, output World!3
Hello, error World!1
Hello, error World!2
skalkoto@darkstar:~/src$
it's the same as piping stderr to a while read loop, save it to a variable, and then append it to the file and print it on the screen

Last edited by perfect_circle; 06-30-2005 at 10:42 PM.
 
Old 07-01-2005, 06:31 AM   #11
Bebo
Member
 
Registered: Jul 2003
Location: Göteborg
Distribution: Arch Linux (current)
Posts: 553

Rep: Reputation: 30
carl.waldbieser and perfect_circle: Yes, this is exactly what I mean when I say that stderr looses context. The errors are put at the bottom of the log file, not where they occur. That is also the best I've managed myself using only redirection and tee.

Hm, here is one really ugly and stupid solution:
Code:
cmd 2>&1 | tee log | grep -2 ':'
Since most error messages are formatted like cmd: this is an error message, one can grep for ':' and also output some context ("-2"). Of course, if stdout also contains lots of ':' then this doesn't work...
 
Old 07-04-2005, 02:59 PM   #12
newbie007007
Member
 
Registered: Dec 2004
Location: India
Distribution: Fedora
Posts: 36

Rep: Reputation: 15
"$command 2>&1" what does it mean????

Can someone plz tell me
what does the " 2>&1 " after the "command" do???

what does the letters 2,1,>,& signify???

I am a newbie.i am interested in this discussion.
Thank you.
 
Old 07-04-2005, 04:41 PM   #13
vladmihaisima
Member
 
Registered: Oct 2002
Location: Delft, Netherlands
Distribution: Gentoo
Posts: 196

Rep: Reputation: 33
2 is standard error (stderr)
1 is standard output (stdout)
2>&1 means redirect 2 into a copy of 1

For more information "man bash" search (click '/', the word you want to search and enter) Redirection.
 
Old 07-05-2005, 02:03 AM   #14
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,251

Rep: Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025
Incidentally, 0 is standard input (stdin)
 
Old 07-05-2005, 02:34 AM   #15
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,486

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Quote:
Yes, this is exactly what I mean when I say that stderr looses context. The errors are put at the bottom of the log file, not where they occur.
The problem you are complaining of is probably due to buffering / flushing issues.
Stdout and stderr loose their synchronization when piped to another program, so you need a solution not using a pipe. I think that would be easy to solve with Solaris dtrace.
Under Linux, the closest solution would be customizing strace source code to your needs.

Here's a start:
Code:
strace -s 1024 -f -e trace=write -e write=1,2 make
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Writing shell script with mail < logfile humbletech99 Programming 2 11-22-2005 08:20 AM
io redirection doesn't work? 2>&1 >> logfile.log Thinking Linux - General 5 06-10-2005 04:31 AM
ntfs logfile error on slack10.0 2.6.9 eitch Slackware 3 11-22-2004 01:23 PM
Can I redirect script output to a file without ">> $LOGFILE" at the end of each line davee Linux - General 1 12-19-2003 05:01 AM
fatal server error: cannot move logfile XFree86.0.old Scruff Linux - Software 3 11-02-2003 01:00 PM


All times are GMT -5. The time now is 03:13 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration