[SOLVED] tail -f with multiple pipes

Sefyir · 07-01-2015, 10:27 PM

Code:

$ sudo tail -f /var/log/messages | grep LOGPREFIX | awk '{print $11}'

I am watching some iptables logging and want to filter it
I want to look from any strings containing LOGPREFIX and then print the 11th column.
However, when I run the above, it sits there without printing anything. Removing the awk command causes it to display the output. Removing the -f (to actively follow it) causes it to work with the awk command.

Is there a limitation in tail to prevent more then one filtering program when using -f or am I writing it wrong?

My output:

Code:

$ sudo tail -f /var/log/messages | grep LOGPREFIX
Jul  1 20:14:25 HOSTNAME kernel: [1245137.142033] LOGPREFIX IN=eth1 OUT= MAC=macaddress SRC=source_ip DST=dest_ip LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=59154 DF PROTO=TCP SPT=43935 DPT=7778 WINDOW=29200 RES=0x00 SYN URGP=0
Jul  1 20:14:25 HOSTNAME kernel: [1245137.142033] LOGPREFIX IN=eth1 OUT= MAC=macaddress SRC=source_ip DST=dest_ip LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=59154 DF PROTO=TCP SPT=43935 DPT=7778 WINDOW=29200 RES=0x00 SYN URGP=0
...

Code:

$ sudo tail -f /var/log/messages | grep LOGPREFIX | awk '{print $11}'
_ (no output)

Code:

$ sudo tail /var/log/messages | grep LOGPREFIX | awk '{print $11}'
SRC=dest_ip
SRC=dest_ip
...

rknichols · 07-01-2015, 10:29 PM

Your problem is that grep is block-buffering its output because its stdout is not a terminal. Try "grep --line-buffered LOGPREFIX".

syg00 · 07-01-2015, 10:33 PM

Or better yet, get rid of the grep altogether and use awk to do the filtering as well as the print.

Sefyir · 07-01-2015, 10:40 PM

Quote:

Originally Posted by rknichols

Your problem is that grep is block-buffering its output because its stdout is not a terminal. Try "grep --line-buffered LOGPREFIX".

Thank you! This worked perfectly.

Quote:

Or better yet, get rid of the grep altogether and use awk to do the filtering as well as the print.

I'm not familiar with awk, I've only recently started to use it for column filtering. How would I use awk in this situation to remove the need for grep?

syg00 · 07-02-2015, 12:03 AM

Code:

sudo tail /var/log/messages | awk '/LOGPREFIX/ {print $11}'

This is a short form of "$0 ~ /LOGPREFIX/" - does the input record contain the regex (in this case a regex constant). Note is is case sensitive.