[SOLVED] How to grep -v and omit all results matching 'x' in column 'y'

zongbot · 09-12-2011, 09:05 AM

Hello,

I'm trying to find out if such a thing is possible.

Scenario: in /var/log/ I have maillog files dating back to 2006. Example:

Code:

-rw-r--r--  1 root    wheel   -     79248 Sep  1  2006 maillog.1939
-rw-r--r--  1 root    wheel   -     49774 Sep  1  2006 maillog.1938

I only need the logs for 2011, which doesn't have a year in the output:

Code:

-rw-r--r--  1 root    wheel   -    322481 Sep  1 22:00 maillog.45588
-rw-r--r--  1 root    wheel   -    300857 Sep  1 21:00 maillog.45587

I've got this much so far:

Code:

ls -lt | grep maillog | grep 'Sep  1'

Question: In the above command could I add "| grep -v ..."

I would like to replace "..." with a function that will omit from the output, all entries that match 2006 from the 9th column.

Many thanks in advance comrades!

thesnow · 09-12-2011, 09:12 AM

Code:

grep -v ' 2006 '

SL00b · 09-12-2011, 09:12 AM

The ones you're after don't have a year, they have a time... and the unique thing about the timestamp is, they have a colon. So try:

Code:

ls -lt | grep maillog | grep :

druuna · 09-12-2011, 09:15 AM

Hi,

If you look at the ls man page you see this option: --time-style=STYLE. If you use long-iso for STYLE you get a cinsistent date output.

Example:

Code:

ls -l
drwxr-x---  3 druuna internet    4096 Feb 28  2011 Documenten
drwxr-x---  3 druuna internet    4096 Nov  9  2010 Nikon_D700
-rw-r-----  1 druuna internet      60 Sep  9 19:15 blaat
-rwxr-x---  1 druuna internet    6070 Aug 30 21:49 cpu.info.org.sh
-rwxr-x---  1 druuna internet    5734 Sep  2 16:51 cpu.info.sh
-rw-r-----  1 druuna internet    1225 Aug 30 13:18 input
-rw-r-----  1 druuna internet    3576 Aug 30 13:20 input2

ls -l --time-style=long-iso
drwxr-x---  3 druuna internet    4096 2011-02-28 13:45 Documenten
drwxr-x---  3 druuna internet    4096 2010-11-09 14:02 Nikon_D700
-rw-r-----  1 druuna internet      60 2011-09-09 19:15 blaat
-rwxr-x---  1 druuna internet    6070 2011-08-30 21:49 cpu.info.org.sh
-rwxr-x---  1 druuna internet    5734 2011-09-02 16:51 cpu.info.sh
-rw-r-----  1 druuna internet    1225 2011-08-30 13:18 input
-rw-r-----  1 druuna internet    3576 2011-08-30 13:20 input2

Hope this helps.

colucix · 09-12-2011, 09:18 AM

Quote:

Originally Posted by zongbot

I only need the logs for 2011, which doesn't have a year in the output:

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 !~ /:/'

Quote:

Originally Posted by zongbot

I would like to replace "..." with a function that will omit from the output, all entries that match 2006 from the 9th column.

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 !~ /2006/'

zongbot · 09-12-2011, 10:26 AM

Quote:

Originally Posted by thesnow

Code:

grep -v ' 2006 '

That would work, except I have logs from 2007, 2008, 2009, etc. I could also grep -v those, but I was trying to find a tidier solution.

---------- Post added 09-12-11 at 11:27 AM ----------

Quote:

Originally Posted by SL00b

The ones you're after don't have a year, they have a time... and the unique thing about the timestamp is, they have a colon. So try:

Code:

ls -lt | grep maillog | grep :

That works perfectly in this instance. But I was hoping to actually omit output based on a value of a column.

zongbot · 09-12-2011, 10:30 AM

Quote:

Originally Posted by colucix

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 !~ /:/'

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 !~ /2006/'

Both codes ended up giving me a similar output as my original command. But I'm curious, what is the $0 supposed to do?

The awk looks promising, but the output doesn't omit based on the year:

Code:

# ls -lt | awk '$0 ~ /maillog/ && $9 !~ /:/'
-rw-r--r--  1 root    wheel   -    207543 Mar 14  2011 maillog.41482
-rw-r--r--  1 root    wheel   -    143629 Mar 14  2011 maillog.41481
-snip-
-rw-r--r--  1 root    wheel   -    194670 Jul 27  2010 maillog.35971
-rw-r--r--  1 root    wheel   -    214481 Jul 27  2010 maillog.35970

colucix · 09-12-2011, 10:45 AM

Quote:

Originally Posted by zongbot

Both codes ended up giving me a similar output as my original command. But I'm curious, what is the $0 supposed to do?

$0 in awk means the whole record (that is a single line of input).

Quote:

Originally Posted by zongbot

The awk looks promising, but the output doesn't omit based on the year:

Yeah, sorry. You were looking for the opposite:

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 ~ /:/'

without the negation operator (the exclamation mark in awk).

druuna · 09-12-2011, 10:49 AM

Never mind, I misread the original question.......

zongbot · 09-12-2011, 11:08 AM

Quote:

Originally Posted by colucix

$0 in awk means the whole record (that is a single line of input).

Yeah, sorry. You were looking for the opposite:

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 ~ /:/'

without the negation operator (the exclamation mark in awk).

Excellent! That works perfectly

If you wouldn't mind terribly, could you please expand on the functions of the tilde, double ampersand, and '/' functions in the awk statement?

colucix · 09-12-2011, 12:47 PM

Quote:

Originally Posted by zongbot

Excellent! That works perfectly

If you wouldn't mind terribly, could you please expand on the functions of the tilde, double ampersand, and '/' functions in the awk statement?

Well... you should read about the basic syntax of awk. Here is some useful links:
http://www.gnu.org/s/gawk/manual/
http://www.grymoire.com/Unix/Awk.html

Two things to take in mind: first awk parses one line at a time (record) and splits the record into fields based on the value of the field separator, FS, which is one or multiple spaces by default. Second, awk applies all the rules to each line where a rule is basically:

Code:

pattern { action }

In the command

Code:

awk '$0 ~ /maillog/ && $9 ~ /:/'

there is only one rule, made of a pattern (if no action is specified the default action is applied, that is the entire record $0 is printed out as is). Regarding the pattern, here it uses regular expressions to match maillog for the entire record and a colon for the 9th field, $9. The ~ operator means "match against a regular expression". The regular expression is embedded between slashes. Finally the && operator is the logical AND.

Translated in real words the command means: if the record matches "maillog" and the 9th field matches ":" print out the record itself. Hope it is clear.

zongbot · 09-12-2011, 02:07 PM

Quote:

Originally Posted by colucix

Translated in real words the command means: if the record matches "maillog" and the 9th field matches ":" print out the record itself. Hope it is clear.

Thank you very much colucix!