LinuxQuestions.org - [SOLVED] How to grep -v and omit all results matching 'x' in column 'y'

- Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)

- - How to grep -v and omit all results matching 'x' in column 'y' (https://www.linuxquestions.org/questions/linux-general-1/how-to-grep-v-and-omit-all-results-matching-x-in-column-y-902578/)

zongbot

09-12-2011 09:05 AM

How to grep -v and omit all results matching 'x' in column 'y'

Hello,

I'm trying to find out if such a thing is possible.

Scenario: in /var/log/ I have maillog files dating back to 2006. Example:

Code:

-rw-r--r--  1 root    wheel  -    79248 Sep  1  2006 maillog.1939

-rw-r--r--  1 root    wheel  -    49774 Sep  1  2006 maillog.1938

I only need the logs for 2011, which doesn't have a year in the output:

Code:

-rw-r--r--  1 root    wheel  -    322481 Sep  1 22:00 maillog.45588

-rw-r--r--  1 root    wheel  -    300857 Sep  1 21:00 maillog.45587

I've got this much so far:

Code:

ls -lt | grep maillog | grep 'Sep 1'

Question: In the above command could I add "| grep -v ..."

I would like to replace "..." with a function that will omit from the output, all entries that match 2006 from the 9th column.

Many thanks in advance comrades!

thesnow

09-12-2011 09:12 AM

Code:

grep -v ' 2006 '

SL00b

09-12-2011 09:12 AM

The ones you're after don't have a year, they have a time... and the unique thing about the timestamp is, they have a colon. So try:

Code:

ls -lt | grep maillog | grep :

druuna

09-12-2011 09:15 AM

Hi,

If you look at the ls man page you see this option: --time-style=STYLE. If you use long-iso for STYLE you get a cinsistent date output.

Example:

Code:

ls -l

drwxr-x---  3 druuna internet    4096 Feb 28  2011 Documenten

drwxr-x---  3 druuna internet    4096 Nov  9  2010 Nikon_D700

-rw-r-----  1 druuna internet      60 Sep  9 19:15 blaat

-rwxr-x---  1 druuna internet    6070 Aug 30 21:49 cpu.info.org.sh

-rwxr-x---  1 druuna internet    5734 Sep  2 16:51 cpu.info.sh

-rw-r-----  1 druuna internet    1225 Aug 30 13:18 input

-rw-r-----  1 druuna internet    3576 Aug 30 13:20 input2



ls -l --time-style=long-iso

drwxr-x---  3 druuna internet    4096 2011-02-28 13:45 Documenten

drwxr-x---  3 druuna internet    4096 2010-11-09 14:02 Nikon_D700

-rw-r-----  1 druuna internet      60 2011-09-09 19:15 blaat

-rwxr-x---  1 druuna internet    6070 2011-08-30 21:49 cpu.info.org.sh

-rwxr-x---  1 druuna internet    5734 2011-09-02 16:51 cpu.info.sh

-rw-r-----  1 druuna internet    1225 2011-08-30 13:18 input

-rw-r-----  1 druuna internet    3576 2011-08-30 13:20 input2

Hope this helps.

colucix

09-12-2011 09:18 AM

Quote:

Originally Posted by zongbot (Post 4469686)

I only need the logs for 2011, which doesn't have a year in the output:

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 !~ /:/'

Quote:

Originally Posted by zongbot (Post 4469686)

I would like to replace "..." with a function that will omit from the output, all entries that match 2006 from the 9th column.

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 !~ /2006/'

zongbot

09-12-2011 10:26 AM

Quote:

Originally Posted by thesnow (Post 4469691)

Code:

grep -v ' 2006 '

That would work, except I have logs from 2007, 2008, 2009, etc. I could also grep -v those, but I was trying to find a tidier solution.

---------- Post added 09-12-11 at 11:27 AM ----------

Quote:

Originally Posted by SL00b (Post 4469692)

The ones you're after don't have a year, they have a time... and the unique thing about the timestamp is, they have a colon. So try:

Code:

ls -lt | grep maillog | grep :

That works perfectly in this instance. But I was hoping to actually omit output based on a value of a column.

zongbot

09-12-2011 10:30 AM

Quote:

Originally Posted by colucix (Post 4469696)

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 !~ /:/'

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 !~ /2006/'

Both codes ended up giving me a similar output as my original command. But I'm curious, what is the $0 supposed to do?

The awk looks promising, but the output doesn't omit based on the year:

Code:

# ls -lt | awk '$0 ~ /maillog/ && $9 !~ /:/'

-rw-r--r--  1 root    wheel  -    207543 Mar 14  2011 maillog.41482

-rw-r--r--  1 root    wheel  -    143629 Mar 14  2011 maillog.41481

-snip-

-rw-r--r--  1 root    wheel  -    194670 Jul 27  2010 maillog.35971

-rw-r--r--  1 root    wheel  -    214481 Jul 27  2010 maillog.35970

colucix

09-12-2011 10:45 AM

Quote:

Originally Posted by zongbot (Post 4469754)

Both codes ended up giving me a similar output as my original command. But I'm curious, what is the $0 supposed to do?

$0 in awk means the whole record (that is a single line of input).

Quote:

Originally Posted by zongbot (Post 4469754)

The awk looks promising, but the output doesn't omit based on the year:

Yeah, sorry. You were looking for the opposite:

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 ~ /:/'

without the negation operator (the exclamation mark in awk).

druuna

09-12-2011 10:49 AM

Never mind, I misread the original question.......

zongbot

09-12-2011 11:08 AM

Quote:

Originally Posted by colucix (Post 4469775)

$0 in awk means the whole record (that is a single line of input).

Yeah, sorry. You were looking for the opposite:

Code:

ls -lt | awk '$0 ~ /maillog/ && $9 ~ /:/'

without the negation operator (the exclamation mark in awk).

Excellent! That works perfectly :)

If you wouldn't mind terribly, could you please expand on the functions of the tilde, double ampersand, and '/' functions in the awk statement?

colucix

09-12-2011 12:47 PM

Quote:

Originally Posted by zongbot (Post 4469798)

Excellent! That works perfectly :)

If you wouldn't mind terribly, could you please expand on the functions of the tilde, double ampersand, and '/' functions in the awk statement?

Well... you should read about the basic syntax of awk. Here is some useful links:
http://www.gnu.org/s/gawk/manual/
http://www.grymoire.com/Unix/Awk.html

Two things to take in mind: first awk parses one line at a time (record) and splits the record into fields based on the value of the field separator, FS, which is one or multiple spaces by default. Second, awk applies all the rules to each line where a rule is basically:

Code:

pattern { action }

In the command

Code:

awk '$0 ~ /maillog/ && $9 ~ /:/'

there is only one rule, made of a pattern (if no action is specified the default action is applied, that is the entire record $0 is printed out as is). Regarding the pattern, here it uses regular expressions to match maillog for the entire record and a colon for the 9th field, $9. The ~ operator means "match against a regular expression". The regular expression is embedded between slashes. Finally the && operator is the logical AND.

Translated in real words the command means: if the record matches "maillog" and the 9th field matches ":" print out the record itself. Hope it is clear.

zongbot

09-12-2011 02:07 PM

Quote:

Originally Posted by colucix (Post 4469897)

Translated in real words the command means: if the record matches "maillog" and the 9th field matches ":" print out the record itself. Hope it is clear.

Thank you very much colucix!

All times are GMT -5. The time now is 07:12 PM.