[SOLVED] Expand grep command usage

masatheesh · 04-20-2012, 02:41 AM

Hi,

I am using grep as below to extract after two lines from line which has words "SENT: RCPT".

grep -B 2 -i "SENT: RCPT" MailLog.txt

Here I need to check another thing also. That is, the line which has words "SENT: RCPT" should not have word "XYZ".

So I need every two lines before a line which contains "SENT: RCPT" and doesnt contain "xyz".

Please help me.

fukawi1 · 04-20-2012, 02:52 AM

It can probably be done with regex, but well, I suck at regex, so I would pipe grep to grep.

Code:

grep -B 2 -i "SENT: RCPT" MailLog.txt | grep -v "xyz"

masatheesh · 04-20-2012, 03:40 AM

grep -B 2 -i "SENT: RCPT" MailLog.txt . If I run this, output will be as below,

"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<abcd@xyz.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<efgh@xyz.com>"

grep -B 2 -i "SENT: RCPT" MailLog.txt | grep -v "xyz" . If I run this, output will be as below

"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"

But my requirement is, for example, if third line of first output contains "SENT: RCPT TO:<efgh@domain.com>"
I need to get output as this line and above two lines also.

fukawi1 · 04-20-2012, 03:43 AM

Hang five, I misread your last post.

druuna · 04-20-2012, 03:59 AM

Hi,

Is this what you are looking for:

Code:

sed '/SENT: MAIL FROM/{N;N;/SENT: RCPT TO/{/xyz/d}}' MailLog.txt

You don't show the full layout of the file, maybe this will suffice:

Code:

sed '/SENT: MAIL FROM/{N;N;/xyz/d}' MailLog.txt

Hope this helps.

masatheesh · 04-20-2012, 07:30 AM

I give some more information about my requirement. For example, below lines are excertps from mail log

"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<abcd@xyz.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<efgh@xyz.com>"

By using this above three lines, I come to know that mail has sent from abcd@xyz.com to efgh@xyz.com. So this mail communication has happened internally as From mail ID and To mail ID are having same domain as xyz.com. I dont want this internal communication log.

I need to capture all mail communication which sent to domain other than xyz.com. For example, if mail has sent from abc@xyz.com to test@testing.com. Here I will come to know that abc@xyz.com has sent to test@testing.com. So I can capture this user. I want this kind mail logs only.

schneidz · 04-20-2012, 08:07 AM

this worx:

Code:

[schneidz@hyper ~]$ cat masatheesh.txt
"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<abcd@xyz.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<efgh@xyz.com>"
[schneidz@hyper ~]$ grep -B 2 RCPT masatheesh.txt | awk 'ORS=(NR%3)?" ":"\n"' | grep -v "RCPT TO:.*@xyz"

it essentially takes the 3 lines and appends them into 1 so that grep -v works more easily.
you would need to add some logic if you want to put the non @xyz lines back into 3 separate lines.

druuna · 04-20-2012, 08:12 AM

@masatheesh: Have you tried my solution?

masatheesh · 04-21-2012, 05:47 AM

@druuna: I tried. It shows almost all lines. Its not extracting.

druuna · 04-21-2012, 06:05 AM

Hi,

It looks like I'm still not clear about what it is you want.

Can you post an unfiltered relevant example of the file you start with and an example of the expected output based on that file?

This is what I assumed based on your previous posts:

Code:

$ cat infile
"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<abcd@xyz.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<efgh@xyz.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<abcd@domain.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<efgh@domain.com>"

$ sed '/SENT: MAIL FROM/{N;N;/SENT: RCPT TO/{/xyz/d}}' infile 
"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<abcd@domain.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<efgh@domain.com>"

The italic lines contain RCPT TO. And you only want those lines and 2 before it when it does not contain xyz. My solution seems to do just that.

Please elaborate.

masatheesh · 04-25-2012, 12:41 AM

For example consider below log file content,

"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<abcd@xyz.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<efgh@xyz.com>"
"POP3D" 12160138 "2012-04-19 13:42:41.984" "192.168.0.100" "RECEIVED: AUTH "
"POP3D" 12160138 "2012-04-19 13:42:41.984" "192.168.0.100" "SENT: -ERR Invalid command in current state."
"POP3D" 6264 138 "2012-04-19 13:42:42.000" "192.168.0.100" "RECEIVED: USER It@xyz.com"
"POP3D" 6264 138 "2012-04-19 13:42:42.000" "192.168.0.100" "SENT: +OK Send your password"
"POP3D" 11736 138 "2012-04-19 13:42:42.000" "192.168.0.100" "RECEIVED: PASS ***"
"POP3D" 11736 138 "2012-04-19 13:42:42.000" "192.168.0.100" "SENT: +OK Mailbox locked and ready"
"POP3D" 11736 138 "2012-04-19 13:42:42.015" "192.168.0.100" "RECEIVED: STAT"
"POP3D" 11736 138 "2012-04-19 13:42:42.015" "192.168.0.100" "SENT: +OK 0 0"
"POP3D" 11736 138 "2012-04-19 13:42:42.031" "192.168.0.100" "RECEIVED: QUIT"
"POP3D" 11736 138 "2012-04-19 13:42:42.031" "192.168.0.100" "SENT: +OK POP3 server saying goodbye..."
"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<abcd@xyz.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<efgh@domain.com>"
"POP3D" 6264 138 "2012-04-19 13:42:42.000" "192.168.0.100" "SENT: +OK Send your password"
"POP3D" 11736 138 "2012-04-19 13:42:42.000" "192.168.0.100" "RECEIVED: PASS ***"
"POP3D" 11736 138 "2012-04-19 13:42:42.000" "192.168.0.100" "SENT: +OK Mailbox locked and ready"
"POP3D" 11736 138 "2012-04-19 13:42:42.015" "192.168.0.100" "RECEIVED: STAT"
"POP3D" 11736 138 "2012-04-19 13:42:42.015" "192.168.0.100" "SENT: +OK 0 0"
"POP3D" 11736 138 "2012-04-19 13:42:42.031" "192.168.0.100" "RECEIVED: QUIT"
"POP3D" 11736 138 "2012-04-19 13:42:42.031" "192.168.0.100" "SENT: +OK POP3 server saying goodbye..."
"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<gh@xyz.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<xyz@domain.com>"

From above log, I need output as Italic marked lines

druuna · 04-25-2012, 12:48 AM

Hi,

Code:

sed -n '/SENT: MAIL FROM/{N;N;/SENT: RCPT TO/{/domain/p}}' infile

A test run on the above given data:

Code:

$ sed -n '/SENT: MAIL FROM/{N;N;/SENT: RCPT TO/{/domain/p}}' infile
"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<abcd@xyz.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<efgh@domain.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:39.671" "192.168.0.200" "SENT: MAIL FROM:<gh@xyz.com>"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "RECEIVED: 250 OK"
"SMTPC" 6264 898 "2012-04-19 18:44:40.015" "192.168.0.200" "SENT: RCPT TO:<xyz@domain.com>

Hope this helps.

masatheesh · 05-19-2012, 04:43 AM

Hi,

Thanks for your time and effort.

If a line which has "RCPT TO" and doesnt have xyz.com, then the command has to give that line and two lines before also. Because sometimes domain.com can be replaced by domain1.com,domain2.com etc.

Once again thanks for your help.

druuna · 05-19-2012, 05:32 AM

Guess the info given in post #11 isn't relevant after all.....

Try this:

Code:

sed -n '/SENT: MAIL FROM/{N;N;{/SENT: RCPT TO:.*xyz.com>/!p}}' infile

pan64 · 05-19-2012, 05:38 AM

If you do not need the complete lines you can do the following:

Code:

awk ' BEGIN { RS="SENT: MAIL FROM"; FS="\n" } index($3, "xyz.com") == 0 { print $1 "\n" $2 "\n" $3 } ' inputfile

will display the relevant data I think.