LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   Using grep -v to sort out a log (https://www.linuxquestions.org/questions/programming-9/using-grep-v-to-sort-out-a-log-880071/)

rhbegin 05-11-2011 07:52 AM
Using grep -v to sort out a log
 
I have logs from email servers and I am trying to sort out our IP ranges from the log files.

I am using the following:

grep -v "192.168.0.25" s20110511serv.log > s2011_25_removed.log
grep -v "192.168.0.27" s2011_25_removed.log > s2011_2527_gone.log

How can I issue this on one line without having to output to multiple files, also can I pattern match an IP range like 192.168.4.0/18 from the same line???

grep -v "192.168.0.25" "192.168.0.27" "192.168.4.0" the /18 range from the log file?

Any help would be great, I am still learning on grep and I am a bit confused on the syntax to pull out data in one line.

Thanks

theNbomr 05-11-2011 08:40 AM
You can pipe the output of grep to the standard input of any other command, including grep:
Code:
grep -v "192.168.0.25" s20110511serv.log | grep -v "192.168.0.27" > s2011_2527_gone.log
Grep doesn't know anything about the notation used to describe IP subnetting.
--- rod.

rhbegin 05-11-2011 02:02 PM
Quote:
Originally Posted by theNbomr (Post 4353254)
You can pipe the output of grep to the standard input of any other command, including grep:
Code:
grep -v "192.168.0.25" s20110511serv.log | grep -v "192.168.0.27" > s2011_2527_gone.log
Grep doesn't know anything about the notation used to describe IP subnetting.
--- rod.
So you can pipe your source ip then again then output to 1 file, that will work.

Is it possible to take out an ip range in a subnet like 192.168.4.0/24 out of the log file then with a different method?

Thank you for the quick response!

:)

jcmlq 05-11-2011 02:27 PM
Your first command is saying 'give me everything without 192.168.0.25', and your second line is saying 'give me everything without 192.168.0.27'. You can say in a single line 'give me everything without 192.168.0.2' followed by 5 or 7 with a regular expression.

Code:
grep -v '192.168.0.2[57]'
You could remove an entire /24 with an extended regex like
Code:
grep -v -E '192\.168\.0\.[0-9]{1,3}'
The \. is used to match the '.' characters because '.' is itself a special regular expression character.

The -E means extended grep syntax is enabled.

The meaning of the match expression is '192.168.0.', followed by 1 to 3 instances of the characters 0-9.

rhbegin 05-11-2011 03:21 PM
Quote:
Originally Posted by jcmlq (Post 4353565)
Your first command is saying 'give me everything without 192.168.0.25', and your second line is saying 'give me everything without 192.168.0.27'. You can say in a single line 'give me everything without 192.168.0.2' followed by 5 or 7 with a regular expression.

Code:
grep -v '192.168.0.2[57]'
You could remove an entire /24 with an extended regex like
Code:
grep -v -E '192\.168\.0\.[0-9]{1,3}'
The \. is used to match the '.' characters because '.' is itself a special regular expression character.

The -E means extended grep syntax is enabled.

The meaning of the match expression is '192.168.0.', followed by 1 to 3 instances of the characters 0-9.
Thanks a million! :)

I need to write this down and study it to get a good understanding, I learn something new everything with the command-line.

:)

jcmlq 05-11-2011 03:31 PM
One thing I should make absolutely clear about the example to match a /24 that I provided - it works, but it isn't actually correct. It is just as happy to match 192.168.0.999 as it is to match 192.168.0.255

Much (most) of the time that kind of sloppiness is just fine, but if you really need to make sure and only match valid ip addresses then you probably need to use a full blown scripting language and not just grep.

EDIT: the proper ip octet match scheme is something like

Code:
grep -E -v '192\.168\.0\.(1*[0-9]{1,2}|2[0-4][0-9]|25[0-5])'
Since you seem to be pretty new to regex I hesitate to dump that mess in your lap when something much easier to understand does pretty much what you need.

rhbegin 05-11-2011 04:13 PM
Quote:
Originally Posted by jcmlq (Post 4353633)
One thing I should make absolutely clear about the example to match a /24 that I provided - it works, but it isn't actually correct. It is just as happy to match 192.168.0.999 as it is to match 192.168.0.255

Much (most) of the time that kind of sloppiness is just fine, but if you really need to make sure and only match valid ip addresses then you probably need to use a full blown scripting language and not just grep.

EDIT: the proper ip octet match scheme is something like

Code:
grep -E -v '192\.168\.0\.(1*[0-9]{1,2}|2[0-4][0-9]|25[0-5])'
Since you seem to be pretty new to regex I hesitate to dump that mess in your lap when something much easier to understand does pretty much what you need.
Thank you again, I am still learning, once I get the basic concepts I will understand it. It takes me a little while on the front end but once I understand it, I got it.

:)

Thank you again!

:)

jcmlq 05-11-2011 04:29 PM
You're welcome, good luck in cracking into regular expressions - they are an essential tool for admin and operations work in my opinion.


All times are GMT -5. The time now is 10:09 PM.