LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 07-07-2008, 10:01 PM   #1
rk4k
LQ Newbie
 
Registered: Feb 2007
Posts: 9

Rep: Reputation: 0
Awk Question to search specific strings grouped by blank lines


Halo ,

I'm want to do snort real time alerting by searching specific string in snort log files.

Is it possible to search specific string in awk (or whatever) and return the string with the line(s) above and below the string separated with blank line ?

The number of lines above and below the string is random. But always separated with blank lines before and after the specific logs files.

The sample log files is below :
Quote:
[**] [1:1918:6] SCAN SolarWinds IP scan attempt [**]
[Classification: Detection of a Network Scan] [Priority: 3]
07/08-02:51:19.737684 202.59.167.55 -> ip.add.res.ses
ICMP TTL:27 TOS:0x0 ID:4290 IpLen:20 DgmLen:141
Type:8 Code:0 ID:512 Seq:46109 ECHO

[**] [1:486:5] ICMP Destination Unreachable Communication with Destination Host is Administratively Prohibited [**]
[Classification: Misc activity] [Priority: 1]
07/08-02:51:19.958887 65.23.153.214 -> ip.add.res.ses
ICMP TTL:242 TOS:0x0 ID:6187 IpLen:20 DgmLen:80
Type:3 Code:10 DESTINATION UNREACHABLE: ADMINISTRATIVELY PROHIBITED HOST FILTERED
** ORIGINAL DATAGRAM DUMP:
ip.add.res.ses:60776 -> 65.23.153.214:80
TCP TTL:48 TOS:0x0 ID:31990 IpLen:20 DgmLen:52 DF
Seq: 0xDE3F87E7
(24 more bytes of original packet)
** END OF DUMP

[**] [119:15:1] (http_inspect) OVERSIZE REQUEST-URI DIRECTORY [**]
[Priority: 1]
07/08-02:51:22.800548 202.138.246.7:52620 -> ip.add.res.ses
TCP TTL:60 TOS:0x0 ID:65210 IpLen:20 DgmLen:1500 DF
***A**** Seq: 0x512DC8CC Ack: 0x82E8E940 Win: 0xFFFF TcpLen: 32
TCP Options (3) => NOP NOP TS: 2819121856 56104715

[**] [1:1918:6] SCAN SolarWinds IP scan attempt [**]
[Classification: Detection of a Network Scan] [Priority: 1]
07/08-02:51:35.742359 202.59.167.55 -> ip.add.res.ses
ICMP TTL:27 TOS:0x0 ID:5075 IpLen:20 DgmLen:141
Type:8 Code:0 ID:512 Seq:27680 ECHO

[**] [119:15:1] (http_inspect) OVERSIZE REQUEST-URI DIRECTORY [**]
[Priority: 3]
07/08-02:51:37.879363 189.63.254.26:1042 -> ip.add.res.ses
TCP TTL:113 TOS:0x0 ID:186 IpLen:20 DgmLen:1500 DF
***A**** Seq: 0x73ED4BF1 Ack: 0x8550920B Win: 0xFFFF TcpLen: 20

[**] [1:483:6] ICMP PING CyberKit 2.2 Windows [**]
[Classification: Misc activity] [Priority: 3]
07/08-02:51:38.347096 202.59.167.55 -> ip.add.res.ses
ICMP TTL:1 TOS:0x0 ID:5198 IpLen:20 DgmLen:84
Type:8 Code:0 ID:512 Seq:56096 ECHO
[Xref => http://www.whitehats.com/info/IDS154]
The string I want to search is :
Quote:
"[Priority: 1]"
The search results should be like these :

Quote:
[**] [1:486:5] ICMP Destination Unreachable Communication with Destination Host is Administratively Prohibited [**]
[Classification: Misc activity] [Priority: 1] ----------------------------------------> Searched string
07/08-02:51:19.958887 65.23.153.214 -> ip.add.res.ses
ICMP TTL:242 TOS:0x0 ID:6187 IpLen:20 DgmLen:80
Type:3 Code:10 DESTINATION UNREACHABLE: ADMINISTRATIVELY PROHIBITED HOST FILTERED
** ORIGINAL DATAGRAM DUMP:
ip.add.res.ses:60776 -> 65.23.153.214:80
TCP TTL:48 TOS:0x0 ID:31990 IpLen:20 DgmLen:52 DF
Seq: 0xDE3F87E7
(24 more bytes of original packet)
** END OF DUMP

[**] [119:15:1] (http_inspect) OVERSIZE REQUEST-URI DIRECTORY [**]
[Priority: 1] ----------------------------------------------------------> Searched string
07/08-02:51:22.800548 202.138.246.7:52620 -> ip.add.res.ses
TCP TTL:60 TOS:0x0 ID:65210 IpLen:20 DgmLen:1500 DF
***A**** Seq: 0x512DC8CC Ack: 0x82E8E940 Win: 0xFFFF TcpLen: 32
TCP Options (3) => NOP NOP TS: 2819121856 56104715

[**] [1:1918:6] SCAN SolarWinds IP scan attempt [**]
[Classification: Detection of a Network Scan] [Priority: 1] -------------> Searched string
07/08-02:51:35.742359 202.59.167.55 -> ip.add.res.ses
ICMP TTL:27 TOS:0x0 ID:5075 IpLen:20 DgmLen:141
Type:8 Code:0 ID:512 Seq:27680 ECHO
I hope somebody get the idea ..
I've try many possible way to do these , but I think I'm stuck now

Thank you.
 
Old 07-07-2008, 10:15 PM   #2
Mr. C.
Senior Member
 
Registered: Jun 2008
Posts: 2,529

Rep: Reputation: 59
The easiest way for beginners to think of this type of problem is to think about maintaining a previous line variable. You read each line, and at the end of your block, save the current line to the previous line, and continue the loop. Then, when your pattern is found, output your previous line variable, the current line, and then read the next line and output it.

I think you can do the implementation based on that.
 
Old 07-07-2008, 10:37 PM   #3
fptt
LQ Newbie
 
Registered: Jun 2008
Posts: 3

Rep: Reputation: 4
You could of course roll your own, but you might get some good mileage out of egrep.


$ egrep --help
...
-e, --regexp=PATTERN use PATTERN for matching
...
Context control:
-B, --before-context=NUM print NUM lines of leading context
-A, --after-context=NUM print NUM lines of trailing context

If you need tighter control, you could use perl's multi-line matching mode provided you have well defined start/stop patterns.

Something similar to:

$startpat = 'START_DELIM';
$stoppat = 'STOP_DELIM';
my( $re ) = "\($startpat" . '.*?' . "$stoppat\)";
$data =~ s/$re//smi;
$matched = $1;

This puts data between your delimiters into $matched.

You can then push/poke the chunks of data to your liking.

If that doesn't work, you can custom code loops to do your bidding in your language of choice.

Your mileage may vary. Good luck.
 
Old 07-07-2008, 11:19 PM   #4
rk4k
LQ Newbie
 
Registered: Feb 2007
Posts: 9

Original Poster
Rep: Reputation: 0
fptt,

The context control must specified the number of lines. In my case the number of lines is random especially the lines below the string I want to search :

Code:
Context control:
-B, --before-context=NUM print NUM lines of leading context
-A, --after-context=NUM print NUM lines of trailing context
I come this far to awk documentation (googling around) to :
Quote:
# print the line immediately before a regex, but not the line
# containing the regex
awk '/regex/{print x};{x=$0}'
awk '/regex/{print (x=="" ? "match on line 1" : x)};{x=$0}'

# print the line immediately after a regex, but not the line
# containing the regex
awk '/regex/{getline;print}'
Nothing more .. and still searching
 
Old 07-07-2008, 11:31 PM   #5
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,696
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Code:
awk 'BEGIN{RS="";ORS="\n\n"}/Priority: 1/' file
 
Old 07-07-2008, 11:52 PM   #6
rk4k
LQ Newbie
 
Registered: Feb 2007
Posts: 9

Original Poster
Rep: Reputation: 0
ghostdog74

Awesome !!

Thanks
 
Old 07-07-2008, 11:56 PM   #7
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,351

Rep: Reputation: 1041Reputation: 1041Reputation: 1041Reputation: 1041Reputation: 1041Reputation: 1041Reputation: 1041Reputation: 1041
KISS wins ...
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
awk print correct lines when certain columns are blank schneidz Programming 11 04-04-2008 04:06 PM
Delete specific Range of lines Using sed , awk, grep etc. joyds219 Linux - Newbie 4 03-28-2008 08:59 AM
Replacing text on specific lines with sed or awk? Lantzvillian Linux - Newbie 5 10-17-2007 09:00 AM
search for specific text in fields using awk Helene Programming 2 04-23-2004 12:13 AM


All times are GMT -5. The time now is 06:20 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration