Awk Question to search specific strings grouped by blank lines

rk4k · 07-07-2008, 10:01 PM

Halo ,

I'm want to do snort real time alerting by searching specific string in snort log files.

Is it possible to search specific string in awk (or whatever) and return the string with the line(s) above and below the string separated with blank line ?

The number of lines above and below the string is random. But always separated with blank lines before and after the specific logs files.

The sample log files is below :

Quote:

[**] [1:1918:6] SCAN SolarWinds IP scan attempt [**]
[Classification: Detection of a Network Scan] [Priority: 3]
07/08-02:51:19.737684 202.59.167.55 -> ip.add.res.ses
ICMP TTL:27 TOS:0x0 ID:4290 IpLen:20 DgmLen:141
Type:8 Code:0 ID:512 Seq:46109 ECHO

[**] [1:486:5] ICMP Destination Unreachable Communication with Destination Host is Administratively Prohibited [**]
[Classification: Misc activity] [Priority: 1]
07/08-02:51:19.958887 65.23.153.214 -> ip.add.res.ses
ICMP TTL:242 TOS:0x0 ID:6187 IpLen:20 DgmLen:80
Type:3 Code:10 DESTINATION UNREACHABLE: ADMINISTRATIVELY PROHIBITED HOST FILTERED
** ORIGINAL DATAGRAM DUMP:
ip.add.res.ses:60776 -> 65.23.153.214:80
TCP TTL:48 TOS:0x0 ID:31990 IpLen:20 DgmLen:52 DF
Seq: 0xDE3F87E7
(24 more bytes of original packet)
** END OF DUMP

[**] [119:15:1] (http_inspect) OVERSIZE REQUEST-URI DIRECTORY [**]
[Priority: 1]
07/08-02:51:22.800548 202.138.246.7:52620 -> ip.add.res.ses
TCP TTL:60 TOS:0x0 ID:65210 IpLen:20 DgmLen:1500 DF
***A**** Seq: 0x512DC8CC Ack: 0x82E8E940 Win: 0xFFFF TcpLen: 32
TCP Options (3) => NOP NOP TS: 2819121856 56104715

[**] [1:1918:6] SCAN SolarWinds IP scan attempt [**]
[Classification: Detection of a Network Scan] [Priority: 1]
07/08-02:51:35.742359 202.59.167.55 -> ip.add.res.ses
ICMP TTL:27 TOS:0x0 ID:5075 IpLen:20 DgmLen:141
Type:8 Code:0 ID:512 Seq:27680 ECHO

[**] [119:15:1] (http_inspect) OVERSIZE REQUEST-URI DIRECTORY [**]
[Priority: 3]
07/08-02:51:37.879363 189.63.254.26:1042 -> ip.add.res.ses
TCP TTL:113 TOS:0x0 ID:186 IpLen:20 DgmLen:1500 DF
***A**** Seq: 0x73ED4BF1 Ack: 0x8550920B Win: 0xFFFF TcpLen: 20

[**] [1:483:6] ICMP PING CyberKit 2.2 Windows [**]
[Classification: Misc activity] [Priority: 3]
07/08-02:51:38.347096 202.59.167.55 -> ip.add.res.ses
ICMP TTL:1 TOS:0x0 ID:5198 IpLen:20 DgmLen:84
Type:8 Code:0 ID:512 Seq:56096 ECHO
[Xref => http://www.whitehats.com/info/IDS154]

The string I want to search is :

Quote:

"[Priority: 1]"

The search results should be like these :

Quote:

[**] [1:486:5] ICMP Destination Unreachable Communication with Destination Host is Administratively Prohibited [**]
[Classification: Misc activity] [Priority: 1] ----------------------------------------> Searched string
07/08-02:51:19.958887 65.23.153.214 -> ip.add.res.ses
ICMP TTL:242 TOS:0x0 ID:6187 IpLen:20 DgmLen:80
Type:3 Code:10 DESTINATION UNREACHABLE: ADMINISTRATIVELY PROHIBITED HOST FILTERED
** ORIGINAL DATAGRAM DUMP:
ip.add.res.ses:60776 -> 65.23.153.214:80
TCP TTL:48 TOS:0x0 ID:31990 IpLen:20 DgmLen:52 DF
Seq: 0xDE3F87E7
(24 more bytes of original packet)
** END OF DUMP

[**] [119:15:1] (http_inspect) OVERSIZE REQUEST-URI DIRECTORY [**]
[Priority: 1] ----------------------------------------------------------> Searched string
07/08-02:51:22.800548 202.138.246.7:52620 -> ip.add.res.ses
TCP TTL:60 TOS:0x0 ID:65210 IpLen:20 DgmLen:1500 DF
***A**** Seq: 0x512DC8CC Ack: 0x82E8E940 Win: 0xFFFF TcpLen: 32
TCP Options (3) => NOP NOP TS: 2819121856 56104715

[**] [1:1918:6] SCAN SolarWinds IP scan attempt [**]
[Classification: Detection of a Network Scan] [Priority: 1] -------------> Searched string
07/08-02:51:35.742359 202.59.167.55 -> ip.add.res.ses
ICMP TTL:27 TOS:0x0 ID:5075 IpLen:20 DgmLen:141
Type:8 Code:0 ID:512 Seq:27680 ECHO

I hope somebody get the idea ..

I've try many possible way to do these , but I think I'm stuck now

Thank you.

Mr. C. · 07-07-2008, 10:15 PM

The easiest way for beginners to think of this type of problem is to think about maintaining a previous line variable. You read each line, and at the end of your block, save the current line to the previous line, and continue the loop. Then, when your pattern is found, output your previous line variable, the current line, and then read the next line and output it.

I think you can do the implementation based on that.

fptt · 07-07-2008, 10:37 PM

You could of course roll your own, but you might get some good mileage out of egrep.

$ egrep --help
...
-e, --regexp=PATTERN use PATTERN for matching
...
Context control:
-B, --before-context=NUM print NUM lines of leading context
-A, --after-context=NUM print NUM lines of trailing context

If you need tighter control, you could use perl's multi-line matching mode provided you have well defined start/stop patterns.

Something similar to:

$startpat = 'START_DELIM';
$stoppat = 'STOP_DELIM';
my( $re ) = "$$startpat" . '.*?' . "$stoppat$";
$data =~ s/$re//smi;
$matched = $1;

This puts data between your delimiters into $matched.

You can then push/poke the chunks of data to your liking.

If that doesn't work, you can custom code loops to do your bidding in your language of choice.

Your mileage may vary. Good luck.

rk4k · 07-07-2008, 11:19 PM

fptt,

The context control must specified the number of lines. In my case the number of lines is random especially the lines below the string I want to search :

Code:

Context control:
-B, --before-context=NUM print NUM lines of leading context
-A, --after-context=NUM print NUM lines of trailing context

I come this far to awk documentation (googling around) to :

Quote:

# print the line immediately before a regex, but not the line
# containing the regex
awk '/regex/{print x};{x=$0}'
awk '/regex/{print (x=="" ? "match on line 1" : x)};{x=$0}'

# print the line immediately after a regex, but not the line
# containing the regex
awk '/regex/{getline;print}'

Nothing more .. and still searching

ghostdog74 · 07-07-2008, 11:31 PM

Code:

awk 'BEGIN{RS="";ORS="\n\n"}/Priority: 1/' file

rk4k · 07-07-2008, 11:52 PM

ghostdog74

Awesome !!

Thanks

syg00 · 07-07-2008, 11:56 PM

KISS wins ...