LinuxQuestions.org - using sed to remove all but ip addresses

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - using sed to remove all but ip addresses (https://www.linuxquestions.org/questions/programming-9/using-sed-to-remove-all-but-ip-addresses-565827/)

using sed to remove all but ip addresses

I am trying to extract out a list of ip addresses from a firewall log, with one ip on a line. So far I have come up with this:

cat log.txt > sed 's/^.*[0-9]\{1,3\}.[0-9]\{1,3\}.[0-9]\{1,3\}.[0-9]\{1,3\}//g'

but that's not working. If someone could help me, I would be very grateful. Thanks.

Include a little snippet of your log file so everybody is on the same page.
Ken

Ok, thanks, here goes:

Code:

Jul  1 12:35:01 puffy pf: Jul 01 12:32:24.830687 rule 0/(match) block in on sis0: 129.42.58.103.80 > 111.22.33.44.64702: F 0:0(0) ack 1 win 8190 [tos 0x60] 

Jul  1 12:35:01 puffy pf: Jul 01 12:32:24.830831 rule 0/(match) block in on sis0: 129.42.58.103.80 > 111.22.33.44.50012: F 0:0(0) ack 1 win 8190 [tos 0x60] 

Jul  1 12:35:01 puffy pf: Jul 01 12:33:54.090446 rule 0/(match) block in on sis0: 221.209.110.50.49214 > 111.22.33.44.1027: udp 457 (DF)

The "111.22.33.44" is my own ip, changed for the purposes of posting here. Basically, I am trying to extract out the incoming ips, in this case the 129.42.58.103 and 221.209.110.50 ip addresses, into a log file so I can see the occurrences.

Code:

awk '{print $15}' /path/to/logfile | awk -F. '{print $1"."$2"."$3"."$4}'

It is so much easier to use awk here... instead of sed. Note: the second half isn't even needed... except that I use it to chop off the port number.

That works great, thanks frob23!

there's no need to use awk 2 times

Code:



awk '

    { n=split($15,ip,".")

          r=ip[1]          

      for(i=2;i<n;i++){

            r=r"."ip[i] 

          }

          print r

    }

' "file"

Hi,

Or, using multiple field separators:

Code:

awk -F"[ .]" '{ print $17"."$18"."$19"."$20 }' infile

There's a space and a dot inside the square brackets.

Nice answers, but I'm inclined to parse for known data rather than rely on position.
Ya never know when the format will change ...

As always, each to their own - I'd probably do it in perl.

well, given the sample input and from what i see, the only place that formatting can change is after the destination ip portion..so we can safely assuming fields 1 to 17 is kind of fixed. However, this is only a very small sample though..

Here is a longer sampling:

Code:

Jul  1 23:00:02 puffy pf: Jul 01 22:57:28.831284 rule 0/(match) block in on sis0

: 198.26.188.185.31169 > 111.22.33.44.1026: udp 373 

Jul  1 23:00:02 puffy pf: Jul 01 22:58:10.370293 rule 0/(match) block in on sis0

: 159.46.97.65.31169 > 111.22.33.44.1026: udp 493 

Jul  1 23:00:02 puffy pf: Jul 01 22:58:10.370472 rule 0/(match) block in on sis0

: 159.46.97.65.31169 > 111.22.33.44.1027: udp 493 

Jul  1 23:05:02 puffy pf: Jul 01 23:00:41.681482 rule 0/(match) block in on sis0

: 218.234.41.8.6000 > 111.22.33.44.6588: S 1851981824:1851981824(0) win 16384 

Jul  1 23:10:02 puffy pf: Jul 01 23:05:08.790557 rule 0/(match) block in on sis0

: 24.64.70.164.32015 > 111.22.33.44.1027: udp 484 

Jul  1 23:10:02 puffy pf: Jul 01 23:05:08.790837 rule 0/(match) block in on sis0

: 24.64.70.164.32015 > 111.22.33.44.1026: udp 484 

Jul  1 23:10:02 puffy pf: Jul 01 23:05:08.791129 rule 0/(match) block in on sis0

: 24.64.70.164.32015 > 111.22.33.44.1028: udp 484 

Jul  1 23:10:02 puffy pf: Jul 01 23:07:03.131783 rule 0/(match) block in on sis0

: 24.64.170.225.10492 > 111.22.33.44.1026: udp 484 

Jul  1 23:10:02 puffy pf: Jul 01 23:07:03.132086 rule 0/(match) block in on sis0

: 24.64.170.225.10492 > 111.22.33.44.1028: udp 484 

Jul  1 23:10:02 puffy pf: Jul 01 23:07:03.132375 rule 0/(match) block in on sis0

: 24.64.170.225.10492 > 111.22.33.44.1027: udp 484 

Jul  1 23:10:02 puffy pf: Jul 01 23:07:42.492680 rule 0/(match) block in on sis0

: 24.64.54.41.28243 > 111.22.33.44.1026: udp 484 

Jul  1 23:10:02 puffy pf: Jul 01 23:07:42.493063 rule 0/(match) block in on sis0

: 24.64.54.41.28243 > 111.22.33.44.1027: udp 484 

Jul  1 23:10:02 puffy pf: Jul 01 23:07:42.500104 rule 0/(match) block in on sis0

: 24.64.54.41.28243 > 111.22.33.44.1028: udp 484 

Jul  1 23:15:02 puffy pf: Jul 01 23:12:05.750283 rule 0/(match) block in on sis0

: 24.121.217.166.1895 > 111.22.33.44.2968: S 3727943993:3727943993(0) win 1638

4 <mss 1460,nop,nop,sackOK> (DF) 

Jul  1 23:15:02 puffy pf: Jul 01 23:12:08.750281 rule 0/(match) block in on sis0

: 24.121.217.166.1895 > 111.22.33.44.2968: S 3727943993:3727943993(0) win 1638

4 <mss 1460,nop,nop,sackOK> (DF) 

Jul  1 23:15:02 puffy pf: Jul 01 23:12:46.380275 rule 0/(match) block in on sis0

: 63.251.178.30.13223 > 111.22.33.44.33436: udp 4 [ttl 1] 

Jul  1 23:15:02 puffy pf: Jul 01 23:12:51.070190 rule 0/(match) block in on sis0

: 63.251.178.30.13223 > 111.22.33.44.33436: udp 4 [ttl 1] 

Jul  1 23:15:02 puffy pf: Jul 01 23:12:56.040218 rule 0/(match) block in on sis0

: 63.251.178.30.13223 > 111.22.33.44.33436: udp 4 

Jul  1 23:15:02 puffy pf: Jul 01 23:13:01.050217 rule 0/(match) block in on sis0

: 63.251.178.30.13223 > 111.22.33.44.33436: udp 4

All of your suggestions have been great, thanks everyone. Just in case anyone can't tell or is curious, this is an OpenBSD install on a Soekris 4801 running off a compact flash card. It's working great. Anyway, I have the regular pflog run through a script that does a tcpdump into a text file, which is what you see above. I have been watching these logs and for some reason there is an inordinate number of hits from various IPs all within the Shaw Communications range of 24.64.0.0 - 24.71.255.255. I googled around and found a forum thread at the zonealarm forums where other people have found similar hits. (http://forums.zonealarm.com/zonelabs...ssage.id=16151) . That thread starts off about Firefox but then discusses this Shaw Communications issue. I guess there are a lot of Shaw customers with infected computers.

Whichever ISP you're with, you'll always get a large amt of hits from within that (ie the ISP's) range.