[SOLVED] How to generate list of unique IPs out of access_log

anon091 · 09-15-2014, 03:06 PM

I'm looking at an access_log for an old lampp install, where all the lines in it are similar to this

Code:

12.34.56.78 - - [15/Sep/2014:15:48:09 -0400] "GET /dir/file.png HTTP/1.1" 200 41219

Is there a clever command I could run to just give me a list of all the unique IP addresses in it?

I'm trying to kill this old server, so I want to know which machines i need to visit to point to the new server, and this list would save me a lot of time.

AlucardZero · 09-15-2014, 03:36 PM

Code:

grep -o "^[^ ]*" access_log | sort -nu

anon091 · 09-15-2014, 03:57 PM

Thanks. That only lists the first IP in the file it appears though. Did I maybe re-type it wrong?

syg00 · 09-15-2014, 09:49 PM

Do a search on how to use GNU sort to sort IP addresses. Simple fix.

szboardstretcher · 09-15-2014, 10:06 PM

Assuming that your log file is called logfile,.. then you could do something like this:

Code:

grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' logfile > only_ips.unformatted
uniq only_ips.unformatted > only_ips.uniq
sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 only_ips.uniq > only_ips.sorted

Which I cobbled together after I became frustrated trying to use awk. It's generally less expensive processor/memory wise if you can streamline these kinds of things into one program, but I'm having a dumb and can't brain.

For a good explanation of the sort, see here: https://www.madboa.com/geek/sort-addr/

anon091 · 09-16-2014, 09:04 AM

Thanks sz. Looks like there's a couple dupes in here, but this list is a TON shorter and will work perfectly for what I need. Thanks a lot!

szboardstretcher · 09-16-2014, 09:25 AM

No problem. I wonder why there would be dupes...

anon091 · 09-16-2014, 09:26 AM

There's only a couple of each, which is better than the 100s of each in the original log file. no complaints here!

syg00 · 09-16-2014, 09:32 AM

Quote:

Originally Posted by szboardstretcher

No problem. I wonder why there would be dupes...

Uniq requires sorted input - better to use "sort -nu" as per the earlier response.