BASH - parse apache log for requests per CIDR /24

Scruff · 02-06-2011, 09:25 AM

I'm trying to figure out how I can get a request count per CIDR/24 from an apache log in combined format - eg:

Code:

24.147.44.122 - frank [10/Oct/2011:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "http://www.example.com/start.html" "Mozilla/4.08 [en] (Win98; I ;Nav)"

I'm stuck using BASH for this and I generally write everything in Python, or even Perl if necessary so BASH isn't the most comfortable for me.

I've got enough to extract the IPs and get a count I just need a slick way to come up with a sum on a per CIDR basis.

Scruff · 02-06-2011, 09:42 AM

I should add, this is what I'm using to pull the count of requests on a per IP basis:

Code:

cat access.log | cut -d ' ' -f1 |  sort -t' ' -k2 | uniq -c | sort -n -k1
...

   4317 76.24.55.8
   4812 24.61.244.136
   5144 76.119.58.203
   5714 74.104.26.126
   5988 96.240.228.31
   7682 67.189.152.233

Nominal Animal · 02-06-2011, 01:05 PM

Perhaps I've misunderstood something, but shouldn't

Code:

cut -d . -f1-3 access.log | sort | uniq -c | sort -bg

do the job?

Nominal Animal

colucix · 02-06-2011, 01:38 PM

A gawk solution:

Code:

awk '{_[gensub(/\.[0-9]+$/,"","g",$1)]++}END{print length(_)}' access.log