LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 02-06-2011, 09:25 AM   #1
Scruff
Member
 
Registered: Oct 2002
Location: Stoughton, MA
Distribution: Gentoo x86_64 & PPC
Posts: 949

Rep: Reputation: 30
BASH - parse apache log for requests per CIDR /24


I'm trying to figure out how I can get a request count per CIDR/24 from an apache log in combined format - eg:

Code:
24.147.44.122 - frank [10/Oct/2011:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "http://www.example.com/start.html" "Mozilla/4.08 [en] (Win98; I ;Nav)"
I'm stuck using BASH for this and I generally write everything in Python, or even Perl if necessary so BASH isn't the most comfortable for me.

I've got enough to extract the IPs and get a count I just need a slick way to come up with a sum on a per CIDR basis.
 
Old 02-06-2011, 09:42 AM   #2
Scruff
Member
 
Registered: Oct 2002
Location: Stoughton, MA
Distribution: Gentoo x86_64 & PPC
Posts: 949

Original Poster
Rep: Reputation: 30
I should add, this is what I'm using to pull the count of requests on a per IP basis:

Code:
cat access.log | cut -d ' ' -f1 |  sort -t' ' -k2 | uniq -c | sort -n -k1
...

   4317 76.24.55.8
   4812 24.61.244.136
   5144 76.119.58.203
   5714 74.104.26.126
   5988 96.240.228.31
   7682 67.189.152.233
 
Old 02-06-2011, 01:05 PM   #3
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 947Reputation: 947Reputation: 947Reputation: 947Reputation: 947Reputation: 947Reputation: 947Reputation: 947
Perhaps I've misunderstood something, but shouldn't
Code:
cut -d . -f1-3 access.log | sort | uniq -c | sort -bg
do the job?
Nominal Animal

Last edited by Nominal Animal; 03-21-2011 at 07:20 AM.
 
Old 02-06-2011, 01:38 PM   #4
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978
A gawk solution:
Code:
awk '{_[gensub(/\.[0-9]+$/,"","g",$1)]++}END{print length(_)}' access.log
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
bash CIDR calculator genderbender Programming 18 10-29-2014 10:13 AM
Parse log file with bash mijohnst Linux - General 8 09-09-2010 10:46 PM
Redirect https apache requests to folder not in apache root bax Linux - Enterprise 4 04-12-2010 01:16 PM
Parse Log files nima0102 Linux - Server 2 11-23-2009 11:24 AM
Constant Log in Requests john3 LQ Suggestions & Feedback 1 03-27-2006 08:49 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 05:45 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration