LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 08-31-2009, 09:36 AM   #1
sebelk
Member
 
Registered: Jan 2007
Posts: 66

Rep: Reputation: 15
Parsing log file with awk


I have logs files from freeradius that have looks as follows:


$ grep "Login incorrect (rlm_ldap: User not found" /var/log/radius/radiusd-inner-tunnel-20090831.log
Mon Aug 31 09:25:27 2009 : Auth: Login incorrect (rlm_ldap: User not found): [John Doe] (from client oficina port 0 via TLS tunnel)
Mon Aug 31 09:25:27 2009 : Auth: Login incorrect (rlm_ldap: User not found): [Joon Williams] (from client oficina port 0 via TLS tunnel)
Mon Aug 31 09:25:27 2009 : Auth: Login incorrect (rlm_ldap: User not found): [hollywood] (from client oficina port 0 via TLS tunnel)
Mon Aug 31 09:25:33 2009 : Auth: Login incorrect (rlm_ldap: User not found): [hollywood] (from client oficina port 0 via TLS tunnel)
Mon Aug 31 09:25:38 2009 : Auth: Login incorrect (rlm_ldap: User not found): [hollywood] (from client oficina port 0 via TLS tunnel)
Mon Aug 31 09:25:44 2009 : Auth: Login incorrect (rlm_ldap: User not found): [hollywood] (from client oficina port 0 via TLS tunnel)
Mon Aug 31 09:25:49 2009 : Auth: Login incorrect (rlm_ldap: User not found): [hollywood] (from client oficina port 0 via TLS tunnel)
Mon Aug 31 09:25:54 2009 : Auth: Login incorrect (rlm_ldap: User not found): [hollywood] (from client oficina port 0 via TLS tunnel)
Mon Aug 31 10:20:44 2009 : Auth: Login incorrect (rlm_ldap: User not found): [John Olmedo] (from client PP-PI-5 port 0 via TLS tunnel)

I use the following line to get the amount of users that don't exist on ldap:

Code:
grep "Login incorrect (rlm_ldap: User not found" /var/log/radius/radiusd-inner-tunnel-20090831.log |   awk '{print $14}' | sort -fu | wc -l
Now, awk on line one for example parses [John Doe] and [Joon Williams] as "[John" and that it's not what I'd want. I mean how could I do for awk looks username field as closed between squared brackets?

Thanks in advance!
 
Old 08-31-2009, 09:47 AM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Since the square brackets are used to embed only the username (correct me if I'm wrong) you can use them as field separator and act accordingly:
Code:
grep "Login incorrect (rlm_ldap: User not found" /var/log/radius/radiusd-inner-tunnel-20090831.log | awk -F[ '{print $2}' | awk -F] '{print $1}'
Edit: another way is to use only awk to accomplish the whole task (no need to grep, sort, wc and so on). For example
Code:
awk '/Login incorrect/{ user=gensub(/.*\[(.*)\].*/,"\\1","g")
                        array[user] += 1 }
END { print "Total number of users with incorrect logins:", length(array)
      for (user in array)
         printf "User %-15s has %3d incorrect logins\n", user, array[user]
}' /var/log/radius/radiusd-inner-tunnel-20090831.log

Last edited by colucix; 08-31-2009 at 10:04 AM. Reason: Added more compact code (pure awk)
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
A data parsing problem - AWK indiancosmonaut Programming 1 02-24-2009 11:34 AM
Parsing a log file jpostma Linux - Newbie 5 12-05-2008 04:58 PM
Parsing using awk tungaw2001 Programming 2 03-27-2007 06:45 AM
awk question - parsing xml file epoo Programming 7 01-24-2007 03:13 PM
Help on parsing a log file in BASH globemast Programming 5 01-11-2007 02:56 AM


All times are GMT -5. The time now is 03:31 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration