LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-31-2012, 11:39 AM   #1
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Rep: Reputation: 16
add a count to sort -u


I run this over a log file

Code:
casper@casperbox ~ $ more badthings20120531 | perl -nle 'print /^<br\s\/>(\w+)\s/' | sort -u

casper2011
casper2023
casper2030
casper2031
casper2032
casper2040
casper2041
casper2042
I sort uniqued them out -
Would there be a way to put a wc -l number near each instance to show how many times it showed up.

like it would be great to not only indicate what servers showed up in log file, but how many times they showed up in the log file.

Last edited by casperdaghost; 05-31-2012 at 11:41 AM.
 
Old 05-31-2012, 01:02 PM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,104

Rep: Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267
I would solve it in perl, without more, sort and others.
Code:
#!/usr/bin/perl -w
use strict;

die if ! defined $ARGV[0];
my $file = $ARGV[0];

my %sum;

open (F, $file) or die "Error: cannot open file: $file, $!";
while (<F>) {
    next if $_ !~ /^<br\s\/>(\w+)\s/;
    $sum{$1}++;
}
close F or die "Error: cannot read file: $file, $!";

foreach (keys %sum) {
    print "$_, $sum{$_}\n";
}
you should give the filename as argument to it






__________________________________
Happy with solution ... mark as SOLVED
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.

Last edited by pan64; 05-31-2012 at 01:03 PM.
 
Old 05-31-2012, 01:17 PM   #3
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,243

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
I could probably whip up something in awk or ruby but would need to see the format.
 
Old 06-01-2012, 11:27 AM   #4
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
i use this alot
Code:
 netstat -an | grep ESTABLISHED | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | awk '{ printf("%s\t%s\t",$2,$1) ; for (i = 0; i < $1; i++) {printf("*")}; print "" }'

i am having trouble applying it to this issue - I don't understand the "
awk '{ printf("%s\t%s\t",$2,$1)"
Code:
walt@walt-nasdaq:~$ cat /home/walt/noo | sed -e 's/pats/casper/g'
<br />casper0053 19:40:08 init: process \'/etc/rc.d/rc.start_java 2>&1 | logger\' (pid 532) exited. Scheduling for restart.
<br />casper0053 19:40:08 init: starting pid 1339, tty \'/dev/null\': \'/etc/rc.d/rc.start_java 2>&1 | logger\'
<br />casper0038 19:40:09 kernel: e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
<br />casper0054 19:40:09 init: process \'/etc/rc.d/rc.start_java 2>&1 | logger\' (pid 890) exited. Scheduling for restart.
<br />casper0054 netstat -an | grep ESTABLISHED | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | awk '{ printf("%s\t%s\t",$2,$1) ; for (i = 0; i < $1; i++) {printf("*")}; print "" }'
 19:40:09 init: starting pid 1606, tty \'/dev/null\': \'/etc/rc.d/rc.start_java 2>&1 | logger\'
<br />casper0048 19:40:10 kernel: e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
<br />casper0055 19:40:11 init: process \'/etc/rc.d/rc.start_java 2>&1 | logger\' (pid 533) exited. Scheduling for restart.
<br />casper0055 19:40:11 init: starting pid 815, tty \'/dev/null\': \'/etc/rc.d/rc.start_java 2>&1 | logger\'
<br />casper0052 19:40:12 kernel: e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
<br />casper0059 19:40:12 init: process \'/etc/rc.d/rc.start_java 2>&1 | logger\' (pid 10090) exited. Scheduling for restart.
<br />casper0059 19:40:12 init: starting pid 10773, tty \'/dev/null\': \'/etc/rc.d/rc.start_java 2>&1 | logger\'
<br />casper0063 19:40:14 init: process \'/etc/rc.d/rc.start_java 2>&1 | logger\' (pid 23802) exited. Scheduling for restart.
<br />casper0063 19:40:14 init: starting pid 24329, tty \'/dev/null\': \'/etc/rc.d/rc.start_java 2>&1 | logger\'
<br />casper0053 19:40:14 kernel: e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
<br />casper0068 19:40:15 init: process \'/etc/rc.d/rc.start_java 2>&1 | logger\' (pid 3046) exited. Scheduling for restart.
<br />casper0068 19:40:15 init: starting pid 3583, tty \'/dev/null\': \'/etc/rc.d/rc.start_java 2>&1 | logger\'
<br />casper0054 19:40:16 kernel: e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
<br />casper0055 19:40:16 kernel: e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
<br />casper0070 19:40:17 init: process \'/etc/rc.d/rc.start_java 2>&1 | logger\' (pid 532) exited. Scheduling for restart.
<br />casper0070 19:40:17 init: starting pid 802, tty \'/dev/null\': \'/etc/rc.d/rc.start_java 2>&1 | logger\'
 
Old 06-01-2012, 11:57 AM   #5
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,027

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
does
Code:
sort | uniq -c
help ?
 
Old 06-01-2012, 12:05 PM   #6
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
oh snap schneidz - that does help - BIGTIME EASY

how can I sort by number and then just take the top ten avec bash?
 
Old 06-01-2012, 01:36 PM   #7
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,027

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
Quote:
Originally Posted by casperdaghost View Post
oh snap schneidz - that does help - BIGTIME EASY

how can I sort by number and then just take the top ten avec bash?
Code:
sort | uniq -c | sort -n -r | head
worx, but theres gotta' be a more efficient way.

Last edited by schneidz; 06-01-2012 at 01:38 PM.
 
Old 06-01-2012, 01:55 PM   #8
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,243

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Well firstly I would say that grep and multiple awks are never the way to go:
Code:
netstat -an | awk 'BEGIN{FS="[ \t:]+";OFS="*"}/ESTAB/{a[$(NF-2)]++}END{asorti(a,b);while(++i < length(b)){$0="";$(a[b[i]]+1)="";print b[i]"\t"a[b[i]]"\t"$0}}'
I am not sure how the rest relates to your question though?
 
Old 06-02-2012, 04:31 AM   #9
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,104

Rep: Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267
what about the perl script I gave you? This should work and also easily can be extended to sort and display the first ten:
Code:
#!/usr/bin/perl -w
use strict;

die if ! defined $ARGV[0];
my $file = $ARGV[0];

my %sum;

open (F, $file) or die "Error: cannot open file: $file, $!";
while (<F>) {
    next if $_ !~ /^<br\s\/>(\w+)\s/;
    $sum{$1}++;
}
close F or die "Error: cannot read file: $file, $!";

my $i = 0;
foreach ( sort { $sum{$b} <=> $sum{$a} } keys %sum )
{
        print "$_, $sum{$_}\n";
        last if (++$i > 4);   # we print only the first 5 lines
}
You can use it as:
perlscript.pl filename > result
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Warning: [fnn_insert] Column count doesn't match value count at row 1 in bondoq Programming 2 09-27-2011 05:11 PM
LXer: Add a Word Count Feature to Tomboy LXer Syndicated Linux News 0 08-11-2010 12:01 AM
Is there a sort of 2nd SSH daemon server, sort of JAILKIT, in the debian repositories frenchn00b Debian 5 06-20-2010 05:03 AM
DBD::mysql::st execute failed: Column count doesn't match value count at row 1 shifter Programming 2 02-24-2010 08:42 PM
Sort polls by vote count jonaskoelker LQ Suggestions & Feedback 2 07-18-2005 03:59 PM


All times are GMT -5. The time now is 12:07 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration