[SOLVED] Arrange output by frequency of occurrence

oliviaxinw · 08-23-2012, 01:34 PM

Hello,

I have a file with entries like:
A
A
B
C
A
B
B
A
...

I want the output to look something like: (arrange by how many times a particular entry occurs and write down how many times it occurs)
A 4
B 3
C 1

What code should I use? My file is very large and have tens of thousands of different entries, so it's not practical for me to define individually what A, B, C...are.

Thank you =)

byannoni · 08-23-2012, 01:52 PM

Code:

uniq -c file | sort -nr

schneidz · 08-23-2012, 03:27 PM

you would need to sort before you uniq

Code:

sort olivia.txt | uniq -c | sort -n -r # | awk '{print $2 " " $1}'

byannoni · 08-23-2012, 04:33 PM

Quote:

Originally Posted by schneidz

you would need to sort before you uniq

Code:

sort olivia.txt | uniq -c | sort -n -r # | awk '{print $2 " " $1}'

Did you try mine before saying what it needs? It works fine and the output is the same as yours.

schneidz · 08-23-2012, 04:38 PM

no i didnt, i'll try it now... i always learned that a sort was necessary before a uniq since uniq counts consecutive rows.

edit:
i think i'm rite:

Code:

[schneidz@hyper abg]$ uniq -c olivia.txt | sort -nr
      2 B
      2 A
      1 C
      1 B
      1 A
      1 A
[schneidz@hyper abg]$ sort olivia.txt | uniq -c | sort -n -r | awk '{print $2 " " $1}'
A 4
B 3
C 1
[schneidz@hyper abg]$ uniq --version
uniq (GNU coreutils) 8.10
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Richard M. Stallman and David MacKenzie.
[schneidz@hyper abg]$ sort --version
sort (GNU coreutils) 8.10
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and Paul Eggert.

David the H. · 08-23-2012, 05:09 PM

Here's a slightly more advanced gawk solution.

Code:

gawk 'BEGIN{ PROCINFO["sorted_in"]="@val_num_desc" } { a[$1]+=1 } END{ for ( i in a ) { print i,a[i] } }'  infile.txt

It counts the number of each entry using an array, and prints the results out at the end.

Sorting is handled by a PROCINFO setting, which is why it requires a recent version.

For older versions of gawk or other awks, remove the BEGIN section and just pipe the output through "sort -rn -k2".

byannoni · 08-23-2012, 05:09 PM

Quote:

Originally Posted by schneidz

no i didnt, i'll try it now... i always learned that a sort was necessary before a uniq since uniq counts consecutive rows.

edit:
i think i'm rite:

Code:

[schneidz@hyper abg]$ uniq -c olivia.txt | sort -nr
      2 B
      2 A
      1 C
      1 B
      1 A
      1 A
[schneidz@hyper abg]$ sort olivia.txt | uniq -c | sort -n -r | awk '{print $2 " " $1}'
A 4
B 3
C 1
[schneidz@hyper abg]$ uniq --version
uniq (GNU coreutils) 8.10
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Richard M. Stallman and David MacKenzie.
[schneidz@hyper abg]$ sort --version
sort (GNU coreutils) 8.10
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and Paul Eggert.

I stand corrected. Sorry about that, I should have tested with values that weren't already in order.