Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
06-23-2017, 01:37 AM
|
#1
|
LQ Newbie
Registered: Mar 2017
Posts: 12
Rep: 
|
Proper linux command
Hi,
I have file which contain data like -
1311629642,abc
1311629642,def
1311629906,ghi
1311630077,jkl
1311631671,mno
1311632245,pqr
I need o/p as -
2 1311629642 abc
def
1 1311629906,ghi
1 1311630077,jkl
1 1311631671,mno
1 1311632245,pqr
I already tried with - cat filename | cut -d "," -f1,2 | sort | uniq -c
but no luck.... could someone let me know the correct command ?
Thanks.
|
|
|
06-23-2017, 02:24 AM
|
#2
|
LQ Guru
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,678
|
One of the correct methods would be to use the language awk.
Another way, if you want to eliminate the whole line, would be to take advantage of sort utility's full capabilities.
Code:
sort -t, -k1,1 -u /path/to/some/filename
sort -t, -k1,1 /path/to/some/filename | uniq -c -w 10
See "man sort" and "man uniq".
However, if you need to retain the line with "def" or someting else more fancy then awk or perl is needed.
Last edited by Turbocapitalist; 06-23-2017 at 02:31 AM.
|
|
|
06-23-2017, 03:22 AM
|
#3
|
LQ Newbie
Registered: Mar 2017
Posts: 12
Original Poster
Rep: 
|
tried with sort -t, -k1,1 /path/to/some/filename | uniq -c -w 10 command but gives o/p as -
2 1311629642 abc
1 1311629906 ghi
1 1311630077 jkl
1 1311631671 mno
1 1311632245 pqr
def missing in this... it should be print as
2 1311629642 abc
def
|
|
|
06-23-2017, 03:33 AM
|
#4
|
Member
Registered: Jan 2017
Location: Manhattan, NYC NY
Distribution: Mac OS X, iOS, Solaris
Posts: 508
Rep: 
|
use Perl.
Code:
| => cat keys.pl
#!/usr/bin/perl
while (<>) {
chomp;
my ($num, $code) = split /,/;
$hash{$num}{$code} = 1;
}
foreach $key (sort keys {map { $_ => 1 } keys %hash }) {
@values = sort keys $hash{$key};
$length = @values;
print $length, " ", $key," ", join(" ", @values), "\n";
}
exit(0);
Output:
Code:
| => ./keys.pl < keys.txt
2 1311629642 abc def
1 1311629906 ghi
1 1311630077 jkl
1 1311631671 mno
1 1311632245 pqr
|
|
1 members found this post helpful.
|
06-23-2017, 03:43 AM
|
#5
|
LQ Newbie
Registered: Mar 2017
Posts: 12
Original Poster
Rep: 
|
thanks for this
but will it be possible by any shell script / linux command instead of perl
|
|
|
06-23-2017, 04:20 AM
|
#6
|
LQ Guru
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,678
|
perl actually is a "linux" "command" You will find it on all systems, just like awk. You will even find it on the various BSD operating systems, such as OS X.
|
|
|
06-23-2017, 04:27 AM
|
#7
|
LQ Newbie
Registered: Mar 2017
Posts: 12
Original Poster
Rep: 
|
Sorry, but I am not familier to perl and need this in shell scripts / any linux command only
Now, I tried by creating 2 files such as -
cat file name | cut -d "," -f1 | uniq -c > test1
cat file name | cut -d "," -f2 > test2
So now, I have 2 files with o/p as -
test1
2 1311629642
1 1311629906
1 1311630077
1 1311631671
1 1311632245
test2
abc
def
ghi
jkl
mno
pqr
now just want to paste the 2 files (test1 and test2) in such way that it should print last o/p as -
2 1311629642 abc def
1 1311629906 ghi
1 1311630077 jkl
1 1311631671 mno
1 1311632245 pqr
Last edited by krishnapa; 06-23-2017 at 04:29 AM.
|
|
|
06-23-2017, 05:00 AM
|
#8
|
LQ Guru
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,678
|
If there were a common field between the two files you could have used join. But going over the data as you have originally presented it you can use Linux's awk or perl commands.
|
|
|
06-23-2017, 05:12 AM
|
#9
|
LQ Newbie
Registered: Mar 2017
Posts: 12
Original Poster
Rep: 
|
better to leave everything... can you help me to get desired o/p using awk / sort / any shell script ?
|
|
|
06-23-2017, 05:21 AM
|
#10
|
LQ Guru
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,678
|
Quote:
Originally Posted by krishnapa
better to leave everything... can you help me to get desired o/p using awk / sort / any shell script ?
|
Well, if you aren't going to use Linux's perl command, I'd still recommend that you put it on your todo list for the future. It is going to come up again and again in similar circumstances.
Back to awk, my approach would be to set the input Field Separator to a comma and then iterate through the data file. As each line is read, make an array element with the first field as a key and increment that element. Also make an element in a second array with the first field as the key, too, and append field two to its value using a comma or other separator. Then the output occurs in the END {} block by looping through the keys in the first array and printing the relevant key and element value. Then use split() to divide the corresponding element in the second array and loop through it, printing each element.
The printf() function is useful in all that.
It may not be the best approach, but it is simple to implement.
Add it in a piece at a time. We can walk you through the steps in awk or perl, but do show us what you have started and which guides you are working from.
|
|
|
06-23-2017, 06:39 AM
|
#11
|
LQ Addict
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,635
|
Is that original input file sorted ?
|
|
|
06-23-2017, 08:15 AM
|
#12
|
Member
Registered: Jan 2017
Location: Manhattan, NYC NY
Distribution: Mac OS X, iOS, Solaris
Posts: 508
Rep: 
|
You don't seem to get the fact that Perl is every bit a standard UNIX/Linux command as awk/sed/etc. Plus, it's much faster, more efficient and more powerful.
|
|
|
06-23-2017, 09:44 AM
|
#13
|
Member
Registered: Jan 2017
Location: Manhattan, NYC NY
Distribution: Mac OS X, iOS, Solaris
Posts: 508
Rep: 
|
I cleaned up the code a bit to add
Code:
use warnings "all";
and
Plus some other stuff.
This is always good Perl programming practice:
Code:
#!/usr/bin/perl
use strict;
use warnings "all";
my %hash = ();
while (<>) {
chomp;
my ($num, $code) = split /\s*,\s*/;
exists $hash{$num}{$code} ? $hash{$num}{$code} += 1 : $hash{$num}{$code} = 1;
}
foreach my $key (sort keys {map { $_ => 1 } keys %hash }) {
my @values = sort keys $hash{$key};
my $length = @values;
print join (" ", $length, $key, @values), "\n";
}
exit(0);
That helps catch a lot of bugs, especially in very large programs.
Last edited by Laserbeak; 06-23-2017 at 09:58 AM.
|
|
|
All times are GMT -5. The time now is 02:41 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|