LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-23-2017, 01:37 AM   #1
krishnapa
LQ Newbie
 
Registered: Mar 2017
Posts: 12

Rep: Reputation: Disabled
Proper linux command


Hi,

I have file which contain data like -
1311629642,abc
1311629642,def
1311629906,ghi
1311630077,jkl
1311631671,mno
1311632245,pqr

I need o/p as -
2 1311629642 abc
def
1 1311629906,ghi
1 1311630077,jkl
1 1311631671,mno
1 1311632245,pqr

I already tried with - cat filename | cut -d "," -f1,2 | sort | uniq -c
but no luck.... could someone let me know the correct command ?

Thanks.
 
Old 06-23-2017, 02:24 AM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,678
Blog Entries: 4

Rep: Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927
One of the correct methods would be to use the language awk.

Another way, if you want to eliminate the whole line, would be to take advantage of sort utility's full capabilities.

Code:
sort -t, -k1,1 -u /path/to/some/filename
sort -t, -k1,1 /path/to/some/filename | uniq -c -w 10
See "man sort" and "man uniq".

However, if you need to retain the line with "def" or someting else more fancy then awk or perl is needed.

Last edited by Turbocapitalist; 06-23-2017 at 02:31 AM.
 
Old 06-23-2017, 03:22 AM   #3
krishnapa
LQ Newbie
 
Registered: Mar 2017
Posts: 12

Original Poster
Rep: Reputation: Disabled
tried with sort -t, -k1,1 /path/to/some/filename | uniq -c -w 10 command but gives o/p as -
2 1311629642 abc
1 1311629906 ghi
1 1311630077 jkl
1 1311631671 mno
1 1311632245 pqr

def missing in this... it should be print as

2 1311629642 abc
def
 
Old 06-23-2017, 03:33 AM   #4
Laserbeak
Member
 
Registered: Jan 2017
Location: Manhattan, NYC NY
Distribution: Mac OS X, iOS, Solaris
Posts: 508

Rep: Reputation: 143Reputation: 143
use Perl.

Code:
| => cat keys.pl
#!/usr/bin/perl

while (<>) {
   chomp;
   my ($num, $code) = split /,/;
   $hash{$num}{$code} = 1;
}

foreach $key (sort keys {map { $_ => 1 } keys %hash }) {
   @values = sort keys $hash{$key};
   $length = @values;
   print $length, " ", $key," ", join(" ", @values), "\n";
}

exit(0);
Output:

Code:
| => ./keys.pl < keys.txt
2 1311629642 abc def
1 1311629906 ghi
1 1311630077 jkl
1 1311631671 mno
1 1311632245 pqr
 
1 members found this post helpful.
Old 06-23-2017, 03:43 AM   #5
krishnapa
LQ Newbie
 
Registered: Mar 2017
Posts: 12

Original Poster
Rep: Reputation: Disabled
thanks for this

but will it be possible by any shell script / linux command instead of perl
 
Old 06-23-2017, 04:20 AM   #6
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,678
Blog Entries: 4

Rep: Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927
perl actually is a "linux" "command" You will find it on all systems, just like awk. You will even find it on the various BSD operating systems, such as OS X.
 
Old 06-23-2017, 04:27 AM   #7
krishnapa
LQ Newbie
 
Registered: Mar 2017
Posts: 12

Original Poster
Rep: Reputation: Disabled
Sorry, but I am not familier to perl and need this in shell scripts / any linux command only

Now, I tried by creating 2 files such as -
cat file name | cut -d "," -f1 | uniq -c > test1
cat file name | cut -d "," -f2 > test2

So now, I have 2 files with o/p as -
test1
2 1311629642
1 1311629906
1 1311630077
1 1311631671
1 1311632245

test2
abc
def
ghi
jkl
mno
pqr

now just want to paste the 2 files (test1 and test2) in such way that it should print last o/p as -
2 1311629642 abc def
1 1311629906 ghi
1 1311630077 jkl
1 1311631671 mno
1 1311632245 pqr

Last edited by krishnapa; 06-23-2017 at 04:29 AM.
 
Old 06-23-2017, 05:00 AM   #8
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,678
Blog Entries: 4

Rep: Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927
If there were a common field between the two files you could have used join. But going over the data as you have originally presented it you can use Linux's awk or perl commands.
 
Old 06-23-2017, 05:12 AM   #9
krishnapa
LQ Newbie
 
Registered: Mar 2017
Posts: 12

Original Poster
Rep: Reputation: Disabled
better to leave everything... can you help me to get desired o/p using awk / sort / any shell script ?
 
Old 06-23-2017, 05:21 AM   #10
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,678
Blog Entries: 4

Rep: Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927Reputation: 3927
Quote:
Originally Posted by krishnapa View Post
better to leave everything... can you help me to get desired o/p using awk / sort / any shell script ?
Well, if you aren't going to use Linux's perl command, I'd still recommend that you put it on your todo list for the future. It is going to come up again and again in similar circumstances.

Back to awk, my approach would be to set the input Field Separator to a comma and then iterate through the data file. As each line is read, make an array element with the first field as a key and increment that element. Also make an element in a second array with the first field as the key, too, and append field two to its value using a comma or other separator. Then the output occurs in the END {} block by looping through the keys in the first array and printing the relevant key and element value. Then use split() to divide the corresponding element in the second array and loop through it, printing each element.

The printf() function is useful in all that.

It may not be the best approach, but it is simple to implement.

Add it in a piece at a time. We can walk you through the steps in awk or perl, but do show us what you have started and which guides you are working from.
 
Old 06-23-2017, 06:39 AM   #11
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,635

Rep: Reputation: 7813Reputation: 7813Reputation: 7813Reputation: 7813Reputation: 7813Reputation: 7813Reputation: 7813Reputation: 7813Reputation: 7813Reputation: 7813Reputation: 7813
Is that original input file sorted ?
 
Old 06-23-2017, 08:15 AM   #12
Laserbeak
Member
 
Registered: Jan 2017
Location: Manhattan, NYC NY
Distribution: Mac OS X, iOS, Solaris
Posts: 508

Rep: Reputation: 143Reputation: 143
You don't seem to get the fact that Perl is every bit a standard UNIX/Linux command as awk/sed/etc. Plus, it's much faster, more efficient and more powerful.
 
Old 06-23-2017, 09:44 AM   #13
Laserbeak
Member
 
Registered: Jan 2017
Location: Manhattan, NYC NY
Distribution: Mac OS X, iOS, Solaris
Posts: 508

Rep: Reputation: 143Reputation: 143
I cleaned up the code a bit to add

Code:
use warnings "all";
and

Code:
use strict;
Plus some other stuff.

This is always good Perl programming practice:

Code:
#!/usr/bin/perl

use strict;
use warnings "all";

my %hash = ();

while (<>) {
   chomp;
   my ($num, $code) = split /\s*,\s*/;
   exists $hash{$num}{$code} ? $hash{$num}{$code} += 1 : $hash{$num}{$code} = 1;
}

foreach my $key (sort keys {map { $_ => 1 } keys %hash }) {
   my @values = sort keys $hash{$key};
   my $length = @values;
   print join (" ", $length, $key, @values), "\n";
}

exit(0);
That helps catch a lot of bugs, especially in very large programs.

Last edited by Laserbeak; 06-23-2017 at 09:58 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] the sync command and proper use cpu0 Linux - Newbie 11 01-11-2017 01:01 PM
[SOLVED] proper syntax for a qemu command jr_bob_dobbs Slackware 5 02-26-2016 10:22 PM
Info on proper command line use chipado LinuxQuestions.org Member Intro 2 09-22-2012 01:36 PM
proper syntax for 'more' command bourne Linux - General 6 09-14-2007 10:26 AM
Shell Script help or proper command jmikeneedham Linux - General 2 04-30-2006 02:25 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 02:41 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration