LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-31-2016, 06:25 AM   #1
arun natarajan
Member
 
Registered: Jun 2014
Posts: 111

Rep: Reputation: Disabled
duplicate not removed in column wise sort


hi,

am trying to print only unique records by comparing the values only from 1st column.

But its not working as expected.

May i know the required changes to do ???

[root@localhost ~]# uname -a
Linux localhost.localdomain 2.6.32-71.el6.x86_64 #1 SMP Wed Sep 1 01:33:01 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

[root@localhost ~]# sort -u -k1 ee
Ankit m 28 developer 7000 Jun 11-06-2016
bavana f 30 tester 10000 Dec 02-12-2016
bavana f 30 tester 900 Dec 02-12-2016
nilakshi f 25 engineer 20000 Feb 23-02-16
praveen m 22 analyst 30000 May 15-05-16
sakshi f 36 manager 60000 Mar 22-03-16
sakshi f 42 manager 60000 Sep 05-09-2016

===================================================
Am expecting the output like below:

[root@localhost ~]# sort -u -k1 ee
Ankit m 28 developer 7000 Jun 11-06-2016
nilakshi f 25 engineer 20000 Feb 23-02-16
praveen m 22 analyst 30000 May 15-05-16

============================================
 
Old 12-31-2016, 06:31 AM   #2
Jjanel
Member
 
Registered: Jun 2016
Distribution: any&all, in VBox; Ol'UnixCLI; NO GUI resources
Posts: 999
Blog Entries: 12

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
awk? print only if key field(s) *seen only/exactly ONCE*

Try -k1,1 http://stackoverflow.com/questions/1...n-sort-command
But that doesn't *eliminate* if dups; it just leaves one. (My mistake, sorry)
So, it looks like a more complex solution idea is needed. uniq -u (but on 1 field!)
(maybe |awk instead of -u; 'remember' previous and skip if key-match)

I tried awk'ing for hourS: no success How-to iff key field(s) seen *once*?
It's like a 'inventory' algorithm:
if you have >1, don't list/order; if 1, list=order (0/none N/A)

Anyone want to help? Thanks! Happy New Year

UPDATE: I was gifted this (to solve my 'seen-once' failure):
Code:
#!/usr/bin/awk -f

{
  if( $1 in keep)
    del[$1]

  keep[$1] = $0
}

END{
  asorti(keep, nkeep)

  for(i in nkeep)
    if(!(nkeep[i] in del))
    print keep[nkeep[i]]
}

Last edited by Jjanel; 01-03-2017 at 03:55 AM.
 
Old 01-01-2017, 06:16 AM   #3
arun natarajan
Member
 
Registered: Jun 2014
Posts: 111

Original Poster
Rep: Reputation: Disabled
thanks janel,

it worked with "sort -u -k1,1"
 
Old 01-02-2017, 10:20 AM   #4
arun natarajan
Member
 
Registered: Jun 2014
Posts: 111

Original Poster
Rep: Reputation: Disabled
Hi Janel,

My data is not getting sorted when i want to sort 4th column and then based on 4th column values, 5th column should be sorted.
From the below output, 4th column is sorted properly, but 5th column based on 4th column is not sorted.


[root@localhost ~]# sort -k4,4 -k5,5 emp
praveen m 22 analyst 30000 May 15-05-16
saba f 23 analyst 40000 Jan 22-01-16
Nilan m 36 assistant 800 Apr 05-04-16
ghosh m 29 consultant 80000 Oct 08-10-16
Ankit m 28 developer 7000 Jun 11-06-16
rasheed m 32 developer 9000 Jul 16-07-16
nilakshi f 25 engineer 20000 Feb 23-02-16
Tina f 30 hr 8000 Aug 30-08-16
sakshi f 42 manager 60000 Sep 05-09-16
bavana f 30 tester 10000 Dec 02-12-16
bavana f 30 tester 900 Dec 02-12-16
shreya f 33 vendor 5000 Nov 06-11-09

When i tried with numeric sort in 5th column based on 4th column, it is sorting only based on 5th column.

[root@localhost ~]# sort -k4,4 -nk5,5 emp
Nilan m 36 assistant 800 Apr 05-04-16
bavana f 30 tester 900 Dec 02-12-16
shreya f 33 vendor 5000 Nov 06-11-09
Ankit m 28 developer 7000 Jun 11-06-16
Tina f 30 hr 8000 Aug 30-08-16
rasheed m 32 developer 9000 Jul 16-07-16
bavana f 30 tester 10000 Dec 02-12-16
nilakshi f 25 engineer 20000 Feb 23-02-16
praveen m 22 analyst 30000 May 15-05-16
saba f 23 analyst 40000 Jan 22-01-16
sakshi f 36 manager 60000 Mar 22-03-16
ghosh m 29 consultant 80000 Oct 08-10-16


Expected output:

praveen m 22 analyst 30000 May 15-05-16
saba f 23 analyst 40000 Jan 22-01-16
Nilan m 36 assistant 800 Apr 05-04-16
ghosh m 29 consultant 80000 Oct 08-10-16
Ankit m 28 developer 7000 Jun 11-06-16
rasheed m 32 developer 9000 Jul 16-07-16
nilakshi f 25 engineer 20000 Feb 23-02-16
Tina f 30 hr 8000 Aug 30-08-16
sakshi f 42 manager 60000 Sep 05-09-16
bavana f 30 tester 900 Dec 02-12-16
bavana f 30 tester 10000 Dec 02-12-16
shreya f 33 vendor 5000 Nov 06-11-09
 
Old 01-02-2017, 05:20 PM   #5
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,785

Rep: Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574
Deleted - testing.

Last edited by syg00; 01-02-2017 at 05:21 PM.
 
2 members found this post helpful.
Old 01-02-2017, 05:27 PM   #6
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,785

Rep: Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574Reputation: 3574
Read the manpage.
That specification doesn't match what is documented. You do need to specify 2 keys but your testing obviously isn't very thorough.
 
1 members found this post helpful.
Old 01-03-2017, 01:36 AM   #7
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,821

Rep: Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084
I feel this should have been raised as a new question as it is not related to the current topic
 
Old 01-03-2017, 02:55 AM   #8
Jjanel
Member
 
Registered: Jun 2016
Distribution: any&all, in VBox; Ol'UnixCLI; NO GUI resources
Posts: 999
Blog Entries: 12

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
Your -n is 'syntactically' *not* part of -k5, *when* you put it *before* the -k!
From man sort: KEYDEF is F[.C][OPTS][,F[.C][OPTS]]

Yes, new non- -u thread! p.s. thanks grail, 2x
(unless Title changed to: learning sort command )

p.s. you have a conflict, tho irrelevant to k4&5, in sakshi age in middle list: 36/42

Last edited by Jjanel; 01-03-2017 at 03:33 AM.
 
  


Reply

Tags
awk


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] HELP! How to merge two files as column wise?? SAAAD Linux - Newbie 9 03-24-2013 01:31 AM
[SOLVED] Append output of a command in column wise manner shivaa Linux - Newbie 5 02-18-2013 02:43 AM
how to sort the 2nd column on the basis of first column without repeating the value ? zediok Linux - Newbie 15 12-20-2011 11:48 AM
printing column wise bharatbsharma Programming 4 09-11-2008 11:00 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 09:14 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration