duplicate not removed in column wise sort

arun natarajan · 12-31-2016, 06:25 AM

hi,

am trying to print only unique records by comparing the values only from 1st column.

But its not working as expected.

May i know the required changes to do ???

[root@localhost ~]# uname -a
Linux localhost.localdomain 2.6.32-71.el6.x86_64 #1 SMP Wed Sep 1 01:33:01 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

[root@localhost ~]# sort -u -k1 ee
Ankit m 28 developer 7000 Jun 11-06-2016
bavana f 30 tester 10000 Dec 02-12-2016
bavana f 30 tester 900 Dec 02-12-2016
nilakshi f 25 engineer 20000 Feb 23-02-16
praveen m 22 analyst 30000 May 15-05-16
sakshi f 36 manager 60000 Mar 22-03-16
sakshi f 42 manager 60000 Sep 05-09-2016

===================================================
Am expecting the output like below:

[root@localhost ~]# sort -u -k1 ee
Ankit m 28 developer 7000 Jun 11-06-2016
nilakshi f 25 engineer 20000 Feb 23-02-16
praveen m 22 analyst 30000 May 15-05-16
============================================

Jjanel · 12-31-2016, 06:31 AM

Try -k1,1 http://stackoverflow.com/questions/1...n-sort-command
But that doesn't *eliminate* if dups; it just leaves one. (My mistake, sorry)
So, it looks like a more complex solution idea is needed. uniq -u (but on 1 field!)
(maybe |awk instead of -u; 'remember' previous and skip if key-match)

I tried awk'ing for hourS: no success

How-to iff key field(s) seen *once*?
It's like a 'inventory' algorithm:
if you have >1, don't list/order; if 1, list=order (0/none N/A)

Anyone want to help? Thanks! Happy New Year

UPDATE: I was gifted this (to solve my 'seen-once' failure):

Code:

#!/usr/bin/awk -f

{
  if( $1 in keep)
    del[$1]

  keep[$1] = $0
}

END{
  asorti(keep, nkeep)

  for(i in nkeep)
    if(!(nkeep[i] in del))
    print keep[nkeep[i]]
}

arun natarajan · 01-01-2017, 06:16 AM

thanks janel,

it worked with "sort -u -k1,1"

arun natarajan · 01-02-2017, 10:20 AM

Hi Janel,

My data is not getting sorted when i want to sort 4th column and then based on 4th column values, 5th column should be sorted.
From the below output, 4th column is sorted properly, but 5th column based on 4th column is not sorted.

[root@localhost ~]# sort -k4,4 -k5,5 emp
praveen m 22 analyst 30000 May 15-05-16
saba f 23 analyst 40000 Jan 22-01-16
Nilan m 36 assistant 800 Apr 05-04-16
ghosh m 29 consultant 80000 Oct 08-10-16
Ankit m 28 developer 7000 Jun 11-06-16
rasheed m 32 developer 9000 Jul 16-07-16
nilakshi f 25 engineer 20000 Feb 23-02-16
Tina f 30 hr 8000 Aug 30-08-16
sakshi f 42 manager 60000 Sep 05-09-16
bavana f 30 tester 10000 Dec 02-12-16
bavana f 30 tester 900 Dec 02-12-16
shreya f 33 vendor 5000 Nov 06-11-09

When i tried with numeric sort in 5th column based on 4th column, it is sorting only based on 5th column.

[root@localhost ~]# sort -k4,4 -nk5,5 emp
Nilan m 36 assistant 800 Apr 05-04-16
bavana f 30 tester 900 Dec 02-12-16
shreya f 33 vendor 5000 Nov 06-11-09
Ankit m 28 developer 7000 Jun 11-06-16
Tina f 30 hr 8000 Aug 30-08-16
rasheed m 32 developer 9000 Jul 16-07-16
bavana f 30 tester 10000 Dec 02-12-16
nilakshi f 25 engineer 20000 Feb 23-02-16
praveen m 22 analyst 30000 May 15-05-16
saba f 23 analyst 40000 Jan 22-01-16
sakshi f 36 manager 60000 Mar 22-03-16
ghosh m 29 consultant 80000 Oct 08-10-16

Expected output:

praveen m 22 analyst 30000 May 15-05-16
saba f 23 analyst 40000 Jan 22-01-16
Nilan m 36 assistant 800 Apr 05-04-16
ghosh m 29 consultant 80000 Oct 08-10-16
Ankit m 28 developer 7000 Jun 11-06-16
rasheed m 32 developer 9000 Jul 16-07-16
nilakshi f 25 engineer 20000 Feb 23-02-16
Tina f 30 hr 8000 Aug 30-08-16
sakshi f 42 manager 60000 Sep 05-09-16
bavana f 30 tester 900 Dec 02-12-16
bavana f 30 tester 10000 Dec 02-12-16
shreya f 33 vendor 5000 Nov 06-11-09

syg00 · 01-02-2017, 05:20 PM

Deleted - testing.

syg00 · 01-02-2017, 05:27 PM

Read the manpage.
That specification doesn't match what is documented. You do need to specify 2 keys but your testing obviously isn't very thorough.

grail · 01-03-2017, 01:36 AM

I feel this should have been raised as a new question as it is not related to the current topic

Jjanel · 01-03-2017, 02:55 AM

Your -n is 'syntactically' *not* part of -k5, *when* you put it *before* the -k!
From man sort: KEYDEF is F[.C][OPTS][,F[.C][OPTS]]

Yes, new non- -u thread! p.s. thanks grail, 2x

(unless Title changed to: learning sort command

)

p.s. you have a conflict, tho irrelevant to k4&5, in sakshi age in middle list: 36/42