Hello folks, i am on hp-ux box running korn shell. I am trying to sort a file on a particular column as well i need to make it unique. I have tried so many different things such as:

cat goodtest|sort -u -k 2,2
cat goodtest|sort -u +1

nothings seems to work, i need the second column to be unique.i am still getting duplicates. Here is my file. Please help me

/dev/rdsk/c4t1d3 06BD 09A:1 15A:C2 RAID-5 Grp'd (M) RW 34526
/dev/rdsk/c6t1d3 06BD 09A:1 15A:C2 RAID-5 Grp'd (M) RW 34526
/dev/rdsk/c8t1d3 06BD 08A:1 15A:C2 RAID-5 Grp'd (M) RW 34526
/dev/rdsk/c10t1d3 06BD 08A:1 15A:C2 RAID-5 Grp'd (M) RW 34526

Form #1 works fine on my MEPIS 3.3.2 GNU/Linux box running bash, although I would have used the form: Are you using a different ver. of sort?:Sorry, I pretty much use only bash & definitely only GNU/Linux, so I have no idea if you're having a korn or hp-ux problem.

where $2 is the second column

And yes, his sort is probably not the GNU version - he's on hpux.

strange ..but this is what i get, for some reason it removes two instances of 06BD ..but leaves two there even though i did -u. Any ideas guys?

# sort -uk 2,2 goodtest
/dev/rdsk/c4t1d3 06BD 09A:1 15A:C2 RAID-5 Grp'd (M) RW 34526
/dev/rdsk/c10t1d3 06BD 08A:1 15A:C2 RAID-5 Grp'd (M) RW 34526

Jim,

your example did work , I've used awk before ..mostly something like this: "cat filename|awk '{print $2}'". Can you explain the syntax of your command. Thank you very much ..

Yes Jim, please explain.

FWIW, I think this is what's happening:
x[] is an array.
Its indices are values of $2.
Every time it "sees" a $2, the '++' increments the value associated w/ that index, creating a new array element if necessary.
This works because awk arrays can have non-numeric indexing, like a hash.
This may seem backward -- the indices are strings & the array elements are numbers.

The part that I 'm not sure about is why the logical negation, '!', makes it work -- w/ it in, only the 1st instance of a value for $2 prints; remove it, & everything after the 1st instance prints. I suspect that the '!' is operating on the logical value of the array element "x[$2]", which, before it is created by the '++', is false. See: http://www.gnu.org/software/gawk/man...l#Truth-Values. Note: only the trailing '++', "post-decrement", will work.
I believe he is also relying on an implicit "print $0" when no other action is specified.

If you have just '<condition> ' as some testable value (true/false), when true, then awk prints $0 - the default action.

awk supports associative arrays. x[] is an array - the test is
if !x[column value] then print $0. The ++ then increments the element. From then on x[column value] is always non-zero.

Jim,

thank you very much for taking the time to explain this. My programming skills are rather weak. So "!x[column value] then print $0" means that the first time around x is nothing so when x is compared against the value in [column value] it will be true since x is not equal to 06BD hence it goes to stdout. Now the second time around did x get assigned the value from [column value] and now when x is tested against [column value] it equal to 06BD and therefore is not printed to stdout?

Thank you again

Not at all.

x is an array.

x[$2] is an element of that array -- one such element is created for each unique $2.

In addition to being a field value in your input, each $2 is an index of the array.

The value of the array element x[$2] is the number of lines that contain that particular $2. This results from x[$2] being incremented each time the script reads a line that contains that particular $2.

The first time the script reads a line containing some new $2, x[$2] tests false because it is empty, as yet undefined. (It is then incremented to '1', after it is tested). The '!' negates the false to true, & the std. awk default action, print $0, is performed. (Print $0 means print the whole line).

The values of the array x are not the values of $2, but the number of occurrences of those values. Even though they are strings, the $2's are indices (names) of the elements of x.

archtoad6 ..i think i need to do more reading on associative arrays and awk. I am still struggling to understand. But i was playing around with sort and found this command to work, althogh i am still not clear what "2.2b" option does. I know that the first 2 means second filed but have not clue about 2b. Here is what worked for me:

sort -k 2.2b,2 -u filename

Thanks

In GNU/Linux, & not necessarily on your HP-UX, Would mean "Sort <filename> on the 2nd field only, starting w/ the 2nd character (origin 1), ignoring blanks; & show only unique lines". To see if it might be different for you, I suggest you: a) post the ver. of your sort & b) check your man page -- it may be different from mine.