LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-03-2009, 07:06 PM   #1
ankursinha
LQ Newbie
 
Registered: Feb 2009
Posts: 5

Rep: Reputation: 0
Linux sort


Hi,

I need to sort data numerically by first column and if they are equal then by second column. The columns are separated by spaces and the data looks something like this.

-9.21572e-19 0.812056
0.0275 0.812056
-0.0275 0.814485
0.055 0.812056
-0.055 0.820524
0.0825 0.812056
-0.0825 0.832067
-0.085028 0.833333
0.11 0.812056
-0.11 0.848593
-0.120218 0.85625
0.1375 0.812056

Could someone help me out how to write a script for this. I tried the following but it doesnt seem to work.

sort -t ' ' -k1,1n -k2,2nr file.txt

Thanks,
Anks

Last edited by ankursinha; 02-03-2009 at 07:07 PM.
 
Old 02-03-2009, 08:02 PM   #2
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738
When you say "doesn't work", can you be more specific? i.e. tell us what DOES happen.

I wonder if the minus signs are relevant to what is happening?
 
Old 02-03-2009, 08:23 PM   #3
norobro
Member
 
Registered: Feb 2006
Distribution: Debian Sid
Posts: 515

Rep: Reputation: 176Reputation: 176
From the sort man page:
Quote:
-g, --general-numeric-sort : compare according to general numerical value
I think the number in scientific notation will be a problem though. [ EDIT: Never mind. Posted before trying it]

Last edited by norobro; 02-03-2009 at 08:31 PM.
 
Old 02-03-2009, 09:13 PM   #4
BCarey
Senior Member
 
Registered: Oct 2005
Location: New Mexico
Distribution: Slackware
Posts: 1,596

Rep: Reputation: Disabled
This works in perl:

Code:
#!/usr/bin/perl -w
#
use strict;

my @unsorted;

for (<DATA>) {
	next if /^\s$/;
	push @unsorted, [split /\s/];
}

my @sorted = sort { $a->[0] <=> $b->[0] or $a->[1] <=> $b->[1]} @unsorted;

for (@sorted) {
	print "$_->[0]   $_->[1]\n";
}

__DATA__
-9.21572e-19 0.812056
0.0275 0.812056
-0.0275 0.814485
0.055 0.812056
0.055 0.820524
0.0825 0.812056
-0.0825 0.832067
-0.085028 0.833333
0.11 0.812056
-0.11 0.848593
-0.120218 0.85625
0.1375 0.812056
I slightly changed your data so you would see that it works if the first value is equal.

Brian
 
Old 02-03-2009, 10:16 PM   #5
norobro
Member
 
Registered: Feb 2006
Distribution: Debian Sid
Posts: 515

Rep: Reputation: 176Reputation: 176
Just to clarify my previous post (#3): setting the -g switch works.
The edit was about my comment on the number in scientific notation.

Using Brian's data:
Code:
~/temp: sort -g -k1 -k2 sort.txt
-0.120218 0.85625
-0.11 0.848593
-0.085028 0.833333
-0.0825 0.832067
-0.0275 0.814485
-9.21572e-19 0.812056
0.0275 0.812056
0.055 0.812056
0.055 0.820524
0.0825 0.812056
0.11 0.812056
0.1375 0.812056
 
Old 02-04-2009, 03:36 AM   #6
ankursinha
LQ Newbie
 
Registered: Feb 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Hey Norobro,

Thats exactly the kind of output I want but how did you get it to work.

sort -g -k1 -k2 sort.txt

doesnt help me out. It seems to ignore the negative signs before the numbers. It gives exactly the same output as the numbers in the orginal file without doing any sorting for the file I gave.

I am working on mandriva, I am so annoyed that I am caught in something so trivial and my work is stuck.

I do not wish to go for a C or perl code when I believe this can be done in one line. Please help me out.

Regards,
Anks

Last edited by ankursinha; 02-04-2009 at 03:39 AM.
 
Old 02-04-2009, 03:52 AM   #7
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 670Reputation: 670Reputation: 670Reputation: 670Reputation: 670Reputation: 670
sort -n -t' ' sort.txt

I added an additional line in the data to check second column sorting.
Code:
cat tempfile
-9.21572e-19 0.812056
0.0275 0.812056
-0.0275 0.814485
0.0275 0.110
0.055 0.812056
-0.055 0.820524
0.0825 0.812056
-0.0825 0.832067
-0.085028 0.833333
0.11 0.812056
-0.11 0.848593
-0.120218 0.85625
0.1375 0.812056
[jschiwal@hpamd64 ~]$ sort -n -t' ' tempfile
-9.21572e-19 0.812056
-0.120218 0.85625
-0.11 0.848593
-0.085028 0.833333
-0.0825 0.832067
-0.055 0.820524
-0.0275 0.814485
0.0275 0.110
0.0275 0.812056
0.055 0.812056
0.0825 0.812056
0.11 0.812056
0.1375 0.812056
But if you use scientific notation, you may need to use a generic sort (-g) instead.
 
Old 02-04-2009, 03:54 AM   #8
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
Code:
sort -n -k1,2 sort.txt
Edit: jschiwal beat me!
 
Old 02-04-2009, 05:27 AM   #9
ankursinha
LQ Newbie
 
Registered: Feb 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by pixellany View Post
When you say "doesn't work", can you be more specific? i.e. tell us what DOES happen.

I wonder if the minus signs are relevant to what is happening?
pixellany, you got it right, it has to do something with the minus sign...if I keep all the numbers positive it is working well.

It somehow seems to be ignoring the negative signs while sorting. I tried all the above responses but it doesnt help. Could it be a bug (I am on Mandriva), or some kind of setting needs to be done so that negative signs are not ignored.

Last edited by ankursinha; 02-04-2009 at 05:33 AM.
 
Old 02-04-2009, 08:20 AM   #10
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
Quote:
Originally Posted by ankursinha View Post
I tried all the above responses but it doesnt help.
Can you elaborate, please? Copy and paste what do you see in your terminal, otherwise we cannot be of much help. The following is a working example showing the input, the command, the output:
Code:
$ cat testfile
-9.21572e-19 0.812056
 0.0275      0.812056
-0.0275      0.814485
 0.055       0.812056
-0.055       0.820524
 0.0825      0.812056
-0.0825      0.832067
-0.085028    0.833333
 0.11        0.812056
-0.11        0.848593
-0.120218    0.85625
 0.1375      0.812056
$ sort -n testfile
-9.21572e-19 0.812056
-0.120218    0.85625
-0.11        0.848593
-0.085028    0.833333
-0.0825      0.832067
-0.055       0.820524
-0.0275      0.814485
 0.0275      0.812056
 0.055       0.812056
 0.0825      0.812056
 0.11        0.812056
 0.1375      0.812056
 
Old 02-04-2009, 10:27 AM   #11
norobro
Member
 
Registered: Feb 2006
Distribution: Debian Sid
Posts: 515

Rep: Reputation: 176Reputation: 176
Anks,

As I indicated in my post I used the Brian's (BCarey) data to test for identical first columns. As Colucix pointed out the command can be shortened to:
Code:
sort -g -k1,2 foo
We have a pretty good sampling of distros in this thread where the sort command works. I can't imagine that Mandriva would be any different.

Just a shot in the dark, but could this be your problem? From the sort man page:
Quote:
*** WARNING *** The locale specified by the environment affects sort order. Set LC_ALL=C to get the traditional
sort order that uses native byte values.
 
Old 02-05-2009, 04:20 PM   #12
ankursinha
LQ Newbie
 
Registered: Feb 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Hey Guys,

Here is a screenshot of the shell. See if this can be of any help for you guys in finding the problem.

Thanks,
Anks
Attached Thumbnails
Click image for larger version

Name:	screenshot.png
Views:	46
Size:	89.5 KB
ID:	114  
 
Old 02-05-2009, 05:51 PM   #13
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
I agree with norobro's suggestion. Maybe your current LOCALE affects the sort order. Please try
Code:
env LC_ALL=C sort -n temp.dat
 
Old 02-06-2009, 06:09 AM   #14
ankursinha
LQ Newbie
 
Registered: Feb 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Superb Guys,

It works :-)

Thanks a lot for all your effors and help. I was really banging my head on my comp for this trivial problem.

Anks
 
Old 02-06-2009, 06:58 AM   #15
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
Quote:
Originally Posted by ankursinha View Post
Superb Guys,

It works :-)
Nice to hear the good news! For your information, you can use the command
Code:
locale
to see the current settings based on the language you chose for your computer. They affect the behaviour of many different commands. Even a simple ls can sort entries in a different way if you change your current locale settings. Giving a different environment to some specific commands, using the env method as suggested above, is a very common task in order to prevent strange or unexpected behaviours.

Cheers!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How do I do filtering in Perl (keep sort order and sort again by another means)? RavenLX Programming 9 12-19-2008 11:12 AM
selection sort compiles but does not sort the array as desired ganesha Programming 2 04-20-2008 08:44 AM
New to Linux...sort of trekr200 LinuxQuestions.org Member Intro 0 06-25-2007 05:20 PM
Is there a line limit with the sort utility? Trying to sort 130 million lines of text gruffy Linux - General 4 08-10-2006 09:40 PM


All times are GMT -5. The time now is 01:18 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration