LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 08-09-2006, 03:05 AM   #1
man_linux
LQ Newbie
 
Registered: Aug 2006
Posts: 19

Rep: Reputation: 0
sorting text file - sort command


I am trying to sort a text file with thousands of lines in linux. sort isnt helping much.

problem is this - my file has line numbers in text in random order
And all data is in MySQL format
('line-3','xyz','.....'),
('line-99','abc','.....'),
just the number after 'line-' is changing in random way. i need to sort it.
2
1
8
22
6
4
7
5
10
....... and more

When i sort it, it becomes, I dont want sorting this way.
1
10
2
22
3
4
5
6......


I am expecting -

1
2
3
4
.....
10
....
22


How to do it, what switches in sort can do this? or any other tool which can perform this.

Thanks in advance.

Last edited by man_linux; 08-09-2006 at 04:00 AM.
 
Old 08-09-2006, 03:53 AM   #2
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,986
Blog Entries: 11

Rep: Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880
You may have to split the 'line-<n>' and then sort separately by the
number. Something like
sed 's/line-\([0-9]+'/line \1 /' file | sort -k2,2g


Cheers,
Tink
 
Old 08-09-2006, 04:00 AM   #3
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,245
Blog Entries: 15

Rep: Reputation: 233Reputation: 233Reputation: 233
try 'sort -n'
 
Old 08-09-2006, 04:10 AM   #4
man_linux
LQ Newbie
 
Registered: Aug 2006
Posts: 19

Original Poster
Rep: Reputation: 0
Tink, I tried it, didnt work.
 
Old 08-09-2006, 04:17 AM   #5
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,986
Blog Entries: 11

Rep: Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880
Can you post let's say 10 actual lines of your file?

Sorry, I got that expression wrong.
Code:
sed 's/line-\([0-9]+\)\?/line \1 /' file | sort -k2,2g

Cheers,
Tink
 
Old 08-09-2006, 04:20 AM   #6
man_linux
LQ Newbie
 
Registered: Aug 2006
Posts: 19

Original Poster
Rep: Reputation: 0
cool, worked......

Thanx, is this going to work if i have thousands of lines.

My data is from PIX firewall, I convereted it to MySQL format.

('line-32','permit','udp','host-10.7.85.166',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-33','permit','udp','host-10.7.85.166',' ','host-10.124.21.143','range-7001-7006','(hitcnt=0)',' '),
('line-34','permit','udp','host-10.7.85.166',' ','host-10.124.21.144','range-7001-7006','(hitcnt=0)',' '),
('line-35','permit','udp','host-10.7.85.166',' ','host-10.124.21.145','range-7001-7006','(hitcnt=0)',' '),
('line-36','permit','tcp','host-10.7.67.27',' ','host-10.124.21.139','eq-1882','(hitcnt=583125)',' '),
('line-37','permit','tcp','host-10.7.67.27',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-38','permit','tcp','host-10.7.67.27',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
('line-39','permit','udp','host-10.7.67.27',' ','host-10.124.21.139','eq-1882','(hitcnt=0)',' '),
('line-40','permit','udp','host-10.7.67.27',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-41','permit','udp','host-10.7.67.27',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
('line-42','permit','udp','host-10.7.67.27',' ','host-10.124.21.142','range-7001-7006','(hitcnt=0)',' '),
('line-43','permit','udp','host-10.7.67.27',' ','host-10.124.21.143','range-7001-7006','(hitcnt=0)',' '),
('line-44','permit','udp','host-10.7.67.27',' ','host-10.124.21.144','range-7001-7006','(hitcnt=0)',' '),
('line-45','permit','udp','host-10.7.85.166',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),


Thanx
 
Old 08-09-2006, 04:25 AM   #7
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,986
Blog Entries: 11

Rep: Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880
Yes, it is ... it's working.

Code:
]$ cat sortme.txt 
('line-32','permit','udp','host-10.7.85.166',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-38','permit','tcp','host-10.7.67.27',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
('line-43','permit','udp','host-10.7.67.27',' ','host-10.124.21.143','range-7001-7006','(hitcnt=0)',' '),
('line-33','permit','udp','host-10.7.85.166',' ','host-10.124.21.143','range-7001-7006','(hitcnt=0)',' '),
('line-36','permit','tcp','host-10.7.67.27',' ','host-10.124.21.139','eq-1882','(hitcnt=583125)',' '),
('line-37','permit','tcp','host-10.7.67.27',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-39','permit','udp','host-10.7.67.27',' ','host-10.124.21.139','eq-1882','(hitcnt=0)',' '),
('line-40','permit','udp','host-10.7.67.27',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-35','permit','udp','host-10.7.85.166',' ','host-10.124.21.145','range-7001-7006','(hitcnt=0)',' '),
('line-41','permit','udp','host-10.7.67.27',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
('line-34','permit','udp','host-10.7.85.166',' ','host-10.124.21.144','range-7001-7006','(hitcnt=0)',' '),
('line-42','permit','udp','host-10.7.67.27',' ','host-10.124.21.142','range-7001-7006','(hitcnt=0)',' '),
('line-44','permit','udp','host-10.7.67.27',' ','host-10.124.21.144','range-7001-7006','(hitcnt=0)',' '),
('line-45','permit','udp','host-10.7.85.166',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
$
$ sed 's/line-\([0-9]+\)\?/line \1 /' sortme.txt | sort -k2,2g
('line  32','permit','udp','host-10.7.85.166',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line  33','permit','udp','host-10.7.85.166',' ','host-10.124.21.143','range-7001-7006','(hitcnt=0)',' '),
('line  34','permit','udp','host-10.7.85.166',' ','host-10.124.21.144','range-7001-7006','(hitcnt=0)',' '),
('line  35','permit','udp','host-10.7.85.166',' ','host-10.124.21.145','range-7001-7006','(hitcnt=0)',' '),
('line  36','permit','tcp','host-10.7.67.27',' ','host-10.124.21.139','eq-1882','(hitcnt=583125)',' '),
('line  37','permit','tcp','host-10.7.67.27',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line  38','permit','tcp','host-10.7.67.27',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
('line  39','permit','udp','host-10.7.67.27',' ','host-10.124.21.139','eq-1882','(hitcnt=0)',' '),
('line  40','permit','udp','host-10.7.67.27',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line  41','permit','udp','host-10.7.67.27',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
('line  42','permit','udp','host-10.7.67.27',' ','host-10.124.21.142','range-7001-7006','(hitcnt=0)',' '),
('line  43','permit','udp','host-10.7.67.27',' ','host-10.124.21.143','range-7001-7006','(hitcnt=0)',' '),
('line  44','permit','udp','host-10.7.67.27',' ','host-10.124.21.144','range-7001-7006','(hitcnt=0)',' '),
('line  45','permit','udp','host-10.7.85.166',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),

Cheers,
Tink
 
Old 08-09-2006, 04:34 AM   #8
man_linux
LQ Newbie
 
Registered: Aug 2006
Posts: 19

Original Poster
Rep: Reputation: 0
I dont have any file of more than thousand lines, just doing test on small PIX access list file to put in DB. Hope, it wont fail when i get my next big file.

It just replaced "-" with " ", but that ok. easy to fix.

Very Thanx to you.
 
Old 08-09-2006, 05:03 AM   #9
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,986
Blog Entries: 11

Rep: Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880
I HAD to replace the "-" :} ... otherwise the numeric sort won't work on
lines-<number>, and you'll see what you had, sorting by ASCII values.

You can easily just pipe the result through sed again after the sort.
And I realised the sed instruction was too complex ;} ... bedtime.
Code:
sed 's/line-/line /' sortme.txt | sort -k2,2g | sed 's/line /line-/'

Cheers,
Tink
 
Old 08-09-2006, 05:13 AM   #10
spirit receiver
Member
 
Registered: May 2006
Location: Frankfurt, Germany
Distribution: SUSE 10.2
Posts: 424

Rep: Reputation: 33
Seems like the pattern matching is a little muddled here. I guess Tink originally wanted to use this:
Code:
sed 's/line-\([0-9]\+\)\?/line \1 /' kram | sort -k2,2g
You can also get rid of the sed command using
Code:
sort -t"-" -k2,2 -n filename
 
Old 08-09-2006, 05:43 AM   #11
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,245
Blog Entries: 15

Rep: Reputation: 233Reputation: 233Reputation: 233
this is weird. sort -n works well for me.

sort --version
Code:
sort (GNU coreutils) 5.94
cat sort.txt
Code:
('line-32','permit','udp','host-10.7.85.166',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-38','permit','tcp','host-10.7.67.27',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
('line-43','permit','udp','host-10.7.67.27',' ','host-10.124.21.143','range-7001-7006','(hitcnt=0)',' '),
('line-33','permit','udp','host-10.7.85.166',' ','host-10.124.21.143','range-7001-7006','(hitcnt=0)',' '),
('line-36','permit','tcp','host-10.7.67.27',' ','host-10.124.21.139','eq-1882','(hitcnt=583125)',' '),
('line-37','permit','tcp','host-10.7.67.27',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-39','permit','udp','host-10.7.67.27',' ','host-10.124.21.139','eq-1882','(hitcnt=0)',' '),
('line-40','permit','udp','host-10.7.67.27',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-35','permit','udp','host-10.7.85.166',' ','host-10.124.21.145','range-7001-7006','(hitcnt=0)',' '),
('line-41','permit','udp','host-10.7.67.27',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
('line-34','permit','udp','host-10.7.85.166',' ','host-10.124.21.144','range-7001-7006','(hitcnt=0)',' '),
('line-42','permit','udp','host-10.7.67.27',' ','host-10.124.21.142','range-7001-7006','(hitcnt=0)',' '),
('line-44','permit','udp','host-10.7.67.27',' ','host-10.124.21.144','range-7001-7006','(hitcnt=0)',' '),
('line-45','permit','udp','host-10.7.85.166',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
cat sort.txt | sort -n
Code:
('line-32','permit','udp','host-10.7.85.166',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-33','permit','udp','host-10.7.85.166',' ','host-10.124.21.143','range-7001-7006','(hitcnt=0)',' '),
('line-34','permit','udp','host-10.7.85.166',' ','host-10.124.21.144','range-7001-7006','(hitcnt=0)',' '),
('line-35','permit','udp','host-10.7.85.166',' ','host-10.124.21.145','range-7001-7006','(hitcnt=0)',' '),
('line-36','permit','tcp','host-10.7.67.27',' ','host-10.124.21.139','eq-1882','(hitcnt=583125)',' '),
('line-37','permit','tcp','host-10.7.67.27',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-38','permit','tcp','host-10.7.67.27',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
('line-39','permit','udp','host-10.7.67.27',' ','host-10.124.21.139','eq-1882','(hitcnt=0)',' '),
('line-40','permit','udp','host-10.7.67.27',' ','host-10.124.21.140','eq-1882','(hitcnt=0)',' '),
('line-41','permit','udp','host-10.7.67.27',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
('line-42','permit','udp','host-10.7.67.27',' ','host-10.124.21.142','range-7001-7006','(hitcnt=0)',' '),
('line-43','permit','udp','host-10.7.67.27',' ','host-10.124.21.143','range-7001-7006','(hitcnt=0)',' '),
('line-44','permit','udp','host-10.7.67.27',' ','host-10.124.21.144','range-7001-7006','(hitcnt=0)',' '),
('line-45','permit','udp','host-10.7.85.166',' ','host-10.124.21.141','eq-1882','(hitcnt=0)',' '),
 
Old 08-09-2006, 05:56 AM   #12
man_linux
LQ Newbie
 
Registered: Aug 2006
Posts: 19

Original Poster
Rep: Reputation: 0
Tried on many files, working very well......
 
Old 08-09-2006, 06:49 AM   #13
spirit receiver
Member
 
Registered: May 2006
Location: Frankfurt, Germany
Distribution: SUSE 10.2
Posts: 424

Rep: Reputation: 33
Quote:
Originally Posted by konsolebox
this is weird. sort -n works well for me.
All those numbers have two digits, that's why a simple asciibetical sort will do. But that's not what we're looking for. Try inserting line-123 into the data.
 
Old 08-09-2006, 07:20 AM   #14
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,245
Blog Entries: 15

Rep: Reputation: 233Reputation: 233Reputation: 233
yah i guess you're right
 
Old 08-09-2006, 01:18 PM   #15
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,986
Blog Entries: 11

Rep: Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880
Quote:
Originally Posted by spirit receiver
You can also get rid of the sed command using
Code:
sort -t"-" -k2,2 -n filename
I must say that I'm surprised that this actually works. After all, the compared
values are
Code:
32','permit','udp','host
and so forth.... which is not really numerical.

Oh well :}


Cheers,
Tink
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Sorting large text files tmaxx AIX 14 02-19-2009 06:32 PM
Advanced Linux Sort text file Command soulxcavtor Linux - Software 1 06-07-2006 10:58 PM
How to run a text file through a command prompt? Fillys6 Linux - Newbie 8 03-24-2005 02:21 PM
Command to find file with text chamanrana Linux - Software 10 12-02-2004 10:47 AM
Reverse Sort Text File BxBoy Linux - General 1 08-02-2004 10:13 AM


All times are GMT -5. The time now is 01:08 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration