Visit Jeremy's Blog.
Go Back > Forums > Linux Forums > Linux - General
User Name
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.


  Search this Thread
Old 04-17-2009, 07:50 PM   #1
LQ Newbie
Registered: Apr 2009
Posts: 2

Rep: Reputation: 0
Make sort ignore comment lines

I have a file in the following format

# comment 1
# comment 2
0 0 BBAAC 16
0 16 AACAD 16
0 32 ACDAC 16
I want to sort(1) it by the third column as string, which can be easily done by

sort --key=3 my_file
However, after sort, the first two comment lines may not be still at the head of the file. Is there a way to tell sort(1) to ignore the comment lines?

I assume there is no such convenient way. Then what I did is to
1) grep the comment lines to my_comments
2) delete the comment lines from my_file
3) sort my_file
4) concatenate my_comments with my_file

Step 1 can be easily done by

head -n 20 my_file | grep \# > my_comments
assuming there is always no more than 20 comment lines.

Step 2 is the source of headache. There seems no way to truncate file at head. I need to grep or sed my_file to a temporary new file, and sort the temporary file instead in step 3, right? If my_file is large, say 1GB, this will exhibit high performance and storage overhead. Is there in-place way?

Step 4, how to do it quickly without commands like

cat my_comments temp_file > my_file

In one word, is there a way to do in-place sort by taking advantage of knowing that the comment lines are few and only appear at head?

Thank you very much!
Old 04-17-2009, 08:56 PM   #2
Senior Member
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
grep -v "^#" file |sort ...
Old 04-17-2009, 09:38 PM   #3
Senior Member
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,186

Rep: Reputation: 346Reputation: 346Reputation: 346Reputation: 346
Or, if you don't have other comments in the file:

grep /^#/ file >/tmp/head;grep -v /^#/ | sort ... >/tmp/body; cat /tmp/head /tmp/body;rm -f /tmp/head /tmp/body

Note: Untested code.
Old 04-17-2009, 10:44 PM   #4
LQ Newbie
Registered: Apr 2009
Posts: 2

Original Poster
Rep: Reputation: 0
Thanks, folks. After having some understanding about sort's algorithm, I know that directly calling sort and grep | sort has no performance difference.

I noticed that zsh has ability to concatenate files with redirection (using >>). Thus the script can be

grep "^#" my_file > my_sorted
grep -v "^#" my_file | sort >> my_sorted
and the sort check and be done by

if [[ -n $(grep -v "^#" my_file | sort --check 2>&1) ]]; then
Perfect! Linux is really fascinating...

Last edited by CrendKing; 04-17-2009 at 11:30 PM.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Comment lines 1-9, 11-19, 21-29 using Sed kushalkoolwal Programming 8 02-03-2009 04:02 PM
Comment/uncomment lines? Romanus81 Programming 4 03-31-2008 08:25 PM
listing non-comment lines scbops Linux - Newbie 5 09-18-2007 08:55 PM
Is there a line limit with the sort utility? Trying to sort 130 million lines of text gruffy Linux - General 4 08-10-2006 08:40 PM
awk/gawk/sed - read lines from file1, comment out or delete matching lines in file2 rascal84 Linux - General 1 05-24-2006 09:19 AM

All times are GMT -5. The time now is 04:17 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration