LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 08-25-2009, 02:39 AM   #1
lothario
Member
 
Registered: Apr 2004
Posts: 340

Rep: Reputation: 30
bash script to sort data by field


I have some very large data files that look like this:

"blue","right","mhduei573","0","0"
"pink","left","bvop4967","0","0"
"yellow,"top","zpbm2368","0","0"
"green","right","cdw432lmjt","0","0"
"purple,"left","plmnhy3zd21","0","0"
"white","top","yg36c0wxqa","0","0"
"blue","left","bvop4967","0","0"
"pink","top","yg36c0wxqa","0","0"
"gray,"right","bvop4967","0","0"

Each field has quotes around it and there is a comma between fields.

I would like to sort this data by the 3rd field then the 1st field, using a bash script.
How do I do that?
 
Old 08-25-2009, 02:44 AM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,453

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
No need for bash script, just a one-liner with the proper options:
Code:
sort -t, -k3 -k1 file
 
Old 08-25-2009, 01:09 PM   #3
lothario
Member
 
Registered: Apr 2004
Posts: 340

Original Poster
Rep: Reputation: 30
sort -t, -k3 -k1 ./a1.txt
works perfectly on this file:

"blue","right","mhduei573","0","0"
"pink","left","bvop4967","0","0"
"yellow,"top","zpbm2368","0","0"
"green","right","cdw432lmjt","0","0"
"purple,"left","plmnhy3zd21","0","0"
"white","top","yg36c0wxqa","0","0"
"blue","left","bvop4967","0","0"
"pink","top","yg36c0wxqa","0","0"
"gray,"right","bvop4967","0","0"

It results in:

"blue","left","bvop4967","0","0"
"gray,"right","bvop4967","0","0"
"pink","left","bvop4967","0","0"
"green","right","cdw432lmjt","0","0"
"blue","right","mhduei573","0","0"
"purple,"left","plmnhy3zd21","0","0"
"pink","top","yg36c0wxqa","0","0"
"white","top","yg36c0wxqa","0","0"
"yellow,"top","zpbm2368","0","0"

Exactly what I needed.

To take it a step further, I need to group the data by the 3rd field.
So that it adds a blank line every time there is a change in the 3rd field.
How do you make it look like this?

"blue","left","bvop4967","0","0"
"gray,"right","bvop4967","0","0"
"pink","left","bvop4967","0","0"

"green","right","cdw432lmjt","0","0"

"blue","right","mhduei573","0","0"

"purple,"left","plmnhy3zd21","0","0"

"pink","top","yg36c0wxqa","0","0"
"white","top","yg36c0wxqa","0","0"

"yellow,"top","zpbm2368","0","0"
 
Old 08-25-2009, 01:13 PM   #4
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,962
Blog Entries: 11

Rep: Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865
Code:
sort -t, -k3 -k1 file |awk -F, '{if($3 != hold){print ""}; hold=$3;print $0}'
I'm too lazy to strip the first empty line ;}



Cheers,
Tink
 
Old 08-26-2009, 02:23 AM   #5
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,453

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Quote:
Originally Posted by Tinkster View Post
I'm too lazy to strip the first empty line ;}
Just a slight modification: NR > 1
Code:
sort -t, -k3 -k1 file | awk -F, 'NR>1{if($3 != hold)print ""}{hold=$3; print}'
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Can I use GNU sort to sort one field in order, another in reverse? zombieite Linux - Newbie 4 04-27-2009 12:23 AM
Need help with bash script /simplified sort command/ szcs Linux - General 0 10-26-2008 11:40 AM
bash script to sort mp3 bitrates above and below 128k Damoek Linux - Newbie 3 03-25-2008 10:15 AM
Bash script to sort image files dtcs Programming 5 09-26-2006 09:50 PM
bash script to sort files by extension otheralex Programming 7 08-19-2005 02:40 AM


All times are GMT -5. The time now is 05:20 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration