LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-11-2018, 12:15 AM   #1
amy441
LQ Newbie
 
Registered: Jun 2018
Posts: 10

Rep: Reputation: Disabled
Question Separate Column Using SED.


Hope you guys are doing well.

I'm stuck with separating column using SED. Here are two entries of my CSV file and I need help in separating "Organization Location" column into two columns "Organization City" and "Organization State".

Can someone please help me?

Year Rank Organization ID Organization Name Organization Location Private Income
2004 1 321092 Salvation Army Alexandria, Va. $1,324,089,000
2004 2 321148 American Cancer Society Alexandria, Va. $794,000,000
 
Old 06-11-2018, 12:33 AM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,321
Blog Entries: 3

Rep: Reputation: 3725Reputation: 3725Reputation: 3725Reputation: 3725Reputation: 3725Reputation: 3725Reputation: 3725Reputation: 3725Reputation: 3725Reputation: 3725Reputation: 3725
Welcome.

sed may not be the best language for the task and maybe not AWK either. But if the file has tabs between the fields, you might get by with either. If not, then perl5 with Text::CSV would be the way to parse it.

So, two questions: are their tabs between the fields and can you show us what you have so far written in sed?
 
Old 06-11-2018, 12:41 AM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,130

Rep: Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121Reputation: 4121
Doesn't much look like a CSV - but that dollar field might make a mess of things. Let's see some real data - anonymised would be a good idea.
Agree sed is the wrong tool, but awk should be usable if you know it.
 
Old 06-11-2018, 01:33 AM   #4
amy441
LQ Newbie
 
Registered: Jun 2018
Posts: 10

Original Poster
Rep: Reputation: Disabled
Thanks for your reply guys, I need a solution only using SED.

Please check attached sample data screenshot so you will get some more idea :- https://prnt.sc/jtib1t

Thanks again.

-Amy
 
Old 06-11-2018, 02:17 AM   #5
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,895

Rep: Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317
why do you need sed? is this a homework?
In your example state is missing in line 3. How do you handle that case?
Is this really a CSV (comma separated values) file?
 
Old 06-11-2018, 02:17 AM   #6
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.9.2009
Posts: 5,732

Rep: Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212
Can you show us the data in the file directly, as with cat or less, rather than as it's shown in a spreadsheet program?
The spreadsheet program may be formatting data to display it by adding dollar signs and commas to the numeric data, etc.

And please explain why sed is a requirement. Is this homework?
[simulpost]

Last edited by scasey; 06-11-2018 at 02:18 AM.
 
Old 06-11-2018, 02:20 AM   #7
AwesomeMachine
LQ Guru
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524

Rep: Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015
It sounds like homework. What have you tried so far? You would need to nest awk statements in sed. But if you can't use awk, that won't work. If you are able to use more than one iteration sed might be feasible.
 
Old 06-11-2018, 02:27 AM   #8
amy441
LQ Newbie
 
Registered: Jun 2018
Posts: 10

Original Poster
Rep: Reputation: Disabled
This is an output of cat abc.csv file command. This is actually a homework given by my teacher and he specifically mentioned to split the column using SED.

Year,Rank,Organization ID,Organization Name,Organization Location,Private Income,Total Assets,Service Expense,Fundraising Expense
2004,1,321092,Salvation Army,"Alexandria, Va.","$1,324,089,000","$3,039,922,000","$2,126,200,000","$123,791,000"
2004,2,321148,American Cancer Society,Atlanta,"$794,000,000","$836,295,000","$610,639,000","$188,150,000"
2004,3,321036,Gifts In Kind International,"Alexandria, Va.","$787,192,199","$790,561,819","$792,432,766","$167,242"
 
Old 06-11-2018, 02:34 AM   #9
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.9.2009
Posts: 5,732

Rep: Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212
OK. That's your work to do, then.
We may help if you get stuck, but you need to try first. Take a shot. See what happens.

Know that
Code:
sed <script> file
will not change the contents of the file, but will only display the result on the terminal, so there's no harm in trying anything.

Last edited by scasey; 06-11-2018 at 02:35 AM.
 
1 members found this post helpful.
Old 06-11-2018, 02:45 AM   #10
amy441
LQ Newbie
 
Registered: Jun 2018
Posts: 10

Original Poster
Rep: Reputation: Disabled
Hi Sean,

Thank you for your reply!

I did try but can you please provide me a hint how to get the exact column using SED?
 
Old 06-11-2018, 02:49 AM   #11
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.9.2009
Posts: 5,732

Rep: Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212Reputation: 2212
Show us what you tried...
 
Old 06-11-2018, 02:53 AM   #12
amy441
LQ Newbie
 
Registered: Jun 2018
Posts: 10

Original Poster
Rep: Reputation: Disabled
I tried this command:- sed -n '1p'|awk '{print $5}' abc.csv but it's not providing correct output:-
 
Old 06-11-2018, 02:57 AM   #13
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,895

Rep: Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317Reputation: 7317
it is not trivial, because commas in that file are not always separators, but part of the string (like "$123,791,000").
I would try something like:
1. replace comma within " " with : or ; or something else - but probably not required
2. find the 5th field
3. split it
4. restore the original comma
 
1 members found this post helpful.
Old 06-11-2018, 03:04 AM   #14
amy441
LQ Newbie
 
Registered: Jun 2018
Posts: 10

Original Poster
Rep: Reputation: Disabled
Can you please explain little bit more?
 
Old 06-11-2018, 03:15 AM   #15
AwesomeMachine
LQ Guru
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,524

Rep: Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015
Hi amy441,

The reason your instructor told you to use sed is because he wants you to learn the sed command. It was not because he wanted you to learn only how to do the assignment. Would you like to go to a doctor who only knows the examples given in class? Or would you rather go to a doctor who knows the practice of medicine.

If we simply tell you the answer, the whole purpose of school is lost. You won't learn anything. So, read the sed man page. You will probably need to use more than one command line, so you can write a script.
 
1 members found this post helpful.
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] separate information with \1 \2 in sed vincix Linux - Newbie 12 05-02-2017 04:41 AM
How do I separate columns using sed Basse1 Programming 4 07-01-2010 12:54 PM
File name in separate column jeesun Linux - General 5 04-26-2010 05:27 AM
sed/awk group on first column Eddie Adams Linux - General 4 04-09-2009 10:23 AM
Sed column replacement. keysorsoze Programming 11 12-01-2008 09:50 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:35 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration