LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 10-08-2013, 03:08 PM   #1
scream
Member
 
Registered: Feb 2008
Distribution: Oracle Linux
Posts: 32

Rep: Reputation: 0
CSV formatting


Hi,
i have a csv like below:

USA
1 newyork
2 wasington
UK
1 London
2 Manchester
Japan
1 Tokyo

i need to format it like this:
1 USA Newyork
2 USA wasington
3 UK LONDOn
4 UK Manchester
5 Japan Tokyo

preferable lang shell script. can anyone please help me.............
 
Old 10-08-2013, 03:33 PM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191
Well I am not seeing many c's in the csv, but that aside, what have you done so far to solve the problem?
 
1 members found this post helpful.
Old 10-08-2013, 04:30 PM   #3
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,881

Rep: Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660
With this InFile ...
Code:
USA
1 New York
2 Washington
3 Chicago
4 Los Angeles
UK
1 London
2 Manchester
Japan
1 Tokyo
... this awk ...
Code:
awk '{if ($0!~/[0-9]/) country=$0; else {$1=$1" "country; print}}' $InFile >$OutFile
... produced this OutFile ...
Code:
1 USA New York
2 USA Washington
3 USA Chicago
4 USA Los Angeles
1 UK London
2 UK Manchester
1 Japan Tokyo
Daniel B. Martin
 
Old 10-12-2013, 01:32 PM   #4
scream
Member
 
Registered: Feb 2008
Distribution: Oracle Linux
Posts: 32

Original Poster
Rep: Reputation: 0
thanks daniel. can you please help me to remove trailing "," from the line like below:

2013-10-13 06:00:00, USA, NEWYORK,123,,,,,,,,,,,,
 
Old 10-12-2013, 02:02 PM   #5
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,881

Rep: Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660
Quote:
Originally Posted by scream View Post
thanks daniel. can you please help me to remove trailing "," from the line like below:

2013-10-13 06:00:00, USA, NEWYORK,123,,,,,,,,,,,,
Try this ...
Code:
sed 's/,*$//'
Daniel B. Martin
 
Old 10-13-2013, 04:09 PM   #6
scream
Member
 
Registered: Feb 2008
Distribution: Oracle Linux
Posts: 32

Original Poster
Rep: Reputation: 0
thanks again daniel. i have used "sed -i -e 's/,*$//' OutFile" but didnt got any changes
 
Old 10-13-2013, 04:19 PM   #7
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,634

Rep: Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965
Quote:
Originally Posted by scream View Post
thanks again daniel. i have used "sed -i -e 's/,*$//' OutFile" but didnt got any changes
Well, as grail asked you: can you show us what YOU have done/tried to solve this?? Can you not experiment and think about the solutions/tools given to you so far, to find the solution on your own?

That's the point of homework, isn't it?
 
Old 10-13-2013, 07:14 PM   #8
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,881

Rep: Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660
Quote:
Originally Posted by scream View Post
i have used "sed -i -e 's/,*$//' OutFile" but didnt got any changes
Take a close look at your code. The sed needs some kind of input to chew on. Where did you give it the input?

Daniel B. Martin
 
Old 10-14-2013, 01:55 PM   #9
scream
Member
 
Registered: Feb 2008
Distribution: Oracle Linux
Posts: 32

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by TB0ne View Post
Well, as grail asked you: can you show us what YOU have done/tried to solve this?? Can you not experiment and think about the solutions/tools given to you so far, to find the solution on your own?

That's the point of homework, isn't it?

Dear TB0ne,
i am trying to develop a script to format a csv file (cu.csv) and load into database. my file look like below

cu.csv
(group--121 XYZ) ,,,,,,,,,,,,,,,,,,,,,,,,,
2013-09-12 01:00:00+06:00,9.95,20.61,463,2.15,4.45,6.60,1,15,,,,,,,,,,,,,,,,,
2013-09-12 02:00:00+06:00,5.1,13.09,463,1.10,2.83,3.93,1,15,,,,,,,,,,,,,,,,,
2013-09-12 03:00:00+06:00,2.36,7.88,463,0.51,1.70,2.21,0,15,,,,,,,,,,,,,,,,,
2013-09-12 04:00:00+06:00,1.65,6.21,463,0.36,1.34,1.70,0,15,,,,,,,,,,,,,,,,,

i need the output (NFILE) like

NFILE
2013-09-12 01:00:00 121 9.95 20.61 463 2.15 4.45 6.60 1 15
2013-09-12 02:00:00 121 5.1 13.09 463 1.10 2.83 3.93 1 15

I have used below code to format the output

awk '{if ($1!~/[0-9]/) c=$2; else {$1=$1","c","; print $0}}' cu.csv >OutFile
sed -i -e 's/group--//g' OutFile
sed -i -e 's/ //g' OutFile
sed -i -e 's/+06:00//g' OutFile

after all this till now i got

(OutFile)
2013-09-12,121,01:00:00,9.95,20.61,463,2.15,4.45,6.60,1,15,,,,,,,,,,,,,,,,,
2013-09-12,121,02:00:00,5.1,13.09,463,1.10,2.83,3.93,1,15,,,,,,,,,,,,,,,,,
2013-09-12,121,03:00:00,2.36,7.88,463,0.51,1.70,2.21,0,15,,,,,,,,,,,,,,,,,
2013-09-12,121,04:00:00,1.65,6.21,463,0.36,1.34,1.70,0,15,,,,,,,,,,,,,,,,,

i need to remove the "," and re-arrange the fields. i tried below code but 1st row remains same

awk '{FS=",";print $1"\t"$3"\t"$2"\t"$4"\t"$5"\t"$6"\t"$7"\t"$8"\t"$9"\t"$10"\t"$11"\t"$12;}' OutFile >NFile

(NFile)
2013-09-12,121,01:00:00,9.95,20.61,463,2.15,4.45,6.60,1,15,,,,,,,,,,,,,,,,,
2013-09-12 02:00:00 121 5.1 13.09 463 1.10 2.83 3.93 1 15
2013-09-12 03:00:00 121 2.36 7.88 463 0.51 1.70 2.21 0 15
2013-09-12 04:00:00 121 1.65 6.21 463 0.36 1.34 1.70 0 15

can you tell me , where i missed something?

---------- Post added 10-15-13 at 12:56 AM ----------

Quote:
Originally Posted by danielbmartin View Post
Take a close look at your code. The sed needs some kind of input to chew on. Where did you give it the input?

Daniel B. Martin
Martin
Outfile is the input file. i tried to modify that.
 
Old 10-14-2013, 02:16 PM   #10
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,881

Rep: Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660
Quote:
Originally Posted by scream View Post
Outfile is the input file. i tried to modify that.
Starting with InFile2 ...
Code:
2013-09-12,121,01:00:00,9.95,20.61,463,2.15,4.45,6.60,1,15,,,,,,,,,,,,,,,,,
2013-09-12,121,02:00:00,5.1,13.09,463,1.10,2.83,3.93,1,15,,,,,,,,,,,,,,,,,
2013-09-12,121,03:00:00,2.36,7.88,463,0.51,1.70,2.21,0,15,,,,,,,,,,,,,,,,,
2013-09-12,121,04:00:00,1.65,6.21,463,0.36,1.34,1.70,0,15,,,,,,,,,,,,,,,,,
... this code ...
Code:
   Path=$(cut -d'.' -f1 <<< ${0})
InFile2=$Path"inp2.txt"
sed -i -e 's/,*$//' $InFile2
... changed InFile2 to this ...
Code:
2013-09-12,121,01:00:00,9.95,20.61,463,2.15,4.45,6.60,1,15
2013-09-12,121,02:00:00,5.1,13.09,463,1.10,2.83,3.93,1,15
2013-09-12,121,03:00:00,2.36,7.88,463,0.51,1.70,2.21,0,15
2013-09-12,121,04:00:00,1.65,6.21,463,0.36,1.34,1.70,0,15
This code snippet assumes you have the files InFile2 and the script itself in the same folder.

Daniel B. Martin
 
Old 10-14-2013, 03:08 PM   #11
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian sid
Posts: 2,683

Rep: Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783
Code:
awk 'BEGIN{FS=","}{print $1"\t"$3"\t"$2"\t"$4"\t"$5"\t"$6"\t"$7"\t"$8"\t"$9"\t"$10"\t"$11"\t"$12;}' OutFile >NFile
if you don't mind trailing tabs

Code:
awk 'BEGIN{FS=",";OFS="\t"}{t=$3;$3=$2;$2=t;print}' OutFile
if you don't want the trailing tabs

Code:
awk 'BEGIN{FS=","}{t=$3;$3=$2;$2=t}{printf $1}{for (i=2;i<12;i++)printf "\t"$i}{printf "\n"}' OutFile
 
Old 10-16-2013, 02:29 PM   #12
scream
Member
 
Registered: Feb 2008
Distribution: Oracle Linux
Posts: 32

Original Poster
Rep: Reputation: 0
thanks Firerat and martin. my problem solved.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
need help on csv formatting linuxunix Linux - Newbie 13 03-16-2015 07:23 PM
[SOLVED] A challenging script - Replace field of CSV file based on another CSV file arbex5 Programming 11 06-12-2013 06:56 AM
[SOLVED] How to script csv editing? Remove rows from csv file that do not contain certain text ingram87 Linux - Software 9 08-03-2012 12:45 PM
Using awk/sed to convert linefeed to csv, with some formatting jaykup Programming 1 04-03-2009 05:18 PM
Comparing two csv files and write different record in third CSV file irfanb146 Linux - Newbie 3 06-30-2008 09:15 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 04:58 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration