LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 07-07-2010, 03:50 AM   #1
shakes82
LQ Newbie
 
Registered: Jun 2010
Posts: 21

Rep: Reputation: 15
Help using awk,sed and grep


Hi. I have some financial data in the the following format:

20090302 18:02:03 1.5 1.6

I want to change this to the following format using grep awk and sed:

20090302180203,1.5,1.6,SYM

Please suggest how I can use the commands to get this formatting.

Thank you.
 
Old 07-07-2010, 03:56 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

Something like this?

awk -F"[ :]" '{ print $1 $2 $3 $4","$5","$6",SYM" }' infile

Hope this helps.
 
Old 07-07-2010, 04:10 AM   #3
shakes82
LQ Newbie
 
Registered: Jun 2010
Posts: 21

Original Poster
Rep: Reputation: 15
Thanks Druuna but it didn't work. I am getting the following output with your suggestion:

,,,sym02 180203 1.5 1.6

Any other suggestions? Thanks for your time.
 
Old 07-07-2010, 04:26 AM   #4
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,508

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Quote:
Originally Posted by shakes82 View Post
,,,sym02 180203 1.5 1.6
Hmm.. this means that the input you've used is not what you've shown in the original post, since there is no way for the string "sym02" to appear from the command suggested by druuna (if copied exactly). Also which version of awk are you running and which Linux/Unix release?

Anyway, I suspect the fields in the input file are not separated by blank spaces. Maybe tabs?
 
Old 07-07-2010, 04:26 AM   #5
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

It works on my side.
Code:
$ cat infile
20090302 18:02:03 1.5 1.6

$ awk -F"[ :]" '{ print $1 $2 $3 $4","$5","$6",SYM" }' infile
20090302180203,1.5,1.6,SYM
I just noticed I'm using mawk.

Tried it with awk: Works as well.

Code:
awk --version
GNU Awk 3.1.6
Which distro and which awk version are you using?
 
Old 07-07-2010, 04:52 AM   #6
shakes82
LQ Newbie
 
Registered: Jun 2010
Posts: 21

Original Poster
Rep: Reputation: 15
@colucix: The fields are separated by a space. I think I am getting sym02 because it is taking 02 from 20090302 and adding sym before.
I am using fedora 13. awk version: 3.1.7

@druuna: I am using fedora 13. awk version: 3.1.7
I tried it again exactly as:
awk -F"[ :]" '{ print $1 $2 $3 $4","$5","$6",SYM" }' infile
with the same spacing and everything but I am getting the following output:
,sym0302180203,1.5,1.6
I think what it is adding ,sym infront and that is why 2009 is replaced by ,sym

Can you please suggest what I can do to make it right?

Thank you
 
Old 07-07-2010, 04:57 AM   #7
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

Could you post a relevant example? Like colucix already said it looks like your input file is not the same as your example posted in post #1.

The problem is not the awk command I posted with the example posted by you (shown by me and confirmed by colucix).

I can come up with one thing that could be causing this: Is the infile a unix or a dos file?
 
Old 07-07-2010, 05:00 AM   #8
shakes82
LQ Newbie
 
Registered: Jun 2010
Posts: 21

Original Poster
Rep: Reputation: 15
Following are a few lines of the data:

20090102 18:03:03 1.280550 1.281550
20090102 18:23:20 1.280570 1.281570
20090102 18:23:24 1.280270 1.281270
20090102 18:53:53 1.279970 1.280970
20090102 18:54:10 1.279810 1.280810

It is a *.txt (text) file.
 
Old 07-07-2010, 05:04 AM   #9
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

Unix/linux works different then windows. The fact that the file has a .txt extension doesn't say anything at all.

What does the following command show you: file infile.txt
 
Old 07-07-2010, 05:06 AM   #10
shakes82
LQ Newbie
 
Registered: Jun 2010
Posts: 21

Original Poster
Rep: Reputation: 15
It shows me:
fx.txt: ASCII text, with CRLF line terminators

Thanks
 
Old 07-07-2010, 05:15 AM   #11
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

That is a file with dos/windows terminators (CRLF) and it is also the reason why it doesn't work.

Most (all?) unix/linux tools do not work too well with dos/windows files.

Here a link that gives a few examples of how to change a dos file to a unix file (do make a backup of the original before trying them out!!).

HowTo: UNIX / Linux Convert DOS Newlines CR-LF to Unix/Linux Format

Hope this helps.
 
Old 07-07-2010, 05:16 AM   #12
shakes82
LQ Newbie
 
Registered: Jun 2010
Posts: 21

Original Poster
Rep: Reputation: 15
Thanks Druuna. I will try it and let you know how it goes. Thanks again.
 
Old 07-07-2010, 05:18 AM   #13
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
You're welcome
 
Old 07-07-2010, 05:19 AM   #14
shakes82
LQ Newbie
 
Registered: Jun 2010
Posts: 21

Original Poster
Rep: Reputation: 15
Thanks alot Druuna. It worked perfectly. I appreciate your help. Thanks for your time.
 
Old 07-07-2010, 05:31 AM   #15
shakes82
LQ Newbie
 
Registered: Jun 2010
Posts: 21

Original Poster
Rep: Reputation: 15
Since I am new with unix, can you please suggest how I can save the changes to the file after using the command. Thanks
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] awk, sed, grep and paragraphs ThinkLinux Linux - Newbie 3 04-09-2010 01:22 PM
help with grep/sed/awk nikunjbadjatya Programming 8 02-17-2010 07:29 PM
bash - awk, sed, grep, ... advice schneidz Programming 13 08-25-2008 09:30 AM
awk/sed to grep the text ahpin Linux - Software 3 10-17-2007 12:34 AM
diffrence between grep, sed, awk and egrep Fond_of_Opensource Linux - Newbie 3 08-18-2006 08:15 AM


All times are GMT -5. The time now is 06:19 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration