LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices



Reply
 
Search this Thread
Old 10-30-2009, 05:37 AM   #1
webs
LQ Newbie
 
Registered: Oct 2009
Posts: 7

Rep: Reputation: 0
Sed / Awk / removing chars from field 5 only


Hi All,

Ive been racking my brain for ages and cannot get this one thing to work.

We all know who to remove chars using sed from a whole line, but I have a comma separated file and i have a time stamp in field $5 which looks like the following.


20081009 21:00:46.160


I need to run a sed / awk to remove the :, but effect any data in the other fields as the : are valid in the other fields of the file.

I cant get mysql to use milli second time stamps and need to create the field to the following


20081009210046.160 so that I can use a float(17,3) to upload the timestamp.


theres $8 fields, comma seperated, but I onlt want to remove : from field 5..

Any takers ? Its harder than you think.



WEBS
 
Old 10-30-2009, 05:48 AM   #2
vijay_babu1981
Member
 
Registered: Nov 2004
Location: india
Distribution: fedora 10
Posts: 82

Rep: Reputation: 19
Quote:
Originally Posted by webs View Post
Hi All,

Ive been racking my brain for ages and cannot get this one thing to work.

We all know who to remove chars using sed from a whole line, but I have a comma separated file and i have a time stamp in field $5 which looks like the following.


20081009 21:00:46.160


I need to run a sed / awk to remove the :, but effect any data in the other fields as the : are valid in the other fields of the file.

I cant get mysql to use milli second time stamps and need to create the field to the following


20081009210046.160 so that I can use a float(17,3) to upload the timestamp.


theres $8 fields, comma seperated, but I onlt want to remove : from field 5..

Any takers ? Its harder than you think.



WEBS
You said comma separated file and 5th column you want to operate on?

Try:
Code:
sed 's/\(.*\),\(.*\),\(.*\),\(.*\),\(.*\):\(.*\)/\1,\2,\3,\4,\5\6/' FILE
 
Old 10-30-2009, 05:53 AM   #3
allez
Member
 
Registered: Jul 2008
Location: Russia/Siberia/Krasnoyarsk
Distribution: SuSE, CentOS, FreeBSD
Posts: 77

Rep: Reputation: 21
Just a "quick and dirty" example:
Code:
$echo 'bla:1,bla:2,bla:3,bla:4,20081009 21:00:46.160,bla:6' | awk -F "," '{ printf "%s,%s,%s,%s,%s,%s\n", $1, $2, $3, $4, gensub("(:| )", "", "g", $5), $6 }'

bla:1,bla:2,bla:3,bla:4,20081009210046.160,bla:6

Last edited by allez; 10-30-2009 at 05:55 AM.
 
Old 10-30-2009, 05:59 AM   #4
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
if you are working with fields, use awk. its easier than having to craft out complicated regex
Code:
awk -F"," '{gsub(/[: ]/,"",$5)}1' OFS="," file

Last edited by ghostdog74; 10-30-2009 at 06:12 AM.
 
Old 10-30-2009, 06:00 AM   #5
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by vijay_babu1981 View Post
You said comma separated file and 5th column you want to operate on?

Try:
Code:
sed 's/\(.*\),\(.*\),\(.*\),\(.*\),\(.*\):\(.*\)/\1,\2,\3,\4,\5\6/' FILE
are you sure it will work.?
 
Old 10-30-2009, 06:03 AM   #6
vijay_babu1981
Member
 
Registered: Nov 2004
Location: india
Distribution: fedora 10
Posts: 82

Rep: Reputation: 19
Quote:
Originally Posted by ghostdog74 View Post
are you sure it will work.?
Yes. Tried just now. Worked!
 
Old 10-30-2009, 06:05 AM   #7
vijay_babu1981
Member
 
Registered: Nov 2004
Location: india
Distribution: fedora 10
Posts: 82

Rep: Reputation: 19
Quote:
Originally Posted by vijay_babu1981 View Post
Yes. Tried just now. Worked!
I mean I tried it in Vi, But as I understand Vi and sed are so similar in using reg ex. So it should work in sed as well.
 
Old 10-30-2009, 06:10 AM   #8
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by vijay_babu1981 View Post
I mean I tried it in Vi, But as I understand Vi and sed are so similar in using reg ex. So it should work in sed as well.
Code:
$ more file
1,2,3,4,20081009 21:00:46.160,6,7,another:field

$ sed 's/\(.*\),\(.*\),\(.*\),\(.*\),\(.*\):\(.*\)/\1,\2,\3,\4,\5\6/' file
1,2,3,4,20081009 21:00:46.160,6,7,anotherfield
now the 8th field is changed. 5th field remain unchanged. OP requires ":" to be removed on $5.

Last edited by ghostdog74; 10-30-2009 at 06:12 AM.
 
Old 10-30-2009, 06:25 AM   #9
vijay_babu1981
Member
 
Registered: Nov 2004
Location: india
Distribution: fedora 10
Posts: 82

Rep: Reputation: 19
Quote:
Originally Posted by ghostdog74 View Post
Code:
$ more file
1,2,3,4,20081009 21:00:46.160,6,7,another:field

$ sed 's/\(.*\),\(.*\),\(.*\),\(.*\),\(.*\):\(.*\)/\1,\2,\3,\4,\5\6/' file
1,2,3,4,20081009 21:00:46.160,6,7,anotherfield
now the 8th field is changed. 5th field remain unchanged. OP requires ":" to be removed on $5.
Okay, My reg ex is being greedy. Its picking upto the last :

I did not work out with the OPs exact text.

Code:
$ sed "s/\(.*\),\(.*\),\(.*\),\(.*\),\(.*\):\(.*\),\(.*\),\(.*\),\(.*\)/\1,\2,\3,\4,\5\6,\7,\8,\9/" file
1,2,3,4,20081009 21:0046.160,6,7,another:field
OP can get some idea of what I am doing here.

But you are right. awk is the way to go, it can grab the column. I have started learning it!!
 
Old 10-30-2009, 06:38 AM   #10
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by vijay_babu1981 View Post
I have started learning it!!
learn and use awk, and you never have to use sed again.

Last edited by ghostdog74; 10-30-2009 at 06:40 AM.
 
Old 10-31-2009, 04:19 AM   #11
webs
LQ Newbie
 
Registered: Oct 2009
Posts: 7

Original Poster
Rep: Reputation: 0
Thanks guys, didnt expect something so quick......messed with substr, but didnt know about gsub....


awk -F"," '{gsub(/[: ]/,"",$5)}1' OFS="," file


That works a treat, its easy to understand and I think I'll use this command to manipulate so many problem files !!!!




-WEBS


Thanx
 
Old 10-31-2009, 04:46 AM   #12
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by webs View Post
messed with substr, but didnt know about gsub....
Read the docs. See my sig
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How do you insert chars with sed when the char itself is '\'... trist007 Linux - Newbie 8 08-07-2009 06:41 PM
AWK: delete last chars freeindy Programming 3 07-25-2009 05:35 AM
Awk script - Converting newline chars sleeper0110 Programming 6 06-05-2009 07:50 PM
Error in replacing special chars using awk linux_vidhyarthi Linux - General 2 04-09-2008 08:39 PM
My field separator changes when using awk Helene Programming 3 05-01-2004 09:10 AM


All times are GMT -5. The time now is 04:10 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration