LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (http://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Sed / Awk / removing chars from field 5 only (http://www.linuxquestions.org/questions/linux-newbie-8/sed-awk-removing-chars-from-field-5-only-765524/)

webs 10-30-2009 04:37 AM

Sed / Awk / removing chars from field 5 only
 
Hi All,

Ive been racking my brain for ages and cannot get this one thing to work.

We all know who to remove chars using sed from a whole line, but I have a comma separated file and i have a time stamp in field $5 which looks like the following.


20081009 21:00:46.160


I need to run a sed / awk to remove the :, but effect any data in the other fields as the : are valid in the other fields of the file.

I cant get mysql to use milli second time stamps and need to create the field to the following


20081009210046.160 so that I can use a float(17,3) to upload the timestamp.


theres $8 fields, comma seperated, but I onlt want to remove : from field 5..

Any takers ? Its harder than you think.



WEBS

vijay_babu1981 10-30-2009 04:48 AM

Quote:

Originally Posted by webs (Post 3737720)
Hi All,

Ive been racking my brain for ages and cannot get this one thing to work.

We all know who to remove chars using sed from a whole line, but I have a comma separated file and i have a time stamp in field $5 which looks like the following.


20081009 21:00:46.160


I need to run a sed / awk to remove the :, but effect any data in the other fields as the : are valid in the other fields of the file.

I cant get mysql to use milli second time stamps and need to create the field to the following


20081009210046.160 so that I can use a float(17,3) to upload the timestamp.


theres $8 fields, comma seperated, but I onlt want to remove : from field 5..

Any takers ? Its harder than you think.



WEBS

You said comma separated file and 5th column you want to operate on?

Try:
Code:

sed 's/\(.*\),\(.*\),\(.*\),\(.*\),\(.*\):\(.*\)/\1,\2,\3,\4,\5\6/' FILE

allez 10-30-2009 04:53 AM

Just a "quick and dirty" example: :)
Code:

$echo 'bla:1,bla:2,bla:3,bla:4,20081009 21:00:46.160,bla:6' | awk -F "," '{ printf "%s,%s,%s,%s,%s,%s\n", $1, $2, $3, $4, gensub("(:| )", "", "g", $5), $6 }'

bla:1,bla:2,bla:3,bla:4,20081009210046.160,bla:6


ghostdog74 10-30-2009 04:59 AM

if you are working with fields, use awk. its easier than having to craft out complicated regex
Code:

awk -F"," '{gsub(/[: ]/,"",$5)}1' OFS="," file

ghostdog74 10-30-2009 05:00 AM

Quote:

Originally Posted by vijay_babu1981 (Post 3737726)
You said comma separated file and 5th column you want to operate on?

Try:
Code:

sed 's/\(.*\),\(.*\),\(.*\),\(.*\),\(.*\):\(.*\)/\1,\2,\3,\4,\5\6/' FILE

are you sure it will work.?

vijay_babu1981 10-30-2009 05:03 AM

Quote:

Originally Posted by ghostdog74 (Post 3737741)
are you sure it will work.?

Yes. Tried just now. Worked!

vijay_babu1981 10-30-2009 05:05 AM

Quote:

Originally Posted by vijay_babu1981 (Post 3737744)
Yes. Tried just now. Worked!

I mean I tried it in Vi, But as I understand Vi and sed are so similar in using reg ex. So it should work in sed as well.

ghostdog74 10-30-2009 05:10 AM

Quote:

Originally Posted by vijay_babu1981 (Post 3737746)
I mean I tried it in Vi, But as I understand Vi and sed are so similar in using reg ex. So it should work in sed as well.

Code:

$ more file
1,2,3,4,20081009 21:00:46.160,6,7,another:field

$ sed 's/\(.*\),\(.*\),\(.*\),\(.*\),\(.*\):\(.*\)/\1,\2,\3,\4,\5\6/' file
1,2,3,4,20081009 21:00:46.160,6,7,anotherfield

now the 8th field is changed. 5th field remain unchanged. OP requires ":" to be removed on $5.

vijay_babu1981 10-30-2009 05:25 AM

Quote:

Originally Posted by ghostdog74 (Post 3737752)
Code:

$ more file
1,2,3,4,20081009 21:00:46.160,6,7,another:field

$ sed 's/\(.*\),\(.*\),\(.*\),\(.*\),\(.*\):\(.*\)/\1,\2,\3,\4,\5\6/' file
1,2,3,4,20081009 21:00:46.160,6,7,anotherfield

now the 8th field is changed. 5th field remain unchanged. OP requires ":" to be removed on $5.

Okay, My reg ex is being greedy. Its picking upto the last :

I did not work out with the OPs exact text.

Code:

$ sed "s/\(.*\),\(.*\),\(.*\),\(.*\),\(.*\):\(.*\),\(.*\),\(.*\),\(.*\)/\1,\2,\3,\4,\5\6,\7,\8,\9/" file
1,2,3,4,20081009 21:0046.160,6,7,another:field

OP can get some idea of what I am doing here.

But you are right. awk is the way to go, it can grab the column. I have started learning it!!

ghostdog74 10-30-2009 05:38 AM

Quote:

Originally Posted by vijay_babu1981 (Post 3737761)
I have started learning it!!

learn and use awk, and you never have to use sed again.

webs 10-31-2009 03:19 AM

Thanks guys, didnt expect something so quick......messed with substr, but didnt know about gsub....


awk -F"," '{gsub(/[: ]/,"",$5)}1' OFS="," file


That works a treat, its easy to understand and I think I'll use this command to manipulate so many problem files !!!!




-WEBS


Thanx

ghostdog74 10-31-2009 03:46 AM

Quote:

Originally Posted by webs (Post 3738878)
messed with substr, but didnt know about gsub....

Read the docs. See my sig


All times are GMT -5. The time now is 10:41 PM.