[SOLVED] Simple Code to convert a field in the same file

shayno90 · 10-14-2009, 10:01 AM

Hi, I want to change/convert one field that is in a line of text(about 10 fields in a line) So in this line

'Mop-21050905','auth','info','info','26','2009-10-09 12:45:11','snort','snort[4574]: [1:408:5] ICMP Echo Reply [Classification: Misc activity] [Priority: 3]: {ICMP} 208.67.220.220 -> 95.224.96.106',8764647

I want to replace the datetime field '2009-10-09 12:45:11' with UTC '1255088707' e.g.

'Mop-21050905','auth','info','info','26','1255088707','snort','snort[4574]: [1:408:5] ICMP Echo Reply [Classification: Misc activity] [Priority: 3]: {ICMP} 208.67.220.220 -> 95.224.96.106',8764647

Would this command work? Also I would need to change multiple lines in the same way.

sed -e 's/.*'//' | date -d "XXX" +%s filename

I think these commands might do it but they are not correct

Any suggestions?

vonbiber · 10-14-2009, 11:12 AM

I ran a quick test with this.
First I wrote this script:

Code:

#!/bin/sh

cat > input.txt <<EOF
'Mop-21050905','auth','info','info','26','2009-10-09 12:45:11','snort','snort[45
74]: [1:408:5] ICMP Echo Reply [Classification: Misc activity] [Priority: 3]: {I
CMP} 208.67.220.220 -> 95.224.96.106',8764647
'Mop-21050904','auth','info','info','26','2009-10-07 11:45:11','snort','snort[45
74]: [1:408:5] ICMP Echo Reply [Classification: Misc activity] [Priority: 3]: {I
CMP} 208.67.220.220 -> 95.224.96.106',8764647
EOF

cat input.txt
echo '-----------------'

UTC=1255088707

cmd="sed 's/[0-9]\{4,4\}-[0-9]\{2,2\}-[0-9]\{2,2\} [0-9]\{2,2\}:[0-9]\{2,2\}:[0-
9]\{2,2\}/$UTC/g' input.txt"

eval "$cmd"

made the script (bogus.sh) executable then ran it:

Code:

$ ./bogus.sh
'Mop-21050905','auth','info','info','26','2009-10-09 12:45:11','snort','snort[4574]: [1:408:5] ICMP Echo Reply [Classification: Misc activity] [Priority: 3]: {ICMP} 208.67.220.220 -> 95.224.96.106',8764647
'Mop-21050904','auth','info','info','26','2009-10-07 11:45:11','snort','snort[4574]: [1:408:5] ICMP Echo Reply [Classification: Misc activity] [Priority: 3]: {ICMP} 208.67.220.220 -> 95.224.96.106',8764647
-----------------
'Mop-21050905','auth','info','info','26','1255088707','snort','snort[4574]: [1:408:5] ICMP Echo Reply [Classification: Misc activity] [Priority: 3]: {ICMP} 208.67.220.220 -> 95.224.96.106',8764647
'Mop-21050904','auth','info','info','26','1255088707','snort','snort[4574]: [1:408:5] ICMP Echo Reply [Classification: Misc activity] [Priority: 3]: {ICMP} 208.67.220.220 -> 95.224.96.106',8764647

If you want the file to be replaced replace

Code:

cmd="sed 's/..... input.txt"

with

Code:

cmd="sed -i 's/..... input.txt"

shayno90 · 10-14-2009, 12:16 PM

Well I have a specific command to change the datetime to UTC, those integers I provided were an example of how it should be converted.

I think this code may work but needs some improvement

filename | awk -F"/" '{ split($NF,tmp,"'"); print tmp[6] }' `

vonbiber · 10-14-2009, 02:58 PM

Quote:

Well I have a specific command to change the datetime to UTC, those integers I provided were an example of how it should be converted.

I figured that. That's why I stored it in a variable in
the shell script. All you have to do is
replace the line

Code:

UTC=1255088707

with that line

Code:

UTC=$(your command here)

where you write your command inside the parenthesis

shayno90 · 10-15-2009, 07:42 AM

Hi vonbiber, i am having trouble running the command

1st not sure how to run ./ bogus.sh, do u remove #!/bin/sh first
2nd think my date command is not been read properly, (maybe can't field to change)--> problem with "XXX" maybe
3rd you replace input.txt with your filename
4th i went to the bottom of my text document and copied the last line so the EOF would stop there

This is what I ran

#!/bin/sh

./ bogus.sh

cat > filename <<EOF
'Mop-21050905','auth','info','info','26','2009-10-09 12:45:20','snort','snort[4574]: [1:2050:14] SQL version overflow attempt [Classification: Attempted Administrator Privilege Gain] [Priority: 1]: {UDP} 202.103.9.51:1096 -> 95.224.96.106:1434',8764657
EOF

cat filename
echo '-----------------'

UTC=$(date -d "XXX" +%s)

cmd="sed 's/[0-9]\{4,4\}-[0-9]\{2,2\}-[0-9]\{2,2\} [0-9]\{2,2\}:[0-9]\{2,2\}:[0-
9]\{2,2\}/$UTC/g' filename"

eval "$cmd"

Sorry I don't understand, it is difficult to make the changes when I can't understand some code

chrism01 · 10-15-2009, 08:33 PM

bogus.sh is the name he made up for the test script he wrote; a replacement for yours.

vonbiber · 10-16-2009, 04:36 AM

If you have already a file to read from, copy and paste
the code below to a file that you name whatever you like
(eg bogus.sh) and make executable (chmod +x bogus.sh)

Code:

#!/bin/sh

if [ -z "$1" ]; then
  exit 1
fi
if [ ! -e "$1" ]; then
  exit 1
fi
INPUT="$1"
UTC=$(date --utc +%s)
cmd="sed 's/[0-9]\{4,4\}-[0-9]\{2,2\}-[0-9]\{2,2\} [0-9]\{2,2\}:[0-9]\{2,2\}:[0-
9]\{2,2\}/$UTC/g' $INPUT"

eval "$cmd"

then you run like this (replace bogus.sh by the name you
gave to your script shell and replace filename by the actual
name of the file you're reading from)

Code:

./bogus.sh filename

some comments

Code:

UTC=$(date --utc +%s)

This stores the date in UTC format in the variable UTC.
It can be retrieved by invoking its value $UTC.

Code:

cmd="sed ..."

This writes the replacement command to apply to the input file

Code:

eval "$cmd"

This executes the command written above.