awk and strftime/mktime - characters added to end of date/time conversion string

jonbob75 · 10-07-2013, 05:45 AM

Hi,
I'm parsing log files using awk and changing their format to make them readable to a log file analyser. The awk command I'm using works fine, except for the date/time conversion.
The log file I'm parsing has entries like this:

Code:

2013-01-01 00:00:44 127.0.0.1 12345W@uni.edu 8hd86863gd 200 8042 http://journal.com

And I'm trying to get them in this format:

Code:

127.0.0.1 8hd86863gd 12345W@uni.edu [01/Jan/2013:00:00:44 +0000] "GET http://journal.com HTTP/1.1" 200 8042

The awk command I'm using to convert them is this:

Code:

awk 'BEGIN {FS=OFS=" "}
{format = "%d/%b/%Y:%H:%M:%S %z" split($1,date,"-") split($2,time,":")}
{datetime = (date[1] " " date[2] " " date[3] " " time[1] " " time[2] " " time[3] " 0")}
{print $3,$5,$4,"["strftime(format, mktime(datetime))"]","\"GET " $8 " HTTP/1.1\"",$6,$7}' 
oldFormat.log > newFormat.log

The newFormat.log file created is exactly as I need it, except the new date/time has "33" appended to it. So the time part looks like this:

Code:

[01/Jan/2013:00:00:44 +000033]

Any idea why this is happening? I've tried changing the formation of the datetime string, but this "33" is appended to it no matter what the format.

Thanks,
jb

colucix · 10-07-2013, 06:00 AM

Quote:

Originally Posted by jonbob75

Any idea why this is happening?

Yes. The following statement

Code:

format = "%d/%b/%Y:%H:%M:%S %z" split($1,date,"-") split($2,time,":")

builds a string from three substrings. Take in mind that string concatenation in awk is performed simply by using a blank space as separator, therefore the string "%d/%b/%Y:%H:%M:%S %z" is concatenated with the return values of the two split functions, which is 3 in both cases.

Moreover, your script can be simplified by removing unnecessary stuff:

Code:

awk '{
  format = "%d/%b/%Y:%H:%M:%S %z";
  split($1,date,"-")
  split($2,time,":")
  datetime = (date[1] " " date[2] " " date[3] " " time[1] " " time[2] " " time[3] " 0")
  print $3,$5,$4,"["strftime(format, mktime(datetime))"]","\"GET " $8 " HTTP/1.1\"",$6,$7
}' oldFormat.log > newFormat.log

Hope this helps.

jonbob75 · 10-07-2013, 06:12 AM

Quote:

Originally Posted by colucix

Yes. The following statement

Code:

format = "%d/%b/%Y:%H:%M:%S %z" split($1,date,"-") split($2,time,":")

builds a string from three substrings.

Of course! Thanks so much for spotting that!!

Quote:

Originally Posted by colucix

Moreover, your script can be simplified by removing unnecessary stuff:

Code:

awk '{
  format = "%d/%b/%Y:%H:%M:%S %z";
  split($1,date,"-")
  split($2,time,":")
  datetime = (date[1] " " date[2] " " date[3] " " time[1] " " time[2] " " time[3] " 0")
  print $3,$5,$4,"["strftime(format, mktime(datetime))"]","\"GET " $8 " HTTP/1.1\"",$6,$7
}' oldFormat.log > newFormat.log

Hope this helps.

Thanks! Tried putting that in, but it threw syntax errors at the "datetime" and "print" lines until I braced them like this:

Code:

awk '{
   format = "%d/%b/%Y:%H:%M:%S %z";
   split($1,date,"-")
   split($2,time,":")}
  {datetime = (date[1] " " date[2] " " date[3] " " time[1] " " time[2] " " time[3] " 0")}
  {print $3,$5,$4,"["strftime(format, mktime(datetime))"]","\"GET " $8 " HTTP/1.1\"",$6,$7
}' oldFormat.log > newFormat.log

But everything works now anyway, so thanks a lot again!

grail · 10-07-2013, 08:59 AM

You simply need semi-colons after each statement