How to pass multiline output of a command into 'date -j'

chr15t0 · 03-03-2011, 01:59 AM

Hey folks

I am doing some NSCA log parsing and I want to get an output like this:

2011-Feb-18:11:00:07
2011-Feb-18:11:00:07
2011-Feb-18:11:00:07
2011-Feb-18:11:00:08
2011-Feb-18:11:00:08

converted to this:

1298026807
1298026807
1298026807
1298026808
1298026808

Of course I'm making it hard for myself by cramming it into a shell one liner - this is where I'm at:

The following will create the formatted-date output:

Code:

sort -b -k4.9,4.12 -k4.5b,4.7Mb -k4.2,4.3 -k4.14,4 foo.log| awk -F '[ \[\/:]' '{print $7"-"$6"-"$5":"$8":"$9":"$10}'

and I want to pass that into the second parameter of 'date -j' like this:

Code:

date -j -f "%Y-%b-%d:%T" "+%s" "<pass each line into here>"

I have tried this, but it treats the entire output as a single line, then plops a timestamp on the end (I think):

Code:

sort -b -k4.9,4.12 -k4.5b,4.7Mb -k4.2,4.3 -k4.14,4 foo.log| date -j -f "%Y-%b-%d:%T" "+%s" "`awk -F '[ \[\/:]' '{print $7"-"$6"-"$5":"$8":"$9":"$10}'`"

I'm pretty sure this is dead easy, but I just wasted an hour on it

Any ideas?

christo

ruario · 03-03-2011, 02:14 AM

Use GNU Parallel

Code:

sort -b -k4.9,4.12 -k4.5b,4.7Mb -k4.2,4.3 -k4.14,4 foo.log| awk -F '[ \[\/:]' '{print $7"-"$6"-"$5":"$8":"$9":"$10}' | parallel date -j -f "%Y-%b-%d:%T" "+%s"

ruario · 03-03-2011, 02:25 AM

Actually you could do this with xargs but it is uglier:

Code:

sort -b -k4.9,4.12 -k4.5b,4.7Mb -k4.2,4.3 -k4.14,4 foo.log| awk -F '[ \[\/:]' '{print $7"-"$6"-"$5":"$8":"$9":"$10}' | xargs -I {} date -j -f "%Y-%b-%d:%T" "+%s" {}

I'd still recommend you install Parallel. It will make life easier later on as it is much more powerful.

chr15t0 · 03-03-2011, 04:22 AM

two great responses. Thanks guys

christo

ruario · 03-03-2011, 04:42 AM

Well I am just one guy, but you are welcome.

tange · 03-06-2011, 12:15 PM

I am not really sure what you do by the sorting. Also 'date -j' seems not to be part of 'date (GNU coreutils) 8.5'.

But the following gives the output you have shown for the input you have shown:

Code:

cat foo.log | parallel -k --colsep '-|:' date +%s --date "'{2} {3} {1} {4}:{5}:{6}'"

It will spawn date for each line, so if you have a lot of lines, you might consider using perl with Time::Local.

ruario · 03-06-2011, 01:21 PM

Quote:

Originally Posted by tange

Also 'date -j' seems not to be part of 'date (GNU coreutils) 8.5'.

If you look at his icon it is from a Mac, so he is using FreeBSD date.