LinuxQuestions.org - combine multiple awk statements into single line

Page 1 of 2

Show 50 post(s) from this thread on one page

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - combine multiple awk statements into single line (https://www.linuxquestions.org/questions/linux-newbie-8/combine-multiple-awk-statements-into-single-line-4175560662/)

tlegend33

12-04-2015 03:00 PM

combine multiple awk statements into single line

I'm mining logs in an attempt to create a pipe delimited file. I have the below awk statement, but I can't seem to get the appropriate output onto the correct line:

Quote:

awk '/nz_migrate table/ {print $3}/.....migration process started/ {print $5,$6} end' tony.log
table1
2015-11-29 07:00:09
table2
table3
2015-11-29 07:00:20
table4
2015-11-29 07:00:48
table5
table6
2015-11-29 07:01:19
table7
table8
2015-11-29 07:01:31
table9
table10
2015-11-29 07:01:41

The desired output is:

Quote:

table1 2015-11-29 07:00:09
table2
table3 2015-11-29 07:00:20
table4 2015-11-29 07:00:48
table5
table6 2015-11-29 07:01:19
table7
table8 2015-11-29 07:01:31
table9
table10 2015-11-29 07:01:41

Thanks!

berndbausch

12-04-2015 07:19 PM

print always ends with ORS, usually a newline character. Use printf instead. See also my signature.

What is the end for, by the way?

tlegend33

12-07-2015 08:01 AM

The end was most likely left over from previous attempts at this bit of functionality. printf gets everything into a single line. I need to have my output as above. Some tables will have migrate dates while others do not. If a table has a migrate date, then I need a new line. If a table does not have a migrate date, I also need a new line to start with the next table name. Thanks for your help and also the link provided. I will continue to research.

Code:

awk  '/nz_migrate table/ {printf $3}/.....migration process                              started/ {printf $5,$6}' tony.log

table1 2015-11-29 07:00:09table2table3 2015-11-29 07:00:20table4 2015-11-29 07:00:48table5table6 2015-11-29 07:01:19table7table8 2015-11-29 07:01:31table9table10 2015-11-29 07:01:41

syg00

12-07-2015 08:06 AM

If you want a newline, add it to the printf - "\n".

berndbausch

12-07-2015 08:06 AM

Convert the first print into printf. Add code that prints a newline when the input line doesn't contain /migration process started/.

tlegend33

12-07-2015 12:12 PM

I believe I've already converted the first print to a printf. Additionally, that last input line will not always contain /migration process started/. My biggest challenge throughout the logfile is that not all the input will be identical in format. Thanks for your assistance, I will continue to research.

tlegend33

12-07-2015 12:14 PM

Code:

nz_migrate table table1



.....processing table 1 of 1396

.....truncating the target table

.....migration process                              started at  2015-11-29 07:00:09

.....estimated # of records                                    2

.....nzload starting            ( thread 1 of 1 )

.....unloading data              ( thread 1 of 1 )

.....data flowing.....

.....unload results              ( thread 1 of 1 )              INSERT 0 2

.....unload finished            ( thread 1 of 1 )              elapsed seconds: 3

.....nzload finished            ( thread 1 of 1 )              elapsed seconds: 3

.....nzload successful          ( thread 1 of 1 )

.....migration process                              ended at    2015-11-29 07:00:12

.....data flow finished

.....actual # of records unloaded                              2

.....

.....migration completed                                        TOTAL seconds: 3

.....

.....cksum process                                  started at  2015-11-29 07:00:12

.....cksum process                                  ended at    2015-11-29 07:00:14

.....confirmed cksum: 0.0 2 table1

.....

.....cksum completed                                            TOTAL seconds: 2

.....

.....launching generate statistics (in the background)



nz_migrate table table2



.....processing table 2 of 1396

.....truncating the target table

.....source table is empty, no data to migrate

.....actual # of records unloaded                              0

.....

.....migration completed                                        TOTAL seconds: 0

MadeInGermany

12-07-2015 12:38 PM

It's a bit tricky to get the newlines alright, especially in the first and the last section (output line).
The following uses a variable nl that is empty in the first line. At the END a pending newline is printed.

Code:

awk '/nz_migrate table/ {printf "%s", nl $3} /.....migration process/ {printf "|%s %s", $5, $6} {nl="\n"} END {printf nl}' logfile

tlegend33

12-07-2015 02:04 PM

This has me very close to where I need to be:

Code:

awk  '/nz_migrate table/ {printf $3"|"}/.....migration process                              started/{if ($1) print $5" "$6} end {printf "\n"}' tony.log

table1|2015-11-29 07:00:09

table2|table3|2015-11-29 07:00:20

table4|2015-11-29 07:00:48

table5|table6|2015-11-29 07:01:19

table7|table8|2015-11-29 07:01:31

table9|table10|2015-11-29 07:01:41

needs to be:

Code:

table1|2015-11-29 07:00:09

table2|

table3|2015-11-29 07:00:20

table4|2015-11-29 07:00:48

table5|

table6|2015-11-29 07:01:19

table7|

table8|2015-11-29 07:01:31

table9|

table10|2015-11-29 07:01:41

I want the 'if' statement to function where if the string is not found, then to printf "\n".

tlegend33

12-07-2015 03:12 PM

This still gives me the same result:

Code:

awk  '/nz_migrate table/ {printf $3"|"}/.....migration process                              started/{if (length($1)==0) {print "\n";} else {print $5" "$6;}} end {printf "\n"}' tony.log

table1|2015-11-29 07:00:09

table2|table3|2015-11-29 07:00:20

table4|2015-11-29 07:00:48

table5|table6|2015-11-29 07:01:19

table7|table8|2015-11-29 07:01:31

table9|table10|2015-11-29 07:01:41

Does awk have something for a null condition? table2 should have a null 'migration process' in this scenario.

syg00

12-07-2015 04:44 PM

There are innumerable ways to fix this. Testing for a condition that doesn't exist is not a good one.
Simplest probably just to force a newline first in the initial printf. Will give you a null line as first line of output, but in need that can be worked around using something like the nl in post #8.
Better to use regex rather than a fixed number of blanks in the test too ... output formats have been known to change without warning.

berndbausch

12-07-2015 05:25 PM

You can actually check for the absence of a pattern, see http://www.gnu.org/software/gawk/man...ssion-Patterns or use the !~ operator.

But it might be better to work with next statements in the actions - that is, when you are done with the processing of /migrate table/ lines, next; when you are done with the proceesing of /migration process/ lines, next, and then print a newline.

Depends on the overall logic of the program.

grail

12-07-2015 07:01 PM

Code:

awk '/nz_migrate/{printf (x?"\n":"")$3"|";x=1}/migration process\s*started/{print $5,$6;x=0}'

Can probably be tidied up

syg00

12-08-2015 01:22 AM

I like that too grail, but difficult to explain to someone not well versed.

tlegend33

12-08-2015 12:00 PM

syg00, you are correct. I am not terribly well versed on awk programming. However, I sincerely appreciate everyone's comments on my issue. Grails solution provides what I need however I submitted only two search conditions for my scripts as I thought it would a good jumping off point for the several other strings I need to search for also:

Code:

 awk '

> /nz_migrate/{printf (x?"\n":"")$3"|";x=2} \

> /.....migration process                              started/{print $5" "$6"|";x=1 \

> /.....migration process                              ended/{print $5" "$6"|";x=0 \

> }' tony.log

awk: cmd. line:3: /.....migration process                              ended/{print $5" "$6"|";x=0 \

awk: cmd. line:3:  ^ syntax error

awk: cmd. line:3: /.....migration process                              ended/{print $5" "$6"|";x=0 \

awk: cmd. line:3:                                                            ^ syntax error

awk: cmd. line:5: }

awk: cmd. line:5:  ^ unexpected newline or end of string

I'm researching arrays so I can see how to incorporate grail's suggestions to additional criteria. Thanks to all who replied. I'll get this figured out.

Thanks!

All times are GMT -5. The time now is 02:13 AM.

Page 1 of 2

Show 50 post(s) from this thread on one page