AWK looping though fields

casperdaghost · 12-29-2011, 10:06 AM

Code:

casperbox 12292011 09:12:49 bash: user: casper as casper from ip: 10.80.6.100:pts/55 execs: \'sudo grep closedata /data/*\'
casperbox 12292011 09:13:15 bash: user: casper as casper from ip: 10.80.6.100:pts/55 execs: \'cd ~/nocteam/group/\'
casperbox 12292011 09:13:19 bash: user: casper as casper from ip: 10.80.6.100:pts/55 execs: \'xcd /var/ftp/\'
casperbox 12292011 09:13:22 bash: user: casper as casper from ip: 10.80.6.100:pts/55 execs: 'cd /var/ftp/\'

I pipe the more into this awk to get just the fields 14 through the end.

| awk '{ for (i=14; i<=NF; i++) printf "%s ", $i ; printf "\n"; }

However i want to include fields 3 and 6 - but when i do that the output is stacked :

awk '{ print $3 ; print $6 ; for (i=14; i<=NF; i++) printf "%s", $i ; printf "\n"; }

091319
casper
'cd /var/ftp/\'

I am going to need it inline, like this:

09:13:19 casper \'cd /var/ftp/\'

Telengard · 12-29-2011, 11:39 AM

The print command automatically adds a newline. Try using printf instead.

I don't understand why you are backslash escaping all your quote marks. Is your shell not Bash?

casperdaghost · 12-29-2011, 12:50 PM

Code:

 more /var/log/bashlog | grep casper  | awk '{ printf $3 ," "  ; printf $6 ; for (i=14; i<=NF; i++) printf "%s", $i ; printf "\n"; }'

09:12:49caspergrepclosedata/data/*\'
09:13:15casper~/nocgroup/\'
09:13:19casper/var/ftp/\'
09:13:22casper/var/ftp/\'

The printf works great - however now I am missing a space - how do I get that?

Please excuse some of the backslashes from previous posts - taht is the way the bash log outputs the commands.

David the H. · 12-29-2011, 01:15 PM

printf is the "formatted printing" function. You define a fixed printing pattern to use as the first argument, and the input you want printed in that format as the subsequent arguments. If you want to insert a space (or a newline, or anything else) in the output, you have to put it in the format string.

See here for a full description of printf on gawk:

http://www.gnu.org/software/gawk/man...de/Printf.html

casperdaghost · 12-29-2011, 01:41 PM

awk '{ printf "%s ", $3 ; printf "%s ",$6 ; for (i=13; i<=NF; i++) printf "%s ", $i ; printf "\n"; }'

ugly - but it works - thank you for the reference page

David the H. · 12-29-2011, 02:04 PM

Nah, not too bad. I'd say it's pretty much exactly what you need to use in this case.

You can combine the first two fields into a single printing, however.

Code:

printf( "%s %s " , $3 , $6 )

Parenthesis are optional around the print functions. It may help readability if you used them.

There may also be some way to use a loop directly as an argument to printf, but I'm not too sure about that.

BTW, it would be easier to read if you used [code][/code] tags around all your code and data. The significant blocks, at least.

casperdaghost · 12-30-2011, 04:05 PM

Hey dave - -check this out

Code:

#!/usr/bin/perl 
open (FILE, "/tmp/awk_loop") or die ("Can not open the File $!"); 
while (<FILE>) { 
  @array = <FILE> ; 
  foreach $line (@array) {
  @split_array=split(" ", $line); 
  @splice =  @split_array[2,7,13..24] ;
  print "@splice\n";
  }
}

this works almost - the formatting is good, but it does not print out the first line --t i guess i will have to open up another thread.

Telengard · 12-30-2011, 04:34 PM

Quote:

Originally Posted by casperdaghost

Hey dave - -check this out

Code:

#!/usr/bin/perl 
open (FILE, "/tmp/awk_loop") or die ("Can not open the File $!"); 
while (<FILE>) { 
  @array = <FILE> ; 
  foreach $line (@array) {
  @split_array=split(" ", $line); 
  @splice =  @split_array[2,7,13..24] ;
  print "@splice\n";
  }
}

I'm confident some of the resident perl fans can trim that down to three lines of under 80 characters each. Maybe even less.

casperdaghost · 12-30-2011, 04:44 PM

If you could trim it that is great -

Code:

#!/usr/bin/perl 
@array = `more /tmp/awk_loop`; 
foreach $line (@array) {
@split_array=split(" ", $line); 
@splice =  @split_array[2,7,13..24] ;
print "@splice\n";
}

the code works fine when I more the whole file and backtick it into an array - however that eats up a whole load of memory.
I need to read it into a FILEHANDLE and process it line by line.

There is something going on with the while loop and loading the FILEHANDLE into the array.

the results :

Code:

09:12:49 casper grep closedata /data/*\'         
09:13:15 casper ~/nocteam/group/\'           
09:13:19 casper /var/ftp/\'

Telengard · 12-30-2011, 05:01 PM

Quote:

Originally Posted by casperdaghost

If you could trim it that is great

Nope, haven't written anything in perl for years, sorry. I find bash and awk are enough for almost anything I might want to do. When my needs are more complex I reach for C or Java, but even that is pretty rare.

David the H. · 12-31-2011, 09:31 AM

I'm a rank beginner in perl myself. I know just enough to understand what that code is doing basically, but not enough to really help you with the problem.

It looks like it really isn't doing all that much that's different from awk though. Only instead of printing the fields directly, it extracts the fields you want from each line into an array, then prints that.

awk could handle it that way as well, but you'd have to deal with the limitations it has in sorting the array elements. It can sometimes be a bit frustrating to get awk to print its arrays in the order you want, as it doesn't keep track of the initial input order for the indexes.

I think what awk really needs is a way to print ranges of fields (and array elements) without having to use loops all the time, like you can do in perl. I really wonder why, in all the years it's been around, nobody has ever bothered to extend it to include such features.