LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-29-2011, 11:06 AM   #1
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Rep: Reputation: 16
AWK looping though fields


Code:
casperbox 12292011 09:12:49 bash: user: casper as casper from ip: 10.80.6.100:pts/55 execs: \'sudo grep closedata /data/*\'
casperbox 12292011 09:13:15 bash: user: casper as casper from ip: 10.80.6.100:pts/55 execs: \'cd ~/nocteam/group/\'
casperbox 12292011 09:13:19 bash: user: casper as casper from ip: 10.80.6.100:pts/55 execs: \'xcd /var/ftp/\'
casperbox 12292011 09:13:22 bash: user: casper as casper from ip: 10.80.6.100:pts/55 execs: 'cd /var/ftp/\'

I pipe the more into this awk to get just the fields 14 through the end.

| awk '{ for (i=14; i<=NF; i++) printf "%s ", $i ; printf "\n"; }

However i want to include fields 3 and 6 - but when i do that the output is stacked :

awk '{ print $3 ; print $6 ; for (i=14; i<=NF; i++) printf "%s", $i ; printf "\n"; }

091319
casper
'cd /var/ftp/\'

I am going to need it inline, like this:

09:13:19 casper \'cd /var/ftp/\'

Last edited by casperdaghost; 12-29-2011 at 01:51 PM.
 
Old 12-29-2011, 12:39 PM   #2
Telengard
Member
 
Registered: Apr 2007
Location: USA
Distribution: Kubuntu 8.04
Posts: 579
Blog Entries: 8

Rep: Reputation: 148Reputation: 148
The print command automatically adds a newline. Try using printf instead.

I don't understand why you are backslash escaping all your quote marks. Is your shell not Bash?
 
1 members found this post helpful.
Old 12-29-2011, 01:50 PM   #3
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
Code:
 more /var/log/bashlog | grep casper  | awk '{ printf $3 ," "  ; printf $6 ; for (i=14; i<=NF; i++) printf "%s", $i ; printf "\n"; }'

09:12:49caspergrepclosedata/data/*\'
09:13:15casper~/nocgroup/\'
09:13:19casper/var/ftp/\'
09:13:22casper/var/ftp/\'
The printf works great - however now I am missing a space - how do I get that?

Please excuse some of the backslashes from previous posts - taht is the way the bash log outputs the commands.

Last edited by casperdaghost; 12-29-2011 at 01:52 PM.
 
Old 12-29-2011, 02:15 PM   #4
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,834

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
printf is the "formatted printing" function. You define a fixed printing pattern to use as the first argument, and the input you want printed in that format as the subsequent arguments. If you want to insert a space (or a newline, or anything else) in the output, you have to put it in the format string.

See here for a full description of printf on gawk:

http://www.gnu.org/software/gawk/man...de/Printf.html
 
1 members found this post helpful.
Old 12-29-2011, 02:41 PM   #5
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
awk '{ printf "%s ", $3 ; printf "%s ",$6 ; for (i=13; i<=NF; i++) printf "%s ", $i ; printf "\n"; }'


ugly - but it works - thank you for the reference page

Last edited by casperdaghost; 12-29-2011 at 03:00 PM.
 
Old 12-29-2011, 03:04 PM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,834

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
Nah, not too bad. I'd say it's pretty much exactly what you need to use in this case.

You can combine the first two fields into a single printing, however.

Code:
printf( "%s %s " , $3 , $6 )
Parenthesis are optional around the print functions. It may help readability if you used them.

There may also be some way to use a loop directly as an argument to printf, but I'm not too sure about that.

BTW, it would be easier to read if you used [code][/code] tags around all your code and data. The significant blocks, at least.
 
2 members found this post helpful.
Old 12-30-2011, 05:05 PM   #7
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
Hey dave - -check this out
Code:
#!/usr/bin/perl 
open (FILE, "/tmp/awk_loop") or die ("Can not open the File $!"); 
while (<FILE>) { 
  @array = <FILE> ; 
  foreach $line (@array) {
  @split_array=split(" ", $line); 
  @splice =  @split_array[2,7,13..24] ;
  print "@splice\n";
  }
}
this works almost - the formatting is good, but it does not print out the first line --t i guess i will have to open up another thread.

Last edited by casperdaghost; 12-30-2011 at 05:07 PM.
 
Old 12-30-2011, 05:34 PM   #8
Telengard
Member
 
Registered: Apr 2007
Location: USA
Distribution: Kubuntu 8.04
Posts: 579
Blog Entries: 8

Rep: Reputation: 148Reputation: 148
Quote:
Originally Posted by casperdaghost View Post
Hey dave - -check this out
Code:
#!/usr/bin/perl 
open (FILE, "/tmp/awk_loop") or die ("Can not open the File $!"); 
while (<FILE>) { 
  @array = <FILE> ; 
  foreach $line (@array) {
  @split_array=split(" ", $line); 
  @splice =  @split_array[2,7,13..24] ;
  print "@splice\n";
  }
}
I'm confident some of the resident perl fans can trim that down to three lines of under 80 characters each. Maybe even less.

Last edited by Telengard; 12-30-2011 at 05:58 PM.
 
Old 12-30-2011, 05:44 PM   #9
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
If you could trim it that is great -

Code:
#!/usr/bin/perl 
@array = `more /tmp/awk_loop`; 
foreach $line (@array) {
@split_array=split(" ", $line); 
@splice =  @split_array[2,7,13..24] ;
print "@splice\n";
}
the code works fine when I more the whole file and backtick it into an array - however that eats up a whole load of memory.
I need to read it into a FILEHANDLE and process it line by line.

There is something going on with the while loop and loading the FILEHANDLE into the array.

the results :
Code:
09:12:49 casper grep closedata /data/*\'         
09:13:15 casper ~/nocteam/group/\'           
09:13:19 casper /var/ftp/\'

Last edited by casperdaghost; 12-30-2011 at 05:47 PM.
 
Old 12-30-2011, 06:01 PM   #10
Telengard
Member
 
Registered: Apr 2007
Location: USA
Distribution: Kubuntu 8.04
Posts: 579
Blog Entries: 8

Rep: Reputation: 148Reputation: 148
Quote:
Originally Posted by casperdaghost View Post
If you could trim it that is great
Nope, haven't written anything in perl for years, sorry. I find bash and awk are enough for almost anything I might want to do. When my needs are more complex I reach for C or Java, but even that is pretty rare.
 
Old 12-31-2011, 10:31 AM   #11
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,834

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
I'm a rank beginner in perl myself. I know just enough to understand what that code is doing basically, but not enough to really help you with the problem.

It looks like it really isn't doing all that much that's different from awk though. Only instead of printing the fields directly, it extracts the fields you want from each line into an array, then prints that.

awk could handle it that way as well, but you'd have to deal with the limitations it has in sorting the array elements. It can sometimes be a bit frustrating to get awk to print its arrays in the order you want, as it doesn't keep track of the initial input order for the indexes.

I think what awk really needs is a way to print ranges of fields (and array elements) without having to use loops all the time, like you can do in perl. I really wonder why, in all the years it's been around, nobody has ever bothered to extend it to include such features.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] [awk] looping description problem dhodho Programming 7 07-26-2010 11:18 PM
awk question on handling *.CSV "text fields" in awk jschiwal Programming 8 05-27-2010 07:23 AM
[SOLVED] get fields using awk ashok.g Programming 9 12-09-2009 02:21 AM
shell command using awk fields inside awk one71 Programming 6 06-26-2008 05:11 PM
Supressing Fields w/ AWK Rv5 Programming 3 10-19-2004 12:06 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 04:59 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration