LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
LinkBack Search this Thread
Old 05-31-2012, 10:10 AM   #1
nano2
Member
 
Registered: May 2007
Posts: 100

Rep: Reputation: 15
using awk on a file


Hi
I have the following jboss_gc.log and i want to convert to csv file .

Any ideas why the awk below is not paring the file correctly ?


gc.log file
Code:
3.811: [GC 129152K->9220K(261184K), 0.0536640 secs]
5.971: [GC 138372K->18688K(261184K), 0.0488410 secs]
7.490: [GC 147840K->28092K(261184K), 0.0252540 secs]
8.291: [GC 157244K->28483K(261184K), 0.0025680 secs]
8.797: [GC 157635K->28655K(261184K), 0.0012670 secs]
9.326: [GC 157807K->29009K(261184K), 0.0016340 secs]
10.062: [GC 158161K->32141K(261184K), 0.0152050 secs]
10.796: [GC 161293K->34653K(261184K), 0.0236380 secs]
17.251: [GC 163805K->39059K(261184K), 0.0163560 secs]
17.879: [GC 168211K->41905K(261184K), 0.0119190 secs]
18.589: [GC 171057K->46506K(261184K), 0.0186890 secs]
20.031: [GC 175658K->53395K(261184K), 0.0597250 secs]
20.091: [GC 55959K(261184K), 0.0096890 secs]
20.414: [GC 182547K->55929K(261184K), 0.0236070 secs]
20.726: [Full GC 120483K->55851K(261184K), 0.3002330 secs]
21.070: [GC 58433K(261184K), 0.0023200 secs]
21.772: [GC 185003K->65093K(261184K), 0.0550760 secs]
22.726: [GC 143416K(261184K), 0.0396670 secs]
23.196: [Full GC 193833K->68977K(261184K), 0.0352740 secs]
23.232: [GC 71357K(261184K), 0.0022760 secs]
24.399: [GC 198129K->70638K(261184K), 0.0158770 secs]
Code:
GC_LOG=$JBOSS"/jboss_gc.log"
grep "Full GC" $GC_LOG | awk '{print $8}' | awk 'BEGIN {FS="K->"};{print $1""$2}' | awk 'BEGIN {FS="K"};{print $1}'| awk '{print $1","$2}'>>/tmp/out.csv
 
Old 05-31-2012, 10:23 AM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,360

Rep: Reputation: 1909Reputation: 1909Reputation: 1909Reputation: 1909Reputation: 1909Reputation: 1909Reputation: 1909Reputation: 1909Reputation: 1909Reputation: 1909Reputation: 1909
None of the lines you posted has 8 fields, so the first
Code:
awk '{print $8}'
prints nothing. I would not use a similar chain of grep and multiple awk processes. You can do it all with a single call to awk. Please, post an example of the desired output, based on the input above, and we could give some better advice.
 
Old 05-31-2012, 10:51 AM   #3
nano2
Member
 
Registered: May 2007
Posts: 100

Original Poster
Rep: Reputation: 15
This is the file
Code:
64519.455: [Full GC 784436K->538165K(784448K), 12.7143926 secs]
164534.199: [GC 547731K(784448K), 0.2120279 secs]
164547.339: [GC 583344K(784448K), 0.3589552 secs]
164551.387: [GC 591125K(784448K), 0.5363759 secs]
164558.084: [GC 605668K(784448K), 0.5198986 secs]
164561.962: [GC 614448K(784448K), 0.6760577 secs]
164568.773: [GC 628705K(784448K), 0.6676036 secs]
164571.995: [GC 636129K(784448K), 0.8861083 secs]
164579.121: [GC 650744K(784448K), 0.8984039 secs]
164583.355: [GC 660185K(784448K), 1.0209936 secs]
164590.619: [GC 675616K(784448K), 1.0320532 secs]
164594.169: [GC 681971K(784448K), 1.1254689 secs]
164601.674: [GC 706273K(784448K), 1.0027303 secs]
164605.759: [GC 711446K(784448K), 1.3872742 secs]
164613.461: [GC 729877K(784448K), 1.3498643 secs]
164617.313: [GC 736181K(784448K), 1.6499854 secs]
164625.248: [GC 753574K(784448K), 1.5982310 secs]
164629.200: [GC 757053K(784448K), 1.5155503 secs]
164637.251: [GC 776190K(784448K), 1.6940726 secs]
164641.190: [GC 780727K(784448K), 1.9722320 secs]
164649.339: [Full GC 784440K->539326K(784448K), 14.2416926 secs]
so i want to take the value 784448K, 14.2416926 secs and graph this
 
Old 05-31-2012, 11:28 AM   #4
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 3,984

Rep: Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000
avoid writing such chains: grep | awk | awk | awk | awk
You need only one awk to implement this:
Code:
awk ' BEGIN { FS="[- ,K>()]*" } /Full GC/ { print $6 "," $7 } ' >> /tmp/out.csv




__________________________________
Happy with solution ... mark as SOLVED
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.

Last edited by pan64; 05-31-2012 at 11:31 AM. Reason: script corrected
 
Old 06-01-2012, 03:26 AM   #5
nano2
Member
 
Registered: May 2007
Posts: 100

Original Poster
Rep: Reputation: 15
This still gives me nothing back

Any ideas why ?
Code:
grep "Full GC"  $GC_LOG | awk ' BEGIN { FS="[- ,K>()]*" } /Full GC/ { print $6 "," $7 } ' >> /tmp/out.csv
 
Old 06-01-2012, 03:33 AM   #6
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 3,984

Rep: Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000
Code:
awk ' BEGIN { FS="[- ,K>()]*" } /Full GC/ { print $6 "," $7 } ' $GC_LOG >> /tmp/out.csv
would be better, it worked for me. probably you can remove >>/tmp/out.csv and test, what can you see?
 
Old 06-01-2012, 03:55 AM   #7
nano2
Member
 
Registered: May 2007
Posts: 100

Original Poster
Rep: Reputation: 15
Thanks that worked now if i also want to check if there was more than one Full GC string in the file i will throw an error
How could i handle that ?
 
Old 06-01-2012, 04:11 AM   #8
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 3,984

Rep: Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000Reputation: 1000
Code:
awk ' BEGIN { FS="[- ,K>()]*"; counter=0 } /Full GC/ { if ( counter++ ) { exit 1 } else print $6 "," $7 } '  $GC_LOG >> /tmp/out.csv
you can modify { exit 1 } to print a message also if you want to do so:
{ print "THIS IS AN ERROR" > "/dev/stderr"; exit 1 }


_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
(located in the "thread tools")
 
  


Reply

Tags
awk, bash scripting


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
running series of awk commands from an awk file oreka18 Programming 3 05-16-2012 01:13 AM
awk error awk: line 2: missing } near end of file boscop Linux - Networking 2 04-08-2012 10:49 AM
[SOLVED] replace matched pattern from 1st file into 2nd file using awk sopier Programming 6 12-13-2011 09:58 AM
AWK/BASH: get nth line from a file by getline feed to actions in a same awk line cristalp Programming 3 11-23-2011 11:38 AM
parsing a text file - to awk or not to awk ? rollyah Programming 9 08-18-2011 02:20 PM


All times are GMT -5. The time now is 12:09 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration