[SOLVED] how do i keep the header

grail · 01-21-2014, 12:34 AM

You will need to add 2 more fields smeezekitty, there are 16 space separated columns.

smeezekitty · 01-21-2014, 02:05 AM

Quote:

Originally Posted by grail

You will need to add 2 more fields smeezekitty, there are 16 space separated columns.

Thanks for noticing. The code had a bunch of bugs now it really works

schneidz · 01-21-2014, 07:57 AM

Quote:

Originally Posted by tabbygirl1990

schneidz - i'm sorry, i didn't mean to be snooty at all, one of those days, thank you for your help.

when i ran firerat's script in post 5, my output file came back empty

when i ran grail's command line in post #10, my output file came back with the line after the line that met the filter criteria, but alos no header

so, here's an example space deliminated input file

Code:

DATE		TIME       OPERATOR VERSION  RUN_ID  SPEC      DTG      FAIL  END    ONGOING  FEATURE_1   FEATURE_2      FEATURE_3 GOODNESS     
12/04/2013      6:00:011.27   SM    2.6.8 6   90501   5   921996008.31 FALSE 5  *     5      0.711503131 5660093.929    6.22      0.91
12/05/2013      6:00:011.3     DK    2.6.8 6   90501   4   921996009.31 FALSE 8  *     8      0.567142359 5660095.848    0.53      0.90
12/06/2013      6:00:011.41    SM    2.6.8 5   90503   2   921996009.01 FALSE 8  *     8      0.708699814 5660097.221    0.54      0.91
12/06/2013      6:00:011.41    JF    2.6.8 6   90501   5   921996010.31 FALSE 3  *     3      0.142189285 5660100.259    -0.27      0.08
12/09/2013      6:00:011.55    SM    2.6.8 6   90501   1   921996010.01 FALSE 8  *     8      0.213247275 5660103.596    -0.27      0.08
12/10/2013      6:00:011.41   SM    2.6.8 4   90503   5   921996011.31 FALSE 8  *     8      0.91836074 5660103.492    0.53       0.91
12/10/2013      6:00:011.32   SM    2.6.8 4   90501   5   921996011.01 TRUE 1
12/11/2013      6:00:011.21   DK    2.6.8 4   90501   3   921996015.01 TRUE 1
12/11/2013      6:00:011.42   SM    2.6.8 4   90501   3   921996015.01 FALSE 10  *    10      0.864147301 5660105.265    0.622     0.91
12/12/2013      6:00:011.50   JF    2.6.8 4   90501   3   921996015.31 FALSE 8  *     8      0.539123318 5660104.795    0.622     0.92
12/13/2013      6:00:011.15   SM    2.6.8 4   90503   5   921996016.01 FALSE 2  *     2      0.922633758 5660109.457    7.05      0.96

if i filter on OPERATOR=SM and SPEC=5 then what I'd like to getout is

Code:

DATE		TIME       OPERATOR VERSION  RUN_ID  SPEC      DTG      FAIL  END FEATURE_1   FEATURE_2      FEATURE_3 GOODNESS     
12/04/2013      6:00:011.27   SM    2.6.8 6   90501   5   921996008.31  FALSE  5  0.711503131 5660093.929    6.22      0.91
12/10/2013      6:00:011.41   SM    2.6.8 4   90503   5   921996011.31  FALSE  8  0.91836074 5660103.492    0.53      0.88
12/10/2013      6:00:011.32   SM    2.6.8 4   90501   5   921996011.01  TRUE  1
12/13/2013      6:00:011.15   SM    2.6.8 4   90503   5   921996016.01  FALSE  2  0.922633758 5660109.457    7.05      0.96

the files that i'm trying to process are much much bigger but i think this little one covers all the cases

thanks so much guys!!!

tabby

sure no prob... does this help as far as text padding go:

Code:

awk 'NR == 1 || ( $3 == "SM" && $7 == "5" ) {printf("%-10s %-10s %-10s %-10s %-10s %-10s\n", $1, $2, $3, $11, $12, $5)}' tabbygirl.lst
DATE       TIME       OPERATOR   FEATURE_1  FEATURE_2  RUN_ID    
12/04/2013 6:00:011.27 SM         *          5          6         
12/10/2013 6:00:011.41 SM         *          8          4         
12/10/2013 6:00:011.32 SM                               4         
12/13/2013 6:00:011.15 SM         *          2          4

also, can someone help me understand why 5 is matched against $7 eventhough it should be the $6 field ?

edit: oh i get it, i see that the fields are mis-aligned in the output for some reason...
is version supposed to have a space ?; does fail end with a number or does end begin with a number (or possible column heading missing between fail and end) ?
perhaps comma-delimited input would work better ?

tabbygirl1990 · 01-21-2014, 09:03 AM

morning guys

my office mate said that both the 2.6.8 and the number immeadiately following 2.6.8 ( that would be the 6 in the first row ) are part of the VERSION. she said that the the 2.6.8 is a "known" and when i do a grep statement on all the files the 2.6.8 shows up in all of them, but the number following the 2.6.8 does change. so she thinks i can ignore the 2.6.8 no need to repeat it, and just keep the number following it.

i'm going to test out some of your all's help, and see what comes back.

many thanks, tabby

tabbygirl1990 · 01-21-2014, 09:37 AM

smeezekitty, i know it's very powerful, Perl is pretty scary to me. i blunder my way through awk sed and bash getting lots of help from the guys here learning as i go and i am soooo very thankful to them!

unless i could sit through a class on Perl (and i don't know when i could), i can't see my learning it. for me Perl is kinda like i gladly appreciate the gifts!

grail · 01-21-2014, 10:02 AM

As an alternative to padding may I suggest you also look at the column command. I find it quite useful to simply print from awk with a standard space separation and let column handle the formatting

smeezekitty · 01-21-2014, 12:38 PM

Quote:

Originally Posted by tabbygirl1990

smeezekitty, i know it's very powerful, Perl is pretty scary to me. i blunder my way through awk sed and bash getting lots of help from the guys here learning as i go and i am soooo very thankful to them!

unless i could sit through a class on Perl (and i don't know when i could), i can't see my learning it. for me Perl is kinda like i gladly appreciate the gifts!

Perl isn't so bad. And any many applications its actually simpler than awk

tabbygirl1990 · 01-21-2014, 02:17 PM

hi guys

here's the script i put together after playing with grail's in the end it looks kinda simple but getting there was hard

Code:

#                    3   4    5   6  7   8    9    10    11    12  13  14  15
BEGIN  {  fmt[1] = "%s  %s   %s  %s  %s  %s  %.0s   %s   %s    %s  %s  %s  %s \n" # header
          fmt[2] = "%s  %.0s %d  %d  %d  %f   %s   %.0s  %.0s  %f  %f  %f  %f \n" # the columns types
       }
          NR == 1 || ( $3 == "SM" && $7 == 5 ){ 
 
          if($9=="TRUE")
            printf(fmt[NR==1?1:2], $3, $4, $5, $6, $7, $8, $9, $11, $12, "NaN", "NaN", "NaN", "NaN");
          else
            printf(fmt[NR==1?1:2], $3, $4, $5, $6, $7, $8, $9, $11, $12,  $13,   $14,   $15,   $16);
       }
END{}

wohooo look, i even added an if statement, now i'm a programmer

i put in all the extra spces myself just to line everything up, but i need to learn how to do that too...

Code:

OPERATOR VERSION  RUN_ID  SPEC      DTG      FAIL  END  FEATURE_1    FEATURE_2      FEATURE_3  GOODNESS     
   SM       6      90501   5   921996008.31  FALSE  5   0.711503131  5660093.929    6.22        0.91
   SM       4      90503   5   921996011.31  FALSE  8   0.91836074   5660103.492    0.53        0.88
   SM       4      90501   5   921996011.01  TRUE   1   NaN          NaN            NaN         NaN
   SM       4      90503   5   921996016.01  FALSE  2   0.922633758  5660109.457    7.05        0.96

it is kinda tricky to figure out what goes into fmt[1] fmt[2] and the printf statements to get out the right stuff, but i think i'm ready to tackle the next bigger file of 53 columns - yikes!

tabbygirl1990 · 01-21-2014, 02:29 PM

smeezekitty,

i'm not really a "programer" you can probably tell

i needed a job and i had a class at USC and i'm good in math, so they hired me. there's a guy here (well up in San Fransico) that can help, but he's not here very often, glad i don't have a demanding boss, so it's just me and my cube-mate, and she's less technical than me, but someday maybe???

so between work, apartment life, and playtime in the ocean, i'm one busy tabbygirl

but i do really really appreciate all the help you guys give and i think i'm learning a few "geekie" programmer things

grail · 01-21-2014, 08:22 PM

So not sure if you typed it for later, but just to let you know, you do not need the END{}.

Also, you can turn file into a script by placing the interpreter inside the file at the top line:

Code:

/usr/bin/awk -f

Of course you need to point to wherever your copy of awk is located.

tabbygirl1990 · 01-22-2014, 03:17 PM

firerat,

i was able to make some other "formatting" changes to my input file and so now I can almost get your script on post number #5 to go

Code:

BEGIN {
   FS = ' '
   }
   {
   if ( NR == 1 ) {
       print $0
       } else {
        if ($3=="42" && $5=="the answer to the universe")
           printf("%f %f %d %f %f %s\, $1, $2, $3, $11, $12, $5)
       }
   }
END{}

i wrote it like this with an additional if-else stament

Code:

BEGIN {
       FS = " "
      }
      {
      if( NR == 1) {
       printf("%s,%s,%s,%s,%s,%s\n")
       #print $0
       } else {
          if($4==9501 && $5== 9 && $21!=0)
           printf("%f %f %d %f %f %s\, $1, $2, $3, "NaN", "NaN", "NaN");
         else
           printf("%f %f %d %f %f %s\, $1, $2, $3, $11, $12, $5);
       }
       }
END{}

i get

Code:

fatal: not enough arguments to satisfy formating string  '%s,%s,%s,%s,%s,%s  ^ ran out for this one

if i just use the print 0 command, i get 29 headers and 6 columns of data, but i only want the 6 heeaders not the 29, so i know the logic is working because with print $0 it gives 29.

can you see my error?

thanks sooo much! tabby

grail · 01-23-2014, 06:59 AM

The error comes from not supplying any input to the printf command:

Code:

printf("%s,%s,%s,%s,%s,%s\n")

So here you have told printf that there will be 6 string items to print but you pass in none, ie after the last double quotes should appear 6 comma separated items.

i would also note that as they stand, neither of the other 2 printf commands would work either. Here you need to review the number of double quotes you have used and also note that you have no new line
in either so all data presented by these lines will be continuous.

tabbygirl1990 · 01-23-2014, 09:31 AM

good mornig grail,

yep, thanks you, as always, you are absoluetly correct

the problem that i have is that the data files and awk scripts i write are not on a machine that's connected to the internet, and i'm not allowed to copy data files to an internet machine because of privacy rules. so i my only choice is to look at one screen and type on the other, and so sometimes i mis-type or forget to type stuff

and i know how syntax specific awk, sed, and bash scripts are, sorry.

yes the two last printf statements should be

Code:

printf("%f %f %d %s %s %s\n", $1, $2, $3, "NaN", "NaN", "NaN");

and

Code:

printf("%f %f %d %f %f %s\n", $1, $2, $3, $11, $12, $5);

and they are written that way in the script on my linux machine

with the first printf statement, i know i could write

Code:

printf("%s,%s,%s,%s,%s,%s\n", "A", "B", "C", "D", "E", "F");

but since

Code:

print $0

gave me everything in the first line, I was hoping (although i didn't know how it could as it's currently written) not to have to declare (other than as strings) the 6 fields that i want, that the script would ust pull the 6 header field's that align with the printf statments

so i guess i'll have to write the 6 header fileds out explicitly

thanks soooo much guys, muah,

tabby

tabbygirl1990 · 01-23-2014, 12:37 PM

well that did it, it all works.

a modified version of firerat's post 5 and the corrections to my erros by grail

thanks sooooo much guys

tabby