[SOLVED] Awk inside Awk ?

acc_Wk · 05-23-2012, 01:34 AM

Hi,
i'm using awk inside awk(via command substitution) and I want to apply the FS of the outer awk to inner awk..

My main goal is to actually insert a line into a table conforming to the format of that table :
Eg :- i ve two files : (i wasnt able to show multiple spaces as they were getting truncated, so i have indicated a space with <space>
---------------------------------------------------------
cat f1
hdr1<space>hdr2<space>hdr3<space>hdr4

abcd<space>bcef<space>pqrs<space>xyza
---------------------------------------------------------
cat f2
hdr1<space><space>hdr2<space><space>hdr3<space><space>hdr4

1111<space><space>2222<space><space>3333<space><space>4444

My desired output is :
----------------------------------------------------------
hdr1<space><space>hdr2<space><space>hdr3<space><space>hdr4

abcd<space><space>bcef<space><space>pqrs<space><space>xyza
1111<space><space>2222<space><space>3333<space><space>4444
----------------------------------------------------------

(observe that though headers are same in both files, spacing is different in the two files, so i want line in f1 in f2 with f2's format)

For this I was planning on using the following awk code :

cat f2 | awk -v "n=2" -v "s=` cat f1| sed 1d | awk 'BEGIN { FS = "<outer awk's FS>" }; {printf "%s %s %s %s %s %s %s\n", $1,$2,$3,$4,$5,$6,$7}'`" '(NR==n) { print s } 1'

Is this actually possible ? or is there better approach to achieve the desired output ? Any help/hints would be much appreciated!!

Thanks!!

jhwilliams · 05-23-2012, 02:14 AM

The following AWK script would accomplish the output you desire:

Code:

{
    gsub(/[ \t]+/,"  ",$0)
}

/^hdr/ {
    if (0 == seen) {
        print $0"\n"
    }

    seen++
    next
}

!/^$/ {
    print $0
}

Save that as parse.awk, and then invoke like this:

Code:

awk -f parse.awk f1 f2

acc_Wk · 05-23-2012, 02:33 AM

Hi,

Sorry I wasnt very clear about this.. The second file (f2) doesnt have a uniform spacing like (2 spaces between the cols), it can have a more random format like :

hdr1<space><space>hdr2<space><space><space><space>hdr3<space>hdr4
aaaa<space><space>bbbb<space><space><space><space>cccc<space>dddd

So basically there's no clarity on the spacings.. All that i'm sure of is, both the files have the same headers and I need to merge the (only) line from one file to the another file having many lines (with same column headers)..

Thanks for your time though!!

jhwilliams · 05-23-2012, 02:52 AM

Hi acc, that shouldn't matter. Did you try it? The first action of the script I provided is to normalize spacing to two spaces, everywhere.

acc_Wk · 05-23-2012, 03:46 AM

Hi jhwilliams, yes i did try.. but also i realized now, I am not giving the best sample example data..

The values under the headers extend longer than the headers...

I tried attaching a file but didnt work.. So please see if below is easy to understand ( _ == space)

Code:

f1 :
Header1_Header2_________Header3______Header4_Header5_Header6_____Header7

AAAAA___QQQQQQQ_________FFFFFF_______SSSSSS__-_______-___________-


f2:
Header1_____________________________Header2_________Header3______Header4_Header5_Header6_____Header7

AAAAAAAAAAAAAAAAAAAAAAAAAAA_________BBBBBBBB________CCCCCCC______DDDDDD__XXXXXX__-___________-
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP_____EEEEEEEE________FFFFFFF______GGGGGG__YYYYYY__-___________-

desired:

Header1_____________________________Header2_________Header3______Header4_Header5_Header6_____Header7

AAAAA_______________________________QQQQQQQ_________FFFFFF_______SSSSSS__-_______-___________-
AAAAAAAAAAAAAAAAAAAAAAAAAAA_________BBBBBBBB________CCCCCCC______DDDDDD__XXXXXX__-___________-
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP_____EEEEEEEE________FFFFFFF______GGGGGG__YYYYYY__-___________-

Ok i really am finding it hard to convey the intended format.. In edit mode, it looks ok but once posted the format is changing.. Anyway i hope i've given a fair idea of what i'm looking for.. In the above sample data, the only thing to keep in mind for column values is, they start from where it's corresponding column header starts.. barring that the above sample data is fine..

Appreciate your helP!!

Thanks!

syg00 · 05-23-2012, 05:30 AM

Use [code] tags as above - from "advanced" it's the "#" token.

grail · 05-23-2012, 06:46 AM

Maybe something like:

Code:

awk 'FNR==NR || (FNR > 1 && NF){$1=$1;print}' OFS="  " f1 f2

acc_Wk · 05-23-2012, 06:57 AM

Hi,

Thanks syg00

So finally, here is the format :

Code:

f1
Header1 Header2         Header3      Header4 Header5 Header6     Header7

AAAAA   QQQQQQQ         FFFFFF       SSSSSS  -       -           -

___________________________________________________________________________________________________
f2
Header1                             Header2         Header3      Header4 Header5 Header6     Header7

AAAAAAAAAAAAAAAAAAAAAAAAAAA         BBBBBBBB        CCCCCCC      DDDDDD  XXXXXX  -           -
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP     EEEEEEEE        FFFFFFF      GGGGGG  YYYYYY  -           -

___________________________________________________________________________________________________
Desired :

Header1                             Header2         Header3      Header4 Header5 Header6     Header7

AAAAA                               QQQQQQQ         FFFFFF       SSSSSS  -       -           -
AAAAAAAAAAAAAAAAAAAAAAAAAAA         BBBBBBBB        CCCCCCC      DDDDDD  XXXXXX  -           -
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP     EEEEEEEE        FFFFFFF      GGGGGG  YYYYYY  -           -

@grail : It didnt solve the formatting problem for me , when I gave the two files as inputs to the awk line you suggested...

Code:

# 
awk 'FNR==NR || (FNR > 1 && NF){$1=$1;print}' OFS="  " t1 t2
Header1  Header2  Header3  Header4  Header5  Header6  Header7

AAAAA  QQQQQQQ  FFFFFF  SSSSSS  -  -  -
AAAAAAAAAAAAAAAAAAAAAAAAAAA  BBBBBBBB  CCCCCCC  DDDDDD  XXXXXX  -  -
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP  EEEEEEEE  FFFFFFF  GGGGGG  YYYYYY  -  -

acc_Wk · 05-23-2012, 08:29 AM

Hi,

Thanks syg00

So finally, here is the format :

Code:

f1
Header1 Header2         Header3      Header4 Header5 Header6     Header7

AAAAA   QQQQQQQ         FFFFFF       SSSSSS  -       -           -

___________________________________________________________________________________________________
f2
Header1                             Header2         Header3      Header4 Header5 Header6     Header7

AAAAAAAAAAAAAAAAAAAAAAAAAAA         BBBBBBBB        CCCCCCC      DDDDDD  XXXXXX  -           -
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP     EEEEEEEE        FFFFFFF      GGGGGG  YYYYYY  -           -

___________________________________________________________________________________________________
Desired :

Header1                             Header2         Header3      Header4 Header5 Header6     Header7

AAAAA                               QQQQQQQ         FFFFFF       SSSSSS  -       -           -
AAAAAAAAAAAAAAAAAAAAAAAAAAA         BBBBBBBB        CCCCCCC      DDDDDD  XXXXXX  -           -
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP     EEEEEEEE        FFFFFFF      GGGGGG  YYYYYY  -           -

@grail : It didnt solve the formatting problem for me , when I gave the two files as inputs to the awk line you suggested...

Code:

# 
awk 'FNR==NR || (FNR > 1 && NF){$1=$1;print}' OFS="  " t1 t2
Header1  Header2  Header3  Header4  Header5  Header6  Header7

AAAAA  QQQQQQQ  FFFFFF  SSSSSS  -  -  -
AAAAAAAAAAAAAAAAAAAAAAAAAAA  BBBBBBBB  CCCCCCC  DDDDDD  XXXXXX  -  -
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP  EEEEEEEE  FFFFFFF  GGGGGG  YYYYYY  -  -

colucix · 05-23-2012, 09:47 AM

Quote:

Originally Posted by acc_Wk

In the above sample data, the only thing to keep in mind for column values is, they start from where it's corresponding column header starts.. barring that the above sample data is fine..

In your words there is the solution. Just check where the headers start and set the proper format for each field. This assumes that all the headers are different from each other, otherwise the index function fails because it returns the starting position of the leftmost occurrence.

Code:

awk 'BEGIN {

  getline < ARGV[2]
  
  for ( i = 1; i <= NF; i++ )
    n[i] = index($0, $i)
    
  for ( i = 2; i <= NF; i++ )
    f[i-1] = ( "%-" n[i]-n[i-1] "s" )
  
  f[i-1] = "%-s"
  
  print
  print ""
  
}

FNR > 2 {

  for ( i = 1; i <= NF; i++ )
    printf f[i], $i
  
  print ""
  
}' f1 f2

Notice that the arguments must be exactly in this order, f1 then f2. The BEGIN section reads the header of the second file to retrieve the format (and prints it followed by an empty line, as in your example). Then the content of f1 is printed out with the format retrieved from f2. Finally the content of f2 is appended. Hope this helps.

grail · 05-23-2012, 12:54 PM

Quote:

It didnt solve the formatting problem for me , when I gave the two files as inputs to the awk line you suggested...

Actually ... it provided exactly the output you initially requested, ie all data from both files with only a single header and 2 spaces between each data item.

Your new requirement to have the data formatted in columns is best suited to the column command:

Code:

awk 'FNR==NR || (FNR > 1 && NF){$1=$1;print}' f1 f2 | column -t

If you still want the blank line you can play with awk some more.

jhwilliams · 05-23-2012, 06:41 PM

Here's a tweak to my original post, to meet your additional "pretty print" requirement:

Code:

column -t f1 f2 | awk '
    /^Header/ {
        if (!seen) {
            print $0"\n"
        }
    
        seen++
        next
    } 

    !/^$/'

acc_Wk · 05-24-2012, 02:21 AM

colucix, grail, jhwilliams... Thanks so much!! I was struggling for a long time to get one and now I have 3 ways for it.. You guys are awKsome!

.. Thanks!

David the H. · 05-24-2012, 06:15 PM

Could you please go back and edit your previous posts to include code tags as well? The long lines are making my screen side-scroll. Thanks.