[SOLVED] Search multiple patterns & print matching patterns instead of whole line

grail · 02-29-2012, 01:57 PM

This version removes duplicates. I also removed the function as it added a large portion of time to the calculation:

Code:

#!/usr/bin/awk -f

BEGIN{ OFS = FS = "\t" }

FILENAME ~ /file/{
    if(FILENAME ~ /1/)
        pat1=pat1 (pat1?"|":"\\<(") $0
    else
        pat2=pat2 (pat2?"|":"\\<(") $0

    next
}

!x{ pat1 = pat1 ")\\>"
    pat2 = pat2 ")\\>"
    x=1
}

{
    str = $3
    $3 = $4 = ""
    l = k = 0

    while(match(substr(str,k), pat1, f)){
        if($3 !~ f[0])
            $3 = $3 ($3?",":"") f[0]
        k = k + f[0,"start"] + f[0,"length"]
    }
    if($3){
        while(match(substr(str,l), pat2, f)){
        if($4 !~ f[0])
            $4 = $4 ($4?",":"") f[0]
            l = l + f[0,"start"] + f[0,"length"]
        }
        if($4)
            print
    }
}

firstfire · 02-29-2012, 02:11 PM

I found that mawk runs much faster than gawk. Last solution could probably be made even faster, but, unfortunately, mawk does not supports match() with three arguments.

Trd300 · 02-29-2012, 07:47 PM

%%%%%

Trd300 · 02-29-2012, 08:04 PM

%%%%%

grail · 02-29-2012, 11:03 PM

Firstly, to run you only need to make it executable and run like so:

Code:

./search.awk file1.tab file2.tab bigdb.tab

Please find out which version of awk you are linked to:

Code:

ls -l /usr/bin/awk

Trd300 · 02-29-2012, 11:34 PM

Yep, I forgot to change rights !

My version of awk:

Code:

-rwxr-xr-x  1 root  wheel  238400 25 Jun  2010 /usr/bin/awk

Trd300 · 02-29-2012, 11:40 PM

Still the same error though:

Code:

./search.awk file1.tab file2.tab bigdb.tab
/usr/bin/awk: syntax error at source line 24 source file ./search.awk
 context is
        (match(substr(str,k), >>>  pat1, <<<  f)){
/usr/bin/awk: illegal statement at source line 25 source file ./search.awk
/usr/bin/awk: illegal statement at source line 25 source file ./search.awk

grail · 02-29-2012, 11:53 PM

Hmmm ... is there any chance you have gawk installed as well? By the look of it I would say the straight form of awk does not support some the features I have used.

Trd300 · 02-29-2012, 11:56 PM

I don't have gawk installed.

Which version would suit?

grail · 03-01-2012, 12:01 AM

3.1.8 up to the latest

grail · 03-01-2012, 10:14 AM

Hey I am not sure if you would have ruby installed, but I thought the following solution was sweet

Code:

#!/usr/bin/ruby

BEGIN{ $, = $; = "\t" }

while gets
    if $FILENAME[/file/]
        if $FILENAME[/1/]
            pat1 = "#{pat1}#{pat1 ? "|" : ""}#{$_.chomp}"
        else
            pat2 = "#{pat2}#{pat2 ? "|" : ""}#{$_.chomp}"
        end
    end

    if $FILENAME[/big/]
        line = $_.split

        puts line.join if (line[3] = line[2].scan(/\b(#{pat2})\b/).uniq.join(",")) && !line[3].empty? &&
                          (line[2] = line[2].scan(/\b(#{pat1})\b/).uniq.join(",")) && !line[2].empty?
    end
end

It does run a little slower than the gawk solution but it is also quite a bit tidier (although I am sure one of the ruby pros could probably do more

)

This is run the same as the gawk solution.

Trd300 · 03-04-2012, 08:13 PM

Sorry for the late response.

It works perfectly with gawk, and it is pretty fast.

Thanks a lot grail !

Thanks firstfire !

Trd300 · 03-04-2012, 11:20 PM

%%%%%

grail · 03-05-2012, 03:06 AM

The main idea for the first part of the script is that it needs to know the names you are looking for. To help facilitate we can make the following change:

Code:

BEGIN{ OFS = FS = "\t"
       if(ARGC != 4){
           print "Wrong number of arguments"
           exit
       }

       file1 = ARGV[1]
       file2 = ARGV[2]
}

FILENAME == file1 || FILENAME == file2{
    if( FILENAME == file1 )
        pat1=pat1 (pat1?"|":"\\<(") $0
    else
        pat2=pat2 (pat2?"|":"\\<(") $0

    next
}

As long as you now place the files being used for the checking first you should be fine

Trd300 · 03-05-2012, 07:41 PM

%%%%%