testgrep

picobyte · 09-19-2008, 06:46 PM

This script matches a string to an regexp, and in the case of a failure, shows where the failure occurs.

Save it as testgrep and chmod as required.

For instance test

testgrep "^http://www\.[a-zA-Z0-9.]*\.(com|olg|net)/$" "http://www.linuxquestions.org/"

and you'll get

Code:

^http://www\.[a-zA-Z0-9.]*\.(com|olg|net)/$
                            ^

or testgrep "^h[ea]l*(w?[eo][r ]|e?ld){2,3}$" "hello world"

which will not complain

There might be an error in it, please tell if it misbehaves.
Comments welcome.

Code:

#!/bin/bash

# (C) Roel Kluin, 2008 GPL v.2

# usage: testgrep "match" "teststring"
# output: indicates where grep -E goes wrong
# tests from the left

point_it()
{
  while [ $pt -ne 0 ]; do
    echo -n " ";
    let "pt--";
  done
  echo "^";
  esc=2;
}


# main
i=0;
sb=0;
rb=0
cb=0;
pt=0;
esc=0;
echo "$1"
while [ $i -ne ${#1} ]; do
  if [ $esc -eq 2 ]; then
    break;
  elif [ $esc -eq 1 ]; then
    esc=0;
  elif [ "${1:$i:1}" = "]" ]; then
    [ $sb -lt 1 -o $cb -gt 0 ] && point_it;
    let "sb--";
  elif [ "${1:$i:1}" = ")" ]; then
    [ $rb -lt 1 ] && point_it;
    let "rb--";
  elif [ "${1:$i:1}" = "}" ]; then
    [ $cb -lt 1 -o $sb -gt 0 ] && point_it;
    let "cb--";
  elif [ "${1:$i:1}" = "\\" ]; then
    esc=1;
    [ $sb -gt 0 -o $cb -gt 0 ] && point_it;
  elif [ "${1:$i:1}" = "[" ]; then
    [ $cb -gt 0 ] && point_it;
    let "sb++";
  elif [ "${1:$i:1}" = "(" ]; then
    [ $sb -gt 0 -o $cb -gt 0 ] && point_it;
    let "rb++";
  fi
  let "i++";
  if [ $sb -ne 0 -o $rb -ne 0 -o $esc -ne 0 -o $cb -ne 0 ]; then
    continue;
  elif [ "${1:$i:1}" = "?" ]; then
    [ $sb -gt 0 -o $cb -gt 0 ] && point_it;
    continue;
  elif [ "${1:$i:1}" = "*" ]; then
    [ $sb -gt 0 -o $cb -gt 0 ] && point_it;
    continue;
  elif [ "${1:$i:1}" = "{" ]; then
    [ $sb -gt 0 ] && point_it;
    let "cb++";
    continue;
  fi
  echo "$2" | grep -qE "${1:0:$i}"
  [ $? -ne 0 ] && point_it;
  pt=$i;
done
[ $sb -ne 0 -o $rb -ne 0 -o $esc -eq 1 -o $cb -ne 0 ] && point_it;

ta0kira · 09-19-2008, 08:15 PM

I didn't really take a good look at it, but it sounds like a replacement for something like test $( echo string | egrep -c '^regex$' ) -eq 1? Certainly more graceful.
ta0kira

picobyte · 09-20-2008, 04:33 AM

no, I don't think so. I'll illustrate its usage with an example:

say I want to find all instances in the linux kernel
of :

Code:

x = kmalloc(sizeof(y), GFP_KERNEL)

And I came up with this, which I fed this to my (bash) term:

Code:

al_="[A-Za-z_]";
an_="[A-Za-z0-9_]";
int="[0-9]"
hex="[a-f0-9]"

#whitespace
s="[[:space:]]*";
S="[[:space:]]+";

# to match something like 1ul, floats or hexes as well:
D="$int*\.?$int+x?$hex*[uUlL]{0,3}[fF]?"

# can be used for a variable/function name
V="$al_+$an_*";

# same, but also catches variables that are members or arrays
w="($V|${V}\[$s$an_*$s\]|$V\.|$V->)+"

# match the end of the line, including comments
cendl="$s(\/[*\/].*)?$"

git grep -E "$s$w$s=${s}kmalloc$s\(${s}sizeof$s\($s$w$s\)$s,GFP_KERNEL$s\)$s;$cendl"

But it only finds one match, there's obviously something wrong with my match. I look and find in drivers/atm/nicstar.c, line 920:

Code:

  scq = kmalloc(sizeof(scq_info), GFP_KERNEL);

which should match. Then I do

Code:

testgrep "$s$w$s=${s}kmalloc$s\(${s}sizeof$s\($s$w$s\)$s,GFP_KERNEL$s\)$s;$cendl" "  scq = kmalloc(sizeof(scq_info), GFP_KERNEL);"

and it shows me that the match fails before GFP_KERNEL.

solution: there is an optional space missing:

Code:

git grep -E "$s$w$s=${s}kmalloc$s\(${s}sizeof$s\($s$w$s\)$s,${s}GFP_KERNEL$s\)$s;$cendl"

ta0kira · 09-20-2008, 12:30 PM

Oh, I see now. It took me a while to get it. So you feed the script an expression and a string, knowing it isn't a match, then your script tells you where exactly the discrepancy is since you only have match/no-match otherwise? That sounds very useful.
ta0kira

picobyte · 09-20-2008, 12:52 PM

Yes, instead of success/failure, this script shows where the failure occurs. And thank you, yes, I hope it is useful.

I can think of a few improvements which I may add when I have time:

* also indicate from the right, where the discrepancy ends.
* display discrepant section in colour, instead of just pointing at.

ta0kira · 09-20-2008, 01:06 PM

Maybe break the expression up into a tree using tabs or spaces and show how the string matches up along side it?

Code:

'^h[ea]l*'        ^'hell'
(
  'w?[eo][r ]'    'o '{1}    'wor'{2}
  'e?ld'                     'ld'{2}
){2,3}            'o '{1}    'world'{2}
'$'               $

That's a rash example, but you get the idea.
ta0kira