Awk / Grep / sed

rahul090590 · 06-18-2015, 08:25 AM

Hi I have a java file,

public class Test {

public static void main(String args[]) {
int x = 10;

while( x < 5)
{
}

while( x < 20 ) {
System.out.print("value of x : " + x );
x++;
System.out.print("\n");
}
}
}

Now as you can see there are two while loops , one while loop is empty and other while loop has got some logic.

Now I would like to print the while loop which has got no logic in it. In other words I would like to grep and print the while loop which does not have any logic. How can I grep Multiple lines. So that I can print the while loop which has no logic.

Thanks.

netnix99 · 06-18-2015, 08:43 AM

I don't understand what you want to print. I am not a Java programmer, but it appears that you initialize a variable (x) and set it equal to 10. In your first while loop, you state while X < 5, but X will never be less than 5 in this case, so what would you expect to print.

rahul090590 · 06-18-2015, 10:05 AM

Hi, I would just like to grep (do a pattern matching) on the java file. This java file is just a sample. In real scenario I would like to do a pattern match that identifies all while loops (the loops which are empty i.e does not have logic between { and } ) from many java files in my project.

Hope I am clear.

firstfire · 06-18-2015, 10:25 AM

Hi.

In this particular case you may use something like this:

Code:

$ perl -0777 -ne 'print "$&\n" if /while[^{]*{[ \n]*}/' infile
while( x < 5)
{
}

Here -0777 enables so called slurp mode -- perl will read entire file into memory before attempting pattern match (in this case the paragraph mode -00 could be used too). This allows to match multiline patterns.

In general you should consider using a parser for such kind of processing. It may be a complete java parser (e.g. this one) or a surface grammar (the one capturing only specific syntactic structures and ignoring everything else) parser. In the latter case Parsing Expression Grammars come in handy.

danielbmartin · 06-18-2015, 10:43 AM

Quote:

Originally Posted by rahul090590

... I would like to print the while loop which has got no logic in it. ...

Can these files contain multiple instances of "empty" (i.e. no logic) while loops? If so, do you want to print all such code segments?
Example:

Code:

public class Test {

public static void main(String args[]) {
int x = 10;

while( x < 5)
{
}

while( x < 99)
{
}

while( x < 20 ) {
System.out.print("value of x : " + x );
x++;

System.out.print("\n");
}
}
}

Daniel B. Martin

danielbmartin · 06-18-2015, 11:01 AM

With this InFile ...

Code:

public class Test {

public static void main(String args[]) {
int x = 10;

while( x < 5)
{
}

while( x < 99)
{
}

while( x < 20 ) {
System.out.print("value of x : " + x );
x++;

System.out.print("\n");
}
}
}

... this code ...

Code:

 tr "\n" "~" <$InFile  \
|sed 's/while(/\nwhile(/g'  \
|grep "{"~*"}"              \
|tr "~" "\n"                \
>$OutFile

... produced this OutFile ...

Code:

while( x < 5)
{
}


while( x < 99)
{
}

Daniel B. Martin

rahul090590 · 06-18-2015, 12:59 PM

Thanks alot. And one more small doubt here. If I would like to look for comments also

suppose

while (x<5) {
// some comments here

}

Then how would be the regular expression.

danielbmartin · 06-18-2015, 01:08 PM

Quote:

Originally Posted by rahul090590

Thanks alot. And one more small doubt here. If I would like to look for comments also

suppose

while (x<5) {
// some comments here

}

Then how would be the regular expression.

Be more specific. Do you want such "commented" WHILE loops reported instead of, or in addition to, the empty loops.

Daniel B. Martin

danielbmartin · 06-18-2015, 02:40 PM

This code ...

Code:

 tr "\n" "~" <$InFile       \
|sed 's/while(/\nwhile(/g'  \
|grep "{[~ ]*}"             \
|tr "~" "\n"                \
>$OutFile

... read this InFile ...

Code:

public class Test {

public static void main(String args[]) {
int x = 10;

while( x < 5 )
{
}

while( x < 6 )
{

}

while( x < 7 ) {}


while( x < 8 ) {  }

while (x<9) {
// some comments here

}

while( x < 20 ) {
System.out.print("value of x : " + x );
x++;

System.out.print("\n");
}
}
}

... and produced this OutFile ...

Code:

while( x < 5 )
{
}


while( x < 6 )
{

}


while( x < 7 ) {}



while( x < 8 ) {  }

while (x<9) {
// some comments here

}

Daniel B. Martin

firstfire · 06-18-2015, 03:03 PM

Hi.

To illustrate my point about using parsers here is an example PEG parser written in python using pyparsing library:

Code:

$ cat parser.py
import pyparsing as pp
import sys

emptyLoopBody = pp.ZeroOrMore( pp.javaStyleComment ^ pp.White() )
emptyLoop = pp.Combine( pp.Regex(r"while[^{]+{") + emptyLoopBody + "}" )
parser = pp.ZeroOrMore( pp.Suppress(pp.SkipTo(emptyLoop)) + emptyLoop)

for fn in sys.argv[1:]:
    for r in parser.parseFile(fn):
        print r

$ #demo run
$ python parser.py infile
while( x < 5 )
{
}
while( x < 6 )
{

}
while( x < 7 ) {}
while( x < 8 ) {  }
while (x<9) {
// some comments here

}

where infile is the same input data as used by Daniel above.

To install pyparsing in Debian based distro:

Code:

$ sudo apt-get install pyhon-pyparsing

Or use pip install pyparsing (sort of).

Sure, it runs slower than regex based solutions, but gives more flexibility.

EDIT: Here is a good reference guide on pyparsing.

rahul090590 · 06-18-2015, 09:21 PM

Hi thanks alot. Would surely take a look in to this.

And in perl if I had to match //comments then what change would be required in the regular expression. Thanks again.

rahul090590 · 06-19-2015, 04:17 AM

I used pyparsing and it works well and I am very new to all these I dont know how to edit these scripts.

Can you also tell me how to get the line number.

firstfire · 06-19-2015, 05:21 AM

Hi.

Code:

import pyparsing as pp
import sys

emptyLoopBody = pp.ZeroOrMore( pp.javaStyleComment ^ pp.White() )
emptyLoop = pp.Combine( pp.Regex(r"while[^{]+{") + emptyLoopBody + "}" )
parser = pp.ZeroOrMore( pp.Suppress(pp.SkipTo(emptyLoop)) + emptyLoop )

def printLocation(s, loc, toks):
    print "###### Match at line", pp.lineno(loc, s)
    print toks[0]

emptyLoop.setParseAction(printLocation)

for fn in sys.argv[1:]:
    parser.parseFile(fn)

Code:

$ python parser.py ./infile
###### Match at line 6
while( x < 5 )
{
}
###### Match at line 10
while( x < 6 )
{

}
###### Match at line 15
while( x < 7 ) {}
###### Match at line 18
while( x < 8 ) {  }
###### Match at line 20
while (x<9) {
// some comments here

}

rahul090590 · 06-24-2015, 08:16 AM

Hi

Thanks for you help.

Suppose If I want to take only loops with java comments and to get that I did

import pyparsing as pp
import sys

emptyLoopBody = pp.ZeroOrMore( pp.javaStyleComment )
emptyLoop = pp.Combine( pp.Regex(r"while[^{]+{") + emptyLoopBody + "}" )
............

When I give this it does not find any loop which has java style comments.

And for getting white space if I give

emptyLoopBody = pp.ZeroOrMore( pp.White())
.............

it matches white space and returns the match.

Am I doing anything wrong for java comments.

Thanks again.

firstfire · 06-24-2015, 02:55 PM

Hi.

The following code searches for loops with one or more comments inside:

Code:

import pyparsing as pp
import sys

emptyLoopBody = pp.OneOrMore( pp.javaStyleComment )
emptyLoop = pp.Regex(r"while[^{]+{") + emptyLoopBody + "}"

parser = pp.ZeroOrMore( pp.Suppress(pp.SkipTo(emptyLoop)) + emptyLoop )

def printLocation(s, loc, toks):
    print "###### Match at line", pp.lineno(loc, s)
    end = pp.getTokensEndLoc()
    print s[loc:end]

emptyLoop.setParseAction(printLocation)

for fn in sys.argv[1:]:
    parser.parseFile(fn)

By default pyparsing ignores whitespaces (spaces, tabs and newlines). I removed pp.Combine() because it treats whitespaces specially and instead used pp.getTokensEndLoc() function to determine where is the end of the match. This allows to remove pp.White() and tell pp.OneOrMore(pp.javaStyleComment) since our grammar is whitespace agnostic now.