LinuxQuestions.org - help with glob - perl invoked from shell

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - help with glob - perl invoked from shell (https://www.linuxquestions.org/questions/programming-9/help-with-glob-perl-invoked-from-shell-682930/)

johngreg

11-12-2008 11:57 AM

help with glob - perl invoked from shell

i am facing difficulty with glob - what i want is - i need to parse all '.log' files and generate a report. I am using perl to do this - this is invoked from a simple shell script which is as follows
globtry.sh:-

Code:

#!/bin/sh

perl globreport.pl $@

perl script is as below
-------------------------

Code:

#!/usr/bin/perl -w



use strict;

use warnings;

use Data::Dumper;

use Getopt::Long;

use File::Spec;



my $file = pop @ARGV;

my @files = glob($file);

showUsage() unless $file;



foreach my $inputfile (@files)  {

    if ( -r $inputfile ) {

        print "Processing file :: $inputfile\n";

        #all logic to generate report

    }else{

        print "couldnot do anything on :: $inputfile";

    }

}

the problem is, when i run

Code:

./globtry.sh *.log

it is picking up only one file and exiting application.
Please help.

keefaz

11-12-2008 12:05 PM

It would be because the shell expands *.log before perl, I mean in this case perl see the files list expanded

Try:

Code:

#!/usr/bin/perl

for my $x(@ARGV) {

  print $x."\n";

}

then execute it, like

Code:

./myscript.pl *.log

and try once again with:

Code:

./myscript.pl '*.log'

Telemachos

11-12-2008 12:45 PM

As Keefaz says, the quoting is messing you up. It only gets one file because you pop one file off of @ARGV. You then use the glob function on the single file.

You could fix the quoting or as an alternative, just set what you want to glob in the script itself.

Edit - here's an example quickly:

Code:

#!/usr/bin/perl

use strict;

use warnings;



my @files = glob "*";

print "Globbing *: @files\n";



my $file = pop @ARGV;

my @other_files = glob $file;

print "\nGlobbing \@ARGV: @other_files\n";

And here's two runs, once using * (no quotes) and once quoting '*':

Code:

hektor ~/iliumSvn/wordpress $ ./glob *

Globbing *: backup backup.sh blogFixer changePerms disk_usage doc_check themeEditor



Globbing @ARGV: themeEditor

hektor ~/iliumSvn/wordpress $ ./glob '*'

Globbing *: backup backup.sh blogFixer changePerms disk_usage doc_check themeEditor



Globbing @ARGV: backup backup.sh blogFixer changePerms disk_usage doc_check themeEditor

I'm also not sure why you call the Perl script from the shell script. That seems to add unnecessary complications, but there may be a larger reason.

johngreg

11-12-2008 02:12 PM

Quote:

Originally Posted by Telemachos (Post 3339711)

I'm also not sure why you call the Perl script from the shell script. That seems to add unnecessary complications, but there may be a larger reason.

am invoking perl from shell because i need to cron it. the suggestions worked when i call the perl script directly, but calling through shell is still not working
this is working

Code:

./myscript.pl '*.log'

this is not working

Code:

./myscript.sh '*.log'

please help.

keefaz

11-12-2008 02:22 PM

Where did you see that cron can not execute a perl script?
I think cron can execute any file that is executable

johngreg

11-12-2008 02:26 PM

Quote:

Originally Posted by keefaz (Post 3339799)

Where did you see that cron can not execute a perl script?
I think cron can execute any file that is executable

okay, does it work in all AIX boxes? just to confirm, as i am not sure.

Sergei Steshenko

11-12-2008 02:28 PM

Quote:

Originally Posted by johngreg (Post 3339671)

Code:

#!/bin/sh

perl globreport.pl $@

perl script is as below
-------------------------

Code:

#!/usr/bin/perl -w



use strict;

use warnings;

use Data::Dumper;

use Getopt::Long;

use File::Spec;



my $file = pop @ARGV;

my @files = glob($file);

showUsage() unless $file;



foreach my $inputfile (@files)  {

    if ( -r $inputfile ) {

        print "Processing file :: $inputfile\n";

        #all logic to generate report

    }else{

        print "couldnot do anything on :: $inputfile";

    }

}

the problem is, when i run

Code:

./globtry.sh *.log

it is picking up only one file and exiting application.
Please help.

Why do you need

Code:

my @files = glob($file);

in the first place ?

johngreg

11-12-2008 02:35 PM

Quote:

Originally Posted by Sergei Steshenko (Post 3339807)

Why do you need

Code:

my @files = glob($file);

in the first place ?

Else, how does all the file names get resolved?

keefaz

11-12-2008 02:35 PM

Quote:

Originally Posted by johngreg (Post 3339805)

okay, does it work in all AIX boxes? just to confirm, as i am not sure.

I don't know for sure, I use cron on Linux system
Maybe post this question in AIX section of this forum ?

Sergei Steshenko

11-12-2008 02:44 PM

Quote:

Originally Posted by johngreg (Post 3339811)

Else, how does all the file names get resolved?

You didn't answer my question.

Do you need to pick all the files in a directory ?

If yes, have you considered what

Code:

echo *

prints on command line ?

Also, have you considered what will @ARGV in a Perl script contain when the script is called as

Code:

..../script *

?

Also, have you heard of 'opendiir', 'readdir' functions in Perl ?

johngreg

11-12-2008 02:52 PM

Quote:

Originally Posted by Sergei Steshenko (Post 3339817)

You didn't answer my question.

Do you need to pick all the files in a directory ?

If yes, have you considered what

Code:

echo *

prints on command line ?

Also, have you considered what will @ARGV in a Perl script contain when the script is called as

Code:

..../script *

?

Also, have you heard of 'opendiir', 'readdir' functions in Perl ?

okay, if the user inputs *.log as argument, i want to retrieve all files with .log extension. I am very new perl and i have not used opendir and readir functions till now.

Telemachos

11-12-2008 02:57 PM

If you will want to get a list of all the files ending in .log every time you run the script, then you should just put that directly into the script (rather than adding it on the command line). In that case, you don't need to pop the item from @ARGV and then glob that. Just hard-code the information into the script.

On the other hand, if you will sometimes invoke it to parse all .log files and other times to invoke .pl files and other times etc., then I would call the Perl script from cron. I don't know AIX, but in general, you need to call a script in cron with its full path name, e.g. /path/to/my/script.pl - the regular ./script.pl won't work, since cron has a very limited environment (and it isn't run from the directory where your script lives).

Finally, as Sergei suggests, you might want to consider opendir and readdir instead of glob. The modern Perl version of glob is supposed to be much better than its earlier versions (which were buggy), but last I heard it invokes stat on each file and is often not necessary. (That said, opendir and readdir require a bit more manual work than glob. Items are not sorted and everything - including . and .. - gets listed.)

Sergei Steshenko

11-12-2008 03:05 PM

Quote:

Originally Posted by johngreg (Post 3339826)

okay, if the user inputs *.log as argument, i want to retrieve all files with .log extension. I am very new perl and i have not used opendir and readir functions till now.

Again, first try to do this:

Code:

echo *.log

, then try the following script:

Code:

#!/usr/bin/perl -w

use strict;

warn "contents of \@ARGV - one item per line:\n", join("\n", @ARGV);

, i.e., say, call the script 'test.pl', put it into a directory with *.log files and run it as:

Code:

./test.pl *.log

.

Anyway, 'opendir' + 'readdir' solution would be much cleaner.

Telemachos

11-12-2008 03:10 PM

A quick example of what opendir + readdir looks like versus glob

Code:

#!/usr/bin/perl

use strict;

use warnings;



opendir my $file_handle, '.'

  or die "Can't open current working directory";



my @open_files = readdir $file_handle;



print "Using opendir and readdir:\n";

foreach my $file (@open_files) {

  print "\t$file\n";

}



my @glob_files = glob '*';



print "Using glob:\n";

foreach my $file (@glob_files) {

  print "\t$file\n";

}

Save that and run it in any directory with various files in it (hidden and non-hidden) to get an idea of the differences.

Sergei Steshenko

11-12-2008 03:25 PM

Quote:

Originally Posted by Telemachos (Post 3339844)

A quick example of what opendir + readdir looks like versus glob

Code:

#!/usr/bin/perl

use strict;

use warnings;



opendir my $file_handle, '.'

  or die "Can't open current working directory";



my @open_files = readdir $file_handle;



print "Using opendir and readdir:\n";

foreach my $file (@open_files) {

  print "\t$file\n";

}



my @glob_files = glob '*';



print "Using glob:\n";

foreach my $file (@glob_files) {

  print "\t$file\n";

}

Save that and run it in any directory with various files in it (hidden and non-hidden) to get an idea of the differences.

You really don't need this:

Code:

my @open_files = readdir $file_handle;

A typical idiom, adjusted for the needed *.log files, would be:

Code:

opendir(my $dh, '.') or die "cannot open '.' directory";



while(defined(my $dir_entry = readdir($dh)))

  {

  next if not -f $dir_entry;

  next if not $dir_entry =~ m/\.log$/;

  # do something with $dir_entry since it's one of *.log files

  }

Telemachos

11-12-2008 03:31 PM

Quote:

Originally Posted by Sergei Steshenko (Post 3339854)

You really don't need this:

Code:

my @open_files = readdir $file_handle;

Fair enough, but I think you missed my point. I wasn't trying to write his program. I was simply trying to show him the difference between an array created by using glob and one created by using opendir & readdir.

johngreg

11-12-2008 03:33 PM

thanks, i will try the methods suggested. one more point to add on regarding usage from shell - this perl script is called along with other scripts from a master shell script which takes backup, sent emails alerts and few other tasks.
and again - thanks Telemachos for suggesting the hardcoding - i dint think in that way at all. as i need to do this only on known set of extensions- i can hardcode and sent some parameter to pick the one i need and there by skipping interpretation by shell.

All times are GMT -5. The time now is 01:55 PM.