LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 11-12-2008, 12:57 PM   #1
johngreg
LQ Newbie
 
Registered: Oct 2008
Posts: 14

Rep: Reputation: 0
help with glob - perl invoked from shell


i am facing difficulty with glob - what i want is - i need to parse all '.log' files and generate a report. I am using perl to do this - this is invoked from a simple shell script which is as follows
globtry.sh:-
Code:
#!/bin/sh
perl globreport.pl $@
perl script is as below
-------------------------
Code:
#!/usr/bin/perl -w

use strict;
use warnings;
use Data::Dumper;
use Getopt::Long;
use File::Spec;

my $file = pop @ARGV;
my @files = glob($file);
showUsage() unless $file;

foreach my $inputfile (@files)  {
    if ( -r $inputfile ) {
        print "Processing file :: $inputfile\n";
        #all logic to generate report
    }else{
        print "couldnot do anything on :: $inputfile";
    }
}
the problem is, when i run
Code:
./globtry.sh *.log
it is picking up only one file and exiting application.
Please help.
 
Old 11-12-2008, 01:05 PM   #2
keefaz
Senior Member
 
Registered: Mar 2004
Distribution: Slackware
Posts: 4,617

Rep: Reputation: 136Reputation: 136
It would be because the shell expands *.log before perl, I mean in this case perl see the files list expanded

Try:
Code:
#!/usr/bin/perl
for my $x(@ARGV) {
   print $x."\n";
}
then execute it, like
Code:
./myscript.pl *.log
and try once again with:
Code:
./myscript.pl '*.log'

Last edited by keefaz; 11-12-2008 at 01:10 PM.
 
Old 11-12-2008, 01:45 PM   #3
Telemachos
Member
 
Registered: May 2007
Distribution: Debian
Posts: 754

Rep: Reputation: 59
As Keefaz says, the quoting is messing you up. It only gets one file because you pop one file off of @ARGV. You then use the glob function on the single file.

You could fix the quoting or as an alternative, just set what you want to glob in the script itself.

Edit - here's an example quickly:
Code:
#!/usr/bin/perl
use strict;
use warnings;

my @files = glob "*";
print "Globbing *: @files\n";

my $file = pop @ARGV;
my @other_files = glob $file;
print "\nGlobbing \@ARGV: @other_files\n";
And here's two runs, once using * (no quotes) and once quoting '*':
Code:
hektor ~/iliumSvn/wordpress $ ./glob *
Globbing *: backup backup.sh blogFixer changePerms disk_usage doc_check themeEditor

Globbing @ARGV: themeEditor
hektor ~/iliumSvn/wordpress $ ./glob '*'
Globbing *: backup backup.sh blogFixer changePerms disk_usage doc_check themeEditor

Globbing @ARGV: backup backup.sh blogFixer changePerms disk_usage doc_check themeEditor
I'm also not sure why you call the Perl script from the shell script. That seems to add unnecessary complications, but there may be a larger reason.

Last edited by Telemachos; 11-12-2008 at 02:09 PM.
 
Old 11-12-2008, 03:12 PM   #4
johngreg
LQ Newbie
 
Registered: Oct 2008
Posts: 14

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Telemachos View Post

I'm also not sure why you call the Perl script from the shell script. That seems to add unnecessary complications, but there may be a larger reason.
am invoking perl from shell because i need to cron it. the suggestions worked when i call the perl script directly, but calling through shell is still not working
this is working
Code:
./myscript.pl '*.log'
this is not working
Code:
./myscript.sh '*.log'
please help.
 
Old 11-12-2008, 03:22 PM   #5
keefaz
Senior Member
 
Registered: Mar 2004
Distribution: Slackware
Posts: 4,617

Rep: Reputation: 136Reputation: 136
Where did you see that cron can not execute a perl script?
I think cron can execute any file that is executable
 
Old 11-12-2008, 03:26 PM   #6
johngreg
LQ Newbie
 
Registered: Oct 2008
Posts: 14

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by keefaz View Post
Where did you see that cron can not execute a perl script?
I think cron can execute any file that is executable
okay, does it work in all AIX boxes? just to confirm, as i am not sure.
 
Old 11-12-2008, 03:28 PM   #7
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by johngreg View Post
i am facing difficulty with glob - what i want is - i need to parse all '.log' files and generate a report. I am using perl to do this - this is invoked from a simple shell script which is as follows
globtry.sh:-
Code:
#!/bin/sh
perl globreport.pl $@
perl script is as below
-------------------------
Code:
#!/usr/bin/perl -w

use strict;
use warnings;
use Data::Dumper;
use Getopt::Long;
use File::Spec;

my $file = pop @ARGV;
my @files = glob($file);
showUsage() unless $file;

foreach my $inputfile (@files)  {
    if ( -r $inputfile ) {
        print "Processing file :: $inputfile\n";
        #all logic to generate report
    }else{
        print "couldnot do anything on :: $inputfile";
    }
}
the problem is, when i run
Code:
./globtry.sh *.log
it is picking up only one file and exiting application.
Please help.
Why do you need

Code:
my @files = glob($file);
in the first place ?
 
Old 11-12-2008, 03:35 PM   #8
johngreg
LQ Newbie
 
Registered: Oct 2008
Posts: 14

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Sergei Steshenko View Post
Why do you need

Code:
my @files = glob($file);
in the first place ?
Else, how does all the file names get resolved?
 
Old 11-12-2008, 03:35 PM   #9
keefaz
Senior Member
 
Registered: Mar 2004
Distribution: Slackware
Posts: 4,617

Rep: Reputation: 136Reputation: 136
Quote:
Originally Posted by johngreg View Post
okay, does it work in all AIX boxes? just to confirm, as i am not sure.
I don't know for sure, I use cron on Linux system
Maybe post this question in AIX section of this forum ?
 
Old 11-12-2008, 03:44 PM   #10
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by johngreg View Post
Else, how does all the file names get resolved?
You didn't answer my question.

Do you need to pick all the files in a directory ?

If yes, have you considered what

Code:
echo *
prints on command line ?

Also, have you considered what will @ARGV in a Perl script contain when the script is called as

Code:
..../script *
?

Also, have you heard of 'opendiir', 'readdir' functions in Perl ?
 
Old 11-12-2008, 03:52 PM   #11
johngreg
LQ Newbie
 
Registered: Oct 2008
Posts: 14

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Sergei Steshenko View Post
You didn't answer my question.

Do you need to pick all the files in a directory ?

If yes, have you considered what

Code:
echo *
prints on command line ?

Also, have you considered what will @ARGV in a Perl script contain when the script is called as

Code:
..../script *
?

Also, have you heard of 'opendiir', 'readdir' functions in Perl ?
okay, if the user inputs *.log as argument, i want to retrieve all files with .log extension. I am very new perl and i have not used opendir and readir functions till now.
 
Old 11-12-2008, 03:57 PM   #12
Telemachos
Member
 
Registered: May 2007
Distribution: Debian
Posts: 754

Rep: Reputation: 59
If you will want to get a list of all the files ending in .log every time you run the script, then you should just put that directly into the script (rather than adding it on the command line). In that case, you don't need to pop the item from @ARGV and then glob that. Just hard-code the information into the script.

On the other hand, if you will sometimes invoke it to parse all .log files and other times to invoke .pl files and other times etc., then I would call the Perl script from cron. I don't know AIX, but in general, you need to call a script in cron with its full path name, e.g. /path/to/my/script.pl - the regular ./script.pl won't work, since cron has a very limited environment (and it isn't run from the directory where your script lives).

Finally, as Sergei suggests, you might want to consider opendir and readdir instead of glob. The modern Perl version of glob is supposed to be much better than its earlier versions (which were buggy), but last I heard it invokes stat on each file and is often not necessary. (That said, opendir and readdir require a bit more manual work than glob. Items are not sorted and everything - including . and .. - gets listed.)

Last edited by Telemachos; 11-12-2008 at 03:59 PM.
 
Old 11-12-2008, 04:05 PM   #13
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by johngreg View Post
okay, if the user inputs *.log as argument, i want to retrieve all files with .log extension. I am very new perl and i have not used opendir and readir functions till now.
Again, first try to do this:

Code:
echo *.log
, then try the following script:

Code:
#!/usr/bin/perl -w
use strict;
warn "contents of \@ARGV - one item per line:\n", join("\n", @ARGV);
, i.e., say, call the script 'test.pl', put it into a directory with *.log files and run it as:

Code:
./test.pl *.log
.

Anyway, 'opendir' + 'readdir' solution would be much cleaner.
 
Old 11-12-2008, 04:10 PM   #14
Telemachos
Member
 
Registered: May 2007
Distribution: Debian
Posts: 754

Rep: Reputation: 59
A quick example of what opendir + readdir looks like versus glob
Code:
#!/usr/bin/perl
use strict;
use warnings;

opendir my $file_handle, '.'
  or die "Can't open current working directory";

my @open_files = readdir $file_handle;

print "Using opendir and readdir:\n";
foreach my $file (@open_files) {
  print "\t$file\n";
}

my @glob_files = glob '*';

print "Using glob:\n";
foreach my $file (@glob_files) {
  print "\t$file\n";
}
Save that and run it in any directory with various files in it (hidden and non-hidden) to get an idea of the differences.
 
Old 11-12-2008, 04:25 PM   #15
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by Telemachos View Post
A quick example of what opendir + readdir looks like versus glob
Code:
#!/usr/bin/perl
use strict;
use warnings;

opendir my $file_handle, '.'
  or die "Can't open current working directory";

my @open_files = readdir $file_handle;

print "Using opendir and readdir:\n";
foreach my $file (@open_files) {
  print "\t$file\n";
}

my @glob_files = glob '*';

print "Using glob:\n";
foreach my $file (@glob_files) {
  print "\t$file\n";
}
Save that and run it in any directory with various files in it (hidden and non-hidden) to get an idea of the differences.
You really don't need this:

Code:
my @open_files = readdir $file_handle;
A typical idiom, adjusted for the needed *.log files, would be:

Code:
opendir(my $dh, '.') or die "cannot open '.' directory";

while(defined(my $dir_entry = readdir($dh)))
  {
  next if not -f $dir_entry;
  next if not $dir_entry =~ m/\.log$/;
  # do something with $dir_entry since it's one of *.log files
  }
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
kill the process invoked from a shell script, when the script is killed kskkumar Linux - Software 8 05-23-2007 12:29 PM
[Perl] unlink/ glob and warn noir911 Programming 1 12-11-2006 07:55 AM
[Perl] unlink/ glob and warn noir911 Programming 1 12-11-2006 06:55 AM
How to tell the shell program is invoked from GUI environment? oldbee Programming 9 01-20-2002 10:17 PM


All times are GMT -5. The time now is 12:22 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration