LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 03-10-2005, 03:40 PM   #1
ivanatora
Member
 
Registered: Sep 2003
Location: Bulgaria
Distribution: Ubuntu 9.10, FreeBSD 7.2
Posts: 459

Rep: Reputation: 31
Perl, recursion


I tried to make a file indexer, that simply gets all the files from a directory and its sub-directories and writes that list to a file.
The problem is that in the end all get files are the first-dir files. Here is the code:
Code:
#!/usr/bin/perl -w
sub get_dir {
my @fs;
my @ds;
$dir = shift;
chdir $dir;
$pwd = `pwd`; 
chomp($pwd);
my $files;
$files = `ls -Fm .`;     
$files =~ s/\n/ /g;
$files =~ s/\*//g;
@names = split(/, /,$files);
foreach (@names){
        if ($_ !~ /\//){
        push (@fs, $_);
        }
        else { 
        s/\///;
        push (@ds, $_);
        }
}
#print "PWD  \t$pwd\n"; 
#print "FILES\t@fs\n\n";
my $n;
open (OUT,">> $out") or die "$!";
foreach (@fs){
        print OUT "$pwd/","$_\n";
        }   
close (OUT);
#print "DIRS\t@ds\n\n";
print "*"x80,"\n";
my $i = 0;
foreach (@ds){
        &get_dir($_);
        }
} #end of sub
$out = "/home/ivanatora/perl_razni/index.fls";
unlink ($out);
&get_dir("/home/ivanatora/shits/mp3/Morgul/");
And here is how it works:
I'm getting the dir name and then via 'ls' - all files and directories within. I put them in two different arrays - @fs for files and @ds for dirs. Then I do recursion - I call the same get_dir subroutinge for every dir in @dirs. I'm not sure of how I define some of the variables.. should they be 'my' or 'local' or nothing at all Here is the output (if all these prints would be uncommented):
Quote:
(23:30:45)[ivanatora@~/perl_razni]$ perl getdir.sub
PWD /home/ivanatora/shits/mp3/Morgul
FILES thisisafile

DIRS 1998 - Parody of the Mass 2000 - The Horror Grandeur 2001 - Sketch Of The Supposed Murderer

********************************************************************************
PWD /home/ivanatora/shits/mp3/Morgul/1998 - Parody of the Mass
FILES 01 Black Hearts Domain.mp3 02 Healing The Blind.mp3 03 Torn.mp3 04 Ballad Of Revolt.mp3 05 - Adoration of the Profane.mp3 06 Author Of Pain.mp3 07 The End.mp3 album.jpg

DIRS

********************************************************************************
PWD /home/ivanatora/shits/mp3/Morgul/1998 - Parody of the Mass
FILES 01 Black Hearts Domain.mp3 02 Healing The Blind.mp3 03 Torn.mp3 04 Ballad Of Revolt.mp3 05 - Adoration of the Profane.mp3 06 Author Of Pain.mp3 07 The End.mp3 album.jpg

DIRS

********************************************************************************
PWD /home/ivanatora/shits/mp3/Morgul/1998 - Parody of the Mass
FILES 01 Black Hearts Domain.mp3 02 Healing The Blind.mp3 03 Torn.mp3 04 Ballad Of Revolt.mp3 05 - Adoration of the Profane.mp3 06 Author Of Pain.mp3 07 The End.mp3 album.jpg

DIRS

********************************************************************************
You can see all FILES are from the first folder..
 
Old 03-10-2005, 04:07 PM   #2
TheLinuxDuck
Member
 
Registered: Sep 2002
Location: Tulsa, OK
Distribution: Slack, baby!
Posts: 349

Rep: Reputation: 33
I once had a perl guru tell me something when I was first learning perl. He said "When coding in perl, code in perl". That means, why in the world are you using shell commands like this? The perl part of it is easier than you think, and looks much nicer than shell calls. Here is a simple by good recursive dir scanner sub:
Code:
sub recursedir($$)
{
  my($dir)  = shift @_;
  my($list) = shift @_;

  if(opendir(DIR, "$dir")) {
    #  get files, skipping hidden . and ..
    #
    for my $file(grep { !/^\./ } readdir DIR) {
      if(-d "$dir/$file") {
        #  recurse subdirs
        #
        recursedir "$dir/$file", $list;
      }
      elsif(-f "$dir/$file") {
        #  add files
        #
        push @$list, "$dir/$file";
      }
    }
    closedir DIR;
  }
  else {
    warn "Cannot open dir '$dir': $!\n";
  }
}
You would execute it something like this:
Code:
my($dir)      = '.';   #  start in the current dir
my($filelist) = [];   #  a blank array ref

recursedir $dir, $filelist;

#  then print it
#

print $_, "\n" for(@$filelist);
You could also make a few minor modifications and make it dump to a file, by changing:
Code:
push @$list, "$dir/$file"
to:
Code:
 print $list "$dir/$file\n";
and then instead of using an array reference, you would open a file and pass in the filehandle:
Code:
  my($dir)  = '/usr/local/share/vim';

  open OUT,">/tmp/fgfgfgfg" or die "Cannot open file: $!\n";
  recursedir $dir, *OUT;
  close OUT;

  exit;
Lots of wonderful opportunities. (=

I hope that this helps and makes your project easier!

Happy coding!
 
Old 03-10-2005, 04:26 PM   #3
ivanatora
Member
 
Registered: Sep 2003
Location: Bulgaria
Distribution: Ubuntu 9.10, FreeBSD 7.2
Posts: 459

Original Poster
Rep: Reputation: 31
Thanks ;] Ugh.. I'm pretty new to perl.. I'll need a little time to catch all these stuff..
 
Old 03-10-2005, 04:30 PM   #4
TheLinuxDuck
Member
 
Registered: Sep 2002
Location: Tulsa, OK
Distribution: Slack, baby!
Posts: 349

Rep: Reputation: 33
If you have any questions about anything that I did, please don't hesitate to ask me.. I'll explain it as best as I can. I had alot of help when I learned, and if you need help, then I would be happy to offer all I can. (=
 
Old 03-11-2005, 12:09 AM   #5
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,269

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
Of course you could use File::Find.pm, but that would take some of the fun out of it
 
Old 03-11-2005, 06:51 AM   #6
ivanatora
Member
 
Registered: Sep 2003
Location: Bulgaria
Distribution: Ubuntu 9.10, FreeBSD 7.2
Posts: 459

Original Poster
Rep: Reputation: 31
Well
First this thing:
Code:
 if(-d "$dir/$file") {
It remains of BASH 'is directory' checking. I've searched all over the Google again and again, but didn't find such a condition. Is there a place where all such conditional operators are listed? I've found only the regular == eq != ne > gt < lt ... Not a word from these -d -f -T (I found that once )
 
Old 03-11-2005, 08:11 AM   #7
TheLinuxDuck
Member
 
Registered: Sep 2002
Location: Tulsa, OK
Distribution: Slack, baby!
Posts: 349

Rep: Reputation: 33
Quote:
Originally posted by chrism01
Of course you could use File::Find.pm, but that would take some of the fun out of it
File::Find.. ugh.. I think that is probably one of the most poorly written perl functions I've ever seen or dealt with. Writing your own recursion is much cleaner and easier, IMHO.
 
Old 03-11-2005, 08:16 AM   #8
keefaz
Senior Member
 
Registered: Mar 2004
Distribution: Slackware
Posts: 4,564

Rep: Reputation: 124Reputation: 124
Quote:
Is there a place where all such conditional operators are listed?
try the builtin perldoc command line tool, like
perldoc -f -d

(-f : docs for perl functions)
 
Old 03-11-2005, 08:19 AM   #9
TheLinuxDuck
Member
 
Registered: Sep 2002
Location: Tulsa, OK
Distribution: Slack, baby!
Posts: 349

Rep: Reputation: 33
Quote:
Originally posted by ivanatora
Well :)
First this thing:
Code:
 if(-d "$dir/$file") {
It remains of BASH 'is directory' checking. I've searched all over the Google again and again, but didn't find such a condition. Is there a place where all such conditional operators are listed? I've found only the regular == eq != ne > gt < lt ... Not a word from these -d -f -T (I found that once :) )
You're right that it is similar to BASH's file tests.. in fact, perl expands on them twice as much. These types of operators are called "file test operators", just so you'll know what to search for, if you need to.

Here is a link to file test operators for perl. Scroll down the page to 4.5.8.

One of my favorite perl books to own is "Programming Perl 3rd edition", AKA The Camel Book. It's not an instructive book as much as it is a complete reference. Once I was familiar with perl's syntax, I keep this book near me, and use it often. (= Although O'reilly is asking $50 for it, you can find it usually for sale, through bestbookbuys.com, for $20-25, usually used. That is a great place to go to pick up any computer books, IMHO.
 
Old 03-13-2005, 03:45 AM   #10
ivanatora
Member
 
Registered: Sep 2003
Location: Bulgaria
Distribution: Ubuntu 9.10, FreeBSD 7.2
Posts: 459

Original Poster
Rep: Reputation: 31
Thanks for the references, I'll start working right now on that
About the book - it's really hard to find a good perl book in my city.. I have a few books bought long time ago and they are fine, but now I have to travel many miles to get a good book.. and I don't have a CC for online shoping :/

PS. I made it!!! Thank you! Here is the source, what do you think about it? I admit that I used your idea
Code:
#!/usr/bin/perl
sub get_dir {
my ($dir) = shift;

opendir (DIR, $dir) or die "$!";
foreach my $file(readdir DIR){  
if ($file =~ /^[^\.]/){
        if (-d "$dir/$file"){
                &get_dir("$dir$file");
                }
        elsif (-f "$dir/$file"){
                print OUT "$dir$file\n";
                }
        }
}
closedir(DIR);
} #end of sub
 
$out = "/home/ivanatora/perl_razni/index.fls";
unlink($out);
open(OUT,"> $out") or die "$!";
&get_dir("/home/ivanatora/shits/mp3/Morgul/");
The second thing of my stupidity: what does that do:
Code:
push @$list, "$dir/$file"
This is some kind of reference.. a variable that points to a memory where other variable is stored.. that's all I know of it. Why do you use it, instead of just @list? And why is it @$ ? There are so many types of references ..

Last edited by ivanatora; 03-13-2005 at 04:13 AM.
 
Old 03-14-2005, 09:44 AM   #11
TheLinuxDuck
Member
 
Registered: Sep 2002
Location: Tulsa, OK
Distribution: Slack, baby!
Posts: 349

Rep: Reputation: 33
Quote:
Originally posted by ivanatora
Code:
push @$list, "$dir/$file"
This is some kind of reference.. a variable that points to a memory where other
variable is stored.. that's all I know of it. Why do you use it, instead of just
@list? And why is it @$ ? There are so many types of references :( ..
You're exactly right that $list is a reference. Earlier on in my code, you'll
see that I created $list as:
Code:
my($list) = [];
This simply creates an empty array reference. I don't know your programming history,
but just for the sake of explanation, it is much faster to pass an array reference
into a function (or out of for that matter), than it is to pass all the items of an
array. And, since the recursive function in my original code passes the dir listing
into itself, we use the reference to speed it up.

@$ is a perlism that you just get used it. (= It's not visually self-explanatory,
that's for certain. Since $list is an array reference, we have to convert it, for
the sake of the push command, into a regular array. The @$ simply
dereferences the reference.

I think your code looks fine! I have a couple of suggestions for you to consider, too.

1. In reference to the opendir command, consider the final product you're trying to
achieve, in regards to the die command. If for some reason the script
encounters an error trying to open a dir, this will terminate the script. Perhaps this is
what you want. Typically, though, a better way to handle it is to generate a warning,
or simply silently ignore the directory on which the error was triggered (or write the
error to a log file).
2. This one is only my opinion, but I feel strongly enough about it to share it with you.
I feel that it is a bad idea to get into the habit of using & when calling functions. The
reason for this is that & allows the user to ignore any prototyping done with the function
declaration/definition. IMHO prototyping is a good idea, because it not only forces the
coder to pay attention to what needs to be passed into a function, but it also allows
the coder to reference the function w/o the need for parenthesis (mostly). If you're not
familiar with prototyping, it's similar to how C declares/defines functions. Consider this
code:
Code:
#!/usr/bin/perl
use strict;
use warnings;

sub getMyInfo($$$) {
  my($filename) = shift @_;
  my($arrayRef) = shift @_;
  my($regexp)   = shift @_;

  print $filename, scalar(@$arrayRef), $regexp, "\n";
}

#MAIN
{
  #  This produces an error
  #
  getMyInfo "Bacon.txt";
}
The user of this code would be forced to provide three items to this function, otherwise,
the perl interpreter would complain:
Code:
~> ./prototype.pl
Not enough arguments for main::getMyInfo at ./prototype.pl line 17, near ""Bacon.txt";"
Execution of ./prototype.pl aborted due to compilation errors.
If the code calls the function using the & operator, here is the output:
Code:
~> ./prototype.pl
Can't use an undefined value as an ARRAY reference at ./prototype.pl line 10.
The second error will more than likely leave a coder scratching their heads, saying
"This sub looks right, why is it giving me an error?" when the error lies elsewhere.

Anyway, as I said, this is just my opinion, and I'm sure that there are those who would
disagree with me.. (= but, if you learned something, then that is what matters to me.

Hope you had a good weekend!
 
Old 03-14-2005, 01:49 PM   #12
ivanatora
Member
 
Registered: Sep 2003
Location: Bulgaria
Distribution: Ubuntu 9.10, FreeBSD 7.2
Posts: 459

Original Poster
Rep: Reputation: 31
Thanks for the suggestions!
I knew this about prototyping, but I haven't thought it will be so important. But when I saw the different error outputs, and remembering how loudly did I cursed the last time when I had to debug 100 lines of source.. Thanks for that example, too!
I'm gonna meditate on that knowledge, now :]
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
chmod recursion -- files only Risc91 AIX 12 09-18-2012 08:58 AM
Recursion in C hubabuba Programming 12 10-03-2005 07:46 AM
tar: '--no-recursion' option doesn't prevent recursion Earl Parker II Slackware 12 08-17-2004 02:49 AM
Writing bash script with recursion.. ray5_83 Programming 4 08-04-2004 05:44 PM
help with recursion function debdas Programming 4 05-14-2003 03:03 AM


All times are GMT -5. The time now is 09:41 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration