[SOLVED] check all files in directory for last modify date in perl

Regnets1 · 07-17-2013, 09:14 AM

I am writing a script in Perl to check the last modified date of all the files in a specific directory and make backups if they are older than 24 hours.
I have two working stand alone "proof of Concept" scripts written. One will verify the last modified time since epoch and tell me if a specific file needs to be updated. The other will copy all the files in a directory to a backup location and add "bak" to the file name. What I am having trouble figuring out is how to merge both scripts so that the new finished script will check the last modified date of all files in a specific directory and perform the backup if any are older than 24 hours.

Here is the script that checks the last modified time:

Code:

#!/usr/bin/perl
##
use strict;
use warnings;
##
use File::stat;
use Time::localtime;
use Time::Local;
use File::Copy;

##


my $source = "/home/rstenger/backup/working/add_td.sh.bak";
my $today = time;
my $modtime = (stat($source)->mtime);
my $interval = 86400;
my $delta = ($today - $modtime);

print "\n" x3;
print "the time is $today\n";

print "\n" x3;
print "Info for calculations:\n";
print "File was created $modtime seconds since epoch\n";
print "Today is $today seconds from epoch\n";
print "There are $interval seconds in 24 hours\n";
print "The difference between source file creation and now is $delta seconds\n";
print "\n" x3;

if ( $delta >= $interval ) {
                print "the file $source is old and needs to be backed up!\n";
        } else {
                print "Backup files are current, no action taken...\n";
}

The script that copies all the files in a directory:

Code:

#!/usr/bin/perl
#
use strict;
use warnings;
#
use File::Copy;

my $SRC_SCRIPTS = "/home/rstenger/scripts/";
my $SRC_MODULES = "/home/rstenger/scripts/modules/";
my $SRC_DATA = "/home/rstenger/scripts/data/";
my $SRC_CONSTRUCTION = "/home/rstenger/scripts/construction";
my $SRC_PERL = "/home/rstenger/scripts/construction/perl/";
my $TARGET_CONST = "/home/rstenger/backup/sandbox/";
my $TARGET_SCPTS = "/home/rstenger/backup/working";
################

opendir(my $SCRIPTS, $SRC_SCRIPTS ) || die "can't opendir $SRC_SCRIPTS: $!";

my @SCRIPTS = readdir($SCRIPTS);

foreach my $t (@SCRIPTS)
{ 
        if( -f "$SRC_SCRIPTS/$t" ) {
                copy "$SRC_SCRIPTS/$t", "$TARGET_SCPTS/$t.bak" || warn "Copy of $t to $TARGET_SCPTS failed: $!";
                }
}

closedir($SCRIPTS);
#################

opendir(my $MODULES, $SRC_MODULES ) || die "can't opendir $SRC_MODULES: $!";
my @MODULES = readdir($MODULES);
foreach my $t (@MODULES)
{ 
        if( -f "$SRC_MODULES/$t" ) {
                copy "$SRC_MODULES/$t", "$TARGET_SCPTS/$t.bak" || warn "Copy of $t to $TARGET_SCPTS failed: $!";
                }
}

closedir($MODULES);
##################

opendir(my $DATA, $SRC_DATA ) || die "can't opendir $SRC_DATA: $!";

my @DATA = readdir($DATA);

foreach my $t (@DATA)
{ 
        if( -f "$SRC_DATA/$t" ) {
                copy "$SRC_DATA/$t", "$TARGET_SCPTS/$t.bak" || warn "Copy of $t to $TARGET_SCPTS failed: $!";
                }
}

closedir($DATA);
#################

opendir(my $CONST, $SRC_CONSTRUCTION ) || die "can't opendir $SRC_CONSTRUCTION: $!";

my @CONST = readdir($CONST);

foreach my $t (@CONST)
{ 
        if( -f "$SRC_CONSTRUCTION/$t" ) {
                copy "$SRC_CONSTRUCTION/$t", "$TARGET_CONST/$t.bak" || warn "Copy of $t to $TARGET_CONST failed: $!";
                }
}

closedir($CONST);
################

opendir(my $PRL, $SRC_PERL) || die "can't opendir $SRC_PERL: $!";

my @PRL = readdir($PRL);

foreach my $t (@PRL)
{ 
        if( -f "$SRC_PERL/$t" ) {
                copy "$SRC_PERL/$t", "$TARGET_CONST/$t.bak" || warn "Copy of $t to $TARGET_CONST failed: $!";
                }
}

closedir($PRL);

exit;

Thanks, please let me know if more detail is required.

thirdm · 07-17-2013, 11:13 AM

I would make the first script (without all the print lines) into a function taking a single argument and returning a boolean value (i.e. a predicate function). Then I'd filter your lists in each copy loop using the grep function with your function as the thing to eval for each file. Or actually, once you remove the prints your function is only going to be a line or two so you could stick the lines in the grep expression block directly.

Regnets1 · 07-17-2013, 03:28 PM

Thanks for the response.
First let me tell you I am at best a beginner at scripting and even greener at Perl. I just looked up making the function and I think I understand that part of your reply. However the rest of your post is completely over my head (you lost me at "returning a boolean value"). So in an effort to not waste your advice, I made a function out of the copy script and called it if a specific file in the backup directory were out of date. Which I believe will solve my problem, because if one backup file is older than 24 hours, then all will be because this script will execute when I login. So here is my solution.

Code:

#!/usr/bin/perl
##
use strict; 
use warnings;
##
use File::stat;
use Time::localtime;
use Time::Local;
use File::Copy;

##
my $source = "/home/rstenger/backup/working/add_rs.sh.bak";
my $today = time;
my $modtime = (stat($source)->mtime);
my $interval = 86400;
my $delta = ($today - $modtime);
#
my $SRC_SCRIPTS = "/home/rstenger/scripts/";
my $SRC_MODULES = "/home/rstenger/scripts/modules/";
my $SRC_DATA = "/home/rstenger/scripts/data/";
my $SRC_CONSTRUCTION = "/home/rstenger/scripts/construction";
my $SRC_PERL = "/home/rstenger/scripts/construction/perl/";
my $TARGET_CONST = "/home/rstenger/backup/sandbox/";
my $TARGET_SCPTS = "/home/rstenger/backup/working";
###############################

if ( $delta >= $interval ) {
                print "Performing file backups!\n";
                &backup;

}
else {

                print "\n" x4;
                print "Backup files are current, no action taken\n";

} 
sub backup {


opendir(my $SCRIPTS, $SRC_SCRIPTS ) || die "can't opendir $SRC_SCRIPTS: $!";

my @SCRIPTS = readdir($SCRIPTS);

foreach my $t (@SCRIPTS)
{ 
        if( -f "$SRC_SCRIPTS/$t" ) {
                copy "$SRC_SCRIPTS/$t", "$TARGET_SCPTS/$t.bak" || warn "Copy of $t to $TARGET_SCPTS failed: $!";
                }
}

closedir($SCRIPTS);
#################

opendir(my $MODULES, $SRC_MODULES ) || die "can't opendir $SRC_MODULES: $!";
my @MODULES = readdir($MODULES);
foreach my $t (@MODULES)
{ 
        if( -f "$SRC_MODULES/$t" ) {
                copy "$SRC_MODULES/$t", "$TARGET_SCPTS/$t.bak" || warn "Copy of $t to $TARGET_SCPTS failed: $!";
                }
}

closedir($MODULES);
##################

opendir(my $DATA, $SRC_DATA ) || die "can't opendir $SRC_DATA: $!";

my @DATA = readdir($DATA);

foreach my $t (@DATA)
{ 
        if( -f "$SRC_DATA/$t" ) {
                copy "$SRC_DATA/$t", "$TARGET_SCPTS/$t.bak" || warn "Copy of $t to $TARGET_SCPTS failed: $!";
                }
}

closedir($DATA);
#################

opendir(my $CONST, $SRC_CONSTRUCTION ) || die "can't opendir $SRC_CONSTRUCTION: $!";

my @CONST = readdir($CONST);

foreach my $t (@CONST)
{ 
        if( -f "$SRC_CONSTRUCTION/$t" ) {
                copy "$SRC_CONSTRUCTION/$t", "$TARGET_CONST/$t.bak" || warn "Copy of $t to $TARGET_CONST failed: $!";
                }
}

closedir($CONST);
################

opendir(my $PRL, $SRC_PERL) || die "can't opendir $SRC_PERL: $!";

my @PRL = readdir($PRL);

foreach my $t (@PRL)
{ 
        if( -f "$SRC_PERL/$t" ) {
                copy "$SRC_PERL/$t", "$TARGET_CONST/$t.bak" || warn "Copy of $t to $TARGET_CONST failed: $!";
                }
}

closedir($PRL);

}

exit;

I just ran the script and it appears to have worked as expected. I will know run some test, adjusting my interval variable and see what happens.

I would like to know how to check the modified time on all files in the directory for future reference.

Thanks again for the suggestion.
Robert

thirdm · 07-17-2013, 05:16 PM

Quote:

Originally Posted by Regnets1

Thanks for the response.
First let me tell you I am at best a beginner at scripting and even greener at Perl. I just looked up making the function and I think I understand that part of your reply. However the rest of your post is completely over my head (you lost me at "returning a boolean value"). So in an effort to not waste your advice, I made a function out of the copy script and called it if a specific file in the backup directory were out of date. Which I believe will solve my problem, because if one backup file is older than 24 hours, then all will be because this script will execute when I login. So here is my solution.

Boolean means true or false. Perl defines that in a certain way, with true being a number that's not zero (like C) or a string that's not empty (like Perl) or "0" (totally like Perl).

So the predicate function should, when given a filename, say yes or no (true or false, 1 or 0, etc.) on whether it's older than one day.

Code:

# untested
sub not_fresh {
    my $source = shift;
    return time - stat($source)->mtime > 86400;
}

You could use this function in one of your loops like this:

Code:

opendir(my $SCRIPTS, $SRC_SCRIPTS ) || die "can't opendir $SRC_SCRIPTS: $!";

my @SCRIPTS = readdir($SCRIPTS);

foreach my $t (@SCRIPTS) { 
        if( -f "$SRC_SCRIPTS/$t" && not_fresh("$SRC_SCRIPTS/$t") ) {
                copy "$SRC_SCRIPTS/$t", "$TARGET_SCPTS/$t.bak" || warn "Copy of $t to $TARGET_SCPTS failed: $!";
        }
}

closedir($SCRIPTS);

Which might be better than my way, I dunno, but I like to separate filtering from loop body as much as possible. This is where the grep perl function (http://perldoc.perl.org/functions/grep.html) comes in:

Code:

opendir(my $SCRIPTS, $SRC_SCRIPTS ) || die "can't opendir $SRC_SCRIPTS: $!";

# untested!
foreach my $t (grep { my $f = "$SRC_SCRIPTS/$_"; -f $f && time - stat($f)->mtime > 86400 } readdir($SCRIPTS)) { 
   copy "$SRC_SCRIPTS/$t", "$TARGET_SCPTS/$t.bak" || warn "Copy of $t to $TARGET_SCPTS failed: $!";
}

closedir($SCRIPTS);

Regnets1 · 07-22-2013, 07:10 AM

Thirdm,
Sorry I did not respond sooner! Thank you very much for the explanations and the sample code blocks. I am going to test them soon! I am going to mark the thread solved...

Thanks again for the assistance!