LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 03-19-2008, 05:07 PM   #1
justinjoseph24
LQ Newbie
 
Registered: Feb 2006
Posts: 9

Rep: Reputation: 0
Perl disc maintenance script for Windows - works fine could be improved


LQ people,

This is my second perl script. This script archives and rotates log files (assuming you have gzip and perl 5.8 installed) and have a properly formatted argument file. It works, but it could be improved. For example, if it could recursively check sub directories. Some people have said use File::Find, but I'm not sure how to make it recursively search directories for certain file extensions. Any feedback is welcome!

Run it by: "perl discclean.pl dirlist.txt".
Code:
##This program takes a tab-delimited list passed from the command line and attempts to 1) gzip all of the files in the given folder that match the given extension; 
#2) deletes files that exceed the given keep period (days). An example input file should look like this. Note that backslashes must be forward slashes:
#C:/Events	*.evt	.gz	30
#D:/Logs	*.log	.gz	14


use strict;
use Time::Local;
my $thetime = localtime();
my ($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst)=localtime(time);
my $thismonth = $mon+1;
my $thisyear = $year+1900;
my $timestamp = "$thismonth-$mday-$thisyear";
my $timestamp2 = "$thismonth-$mday-$thisyear-$hour-$min-$sec";
my $timestamp3 = "$thismonth-$mday-$thisyear \@ $hour:$min:$sec";
my $newfilename;
my $logoutputfile = "C:/perlHDcleansummary.log";
open FILEHAND, "$ARGV[0]";
#open FILEHAND, "discspace_filelist.txt";
unless (-e "$logoutputfile") {
	open (LOGHAND, ">>$logoutputfile");
} 
my $logfilecontents = "========================================================================\nperlDiscCleanup\t$thetime\n\n";
my @filelist = (<FILEHAND>);
my @rowdata;
my $line; #control var
my $line2; #control var 2
my @files_for_archive;
my @match_gz_files;
my @files_to_delete;
my $arraycount = 0;
my $row = $arraycount+1;
my $unit;
my $loopcount;
my $globarg;
my $globarg2;
foreach $line (@filelist) {
@rowdata = split /\t/,$filelist[$arraycount];

use File::Glob;
$globarg = "$rowdata[0]$rowdata[1]";
my @match_files = glob "$globarg";
if ($rowdata[2] =~ /gz/i) {
1;
}
else {
	die "This program only utilizes GZIP archival at this time.";
	}

foreach (@match_files) {         #these files match  the glob
	if (-M > 9999) {
	print "The file $_ is ancient.\n";
	}
	#orginally this was to archive only, now to delete. archive arbitrary at files older than 1 days. Delete if older than file argument
	if (-M > 1) {  #just about every globbed file will get added except current files.
		#print "The file $_ meets the criteria for archival.\n";
		
			push @files_for_archive, $_;      # do it and warn to log if there's a problem.
			}
	}
foreach (@files_for_archive) {  #the archive process
my $filescount = @files_for_archive;

	system("gzip -f \"$_\"");
	$loopcount++;
	$newfilename = $_;
	my $currentfile = $newfilename;
	$currentfile .= ".gz";
	$newfilename .= ".$timestamp.gz";
	select (LOGHAND);
	rename $currentfile, $newfilename or print "ERROR: $currentfile $!\n";
	$logfilecontents .= "Processed: $newfilename\n";

}

$globarg2 = "$rowdata[0]*$rowdata[2]";
my $part1 = "$rowdata[0]";
my $part2 = "$rowdata[2]";
  @match_gz_files = glob "$globarg2";   #glob the .gz files
foreach (@match_gz_files) {         #these files match  the glob
	if (-M > 9999) {
	print "The file $_ is ancient.\n";
	}
	if (-M > $rowdata[3]) {        #Here is where we delete if the file is older than the lenght of time the file should be kept.

			push @files_to_delete, $_      #exclude files that aren't read/write
				if (-r) && (-w);
			}
	}
foreach (@files_to_delete) {  #the delete process
	my $filedelete = $_;
	unlink $filedelete or warn "failed to delete $filedelete : $!\n";

	$logfilecontents .= "Deleted: $filedelete\n";

	}

@match_files = ( );
@files_for_archive = ( );
@match_gz_files = ( );
@files_to_delete = ( );
$globarg = ( );
$globarg2 = ( ) ;

$arraycount++;
}

print LOGHAND "$logfilecontents";
close (LOGHAND);
my $newlogfile = "$logoutputfile.$timestamp2.txt";
rename $logoutputfile, $newlogfile;
 
Old 03-19-2008, 07:36 PM   #2
Telemachos
Member
 
Registered: May 2007
Distribution: Debian
Posts: 754

Rep: Reputation: 60
Quote:
Originally Posted by justinjoseph24 View Post
[I]t could be improved. For example, if it could recursively check sub directories. Some people have said use File::Find, but I'm not sure how to make it recursively search directories for certain file extensions.
There are probably lots of good ways to do something like that, but I like File::Find:Rule, which you can get from CPAN - http://cpan.uwinnipeg.ca/module/File::Find::Rule
Here's how easy it is to find anything with a .c extension in any subdirectory starting with my home directory, and put it into an array (for later processing):
Code:
#!/usr/bin/perl
use strict;
use warnings;
use File::Find::Rule;

my @c_files = File::Find::Rule
    ->file
    ->name('*.c')
    ->in("/home/telemachus");
Edit - please format code you put up here a bit better. It's a pain to read if it's longer left to right than it is up and down.

Last edited by Telemachos; 03-19-2008 at 07:53 PM.
 
Old 03-19-2008, 10:44 PM   #3
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,359

Rep: Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751
As per telemachos, always
use warnings;
Or
#!/usr/bin/perl -w

always check file open succeeds eg
open (FILEHAND, "<", "$ARGV[0]") or die "Unable to open $ARGV[0] : $!\n";
always close all your files (and check close happens ok)

when clearing an array, you don't need a space between brackets ie
Code:
@arr = ( );
is the same as
@arr = ();
Also, your probably want to remove line endings from your input file
Code:
my @filelist = (<FILEHAND>);
chomp(@filelist);

Last edited by chrism01; 03-24-2008 at 08:14 PM.
 
Old 03-19-2008, 11:07 PM   #4
justinjoseph24
LQ Newbie
 
Registered: Feb 2006
Posts: 9

Original Poster
Rep: Reputation: 0
Wow, very good feed back; I'm going to make use of all these suggestions right now. : ) Thanks guys.
 
Old 03-20-2008, 07:07 PM   #5
justinjoseph24
LQ Newbie
 
Registered: Feb 2006
Posts: 9

Original Poster
Rep: Reputation: 0
Has anyone been able to get File::Find::Rule installed properly on Windows? I'm using Activeperl 5.8.7 b815 and when I use the PPM to install it 'install File-Find-Rule', it installs for some time and fails on:

Code:
...Installing C:\Perl\site\lib\Module\Build\Platform\VMS.pm
Installing C:\Perl\site\lib\Module\Build\Platform\VOS.pm
Installing C:\Perl\site\lib\Module\Build\Platform\Windows.pm
Installing C:\Perl\bin\config_data
Installing C:\Perl\bin\config_data.bat
Successfully installed Module-Build version 0.2808 in ActivePerl 5.8.7.815.
Error: Package 'Scalar-List-Utils' not found. Please 'search' for it
first.
I read something on the net about earlier version of perl not being able to update core files... Ugh I'm probably going to have to upgrade to a newer perl version.
 
Old 03-20-2008, 07:26 PM   #6
Telemachos
Member
 
Registered: May 2007
Distribution: Debian
Posts: 754

Rep: Reputation: 60
I've only played with it a little, but 5.10.0 seems to have some nice new tricks. That said, ActiveState also still has 5.8.8 available; there's a lot more 5.8.8 code already out there to poach - I mean learn from, to learn from.
 
Old 03-22-2008, 09:32 PM   #7
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,359

Rep: Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751
You need to install Scalar-List-Utils pkg as it says.
 
Old 03-22-2008, 10:01 PM   #8
osor
HCL Maintainer
 
Registered: Jan 2006
Distribution: (H)LFS, Gentoo
Posts: 2,450

Rep: Reputation: 78
The rest of the script seems quite schizophrenic (but that’s ok since it’s only your second script). Here are a few comments:
  • Read “perldoc perlstyle” and use consistent indentation. Good indentation improves readability and will help us help you diagnose problems if they appear.
  • You don’t need Time::Local for localtime().
  • You can discard array elements from a function call you don’t need and modify values directly. For example, in the beginning you might do:
    Code:
    my ($sec, $min, $hour, $mday, $mon, $year) = localtime();
    $mon++;
    $year += 1900;
    my $timestamp = "$mon-$mday-$year";
    my $timestamp2 = "$mon-$mday-$year-$hour-$min-$sec";
    my $timestamp3 = "$mon-$mday-$year \@ $hour:$min:$sec";
  • You should check that an open operation succeeds. Also, it seems that you forgo opening at all if the logfile exists. But this will cause an error later on in the program.
  • You can declare lexical variables inside loops and in the loop condition. E.g.,
    Code:
    foreach my $line (@filelist) {
    	my @files_for_archive;
    	…
    If you put all those declarations inside the loop, you won’t have to clear the contents at the end of the loop body.
  • Instead of using code like this:
    Code:
    for (@array1) { push (@array2, $_) if(condition) }
    you can do the following:
    Code:
    @array2 = grep {condition} @array1;
    Keeping in mind that the condition block may have multiple statements, and only the last will affect whether @array2 gets the element. As a side effect, @array2’s elements are aliases of the corresponding @array1 elements (but in your case it doesn’t matter because you don’t try to modify them any way).
  • You don’t need loop counters (i.e., $arraycount) at all if you use foreach style loops.
 
Old 03-24-2008, 10:14 PM   #9
justinjoseph24
LQ Newbie
 
Registered: Feb 2006
Posts: 9

Original Poster
Rep: Reputation: 0
Thanks for the tips Osor! btw: I added recursive directory processing to the functionality if anyone is interested.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Converting a Windows Perl script to a Linux Perl script. rubbercash Programming 2 07-19-2004 10:22 AM
X croaks at 640x480, but Windows works fine. Luke727 Debian 6 05-28-2004 06:19 PM
Perl script works with Apache1 but not Apache2, why? m3kgt Linux - Software 7 03-11-2004 12:36 PM
No internet access in linux, works fine in windows c0uchm0nster Linux - Networking 8 01-29-2004 06:45 AM
printer works fine in windows but is slow under linux nef Linux - Hardware 4 09-28-2003 02:39 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 03:52 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration