LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 04-07-2009, 11:04 PM   #1
roach7711x
LQ Newbie
 
Registered: Oct 2008
Posts: 18

Rep: Reputation: 1
Cron job script creating corrupt archive


Ive created this cron job:
Code:
# m h  dom mon dow   command
0 3 * * * /home/backup1/backupsite.sh # backup whole site
That executes this script daily:
Code:
#!/bin/bash

# backupsite.sh
tar -cjvf /mnt/disk3/site_backups/site_bkup_`date +\%F_\%H\%M`_auto.tar.bz2 /var/www/* --exclude="/var/www/backup"
cp /mnt/disk3/site_backups/site_bkup_`date +\%F_\%H\%M`_auto.tar.bz2 /var/www/backup/db_backups
chmod -R 755 /mnt/disk3/
This creates a corrupted archive everytime. When I run sudo ./backupsite.sh manually it runs just fine. What is the deal here?
 
Old 04-08-2009, 03:26 AM   #2
rylan76
Senior Member
 
Registered: Apr 2004
Location: Potchefstroom, South Africa
Distribution: Fedora 17 - 3.3.4-5.fc17.x86_64
Posts: 1,477

Rep: Reputation: 87
Hmm...

I do almost exactly what you do, in almost exactly the same way, with no problems.

1. Does your cron run as the user you think it is running as?

I know this sounds counter-intuitive, since it always SHOULD run as root (or not? not sure), but maybe it is not... and this might interfere with creating files in the locations you want, due to permission problems on the target folders?

2. Do you have some kind of processwatch mechanism?

On Hetzner servers I have often encountered this problem with long-running cronjobs, i. e. if processwatch sees on a Hetzner server that a cron-spawned process is taking "too long" it sometimes kills it. If the cron-spawned process was in the middle of tarring or compressing, it resulted in a corrupt, incomplete file.

3. Any hints in the cron log?

Can you spot anything in cron's log file?

4. Paths

Are you sure when you run the cronjob, the paths are in the same location? This stumped me for a few hours when I was setting up my similar backup strategy - when I ran the script manually as a test, I was in a different location (a limited system user) than when the script would run under cron (as root, or spawned from a different location in the filesystem). I needed to change the paths in the final, "real cron" version of the script to be absolute, so they worked from THERE, not when called inside my home directory (where they could remain relative).

Here's what I do:

Each day I dump the entire MySQL database on a domain on a Hetzner server, and tarbzip2 some files on the domain. Then I transmit all this with a PHP script that connects via FTP to a backup server elsewhere.

Here's the script that backs up my files - this is called from my crontab:

Code:
gen_backups.sh:
------------------------
mysqldump --set-variable=max_allowed_packet=4M --add-drop-table -uusername -ppassword -hserver.com dbname_db1 > fwf_db_backup.sql
bzip2 /usr/home/fwfunn/fwf_db_backup.sql
mv /usr/home/fwfunn/fwf_db_backup.sql.bz2 public_html/backup_export/.
tar cvf fwf_files_backup_1.tar /usr/home/fwfunn/public_html/downloads/dynamic/*
bzip2 /usr/home/fwfunn/fwf_files_backup_1.tar
mv /usr/home/fwfunn/fwf_files_backup_1.tar.bz2 public_html/backup_export/.
tar cvf fwf_files_backup_2.tar /usr/home/fwfunn/public_html/images/dynamic/*
bzip2 /usr/home/fwfunn/fwf_files_backup_2.tar
mv /usr/home/fwfunn/fwf_files_backup_2.tar.bz2 public_html/backup_export/.
/home/httpd/cgi-bin/php5 /usr/home/fwfunn/public_html/send_backups.php
Here's my crontab:
Code:
MAILTO=""
2 0 * * * /usr/home/fwfunn/public_html/gen_backups.sh >/dev/null 2>&1
3 0 * * * /home/httpd/cgi-bin/php5 /usr/home/fwfunn/public_html/daily_cron.php >/dev/null 2>&1
4 0 1 * * /home/httpd/cgi-bin/php5 /usr/home/fwfunn/public_html/monthly_cron.php >/dev/null 2>&1
05 00 * * * /home/http/cgi-bin/php5 /usr/home/fwfunn/public_html/fix_funeral_dep_group_type_mismatches.php >/dev/null 2>&1
Here's the code of send_backups.php that transmits the backed up files via FTP to the remote backup server:
Code:
<?php
 //Uses FTP to send backups from primaryserver.com to backupserver.com

include 'cms/class.phpmailer.php';

$ftp_server = "backupserver.com";

if (($conn_id = ftp_connect($ftp_server)))
  {
    if (($login_result = ftp_login($conn_id,"ftp_username","ftp_pass")))
      {
	$file = "backup_export/fwf_db_backup.sql.bz2";
	$file_2 = "backup_export/fwf_files_backup_1.tar.bz2";
	$file_3 = "backup_export/fwf_files_backup_2.tar.bz2";

	if (ftp_put($conn_id,"public_html/auto_backups/" . basename($file),$file,FTP_BINARY))
	  {
	    if (ftp_put($conn_id,"public_html/auto_backups/" . basename($file_2),$file_2,FTP_BINARY))
	      {
		if (ftp_put($conn_id,"public_html/auto_backups/" . basename($file_3),$file_3,FTP_BINARY))
		  {
		    ftp_close($conn_id);

		    $mail = new PHPMailer();

		    $mail->From = "";

		    $mail->FromName = "FWF Backup Script";

		    $mail->AddAddress("someone@somewhere.com");

		    $mail->Subject = "FWF Backup Script Success";

		    $msg = "primaryserver.com/backup_export/fwf_db_backup.sql.bz2 transferred succesfully to backupserver.com/auto_backups/fwf_db_backup.sql.bz2.\n\n";

		    $msg .= "primaryserver.com/backup_export/fwf_files_backup_1.sql.bz2 transferred succesfully to backupserver.com/auto_backups/fwf_files_backup_1.sql.bz2.\n\n";

		    $msg .= "primaryserver.com/backup_export/fwf_files_backup_2.sql.bz2 transferred succesfully to backupserver.com/auto_backups/fwf_files_backup_2.sql.bz2.\n\n";

		    $mail->Body = $msg;

		    if (!$mail->Send())
		      {
			echo "Backup script success email message cannot be sent.";
		      }
		  }

		else

		  {
		    $mail = new PHPMailer();

		    $mail->From = "";

		    $mail->FromName = "FWF Backup Script";

		    $mail->AddAddress("someone@somewhere.com");

		    $mail->Subject = "FWF Backup Script Failure - File 2";

		    $mail->Body = "Failed during FWF file 2 transfer to backupserver.com, after successful connect and login!";

		    if (!$mail->Send())
		      {
			echo "Backup script FWF file transfer failure email message cannot be sent.";
		      }
		  }
	      }

	    else

	      {
		$mail = new PHPMailer();

		$mail->From = "";

		$mail->FromName = "FWF Backup Script";

		$mail->AddAddress("someone@somewhere.com");

		$mail->Subject = "FWF Backup Script Failure - Files 1";

		$mail->Body = "Failed during FWF file 1 transfer to backupserver.com, after successful connect and login!";

		if (!$mail->Send())
		  {
		    echo "Backup script FWF file transfer failure email message cannot be sent.";
		  }
	      }
	  }

	else

	  {
	    $mail = new PHPMailer();

	    $mail->From = "";

	    $mail->FromName = "FWF Backup Script";

	    $mail->AddAddress("someone@somewhere.com");

	    $mail->Subject = "FWF Backup Script Failure - DB";

	    $mail->Body = "Failed during DB transfer to backupserver.com, after successful connect and login!";

	    if (!$mail->Send())
	      {
		echo "Backup script DB transfer failure email message cannot be sent.";
	      }
	  }
      }

    else

      {
	$mail = new PHPMailer();

	$mail->From = "";

	$mail->FromName = "FWF Backup Script";

	$mail->AddAddress("someone@somewhere.com");

	$mail->Subject = "FWF Backup Script Failure";

	$mail->Body = "Failed during login to backupserver.com, after successful connect!";

	if (!$mail->Send())
	  {
	    echo "Backup script login failure email message cannot be sent.";
	  }
      }
  }

 else

   {
     $mail = new PHPMailer();

     $mail->From = "";

     $mail->FromName = "FWF Backup Script";

     $mail->AddAddress("someone@somewhere.com");

     $mail->Subject = "FWF Backup Script Failure";

     $mail->Body = "Failed during connect to backupserver.com!";

     if (!$mail->Send())
       {
	 echo "Backup script connect failure email message cannot be sent.";
       }
   }
?>
I. e. each morning I just check my mail to see that the backup script ran successfully at 00:02 that morning, and that the files have been FTP'ed through to the backup server.

Hope this helps...

Last edited by rylan76; 04-08-2009 at 05:37 AM.
 
Old 04-08-2009, 05:01 AM   #3
fotoguy
Senior Member
 
Registered: Mar 2003
Location: Brisbane Queensland Australia
Distribution: KirraMail Live Email Server
Posts: 1,280

Rep: Reputation: 61
Maybe the script is corrupt, maybe cut and paste the contents into a new script, and see if the error still persists.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Creating a cron job to email me a txt file/format? NewShockerGuy Linux - General 7 05-16-2008 10:23 AM
adding a perl script to cron.daily / cron.d to setup a cron job CrontabNewBIE Linux - Software 6 01-14-2008 09:16 AM
script does not run in cron job kashyapvirgo Linux - General 8 03-20-2007 11:55 AM
Cron Job with a Script bravored Linux - General 4 08-05-2005 11:27 AM
newbie needing help creating a cron job Lleb_KCir Linux - Software 5 12-13-2004 04:58 PM


All times are GMT -5. The time now is 09:27 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration