LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Server (http://www.linuxquestions.org/questions/linux-server-73/)
-   -   Cron job script creating corrupt archive (http://www.linuxquestions.org/questions/linux-server-73/cron-job-script-creating-corrupt-archive-717584/)

roach7711x 04-07-2009 10:04 PM

Cron job script creating corrupt archive
 
Ive created this cron job:
Code:

# m h  dom mon dow  command
0 3 * * * /home/backup1/backupsite.sh # backup whole site

That executes this script daily:
Code:

#!/bin/bash

# backupsite.sh
tar -cjvf /mnt/disk3/site_backups/site_bkup_`date +\%F_\%H\%M`_auto.tar.bz2 /var/www/* --exclude="/var/www/backup"
cp /mnt/disk3/site_backups/site_bkup_`date +\%F_\%H\%M`_auto.tar.bz2 /var/www/backup/db_backups
chmod -R 755 /mnt/disk3/

This creates a corrupted archive everytime. When I run sudo ./backupsite.sh manually it runs just fine. What is the deal here?

rylan76 04-08-2009 02:26 AM

Hmm...

I do almost exactly what you do, in almost exactly the same way, with no problems.

1. Does your cron run as the user you think it is running as?

I know this sounds counter-intuitive, since it always SHOULD run as root (or not? not sure), but maybe it is not... and this might interfere with creating files in the locations you want, due to permission problems on the target folders?

2. Do you have some kind of processwatch mechanism?

On Hetzner servers I have often encountered this problem with long-running cronjobs, i. e. if processwatch sees on a Hetzner server that a cron-spawned process is taking "too long" it sometimes kills it. If the cron-spawned process was in the middle of tarring or compressing, it resulted in a corrupt, incomplete file.

3. Any hints in the cron log?

Can you spot anything in cron's log file?

4. Paths

Are you sure when you run the cronjob, the paths are in the same location? This stumped me for a few hours when I was setting up my similar backup strategy - when I ran the script manually as a test, I was in a different location (a limited system user) than when the script would run under cron (as root, or spawned from a different location in the filesystem). I needed to change the paths in the final, "real cron" version of the script to be absolute, so they worked from THERE, not when called inside my home directory (where they could remain relative).

Here's what I do:

Each day I dump the entire MySQL database on a domain on a Hetzner server, and tarbzip2 some files on the domain. Then I transmit all this with a PHP script that connects via FTP to a backup server elsewhere.

Here's the script that backs up my files - this is called from my crontab:

Code:

gen_backups.sh:
------------------------
mysqldump --set-variable=max_allowed_packet=4M --add-drop-table -uusername -ppassword -hserver.com dbname_db1 > fwf_db_backup.sql
bzip2 /usr/home/fwfunn/fwf_db_backup.sql
mv /usr/home/fwfunn/fwf_db_backup.sql.bz2 public_html/backup_export/.
tar cvf fwf_files_backup_1.tar /usr/home/fwfunn/public_html/downloads/dynamic/*
bzip2 /usr/home/fwfunn/fwf_files_backup_1.tar
mv /usr/home/fwfunn/fwf_files_backup_1.tar.bz2 public_html/backup_export/.
tar cvf fwf_files_backup_2.tar /usr/home/fwfunn/public_html/images/dynamic/*
bzip2 /usr/home/fwfunn/fwf_files_backup_2.tar
mv /usr/home/fwfunn/fwf_files_backup_2.tar.bz2 public_html/backup_export/.
/home/httpd/cgi-bin/php5 /usr/home/fwfunn/public_html/send_backups.php

Here's my crontab:
Code:

MAILTO=""
2 0 * * * /usr/home/fwfunn/public_html/gen_backups.sh >/dev/null 2>&1
3 0 * * * /home/httpd/cgi-bin/php5 /usr/home/fwfunn/public_html/daily_cron.php >/dev/null 2>&1
4 0 1 * * /home/httpd/cgi-bin/php5 /usr/home/fwfunn/public_html/monthly_cron.php >/dev/null 2>&1
05 00 * * * /home/http/cgi-bin/php5 /usr/home/fwfunn/public_html/fix_funeral_dep_group_type_mismatches.php >/dev/null 2>&1

Here's the code of send_backups.php that transmits the backed up files via FTP to the remote backup server:
Code:

<?php
 //Uses FTP to send backups from primaryserver.com to backupserver.com

include 'cms/class.phpmailer.php';

$ftp_server = "backupserver.com";

if (($conn_id = ftp_connect($ftp_server)))
  {
    if (($login_result = ftp_login($conn_id,"ftp_username","ftp_pass")))
      {
        $file = "backup_export/fwf_db_backup.sql.bz2";
        $file_2 = "backup_export/fwf_files_backup_1.tar.bz2";
        $file_3 = "backup_export/fwf_files_backup_2.tar.bz2";

        if (ftp_put($conn_id,"public_html/auto_backups/" . basename($file),$file,FTP_BINARY))
          {
            if (ftp_put($conn_id,"public_html/auto_backups/" . basename($file_2),$file_2,FTP_BINARY))
              {
                if (ftp_put($conn_id,"public_html/auto_backups/" . basename($file_3),$file_3,FTP_BINARY))
                  {
                    ftp_close($conn_id);

                    $mail = new PHPMailer();

                    $mail->From = "";

                    $mail->FromName = "FWF Backup Script";

                    $mail->AddAddress("someone@somewhere.com");

                    $mail->Subject = "FWF Backup Script Success";

                    $msg = "primaryserver.com/backup_export/fwf_db_backup.sql.bz2 transferred succesfully to backupserver.com/auto_backups/fwf_db_backup.sql.bz2.\n\n";

                    $msg .= "primaryserver.com/backup_export/fwf_files_backup_1.sql.bz2 transferred succesfully to backupserver.com/auto_backups/fwf_files_backup_1.sql.bz2.\n\n";

                    $msg .= "primaryserver.com/backup_export/fwf_files_backup_2.sql.bz2 transferred succesfully to backupserver.com/auto_backups/fwf_files_backup_2.sql.bz2.\n\n";

                    $mail->Body = $msg;

                    if (!$mail->Send())
                      {
                        echo "Backup script success email message cannot be sent.";
                      }
                  }

                else

                  {
                    $mail = new PHPMailer();

                    $mail->From = "";

                    $mail->FromName = "FWF Backup Script";

                    $mail->AddAddress("someone@somewhere.com");

                    $mail->Subject = "FWF Backup Script Failure - File 2";

                    $mail->Body = "Failed during FWF file 2 transfer to backupserver.com, after successful connect and login!";

                    if (!$mail->Send())
                      {
                        echo "Backup script FWF file transfer failure email message cannot be sent.";
                      }
                  }
              }

            else

              {
                $mail = new PHPMailer();

                $mail->From = "";

                $mail->FromName = "FWF Backup Script";

                $mail->AddAddress("someone@somewhere.com");

                $mail->Subject = "FWF Backup Script Failure - Files 1";

                $mail->Body = "Failed during FWF file 1 transfer to backupserver.com, after successful connect and login!";

                if (!$mail->Send())
                  {
                    echo "Backup script FWF file transfer failure email message cannot be sent.";
                  }
              }
          }

        else

          {
            $mail = new PHPMailer();

            $mail->From = "";

            $mail->FromName = "FWF Backup Script";

            $mail->AddAddress("someone@somewhere.com");

            $mail->Subject = "FWF Backup Script Failure - DB";

            $mail->Body = "Failed during DB transfer to backupserver.com, after successful connect and login!";

            if (!$mail->Send())
              {
                echo "Backup script DB transfer failure email message cannot be sent.";
              }
          }
      }

    else

      {
        $mail = new PHPMailer();

        $mail->From = "";

        $mail->FromName = "FWF Backup Script";

        $mail->AddAddress("someone@somewhere.com");

        $mail->Subject = "FWF Backup Script Failure";

        $mail->Body = "Failed during login to backupserver.com, after successful connect!";

        if (!$mail->Send())
          {
            echo "Backup script login failure email message cannot be sent.";
          }
      }
  }

 else

  {
    $mail = new PHPMailer();

    $mail->From = "";

    $mail->FromName = "FWF Backup Script";

    $mail->AddAddress("someone@somewhere.com");

    $mail->Subject = "FWF Backup Script Failure";

    $mail->Body = "Failed during connect to backupserver.com!";

    if (!$mail->Send())
      {
        echo "Backup script connect failure email message cannot be sent.";
      }
  }
?>

I. e. each morning I just check my mail to see that the backup script ran successfully at 00:02 that morning, and that the files have been FTP'ed through to the backup server.

Hope this helps...

fotoguy 04-08-2009 04:01 AM

Maybe the script is corrupt, maybe cut and paste the contents into a new script, and see if the error still persists.


All times are GMT -5. The time now is 01:44 PM.