Ubuntu 12.04.1 LTS crashing on a PHP script

Hey LQ,

I have a back up machine in my house. It's a dedicated Ubuntu 12.04.1 LTS machine, it's rarely out of date in terms of software packages for longer than week.

Here is my situation. The machine regularly runs a PHP script from a cron job every day at 11pm. Sometimes this script can take up to 40 minutes to complete. This PHP backup script I've written emails me at the end of the script run so that I know the details of how the back up process went. So when I don't receive this email I know something went wrong. So far I can't seem to get the machine to back up properly for more than 3 - 5 days in a row. What ends up happening is that I lose the ability to SSH or PING the machine. I do not have a monitor connected to it.

I know that I need to learn how to write shell scripts correctly so that I'm not dependent on PHP (but hey, it's what I know as I'm a web developer and this has worked for me before on an older Ubuntu server version, and different hardware).

At any rate, I'd really love to know why the machine becomes unresponsive. Is there any kind of machine log that I could read that would tell me what is causing the machine to not respond at the network level? As far as I can tell the machine is still powered on and running but I just can't connect to it. If it really came down to it I could drag a monitor out from somewhere and plug it in but I still wouldn't know what the issue is.

Any tips or suggestions in the right direction would be great. Thanks for reading.

As you seem to realise, its a question of looking at the logs, particularly (probably!) around the time of the backup process.
Start with the generic /var/log/messages.

Also, you should add some checking and debug logging to your script.
Unfortunately, I don't know PHP, so I can't be specific, but if you were using eg bash, you could check the success/failure of each cmd by checking the completion status stored in the shell var $?.
I'm guessing PHP has a similar ability.
Get it to log this at each stage.

A wild guess(!) says it could be running out of disk space during the backup. A lot of backup type operations use a lot of tmp space in the background (eg gzip etc).

HTH

PS Re bash scripting
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/

and start a bash script with

#!/bin/bash
# debug; tell parser to show everything it does
set -xv

Originally Posted by chrism01 As you seem to realise, its a question of looking at the logs, particularly (probably!) around the time of the backup process. Start with the generic /var/log/messages. Also, you should add some checking and debug logging to your script. Unfortunately, I don't know PHP, so I can't be specific, but if you were using eg bash, you could check the success/failure of each cmd by checking the completion status stored in the shell var $?. I'm guessing PHP has a similar ability. Get it to log this at each stage. A wild guess(!) says it could be running out of disk space during the backup. A lot of backup type operations use a lot of tmp space in the background (eg gzip etc). HTH PS Re bash scripting http://rute.2038bug.com/index.html.gz http://tldp.org/LDP/Bash-Beginners-G...tml/index.html http://www.tldp.org/LDP/abs/html/ and start a bash script with #!/bin/bash # debug; tell parser to show everything it does set -xv

Thanks for the tips! The script builds a huge tarchive (2gb with 10's of thousands of files and directories). The harddrive is only 80gb so I wonder if it's running out of swap space or something? I'll check into those logs and report back.

Here is the ls of my /var/log dir. Where do you suppose I start looking first?

wh33t@wh33tserv:/var/log$ ls
alternatives.log       dmesg.1.gz      mail.err        mysql.log.7.gz
alternatives.log.1     dmesg.2.gz      mail.err.1      news
alternatives.log.2.gz  dmesg.3.gz      mail.err.2.gz   syslog
alternatives.log.3.gz  dmesg.4.gz      mail.err.3.gz   syslog.1
apache2                dpkg.log        mail.err.4.gz   syslog.2.gz
apt                    dpkg.log.1      mail.log        syslog.3.gz
auth.log               dpkg.log.2.gz   mail.log.1      syslog.4.gz
auth.log.1             dpkg.log.3.gz   mail.log.2.gz   syslog.5.gz
auth.log.2.gz          faillog         mail.log.3.gz   syslog.6.gz
auth.log.3.gz          fontconfig.log  mail.log.4.gz   syslog.7.gz
auth.log.4.gz          fsck            mysql           udev
boot                   installer       mysql.err       ufw.log
boot.log               kern.log        mysql.log       unattended-upgrades
btmp                   kern.log.1      mysql.log.1.gz  upstart
btmp.1                 kern.log.2.gz   mysql.log.2.gz  wtmp
ConsoleKit             kern.log.3.gz   mysql.log.3.gz  wtmp.1
dist-upgrade           kern.log.4.gz   mysql.log.4.gz
dmesg                  landscape       mysql.log.5.gz
dmesg.0                lastlog         mysql.log.6.gz