[SOLVED] rsync fails in cron

jonathansfl · 12-08-2010, 09:33 AM

Greetings
I'm using Ubuntu 10.04 LTS server and Postgresql 8.4.
I have a .sh script that is run by cron every other hour. That works fine.
The .sh script includes an rsync command that copies a postgresql dump .tar file to a remote archive location via ssh. That fails when run by cron; I think because it is (quietly) asking for the remote user's password (and not getting it). I set up the public/private ssh key arrangement. The script succeeds when run manually as the same user that the cron job uses, and does not ask for the password. I am able to ssh to the remote server from the source server (using the same username) and not get the password prompt (both directions), so why doesn't rsync work? I even put a .pgpass file in the root of that user's directory with that user's password, and the user/password are identical on both servers.

I think the problem is rsync is not able to use the ssh key correctly. I tried adding this to my script but it didn't help.

Code:

# start ssh-agent so don't need to enter passwords manually
if [ -d $HOME/.ssh ]
  then EXEC="exec ssh-agent"
       EXEC="exec ssh-add"
else EXEC="exec"
fi

Here is the rsync command embedding in the .sh script.

Code:

rsync -v -u -a -e "ssh -i /home/postgres/.ssh/id_rsa" --progress --stats $BACKDIR $USER@$DEST:$DESTDIR

Here is the cron entry:

Code:

0   */2 * * *   postgres /home/postgres/backups/BST_postgres_backup_service.sh

I also set permissions:
private key file - LOCAL (600)
authorized_keys - LOCAL (700)
authorized_keys - REMOTE (600)
.ssh directory - BOTH (700)

thank you for your help!

thesnow · 12-08-2010, 10:21 AM

Do you get any log output from the cron job? If not, try adding something like

Code:

> /tmp/rsynclogfile 2>&1

to the end of your cron entry and see what you get.

jonathansfl · 12-08-2010, 10:34 AM

i put that line into the cron job and got this error in rsynclogfile:

Quote:

sudo: no tty present and no askpass program specified
sudo: no tty present and no askpass program specified
sending incremental file list
rsync: link_stat "/home/postgres/backups/BETA_CMS_MAIN_20101208104801.tar.gz" failed: No such file or directory (2)

Number of files: 0
Number of files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 3
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 12
Total bytes received: 12

sent 12 bytes received 12 bytes 16.00 bytes/sec
total size is 0 speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1060) [sender=3.0.7]
sudo: no tty present and no askpass program specified
sudo: no tty present and no askpass program specified
/home/postgres/backups/BST_postgres_backup_service.sh: 102: cannot create /home/postgres/backups/mybackup_BETA_CMS_MAIN.log: Permission denied

thesnow · 12-08-2010, 02:15 PM

It looks like permission problems, but you said you can run it successfully from the command line? What is the log output when you copy your cron line to a terminal and run it?

jonathansfl · 12-08-2010, 02:37 PM

That succeeds when i do it manually.

I entered:

Code:

sudo bash ./BST_postgres_backup_service.sh   > /tmp/BST_postgres_backup_service 2>&1

Quote:

sending incremental file list
BETA_CMS_MAIN_20101208142722.tar.gz
^M 32768 0% 0.00kB/s 0:00:00 ^M 2490368 24% 2.19MB/s 0:00:03 ^M 3145728 30% 1.32MB/s 0:0$

Number of files: 1
Number of files transferred: 1
Total file size: 10368634 bytes
Total transferred file size: 10368634 bytes
Literal data: 10368634 bytes
Matched data: 0 bytes
File list size: 52
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 10370002
Total bytes received: 31

sent 10370002 bytes received 31 bytes 531796.56 bytes/sec
total size is 10368634 speedup is 1.00

To try and fix permissions, i tried (which failed):

Code:

sudo chmod 777 ~/backups

Allowing cron to run produced the same:

Code:

sudo: no tty present and no askpass program specified
sudo: no tty present and no askpass program specified
sending incremental file list
rsync: link_stat "/home/postgres/backups/BETA_CMS_MAIN_20101208143501.tar.gz" failed: No such file or directory (2)

Number of files: 0
Number of files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 3
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 12
Total bytes received: 12

sent 12 bytes  received 12 bytes  16.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1060) [sender=3.0.7]
sudo: no tty present and no askpass program specified
sudo: no tty present and no askpass program specified
/home/postgres/backups/BST_postgres_backup_service.sh: 102: cannot create /home/postgres/backups/mybackup_BETA_CMS_MAIN.log: Permission denied

thesnow · 12-08-2010, 05:22 PM

It ran fine when you did

Code:

sudo bash ./BST_postgres_backup_service.sh   > /tmp/BST_postgres_backup_service 2>&1

But your cron shows

Code:

postgres /home/postgres/backups/BST_postgres_backup_service.sh

so what happens when you run the second one on the command line? It seems like your sudo is not working for some preceding commands, is the tar file even generated when it runs from the cron? Is your sudoers file set up to not ask for a password, or do you need to enter it when you run the first command manually? I assume since you are using sudo it is not running from the root cron.

jonathansfl · 12-09-2010, 10:48 AM

Hurray for you and your help. You pointed me in the right general direction. What I finally had to do was remove "sudo" from in front of the pg_dump and tar commands in my script. apparently by using sudo, i was changing the user that ran the command, and thus it didn't run correctly from cron (although manually it had no trouble).

FIXED