rsync and scp fails when in a script

keirvt · 03-21-2013, 04:05 PM

I have rsync in a script transfering a postgres backup file SQL text to another machine. Root runs the script and the .ssh key are set up to allow it.

Both machines are Red Hat Enterprise Linux Server release 5.9 (Tikanga)
The script is

#!/bin/sh
export localArchive=/tmp/pgDB.sql
remoteMachine="machine2.library.usyd.edu.au"
rsync -abvz ${localArchive} ${remoteMachine}:/data/

If the commands are issued from a command line it works.
If the commands are run from inside a script the transfer is zero.
I tried changing the shell from bash to sh and about 800 bytes transfered.

I tried changing rsync to scp. When run from a command line it works but when run as a script it fails.

It is current working from within the script on another machine but I observed on previous tests only partial transfers that cured after putting a wait command after the rsync.

It may be a network problem or restriction but I'm mystified that the problem only exists when run from inside a script.

I am baffled - any ideas?

smallpond · 03-21-2013, 05:52 PM

When you run a bash script it executes your .bashrc file. Anything in there that could cause a problem for a script? PATH is set properly? You aren't doing some weird setuid thing, are you?

KinnowGrower · 03-21-2013, 08:10 PM

Script run as root and you also run the rsync and scp command as root or different user?. You can also try to check log on the destination server for errors e.g /var/log/secure or /var/log/auth.log

lleb · 03-23-2013, 10:37 PM

im not seeing a remote user in that script. you have the remote system, but not the remote user.

format should be:

Code:

rsync -options /path/to/source remote_user@remote_system:/path/to/remote/destination

you dont have a remote user. also please use the code flags for posting scripts. thank you.

keirvt · 03-24-2013, 03:57 PM

Using a user name in the rsync destination shouldn't be necessary as it should default to the id of the invoking program, in this case root. Nevertheless I added root as a user and it made no difference.

Looking at /var/log/secure of the target machine the connection has worked (when running rsynv within the script)

Accepted publickey for root from 129.78.139.4 port 41451 ssh2
Mar 25 07:45:52 chara-uat sshd[15369]: pam_unix(sshd:session): session opened for user root by (uid=0)
Mar 25 07:45:52 chara-uat sshd[15369]: pam_unix(sshd:session): session closed for user root

Looking at /var/log/secure on the sending machine reports
sshd[14426]: Bad protocol version identification.

Both machines have the same ssh version
OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008

It remains the case that when run inside a script the rsync creates the file on the target machine end but the file is empty. If the script is run from a command line it works.

keirvt · 03-24-2013, 04:51 PM

The bad protocal message was not relevant. According to our local sysadmin its "check machine" making sure certain daemons are running and not issuing very helpful messages.

keirvt · 03-24-2013, 05:49 PM

I apologise for this post. I left a small part of code out for what I thought was brevity but the extra removed line was the cause of the problem. The full relevant code is

Code:

localArchive=/tmp/pgDspaceChara
remoteMachine="targetMachine.mycompany.com.au"
su --shell=/bin/bash --session-command="/usr/bin/pg_dumpall --clean --oids" postgres > ${localArchive}&
rsync -abvz ${localArchive} root@${remoteMachine}:/data/

The postgres dump has a background ampersand on the end which should not be there.
Before the dump process can complete, rsync attempts to run and the file transfer is incomplete. Hence the reason why sometimes there was a partial transfer.

By the time I checked the file in /tmp the dump completed and the /tmp file looked okay and also thne rsync tried from the command line would work. Running the script again starts rewriting the file and gives the appearance that it works in the command line but not the script.

Thanks for your suggestion anyway they were helpful. Thanx