LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
LinkBack Search this Thread
Old 03-21-2013, 04:05 PM   #1
keirvt
Member
 
Registered: Sep 2006
Location: Sydney Australia
Distribution: fedora/Ubuntu
Posts: 77

Rep: Reputation: 15
rsync and scp fails when in a script


I have rsync in a script transfering a postgres backup file SQL text to another machine. Root runs the script and the .ssh key are set up to allow it.

Both machines are Red Hat Enterprise Linux Server release 5.9 (Tikanga)
The script is

#!/bin/sh
export localArchive=/tmp/pgDB.sql
remoteMachine="machine2.library.usyd.edu.au"
rsync -abvz ${localArchive} ${remoteMachine}:/data/

If the commands are issued from a command line it works.
If the commands are run from inside a script the transfer is zero.
I tried changing the shell from bash to sh and about 800 bytes transfered.

I tried changing rsync to scp. When run from a command line it works but when run as a script it fails.

It is current working from within the script on another machine but I observed on previous tests only partial transfers that cured after putting a wait command after the rsync.

It may be a network problem or restriction but I'm mystified that the problem only exists when run from inside a script.

I am baffled - any ideas?
 
Old 03-21-2013, 05:52 PM   #2
smallpond
Senior Member
 
Registered: Feb 2011
Location: Massachusetts, USA
Distribution: Fedora
Posts: 1,156

Rep: Reputation: 258Reputation: 258Reputation: 258
When you run a bash script it executes your .bashrc file. Anything in there that could cause a problem for a script? PATH is set properly? You aren't doing some weird setuid thing, are you?
 
Old 03-21-2013, 08:10 PM   #3
KinnowGrower
Member
 
Registered: May 2008
Location: Toronto
Distribution: Centos && Debian
Posts: 341

Rep: Reputation: 34
Script run as root and you also run the rsync and scp command as root or different user?. You can also try to check log on the destination server for errors e.g /var/log/secure or /var/log/auth.log
 
Old 03-23-2013, 10:37 PM   #4
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,284

Rep: Reputation: 394Reputation: 394Reputation: 394Reputation: 394
im not seeing a remote user in that script. you have the remote system, but not the remote user.

format should be:

Code:
rsync -options /path/to/source remote_user@remote_system:/path/to/remote/destination
you dont have a remote user. also please use the code flags for posting scripts. thank you.
 
Old 03-24-2013, 03:57 PM   #5
keirvt
Member
 
Registered: Sep 2006
Location: Sydney Australia
Distribution: fedora/Ubuntu
Posts: 77

Original Poster
Rep: Reputation: 15
Using a user name in the rsync destination shouldn't be necessary as it should default to the id of the invoking program, in this case root. Nevertheless I added root as a user and it made no difference.

Looking at /var/log/secure of the target machine the connection has worked (when running rsynv within the script)

Accepted publickey for root from 129.78.139.4 port 41451 ssh2
Mar 25 07:45:52 chara-uat sshd[15369]: pam_unix(sshd:session): session opened for user root by (uid=0)
Mar 25 07:45:52 chara-uat sshd[15369]: pam_unix(sshd:session): session closed for user root

Looking at /var/log/secure on the sending machine reports
sshd[14426]: Bad protocol version identification.

Both machines have the same ssh version
OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008

It remains the case that when run inside a script the rsync creates the file on the target machine end but the file is empty. If the script is run from a command line it works.
 
Old 03-24-2013, 04:51 PM   #6
keirvt
Member
 
Registered: Sep 2006
Location: Sydney Australia
Distribution: fedora/Ubuntu
Posts: 77

Original Poster
Rep: Reputation: 15
Bad protocol

The bad protocal message was not relevant. According to our local sysadmin its "check machine" making sure certain daemons are running and not issuing very helpful messages.
 
Old 03-24-2013, 05:49 PM   #7
keirvt
Member
 
Registered: Sep 2006
Location: Sydney Australia
Distribution: fedora/Ubuntu
Posts: 77

Original Poster
Rep: Reputation: 15
Solved

I apologise for this post. I left a small part of code out for what I thought was brevity but the extra removed line was the cause of the problem. The full relevant code is

Code:
localArchive=/tmp/pgDspaceChara
remoteMachine="targetMachine.mycompany.com.au"
su --shell=/bin/bash --session-command="/usr/bin/pg_dumpall --clean --oids" postgres > ${localArchive}&
rsync -abvz ${localArchive} root@${remoteMachine}:/data/
The postgres dump has a background ampersand on the end which should not be there.
Before the dump process can complete, rsync attempts to run and the file transfer is incomplete. Hence the reason why sometimes there was a partial transfer.

By the time I checked the file in /tmp the dump completed and the /tmp file looked okay and also thne rsync tried from the command line would work. Running the script again starts rewriting the file and gives the appearance that it works in the command line but not the script.

Thanks for your suggestion anyway they were helpful. Thanx

Last edited by keirvt; 03-24-2013 at 05:51 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Linux scp or rsync without password sanjee Linux - Server 15 09-13-2012 02:46 PM
rsync with scp behavior jjj0923 Linux - Newbie 3 07-12-2011 08:22 AM
SCP or rsync? centosfan Linux - General 4 01-09-2011 08:02 PM
[SOLVED] rsync fails in cron - ssh key prob for rsync? jonathansfl Linux - Server 6 12-09-2010 09:48 AM
[SOLVED] rsync -z vs. scp ... which one is faster over network? someshpr Linux - General 5 12-08-2010 06:06 PM


All times are GMT -5. The time now is 04:03 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration