LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 02-23-2016, 04:43 PM   #1
timl
Member
 
Registered: Jan 2009
Location: Sydney, Australia
Distribution: Fedora,CentOS
Posts: 750

Rep: Reputation: 156Reputation: 156
cron and rsync not working for me


Hi, I have had a bit of a nightmare recently and lost 2 disks which ruined my backup strategy. Next step was to backup my server to an old PC I sometimes use which runs Fedora 23 – all good using the following script (IP add mangled) and cron:

Quote:
#!/bin/sh

RSYNC=/usr/bin/rsync
SSH=/usr/bin/ssh
KEY=/home/tim/.ssh/id_rsa
RUSER=tim
RHOST=10.928.57.16
RPATH=/home/doco
LPATH=/home/tim


$RSYNC -a -r -t -e "$SSH -i $KEY" $RUSER@$RHOST:$RPATH $LPATH
Using that PC though is inconvenient as I have to switch it on purely for the backup. So, I decided to use my everyday PC (also F23) to receive the files. Now here comes a familiar problem – the script works from the CLI but not via cron.
I had a look at the journalctl file which tells me:

Quote:
Feb 24 09:28:02 dragon CROND[9884]: (tim) CMDOUT (rsync: connection unexpectedly closed (0 bytes received so far) [Receiver])
Feb 24 09:28:02 dragon CROND[9884]: (tim) CMDOUT (rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1])
Feb 24 09:28:02 dragon audit[9884]: USER_END pid=9884 uid=0 auid=1000 ses=30 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_loginuid,pam_keyinit,pam_limits,pam_systemd acct="tim" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
So the server closed the connection. Over at the server end though (Centos 6) there is nothing in /var/log/messages to explain this and I can't see any other logs which are getting me any closer to a solution.

Any ideas where I can look?

I have identified an adequate workaround but I would like to identify this problem.

Cheers

[EDIT] I have SSH keys to get me between each PC. Password authentication is switched off on the server.

Last edited by timl; 02-23-2016 at 05:39 PM.
 
Old 02-24-2016, 03:44 AM   #2
tshikose
Member
 
Registered: Apr 2010
Location: Kinshasa, Democratic Republic of Congo
Distribution: RHEL, Fedora, CentOS
Posts: 525

Rep: Reputation: 95
Try to run rsync with more verbosity -v or -vv, then let's see the cron logs.
 
Old 02-24-2016, 02:57 PM   #3
timl
Member
 
Registered: Jan 2009
Location: Sydney, Australia
Distribution: Fedora,CentOS
Posts: 750

Original Poster
Rep: Reputation: 156Reputation: 156
thanks, I normally include "-v" (F23 doesn't like -w). I am not sure why this slipped through the net however I am still struggling to find logs. At the Fedora end:
Quote:
Feb 25 07:49:01 dragon CROND[1624]: (tim) CMD (/home/tim/rsync_doco)
Feb 25 07:49:01 dragon CROND[1620]: [75B blob data]
Feb 25 07:49:01 dragon audit[1620]: CRED_DISP pid=1620 uid=0 auid=1000 ses=97 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="tim" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
Feb 25 07:49:01 dragon CROND[1620]: [75B blob data]
Feb 25 07:49:01 dragon CROND[1620]: [75B blob data]
Feb 25 07:49:01 dragon CROND[1620]: [74B blob data]
Feb 25 07:49:01 dragon CROND[1620]: [88B blob data]
Feb 25 07:49:01 dragon CROND[1620]: [49B blob data]
Feb 25 07:49:01 dragon audit[1620]: USER_END pid=1620 uid=0 auid=1000 ses=97 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_loginuid,pam_keyinit,pam_limits,pam_systemd acct="tim" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
Feb 25 07:49:01 dragon CROND[1620]: [65B blob data]
Feb 25 07:49:01 dragon CROND[1620]: [75B blob data]
Feb 25 07:49:01 dragon CROND[1620]: (tim) CMDOUT (rsync: connection unexpectedly closed (0 bytes received so far) [Receiver])
Feb 25 07:49:01 dragon CROND[1620]: (tim) CMDOUT (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.1])
But I guess it is the Centos end which is going to contain the reason for the disconnect but nothing in /var/log/messages and /var/log/cron doesn't:
Quote:
Feb 25 07:40:01 localhost CROND[11217]: (root) CMD (/usr/lib/sa/sa1 1 1)
Feb 25 07:50:01 localhost CROND[11276]: (root) CMD (/usr/lib/sa/sa1 1 1)
Actually though
Quote:
Feb 25 07:49:01 dragon CROND[1620]: (tim) CMDOUT (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.1])
I didn't see that before and will follow up

Cheers
 
Old 02-24-2016, 05:47 PM   #4
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
Also, stop using variable names in all caps. The chances are WAY too high that you'll accidentally replace or set an environment variable you've never even heard of but can screw up the behavior of your script. Variables names in all caps are reserved for environment variables, use lower or mixed case for local variables in a script.

I'm not saying that's your problem, but it's a very bad habit to get into.

Last edited by suicidaleggroll; 02-24-2016 at 05:49 PM.
 
Old 02-25-2016, 09:01 AM   #5
Beryllos
Member
 
Registered: Apr 2013
Location: Massachusetts
Distribution: Debian
Posts: 529

Rep: Reputation: 319Reputation: 319Reputation: 319Reputation: 319
Quote:
Originally Posted by timl View Post
thanks, I normally include "-v" (F23 doesn't like -w).
What tshikose wrote was "-vv", but in the standard font it looks like -w.
 
Old 02-25-2016, 02:19 PM   #6
jdelaporte
LQ Newbie
 
Registered: Jun 2014
Posts: 8

Rep: Reputation: Disabled
/bin/bash vs /bin/sh?

Does your cron use /bin/sh, /bin/bash, or another shell by default? I don't have a Fedora 23 box to compare, but from various sites it appears cron on F23 probably uses /bin/bash by default. Does your script work correctly when run using /bin/bash rather than /bin/sh?

Joanna
 
Old 02-25-2016, 02:26 PM   #7
jdelaporte
LQ Newbie
 
Registered: Jun 2014
Posts: 8

Rep: Reputation: Disabled
Also, I was just reading the rsync manpage again now, and noticed this part that might apply to your situation (if you are using the rsync daemon and a different user on one end vs the other):

"If you need to specify a different remote-shell user, keep in mind that the user@ prefix in front of the host is specifying the rsync-user value (for a module that requires user-based authentication). This means that you must give the '-l user' option to ssh when specifying the remote-shell, as in this example that uses the short version of the --rsh option:
rsync -av -e "ssh -l ssh-user" rsync-user@host::module /dest"

This is only for rsync daemon connections, however. It doesn't appear that is what you are using, so this may not apply at all.

Last edited by jdelaporte; 02-25-2016 at 02:31 PM. Reason: Incomplete information
 
Old 02-25-2016, 02:42 PM   #8
jdelaporte
LQ Newbie
 
Registered: Jun 2014
Posts: 8

Rep: Reputation: Disabled
According to askubuntu, you can also see that rsync error if the remote (or destination) path does not exist. However, if the script runs fine when you run it manually, then the path locations have to be correct and present. Does the user have read and write access to the directory? Is cron running the job as the same user that you used to run the command manually?
 
Old 02-25-2016, 05:41 PM   #9
timl
Member
 
Registered: Jan 2009
Location: Sydney, Australia
Distribution: Fedora,CentOS
Posts: 750

Original Poster
Rep: Reputation: 156Reputation: 156
thanks for all the response everyone. As noted I have a workaround for this and other priorities at the moment.

Still, I would like to get to the bottom of this problem and will try to look at it over the weekend and update my status

Cheers
 
Old 02-26-2016, 09:52 PM   #10
timl
Member
 
Registered: Jan 2009
Location: Sydney, Australia
Distribution: Fedora,CentOS
Posts: 750

Original Poster
Rep: Reputation: 156Reputation: 156
Once again thanks for all the assistance. I'll try and go through your points individually Joanne:

Quote:
Does your cron use /bin/sh, /bin/bash, or another shell by default? I don't have a Fedora 23 box to compare, but from various sites it appears cron on F23 probably uses /bin/bash by default. Does your script work correctly when run using /bin/bash rather than /bin/sh?
I am really not sure about the shells. I am assuming bash but it would be good to know how to find out. I tried /bin/bash but same problem

Quote:
"If you need to specify a different remote-shell user, keep in mind that the user@ prefix in front of the host is specifying the rsync-user value (for a module that requires user-based authentication). This means that you must give the '-l user' option to ssh when specifying the remote-shell, as in this example that uses the short version of the --rsh option:
rsync -av -e "ssh -l ssh-user" rsync-user@host::module /dest"
Yeah, I am pretty lazy and only one user on each computer so the username is the same across boxes. I gave the "-l" a whirl but no joy.

Quote:
According to askubuntu, you can also see that rsync error if the remote (or destination) path does not exist. However, if the script runs fine when you run it manually, then the path locations have to be correct and present. Does the user have read and write access to the directory? Is cron running the job as the same user that you used to run the command manually?
Thanks. I had a root around the net and everything pointed to something wrong at the remote end. There is nothing obvious leaping out at me though.

As the script works not only when invoked manually but also on another F23 box I have been digging around for differences between the 2 PCs. I noticed the privileges on the directory holding the script were different although the problematic box had more privileges. I changed to the privileges on the successful box (700). No difference

I'll keep plugging away as and when I get time.

Cheers
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
/USR/SBIN/CRON --> Suspicious Capital Letter, cron is not working??? arfal Linux - Server 10 06-22-2012 06:40 AM
[SOLVED] rsync fails in cron - ssh key prob for rsync? jonathansfl Linux - Server 6 12-09-2010 09:48 AM
Cron and Rsync SimonTHK Linux - Software 16 10-07-2009 03:09 AM
rsync and cron numtre Linux - Software 1 01-08-2009 06:54 AM
cron not working from crontab nor form /etc/cron/cron.d. What did SuSE change? JZL240I-U SUSE / openSUSE 11 01-04-2007 01:57 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 10:23 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration