LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 09-22-2013, 08:24 PM   #1
timl
Member
 
Registered: Jan 2009
Location: Sydney, Australia
Distribution: Fedora,CentOS
Posts: 750

Rep: Reputation: 156Reputation: 156
rsync/cron/ssh


Hi, I have a cron job which runs every night and rsyncs the contents of one area on disk to a similar area on another computer:

Quote:
#!/bin/sh

RSYNC=/usr/bin/rsync
SSH=/usr/bin/ssh
KEY=/home/tim/.ssh/id_rsa
RUSER=tim
RHOST=10.1.1.51
RPATH=/data/tim/done/
LPATH=/data/tim/done/

$RSYNC -a -r -t -e "$SSH -i $KEY" $RUSER@$RHOST:$RPATH $LPATH
It stopped working last week and I get a mail message:
Quote:
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/home/tim>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=tim>
X-Cron-Env: <USER=tim>
Date: Mon, 23 Sep 2013 02:00:01 +1000 (EST)
Status: R

Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [receive
r=3.0.6]
This computer is running Centos 6. The computer from which the files are being dragged runs Fedora 19. The key mentioned id_rsa was generated back in June using ssh-keygen when I installed F19. Is it relevant that the key was generated on Jun 7 which is approx 100 days before the problems started (Most days there is nothing to copy so no suggestion it did not work)? A quick search suggests keys generated by ssh-keygen has no expiry.

If I try to run the rsync job from the command line it works but asks for the password of 10.1.1.51. So it looks like there is a problem with the key?

I run an automatic yum update every night but I can't see any updates on either end to ssh.

Any other ideas why a key just stops working?

Thanks
 
Old 09-23-2013, 12:19 AM   #2
eklavya
Member
 
Registered: Mar 2013
Posts: 636

Rep: Reputation: 142Reputation: 142
Have you tried to run the script manually?

Are you facing same permission denied error when you run the script on terminal without getting your desired result?

I think you should try to generate authorized_keys on the server once again so when you will try to be inside of server, it will not ask password. If your public key(the client system) is changed somehow, the server will not let you in with saved authorized_keys file.

authorized_keys is a file which contains public key of your system (the client from where you are trying log-into server)
 
Old 09-23-2013, 10:07 AM   #3
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
Quote:
Originally Posted by timl View Post

If I try to run the rsync job from the command line it works but asks for the password of 10.1.1.51. So it looks like there is a problem with the key?
this smells of a permissions issue on either the backup computer (your CentOS box if i follow you correctly) and your host computer, the F19 box.

follow the links in my sig for detailed information on ssh keys and permissions required for all the files involved as well as the directories including the /home directory. if the permissions are wrong keyless access will FAIL.

a few other things to consider is to create a rolling log file that self cleans every 30-45 days or so.

so add a few lines like so:

Code:
dtstamp="`date +%F-%A-%H.%M.%S `"
dow=`date +%A`
HOMEDIR="$HOME"
PWD=`pwd`
log=${PWD}/logs/`date +%Y-%m-%d-%A`-rsync.log

### Checking for logs directory, if not there then create it
#####################################

[ ! -d "${PWD}/logs" ] && mkdir -p ${PWD}/logs >> /dev/null 2>&1

rsync -aviS --exclude-from=${HOMEDIR}/excludes.txt ${PWD}/ ${RUSER}@${RHOST}:${RDIR}/${dow}/ >> ${log} 2>&1

### Clean up log directory to only keep 16 days woth of logs

find ${PWD}/logs/*.log -mtime +15 -exec rm '{}' \;

exit
note my variables are different then yours so please feel free to edit them as required. also note that I enclose all of my called variables inside {} this helps to prevent bad interpretation of the data I am trying to call.

my script cleans the log file up every 16 days, you could set yours to much long time for cleanup due to the limited size your rsync log will be. mine are rather large so i clean mine up more often then you might need require.
 
1 members found this post helpful.
Old 09-23-2013, 11:39 AM   #4
haertig
Senior Member
 
Registered: Nov 2004
Distribution: Debian, Ubuntu, LinuxMint, Slackware, SysrescueCD, Raspbian, Arch
Posts: 2,331

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Quote:
Originally Posted by lleb View Post
this smells of a permissions issue
My thoughts exactly. Look at your destination computer. Make sure you $HOME directory is not writeable by anyone but yourself. Make sure the same is true for your .ssh subdirectory. And make sure your authorized_keys file is not writeable by others as well. It is unlikely that /home itself would be writeable by others unless root made a typo in some chmod command, but if so, that will cause ssh pubkey authentication errors as well.

Test by running ssh from your source computer to your destination computer with the -v (verbose) option. Look at your log fil on your destination computer (might be something like /var/log/auth.log, but depends on your distro). Note that file/directory permission issues like lleb mentioned do NOT give you helpful error messages in either the destination log files or in the output of ssh -v on the source end. Things just fail, and you're left wondering why. When that happens (no helpful log mesages), it is almost always a file/directory permissions issue on the destination computer.
 
1 members found this post helpful.
Old 09-23-2013, 09:25 PM   #5
Beryllos
Member
 
Registered: Apr 2013
Location: Massachusetts
Distribution: Debian
Posts: 529

Rep: Reputation: 319Reputation: 319Reputation: 319Reputation: 319
Are you sure the IP address still points to the same computer?
 
Old 09-23-2013, 09:38 PM   #6
timl
Member
 
Registered: Jan 2009
Location: Sydney, Australia
Distribution: Fedora,CentOS
Posts: 750

Original Poster
Rep: Reputation: 156Reputation: 156
thanks for the hints guys. I am no further forward but I now have something to work on. I have 2 connections where I use a public key. Let's say 61->51 does not work but 51->93 does work.

I thought of privileges but not in the .ssh directory. So I compared the contents of the .ssh directory on all these computers and the privileges look the same. Ditto the protection on .ssh in the home directory.

The "-v" switch on ssh gives me something to dig into. This throws up:

Quote:
debug1: Trying private key: /home/tim/.ssh/identity
debug1: Offering public key: /home/tim/.ssh/id_rsa
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /home/tim/.ssh/id_dsa
debug1: Next authentication method: password
which again suggests something dodgy with the id_rsa file.

Like I say, I'll keep digging when I get a spare moment. There must be something obvious I can't see at the moment.

Cheers
 
Old 09-23-2013, 11:48 PM   #7
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and CentOS
Posts: 6,724

Rep: Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705
Hi,

I think you should be looking at the log from sshd on the remote host. On centos this would be /var/log/secure. If you don't have permission to read this file, then you can start up your ownd sshd in debug mode on a higher numbered port and look at its output when you try to connect to it.

So in one terminal on the remote host:
Code:
/usr/sbin/sshd -d -p 9999
If you get a /etc/ssh/sshd_config permission denied error, try with:
Code:
/usr/sbin/sshd -d -f /dev/null -p 9999
Then in another terminal on the localhost
Code:
ssh -v -i /home/tim/.ssh/id_rsa -p 9999 tim@10.1.1.51
and look at the output from sshd in the first terminal.

Evo2.
 
1 members found this post helpful.
Old 09-24-2013, 05:34 PM   #8
timl
Member
 
Registered: Jan 2009
Location: Sydney, Australia
Distribution: Fedora,CentOS
Posts: 750

Original Poster
Rep: Reputation: 156Reputation: 156
Hmmm, I have never used /var/log/secure before but that helps me solve this. I checked the log and got

Quote:
sshd[27329]: Authentication refused: bad ownership or modes for directory /home/tim
I didn't check my home directory before as I really didn't think this would be the problem. /home/tim was set to 775 so I chmodded this to 770 and the key works now.

I can't work out how /home/tim got into that state and how long it has been like that. And why? I guess that is my problem to solve.

Anyway. SOLVED
 
Old 09-24-2013, 05:42 PM   #9
haertig
Senior Member
 
Registered: Nov 2004
Distribution: Debian, Ubuntu, LinuxMint, Slackware, SysrescueCD, Raspbian, Arch
Posts: 2,331

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Well, I was half right:
Quote:
Originally Posted by haertig
Make sure you $HOME directory is not writeable by anyone but yourself.
... and half wrong:
Quote:
Originally Posted by haertig
Note that file/directory permission issues like lleb mentioned do NOT give you helpful error messages in either the destination log files...
Looks like more recent versions of SSHD are indeed logging the permissions problems into their log files. SSHD didn't used to do this. At least not on Solaris, even recent versions of it, but I didn't checked on a recent Linux system for this lack of logging and I just "assumed". I know, I know - never assume anything!
 
Old 09-24-2013, 05:47 PM   #10
haertig
Senior Member
 
Registered: Nov 2004
Distribution: Debian, Ubuntu, LinuxMint, Slackware, SysrescueCD, Raspbian, Arch
Posts: 2,331

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Quote:
Originally Posted by timl View Post
/home/tim was set to 775 so I chmodded this to 770 and the key works now.
You must have a typo there. If 775 didn't work, then 770 shouldn't have either. ??? 775 doesn't give "others" write permission to a directory any more than 770 does. 755 is a typical $HOME directory permission, or 700 if you are paranoid. 770 would be for "half paranoid" people.
 
Old 09-24-2013, 06:54 PM   #11
timl
Member
 
Registered: Jan 2009
Location: Sydney, Australia
Distribution: Fedora,CentOS
Posts: 750

Original Poster
Rep: Reputation: 156Reputation: 156
typo I changed to 700
 
Old 09-24-2013, 08:26 PM   #12
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and CentOS
Posts: 6,724

Rep: Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705
Hi,

I don't know what implementation of sshd is used on Solaris but most linux distros use openssh from openbsd. As far as I know it has always logged this stuff. IMHO, checking the sever logs when there are ssh problems is always the first thing to do. There are any number of things that can cause authentication problems, and it's no fun compiling a list of possible causes and then checking each one individually.

Cheers,

Evo2.
 
Old 09-27-2013, 06:32 PM   #13
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
Quote:
Originally Posted by timl View Post
Hmmm, I have never used /var/log/secure before but that helps me solve this. I checked the log and got



I didn't check my home directory before as I really didn't think this would be the problem. /home/tim was set to 775 so I chmodded this to 770 and the key works now.

I can't work out how /home/tim got into that state and how long it has been like that. And why? I guess that is my problem to solve.

Anyway. SOLVED
yup glad you found it. its exactly what i suspected.

http://wiki.linuxquestions.org/wiki/...reate_ssh_keys

see that link for permissions and proper settings on the keyless entry for ssh.
 
1 members found this post helpful.
Old 09-30-2013, 06:39 PM   #14
timl
Member
 
Registered: Jan 2009
Location: Sydney, Australia
Distribution: Fedora,CentOS
Posts: 750

Original Poster
Rep: Reputation: 156Reputation: 156
Thanks lieb. I plan to upgrade to F20 alpha shortly so these "how tos" are appreciated
 
Old 09-30-2013, 10:07 PM   #15
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
Quote:
Originally Posted by timl View Post
Thanks lieb. I plan to upgrade to F20 alpha shortly so these "how tos" are appreciated
keep in mind if you are using full disk encryption, there is a known bug with fedup that will cause you to fail the upgrade process. At least there was when F19 went live a few months back. I have not read/heard that it has been resolved yet.

There are work-a-arounds for partitions that are encrypted, but none that ive seen for full disk encryption.

if you are running full disk encryption, and why would you not, then you will need to backup up all of your /home or other personal files/folders/customization/etc... and perform a full fresh install.

As much as Id love to move to F20, im going to wait as all three of my laptops running F19 are currently running full disk encryption. Im not looking forward to those 3 full installs, so im hoping the Fedora devs can figure out how to get around the full disk encryption failures for fedup.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
rsync over ssh VS rsync.d RISKS tripialos Linux - Security 4 02-20-2013 06:22 PM
[SOLVED] rsync fails in cron - ssh key prob for rsync? jonathansfl Linux - Server 6 12-09-2010 09:48 AM
CRON problem: (rsync: Failed to exec ssh: Permission denied (13)) Fairys Fedora 6 01-16-2008 06:34 PM
Auto RSYNC Over SSH via Cron? carlosinfl Linux - General 1 09-10-2007 02:12 PM
Rsync server vs rsync over ssh humbletech99 Linux - Networking 1 10-18-2006 12:10 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 05:58 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration