LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 08-10-2016, 11:01 PM   #1
computer_freak_8
Member
 
Registered: Jul 2008
Location: Iowa
Distribution: Ubuntu and Fedora, mostly
Posts: 103

Rep: Reputation: 15
rsync script backup to encrypted home directory


I've done a lot of searching, but not quite found anything that suits my situation. The closest I found is here.

Here's the setup:

Location A:
  • Backup site
  • Can't guarantee physical safety, so encrypted home directory
  • Running Ubuntu CLI-only
Location B:
  • Home
  • Contains data to be backed up
  • Currently Fedora (but mostly distro-irrelevant)

At location A, I have an rsync script/cron job, that connects to location B, and backs up what I want it to. It works whether running as root or not, as long as I run it manually. However, as soon as I let it try to run on its own, it fails. I should note that I've setup the script with logging and key-based authentication. So, when I say "I run it manually", I mean I SSH into the location A box with username/password, and I execute "/path/to/my/script", and it does everything else.

Here's what I get when it runs on its own:
Code:
2016/08/10 00:01:02 [28952] rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
2016/08/10 00:01:02 [28952] rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.0]
Code:
2016/08/10 00:01:02 [28955] rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
2016/08/10 00:01:02 [28955] rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.0]
What am I doing wrong/how do I make this work?

Last edited by computer_freak_8; 08-10-2016 at 11:01 PM. Reason: added ending/clarified need
 
Old 08-11-2016, 12:58 AM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,903

Rep: Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614
Quote:
I SSH into the location A box with username/password
If you're doing that, you haven't set up ssh-keys correctly; it shouldn't be asking for user/passwd....

Can tell us how you tried to setup ssh-keys and show the target sshd_config file content?
Also show rsync cmd line.
 
Old 08-20-2016, 11:58 PM   #3
computer_freak_8
Member
 
Registered: Jul 2008
Location: Iowa
Distribution: Ubuntu and Fedora, mostly
Posts: 103

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by chrism01 View Post
If you're doing that, you haven't set up ssh-keys correctly; it shouldn't be asking for user/passwd....
Location A is the backup site. Instead of walking up to it witb a keyboard/mouse, I SSH into it manually (username/password), and then, via keys, SSH/rsync the Location B box.

Will post rsync command line when I get a chance.
 
Old 08-21-2016, 03:47 AM   #4
IsaacKuo
Senior Member
 
Registered: Apr 2004
Location: Baton Rouge, Louisiana, USA
Distribution: Debian Stable
Posts: 2,398
Blog Entries: 8

Rep: Reputation: 412Reputation: 412Reputation: 412Reputation: 412Reputation: 412
Quote:
Originally Posted by chrism01 View Post
If you're doing that, you haven't set up ssh-keys correctly; it shouldn't be asking for user/passwd....

Can tell us how you tried to setup ssh-keys and show the target sshd_config file content?
Also show rsync cmd line.
It's perfectly possible to set up (default) password authentication in one direction, but key authentication in the other direction.

Still, the OP might be unaware of another option - to set up key authentication with passphrase on location A. That's possibly more secure than ordinary userid/password authentication, because it still requires a key AND it requires a passphrase to utilize the key.
 
Old 08-21-2016, 09:40 AM   #5
computer_freak_8
Member
 
Registered: Jul 2008
Location: Iowa
Distribution: Ubuntu and Fedora, mostly
Posts: 103

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by IsaacKuo View Post
Still, the OP might be unaware of another option - to set up key authentication with passphrase on location A. That's possibly more secure than ordinary userid/password authentication, because it still requires a key AND it requires a passphrase to utilize the key.
I never know what machine I'm going to need to access Location A box from. It's only accessible from the LAN, so it could be one of many Macs, it could be one of a few Windows boxes, it could be my Chromebook. So it's easier just to use username/password. It's a security vs convenience tradeoff.


Here's the code I'm using, with some redactions (modified text in bold):
crontab -l
Code:
# rsync stuff, added 2016-04-20
01 00 * * * /opt/scripts/rsync-script_no-root.sh
Code:
user@LocationA:~$ cat /opt/scripts/rsync-script_no-root.sh
#!/bin/bash


rsync -vah --del --progress --log-file=/opt/log/rsync-script_line1_`date +%F_%H%M%S`.log -e "ssh -p 22 -i /home/user/.ssh/id_rsa" --exclude 'Path/file-to-ignore.ext' user@LocationB.fqdn:/mnt/path/dir_one/ /home/user/data-from_LocationB/path-to_dir-one/

rsync -vah --del --progress --log-file=/opt/log/rsync-script_line2_`date +%F_%H%M%S`.log -e "ssh -p 22 -i /home/user/.ssh/id_rsa" user@LocationB.fqdn:/mnt/path/dir_two/ /home/user/data-from_LocationB/path-to_dir-two/




#EOF
 
Old 08-22-2016, 02:25 PM   #6
IsaacKuo
Senior Member
 
Registered: Apr 2004
Location: Baton Rouge, Louisiana, USA
Distribution: Debian Stable
Posts: 2,398
Blog Entries: 8

Rep: Reputation: 412Reputation: 412Reputation: 412Reputation: 412Reputation: 412
Hmm... no other responses since yesterday morning? I'm afraid I am personally not an expert in using rsync the "proper" way, but I can help you figure out how to do it "my" way.

I started off using rsync strictly on a local lan level, using it basically as a more sophisticated alternative to cp (only making incremental changes rather than deleting the entire backup folder and copying over everything). This was with a simple nfs file server; my backup script was run on an nfs client workstation.

My variants for off site backups still use rsync in this naive way, I just include commands to mount and unmount an sshfs file share from my main server. So my script file would look something like:

Code:
sshfs HOMESERVER:/home/USER ~/remoteHome
rsync -vax --delete ~/remoteHome/Documents ~/backups
sleep 2
fusermount -u ~/remoteHome
HOMESERVER could be defined in /etc/hosts to provide a convenient place to define its IP address. I don't think it's necessary to specify the use of ~/.ssh/id_rsa because it will use that (private key) file by default. But of course, it's necessary to set up the private key and the public key to log in. Which you've already done since it works when running manually.

This isn't a sophisticated way of doing things, but it is a way of doing it. The script has to be run as the user with access to ~/.ssh/id_rsa, of course.

Umm...could that be your problem? What user is the cron job running as? This user will need read access to /home/USER/.ssh/id_rsa. Default is that no one but the owning user has read access to that file. Root also has access anyway because root is root.
 
Old 08-22-2016, 03:55 PM   #7
computer_freak_8
Member
 
Registered: Jul 2008
Location: Iowa
Distribution: Ubuntu and Fedora, mostly
Posts: 103

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by IsaacKuo View Post
The script has to be run as the user with access to ~/.ssh/id_rsa, of course.

Umm...could that be your problem? What user is the cron job running as? This user will need read access to /home/USER/.ssh/id_rsa. Default is that no one but the owning user has read access to that file. Root also has access anyway because root is root.
The script is in root's crontab. Works when manually running as myself or as root. Doesn't work in the cron job.

Last edited by computer_freak_8; 08-22-2016 at 04:26 PM. Reason: clarified pronoun
 
Old 08-22-2016, 04:53 PM   #8
Pearlseattle
Member
 
Registered: Aug 2007
Location: Zurich, Switzerland
Distribution: Gentoo
Posts: 997

Rep: Reputation: 130Reputation: 130
If you say that everything works perfectly when run manually: maybe the crontab doesn't automatically set the user's home directory or whatever other globals that are set when you login manually into a shell? => What about specifying explicitly in the crontab command (in the ssh-part) the directory and/or files and/or username that it's supposed to use to get private & public key?
 
1 members found this post helpful.
Old 08-22-2016, 05:05 PM   #9
jailbait
LQ Guru
 
Registered: Feb 2003
Location: Mineral, Virginia
Distribution: Debian 8
Posts: 7,893

Rep: Reputation: 339Reputation: 339Reputation: 339Reputation: 339
Quote:
Originally Posted by computer_freak_8 View Post
The script is in root's crontab. Works when manually running as myself or as root. Doesn't work in the cron job.
The crontab PATH is much smaller than the typical PATH for root or a user. In the script try giving the full path to every program that you execute. For example instead of using the command:

rsync

Use the command:

/usr/bin/rsync

You can find out the full path name for a program by using the which command:

which rsync

---------------------
Steve Stites
 
1 members found this post helpful.
Old 09-28-2016, 09:46 AM   #10
computer_freak_8
Member
 
Registered: Jul 2008
Location: Iowa
Distribution: Ubuntu and Fedora, mostly
Posts: 103

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by Pearlseattle View Post
If you say that everything works perfectly when run manually: maybe the crontab doesn't automatically set the user's home directory or whatever other globals that are set when you login manually into a shell? => What about specifying explicitly in the crontab command (in the ssh-part) the directory and/or files and/or username that it's supposed to use to get private & public key?
In the posted command line, I specify the full path to the private key.


Quote:
Originally Posted by jailbait View Post
The crontab PATH is much smaller than the typical PATH for root or a user. In the script try giving the full path to every program that you execute. For example instead of using the command:

rsync

Use the command:

/usr/bin/rsync
You both mentioned this part... so you're saying something more like this?
Code:
/usr/bin/rsync -vah --del --progress --log-file=/opt/log/rsync-script_line1_`date +%F_%H%M%S`.log -e "/usr/bin/ssh -p 22 -i /home/user/.ssh/id_rsa" --exclude 'Path/file-to-ignore.ext' user@LocationB.fqdn:/mnt/path/dir_one/ /home/user/data-from_LocationB/path-to_dir-one/
Note: leaving "date" command alone, as I know for sure it works, since it logs the date and time, whether or not the script actually copies anything.
 
Old 09-29-2016, 11:46 PM   #11
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,903

Rep: Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614Reputation: 2614
Yep; in cron just assume you have to specify the complete path to all progs and files (bash built-ins excluded).
OR, first src a file that defines a better PATH and possibly cd's into the correct start dir.
Basically, assume nothing

If you google it, you'll find millions of matches for that exact issue.

HTH
 
1 members found this post helpful.
Old 09-30-2016, 01:00 AM   #12
c0wb0y
Member
 
Registered: Jan 2012
Location: Inside the oven
Distribution: Windows
Posts: 417

Rep: Reputation: 74
I tidied it up a bit. No need to specify ssh port nor its key as they seem to be the default anyway. I added 'z' to make transfer a bit faster. Assuming that the source machine can ssh without asking for ssh paraphrase.

Code:
rsync -avze ssh --log-file=/opt/log/rsync-script_line1_`date +%F_%H%M%S`.log --exclude=Path/file-to-ignore.ext user@LocationB.fqdn:/mnt/path/dir_one/ /home/user/data-from_LocationB/path-to_dir-one/
 
Old 09-30-2016, 08:19 AM   #13
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 9,342
Blog Entries: 4

Rep: Reputation: 3332Reputation: 3332Reputation: 3332Reputation: 3332Reputation: 3332Reputation: 3332Reputation: 3332Reputation: 3332Reputation: 3332Reputation: 3332Reputation: 3332
I would definitely recommend using "encrypted keys," not passwords of any sort.

Encrypted keys normally involves the use of a background "SSH key agent" (man ssh-agent) which might need to be started on some intermediate host.

An encrypted key is much stronger than any password because it is "a key," therefore non-forgeable, yet it requires a password in order to use the key. The password that's used to decrypt the key is never presented to the remote system and is not what the remote demands to see.
 
Old 10-07-2016, 07:17 PM   #14
computer_freak_8
Member
 
Registered: Jul 2008
Location: Iowa
Distribution: Ubuntu and Fedora, mostly
Posts: 103

Original Poster
Rep: Reputation: 15
This is still not working, even after specifying the full path to the rsync and ssh binaries.

Also of note:
  1. I'm using my user's crontab, not root's crontab. I'd read somewhere that using root might break it because the key doesn't match the username.
  2. I'm changing it tonight. Going to try re-running as root crontab, but with the full path to the binaries.

Quote:
Originally Posted by c0wb0y View Post
I tidied it up a bit. No need to specify ssh port nor its key as they seem to be the default anyway.
Port is actually different. I just sanitized the line. SSH key is in the default spot for the user, though.

Quote:
Originally Posted by sundialsvcs View Post
I would definitely recommend using "encrypted keys," not passwords of any sort.
...is that not what I'm doing? See the -i parameter I'm using in my command line.
 
Old 10-09-2016, 09:21 AM   #15
computer_freak_8
Member
 
Registered: Jul 2008
Location: Iowa
Distribution: Ubuntu and Fedora, mostly
Posts: 103

Original Poster
Rep: Reputation: 15
It works!!

Specifying the full path of the binaries, combined with making sure the entry is under root's crontab, seems to have done the trick.

Thank you!
 
  


Reply

Tags
encrypted, not working, rsync, ssh


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
About Encrypted backup maybe home? PeterUK Slackware 5 07-28-2013 11:24 AM
How to recover my encrypted Home directory pazzport Linux - Desktop 7 05-14-2012 11:30 PM
[SOLVED] Automount other harddrives while encrypted home directory sadarax Linux - Security 5 11-06-2011 07:59 PM
fsck or equivalent for my encrypted home directory cantab Ubuntu 2 09-20-2010 01:41 PM
Cannot mount unencrypted directory to encrypted home directory with fstab? Daravon Ubuntu 35 09-14-2010 03:50 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration