[SOLVED] rsync script backup to encrypted home directory

computer_freak_8 · 08-10-2016, 11:01 PM

I've done a lot of searching, but not quite found anything that suits my situation. The closest I found is here.

Here's the setup:

Location A:

Backup site
Can't guarantee physical safety, so encrypted home directory
Running Ubuntu CLI-only

Location B:

Home
Contains data to be backed up
Currently Fedora (but mostly distro-irrelevant)

At location A, I have an rsync script/cron job, that connects to location B, and backs up what I want it to. It works whether running as root or not, as long as I run it manually. However, as soon as I let it try to run on its own, it fails. I should note that I've setup the script with logging and key-based authentication. So, when I say "I run it manually", I mean I SSH into the location A box with username/password, and I execute "/path/to/my/script", and it does everything else.

Here's what I get when it runs on its own:

Code:

2016/08/10 00:01:02 [28952] rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
2016/08/10 00:01:02 [28952] rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.0]

Code:

2016/08/10 00:01:02 [28955] rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
2016/08/10 00:01:02 [28955] rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.0]

What am I doing wrong/how do I make this work?

chrism01 · 08-11-2016, 12:58 AM

Quote:

I SSH into the location A box with username/password

If you're doing that, you haven't set up ssh-keys correctly; it shouldn't be asking for user/passwd....

Can tell us how you tried to setup ssh-keys and show the target sshd_config file content?
Also show rsync cmd line.

computer_freak_8 · 08-20-2016, 11:58 PM

Quote:

Originally Posted by chrism01

If you're doing that, you haven't set up ssh-keys correctly; it shouldn't be asking for user/passwd....

Location A is the backup site. Instead of walking up to it witb a keyboard/mouse, I SSH into it manually (username/password), and then, via keys, SSH/rsync the Location B box.

Will post rsync command line when I get a chance.

IsaacKuo · 08-21-2016, 03:47 AM

Quote:

Originally Posted by chrism01

If you're doing that, you haven't set up ssh-keys correctly; it shouldn't be asking for user/passwd....

Can tell us how you tried to setup ssh-keys and show the target sshd_config file content?
Also show rsync cmd line.

It's perfectly possible to set up (default) password authentication in one direction, but key authentication in the other direction.

Still, the OP might be unaware of another option - to set up key authentication with passphrase on location A. That's possibly more secure than ordinary userid/password authentication, because it still requires a key AND it requires a passphrase to utilize the key.

computer_freak_8 · 08-21-2016, 09:40 AM

Quote:

Originally Posted by IsaacKuo

Still, the OP might be unaware of another option - to set up key authentication with passphrase on location A. That's possibly more secure than ordinary userid/password authentication, because it still requires a key AND it requires a passphrase to utilize the key.

I never know what machine I'm going to need to access Location A box from. It's only accessible from the LAN, so it could be one of many Macs, it could be one of a few Windows boxes, it could be my Chromebook. So it's easier just to use username/password. It's a security vs convenience tradeoff.

Here's the code I'm using, with some redactions (modified text in bold):
crontab -l

Code:

# rsync stuff, added 2016-04-20
01 00 * * * /opt/scripts/rsync-script_no-root.sh

Code:

user@LocationA:~$ cat /opt/scripts/rsync-script_no-root.sh
#!/bin/bash


rsync -vah --del --progress --log-file=/opt/log/rsync-script_line1_`date +%F_%H%M%S`.log -e "ssh -p 22 -i /home/user/.ssh/id_rsa" --exclude 'Path/file-to-ignore.ext' user@LocationB.fqdn:/mnt/path/dir_one/ /home/user/data-from_LocationB/path-to_dir-one/

rsync -vah --del --progress --log-file=/opt/log/rsync-script_line2_`date +%F_%H%M%S`.log -e "ssh -p 22 -i /home/user/.ssh/id_rsa" user@LocationB.fqdn:/mnt/path/dir_two/ /home/user/data-from_LocationB/path-to_dir-two/




#EOF

IsaacKuo · 08-22-2016, 02:25 PM

Hmm... no other responses since yesterday morning? I'm afraid I am personally not an expert in using rsync the "proper" way, but I can help you figure out how to do it "my" way.

I started off using rsync strictly on a local lan level, using it basically as a more sophisticated alternative to cp (only making incremental changes rather than deleting the entire backup folder and copying over everything). This was with a simple nfs file server; my backup script was run on an nfs client workstation.

My variants for off site backups still use rsync in this naive way, I just include commands to mount and unmount an sshfs file share from my main server. So my script file would look something like:

Code:

sshfs HOMESERVER:/home/USER ~/remoteHome
rsync -vax --delete ~/remoteHome/Documents ~/backups
sleep 2
fusermount -u ~/remoteHome

HOMESERVER could be defined in /etc/hosts to provide a convenient place to define its IP address. I don't think it's necessary to specify the use of ~/.ssh/id_rsa because it will use that (private key) file by default. But of course, it's necessary to set up the private key and the public key to log in. Which you've already done since it works when running manually.

This isn't a sophisticated way of doing things, but it is a way of doing it. The script has to be run as the user with access to ~/.ssh/id_rsa, of course.

Umm...could that be your problem? What user is the cron job running as? This user will need read access to /home/USER/.ssh/id_rsa. Default is that no one but the owning user has read access to that file. Root also has access anyway because root is root.

computer_freak_8 · 08-22-2016, 03:55 PM

Quote:

Originally Posted by IsaacKuo

The script has to be run as the user with access to ~/.ssh/id_rsa, of course.

Umm...could that be your problem? What user is the cron job running as? This user will need read access to /home/USER/.ssh/id_rsa. Default is that no one but the owning user has read access to that file. Root also has access anyway because root is root.

The script is in root's crontab. Works when manually running as myself or as root. Doesn't work in the cron job.

Pearlseattle · 08-22-2016, 04:53 PM

If you say that everything works perfectly when run manually: maybe the crontab doesn't automatically set the user's home directory or whatever other globals that are set when you login manually into a shell? => What about specifying explicitly in the crontab command (in the ssh-part) the directory and/or files and/or username that it's supposed to use to get private & public key?

jailbait · 08-22-2016, 05:05 PM

Quote:

Originally Posted by computer_freak_8

The script is in root's crontab. Works when manually running as myself or as root. Doesn't work in the cron job.

The crontab PATH is much smaller than the typical PATH for root or a user. In the script try giving the full path to every program that you execute. For example instead of using the command:

rsync

Use the command:

/usr/bin/rsync

You can find out the full path name for a program by using the which command:

which rsync

---------------------
Steve Stites

computer_freak_8 · 09-28-2016, 09:46 AM

Quote:

Originally Posted by Pearlseattle

If you say that everything works perfectly when run manually: maybe the crontab doesn't automatically set the user's home directory or whatever other globals that are set when you login manually into a shell? => What about specifying explicitly in the crontab command (in the ssh-part) the directory and/or files and/or username that it's supposed to use to get private & public key?

In the posted command line, I specify the full path to the private key.

Quote:

Originally Posted by jailbait

The crontab PATH is much smaller than the typical PATH for root or a user. In the script try giving the full path to every program that you execute. For example instead of using the command:

rsync

Use the command:

/usr/bin/rsync

You both mentioned this part... so you're saying something more like this?

Code:

/usr/bin/rsync -vah --del --progress --log-file=/opt/log/rsync-script_line1_`date +%F_%H%M%S`.log -e "/usr/bin/ssh -p 22 -i /home/user/.ssh/id_rsa" --exclude 'Path/file-to-ignore.ext' user@LocationB.fqdn:/mnt/path/dir_one/ /home/user/data-from_LocationB/path-to_dir-one/

Note: leaving "date" command alone, as I know for sure it works, since it logs the date and time, whether or not the script actually copies anything.

chrism01 · 09-29-2016, 11:46 PM

Yep; in cron just assume you have to specify the complete path to all progs and files (bash built-ins excluded).
OR, first src a file that defines a better PATH and possibly cd's into the correct start dir.
Basically, assume nothing

If you google it, you'll find millions of matches for that exact issue.

HTH

c0wb0y · 09-30-2016, 01:00 AM

I tidied it up a bit. No need to specify ssh port nor its key as they seem to be the default anyway. I added 'z' to make transfer a bit faster. Assuming that the source machine can ssh without asking for ssh paraphrase.

Code:

rsync -avze ssh --log-file=/opt/log/rsync-script_line1_`date +%F_%H%M%S`.log --exclude=Path/file-to-ignore.ext user@LocationB.fqdn:/mnt/path/dir_one/ /home/user/data-from_LocationB/path-to_dir-one/

sundialsvcs · 09-30-2016, 08:19 AM

I would definitely recommend using "encrypted keys," not passwords of any sort.

Encrypted keys normally involves the use of a background "SSH key agent" (man ssh-agent) which might need to be started on some intermediate host.

An encrypted key is much stronger than any password because it is "a key," therefore non-forgeable, yet it requires a password in order to use the key. The password that's used to decrypt the key is never presented to the remote system and is not what the remote demands to see.

computer_freak_8 · 10-07-2016, 07:17 PM

This is still not working, even after specifying the full path to the rsync and ssh binaries.

Also of note:

I'm using my user's crontab, not root's crontab. I'd read somewhere that using root might break it because the key doesn't match the username.
I'm changing it tonight. Going to try re-running as root crontab, but with the full path to the binaries.

Quote:

Originally Posted by c0wb0y

I tidied it up a bit. No need to specify ssh port nor its key as they seem to be the default anyway.

Port is actually different. I just sanitized the line. SSH key is in the default spot for the user, though.

Quote:

Originally Posted by sundialsvcs

I would definitely recommend using "encrypted keys," not passwords of any sort.

...is that not what I'm doing? See the -i parameter I'm using in my command line.

computer_freak_8 · 10-09-2016, 09:21 AM

It works!!

Specifying the full path of the binaries, combined with making sure the entry is under root's crontab, seems to have done the trick.

Thank you!