Rsync over SSH incremental backups: test existence of a remote directory?

kikinovak · 12-14-2015, 02:51 AM

Hi,

I've spent the weekend writing a script for automated backups with snapshot rotation using rsync over SSH. I got the initial idea from a chapter in O'Reilly's "Linux Server Hacks", which itself is an adaptation from this great online tutorial. My script works differently and doesn't split the hourly snapshots and the rotation, it's more of a KISS approach.

Here's my first draft:

https://github.com/kikinovak/microli.../lan/backup.sh

This first version already works on my local server, and now I'm trying to work out some details.

I want to test for the existence of a remote directory, and if this remote directory exists (on the client), a portion of the local script (on the server) will be executed. So far, here's what I got in a little test script.

Code:

ssh -o BatchMode=yes -o ConnectTimeout=5 root@alphamule [ -d /path/to/remote/dir ] ; echo $?

(Note: SSH key authentication is configured and works.)

This line returns 0 if the remote directory exists, and 1 if it doesn't. Now I wonder if I'm doing things in a uselessly complicated manner, or if there is a more simple way to do it.

Any suggestions?

Niki

berndbausch · 12-14-2015, 03:37 AM

Quote:

Originally Posted by kikinovak

Code:

ssh -o BatchMode=yes -o ConnectTimeout=5 root@alphamule [ -d /path/to/remote/dir ] ; echo $?

This line returns 0 if the remote directory exists, and 1 if it doesn't. Now I wonder if I'm doing things in a uselessly complicated manner, or if there is a more simple way to do it.

Not sure why you need to simplify this further

What I would perhaps do is

Code:

if ssh -o BatchMode=yes -o ConnectTimeout=5 root@alphamule [ -d /path/to/remote/dir ]
then
    run some local script
fi

This will run the local script if both the ssh itself succeeds and the remote dir exists.

EDIT: I am a bit confused about client vs server here but hope I understood your question.

kikinovak · 12-14-2015, 05:13 AM

Quote:

Originally Posted by berndbausch

EDIT: I am a bit confused about client vs server here but hope I understood your question.

First of all, thanks for your suggestion. Let me clarify this a bit.

The script backup.sh is run on the server. That's the machine where all the backups are stored. It connects to all clients on the network, one after another, and does a backup of their data.

I'm not in my office, so I can't test it right now. I'll check it out this evening.

Cheers,

Niki

Gerard Lally · 12-14-2015, 07:26 AM

Quote:

Originally Posted by kikinovak

Now I wonder if I'm doing things in a uselessly complicated manner ...

Yes you are. Why re-invent the wheel? rsnapshot is easy to set up and reliable. It's also easy for someone else to understand and maintain, something you can't necessarily say about a custom script. Since it's backups you're dealing with, you should play on the safe side and use tools already there and widely used - rsnapshot, or dar.

kikinovak · 12-14-2015, 11:14 AM

Quote:

Originally Posted by berndbausch

What I would perhaps do is

Code:

if ssh -o BatchMode=yes -o ConnectTimeout=5 root@alphamule [ -d /path/to/remote/dir ]
then
    run some local script
fi

This will run the local script if both the ssh itself succeeds and the remote dir exists.

Worked like a charm. Thanks!

kikinovak · 12-15-2015, 06:08 AM

I've fleshed out the script, along with some explanations. Here's the new version:

https://github.com/kikinovak/microli.../lan/backup.sh

One problem I had to solve was backing up databases. As far as I know, the most simple way to do this is running a separate SQL dump on the client machines. Here's a little script I wrote for that:

https://github.com/kikinovak/microli...sql/sqldump.sh

Note: I know it's simple to dump all databases into one big SQL file. Except I prefer having my databases separate.

Feel free to give this a spin and report back.

Cheers,

Niki

Edit: here's what this looks like. See screenshot.