Hi all,

I have 4 dedicated servers that need backing up (these are in a datacenter, need backing up nightly, preferably!), my plan was to rsync them with a local server I have setup, however, when using SSH with rsync, the transfer rates start around 25kb/s and quickly dwindle down to around 0.5kb/s! I'm on an 8mb connection and have download speeds of around 800kb/s so it isn't my connection!

Is there an alternative to SSH rsync? Any software I could setup to remote sync them?

Cheers

The problem does not look like it comes from your software. It looks like you have a connection problem. You may want to talk to your ISP.

Hello,

I used rsync also to synchronize files and folders but it didn't quite satisfy me so I searched for other options and found Unison. It does exactly what I want, is easy to set up and maintain. Have a look at it. It's worth checking out.

Kind regards,

Eric

From what I know, unisonfs is especially used to mount a snapshot over a read-only directory.

You cannot go wrong using a simple scheduled bash script that rusn rsync, ssh, tar to do backups. That's how a lot, if not, most Linux admins do native backups.

Another alternative that I like, additional to rsync/ssh is squashfs. It is great because it allows you to create a compressed backup file that can be mounted. This is especially good, since a big compressed backup file can take a long time to decompress.

Are you backing to tape or disk?

Unison is much more than just mounting a read only snapshot. I use it continuously to synchronize bidirectionally different directories between servers. And this over an SSH tunnel with SSL keys as authentication, fully automated.

Kind regards,

Eric

Since you are talking about download speeds of 800kb/s I assume this copying is not on your local LAN. What should your specified upload speed be as seen from your server? You say you are in a data center, and that should mean your UPload to the outside world should be higher. What you get now looks like if you are behind an ADSL router.

jlinkels

I'm not entirely sure of the upload speed from our server. I am indeed behind an ADSL router. We have 4 dedicated servers hosted in a datacenter, I am trying to take nightly backups from those servers and download them to a local server we have in the office so if one of our dedicated servers die, we can bring them backup with a local nightly backup.

That's the plan anyway, I have spoken to a friend of mine who said rsync should not be slow and I should try and tackle why that is so slow.

Thanks for your help guys!

Then I would propose that you do some upload tests from the datacenter to an external server to see if the speed limiting is caused by your connection or something else. Try for example to ftp some fairly large files (10 MB or so) to an external server and see what sustained data rate you get.

jlinkels

If rsync is slow for him, unison will be slow as well, as I believe that is based off of rsync.

We tried rsync to mirror file systems about 10 years ago and found that it goes out into the weeds if you have many files. One of our hosts has 1.6 million files on just one file system. We just again tested rsync (assuming that it would have been fixed after 10 years), and sure enough, it still goes out into the weeds after a few tens of thousands of files. So, we went back to tar and ftp to mirror file systems.

toolz0, hello and welcome to LQ.

Kind of an old post.