Hey all,

I need to transfer a huge amount of data between 2 sites.
Both side of the transfert run Linux systems.
Data must be protected in confidentiality.

However, the upload capability is quite low on each sites (connected through ADSL).

At the moment, i use SFTP to transfert data between each sites.
However, i wondered if having a VPN between the two sites, and performing a classic FTP transfert would be better on a performance point of view.

Any idea ?

Thanks

If its just file transfer you want, I would keep it simple and stick with ssh based methods. I would imagine the overhead is less.. ?

I would recommend rsync for transferring large amounts of data, it has many benefits over the sftp method.

http://rsync.samba.org

zer0x

ssh uses very little overhead once the setup is finalized. vpns typically tunnel traffic, so there is usually 20-40 bytes of overhead per packet.

rsync over ssh is a good option.

@PlatinumX: as you know, both ssh and vpn get you encryption on the wire. As for which will be quicker given your circumstances, I have no idea. You could do some performance testing with a small amount of sample data.

rsync is a good idea if you're going to be doing a one-time large data transfer and then keeping it in sync afterwards.

rsync really only gains on multiple uploads of nearly the same data. If its a one-off, go the easy way with scp.

I can't use rsync because it is not a synchronisation.
It is a transfert of log file to a central point.

Ok, i will stay on SCP.

It might sound obvious, but it's worth mentioning anyway: compress the log file first.

Just to clarify, you 'could' use rsync, because although rsync only txfrs 'differences', the diff between the whole file at src and nothing at target, is indeed the whole file
However, scp would be simpler & quicker (and yes, gzip it first, looking at the compression options - level 9 is highest compression, but slowest to do iirc)

I am 7zip-ing it before transmitting

A nice advantage of VPN is that you can do it in hardware. So, you can set up a router on each end (or use the VPN capabilities that might well be already available on what's already there), and arrange for both of them to exchange everything that must pass between them securely.

As long as you know that this is working correctly, and that nothing will ever be passed "in the clear," then you now have a secure channel that you can "just forget about." The encryption takes place but it is entirely transparent to you.

It is critically important that you use digital certificates (self-generated is fine...) to secure the link: don't rely upon "shared secrets" (passwords). This will allow the two routers not only to provide truly-secure communication with each other, but to identify each other.