hi all,
I need to backup a large number of files (nearly 100) from a server. The total size of the files will be around 150 MB.
Which will be a better method? Rsyncing the files individually (or) take a zip of the files in the source and then rsync it.
Will there be any performance issues, when large number of files are rsynced, because i think during the initial handshake itself, the list of files are first exchanged and then the actual files are transferred.
Please note that this is not an incremental backup. Every time, i need to get all those files to a new location.

Any help is appreciated.

TIA.

I (personally) would just gzip them (or tar and gzip) and just scp them to where I wanted them. Basically you can copy them to where you want, but compressing them would make the copy faster.

hi korozion,
Probably scpying after taking gzip is a better option for my application. But out of curiosity, does rsync perform better if it has to transfer multiple small files or a single large file.
I want to add that, the zip will not reduce the size, because the files i am talking about are already zip files!!

Are you going to be doing this through SSH or just through rsync by itself? The reason for that question is, if you do it through SSH then you have encryption and decryption going on, which will slow down the transfer some. This may not be an issue if you are on the same network as the machine you are pulling the file(s) off of.

Use scp if you send just one file. rsync is good for multiple files or folders. Both I think have compression option when sending but if you use scp and are transferring a already compressed file, it probably won't make any difference when scp'ing it over.

When you say "new location each time", do you really mean a new location each instance, or the same location?

The whole point to rsync is that it only backs up the portion the file that is required. This is not the same as an incremental backup to tape where you need several tapes for complete restore. Rsync simply avoids copying a file that already exists and is identical. After the initial copy, subsequent rsync's are much faster and require far less bandwidth than copying all the files each time. You can explictly invoke ssh as the transport for rsync, although I think the current version defaults to ssh.

I use an old 100 MHZ Dell with an 80 GB IDE drive to backup my server every 6 minutes. I also maintain a weekly copy and monthly copy. This same Dell also backups the user directories of 3 other Linux boxes every 15 minutes.

Don't keep copying the same data over and over again--even compressed.

Don't do it.

Jerre cope,
I thought of rsyncing the files, burn it to a CD and remove the files. Then start the rsync process all over once again. Thats wat i meant by new location each time.
As you point out, rsync works best for incremental backups. But the files i am planning to backup are log files, which get log rotated as ".gz" files. I retain 10 such files. I am planning to take these backups every month. So there will always (well almost) be a different set of log files each month. So it doesnt really matter, if i use rsync or scp. There are 8-9 such log files. So i will have 80-90 compressed files for backup.
So my problem boils down to whether to get those 80-90 files through rsync or zip those files and rsync a single file. I am asking this out of curiosity I have already decided to transfer the 80-90 files, becoz zipping them will again take an equal amount of space in the server.

Loosecannon,
The machines are in the same network. I am planning to use rsync over ssh.

I don't think the difference in time would be humanly discernable.

If you have user traffic on the network 24/7 where the slight delay of copying a large file would be noticed, then I would suggust using rsync daily ( without using the --delete option) so that you pull less across the network at a time. When you get close to 600 MB, burn the CD and remove all but the most recent files (so you don't rsync them again.)