Feedback Requested on a planned multiple filesystem copy
 

Hi there --

I am going to be involved with a massive filesystem copy from a local to a remote server in the next couple of weeks. There are ten filesystems in involved in this process. All, except one, are one hundred gigabytes in size, with the remaining one at twenty gigabytes.

The cp command with the -pr options will be used to copy the directories to their new location. A speed test, involving ten directories, was done to determine the average amount of time it would take to complete the process. The ten directories used in this test ranged in size from 2.3 gigabytes to 4.3 gigabytes. The results indicated the average amount time to complete the copy was around one minute and thirty seconds.

The question I have is the following: Is it better to interactively go to each filesystem and run the cp -pr command there, or should I write a script that will automatically go to each filesystem and run the copy?

Its just a thought but have you considered that by using cp you could have an issue if any of the copies fail. A possible alternative would be to consider the program rsync with the options --whole-file (check the manual page for the best options to suit your environment).

At least using something like rsync your copy process is restartable without having to copy the whole of the directory again.

On the script part, I would suggest that your script is less likely to induce errors (typo's etc...) during your copy process so long as you carefully check it. Possibly putting a sleep with a few echo's to give you time to abort the next copy should the copy process fail and also getting the script to log the output from the copy to a file so if you have a failure you can see what happened by examining the log file.

P.S. rsync has a -n option that allows you to simulate what it would do without actually doing it.

Regards

Lee