copy one large directory or multiple cp's on the sub directories
Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I don't know the answer, but my gut feel is that it's unlikely to make much if any difference, because the constraint on the process will be (i) the disk drive, and (ii) the IO bus. I would fully imagine that one of these will be the bottleneck, and splitting the process into multiple processes won't speed things up. More likely it will slow things down.
Basically, throwing CPU or other power at an IO bound problem probably won't help.
If for some reason throwing CPU power at the problem could relieve this then the answer changes (for example compressing data to copy between systems).
WHat would be interesting would be for you to try a sample and post the results.
The issue is this isn't your average copy, its nearly 750GB of data. Running a cp -r over night is what the cron runs every single day. The whole process (it's a backup system) take over 24 hours, and for some strange reason stops the backup in it's tracks. So I'm trying to make each piece of the puzzle more efficient.
Does anyone know a good benchmarking software so I can test to see which is faster? I have munin right now constantly monitoring the system, but I don't feel it's good for a benchmark of which method would be faster.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.