SlackwareThis Forum is for the discussion of Slackware Linux.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I have a 200GB directory I want to copy to a new machine over my LAN. It's a slow process. I found this command:
Code:
ssh user@source.server ‘tar -cz -C /source/path/ *’ | tar -zxv
that is supposed to do the copy as a single stream rather than one file at a time. My question since I'm not a tar expert is does te command look safe? It is meant to be run from the destination directory.
Last edited by unSpawn; 09-02-2012 at 05:35 PM.
Reason: //I R fixoring vBB tagsesess
I'd be wary of the wildcard, especially so given the use of -C on the tar as it will probably be expanded in the home directory of user before the tar changes directory.
Using '.' will be less prone to errors and will also ensure that you don't miss any hidden files which the wildcard can sometimes miss.
I tested the command I posted on a small directory that has one single subdirectory (that has several more layers of subdirectories as well) and bunch of individual files. All of the individual files made the copy, but the subdirectory and it's contents did not. It did copy 2 files very fast though! lol Looking online it looks like tar should automatically be recursive and keep the subdirectory structures? I'm not sure exactly what went wrong here.
I went ahead with rsync + ssh, and 2.5 hours into the copy I can say two things for sure: I should have picked up that gigabit switch that I forgot about yesterday, and I should maybe have researched this a little further in advance!
The -C means compression. I use this command to copy large directories from the HPC I have access to.
EDIT: should probably say this still transfers files one by one.
I tried various incantations of scp but they all bogged down with the first 5GB file. That's kind of the problem with this particular directory. It's huge, and it has a large number of very large files combine with thousands of very small files. When you hit one of the large ones things just get SLOW! LOL
ssh user@source.server ‘tar -cz -C /source/path/ *’ | tar -zxv
I recall having problems with this once when using it to transfer a WINE directory which contained some circular links (the target just grew and grew until it ran out of space).
Other than that, it would seem pretty safe; perhaps add the -j option to make sure you don't archive non-local directories.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.