LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (http://www.linuxquestions.org/questions/slackware-14/)
-   -   Does this command seem safe? (http://www.linuxquestions.org/questions/slackware-14/does-this-command-seem-safe-4175425310/)

damgar 09-02-2012 05:04 PM

Does this command seem safe?
 
I have a 200GB directory I want to copy to a new machine over my LAN. It's a slow process. I found this command:
Code:

ssh user@source.server ‘tar -cz -C /source/path/ *’ | tar -zxv
that is supposed to do the copy as a single stream rather than one file at a time. My question since I'm not a tar expert is does te command look safe? It is meant to be run from the destination directory.

turtleli 09-02-2012 05:48 PM

I don't really know, but have you tried:
Code:

scp -C -r user@source.server:/remote/path /local/path
The -C means compression. I use this command to copy large directories from the HPC I have access to.

EDIT: should probably say this still transfers files one by one.

GazL 09-02-2012 06:12 PM

I'd be wary of the wildcard, especially so given the use of -C on the tar as it will probably be expanded in the home directory of user before the tar changes directory.

Using '.' will be less prone to errors and will also ensure that you don't miss any hidden files which the wildcard can sometimes miss.

e.g.
Code:

ssh user@source.server ‘tar -C /source/path -cz .' | tar -C /dest/path -xzv

damgar 09-02-2012 07:53 PM

I tested the command I posted on a small directory that has one single subdirectory (that has several more layers of subdirectories as well) and bunch of individual files. All of the individual files made the copy, but the subdirectory and it's contents did not. It did copy 2 files very fast though! lol Looking online it looks like tar should automatically be recursive and keep the subdirectory structures? I'm not sure exactly what went wrong here.

I went ahead with rsync + ssh, and 2.5 hours into the copy I can say two things for sure: I should have picked up that gigabit switch that I forgot about yesterday, and I should maybe have researched this a little further in advance!

damgar 09-02-2012 07:55 PM

Quote:

Originally Posted by turtleli (Post 4770887)
I don't really know, but have you tried:
Code:

scp -C -r user@source.server:/remote/path /local/path
The -C means compression. I use this command to copy large directories from the HPC I have access to.

EDIT: should probably say this still transfers files one by one.

I tried various incantations of scp but they all bogged down with the first 5GB file. That's kind of the problem with this particular directory. It's huge, and it has a large number of very large files combine with thousands of very small files. When you hit one of the large ones things just get SLOW! LOL

saulgoode 09-02-2012 09:06 PM

Quote:

Originally Posted by damgar (Post 4770864)
Code:

ssh user@source.server ‘tar -cz -C /source/path/ *’ | tar -zxv

I recall having problems with this once when using it to transfer a WINE directory which contained some circular links (the target just grew and grew until it ran out of space).

Other than that, it would seem pretty safe; perhaps add the -j option to make sure you don't archive non-local directories.

lopid 09-05-2012 07:04 PM

Code:

rsync -avP -e 'ssh -o ControlPersist=yes' user@source.server:/source/path .
Since it's a LAN, I wouldn't bother with compression.


All times are GMT -5. The time now is 03:27 PM.