LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices

Reply
 
Search this Thread
Old 09-02-2012, 05:04 PM   #1
damgar
Senior Member
 
Registered: Sep 2009
Location: dallas, tx
Distribution: Slackware - current multilib/gsb Arch
Posts: 1,949
Blog Entries: 8

Rep: Reputation: 201Reputation: 201Reputation: 201
Does this command seem safe?


I have a 200GB directory I want to copy to a new machine over my LAN. It's a slow process. I found this command:
Code:
ssh user@source.server ‘tar -cz -C /source/path/ *’ | tar -zxv
that is supposed to do the copy as a single stream rather than one file at a time. My question since I'm not a tar expert is does te command look safe? It is meant to be run from the destination directory.

Last edited by unSpawn; 09-02-2012 at 06:35 PM. Reason: //I R fixoring vBB tagsesess
 
Old 09-02-2012, 05:48 PM   #2
turtleli
Member
 
Registered: Aug 2012
Location: UK
Distribution: Main - Slackware 14.1, Others - Gentoo, Fedora
Posts: 149

Rep: Reputation: Disabled
I don't really know, but have you tried:
Code:
scp -C -r user@source.server:/remote/path /local/path
The -C means compression. I use this command to copy large directories from the HPC I have access to.

EDIT: should probably say this still transfers files one by one.

Last edited by turtleli; 09-02-2012 at 05:54 PM. Reason: More info
 
Old 09-02-2012, 06:12 PM   #3
GazL
Senior Member
 
Registered: May 2008
Posts: 3,481

Rep: Reputation: 1016Reputation: 1016Reputation: 1016Reputation: 1016Reputation: 1016Reputation: 1016Reputation: 1016Reputation: 1016
I'd be wary of the wildcard, especially so given the use of -C on the tar as it will probably be expanded in the home directory of user before the tar changes directory.

Using '.' will be less prone to errors and will also ensure that you don't miss any hidden files which the wildcard can sometimes miss.

e.g.
Code:
ssh user@source.server ‘tar -C /source/path -cz .' | tar -C /dest/path -xzv

Last edited by GazL; 09-02-2012 at 07:06 PM.
 
Old 09-02-2012, 07:53 PM   #4
damgar
Senior Member
 
Registered: Sep 2009
Location: dallas, tx
Distribution: Slackware - current multilib/gsb Arch
Posts: 1,949
Blog Entries: 8

Original Poster
Rep: Reputation: 201Reputation: 201Reputation: 201
I tested the command I posted on a small directory that has one single subdirectory (that has several more layers of subdirectories as well) and bunch of individual files. All of the individual files made the copy, but the subdirectory and it's contents did not. It did copy 2 files very fast though! lol Looking online it looks like tar should automatically be recursive and keep the subdirectory structures? I'm not sure exactly what went wrong here.

I went ahead with rsync + ssh, and 2.5 hours into the copy I can say two things for sure: I should have picked up that gigabit switch that I forgot about yesterday, and I should maybe have researched this a little further in advance!
 
Old 09-02-2012, 07:55 PM   #5
damgar
Senior Member
 
Registered: Sep 2009
Location: dallas, tx
Distribution: Slackware - current multilib/gsb Arch
Posts: 1,949
Blog Entries: 8

Original Poster
Rep: Reputation: 201Reputation: 201Reputation: 201
Quote:
Originally Posted by turtleli View Post
I don't really know, but have you tried:
Code:
scp -C -r user@source.server:/remote/path /local/path
The -C means compression. I use this command to copy large directories from the HPC I have access to.

EDIT: should probably say this still transfers files one by one.
I tried various incantations of scp but they all bogged down with the first 5GB file. That's kind of the problem with this particular directory. It's huge, and it has a large number of very large files combine with thousands of very small files. When you hit one of the large ones things just get SLOW! LOL

Last edited by damgar; 09-02-2012 at 08:22 PM.
 
Old 09-02-2012, 09:06 PM   #6
saulgoode
Member
 
Registered: May 2007
Distribution: Slackware
Posts: 252

Rep: Reputation: 120Reputation: 120
Quote:
Originally Posted by damgar View Post
Code:
ssh user@source.server ‘tar -cz -C /source/path/ *’ | tar -zxv
I recall having problems with this once when using it to transfer a WINE directory which contained some circular links (the target just grew and grew until it ran out of space).

Other than that, it would seem pretty safe; perhaps add the -j option to make sure you don't archive non-local directories.
 
Old 09-05-2012, 07:04 PM   #7
lopid
Member
 
Registered: Jun 2008
Posts: 107

Rep: Reputation: Disabled
Code:
rsync -avP -e 'ssh -o ControlPersist=yes' user@source.server:/source/path .
Since it's a LAN, I wouldn't bother with compression.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Is It safe,..... udazzleme Linux - General 3 01-23-2006 11:46 PM
Is it safe ? Alex_jacobson Solaris / OpenSolaris 2 03-02-2005 02:24 PM
How safe am I? bad_andy Linux - Security 2 01-29-2005 02:47 PM
Is this safe enough? koyi Linux - Security 4 08-18-2004 11:50 PM
is it safe... violaten Linux - Hardware 1 08-03-2003 03:56 AM


All times are GMT -5. The time now is 12:49 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration