Help answer threads with 0 replies.
Go Back > Forums > Linux Forums > Linux - Newbie
User Name
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!


Closed Thread
  Search this Thread
Old 10-20-2010, 06:47 PM   #1
Registered: Dec 2003
Posts: 78

Rep: Reputation: 1
Transfer large number of files host to host


I have two servers, one has an empty / and the other has a subdirectory with a large number (4 gig) with many, many files. I need a way to transfer the files en masse from the server with the large number of files to the one that is essentially blank.

I don't have space on the used host to simply gzip all the files. I've googled this and see that there may be some combination of tar and/or gzip that will let me do this with some sort of redirection.

I really need and example line of how this can be accomplished. If my explanation seems rather sparse, I can supply more details.


Old 10-20-2010, 07:03 PM   #2
Senior Member
Registered: Sep 2009
Location: Orange County, CA
Distribution: Kubuntu x64, Raspbian, CentOS
Posts: 1,850
Blog Entries: 36

Rep: Reputation: 455Reputation: 455Reputation: 455Reputation: 455Reputation: 455
Boot to a live CD on the empty host. Set up an NFS server on the populated host sharing it's whole drive. Then mount the empty host drive on the live CD. Then mount the NFS drive on the live CD. Then you can dd the NFS drive to the unpopulated hosts drive. You might want to zero the free space on the populated host drive with dd.

The catch is the empty drive must be equal or larger in size to the populated host. You can expand the partition later but just in case of data corruption do not delete the old host drive which is populated until you're sure you've got all your data. That solution is essentially OS independent because dd will just grab and write blobs of data from disks.

Last edited by sag47; 10-20-2010 at 07:05 PM.
Old 10-20-2010, 07:05 PM   #3
LQ Guru
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Assuming you have sshd running on at least one of the machines you can use scp or rsync (over ssh)

Eg using rsync.
user@sourcehost% rsync -auv -e ssh /path/to/dir/ desthost:/path/to/some/dir/
The above will work if you have sshd running on the destination machine. If you only have sshd running on the source machine you would just do something like the following

user@desthost% rsync -auv -e ssh sourcehost:/path/to/dir/ /path/to/some/dir/
You can read the rsync man page for details.


Old 10-20-2010, 10:01 PM   #4
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
Please post your thread in only one forum. Posting a single thread in the most relevant forum will make it easier for members to help you and will keep the discussion in one place. This thread is being closed because it is a duplicate.

Closed Thread

files, tar, transfer

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
configuring host.allow & host.deny files gardenair Linux - Newbie 7 05-11-2010 07:27 PM
ext3 performance -- very large number of files, large filesystems, etc. td3201 Linux - Server 5 11-25-2008 10:28 AM
How Do I Transfer Files From Ubuntu (Host) and XP (Guest) ?? taurusx5 Linux - Software 2 04-11-2008 02:41 AM
Required to achieve host to host communication via USB file transfer cable may1284 Linux - General 1 03-14-2007 05:02 PM
Kmail ' Connection to host broken ' with large files from Gmail vangelis Linux - Software 0 10-22-2005 08:11 AM > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 10:31 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration