LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 09-18-2007, 08:35 AM   #1
keimdf
LQ Newbie
 
Registered: Dec 2004
Location: Wabasha, MN
Posts: 24

Rep: Reputation: 15
Move files from old system to new system


I am creating a new Virtual Linux system, or should say that it is done, but I need to move the information from the old system to the new system and I am looking for most expediant way to do this. These are both development machines that I am moving from/to. I need to be able to shut one down at the end of a days business and bring the new one up the next morning ready to go. Here are my systems.

Old system -> Physical Box RedHat Enterprise 3 (ES), /u is pointing to a separate hard device on the system which has been set up as a filesystem. I have this filesysem shared through Samba and is available to everybody.

New System -> Virtual Machine RedHat Enterprise 3 (ES), /u is pointing to a Virtual Disk on a RAID.

Options:
1. I can mount the Samba share on the new box and "cp -pr <old> <new>.

the problem with this is the mount command sets the permissions for all files/directories on the mount.

2. I can tar the files/directories on the old system. copy/ftp/whatever the file over to the new system and untar it into the new system. This would probably restore the files to their orignal owner/group and permissions. The problem with this is it is really slow. I started a tar last night at 10:00 pm and it is still running this morning.

Are there any other options that might one) be faster two) keep permissions/ownership the same?

I have STF for answers, but nothing that I have found stands out with an answer. Maybe I am asking the wrong search keywords. I have to believe that someone has run into this before.
 
Old 09-18-2007, 08:43 AM   #2
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,399
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
I think rsync might do the job for you.
--- rod.
 
Old 09-18-2007, 09:28 AM   #3
keimdf
LQ Newbie
 
Registered: Dec 2004
Location: Wabasha, MN
Posts: 24

Original Poster
Rep: Reputation: 15
I am newbie to Linux commands.

what does rsync require for access of the remote machine.

username/password??

I tried this

rsync -avrn <computername>:/u/logs .

and got this...

<computername>: Connection refused
rsync: connection unexpectedly closed (0 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(165)
 
Old 09-18-2007, 10:02 AM   #4
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,399
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
Check out the '-e' switch. If you are going to use this frequently, it might be convenient to set up ssh keys for passwordless ssh logins. You, as the rsync user, must have an account on the peer host, which can be accessed via ssh or rsh.

--- rod.
 
Old 09-18-2007, 10:19 AM   #5
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682
If you want to transfer an entire filesystem or directory of files, you could try using tar this way to transfer the files:
tar -C sourcedir/ -cf - . | ( tar -C <mounted-target-dir> -xf - )

There may be more options you need to preserve permissions, and deal with links properly. Keep in mind that the smbfs has around a 2 GB filesize limit. You might consider using NFS instead. The permissions, ownerships should transfer properly then, although, you may need to use "star" if you have to deal with ACLs and Security contexts.
 
Old 09-18-2007, 10:28 AM   #6
keimdf
LQ Newbie
 
Registered: Dec 2004
Location: Wabasha, MN
Posts: 24

Original Poster
Rep: Reputation: 15
thanks,

actually, while I was waiting for a response here, I read up more on --rsh and did put it in the command.

rsync -avr --rsh="ssh -l <username>" <username>@anakin:/u/send/ send

About the time you responded, I was able to get it bring files across from the remote host. However, I am still having a few problems.

I want the files to preserve the owner/group/permissions. It appears that it is doing the permissions just fine, but not the owner/group.

also, some of the files are root, so I changed the command to

rsync -avr --rsh="ssh -l <username>" root@anakin:/u/send/ send

this brought over files that the first command could not because it was denied access. But then it created the files on the new system my name and group.

so I changed it again ...

rsync -avr --rsh="ssh -l root" root@anakin:/u/send/ send

this brought the files over the same way. My login name and group. The permissions were the ok though.

I added the options ogt to the command above and got the same results.

this is getting closer, but not there yet.
 
Old 09-18-2007, 11:32 AM   #7
keimdf
LQ Newbie
 
Registered: Dec 2004
Location: Wabasha, MN
Posts: 24

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by jschiwal View Post
If you want to transfer an entire filesystem or directory of files, you could try using tar this way to transfer the files:
tar -C sourcedir/ -cf - . | ( tar -C <mounted-target-dir> -xf - )

There may be more options you need to preserve permissions, and deal with links properly. Keep in mind that the smbfs has around a 2 GB filesize limit. You might consider using NFS instead. The permissions, ownerships should transfer properly then, although, you may need to use "star" if you have to deal with ACLs and Security contexts.
I not sure how to set up the directory for nfs on the source system. You think that this will allow the transfer with owner/group and permissions?

I tried this...

mount -t nfs <host>:Programs /u3

and it came up with this.

mount: RPC: Program not registered

UPDATE: I guess I am getting better at reading forums and manuals. I have been able to create a NFS share and get it mounted on the new system.

using the command ...

cp -prv /u3/* .

... I have been able to start moving the files and they are maintain their original owner/group and permissions. And it seems rather quick!

Unless I have problems, thanks for all of the helpp

Last edited by keimdf; 09-18-2007 at 01:55 PM.
 
Old 09-19-2007, 12:50 AM   #8
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682
rsync may be a better choice if it also deletes files in the target that were deleted in the source. You might consider using tar with the --incremental --listed-incremental=SNAPSHOT-FILE to produce a backup of the source directory and then restore to the destination with the --listed-incremental=SNAPSHOT-FILE option. According to the info manual, this will delete files in the target that were deleted in source since the last incremental backup. This would also kill two birds with one stone. Replication and Backup. I don't know if you can use snapshot files when using tar to copy files from a source to a target machine. If you can, you could just copy updated files and delete obsolete files in the target machine.

-- update --

Code:
tar --incremental --listed-incremental=etap.ssf -cf -  etap | (tar --incremental --listed-incremental=/mnt/hpmedia/etap/etap.ssf -C /media/lbigdisk/etap/ -xf - )
I touched a file and then ran the above command to transfer it over. Testing with "find /media/lbigdisk/etap -cmin -2", the file I touched did replace the copy in the destination.

However moving a file out of the source directory, and rerunning the incremental backup, the file in the destination wasn't deleted. If you were to restore from a real incremental backup, it should be.

If you use NFS, you either want to map uid and gid numbers (such as using NIS), or keep them the same across all your hosts. This could have been the problem with rsync. It is the actual number that is copied. If the same user has a different UID number, that could be the user ownership problems you were seeing. As an example, Red Hat and Mandriva start normal UIDs at 500 while SuSE starts at 1000. To be honest, I used the SuSE NFS server and NFS client config modules in YaST2. They allowed me to search the network for NFS shares in the dialog and configured /etc/exports and /etc/fstab for me.

Last edited by jschiwal; 09-23-2007 at 05:38 AM. Reason: Added UID info.
 
Old 09-26-2007, 11:18 PM   #9
keimdf
LQ Newbie
 
Registered: Dec 2004
Location: Wabasha, MN
Posts: 24

Original Poster
Rep: Reputation: 15
I didn't have that many files to get across and I just couldn't get rsync to work right, so I used the cp command and I am done, I will have to read up more on rsync and try it sometime when I don't have a lot of work to do.

Thanks for your help
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how to transfer files from one system to another system ramvoip Programming 10 04-05-2007 03:40 PM
LXer: Move an entire file system on a live Unix system LXer Syndicated Linux News 0 07-08-2006 05:33 PM
Can I find links in the system that point to certain files throughout the system? HGeneAnthony Linux - Newbie 3 02-18-2005 08:28 AM
How to copy files from A computer of NT System to The other one of Unix system? whepin Linux - Newbie 5 04-06-2003 10:50 AM
How do I move a system from one HD to another? afshin Linux - Hardware 3 03-25-2003 07:42 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 08:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration