Hello--can someone tell me how to RSYNC bteween two servers with IP addresses, username and passwords while removing the files at the senders end? Do I need to login as root on both servers? I appreciate your input.

Hi:

These links might help.
http://www.tomsitpro.com/articles/li...t,2-777-4.html
http://www.tecmint.com/sync-two-apac...s-using-rsync/
http://www.sysadminshare.com/2012/05...tween-two.html

I don't know if you should be logged into both servers as root or not:-

Hi,

you need to learn to use rm, rsync and (indirectly ssh).

Eg to rysnc from server1 to server2 (with IP address 192.168.1.234), get a shell on server1, then run rsync. Eg.

Then once you are completely sure that you copied the "somedir" correctly you can remove it and its contents with 'rm -r'.

Whether or not you need root depends on the permissions of the files in question.

Evo2.

Yes, thank you. It is slowing coming back to me now. But in your code, I believe that there are some missing parts. FYI: I am trying to write a crontab to do this automatically everyday. So, I am wondering about the following modifiers:

1. Rsync has a --remove-system-file modifier that I believe will remove files from Server 1, no?

2. I am wondering about how ssh will know the password of user2 in your example. Reading one of the links, I believe the ssh-keygen command enables this, no?

How would the code combine to include both these command above?

Hi,

your original post never mentioned anything about automating this. If you want this to be automated (eg with cron) you should probably use public key authentication (with the key unprotected by a passphrase).
You can use ssh-keygen and ssh-copy-id to set it up.

I'm not familiar with the --remove-system-file option and it is not documented in the rsync man page (on my Debian 8.0/ system).

Evo2.

Sorry, that should be --remove-source-files for Rsync. Where would I put that in your code? I am little unclear on whether that would retain older files on the destination folder (Server 2) that were previously deleted on the source folder (Server 1)?

In the links provided, the ssh-genkey is implemented separate from the Rsync code. But I remember including that in the code itself. I am setting up several crontabs to rsync between different external computers (Server 1 and Server 2, Server 1 and Server 3). I am a little worried about or unclear setting up multiple ssh-keys. I believe that the ssh-key could be implemented in the code. Is that correct?

I have never heard of ssh-copy-id. How is that different than ssh-genkey?

ssh-keygen & ssh-copy-id http://www.cyberciti.biz/faq/install...y-remote-host/

Hi,
It is just an commandline option to rsync, so put it with the other options. Are you experienced in using the commandline? If not I strongly suggest you spend some time on the basics before trying to implement this. Eg learn and use cd, ls, pwd, mkdir, cp, mv, rm, cat, less, etc. Then learn about basic shell scripts. There are plenty of tutorials online.

Is the documentation unclear? What does it say?


I'm assuming that when you write "ssh-genkey" and "ssh-key" you actually mean "ssh-keygen". But what do you mean by "implemented in the code"? Each client should have its own private key with the corresponding public key installed on the server (here client means the machine on which rsync is run, and server is the other machine).

It is a helpful little script for installing public keys. It exists primarily so that people can avoid common mistakes when installing keys (eg wrong location, wrong permissions, overwriting other keys, etc). Look it up.

Evo2.

It would retain them. To investigate questions like this, try rsync between two folders that you don't need. The folders don't have to be on different servers to find out how most rsync options work.

Example:
So you see that --remove-source-files removes files from the source after copying them. Since you asked, the option which does not retain older files on the destination folder that were previously deleted on the source folder is --delete. It removes files from the destination if they are not present in the source.

Great Example! Yes, indeed, rsync can be used for directories within a server, too. That was very clear Beryllos.

So far, my code has taken this shape:

From the links that people have posted, I can generate a file from ssh-keygen so that the password of the remoteuser on the remotehost can be read. For some reason, I thought that the ssh-keygen was part of the code; but perhaps I'm mistaken. My only question left is that if I put this into a crontab, which directory does the ssh-keygen file need to located to be read? I understand that this file also has to be copied on the other server, too. But where?

Thanks guys!

Hi,
The "ssh" should be an argument to the -e option. Eg
However it won't be needed if the RSYNC_RSH environment variable is set to ssh.
The command ssh-keygen generates two files:
1. The private key (eg ~/.ssh/id_rsa)
2. The corresponding public key (eg ~/ssh/id_rsa.pub)
The contents of the public key file needs to be put into the ~/.ssh/authorized_keys file (or similar) on the remote server. You can use the ssh-copy-id script to do this.

Evo2.

Thank you, Evo. That was very helpful. I am having one problem and have another question:

1. I am trying to rsync to synology, and the ssh-copy-id cannot write to the disk. Any suggestions?

2. How would I go about setting rsync and sshkey-gen on a third party linux box, where computer 3 accesses files from computer 1 and transfers them to computer 2?

Hi,
What is synology? Exactly what error message are you seeing?

What do you mean by "setting"? Do you mean installing? If so, then by using the package manager for that system. Why does there need to be a 3rd computer in this scenario? Why can't you copy directly from computer 1 to computer 2?

Evo2.

I assume that Synology is in reference to a NAS box.

Did you enable ssh/sftp via its control panel? Did you try copying the keys to a regular user (if any were created) or root? The .ssh directory may not exist or if copying to a regular user, home services is not enabled which allows regular users to access their home directories.

Sorry. By Synology, I mean NAS. (I'm a true newbie.)

Here is my problem: when I do the following command, I get an error:
>rsync -av -e ssh --remove-source-files /home/ftp_user/stage/datafolder root@75.36.201.198::backups/system/MachineName/stage
root@75.36.201.198's password:
Password:
sending incremental file list
datafolder/
rsync: failed to set times on "/system/MachineName/stage/datafolder" (in backups): Operation not permitted (1)
datafolder/filename1.zip
datafolder/filename2.zip
rsync: mkstemp "/system/MachineName/stage/datafolder/.filename1.wv1RXC" (in backups) failed: Permission denied (13)
rsync: mkstemp "/system/MachineName/stage/datafolder/.filename2.RfXual" (in backups) failed: Permission denied (13)
sent 1891165115 bytes received 107 bytes 13753928.89 bytes/sec
total size is 1890933885 speedup is 1.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1070) [sender=3.0.9]


However, when I implement the same command using a directory (testfolder containing the same files) that I made by myself, I get no error.

>rsync -av -e ssh --remove-source-files /home/ftp_user/stage/testfolder root@75.36.201.198::backups/system/MachineName/stage

root@'s password:
Permission denied, please try again.
root@75.36.201.198's password:
Password:
sending incremental file list

sent 33 bytes received 9 bytes 1.95 bytes/sec
total size is 0 speedup is 0.00

Now, I thought there are permission issues as owners and groups of the directories and files contained within them. Previously the owner and group were ftp_user. But changing permissions to root for both owner and group did not solve the problem as illustrated above:
>ls -als
4 drwxrwxr-x 2 root root 4096 May 15 17:35 testfolder
4 drwxr-xr-x 2 root root 4096 May 14 15:05 datafolder

So, why is the folder "testfolder" that I made able to rsync, while the "datafolder" that was made by a shell script (that made the zip files) not able to rync?