LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Server (http://www.linuxquestions.org/questions/linux-server-73/)
-   -   rsync pull on ubuntu 8.10 - noob Q's (http://www.linuxquestions.org/questions/linux-server-73/rsync-pull-on-ubuntu-8-10-noob-qs-730750/)

mrloaf 06-04-2009 06:04 PM

rsync pull on ubuntu 8.10 - noob Q's
 
I'm setting up a box running Ubuntu 8.10 to rsync my FreeBSD server(s). I want to pull data down to the Ubuntu box, then send it all upstream to an offsite data center. But first I have to get rsync to work.

This works, sort of:

Code:

rsync -halvz /home/administrator/.gvfs/sftp\ for\ root\ on\ myserver /local/backups/ --log-file=/tmp/rlog --exclude-from=/home/administrator/rsync-exclude --delete
It runs for a while, then it errors with:
Code:

rsync: writefd_unbuffered failed to write 461 bytes [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (1258 bytes received so far) [sender]
rsync error: error allocating core memory buffers (code 22) at io.c(600) [sender=3.0.5]

And after that error, it won't run anymore - the error keeps happening. If it were just running out of memory, a restart ought to fix it, but doesn't. The box should be adequate - 2.4 GHz dual P4 with 1 GB ram.

Also: I know there must be a better way to connect to the source, but how? I've been mounting the server via SFTP in the GUI (hence the .gvfs path) which is probably not the fastest way. But ssh thru the shell doesn't seem to work.

rsync daemon is NOT running on the source FreeBSD server.

I'll send whatever add'l info people would like to see - thanks in advance!

irishbitte 06-04-2009 07:38 PM

This I'm not getting:
Quote:

...../.gvfs/sftp\ for\ root\ on\ myserver /local/backups/.....
Whats the for\ root\ on\ myserver bit about?

billymayday 06-04-2009 07:42 PM

I assume the remote path is "/home/administrator/.gvfs/sftp for root on myserver" and the spaces have been escaped.

However, if you are pulling from a remote server, you don't mention the remote anywhere. Why isn't it something like

rsync -options user@remote:/path /local_path

mrloaf 06-04-2009 10:20 PM

Quote:

Originally Posted by billymayday (Post 3563466)
I assume the remote path is "/home/administrator/.gvfs/sftp for root on myserver" and the spaces have been escaped.

However, if you are pulling from a remote server, you don't mention the remote anywhere. Why isn't it something like

rsync -options user@remote:/path /local_path

I tried user@remote:/path - didn't work. So I resorted to mounting the server in Gnome, and using the local mount point (the .gvfs directory) as the source.

irishbitte 06-05-2009 03:24 PM

Quote:

I tried user@remote:/path - didn't work. So I resorted to mounting the server in Gnome, and using the local mount point (the .gvfs directory) as the source.
I don't think that will work very well, tell me, have you an ssh server running on the remote machine, and the keys setup correctly for remote login?

mrloaf 06-06-2009 04:49 PM

Quote:

Originally Posted by irishbitte (Post 3564507)
I don't think that will work very well, tell me, have you an ssh server running on the remote machine, and the keys setup correctly for remote login?

Yes, I can ssh into the remote machine from the command line. And the rsync error doesn't happen until after it asks me for the password (but I'm not getting a password error). I haven't set up keys yet for unattended login in rsync.

irishbitte 06-07-2009 09:42 PM

read all of this, you may find it quite enlightening: http://sial.org/howto/rsync/

mrloaf 06-08-2009 09:23 PM

Quote:

Originally Posted by irishbitte (Post 3566348)
read all of this, you may find it quite enlightening: http://sial.org/howto/rsync/

thanks - but still no joy on my rsync server.
Do I need to have rsync running on the client as well?

billymayday 06-08-2009 09:32 PM

No you don't

What are the logs on the server telling you?

mrloaf 06-08-2009 10:03 PM

Quote:

Originally Posted by billymayday (Post 3567464)
No you don't

What are the logs on the server telling you?

The logs are just the same error messages I get in the terminal. Not much information there. Unless I'm looking at the wrong log?

mrloaf 06-10-2009 02:15 PM

Quote:

Originally Posted by billymayday (Post 3567464)
No you don't

What are the logs on the server telling you?

Thinking that my partitioning & mountpoint scheme might have been contributing to the problem, I yanked the second hard drive I was using before and re-partitioned using vanilla defaults on one drive.

Now rsync doesn't work at all! Different options give different errors. Here's the log results from several attempts. #2 was the closest, but in no case did anything get copied!

Code:

rsync --log-file=/home/administrator/Desktop/rsync-log root@192.168.1.4:/data /home/administrator/backups/fileserver/
2009/06/10 14:01:41 [18260] rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
2009/06/10 14:01:41 [18260] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [receiver=3.0.5]

Code:

rsync -avz --log-file=/home/administrator/Desktop/rsync-log -e root@192.168.1.4:/data /home/administrator/backups/fileserver/
2009/06/10 14:02:14 [18264] building file list
2009/06/10 14:02:14 [18264] drwxrwx--- 4096 2009/06/08 11:22:41 .
2009/06/10 14:02:15 [18264] sent 57 bytes received 12 bytes 138.00 bytes/sec
2009/06/10 14:02:15 [18264] total size is 0 speedup is 0.00

Code:

rsync -avz --log-file=/home/administrator/Desktop/rsync-log -e 'ssh' root@192.168.1.4:/data /home/administrator/backups/fileserver/
2009/06/10 14:02:35 [18270] rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
2009/06/10 14:02:35 [18270] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [receiver=3.0.5]

irishbitte 06-10-2009 04:29 PM

This will only work if you have SSH keys setup between your machine you are sitting at, and the machine with IP 192.168.1.4! You won't have these problems when you setup keys between your machines for unattended rsync. check out the link I sent you earlier!

mrloaf 06-17-2009 03:25 PM

I mounted the remote volumes via sshfs, and now it works great.
Thanks for the help everyone!


All times are GMT -5. The time now is 07:37 AM.