LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
LinkBack Search this Thread
Old 06-04-2009, 05:04 PM   #1
mrloaf
LQ Newbie
 
Registered: Jun 2008
Location: Massaschusetts
Distribution: Ubuntu
Posts: 9

Rep: Reputation: 0
rsync pull on ubuntu 8.10 - noob Q's


I'm setting up a box running Ubuntu 8.10 to rsync my FreeBSD server(s). I want to pull data down to the Ubuntu box, then send it all upstream to an offsite data center. But first I have to get rsync to work.

This works, sort of:

Code:
rsync -halvz /home/administrator/.gvfs/sftp\ for\ root\ on\ myserver /local/backups/ --log-file=/tmp/rlog --exclude-from=/home/administrator/rsync-exclude --delete
It runs for a while, then it errors with:
Code:
rsync: writefd_unbuffered failed to write 461 bytes [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (1258 bytes received so far) [sender]
rsync error: error allocating core memory buffers (code 22) at io.c(600) [sender=3.0.5]
And after that error, it won't run anymore - the error keeps happening. If it were just running out of memory, a restart ought to fix it, but doesn't. The box should be adequate - 2.4 GHz dual P4 with 1 GB ram.

Also: I know there must be a better way to connect to the source, but how? I've been mounting the server via SFTP in the GUI (hence the .gvfs path) which is probably not the fastest way. But ssh thru the shell doesn't seem to work.

rsync daemon is NOT running on the source FreeBSD server.

I'll send whatever add'l info people would like to see - thanks in advance!
 
Old 06-04-2009, 06:38 PM   #2
irishbitte
Senior Member
 
Registered: Oct 2007
Location: Brighton, UK
Distribution: Ubuntu Hardy, Ubuntu Jaunty, Eeebuntu, Debian, SME-Server
Posts: 1,213
Blog Entries: 1

Rep: Reputation: 82
This I'm not getting:
Quote:
...../.gvfs/sftp\ for\ root\ on\ myserver /local/backups/.....
Whats the for\ root\ on\ myserver bit about?
 
Old 06-04-2009, 06:42 PM   #3
billymayday
Guru
 
Registered: Mar 2006
Location: Sydney, Australia
Distribution: Fedora, CentOS, OpenSuse, Slack, Gentoo, Debian, Arch, PCBSD
Posts: 6,678

Rep: Reputation: 120Reputation: 120
I assume the remote path is "/home/administrator/.gvfs/sftp for root on myserver" and the spaces have been escaped.

However, if you are pulling from a remote server, you don't mention the remote anywhere. Why isn't it something like

rsync -options user@remote:/path /local_path
 
Old 06-04-2009, 09:20 PM   #4
mrloaf
LQ Newbie
 
Registered: Jun 2008
Location: Massaschusetts
Distribution: Ubuntu
Posts: 9

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by billymayday View Post
I assume the remote path is "/home/administrator/.gvfs/sftp for root on myserver" and the spaces have been escaped.

However, if you are pulling from a remote server, you don't mention the remote anywhere. Why isn't it something like

rsync -options user@remote:/path /local_path
I tried user@remote:/path - didn't work. So I resorted to mounting the server in Gnome, and using the local mount point (the .gvfs directory) as the source.
 
Old 06-05-2009, 02:24 PM   #5
irishbitte
Senior Member
 
Registered: Oct 2007
Location: Brighton, UK
Distribution: Ubuntu Hardy, Ubuntu Jaunty, Eeebuntu, Debian, SME-Server
Posts: 1,213
Blog Entries: 1

Rep: Reputation: 82
Quote:
I tried user@remote:/path - didn't work. So I resorted to mounting the server in Gnome, and using the local mount point (the .gvfs directory) as the source.
I don't think that will work very well, tell me, have you an ssh server running on the remote machine, and the keys setup correctly for remote login?
 
Old 06-06-2009, 03:49 PM   #6
mrloaf
LQ Newbie
 
Registered: Jun 2008
Location: Massaschusetts
Distribution: Ubuntu
Posts: 9

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by irishbitte View Post
I don't think that will work very well, tell me, have you an ssh server running on the remote machine, and the keys setup correctly for remote login?
Yes, I can ssh into the remote machine from the command line. And the rsync error doesn't happen until after it asks me for the password (but I'm not getting a password error). I haven't set up keys yet for unattended login in rsync.
 
Old 06-07-2009, 08:42 PM   #7
irishbitte
Senior Member
 
Registered: Oct 2007
Location: Brighton, UK
Distribution: Ubuntu Hardy, Ubuntu Jaunty, Eeebuntu, Debian, SME-Server
Posts: 1,213
Blog Entries: 1

Rep: Reputation: 82
read all of this, you may find it quite enlightening: http://sial.org/howto/rsync/
 
Old 06-08-2009, 08:23 PM   #8
mrloaf
LQ Newbie
 
Registered: Jun 2008
Location: Massaschusetts
Distribution: Ubuntu
Posts: 9

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by irishbitte View Post
read all of this, you may find it quite enlightening: http://sial.org/howto/rsync/
thanks - but still no joy on my rsync server.
Do I need to have rsync running on the client as well?
 
Old 06-08-2009, 08:32 PM   #9
billymayday
Guru
 
Registered: Mar 2006
Location: Sydney, Australia
Distribution: Fedora, CentOS, OpenSuse, Slack, Gentoo, Debian, Arch, PCBSD
Posts: 6,678

Rep: Reputation: 120Reputation: 120
No you don't

What are the logs on the server telling you?
 
Old 06-08-2009, 09:03 PM   #10
mrloaf
LQ Newbie
 
Registered: Jun 2008
Location: Massaschusetts
Distribution: Ubuntu
Posts: 9

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by billymayday View Post
No you don't

What are the logs on the server telling you?
The logs are just the same error messages I get in the terminal. Not much information there. Unless I'm looking at the wrong log?
 
Old 06-10-2009, 01:15 PM   #11
mrloaf
LQ Newbie
 
Registered: Jun 2008
Location: Massaschusetts
Distribution: Ubuntu
Posts: 9

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by billymayday View Post
No you don't

What are the logs on the server telling you?
Thinking that my partitioning & mountpoint scheme might have been contributing to the problem, I yanked the second hard drive I was using before and re-partitioned using vanilla defaults on one drive.

Now rsync doesn't work at all! Different options give different errors. Here's the log results from several attempts. #2 was the closest, but in no case did anything get copied!

Code:
rsync --log-file=/home/administrator/Desktop/rsync-log root@192.168.1.4:/data /home/administrator/backups/fileserver/
2009/06/10 14:01:41 [18260] rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
2009/06/10 14:01:41 [18260] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [receiver=3.0.5]

Code:
rsync -avz --log-file=/home/administrator/Desktop/rsync-log -e root@192.168.1.4:/data /home/administrator/backups/fileserver/
2009/06/10 14:02:14 [18264] building file list
2009/06/10 14:02:14 [18264] drwxrwx--- 4096 2009/06/08 11:22:41 .
2009/06/10 14:02:15 [18264] sent 57 bytes received 12 bytes 138.00 bytes/sec
2009/06/10 14:02:15 [18264] total size is 0 speedup is 0.00

Code:
rsync -avz --log-file=/home/administrator/Desktop/rsync-log -e 'ssh' root@192.168.1.4:/data /home/administrator/backups/fileserver/
2009/06/10 14:02:35 [18270] rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
2009/06/10 14:02:35 [18270] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [receiver=3.0.5]

Last edited by mrloaf; 06-10-2009 at 01:27 PM.
 
Old 06-10-2009, 03:29 PM   #12
irishbitte
Senior Member
 
Registered: Oct 2007
Location: Brighton, UK
Distribution: Ubuntu Hardy, Ubuntu Jaunty, Eeebuntu, Debian, SME-Server
Posts: 1,213
Blog Entries: 1

Rep: Reputation: 82
This will only work if you have SSH keys setup between your machine you are sitting at, and the machine with IP 192.168.1.4! You won't have these problems when you setup keys between your machines for unattended rsync. check out the link I sent you earlier!
 
Old 06-17-2009, 02:25 PM   #13
mrloaf
LQ Newbie
 
Registered: Jun 2008
Location: Massaschusetts
Distribution: Ubuntu
Posts: 9

Original Poster
Rep: Reputation: 0
I mounted the remote volumes via sshfs, and now it works great.
Thanks for the help everyone!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Need coaching with RSync and cron configuration on Ubuntu mkenyon2 Linux - Software 4 12-17-2008 03:09 AM
noob away to try ubuntu cormack Ubuntu 2 11-04-2005 10:54 AM
rsync ubuntu mirrors Cybers19 Linux - Newbie 1 09-12-2005 03:47 AM
A few very noob quick Q's SDmac Linux - Newbie 2 03-30-2005 08:29 AM
3 Noob Q's (Debian4.1 on Xbox) Alfastation Debian 4 10-21-2003 03:19 PM


All times are GMT -5. The time now is 07:25 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration