LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices

Reply
 
Search this Thread
Old 04-08-2013, 06:21 PM   #1
Woodsman
Senior Member
 
Registered: Oct 2005
Distribution: Slackware 14.1
Posts: 3,482

Rep: Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534
Frustrating rsync stoppage


I have some scripts to update my local Slackware repos. They are modified versions of Eric's original rsync scripts. Currently I am updating only 14.0 and Current.

The scripts run fine but about half the time the scripts stall. No error messages.

All I see is something like this:

77.30M 80% 392kB/s 0:01:53

The progress status never changes.

I thought the problem might be the upstream server. Changing the server makes no difference.

I run rsync like this:

rsync -vahP --delete

I would appreciate ideas how to debug.

Thanks.
 
Old 04-08-2013, 06:53 PM   #2
jmccue
Member
 
Registered: Nov 2008
Location: US
Distribution: slackware
Posts: 135

Rep: Reputation: 36
Hi,

Did you try -vvv... ? more v's the more 'debug' info

John
 
Old 04-08-2013, 07:12 PM   #3
GazL
Senior Member
 
Registered: May 2008
Posts: 3,480

Rep: Reputation: 1016Reputation: 1016Reputation: 1016Reputation: 1016Reputation: 1016Reputation: 1016Reputation: 1016Reputation: 1016
You might also want to consider adding a --timeout=60
 
Old 04-08-2013, 07:35 PM   #4
Woodsman
Senior Member
 
Registered: Oct 2005
Distribution: Slackware 14.1
Posts: 3,482

Original Poster
Rep: Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534
I will test both options. Thanks. Of course, posting results likely will take a while since the stoppage is not 100% repeatable. Then again, seamonkey is patched almost daily (I'm exaggerating, a wee bit but not by much), so I might see results sooner.
 
Old 04-09-2013, 10:48 AM   #5
willysr
Senior Member
 
Registered: Jul 2004
Location: Jogja, Indonesia
Distribution: Slackware-Current
Posts: 2,599

Rep: Reputation: 437Reputation: 437Reputation: 437Reputation: 437Reputation: 437
you can also use --progress
 
Old 04-09-2013, 01:06 PM   #6
Woodsman
Senior Member
 
Registered: Oct 2005
Distribution: Slackware 14.1
Posts: 3,482

Original Poster
Rep: Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534
Quote:
you can also use --progress
See the original post --- I'm using the P option.

The P option doesn't expose the stoppage anyway. As I mentioned, the progress information just stops. Nothing more.
 
Old 05-08-2013, 01:20 PM   #7
Woodsman
Senior Member
 
Registered: Oct 2005
Distribution: Slackware 14.1
Posts: 3,482

Original Poster
Rep: Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534
Been a while since rsync stalled on me but today that happened. As I increased the verbosity and set a timeout, today I saw the following output:

[receiver] io timeout after 60 seconds -- exiting
rsync error: timeout in data send/receive (code 30) at io.c(140) [receiver=3.0.9]
rsync: connection unexpectedly closed (645 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(605) [generator=3.0.9]

The script stalled when updating gwenview 4.10.3 in Current. In this case I was using the oregon state server.

The confusing thing is the script works for a while without errors or warnings and then one day pukes. The rsync web site addresses some of these issues but I'm not seeing anything obvious. The web site lists the following as possible causes:

* The destination disk is full (remember that you need at least the size of the largest file that needs to be updated available in free disk space for the transfer to succeed).
* An idle connection caused a router or remote-shell server to close the connection.
* A network error caused the connection to be dropped.
* The remote rsync executable wasn't found.
* Your remote-shell setup isn't working right or isn't "clean" (i.e. it is sending spurious text to rsync).

The only two that seem to apply in my case are the second and third reasons.

I'm going to try changing --delete to --delete-during as suggested at the rsync web site and use a different server.

Otherwise, any ideas?
 
Old 05-10-2013, 07:21 AM   #8
gezley
Member
 
Registered: Sep 2009
Location: Ireland
Distribution: Slackware64, NetBSD
Posts: 535

Rep: Reputation: 248Reputation: 248Reputation: 248
Quote:
Originally Posted by Woodsman View Post
Been a while since rsync stalled on me but today that happened. As I increased the verbosity and set a timeout, today I saw the following output:

[receiver] io timeout after 60 seconds -- exiting
rsync error: timeout in data send/receive (code 30) at io.c(140) [receiver=3.0.9]
rsync: connection unexpectedly closed (645 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(605) [generator=3.0.9]

The script stalled when updating gwenview 4.10.3 in Current. In this case I was using the oregon state server.

The confusing thing is the script works for a while without errors or warnings and then one day pukes. The rsync web site addresses some of these issues but I'm not seeing anything obvious. The web site lists the following as possible causes:

* The destination disk is full (remember that you need at least the size of the largest file that needs to be updated available in free disk space for the transfer to succeed).
* An idle connection caused a router or remote-shell server to close the connection.
* A network error caused the connection to be dropped.
* The remote rsync executable wasn't found.
* Your remote-shell setup isn't working right or isn't "clean" (i.e. it is sending spurious text to rsync).

The only two that seem to apply in my case are the second and third reasons.

I'm going to try changing --delete to --delete-during as suggested at the rsync web site and use a different server.

Otherwise, any ideas?
Buffering limits with the network adapter? Have you tried limiting the bandwidth used?

Code:
rsync -avz --bwlimit=100   # restrict to 100Kb/sec
Is it a wireless adapter? Ethernet? Realtek chip? Can you try another PCI/PCI-E/PCMCIA/USB adapter?
 
Old 05-11-2013, 06:23 PM   #9
rouvas
Member
 
Registered: Aug 2006
Location: Greece
Distribution: Slackware.12.2
Posts: 85
Blog Entries: 2

Rep: Reputation: 6
I've seen this kind of behaviour before.
It's a network issue. Presumably your ADSL/modem/whatever you use to connect to the interwebs or your ISP resets or the connection is somehow lost and rsync just lingers there forever.

Use the network timeouts options of rsync (as in --timeout=SECONDS and --contimeout), do not use --delete-before and when something goes wrong, rsync will eventually (after SECONDS) will give up.
If all this inside a script, check the exit status and restart the whole process if it is 30 or 35.
 
Old 05-11-2013, 08:15 PM   #10
Woodsman
Senior Member
 
Registered: Oct 2005
Distribution: Slackware 14.1
Posts: 3,482

Original Poster
Rep: Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534
Just a 1 Gbit on-board NIC and WRT54GL Linksys router.

I use rsync among all of my systems in my home LAN to keep certain files in sync. I don't recall ever seeing a hang. That includes my local router and switch. I am inclined to believe the problem is external. The ISP service is proprietary wireless (rural area) and I do have occasional problems.

I updated my script wrapper to capture and start again when the exit code is 30 or 35. I only use the script when there are changes in patches and current, so testing requires weeks of observation. We'll see how this goes. :-)
 
1 members found this post helpful.
Old 05-11-2013, 08:32 PM   #11
hitest
Senior Member
 
Registered: Mar 2004
Location: Prince Rupert, B.C., Canada
Distribution: Slackware, OpenBSD
Posts: 4,243

Rep: Reputation: 573Reputation: 573Reputation: 573Reputation: 573Reputation: 573Reputation: 573
Woodsman,

I'll be curious to hear what you find out. I suspect it may be a network issue as I've had very good luck updating -current using the University of Oregon server.
 
Old 05-11-2013, 11:03 PM   #12
Woodsman
Senior Member
 
Registered: Oct 2005
Distribution: Slackware 14.1
Posts: 3,482

Original Poster
Rep: Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534Reputation: 534
Probably the only way I would know for sure of the cause is run wireshark concurrently when I run the script. I don't know exactly what to watch for. Further, as the stoppage occurs randomly and occasionally, I could wait many weeks to catch another stall.

The primary challenge is the ISP wireless system is run on the free 900 MHz band, which means a lot of things could cause interference. Anything from overranged radios to baby monitors use that band.

I do know that somebody in my general areas is using such equipment because I see these random occasional stalls with just basic browsing. One moment the connection is there, then not, then is okay again. The ISP owner knows the electric utility maintenace folks use overranged radios, but they won't admit that. :-)

One problem with this issue is rsync does not seem to have any reconnect-and-retry option. rsync seems poorly designed with that. Thus the exit code capture and while loop seems to be the only work-around. I could revise my script to email me a message any time that happens, but I don't have any way to know the cause of the stoppage unless I do something like monitor with wireshark. The challenge with that is any time there is a large update to the Current tree, I really don't want to sit staring at my monitor for a couple of hours waiting for a possible stall.

Even if I discovered the problem is caused by local radio interference, which is likely, there is nothing I can do except restart the session, which my script should now be able to do.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Very frustrating captain chaos Slackware 9 02-28-2009 03:06 PM
Frustrating RSYNC issue setting up CentOS mirror Maeltor Linux - Server 4 03-05-2008 12:30 AM
Too frustrating Harnic Linux - Networking 6 09-04-2004 04:15 PM
this is getting frustrating thebutler4 Linux - Newbie 5 05-24-2004 07:59 PM
Frustrating PlanetNEO Linux - Networking 24 12-09-2002 04:56 PM


All times are GMT -5. The time now is 06:11 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration