LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Enterprise Linux Forums > Linux - Enterprise
User Name
Password
Linux - Enterprise This forum is for all items relating to using Linux in the Enterprise.

Notices



Reply
 
Search this Thread
Old 07-01-2011, 04:14 PM   #1
larold
Member
 
Registered: Jan 2010
Posts: 42

Rep: Reputation: 15
rsync crashes entire networking subsystem?!?


Wondering if any of you out there have seen anything like this.

We used to be doing a nightly rsync from a redhat physical server to another redhat physical server without issues. Total data moved nightly is maybe 20-25 GB.

For testing, we have switched the destination server to be a SLES 11 SP1 virtual machine running under vmware. Same source server.

After maybe 8-10 minutes, the networking subsystem on the VM craps out completely. The link on the virtual NIC still shows as up, and there are no errors reported on the virtual NIC, BUT... no packets can get in or out of the NIC, either internally to our company, or externally to the Internet. The rsync process itself hangs, and pings both to and from the server immediately show 100% dropped packets. The only fix is to ifdown the interface then ifup it again.

This is 100% reproducible. It will occur on different files on each rsync attempt. Things I am going to test when I get a chance:
- Mess around with possible compression options
- Get a precise time of how long it takes before the network subsystem hangs
- Hang an strace off of the rsync server process and see what's going on at the time of networking wedge.
- Set the bwlimit option of rsync to see if that prevents the hang or causes it at a significantly different time offset from invocation of the sync.

My gut tells me this is a stacked problem. Possibly a problem with rsync that then exposes a SLES or VMWare networking problem. I mean come on... the WHOLE NETWORKING SUBSYSTEM JUST BEING BLACKED OUT BY RSYNC??? (Or it could be something ssh-related).

Anyways - any thoughts you folks out there could offer would be much appreciated. Things to try - things it could be, etc. etc. I'm already using rsync -vv and -P options - that's not revealing much.

Thanks!

(Will provide updates in a few days)
 
Old 07-01-2011, 04:34 PM   #2
aysheaia
LQ Newbie
 
Registered: Jun 2011
Distribution: Ubuntu
Posts: 26

Rep: Reputation: Disabled
That's not trancendental, but :
- tcpdump (around the time of failure, for limiting the size of the generated file) could show some interesting things.
- sar can collect statistics of network interfaces (usage and errors).
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Rsync problem: copying entire file when almost identical genmaicha Linux - Software 1 04-14-2009 11:56 AM
Rsync Entire Filesystem? sutley Linux - Newbie 4 10-04-2006 07:29 PM
rsync.net opens offsite storage to entire SSH suite jbrooks Linux - Software 0 05-02-2006 04:47 PM
Artsd crashes entire system? secho Linux - Software 0 05-20-2004 07:58 PM


All times are GMT -5. The time now is 03:21 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration