LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 10-26-2007, 03:53 AM   #1
PhillipHuang
Member
 
Registered: Aug 2006
Location: Shen Zhen
Distribution: Ubuntu 10.04
Posts: 198

Rep: Reputation: 33
NFS+GFS issue caused by dlm_sendd


Hello all,

My cluster info:
-----------------------
CentOS4.4 as operating system
RedHat Cluster Suite 4.0/GFS
A background storage

After the cluster is created successfully, I try to mount the shares(5TB) building on GFS filesytem across two nodes in another client.
1. when mount the share from node1:
Code:
node1#mount /dev/mapper/test/test /mnt/
It is very quickly, about 1 second.
2. and then mount the share from node2:
Code:
node2#mount /dev/mapper/test/test /mnt/
It is very slowly, and the worst recorder is 20 minutes.Of course, this time must be accepted by customers.

I arbitrarily do the above test, and there's always a node mounted very quicky and the left one is mounted slowly. When checking the slow node by issuing "top" command, it displays the "dlm_sendd" became a grant CPU hog when client is mounting the node's NFS share. I think this process causes the low performance.

I google for the key word "dlm_sendd", somebody said it is a kernel bug that has been fixed, or add numa=off boot strings. However, I do not get successful as following with these hints.

Later, I see the nfsd is waiting for CPU time when dlm_sendd keeps high,then I try to modify NFS to make nfsd running first and faster.
for example, assume node2 is slow one.
Code:
node2# service nfs stop
node2# rpc.nfsd
node2# rpc.mountd
node2# rpc.equotad
node2# exportfs -r
at this moment, client can mount node2's share very quickly. I repeat the mount and umount operation more than 10 times, all the testing are about 3-5 seconds.

It seems to have been resolved this issue. But I'm not sure this resolution is stable and if there will be patient troubles.

How do you think about this issue? and any suggestions?
Thanks in advanced.

Phillip
 
Old 10-30-2007, 10:38 PM   #2
PhillipHuang
Member
 
Registered: Aug 2006
Location: Shen Zhen
Distribution: Ubuntu 10.04
Posts: 198

Original Poster
Rep: Reputation: 33
Update.

David Teigland from RedHat provides the following info.
https://www.redhat.com/archives/linu.../msg00268.html

I'll verify it on 2.6.23-rc.

-Phillip
 
  


Reply

Tags
gfs, nfs


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Pick: GFS or GNBD or NFS To make a single FS Roboam León López Linux - Enterprise 5 10-26-2007 01:52 AM
Network issue in RHCE/GFS environment PhillipHuang Linux - Networking 6 06-21-2007 08:40 PM
Issue with NFS gopinathassrj Linux - Networking 2 02-02-2007 11:46 AM
lilo caused to fail NFS statd shanenin Linux - Software 2 09-04-2003 08:35 PM
nfs and smbfs issue iquadri1 Linux - Networking 3 09-07-2001 01:06 PM


All times are GMT -5. The time now is 07:59 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration