Hi,
I have a two server setup that acts as SMB as well as NFS servers in active/active configuration managed by CTDB(
http://ctdb.samba.org/).
When a file is updated by SMB clients(followed by file-close), other SMB clients can see and modify the file. But when a NFS client (same user) updates the same file(followed by file-close), only one SMB server can see the updates. The clients mounting from other SMB server sees the file corrupted. BUT, when you again update the file from any CIFS clients(mounting from SMB server that can see the updates), the updates are visible on all the CIFS clients.
My first suspect was to do with NFS and SMB caching. I forced NFS server to export with sync option and same with SMB with the following, but does not help:
strict allocate = yes
strict locking = yes
strict sync = yes
sync always = yes
NFS exports:
/mnt/gpfs/nfsexport *(rw,no_root_squash,sync,fsid=222)
Tried NFS mount with sync:
node1:/mnt/gpfs/nfsexport /mnt/nfs nfs rw,tcp,hard,intr,sync,rsize=32768,wsize=32768,vers=3 0 0
smb.conf
[global]
workgroup = TESTDOMAIN2
realm = TESTDOMAIN2.LOCAL
netbios name = CTDB-NAS
server string = Clustered CIFS
security = ADS
auth methods = winbind, sam
password server = 172.16.2.25
private dir = /mnt/gpfs/CTDB_AD
passdb backend = tdbsam
client NTLMv2 auth = Yes
log level = 3 winbind:5 auth:10 passdb:5
syslog = 0
log file = /var/log/samba/log.%m
max log size = 10000
deadtime = 15
socket options = TCP_NODELAY SO_RCVBUF=16384 SO_SNDBUF=16384
use mmap = No
clustering = Yes
machine password timeout = 999999999
local master = No
dns proxy = No
ldap admin dn = cn=ldap,cn=Users,dc=testdomain2,dc=local
ldap idmap suffix = dc=testdomain2,dc=local
ldap suffix = dc=testdomain2,dc=local
idmap backend = ad
idmap uid = 5000-100000000
idmap gid = 5000-100000000
template homedir = /home/%D+%U
template shell = /bin/bash
winbind separator = +
winbind enum users = Yes
winbind enum groups = Yes
nfs4:acedup = merge
nfs4:chown = yes
nfs4:mode = special
gpfs:sharemodes = no
fileid:mapping = fsname
force unknown acl user = Yes
strict allocate = Yes
strict sync = Yes
sync always = Yes
blocking locks = No
strict locking = Yes
vfs objects = gpfs, fileid
[global-share]
comment = GPFS File Share
path = /mnt/gpfs/nfsexport
read only = No
inherit permissions = Yes
inherit acls = Yes
Any idea to what I may be doing wrong?
Thanks in Advance,
-Tim
