File integrity in Samba/NFS environment
Hi,
I have a two server setup that acts as SMB as well as NFS servers in active/active configuration managed by CTDB(http://ctdb.samba.org/). When a file is updated by SMB clients(followed by file-close), other SMB clients can see and modify the file. But when a NFS client (same user) updates the same file(followed by file-close), only one SMB server can see the updates. The clients mounting from other SMB server sees the file corrupted. BUT, when you again update the file from any CIFS clients(mounting from SMB server that can see the updates), the updates are visible on all the CIFS clients. My first suspect was to do with NFS and SMB caching. I forced NFS server to export with sync option and same with SMB with the following, but does not help: strict allocate = yes strict locking = yes strict sync = yes sync always = yes NFS exports: /mnt/gpfs/nfsexport *(rw,no_root_squash,sync,fsid=222) Tried NFS mount with sync: node1:/mnt/gpfs/nfsexport /mnt/nfs nfs rw,tcp,hard,intr,sync,rsize=32768,wsize=32768,vers=3 0 0 smb.conf [global] workgroup = TESTDOMAIN2 realm = TESTDOMAIN2.LOCAL netbios name = CTDB-NAS server string = Clustered CIFS security = ADS auth methods = winbind, sam password server = 172.16.2.25 private dir = /mnt/gpfs/CTDB_AD passdb backend = tdbsam client NTLMv2 auth = Yes log level = 3 winbind:5 auth:10 passdb:5 syslog = 0 log file = /var/log/samba/log.%m max log size = 10000 deadtime = 15 socket options = TCP_NODELAY SO_RCVBUF=16384 SO_SNDBUF=16384 use mmap = No clustering = Yes machine password timeout = 999999999 local master = No dns proxy = No ldap admin dn = cn=ldap,cn=Users,dc=testdomain2,dc=local ldap idmap suffix = dc=testdomain2,dc=local ldap suffix = dc=testdomain2,dc=local idmap backend = ad idmap uid = 5000-100000000 idmap gid = 5000-100000000 template homedir = /home/%D+%U template shell = /bin/bash winbind separator = + winbind enum users = Yes winbind enum groups = Yes nfs4:acedup = merge nfs4:chown = yes nfs4:mode = special gpfs:sharemodes = no fileid:mapping = fsname force unknown acl user = Yes strict allocate = Yes strict sync = Yes sync always = Yes blocking locks = No strict locking = Yes vfs objects = gpfs, fileid [global-share] comment = GPFS File Share path = /mnt/gpfs/nfsexport read only = No inherit permissions = Yes inherit acls = Yes Any idea to what I may be doing wrong? Thanks in Advance, -Tim |
Hey There,
I think I found an answer for you from samba.org. It's not mine, so, if it works, thank the gentleman at the link below :) I would inline it but it's a lengthy explanation, and addresses a few of your concerns. Hope it's of some help to you and best wishes, http://lists.samba.org/archive/samba...ne/019414.html , Mike |
File integrity in Samba/NFS environment
Thanks Mike for the pointers.
With "oplocks = no", still we see the same behaviour of only some SMB clients seeing the file updates from NFS client (even after file is closed after updates). Using native file-system GPFS Linux client, all the SMB clients can see updates from the Linux client. I tried appending contents from NFS client via O_DIRECT flag (to bypass NFS caching), but does not seem to solve. When SMB clients(which cannot see updates) tries to access the file after updated by NFS client, we get the following error: "Cannot open the Z:\FileName_XYZ Make sure a disk is in the drive you specified". The SMB client run Windows 2003 64-bit and all of them have the latest updates. Please note, it seems to be always the same SMB server that is having issues and when the SMB client mounting from the other SMB server updates the NFS file, all the SMB clients can access. The SMB servers are managed via CTDB authenticating via AD 2003. Thoughts/comments/suggestions to resolve this will be greatly appreciated. Thanks, -Tim |
All times are GMT -5. The time now is 11:05 PM. |