LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices



Reply
 
Search this Thread
Old 01-04-2009, 03:52 PM   #1
lqtim
LQ Newbie
 
Registered: Dec 2008
Posts: 5

Rep: Reputation: 0
File integrity in Samba/NFS environment


Hi,

I have a two server setup that acts as SMB as well as NFS servers in active/active configuration managed by CTDB(http://ctdb.samba.org/).

When a file is updated by SMB clients(followed by file-close), other SMB clients can see and modify the file. But when a NFS client (same user) updates the same file(followed by file-close), only one SMB server can see the updates. The clients mounting from other SMB server sees the file corrupted. BUT, when you again update the file from any CIFS clients(mounting from SMB server that can see the updates), the updates are visible on all the CIFS clients.

My first suspect was to do with NFS and SMB caching. I forced NFS server to export with sync option and same with SMB with the following, but does not help:

strict allocate = yes
strict locking = yes
strict sync = yes
sync always = yes

NFS exports:
/mnt/gpfs/nfsexport *(rw,no_root_squash,sync,fsid=222)

Tried NFS mount with sync:
node1:/mnt/gpfs/nfsexport /mnt/nfs nfs rw,tcp,hard,intr,sync,rsize=32768,wsize=32768,vers=3 0 0

smb.conf
[global]
workgroup = TESTDOMAIN2
realm = TESTDOMAIN2.LOCAL
netbios name = CTDB-NAS
server string = Clustered CIFS
security = ADS
auth methods = winbind, sam
password server = 172.16.2.25
private dir = /mnt/gpfs/CTDB_AD
passdb backend = tdbsam
client NTLMv2 auth = Yes
log level = 3 winbind:5 auth:10 passdb:5
syslog = 0
log file = /var/log/samba/log.%m
max log size = 10000
deadtime = 15
socket options = TCP_NODELAY SO_RCVBUF=16384 SO_SNDBUF=16384
use mmap = No
clustering = Yes
machine password timeout = 999999999
local master = No
dns proxy = No
ldap admin dn = cn=ldap,cn=Users,dc=testdomain2,dc=local
ldap idmap suffix = dc=testdomain2,dc=local
ldap suffix = dc=testdomain2,dc=local
idmap backend = ad
idmap uid = 5000-100000000
idmap gid = 5000-100000000
template homedir = /home/%D+%U
template shell = /bin/bash
winbind separator = +
winbind enum users = Yes
winbind enum groups = Yes
nfs4:acedup = merge
nfs4:chown = yes
nfs4:mode = special
gpfs:sharemodes = no
fileid:mapping = fsname
force unknown acl user = Yes
strict allocate = Yes
strict sync = Yes
sync always = Yes
blocking locks = No
strict locking = Yes
vfs objects = gpfs, fileid
[global-share]
comment = GPFS File Share
path = /mnt/gpfs/nfsexport
read only = No
inherit permissions = Yes
inherit acls = Yes

Any idea to what I may be doing wrong?

Thanks in Advance,
-Tim
 
Old 01-04-2009, 10:48 PM   #2
eggixyz
Member
 
Registered: Apr 2008
Posts: 310

Rep: Reputation: 30
Hey There,

I think I found an answer for you from samba.org. It's not mine, so, if it works, thank the gentleman at the link below

I would inline it but it's a lengthy explanation, and addresses a few of your concerns.

Hope it's of some help to you and best wishes,

http://lists.samba.org/archive/samba...ne/019414.html

, Mike
 
Old 01-05-2009, 06:22 PM   #3
lqtim
LQ Newbie
 
Registered: Dec 2008
Posts: 5

Original Poster
Rep: Reputation: 0
File integrity in Samba/NFS environment

Thanks Mike for the pointers.

With "oplocks = no", still we see the same behaviour of only some SMB clients seeing the file updates from NFS client (even after file is closed after updates). Using native file-system GPFS Linux client, all the SMB clients can see updates from the Linux client. I tried appending contents from NFS client via O_DIRECT flag (to bypass NFS caching), but does not seem to solve.

When SMB clients(which cannot see updates) tries to access the file after updated by NFS client, we get the following error:

"Cannot open the Z:\FileName_XYZ
Make sure a disk is in the drive you specified".

The SMB client run Windows 2003 64-bit and all of them have the latest updates.

Please note, it seems to be always the same SMB server that is having issues and when the SMB client mounting from the other SMB server updates the NFS file, all the SMB clients can access. The SMB servers are managed via CTDB authenticating via AD 2003.

Thoughts/comments/suggestions to resolve this will be greatly appreciated.

Thanks,
-Tim
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Samba on NFS File Locking Cottsay Linux - Software 4 01-26-2012 07:15 AM
Stale NFS File handle after samba install Spyder~1 Linux - Newbie 7 04-08-2009 03:36 PM
File integrity in Samba/NFS environment lqtim Linux - Software 1 01-04-2009 04:23 PM
NFS/Samba file transfer hangs the server aZZZ Linux - Server 2 09-04-2007 01:39 PM
Trying to mount W2k3 shareusing Samba 3.0.14a-Debian gives Stale NFS file handle lbclinux Debian 1 08-21-2005 08:07 AM


All times are GMT -5. The time now is 09:32 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration