From: Bruce Fields <bfields@fieldses.org>
To: dai.ngo@oracle.com
Cc: "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>,
CIFS <linux-cifs@vger.kernel.org>
Subject: Re: Locking issue between NFSv4 and SMB client
Date: Thu, 23 Sep 2021 17:50:56 -0400 [thread overview]
Message-ID: <20210923215056.GH18334@fieldses.org> (raw)
In-Reply-To: <5b7be2c0-95a6-048c-581f-17e5e3750daa@oracle.com>
On Thu, Jul 15, 2021 at 04:45:22PM -0700, dai.ngo@oracle.com wrote:
> Hi Bruce,
Oops, sorry for neglecting this.
> I'm doing some locking testing between NFSv4 and SMB client and
> think there are some issues on the server that allows both clients
> to lock the same file at the same time.
It's not too surprising to me that getting consistent locks between the
two would be hard.
Did you get any review from a Samba expert? I seem to recall it having
a lot of options, and I wonder if it's configured correctly for this
case.
It sounds like Samba may be giving out oplocks without getting a lease
from the kernel.
--b.
> Here is what I did:
>
> NOTE: lck is a simple program that use lockf(3) to lock a file from
> offset 0 to the length specified by '-l'.
>
> On NFSv4 client
> ---------------
>
> [root@nfsvmd07 ~]# nfsstat -m
> /tmp/mnt from nfsvmf24:/root/smb_share
> Flags: rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,
> proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.80.62.47,
> local_lock=none,addr=10.80.111.94
> [root@nfsvmd07 ~]#
>
>
> [root@nfsvmd07 ~]# ./lck -p /tmp/mnt/messages -W -l 100000000
> Lck/file: 1, Maxlocks: 10000000
> Locking[/tmp/mnt/messages] Offset[0] Len[100000000] N[0]...doing F_LOCK..
> LOCKED...
>
> Locks[1] files[1] took[2.000s] sleep waiting...Hit Control-C to stop
>
> [NFS client successfully locks the file]
>
> On SMB client
> -------------
>
> [root@nfsvme24 ~]# mount |grep cifs
> //nfsvmf24/smb_share on /tmp/mnt type cifs (rw,relatime,vers=3.1.1,cache=strict,username=root,uid=0,noforceuid,gid=0,noforcegid,addr=10.80.111.94,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1)
> [root@nfsvme24 ~]#
>
> [root@nfsvme24 ~]# smbclient -L nfsvmf24
> Enter SAMBA\root's password:
>
> Sharename Type Comment
> --------- ---- -------
> print$ Disk Printer Drivers
> smb_share Disk Test Samba Share <<===== share to mount
> IPC$ IPC IPC Service (Samba 4.10.16)
> root Disk Home Directories
> Reconnecting with SMB1 for workgroup listing.
>
> Server Comment
> --------- -------
>
> Workgroup Master
> --------- -------
> [root@nfsvme24 ~]#
>
> [root@nfsvme24 ~]# ./lck -p /tmp/mnt/messages -W -l 100000000
> Lck/file: 1, Maxlocks: 10000000
> Locking[/tmp/mnt/messages] Offset[0] Len[100000000] N[0]...doing F_LOCK..
> LOCKED...
>
> Locks[1] files[1] took[2.000s] sleep waiting...Hit Control-C to stop
>
> [SMB client successfully locks the file]
>
> The same issue happens when either client locks the file first.
> I think this is what has happened:
>
> 1. NFSv4 client opens and locks the file first
>
> . NFSv4 client send OPEN and LOCK to server, server replies
> OK on both requests.
>
> . SMB client sends create request with Oplock==Lease for
> the same file.
>
> . server holds off on replying to SMB client's create request,
> recalls delegation from NFSv4 client, waits for NFSv4 client
> to return the delegation then replies success to SMB client's
> create request with lease granted (Oplock==Lease).
>
> NOTE: I think SMB server should replies the create request
> with Oplock==None to force the SMB client to sends the
> lock request.
>
> . Once SMB client receives the reply of the create with
> 'Oplock==Lease', it assumes it has full control of the file
> therefor it does not need to send the lock request.
>
> . both NFSv4 and SMB client now think they have locked the file.
>
> pcap: nfs_lock_smb_lock.pcap
>
> 2. SMB client creates the file with 'Oplock==Lease' first
>
> . SMB sends create request with 'Oplock==Lease' to server,
> server replies OK with 'Oplock==Lease'. SMB client skips
> sending lock request since it assumes it has full control
> of the file with the lease.
>
> . NFSv4 client sends OPEN to server, server replies OK with
> delagation is none. NFSv4 client sends LOCK request, since
> no lock was created in the kernel for the SMB client, the
> lock was granted to the NFSv4 client.
>
> NOTE: I think the SMB server should send lease break
> notification to the SMB client, wait for the lease break
> acknowledgment from SMB client before replying to the
> OPEN of the NFSv4 client. This will force the SMB client
> to send the lock request to the server.
>
> . both NFSv4 and SMB client now think they have locked the file.
>
> Your thought?
>
> Thanks,
>
> -Dai
next prev parent reply other threads:[~2021-09-23 21:50 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-15 23:45 Locking issue between NFSv4 and SMB client dai.ngo
2021-07-15 23:47 ` dai.ngo
2021-09-23 21:50 ` Bruce Fields [this message]
2021-09-23 22:39 ` dai.ngo
2021-09-24 0:51 ` Jeremy Allison
2021-09-24 0:56 ` Bruce Fields
2021-10-07 17:03 ` dai.ngo
2021-10-07 17:38 ` dai.ngo
2021-10-11 16:21 ` Bruce Fields
2021-09-24 3:35 ` Trond Myklebust
2021-09-24 3:46 ` Ralph Boehme
2021-09-24 4:13 ` Trond Myklebust
2021-09-24 4:55 ` Ralph Boehme
2021-09-24 16:36 ` Jeremy Allison
2021-09-24 16:45 ` bfields
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210923215056.GH18334@fieldses.org \
--to=bfields@fieldses.org \
--cc=dai.ngo@oracle.com \
--cc=linux-cifs@vger.kernel.org \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).