From: Amir Goldstein <amir73il@gmail.com>
To: Stefan Metzmacher <metze@samba.org>
Cc: "Ralph Böhme" <slow@samba.org>, "Jeremy Allison" <jra@samba.org>,
"Linux NFS Mailing List" <linux-nfs@vger.kernel.org>,
Volker.Lendecke@sernet.de, devel@lists.nfs-ganesha.org,
"Jeff Layton" <jlayton@kernel.org>,
samba-technical <samba-technical@lists.samba.org>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
"J. Bruce Fields" <bfields@fieldses.org>
Subject: Re: Better interop for NFS/SMB file share mode/reservation
Date: Thu, 25 Apr 2019 21:11:34 +0300 [thread overview]
Message-ID: <CAOQ4uxiBLw_L=SqCjLU6W60LbtWiLaBh=5Cb4HnSAFqCW0z1WA@mail.gmail.com> (raw)
In-Reply-To: <5ebdb58b-26d9-c0f2-bd67-883bc4678ac7@samba.org>
On Thu, Mar 7, 2019 at 1:04 PM Stefan Metzmacher <metze@samba.org> wrote:
>
> Am 06.03.19 um 22:25 schrieb Ralph Böhme via samba-technical:
> >
> > Jeremy Allison wrote:
> >> On Wed, Mar 06, 2019 at 03:31:08PM -0500, Jeff Layton wrote:
> >>> On Wed, 2019-03-06 at 10:11 -0500, J. Bruce Fields wrote:
> >>>>
> >>>> Jeff, wasn't there some work (on Ceph maybe?) on a userspace delegation
> >>>> API? Is that close to what's needed?
> >>>>
> >>>
> >>> Here's the C headers for that stuff:
> >>>
> >>> https://github.com/ceph/ceph/blob/7ba6bece4187eda5d05a9b84211fe6ba8dd287bd/src/include/cephfs/libcephfs.h#L1734
> >>>
> >>> It's simple enough and works for us in ganesha, and I think we can
> >>> probably adapt it to samba without too much difficulty. The callback
> >>> doesn't seem like it'll do for a kernel API though -- you'd almost
> >>> certainly need to do something different there (signals? inotify?).
> >>
> >> SMB3 leases have R/RW and Handle-based leases.
> >
> > Just to be precise: SMB2.1+ has R, RH, RW and RWH leases.
> >
> >> Handle leases allow multiple opens of the same pathname
> >> that get different handles to share the lease, allowing
> >> a client redirector to delay opens or closes locally
> >> so long as it has a handle lease.
> >
> > That'a a propertly of leases in general, not just H-leases. The client provides a lease key which is a GUID with each lease request
> >
> >>
> >> Here are the semantics:
> >>
> >> https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/d8df943d-6ad7-4b30-9f58-96ae90fc6204
> >>
> >> I'm not sure a simple file-descriptor based API is
> >> enough for us. Can he have a uuid or token based
> >> API instead where the server can chose what fd's
> >> to cover with a token ?
> >
> > Yes, that would be ideal.
Getting back to this.
Thanks all for the valuable inputs.
Next week is LSF/MM and I was assigned a 30 minute slot on filesystems track
to discuss "NFS/SMB file share".
So let me try to echo what I read on this thread and how I understand what APIs
samba needs from the kernel.
>
> If we want to design an useful API, we also need to think about
> all features:
> - file oplock/leases
Kernel can have a flavor of leases which are not broken
by opens from threads of the process holding the lease.
Bruce has some patches along those lines for knfsd and SMB R/RW
leases could use this flavor if it was exported to userspace?
For SMB RH/RWH leases and Ganesha delegations, server
could keep track of its own handles/clients and break leases within the
same process without involving the kernel.
Am I wrong?
> - directory leases
I have WIP on fsnotify directory pre modification hooks.
There is opposition from fsnotify maintainer to add new userspace
APIs that can create kernel->user->kernel deadlocks, like the
deadlocks currently reported with fanotify permission events.
Need to see if we can find a middle ground between
"post modification notifications" and "pre modification permission"
API, somewhere along the lines of regular file lease breaking API.
> - share modes
Volker told me he thinks samba can enforce share modes by
a single daemon policing all opens in the system with fanotify.
I think he is right. If anyone thinks differently please speak up.
> - disconnected handles (for durable and persistent handles),
> which exists within the kernel for a while and can be reattached
> to process, using some kind of cookie and the same euid
So this interface exists in the kernel.
Nothing more required from the kernel API. Right?
> - the API needs ways to use epoll in order to do async opens
> and lease breaks. For opens the model of async socket connects
> could be used. Leases could have a signalfd-style api.
I should hope that the new AIO API (http://kernel.dk/io_uring.pdf)
would solve those problems as well as other issues that
samba has w.r.t dispatching AIO.
>
> We may not need everything at once, but we should have the full picture
> in mind. And we need working code in kernel and userspace that passes
> all tests (we may need to add additional test). Otherwise the kernel
> creates new syscalls, which wouldn't be used by Samba in the end.
>
Tested interfaces - good idea ;-)
If anyone has any comments about my view of required new interfaces,
or important things that I missed, please say so before Tuesday!
Thanks,
Amir.
next prev parent reply other threads:[~2019-04-25 18:11 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-08 11:20 Better interop for NFS/SMB file share mode/reservation Amir Goldstein
2019-02-08 13:10 ` Jeff Layton
2019-02-08 14:45 ` Amir Goldstein
2019-02-08 15:50 ` J. Bruce Fields
2019-02-08 20:02 ` Amir Goldstein
2019-02-08 20:16 ` J. Bruce Fields
2019-02-08 20:31 ` Amir Goldstein
2019-02-14 20:51 ` J. Bruce Fields
2019-02-15 7:31 ` Amir Goldstein
2019-02-15 20:09 ` J. Bruce Fields
2019-02-08 22:12 ` Jeremy Allison
2019-02-09 4:04 ` Amir Goldstein
2019-02-14 21:06 ` J. Bruce Fields
2019-03-05 21:47 ` J. Bruce Fields
2019-03-06 7:09 ` Amir Goldstein
2019-03-06 15:17 ` J. Bruce Fields
2019-03-06 15:37 ` [NFS-Ganesha-Devel] " Frank Filz
2019-03-08 21:38 ` 'J. Bruce Fields'
2019-03-08 21:53 ` Frank Filz
2019-03-06 15:11 ` J. Bruce Fields
2019-03-06 20:31 ` Jeff Layton
2019-03-06 21:07 ` Jeremy Allison
2019-03-06 21:25 ` Ralph Böhme
2019-03-07 11:03 ` Stefan Metzmacher
2019-03-07 16:47 ` Simo
2019-04-25 18:11 ` Amir Goldstein [this message]
2019-05-24 7:12 ` Amir Goldstein
2019-05-24 13:15 ` Ralph Boehme
2019-05-24 15:07 ` J. Bruce Fields
2019-03-06 21:55 ` Jeff Layton
2019-02-08 16:03 ` Jeff Layton
2019-02-08 16:28 ` Jeffrey Layton
[not found] ` <CAOQ4uxgQsRaEOxz1aYzP1_1fzRpQbOm2-wuzG=ABAphPB=7Mxg@mail.gmail.com>
[not found] ` <20190426140023.GB25827@fieldses.org>
[not found] ` <CAOQ4uxhuxoEsoBbvenJ8eLGstPc4AH-msrxDC-tBFRhvDxRSNg@mail.gmail.com>
[not found] ` <20190426145006.GD25827@fieldses.org>
[not found] ` <e69d149c80187b84833fec369ad8a51247871f26.camel@kernel.org>
2019-04-27 20:16 ` Amir Goldstein
2019-04-28 12:09 ` Jeff Layton
2019-04-28 13:45 ` Amir Goldstein
2019-04-28 15:06 ` Trond Myklebust
2019-04-28 22:00 ` Amir Goldstein
2019-04-28 22:08 ` Trond Myklebust
2019-04-28 22:33 ` Amir Goldstein
2019-04-29 0:57 ` Trond Myklebust
2019-04-29 11:42 ` Amir Goldstein
2019-04-29 13:10 ` Trond Myklebust
2019-04-29 20:29 ` Jeff Layton
2019-04-29 22:33 ` Pavel Shilovskiy
2019-04-30 0:31 ` Amir Goldstein
2019-04-30 8:12 ` Uri Simchoni
2019-04-30 9:22 ` Amir Goldstein
2019-02-11 5:31 ` ronnie sahlberg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAOQ4uxiBLw_L=SqCjLU6W60LbtWiLaBh=5Cb4HnSAFqCW0z1WA@mail.gmail.com' \
--to=amir73il@gmail.com \
--cc=Volker.Lendecke@sernet.de \
--cc=bfields@fieldses.org \
--cc=devel@lists.nfs-ganesha.org \
--cc=jlayton@kernel.org \
--cc=jra@samba.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-nfs@vger.kernel.org \
--cc=metze@samba.org \
--cc=samba-technical@lists.samba.org \
--cc=slow@samba.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).