Re: [LSF/MM TOPIC ATTEND][RFD] ZUFS - Zero-copy User-mode File System

From: Steve French <smfrench@gmail.com>
To: Chuck Lever <chuck.lever@oracle.com>
Cc: Bruce Fields <bfields@fieldses.org>,
	Boaz Harrosh <boazh@netapp.com>,
	lsf-pc@lists.linux-foundation.org,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Ric Wheeler <rwheeler@redhat.com>, Jan Kara <jack@suse.cz>,
	Steven Whitehouse <swhiteho@redhat.com>,
	coughlan@redhat.com, Jeff Moyer <jmoyer@redhat.com>,
	Sage Weil <sweil@redhat.com>,
	Miklos Szeredi <mszeredi@redhat.com>,
	Andy Rudof <andy.rudoff@intel.com>,
	Anna Schumaker <Anna.Schumaker@netapp.com>,
	Amir Goldstein <amir73il@gmail.com>,
	Stephen Bates <stephen@eideticom.com>,
	Amit Golander <Amit.Golander@netapp.com>,
	Sagi Manole <sagim@netapp.com>,
	Shachar Sharon <Shachar.Sharon@netapp.com>,
	Josef Bacik <josef@redhat.com>
Subject: Re: [LSF/MM TOPIC ATTEND][RFD] ZUFS - Zero-copy User-mode File System
Date: Fri, 9 Feb 2018 11:47:57 -0600	[thread overview]
Message-ID: <CAH2r5mv6QPvEe1Py+yLFBd-Ysasbk+QXTMqMWt+GmhVK_onD6g@mail.gmail.com> (raw)
In-Reply-To: <446CAE69-36E0-4F72-85C3-2DCC60B489B6@oracle.com>

On Fri, Feb 2, 2018 at 10:09 AM, Chuck Lever <chuck.lever@oracle.com> wrote:
>
>
>> On Feb 2, 2018, at 10:49 AM, bfields@fieldses.org wrote:
>>
>> On Thu, Feb 01, 2018 at 08:59:18PM +0200, Boaz Harrosh wrote:
>>> On 01/02/18 20:34, Chuck Lever wrote: <>
>>>> This work was also presented at the SNIA Persistent Memory Summit
>>>> last week.  The use case of course is providing a user space
>>>> platform for the development and deployment of memory-based file
>>>> systems. The value-add of this kind of file system is ultra-low
>>>> latency, which is a challenge for the current most popular such
>>>> framework, FUSE.
>>>>
>>>> To start, I can think of three areas where specific questions might
>>>> be entertained by LSF/MM attendees:
>>>>
>>>> - Spectre mitigations make this whole "user space filesystem"
>>>> arrangement even slower, thanks to additional context switches
>>>> between user space and the kernel.
>>
>> I think you're referring to the KPTI patches, which address Meltdown,
>> not Spectre.
>
> I enabled KPTI on my NFS client and server systems in early
> v4.15-rc, and didn't measure a change in latency or throughput.
>
> But with v4.15 final, which includes some Spectre mitigations,
> write(2) on NFS files, for example, takes about 15us longer.
> Since the RPC round-trip times did not increase, I presume this
> extra latency is incurred on the client, where the user-kernel
> boundary transitions occur.
>
> <shrug>

That is interesting data.

A loosely related question is whether ZUFS would be helpful in
a typical NFS or SMB3 scenario (under the server) - especially
with low latency RDMA (SMB3 Direct) connection to the server
(in the case of SMB3, we would want to consider what this would
look like with i/o from the same client, potentially the same file,
coming in on multiple RDMA cards ).

-- 
Thanks,

Steve