From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christoph Hellwig Subject: Re: xfs: untangle the direct I/O and DAX path, fix DAX locking Date: Tue, 28 Jun 2016 15:10:59 +0200 Message-ID: <20160628131059.GA30475@lst.de> References: <1466609236-23801-1-git-send-email-hch@lst.de> <20160623232446.GA12670@dastard> <20160624072612.GA22205@lst.de> <20160624230045.GG12670@dastard> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <20160624230045.GG12670@dastard> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org Sender: "Linux-nvdimm" To: Dave Chinner Cc: linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org, Christoph Hellwig , xfs-VZNHf3L845pBDgjK7y7TUQ@public.gmane.org List-Id: linux-nvdimm@lists.01.org On Sat, Jun 25, 2016 at 09:00:45AM +1000, Dave Chinner wrote: > > > > Sorry, but this is simply broken - allowing apps to opt-in behavior > > (e.g. like we're using O_DIRECT) is always fine. Requriring > > filesystem-specific tuning that has affect outside the app to get > > existing documented behavior is not how to design APIs. > > Using DAX is an *admin decision*, not an application decision. Of course - that's exactly my point. > Indeed, it's a mount option right now, and that's most definitely not > something the application can turn on or off! Inode flags allow the > admin to decide that two apps working on the same filesystem can use > (or not use) DAX independently, rather than needing to put them on > different filesystems. Right. And an existing application can get DAX turned on under its back, and will now suddently get different synchronization behavior. That is if it's writes happen to be aligned to the fs block size. > > Maybe we'll need to opt-in to use DAX for mmap, but giving the same > > existing behavior for read and write and avoiding a copy to the pagecache > > is an obvious win. > > You can't use DAX just for mmap. It's an inode scope behaviour - > once it's turned on, all accesses to that inode - regardless of user > interface - must use DAX. It's all or nothing, not a per file > descript/mmap context option. Right now it is. But when discussing mmap behavior one option was to require an opt-in to get DAX-specific mmap semantics. For plain read/write we have no such option and thus absolutely need to behave as all normal reads and writes behave. If you think the exclusive lock for writes hurts we have two options: a) implement range locks (although they might be more expensive for typical loads) b) add a new O_* or RWF_* option to not require the synchronization for apps that don't want it. Neither of those cases really is DAX-specific. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from verein.lst.de ([213.95.11.211]:50539 "EHLO newverein.lst.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752071AbcF1NLf (ORCPT ); Tue, 28 Jun 2016 09:11:35 -0400 Date: Tue, 28 Jun 2016 15:10:59 +0200 From: Christoph Hellwig To: Dave Chinner Cc: Christoph Hellwig , xfs@oss.sgi.com, linux-fsdevel@vger.kernel.org, linux-nvdimm@ml01.01.org Subject: Re: xfs: untangle the direct I/O and DAX path, fix DAX locking Message-ID: <20160628131059.GA30475@lst.de> References: <1466609236-23801-1-git-send-email-hch@lst.de> <20160623232446.GA12670@dastard> <20160624072612.GA22205@lst.de> <20160624230045.GG12670@dastard> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160624230045.GG12670@dastard> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Sat, Jun 25, 2016 at 09:00:45AM +1000, Dave Chinner wrote: > > > > Sorry, but this is simply broken - allowing apps to opt-in behavior > > (e.g. like we're using O_DIRECT) is always fine. Requriring > > filesystem-specific tuning that has affect outside the app to get > > existing documented behavior is not how to design APIs. > > Using DAX is an *admin decision*, not an application decision. Of course - that's exactly my point. > Indeed, it's a mount option right now, and that's most definitely not > something the application can turn on or off! Inode flags allow the > admin to decide that two apps working on the same filesystem can use > (or not use) DAX independently, rather than needing to put them on > different filesystems. Right. And an existing application can get DAX turned on under its back, and will now suddently get different synchronization behavior. That is if it's writes happen to be aligned to the fs block size. > > Maybe we'll need to opt-in to use DAX for mmap, but giving the same > > existing behavior for read and write and avoiding a copy to the pagecache > > is an obvious win. > > You can't use DAX just for mmap. It's an inode scope behaviour - > once it's turned on, all accesses to that inode - regardless of user > interface - must use DAX. It's all or nothing, not a per file > descript/mmap context option. Right now it is. But when discussing mmap behavior one option was to require an opt-in to get DAX-specific mmap semantics. For plain read/write we have no such option and thus absolutely need to behave as all normal reads and writes behave. If you think the exclusive lock for writes hurts we have two options: a) implement range locks (although they might be more expensive for typical loads) b) add a new O_* or RWF_* option to not require the synchronization for apps that don't want it. Neither of those cases really is DAX-specific. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 348927CA1 for ; Tue, 28 Jun 2016 08:11:08 -0500 (CDT) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id E73A2304032 for ; Tue, 28 Jun 2016 06:11:04 -0700 (PDT) Received: from newverein.lst.de (verein.lst.de [213.95.11.211]) by cuda.sgi.com with ESMTP id 26jRHzfNwsjJ0bu0 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 28 Jun 2016 06:11:02 -0700 (PDT) Date: Tue, 28 Jun 2016 15:10:59 +0200 From: Christoph Hellwig Subject: Re: xfs: untangle the direct I/O and DAX path, fix DAX locking Message-ID: <20160628131059.GA30475@lst.de> References: <1466609236-23801-1-git-send-email-hch@lst.de> <20160623232446.GA12670@dastard> <20160624072612.GA22205@lst.de> <20160624230045.GG12670@dastard> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20160624230045.GG12670@dastard> List-Id: XFS Filesystem from SGI List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: xfs-bounces@oss.sgi.com Sender: xfs-bounces@oss.sgi.com To: Dave Chinner Cc: linux-fsdevel@vger.kernel.org, linux-nvdimm@ml01.01.org, Christoph Hellwig , xfs@oss.sgi.com On Sat, Jun 25, 2016 at 09:00:45AM +1000, Dave Chinner wrote: > > > > Sorry, but this is simply broken - allowing apps to opt-in behavior > > (e.g. like we're using O_DIRECT) is always fine. Requriring > > filesystem-specific tuning that has affect outside the app to get > > existing documented behavior is not how to design APIs. > > Using DAX is an *admin decision*, not an application decision. Of course - that's exactly my point. > Indeed, it's a mount option right now, and that's most definitely not > something the application can turn on or off! Inode flags allow the > admin to decide that two apps working on the same filesystem can use > (or not use) DAX independently, rather than needing to put them on > different filesystems. Right. And an existing application can get DAX turned on under its back, and will now suddently get different synchronization behavior. That is if it's writes happen to be aligned to the fs block size. > > Maybe we'll need to opt-in to use DAX for mmap, but giving the same > > existing behavior for read and write and avoiding a copy to the pagecache > > is an obvious win. > > You can't use DAX just for mmap. It's an inode scope behaviour - > once it's turned on, all accesses to that inode - regardless of user > interface - must use DAX. It's all or nothing, not a per file > descript/mmap context option. Right now it is. But when discussing mmap behavior one option was to require an opt-in to get DAX-specific mmap semantics. For plain read/write we have no such option and thus absolutely need to behave as all normal reads and writes behave. If you think the exclusive lock for writes hurts we have two options: a) implement range locks (although they might be more expensive for typical loads) b) add a new O_* or RWF_* option to not require the synchronization for apps that don't want it. Neither of those cases really is DAX-specific. _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs