Re: [RFC PATCH V2 01/12] fs/stat: Define DAX statx attribute

From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: Ira Weiny <ira.weiny@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>, Jan Kara <jack@suse.cz>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	Dave Chinner <david@fromorbit.com>,
	Christoph Hellwig <hch@lst.de>,
	"Theodore Y. Ts'o" <tytso@mit.edu>,
	linux-ext4 <linux-ext4@vger.kernel.org>,
	linux-xfs <linux-xfs@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: Re: [RFC PATCH V2 01/12] fs/stat: Define DAX statx attribute
Date: Wed, 15 Jan 2020 21:39:35 -0800	[thread overview]
Message-ID: <20200116053935.GB8235@magnolia> (raw)
In-Reply-To: <20200115223821.GG23311@iweiny-DESK2.sc.intel.com>

On Wed, Jan 15, 2020 at 02:38:21PM -0800, Ira Weiny wrote:
> On Wed, Jan 15, 2020 at 12:10:50PM -0800, Dan Williams wrote:
> > On Wed, Jan 15, 2020 at 11:45 AM Ira Weiny <ira.weiny@intel.com> wrote:
> > >
> > > On Wed, Jan 15, 2020 at 09:38:34AM -0800, Darrick J. Wong wrote:
> > > > On Wed, Jan 15, 2020 at 12:37:15PM +0100, Jan Kara wrote:
> > > > > On Fri 10-01-20 11:29:31, ira.weiny@intel.com wrote:
> > > > > > From: Ira Weiny <ira.weiny@intel.com>
> > > > > >
> 
> [snip]
> 
> > > Ok I changed a couple of things as well.  How does this sound?
> > >
> > >
> > > STATX_ATTR_DAX
> > >
> > >         DAX (cpu direct access) is a file mode that attempts to minimize
> > 
> > s/mode/state/?
> 
> DOH!  yes state...  ;-)
> 
> > 
> > >         software cache effects for both I/O and memory mappings of this
> > >         file.  It requires a block device and file system which have
> > >         been configured to support DAX.
> > 
> > It may not require a block device in the future.
> 
> Ok:
> 
> "It requires a file system which has been configured to support DAX." ?
> 
> I'm trying to separate the user of the individual STATX DAX flag from the Admin
> details of configuring the file system and/or devices which supports it.
> 
> Also, I just realized that we should follow the format of the other STATX_*
> attributes.  They all read something like "the file is..."
> 
> So I'm adding that text as well.
> 
> > 
> > >
> > >         DAX generally assumes all accesses are via cpu load / store
> > >         instructions which can minimize overhead for small accesses, but
> > >         may adversely affect cpu utilization for large transfers.
> > >
> > >         File I/O is done directly to/from user-space buffers and memory
> > >         mapped I/O may be performed with direct memory mappings that
> > >         bypass kernel page cache.
> > >
> > >         While the DAX property tends to result in data being transferred
> > >         synchronously, it does not give the same guarantees of
> > >         synchronous I/O where data and the necessary metadata are
> > 
> > Maybe use "O_SYNC I/O" explicitly to further differentiate the 2
> > meanings of "synchronous" in this sentence?
> 
> Done.
> 
> > 
> > >         transferred together.
> > >
> > >         A DAX file may support being mapped with the MAP_SYNC flag,
> > >         which enables a program to use CPU cache flush operations to
> > 
> > s/operations/instructions/
> 
> Done.
> 
> > 
> > >         persist CPU store operations without an explicit fsync(2).  See
> > >         mmap(2) for more information.
> > 
> > I think this also wants a reference to the Linux interpretation of
> > platform "persistence domains" we were discussing that here [1], but
> > maybe it should be part of a "pmem" manpage that can be referenced
> > from this man page.
> 
> Sure, but for now I think referencing mmap for details on MAP_SYNC works.
> 
> I suspect that we may have some word smithing once I get this series in and we
> submit a change to the statx man page itself.  Can I move forward with the
> following for this patch?
> 
> <quote>
> STATX_ATTR_DAX
> 
>         The file is in the DAX (cpu direct access) state.  DAX state

Hmm, now that I see it written out, I <cough> kind of like "DAX mode"
better now. :/

"The file is in DAX (CPU direct access) mode.  DAX mode attempts..."

>         attempts to minimize software cache effects for both I/O and
>         memory mappings of this file.  It requires a file system which
>         has been configured to support DAX.
> 
>         DAX generally assumes all accesses are via cpu load / store
>         instructions which can minimize overhead for small accesses, but
>         may adversely affect cpu utilization for large transfers.
> 
>         File I/O is done directly to/from user-space buffers and memory
>         mapped I/O may be performed with direct memory mappings that
>         bypass kernel page cache.
> 
>         While the DAX property tends to result in data being transferred
>         synchronously, it does not give the same guarantees of
>         synchronous I/O where data and the necessary metadata are
>         transferred together.

(I'm frankly not sure that synchronous I/O actually guarantees that the
metadata has hit stable storage...)

--D

>         A DAX file may support being mapped with the MAP_SYNC flag,
>         which enables a program to use CPU cache flush instructions to
>         persist CPU store operations without an explicit fsync(2).  See
>         mmap(2) for more information.
> </quote>
> 
> Ira
> 
> > 
> > [1]: http://lore.kernel.org/r/20200108064905.170394-1-aneesh.kumar@linux.ibm.com