Re: [PATCH v3 00/12] Enable per-file/directory DAX operations V3

From: Ira Weiny <ira.weiny@intel.com>
To: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Jeff Moyer <jmoyer@redhat.com>,
	linux-kernel@vger.kernel.org,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	Dan Williams <dan.j.williams@intel.com>,
	Dave Chinner <david@fromorbit.com>,
	Christoph Hellwig <hch@lst.de>,
	"Theodore Y. Ts'o" <tytso@mit.edu>, Jan Kara <jack@suse.cz>,
	linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org,
	linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH v3 00/12] Enable per-file/directory DAX operations V3
Date: Thu, 13 Feb 2020 15:29:24 -0800	[thread overview]
Message-ID: <20200213232923.GC22854@iweiny-DESK2.sc.intel.com> (raw)
In-Reply-To: <20200213195839.GG6870@magnolia>

On Thu, Feb 13, 2020 at 11:58:39AM -0800, Darrick J. Wong wrote:
> On Thu, Feb 13, 2020 at 11:05:13AM -0800, Ira Weiny wrote:
> > On Thu, Feb 13, 2020 at 11:01:57AM -0800, 'Ira Weiny' wrote:
> > > On Wed, Feb 12, 2020 at 02:49:48PM -0500, Jeff Moyer wrote:
> > > > Ira Weiny <ira.weiny@intel.com> writes:
> > > > 
> >  
> > [snip]
> > 
> > > > Given that we document the dax mount
> > > > option as "the way to get dax," it may be a good idea to allow for a
> > > > user to selectively disable dax, even when -o dax is specified.  Is that
> > > > possible?
> > > 
> > > Not with this patch set.  And I'm not sure how that would work.  The idea was
> > > that -o dax was simply an override for users who were used to having their
> > > entire FS be dax.  We wanted to depreciate the use of "-o dax" in general.  The
> > > individual settings are saved so I don't think it makes sense to ignore the -o
> > > dax in favor of those settings.  Basically that would IMO make the -o dax
> > > useless.
> > 
> > Oh and I forgot to mention that setting 'dax' on the root of the FS basically
> > provides '-o dax' functionality by default with the ability to "turn it off"
> > for files.
> 
> Please don't further confuse FS_XFLAG_DAX and S_DAX.

Yes...  the above text is wrong WRT statx.  But setting the physical
XFS_DIFLAG2_DAX flag on the root directory will by default cause all files and
directories created there to be XFS_DIFLAG2_DAX and so forth on down the tree
unless explicitly changed.  This will be the same as mounting with '-o dax' but
with the ability to turn off dax for individual files.  Which I think is the
functionality Jeff is wanting.

>
> They are two
> separate flags with two separate behaviors:
> 
> FS_XFLAG_DAX is a filesystem inode metadata flag.
> 
> Setting FS_XFLAG_DAX on a directory causes all files and directories
> created within that directory to inherit FS_XFLAG_DAX.
> 
> Mounting with -o dax causes all files and directories created to have
> FS_XFLAG_DAX set regardless of the parent's status.

I don't believe this is true, either before _or_ after this patch set.

'-o dax' only causes XFS_MOUNT_DAX to be set which then cause S_DAX to be set.
It does not affect FS_XFLAG_DAX.  This is important because we don't want '-o
dax' to suddenly convert all files to DAX if '-o dax' is not used.

> 
> The FS_XFLAG_DAX can be get and set via the fs[g]etxattr ioctl.

Right statx was the wrong tool...

fs[g|s]etattr via the xfs_io -c 'chatttr|lsattr' is the correct tool.

> 
> -------
> 
> S_DAX is the flag that controls the IO path in the kernel for a given
> inode.
> 
> Loading a file inode into the kernel (via _iget) with FS_XFLAG_DAX set
> or creating a file inode that inherits FS_XFLAG_DAX causes the incore
> inode to have the S_DAX flag set if the storage device supports it.

Yes after reworking "Clean up DAX support check" I believe I've got it correct
now.  Soon to be in V4.

> 
> Files with S_DAX set use the dax IO paths through the kernel.
> 
> The S_DAX flag can be queried via statx.

Yes as a verification that the file is at that moment operating as dax.  It
will not return true for a directory ever.  My bad for saying that.  Sorry I
got my tools flags mixed up...

Ira