Re: [PATCH v3 00/12] Enable per-file/directory DAX operations V3

From: Ira Weiny <ira.weiny@intel.com>
To: Dan Williams <dan.j.williams@intel.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>,
	Jeff Moyer <jmoyer@redhat.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	Dave Chinner <david@fromorbit.com>,
	Christoph Hellwig <hch@lst.de>,
	"Theodore Y. Ts'o" <tytso@mit.edu>, Jan Kara <jack@suse.cz>,
	linux-ext4 <linux-ext4@vger.kernel.org>,
	linux-xfs <linux-xfs@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: Re: [PATCH v3 00/12] Enable per-file/directory DAX operations V3
Date: Fri, 14 Feb 2020 12:06:07 -0800	[thread overview]
Message-ID: <20200214200607.GA18593@iweiny-DESK2.sc.intel.com> (raw)
In-Reply-To: <CAPcyv4hkWoC+xCqicH1DWzmU2DcpY0at_A6HaBsrdLbZ6qzWow@mail.gmail.com>

On Thu, Feb 13, 2020 at 04:16:17PM -0800, Dan Williams wrote:
> On Thu, Feb 13, 2020 at 3:29 PM Ira Weiny <ira.weiny@intel.com> wrote:
> >
> > On Thu, Feb 13, 2020 at 11:58:39AM -0800, Darrick J. Wong wrote:
> > > On Thu, Feb 13, 2020 at 11:05:13AM -0800, Ira Weiny wrote:
> > > > On Thu, Feb 13, 2020 at 11:01:57AM -0800, 'Ira Weiny' wrote:
> > > > > On Wed, Feb 12, 2020 at 02:49:48PM -0500, Jeff Moyer wrote:
> > > > > > Ira Weiny <ira.weiny@intel.com> writes:
> > > > > >
> > > >
> > > > [snip]
> > > >
> > > > > > Given that we document the dax mount
> > > > > > option as "the way to get dax," it may be a good idea to allow for a
> > > > > > user to selectively disable dax, even when -o dax is specified.  Is that
> > > > > > possible?
> > > > >
> > > > > Not with this patch set.  And I'm not sure how that would work.  The idea was
> > > > > that -o dax was simply an override for users who were used to having their
> > > > > entire FS be dax.  We wanted to depreciate the use of "-o dax" in general.  The
> > > > > individual settings are saved so I don't think it makes sense to ignore the -o
> > > > > dax in favor of those settings.  Basically that would IMO make the -o dax
> > > > > useless.
> > > >
> > > > Oh and I forgot to mention that setting 'dax' on the root of the FS basically
> > > > provides '-o dax' functionality by default with the ability to "turn it off"
> > > > for files.
> > >
> > > Please don't further confuse FS_XFLAG_DAX and S_DAX.
> >
> > Yes...  the above text is wrong WRT statx.  But setting the physical
> > XFS_DIFLAG2_DAX flag on the root directory will by default cause all files and
> > directories created there to be XFS_DIFLAG2_DAX and so forth on down the tree
> > unless explicitly changed.  This will be the same as mounting with '-o dax' but
> > with the ability to turn off dax for individual files.  Which I think is the
> > functionality Jeff is wanting.
> 
> To be clear you mean turn off XFS_DIFLAG2_DAX, not mask S_DAX when you
> say "turn off dax", right?

Yes.

[disclaimer: the following assumes the underlying 'device' (superblock)
supports DAX]

... which results in S_DAX == false when the file is opened without the mount
option.  The key would be that all directories/files created under a root with
XFS_DIFLAG2_DAX == true would inherit their flag and be XFS_DIFLAG2_DAX == true
all the way down the tree.  Any file not wanting DAX would need to set
XFS_DIFLAG2_DAX == false.  And setting false could be used on a directory to
allow a user or group to not use dax on files in that sub-tree.

Then without '-o dax' (XFS_MOUNT_DAX == false) all files when opened set S_DAX
equal to XFS_DIFLAG2_DAX value.  (Directories, as of V4, never get S_DAX set.)

If '-o dax' (XFS_MOUNT_DAX == true) then S_DAX is set on all files.

[IF the underlying 'device' (superblock) does _not_ support DAX]

... S_DAX is _never_ set but the underlying XFS_DIFLAG2_DAX flags can be
toggled and will be inherited as above.  Because S_DAX is never set access to
that file will be restricted to "not dax"...[1]

I could go into that level of detail in the doc if needed?  I feel like we need
a more general name for XFS_DIFLAG2_DAX if I do.[2]

> 
> The mount option simply forces "S_DAX" on all regular files as long as
> the underlying device (or soon to be superblock for virtiofs) supports
> it. There is no method to mask S_DAX when the filesystem was mounted
> with -o dax. Otherwise we would seem to need yet another physical flag
> to "always disable" dax.

Exactly.  I don't think we want to support that.  From this thread alone it
seems we have enough complexity and that would be another layer...

;-)

Ira

[1] I'm beginning to think that if I type dax one more time I'm going to go
crazy...  :-P

[2] I have patches in the wings to introduce EXT4_DAX_FL as an ext4 on disk bit
which would be equivalent to XFS_DIFLAG2_DAX.  If anyone wants a better name
let me know.