Re: [PATCH] xfs: dax: facilitate EXPERIMENTAL warning for dax=inode case

[Bill O'Donnell <billodo@redhat.com>]

On Fri, Aug 27, 2021 at 10:35:56AM -0500, Eric Sandeen wrote:
> On 8/27/21 9:25 AM, Bill O'Donnell wrote:
> > On Fri, Aug 27, 2021 at 09:18:32AM -0500, Eric Sandeen wrote:
> > > On 8/27/21 9:03 AM, Bill O'Donnell wrote:
> > > > On Thu, Aug 26, 2021 at 06:43:44PM -0500, Eric Sandeen wrote:
> > > > > On 8/26/21 5:08 PM, Bill O'Donnell wrote:
> > > > > > On Thu, Aug 26, 2021 at 01:16:22PM -0500, Eric Sandeen wrote:
> > > > > > > 
> > > > > > > On 8/26/21 1:09 PM, Darrick J. Wong wrote:
> > > > > > > > On Thu, Aug 26, 2021 at 12:30:12PM -0500, Bill O'Donnell wrote:
> > > > > > > 
> > > > > > > > > @@ -1584,7 +1586,7 @@ xfs_fs_fill_super(
> > > > > > > > >      	if (xfs_has_crc(mp))
> > > > > > > > >      		sb->s_flags |= SB_I_VERSION;
> > > > > > > > > -	if (xfs_has_dax_always(mp)) {
> > > > > > > > > +	if (xfs_has_dax_always(mp) || xfs_has_dax_inode(mp)) {
> > > > > > > > 
> > > > > > > > Er... can't this be done without burning another feature bit by:
> > > > > > > > 
> > > > > > > > 	if (xfs_has_dax_always(mp) || (!xfs_has_dax_always(mp) &&
> > > > > > > > 				       !xfs_has_dax_never(mp))) {
> > > > > > > > 		...
> > > > > > > > 		xfs_warn(mp, "DAX IS EXPERIMENTAL");
> > > > > > > > 	}
> > > > > > > 
> > > > > > > changing this conditional in this way will also fail dax=inode mounts on
> > > > > > > reflink-capable (i.e. default) filesystems, no?
> > > > > > 
> > > > > > Correct. My original patch tests fine, and still handles the reflink and dax
> > > > > > incompatibility. The new suggested logic is problematic.
> > > > > > -Bill
> > > > > 
> > > > > I think that both your proposed patch and Darrick's suggestion have this problem.
> > > > > 
> > > > > "mount -o dax=inode" makes your new xfs_has_dax_inode(mp) true, and in that
> > > > > conditional, if the filesystem has reflink enabled, mount fails:
> > > > > 
> > > > > # mkfs.xfs -f /dev/pmem0p1
> > > > > meta-data=/dev/pmem0p1           isize=512    agcount=4, agsize=4194304 blks
> > > > >            =                       sectsz=4096  attr=2, projid32bit=1
> > > > >            =                       crc=1        finobt=1, sparse=1, rmapbt=0
> > > > >            =                       reflink=1    bigtime=0 inobtcount=0
> > > > > data     =                       bsize=4096   blocks=16777216, imaxpct=25
> > > > >            =                       sunit=0      swidth=0 blks
> > > > > naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
> > > > > log      =internal log           bsize=4096   blocks=8192, version=2
> > > > >            =                       sectsz=4096  sunit=1 blks, lazy-count=1
> > > > > realtime =none                   extsz=4096   blocks=0, rtextents=0
> > > > > 
> > > > > # mount -o dax=inode /dev/pmem0p1 /mnt/test
> > > > > mount: wrong fs type, bad option, bad superblock on /dev/pmem0p1,
> > > > >          missing codepage or helper program, or other error
> > > > > 
> > > > >          In some cases useful info is found in syslog - try
> > > > >          dmesg | tail or so.
> > > > > 
> > > > > # dmesg | tail -n 2
> > > > > [  192.691733] XFS (pmem0p1): DAX enabled. Warning: EXPERIMENTAL, use at your own risk
> > > > > [  192.700300] XFS (pmem0p1): DAX and reflink cannot be used together!
> > > > > 
> > > > 
> > > > So, the "DAX enabled" is a misnomer in this case. However the incompatibility of DAX and reflink is
> > > > reflected in the next message, and indeed the mount fails. Is it now a matter of fixing
> > > > the message output so as not to indicate "DAX enabled..."?
> > > 
> > > The mount should not fail, and it does not fail prior to your change.
> > > 
> > > In the past, we did not allow any mixing of a reflink-capable
> > > filesystem with dax in any way.  Now, with per-inode dax, dax-enabled inodes and
> > > reflink-enabled inodes can exist on the same filesystem, you just cannot have an
> > > inode which is both dax-enabled and reflinked at the same time.
> > 
> > Ah. I missed that nuance. I had thought the incompatibility was
> > absolute. :/
> > 
> > The manpage for mkfs.xfs may need updating for the inode mode
> > (unless mine is old):
> > ----------------snip------------------
> > "Note:  the  filesystem DAX mount option ( -o dax ) is incom‐
> > patible  with  reflink-enabled  XFS  filesystems.   To   use
> > filesystem  DAX with XFS, specify the -m reflink=0 option to
> > mkfs.xfs to disable the reflink feature."
> > -------------------------------------
> 
> Hm, looks like the xfs(5) manpage got updated, but it seems mkfs.xfs(8) did not.
> 
>        dax=value
>               Set  CPU  direct  access (DAX) behavior for the current filesystem.
>               This mount option accepts the following values:
> 
>               "dax=inode"  DAX  will  be  enabled  only  on  regular  files  with
>               FS_XFLAG_DAX applied.
> 
>               "dax=never"  DAX  will  not  be enabled for any files. FS_XFLAG_DAX
>               will be ignored.
> 
>               "dax=always" DAX will be enabled for all regular files,  regardless
>               of the FS_XFLAG_DAX state.
> 
>               If  no  option  is  used when mounting a filesystem stored on a DAX
>               capable device, dax=inode will be used as default.

The documentation here, https://www.kernel.org/doc/Documentation/filesystems/dax.txt
adds to the confusion.
   "-o dax"        is a legacy option which is an alias for "dax=always".
		    This may be removed in the future so "-o dax=always" is
		    the preferred method for specifying this behavior.



> 
>               For details regarding DAX behavior in kernel, please refer to  ker‐
>               nel's documentation
> 
> I'll send a patch to fix up the mkfs manpage, thanks.
> 
> Thanks,
> -Eric
>