Re: [PATCH v1 0/4] setting uuid of online filesystems

[Amir Goldstein <amir73il@gmail.com>]

On Sat, Mar 18, 2023 at 2:24 AM Darrick J. Wong <djwong@kernel.org> wrote:
>
> On Tue, Mar 14, 2023 at 05:28:47PM +1100, Dave Chinner wrote:
> > On Mon, Mar 13, 2023 at 09:21:05PM -0700, Catherine Hoang wrote:
> > > Hi all,
> > >
> > > This series of patches implements a new ioctl to set the uuid of mounted
> > > filesystems. Eventually this will be used by the 'xfs_io fsuuid' command
> > > to allow userspace to update the uuid.
> > >
> > > Comments and feedback appreciated!
> >
> > What's the use case for this?
> >
> > What are the pro's and cons of the implementation?
> >
> > Some problems I see:
> >
> > * How does this interact with pNFS exports (i.e.
> > CONFIG_EXPORTFS_BLOCK_OPS). XFS hands the sb_uuid directly to pNFS
> > server (and remote clients, I think) so that incoming mapping
> > requests can be directed to the correct block device hosting the XFS
> > filesystem the mapping information is for. IIRC, the pNFS data path
> > is just given a byte offset into the device where the UUID in the
> > superblock lives, and if it matches it allows the remote IO to go
> > ahead - it doesn't actually know that there is a filesystem on that
> > device at all...
>
> I think we're going to have to detect the presence of pNFS exports and
> EAGAIN if there are any active.  That probably involves incrementing a
> counter during ->map_blocks and decreasing it during ->commit blocks.
>
> That might still invite races between someone setting the fsuuid and
> exporting via NFS.
>
> > * IIRC, the nfsd export table can also use the filesystem uuid to
> > identify the filesystem being exported, and IIRC that gets encoded
> > in the filehandle. Does changing the filesystem UUID then cause
> > problems with export indentification and/or file handle
> > creation/resolution?
>
> I thought NFS (when not being used for block layouts and pnfs stuff)
> still used the fsid that's also in the superblock?  Presumably we'd
> leave the old m_fixedfsid unchanged while the fs is still mounted, and
> document the caveat that handles will not work after a remount.
>
> On some level, we might simply have to document that changing the
> user-visible uuid will break file handles and pnfs exports, so sysadmins
> had better get that done early.
>
> OTOH that is an argument for leaving the xfs_db-based version that we
> have now.
>
> > * Is the VFS prepared for sb->s_uuid changing underneath running
> > operations on mounted filesystems? I can see that this might cause
> > problems with any sort of fscrypt implementation as it may encode
> > the s_uuid into encryption keys held in xattrs, similarly IMA and
> > EVM integrity protection keys.
>
> I would hope it's not so hard to detect that EVM/fscrypt are active on a
> given xfs mount.  EVM seems pretty hard to detect since it operates
> above the fs.
>
> fscrypt might not be so difficult, since we likely need to add support
> in the ondisk metadata, which means xfs will know.
>
> IMA seems to be using it for rule matching, so I'd say that if the
> sysadmin changes the fsuuid, they had better update the IMA rulebook
> too.
>
> Come to think of it, perhaps reading a filesystem's uuid (whether via
> handle export, direct read of s_uuid, nfs, evm, ima, or fscrypt) should
> set a superblock flag that someone has accessed it; and only if nobody's
> yet accessed it do we allow the fsuuid update?
>
> > * Should the VFS superblock sb->s_uuid use the XFS
> > sb->sb_meta_uuid value and never change because we can encode it
> > into other objects stored in the active filesystem? What does that
> > mean for tools that identify block devices or filesystems by the VFS
> > uuid rather than the filesystem uuid?
>
> I don't know of any other than the ones you found.

FWIW, overlayfs also uses s_uuid in a similar manner to nfsd
as a persistent reference to the origin lower file after copy up.

Like nfsd, the overlayfs persistent origin handle (fsuuid+filehandle) will break
when changing lower fs uuid offline, but I don't see much difference from
changing s_uuid online - worse case that I can think of is that an overlayfs
inode will change its st_ino after evict/lookup on a mounted overlayfs.

Thanks,
Amir.