Re: [PATCH] xfs: use a unique and persistent value for f_fsid

[Amir Goldstein <amir73il@gmail.com>]

On Tue, Mar 23, 2021 at 8:35 AM Darrick J. Wong <djwong@kernel.org> wrote:
>
> On Tue, Mar 23, 2021 at 06:50:44AM +0200, Amir Goldstein wrote:
> > On Tue, Mar 23, 2021 at 1:03 AM Dave Chinner <david@fromorbit.com> wrote:
> > >
> > > On Mon, Mar 22, 2021 at 07:11:18PM +0200, Amir Goldstein wrote:
> > > > Some filesystems on persistent storage backend use a digest of the
> > > > filesystem's persistent uuid as the value for f_fsid returned by
> > > > statfs(2).
> > > >
> > > > xfs, as many other filesystem provide the non-persistent block device
> > > > number as the value of f_fsid.
> > > >
> > > > Since kernel v5.1, fanotify_init(2) supports the flag FAN_REPORT_FID
> > > > for identifying objects using file_handle and f_fsid in events.
> > >
> > > The filesystem id is encoded into the VFS filehandle - it does not
> > > need some special external identifier to identify the filesystem it
> > > belongs to....
> > >
> >
> > Let's take it from the start.
> > There is no requirement for fanotify to get a persistent fs id, we just need
> > a unique fs id that is known to userspace, so the statfs API is good enough
> > for our needs.
> >
> > See quote from fanotify.7:
> >
> > " The fields of the fanotify_event_info_fid structure are as follows:
> > ...
> >        fsid   This  is  a  unique identifier of the filesystem
> > containing the object associated with the event.  It is a structure of
> > type __kernel_fsid_t and contains the same value as f_fsid when
> >               calling statfs(2).
> >
> >        file_handle
> >               This is a variable length structure of type struct
> > file_handle.  It is an opaque handle that corresponds to a specified
> > object on a filesystem as returned by name_to_handle_at(2).  It
> >               can  be  used  to uniquely identify a file on a
> > filesystem and can be passed as an argument to open_by_handle_at(2).
>
> Hmmmm.... so I guess you'd /like/ a file handle that will survive across
> unmount/mount cycles, and possibly even a reboot?
>
> I looked at the first commit, and I guess you use name_to_handle_at,
> which returns a mount_id that is .... that weird number in the leftmost
> column of /proc/mountinfo, which increments monotonically for each mount
> and definitely doesn't survive a remount cycle, let alone a reboot?
>
> Hence wanting to use something less volatile than mnt_id_ida...?
>
> My natural inclination is "just use whatever NFS does", but ... then I
> saw fh_compose and realized that the fsid part of an NFS handle depends
> on the export options and probably isn't all that easy for someone who
> isn't an nfs client to extract.
>
> Was that how you arrived at using the statfs fsid field?

Yes. Exactly.

>
> ...except XFS doesn't guarantee that fsid is particularly unique or
> stable, since a reboot can enumerate blockdevs in a different order and
> hence the dev_t will change.  UUIDs also aren't a great idea because you
> can snapshot an fs and mount it with nouuid, and now a "unique" file
> handle can map ambiguously to two different files.
>

As I explained in my reply to Dave, that's not a big issue.
If program want to listen on events from multiple filesystems,
the program will sample f_fsid of both filesystems before setting up the
filesystem watches. If there is a collision, there are other ways to handle
this case (i.e. run two separate listener programs).

Also, as I wrote to Dave, I can easily handle the special case of "-o nouuid"
by leaving the bdev number for f_fsid in that case.

> Urgh, I'm gonna have to think about this one, all the options suck.
> fanotify might be smart enough to handle ambiguous file handles but now

Actually, fanotify is copletely dumb about this, it's the user of fanotify that
needs to be able to do something useful with this information.

There is a demo I wrote based on inotifywatch that demonstrates that [1].

Thanks,
Amir.

[1] https://github.com/amir73il/inotify-tools/commits/fanotify_name_fid