Re: [RFC][PATCH 3/3] ovl: use persistent s_uuid with index=on

[Amir Goldstein <amir73il@gmail.com>]

On Thu, Jul 6, 2023 at 1:14 PM Amir Goldstein <amir73il@gmail.com> wrote:
>
> On Thu, Jul 6, 2023 at 10:19 AM Amir Goldstein <amir73il@gmail.com> wrote:
> >
> > On Tue, Apr 25, 2023 at 4:22 PM Amir Goldstein <amir73il@gmail.com> wrote:
> > >
> > > With index=on, overlayfs instances are non-migratable, meaning that
> > > the layers cannot be copied without breaking the index.
> > >
> > > So when indexdir exists, store a persistent uuid in xattr on the
> > > indexdir to give the overlayfs instance a persistent identifier.
> > >
> > > This also makes f_fsid persistent and more reliable for reporting
> > > fid info in fanotify events.
> > >
> > > With mount option uuid=nogen, a persistent uuid is not be initialized
> > > on indexdir, but if a persistent uuid already exists, it will be used.
> > >
> >
> > This behavior (along with the grammatical mistakes) was changed in
> > https://github.com/amir73il/linux/commits/ovl_encode_fid
> >
> > uuid=off or uuid=null both set ovl fsid to null regardless of persistent
> > uuid xattr.
> >
>
> Sorry, that was meant to say "set ovl uuid to null..."
> when ovl uuid is null then ovl fsid is not null, it is the fsid of the
> uppermost fs.
>
> This creates a dilemma wrt backward compat.
>
> With index=off, the mounter has a choice between two sub-optimal options:
> 1. persistent ovl fsid (of upper fs)
> 2. unique ovl fsid (from random uuid)
>
> If we change the default from legacy (1) to unique (2), that
> could also break systems that rely on the persistent ovl fsid
> of existing overlayfs layers.
>
> With index=on, the choice is between:
> 1. persistent ovl fsid (of upper fs)
> 2. persistent and unique ovl fsid (from uuid xattr)
>
> option (2) is superior, but still could break existing systems
> that rely on (1) being persistent.
>
> The decision to tie uuid xattr to the index dir and index=on
> was rationalized in the commit message, but persistent and
> unique fsid could also be implemented regardless of index=on.
>
> I think I may have found a dignified way out of this mess:
> - In ovl_fill_super(), check impure xattr on upper root dir
> - If impure xattr does not exist (very likely new overlay),
>   uuid_gen() and write the persistent uuid xattr on upper fs root
> - If uuid xattr is written or already exists, use that to initialize
>   s_uuid otherwise, leave it null
> - in ovl_statfs(), override the upper fs fsid, only if ovl uuid is non-null
>
> This gives:
> 1. Old overlayfs deployments retain old behavior wrt null uuid
>     and upper fsid, as long as they have had at least one subdir
>     of root copied up or looked up to trigger ovl_fix_origin()
> 2. New overlayfs deployments always generate and use a unique
>     and persistent uuid/fsid
> 3. mount option uuid=off/null (*) can be used to retain legacy behavior
>     on old/new overlayfs deployments (for whatever reason) and ignore
>     existing persistent uuid xattr
> 4. mount option uuid=on can be used to force new behavior on an
>     existing overlayfs with impure xattr and without uuid xattr
>
> (*) uuid=off was originally introduced for the use case of copied layers.
>      That is similar to the use case of copying disk images and dropping
>      the old persistent ovl uuid makes sense in that case.
>
> I will try to write this up.
>

OK, this is what I got in overlayfs.rst:

UUID and fsid
-------------

The UUID of overlayfs instance itself and the fsid reported by statfs(2) are
controlled by the "uuid" mount option, which supports these values:

- "null":
    UUID of overlayfs in null, fsid is taken from upper most fs.
- "off":
    UUID of overlayfs in null, fsid is taken from upper most fs
    and UUID of underlying layers not checked.
- "on":
    UUID of overlayfs in generated on first mount used to report a unique fsid.
    If upper filesystem supports xattrs, the UUID is stored in xattr
    "trusted.overlay.uuid", making the fsid unique and persistent.
- "auto": (default)
    Upgrade to "uuid=on" on first time mount of new overlay filesystem.
    Downgrade to "uuid=null" for existing overlay filesystems that were never
    mounted with "uuid=on".

Pushed to:
https://github.com/amir73il/linux/commits/ovl_encode_fid

Will post next week.

Thanks,
Amir.