Re: [RFC][PATCH 3/3] ovl: use persistent s_uuid with index=on

[Amir Goldstein <amir73il@gmail.com>]

On Thu, Jul 6, 2023 at 10:19 AM Amir Goldstein <amir73il@gmail.com> wrote:
>
> On Tue, Apr 25, 2023 at 4:22 PM Amir Goldstein <amir73il@gmail.com> wrote:
> >
> > With index=on, overlayfs instances are non-migratable, meaning that
> > the layers cannot be copied without breaking the index.
> >
> > So when indexdir exists, store a persistent uuid in xattr on the
> > indexdir to give the overlayfs instance a persistent identifier.
> >
> > This also makes f_fsid persistent and more reliable for reporting
> > fid info in fanotify events.
> >
> > With mount option uuid=nogen, a persistent uuid is not be initialized
> > on indexdir, but if a persistent uuid already exists, it will be used.
> >
>
> This behavior (along with the grammatical mistakes) was changed in
> https://github.com/amir73il/linux/commits/ovl_encode_fid
>
> uuid=off or uuid=null both set ovl fsid to null regardless of persistent
> uuid xattr.
>

Sorry, that was meant to say "set ovl uuid to null..."
when ovl uuid is null then ovl fsid is not null, it is the fsid of the
uppermost fs.

This creates a dilemma wrt backward compat.

With index=off, the mounter has a choice between two sub-optimal options:
1. persistent ovl fsid (of upper fs)
2. unique ovl fsid (from random uuid)

If we change the default from legacy (1) to unique (2), that
could also break systems that rely on the persistent ovl fsid
of existing overlayfs layers.

With index=on, the choice is between:
1. persistent ovl fsid (of upper fs)
2. persistent and unique ovl fsid (from uuid xattr)

option (2) is superior, but still could break existing systems
that rely on (1) being persistent.

The decision to tie uuid xattr to the index dir and index=on
was rationalized in the commit message, but persistent and
unique fsid could also be implemented regardless of index=on.

I think I may have found a dignified way out of this mess:
- In ovl_fill_super(), check impure xattr on upper root dir
- If impure xattr does not exist (very likely new overlay),
  uuid_gen() and write the persistent uuid xattr on upper fs root
- If uuid xattr is written or already exists, use that to initialize
  s_uuid otherwise, leave it null
- in ovl_statfs(), override the upper fs fsid, only if ovl uuid is non-null

This gives:
1. Old overlayfs deployments retain old behavior wrt null uuid
    and upper fsid, as long as they have had at least one subdir
    of root copied up or looked up to trigger ovl_fix_origin()
2. New overlayfs deployments always generate and use a unique
    and persistent uuid/fsid
3. mount option uuid=off/null (*) can be used to retain legacy behavior
    on old/new overlayfs deployments (for whatever reason) and ignore
    existing persistent uuid xattr
4. mount option uuid=on can be used to force new behavior on an
    existing overlayfs with impure xattr and without uuid xattr

(*) uuid=off was originally introduced for the use case of copied layers.
     That is similar to the use case of copying disk images and dropping
     the old persistent ovl uuid makes sense in that case.

I will try to write this up.

Thanks,
Amir.