Re: Overlayfs with writable lower layer

[Amir Goldstein <amir73il@gmail.com>]

On Tue, Sep 13, 2022 at 11:33 PM Hao Luo <haoluo@google.com> wrote:
>
> On Tue, Sep 13, 2022 at 11:54 AM Amir Goldstein <amir73il@gmail.com> wrote:
> > OK. IIUC, you have upper fs files only in the root dir?
>
> Sorry, no, the upper fs files need to be in subdir.
>
> > And the lower root dir has only subdirs?
>
> There could be files.
>

And assuming that those files are cgroupfs files, why
did you say there is no need to write to those files?

I seem to recall that was an important distinction from
standard overlayfs when you described the problem in LSFMM.

> > Can you give a small example of an upper a lower and their
> > union trees just for the sake of discussion?
> >
>
> For example, assume lower has the following layout:
> $ tree lower
> .
> └── A
>     ├── B
>     │   └── lower
>     └── lower
>
> I can't create files in the fs in the lower.
> $ touch A/B/file
> touch: cannot touch 'A/B/file': Permission denied
>
> The upper is initially empty.
>
> I would like to overlay a writable fs on top of lower, so the union
> tree looks like
> $ tree union
> .
> └── A
>     ├── B
>     │   └── lower
>     └── lower
> $ touch A/B/file
> $ tree union
> .
> └── A
>     ├── B
>     │   ├── file
>     │   └── lower2
>     └── lower1
>
> Here, 'file' exists in the upper.
>

So B is now called a "merged" dir - it is not a "pure" dir
anymore because it contains both upper and lower files.

Normally in overlayfs before creating 'file' in upper,
the hierarchy A/B/ needs to be created in upper fs
to contain the file.

Unless your upper fs automagically has the same
dirs hierarchy as the lower fs?

You should know that overlayfs does more than just
mkdir("A");mkdir("A/B")
it created tmp dirs, sets xattrs and attrs on them and moves
them into place.
I am not sure if you planned to support all those operations
in your upper fs?

There are probably some other limitations at the moment
related to pseudo filesystems that prevent them from being
used as upper and/or lower fs in overlayfs.

We will need to check what those limitations are and whether
those limitations could be lifted for your specific use case.

> Further, directory B could disappear from lower. When that happens, I
> think there are two possible behaviors:
>  - make 'file' disappear from union as well;
>  - make 'file' and its directory accessible as well.
>
> In behavior 1, it will look like
> $ tree union
> .
> └── A
>     └── lower1
>
> In behavior 2, it will look like
> $ tree union
> .
> └── A
>     ├── B
>     │   └── file
>     └── lower1
>
> IMHO, behavior 1 works better in my use case. But if the FS experts
> think behavior 2 makes more sense, I can work around.
>

Something that I always wanted to try is to get rid of the duplicated
upper fs hierarchy.

It's a bit complicated to explain the details, but if your use case
does not involve any directory renames(?), then the upper path
for the merge directories can be index based and not hierarchical.

IOW:
- union tree lookup/traversal by path is performed on the lower fs
- upper fs is traversed by name only for pure upper dirs
- merged dirs are found by index of lower dir inode

The result will be behavior 1 that you wanted

I have some other use cases that might benefit from this mode.

> >
> > If that is all then it sounds pretty simple.
> > It could be described something like this:
> > 1. merged directories cannot appear/disappear
> > 2. lower pure directories can appear/disappear
> > 3. upper files/dirs can be created inside merge dirs and pure upper dirs
> >
> > I think I have some patches that could help with #2.
> >
>
> These three semantics looks good to me.
>

Except the case about disappearing B that you just described
above breaks rule #1 ;)
so other semantics are needed.

I will need some more clarifications about your use case to
understand if what I have in mind could help your use case.

Thanks,
Amir.