Re: [PATCH 0/2] overlayfs: C/R enhancements

[Alexander Mikhalitsyn <alexander.mikhalitsyn@virtuozzo.com>]

On Fri, 5 Jun 2020 17:36:10 +0300
Amir Goldstein <amir73il@gmail.com> wrote:

> > > While at it, you copy pasted the text:
> > >           For more information, see Documentation/filesystems/overlayfs.txt
> > > but there is no more information to be found.
> >
> > As far as I know documentation patches must be send to another mailing list.
> > Of course I have plan to add information to overlayfs documentation about new feature.
> >
> 
> Please send documentation patch together with the series
> to this list. its fine to wait with that until the concept is approved though.
> 

Yep, I will prepare patch and send.

> 
> > > > > And if this works for you, you don't have to export the layers ovl_fh in
> > > > > /proc/mounts, you can export them in numerous other ways.
> > > > > One way from the top of my head, getxattr on overlay root dir.
> > > > > "trusted.overlay" xattr is anyway a reserved prefix, so "trusted.overlay.layers"
> > > > > for example could work.
> > > >
> > > > Thanks xattr might be a good option, but still don't forget about (a)
> > > > and (b), users like to know all information about mount from
> > > > /proc/pid/mountinfo.
> > > >
> > >
> > > Let's stick to your use cases requirements. If you have other use cases
> > > for this functionality lay them out explicitly.
> >
> > Requirements is very simple, at "dump stage" we need to save all overlayfs mount options
> > sufficient to fully reconstruct overlayfs mount state on "restore stage". We already
> > have proof of concept implementation of Docker overlayfs mounts when docker is running in
> > OpenVZ container. In this case we fully dump all tree of mounts and all mount namespaces.
> > CRIU mounts restore procedure at first reconstruct mount tree in special separate subtrees
> > called "yards", then when all mounts is reconstructed we do "pivot_root" syscall. And
> > with overlayfs it was a problem, because we mounted overlayfs with lowerdir,workdir,upperdir
> > paths with mount namespace "yard" path prefix, and after restore in mount options user may see
> > that lowerdir,workdir,upperdir paths were changed... It's a problem. Also it makes second C/R
> > procedure is impossible, because after first C/R lowerdir,workdir,upperdir paths is invalidated
> > after pivot_root.
> >
> > Example for Docker (after first C/R procedure):
> >
> > options lowerdir=/tmp/.criu.mntns.owMo9C/9-0000000000//var/lib/docker/overlay2/l/4BLZ4WH6GZIVKJE5QF62QUUKVZ:/var/lib/docker/overlay2/l/7FYRGAXT35JMKTXCHDNCQO3HKT,upperdir=/tmp/.criu.mntns.owMo9C/9-0000000000//var/lib/docker/overlay2/30aa26fb5e5671fc0126f2fc0e84cc740ce6bf06ca6ad4ac877a3c60f5aceaf1/diff,workdir=/tmp/.criu.mntns.owMo9C/9-0000000000//var/lib/docker/overlay2/30aa26fb5e5671fc0126f2fc0e84cc740ce6bf06ca6ad4ac877a3c60f5aceaf1/work
> >
> 
> That reminds me.
> I've read somewhere that thoses symlinks l/4BLZ4WH6GZIVKJE5QF62QUUKVZ
> are meant to shorten the mount option string, because the mount
> options are limited by
> page size and with many lower layers limitation can reach.
> 
> That is one of the reasons that new mount API was created for (i.e. fsconfig()).
> I wonder if /proc/mounts also has a similar limitation on options size.
> I also wonder why docker doesn't chdir into /var/lib/docker/overlay2/
> before mounting overlay and use relative paths, though that would have
> been worse for CRIU.
> 
> So at least for the docker use case CRIU knows very well where the
> underlying filesytem is mounted (/var/lib/docker/overlay2/ or above).
> So if you got any API from overlayfs something like:
> getxattr("/var/lib/docker/overlay2/XYZ/merged",
> "trusted.overlay.layers.0.fh",..)
> which reads the ovl_fh encoding of layer 0 (upper) rootdir, CRIU
> can verify that uuid matches the filesystem mounted at /var/vol/docker/overlay2/
> and then call open_by_handle_at() to open fd and resolve it to a path
> under /var/vol/docker/overlay2.
> 
> I don't know if that provides what CRIU needs, but it would be no more
> than a few lines of code in overlayfs:
> 
> if (i < ofs->numlayer)
>     fh = ovl_encode_real_fh(ofs->layers[i].mnt->mnt_root, ...
> 

I will try to experiment with that. Thank you!

> Thanks,
> Amir.

-- 
Regards,
Alex.