Re: [PATCH v5] overlayfs: Provide a mount option "volatile" to skip sync

[Vivek Goyal <vgoyal@redhat.com>]

On Mon, Aug 24, 2020 at 01:59:41PM +0300, Amir Goldstein wrote:
> On Mon, Aug 24, 2020 at 11:15 AM Miklos Szeredi <miklos@szeredi.hu> wrote:
> >
> > On Sat, Aug 22, 2020 at 11:27 AM Giuseppe Scrivano <gscrivan@redhat.com> wrote:
> > >
> > > Vivek Goyal <vgoyal@redhat.com> writes:
> > >
> > > > Container folks are complaining that dnf/yum issues too many sync while
> > > > installing packages and this slows down the image build. Build
> > > > requirement is such that they don't care if a node goes down while
> > > > build was still going on. In that case, they will simply throw away
> > > > unfinished layer and start new build. So they don't care about syncing
> > > > intermediate state to the disk and hence don't want to pay the price
> > > > associated with sync.
> > > >
> >
> > [...]
> >
> > > Ping.
> > >
> > > Is there anything holding this patch?
> >
> > Not sure what happened with protection against mounting a volatile
> > overlay twice, I don't see that in the patch.
> 
> Do you mean protection only for new kernels or old kernels as well?
> 
> The latter can be achieved by using $workdir/volatile/ as upperdir
> instead of $upperdir.
> Or maybe even use $workdir/work/incompat/volatile/upper, so if older
> kernel tries to re-use that $workdir, it will fail to mount rw with error:
> 
>   overlayfs: cleanup of 'incompat/volatile' failed (-39)
> 
> If we agree to that, then upperdir= should not be provided at all when
> specifying "volatile".

If we keep volatile inside workdir, then we fail work and upperdir
being separate subtree checks. And I suspect that all that trap
magic will trigger too.

I think for image building use case, tools to have access to volatile
directory. So that they can persist it, rename it and use it as lower
layer for next layer build. That means we will have to document it
and let users access and rename $workdir/work/incompat/volatile/ or
$workdir/work/volatile.

Once Miklos has suggested to drop a file in workdir say $workdir/volatile
And next remount will refuse to mount that overlay instance if
$workdir/volatile is present. With this approach work/ and upper/
are in separate dir subtrees.

And user will be forced to remove work/ and upper/ if previous instance
was mounted with "volatile".

I am not too worried about protection against older kernels because if
system has been setup to boot into a new kernel, it will boot into
new kernel again. (Until and unless somebody forces it to go back to
old kernel). But if you think providing protection against old kernels
is important, we could create volatile in $workdir/work/dir1/dir2/volatile
instead.

/me is wondering why I don't get error with $workdir/work/dir1/volatile
but I do with $workdir/work/dir1/dir2/volatile. IOW, why in first
case removal of dir1 was successful despite the fact it is non-empty.

Thanks
Vivek