Re: [PATCH v5] overlayfs: Provide a mount option "volatile" to skip sync

[Amir Goldstein <amir73il@gmail.com>]

On Mon, Aug 24, 2020 at 4:51 PM Vivek Goyal <vgoyal@redhat.com> wrote:
>
> On Mon, Aug 24, 2020 at 03:20:20PM +0200, Miklos Szeredi wrote:
> > On Mon, Aug 24, 2020 at 3:02 PM Giuseppe Scrivano <gscrivan@redhat.com> wrote:
> > >
> > > Amir Goldstein <amir73il@gmail.com> writes:
> > >
> > > > On Mon, Aug 24, 2020 at 2:39 PM Giuseppe Scrivano <gscrivan@redhat.com> wrote:
> > > >>
> > > >> Hi Amir,
> > > >>
> > > >> Amir Goldstein <amir73il@gmail.com> writes:
> > > >>
> > > >> > On Mon, Aug 24, 2020 at 11:15 AM Miklos Szeredi <miklos@szeredi.hu> wrote:
> > > >> >>
> > > >> >> On Sat, Aug 22, 2020 at 11:27 AM Giuseppe Scrivano <gscrivan@redhat.com> wrote:
> > > >> >> >
> > > >> >> > Vivek Goyal <vgoyal@redhat.com> writes:
> > > >> >> >
> > > >> >> > > Container folks are complaining that dnf/yum issues too many sync while
> > > >> >> > > installing packages and this slows down the image build. Build
> > > >> >> > > requirement is such that they don't care if a node goes down while
> > > >> >> > > build was still going on. In that case, they will simply throw away
> > > >> >> > > unfinished layer and start new build. So they don't care about syncing
> > > >> >> > > intermediate state to the disk and hence don't want to pay the price
> > > >> >> > > associated with sync.
> > > >> >> > >
> > > >> >>
> > > >> >> [...]
> > > >> >>
> > > >> >> > Ping.
> > > >> >> >
> > > >> >> > Is there anything holding this patch?
> > > >> >>
> > > >> >> Not sure what happened with protection against mounting a volatile
> > > >> >> overlay twice, I don't see that in the patch.
> > > >> >
> > > >> > Do you mean protection only for new kernels or old kernels as well?
> > > >> >
> > > >> > The latter can be achieved by using $workdir/volatile/ as upperdir
> > > >> > instead of $upperdir.
> > > >> > Or maybe even use $workdir/work/incompat/volatile/upper, so if older
> > > >> > kernel tries to re-use that $workdir, it will fail to mount rw with error:
> > > >> >
> > > >> >   overlayfs: cleanup of 'incompat/volatile' failed (-39)
> > > >> >
> > > >> > If we agree to that, then upperdir= should not be provided at all when
> > > >> > specifying "volatile".
> > > >>
> > > >> in this case, what does a program need to do to remount the overlay more
> > > >> than once?  Is it enough to just delete a file?
> > > >>
> > > >
> > > > Do you mean re-mount while forgetting all changes to previous "volatile"
> > > > mount?
> > >
> > > no, without forgetting them.
> > > The original idea was to have a way to disable any sync operation in the
> > > overlay file system and let the upper layers handle it.  IOW, mount
> > > volatile overlay+umount overlay+syncfs upper dir must still be
> > > considered safe.
> > > If we want to make it safer and disallow remounting the same
> > > workdir+upperdir by default when "volatile" is used, that is fine; but I
> > > think there should still be a way to say "I know what I am doing, just
> > > remount it".
> >
> > Indeed.  "Volatile" doesn't mean you can't use the data, just that the
> > data may be lost completely in case of a crash (tmpfs analogue).
> >
> > Maybe just stick
> >
> >   $(workdir)/work/incompat/volatile/donotremove
> >
> > in there to prevent misuse.
>
> So we ask users to remove "$(workdir)/work/incompat/volatile/donotremove"
> if they want to remount with with same upper/ and work/? (Presumably
> after syncing upper/).
>

Sounds right.
Just don't rely on the workdir cleanup error yes?
That protection is for old kernels and it falls back to r/o mount.
New kernel should of course recognise $(workdir)/work/incompat/volatile
fail to mount and explicitly error about unclean "volatile" unmount and
maybe give a hint how to fix it in kernel log.

Thanks,
Amir.