Re: [PATCH v5] overlayfs: Provide a mount option "volatile" to skip sync

[Vivek Goyal <vgoyal@redhat.com>]

On Mon, Aug 24, 2020 at 11:53:30PM +0300, Amir Goldstein wrote:
> On Mon, Aug 24, 2020 at 4:51 PM Vivek Goyal <vgoyal@redhat.com> wrote:
> >
> > On Mon, Aug 24, 2020 at 03:20:20PM +0200, Miklos Szeredi wrote:
> > > On Mon, Aug 24, 2020 at 3:02 PM Giuseppe Scrivano <gscrivan@redhat.com> wrote:
> > > >
> > > > Amir Goldstein <amir73il@gmail.com> writes:
> > > >
> > > > > On Mon, Aug 24, 2020 at 2:39 PM Giuseppe Scrivano <gscrivan@redhat.com> wrote:
> > > > >>
> > > > >> Hi Amir,
> > > > >>
> > > > >> Amir Goldstein <amir73il@gmail.com> writes:
> > > > >>
> > > > >> > On Mon, Aug 24, 2020 at 11:15 AM Miklos Szeredi <miklos@szeredi.hu> wrote:
> > > > >> >>
> > > > >> >> On Sat, Aug 22, 2020 at 11:27 AM Giuseppe Scrivano <gscrivan@redhat.com> wrote:
> > > > >> >> >
> > > > >> >> > Vivek Goyal <vgoyal@redhat.com> writes:
> > > > >> >> >
> > > > >> >> > > Container folks are complaining that dnf/yum issues too many sync while
> > > > >> >> > > installing packages and this slows down the image build. Build
> > > > >> >> > > requirement is such that they don't care if a node goes down while
> > > > >> >> > > build was still going on. In that case, they will simply throw away
> > > > >> >> > > unfinished layer and start new build. So they don't care about syncing
> > > > >> >> > > intermediate state to the disk and hence don't want to pay the price
> > > > >> >> > > associated with sync.
> > > > >> >> > >
> > > > >> >>
> > > > >> >> [...]
> > > > >> >>
> > > > >> >> > Ping.
> > > > >> >> >
> > > > >> >> > Is there anything holding this patch?
> > > > >> >>
> > > > >> >> Not sure what happened with protection against mounting a volatile
> > > > >> >> overlay twice, I don't see that in the patch.
> > > > >> >
> > > > >> > Do you mean protection only for new kernels or old kernels as well?
> > > > >> >
> > > > >> > The latter can be achieved by using $workdir/volatile/ as upperdir
> > > > >> > instead of $upperdir.
> > > > >> > Or maybe even use $workdir/work/incompat/volatile/upper, so if older
> > > > >> > kernel tries to re-use that $workdir, it will fail to mount rw with error:
> > > > >> >
> > > > >> >   overlayfs: cleanup of 'incompat/volatile' failed (-39)
> > > > >> >
> > > > >> > If we agree to that, then upperdir= should not be provided at all when
> > > > >> > specifying "volatile".
> > > > >>
> > > > >> in this case, what does a program need to do to remount the overlay more
> > > > >> than once?  Is it enough to just delete a file?
> > > > >>
> > > > >
> > > > > Do you mean re-mount while forgetting all changes to previous "volatile"
> > > > > mount?
> > > >
> > > > no, without forgetting them.
> > > > The original idea was to have a way to disable any sync operation in the
> > > > overlay file system and let the upper layers handle it.  IOW, mount
> > > > volatile overlay+umount overlay+syncfs upper dir must still be
> > > > considered safe.
> > > > If we want to make it safer and disallow remounting the same
> > > > workdir+upperdir by default when "volatile" is used, that is fine; but I
> > > > think there should still be a way to say "I know what I am doing, just
> > > > remount it".
> > >
> > > Indeed.  "Volatile" doesn't mean you can't use the data, just that the
> > > data may be lost completely in case of a crash (tmpfs analogue).
> > >
> > > Maybe just stick
> > >
> > >   $(workdir)/work/incompat/volatile/donotremove
> > >
> > > in there to prevent misuse.
> >
> > So we ask users to remove "$(workdir)/work/incompat/volatile/donotremove"
> > if they want to remount with with same upper/ and work/? (Presumably
> > after syncing upper/).
> >
> 
> Sounds right.
> Just don't rely on the workdir cleanup error yes?
> That protection is for old kernels and it falls back to r/o mount.
> New kernel should of course recognise $(workdir)/work/incompat/volatile
> fail to mount and explicitly error about unclean "volatile" unmount and
> maybe give a hint how to fix it in kernel log.

Ok, I am wondering why are we concerned about older kernels. I mean,
if we introduce new features, we don't provide compatibility with
older kernels. Say "metacopy", "redirect_dir". If you mount with
older kernel, they will see something which you don't expect.

So why "volatile" is different. We seem to be bending backward and
using an unrelated behavior of overlay to provide this.

Why not simply drop a file $workdir/volatile for volatile mounts
and check for presence of this file when mounting?

Thanks
Vivek