Re: [RFD] posix-timers: CRIU woes

From: Andrey Vagin <avagin@openvz.org>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>,
	Frederic Weisbecker <frederic@kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Anna-Maria Behnsen <anna-maria@linutronix.de>,
	Peter Zijlstra <peterz@infradead.org>,
	syzbot+5c54bd3eb218bb595aa9@syzkaller.appspotmail.com,
	Dmitry Vyukov <dvyukov@google.com>,
	Sebastian Siewior <bigeasy@linutronix.de>,
	Michael Kerrisk <mtk.manpages@gmail.com>,
	Christian Brauner <brauner@kernel.org>,
	Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com>,
	Pavel Emelyanov <xemul@openvz.org>,
	Mike Rapoport <mike.rapoport@gmail.com>,
	Dmitry Safonov <0x7f454c46@gmail.com>,
	Adrian Reber <areber@redhat.com>
Subject: Re: [RFD] posix-timers: CRIU woes
Date: Thu, 11 May 2023 18:21:48 -0700	[thread overview]
Message-ID: <CANaxB-xFs2ZYnFBTtQsZxAGAJ6o9cgWM=k=d_EBXuOK5djLgkA@mail.gmail.com> (raw)
In-Reply-To: <87r0rnciqo.ffs@tglx>

On Thu, May 11, 2023 at 2:36 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> On Thu, May 11 2023 at 11:17, Pavel Tikhomirov wrote:
> > On 10.05.2023 16:16, Andrey Vagin wrote:
> >>>
> >>> So because of that half thought out user space ABI we are now up the
> >>> regression creek without a paddle, unless CRIU can accomodate to a
> >>> different restore mechanism to lift this restriction from the kernel.
> >>>
> >> If you give us a new API to create timers with specified id-s, we will
> >> figure out how to live with it. It isn't good to ask users to update
> >> CRIU to work on new kernels, but here are reasons and event improvements
> >> for CRIU, so I think it's worth it.
> >
> > I agree, any API to create timers with specified id-s would work for new
> > CRIU versions.
>
> The real question is whether this will cause any upheaval when a new
> kernel meets a non-updated CRIU stack.

It depends on what you mean by upheaval. We found that CRIU can be stuck
in a busy loop with the new changes. I suggest thinking about how to
work around this case and make sure that CRIU reports an error. The
error should minimize the time that users will need to spend to find the
reason and ways to resolve the problem.

One of the ways to fix the problem is to return indexes in a backward
direction from INT_MAX to zero. But in the kernel, user indices can be
converted back to "normal" values:
kernel_timer_id = INT_MAX - user_timer_id;

I have one idea of how to make these changes without breaking CRIU. CRIU
does a few special things. First, it does all timer operations from a
thread leader.  Second, it calls timer_settime only after creating all
timers. Third, it calls timer_delete for the last timer only. Any of
these events can be a trigger to switch to the new algo of allocating
timer id-s, but new processes allocate indices according to old rules.
It seems unfortunate that a real application will create a set of very
sparse indices without triggering one of these events.
I don't think that it is worth doing something like this, but if we want to
strictly follow the rules, it is the choice.

>
> You know the UABI regression rules of the kernel...

There is no rule without exceptions... With all pros and cons, we may
consider this case as an exception. From our side, we will try to make
everything to minimize the impact. Here are steps off the top of my
head:
* releasing the criu fix before the kernel release.
* update packages in Linux distros (Debian, Ubuntu, Fedora, and
  others that we will find).
* send an announcement to the criu mailing list and to users that we know.
* add the error to FAQ.
* create a GitHub issue with a full description.

Thanks,
Andrei