Re: Suggestion: Lengthen the review period for stable releases from 48 hours to 7 days.

[Greg KH <gregkh@linuxfoundation.org>]

On Tue, Nov 17, 2020 at 11:29:16PM +0100, Christoph Biedl wrote:
> >> On Sat 2020-11-14 17:40:36, Hussam Al-Tayeb wrote:
> 
> >>> Hello. I would like to suggest lengthening the review period for stable
> >>> releases from 48 hours to 7 days.
> >>> The rationale is that 48 hours is not enough for people to test those
> >>> stable releases and make sure there are no regressions for
> >>> particular workflows.
> 
> Disclaimer: I am mostly a user of stable
> 
> It's hard to make a good decision here. I share your position the 48-ish
> hours are a fairly short amound of time, and increasing it would grant
> more time for tests. As for me, I might resume testing -rc on a regular
> base as I used to in the past - which is a time-consuming procedure, and
> since I do that as a hobby, sometimes more important things are in the
> way. But I have to concede the number of issues that occured only here
> was never high, and I don't expect it would grow significantly.
> 
> On the other hand the pace of the stable patches became fairly high¹, so
> during a week of -rc review a *lot* of them will queue up and I predict
> we'll see requests for fast-laning some of them. Also, a release would
> immediately be followed by the next -rc review period, a procedure that
> gives me a bad feeling.
> 
> So for me, I'd appreciate an extension of the review period, even if
> it's just four days. But I understand if people prefer to keep the
> procedures simple, and get fixes out of the door as soon as possible.

That's the thing, these releases almost always contain fixes that we
know people are having in the real world, or they fix reported security
issues, so we need to get them out to everyone as soon as possible.

If you are only a week or so behind (because your testing framework
takes a week), that's fine, let us know if we broke something last week
and we will be glad to revert it or find the fixup patch for it that is
in Linus's tree.  I work with almost all of the major SoC vendors and
they get back to me on this type of delayed cycle because their tests do
take longer, and it works fine.

Also, note that I do a release when the testers that I have come to rely
on tell me that all is good.  I think they are running some 50k+ tests
on each release at the moment, so while quantity isn't a substitute for
quality, it is a good indication that nothing regressed here which is
what I am looking for.  Recently these tests are coming back sooner than
2 days, which is great and why I do a release quicker at times
(sometimes it is because of just logistics reasons).

So slowing down releases is not the answer.  Getting back to me when you
have issues is the solution.  1-2-4 weeks is fine, just let us know if
you have regressions when you find them, otherwise we don't know that
there is an issue that needs to be resolved.

> ¹ If somebody made statistics on the development of the number of
>   patches for stable kernels (in count/second), I'd be curious to see
>   the numbers.

Yes, I have those numbers, we run about 30-35 patches/day in the stable
releases at the moment.  You can see this in the spreadsheet I keep at:
	https://github.com/gregkh/kernel-history
in the kernel_stats.ods file, look at the tabs on the bottom to see the
rate of change for the different stable releases.  It usually is a few
releases old, but I try to update it monthly.

thanks,

greg k-h