Xen 4.16: Proposed release manager and schedule

Xen 4.16: Proposed release manager and schedule

[Ian Jackson]

We haven't formally appointed a release manager for Xen 4.16.
I was approached and asked if I would do the job, and said yes,
but I think things got stuck there.  Taking this as a prima
faciae indication of community confidence, I hereby volunteer to
take on this role.

And, I would like to tentatively propose the following schedule and
policies for Xen 4.16.  This is based on the 4.15 schedule with some
tweaks, and is intended to align the ultimate date roughly with the
4.10 and 4.13 releases, which were also in early/mid December.

I suggest we use the Lazy Consensus procedure to decide on the Release
Manager appointment, starting now (since we're already rather late).
In particular, if you feel someone else would make a better release
manager, please say so right away.

For the release schedule, assuming I'm the RM, please send comments
ASAP and in any case by noon UTC on Wednesday the 25th of August.  I
hope to finalise the schedule then.

** DRAFT **

  Friday 17th September                 Last posting date

    Patches adding new features should be posted to the mailing list
    by this cate, although perhaps not in their final version.
    (3 weeks)

  Friday 8th October                    Feature freeze
 
    Patches adding new features should be committed by this date.
    Straightforward bugfixes may continue to be accepted by
    maintainers.
    (3 weeks)

  Friday 29th October **tentatve**      Code freeze

    Bugfixes only, all changes to be approved by the Release Manager,
    on the basis of a (progressively stricter[*]) risk assessment.
    (2.5 weeks)

  Tuesday 16nd November **tentative**   Branch off staging-4.16
                                        Hard code freeze [*]

    Bugfixes for serious bugs (including regressions), and low-risk
    fixes only.  All changes to be approved by the Release Manager on
    the basis of a (progressively stricter[*]) risk assessment.

    xen-unstable open again - with caveats to avoid release disruption.

    (2.5 weeks)

  Friday 3rd December **tentative**     Final commits (docs/prep only)
  Week of 6th December **tentative**    Release
    (probably Tuesday or Wednesday)

Any patches containing substantial refactoring are to treated as
new features, even if they intent is to fix bugs.

Freeze exceptions will not be routine, but may be granted in
exceptional cases for small changes (on the basis of risk assessment,
like any release-ack).  Large series will not get exceptions.
Contributors *must not* rely on, or expect, a freeze exception, or
release schedule slip.

If as a feature proponent you feel your feature is at risk and there
is something the Xen Project could do to help, please consult me or
ask here on xen-devel.  In such situations please reach out earlier
rather than later.  I will try to put you in touch with people who may
be able to help.

[*] The distinction between Code Freeze and Hard Code Freeze is a
matter of degree, not kind; the Hard Code Freeze data and associated
tighter policy text is indicative rather than normative.

** END OF DRAFT **

Thanks,
Ian.

Re: Xen 4.16: Proposed release manager and schedule

[Jan Beulich]

On 18.08.2021 17:09, Ian Jackson wrote:
> We haven't formally appointed a release manager for Xen 4.16.
> I was approached and asked if I would do the job, and said yes,
> but I think things got stuck there.  Taking this as a prima
> faciae indication of community confidence, I hereby volunteer to
> take on this role.
> 
> And, I would like to tentatively propose the following schedule and
> policies for Xen 4.16.  This is based on the 4.15 schedule with some
> tweaks, and is intended to align the ultimate date roughly with the
> 4.10 and 4.13 releases, which were also in early/mid December.
> 
> I suggest we use the Lazy Consensus procedure to decide on the Release
> Manager appointment, starting now (since we're already rather late).
> In particular, if you feel someone else would make a better release
> manager, please say so right away.
> 
> For the release schedule, assuming I'm the RM, please send comments
> ASAP and in any case by noon UTC on Wednesday the 25th of August.  I
> hope to finalise the schedule then.
> 
> ** DRAFT **
> 
>   Friday 17th September                 Last posting date
> 
>     Patches adding new features should be posted to the mailing list
>     by this cate, although perhaps not in their final version.
>     (3 weeks)

Without meaning this to express any opposition to the proposed schedule,
I'd like to point out that as of mid July there's been unusually little
changes getting committed. The above leaves less than a month (plus the
3 weeks of until feature freeze); at the present rate this would mean
about no further (larger) changes.

> If as a feature proponent you feel your feature is at risk and there
> is something the Xen Project could do to help, please consult me or
> ask here on xen-devel.  In such situations please reach out earlier
> rather than later.  I will try to put you in touch with people who may
> be able to help.

Code review looks to be even more of a bottleneck than it already used
to be. Some of the series I did post long ago (and hence would never
have thought could be at risk to miss the next release) are still
blocked on not getting reviewed, or having got partially reviewed and
then things ended up stalled. Since even pings didn't lead anywhere,
I'm afraid I'm out of ideas, and I will admit that for some items I've
simply not seen any point anymore in pinging. For a few patches I
didn't even think there was any point posting them in the first place
for this very reason. Examples (totaling to about 70 pending patches,
and not naming any further individual ones):

xvmalloc() / x86 xstate area / x86 CPUID / AMX+XFD
x86: memcpy() / memset() (non-)ERMS flavors plus fallout
x86: more or less log-dirty related improvements
x86/mm: large parts of P2M code and struct p2m_domain are HVM-only
x86emul: a few small steps towards disintegration

(I also have almost 20 patches submission of which is blocked by the
presently pending batch of XSAs. Their handling may also get tight
at the current rate of progress.)

Jan

Re: Xen 4.16: Proposed release manager and schedule

[Ian Jackson]

Jan Beulich writes ("Re: Xen 4.16: Proposed release manager and schedule"):
> On 18.08.2021 17:09, Ian Jackson wrote:
> > ** DRAFT **
> > 
> >   Friday 17th September                 Last posting date
> > 
> >     Patches adding new features should be posted to the mailing list
> >     by this cate, although perhaps not in their final version.
> >     (3 weeks)
> 
> Without meaning this to express any opposition to the proposed schedule,
> I'd like to point out that as of mid July there's been unusually little
> changes getting committed. The above leaves less than a month (plus the
> 3 weeks of until feature freeze); at the present rate this would mean
> about no further (larger) changes.

Thanks for your reply.  I recognise the problems you describe.
From my pov as (putative) RM it is difficult to see what could
constructively be done about this situation.

One option is to slip the whole release, in the expectation (hope!) of
collecting more input.  In practical terms because of the impact of
Christmas and New Year on many of our contributors, a month's slip now
is probably more like two months' slip to the release.  So arguably
for this to be worth it, we ought to plan to slip 2 months now and
release in February.

Some of the thinness of this release in particular relates to an
unusual combination of substantial leave taken by many key
contributors, so maybe this is a thing we should consider.
Even my proposing this schedule was rather late, in part for those
reasons.

I don't personally have a strong opinion about the right decision.
But if we are intending to release in December, I think something like
my proposed schedule is probably about right.  Perhaps it could be
compressed a bit due to the lack of major contributions so far.

I would love to hear other opinions.

Ian.

Re: Xen 4.16: Proposed release manager and schedule

[Julien Grall]

Hi,

On 18/08/2021 16:09, Ian Jackson wrote:
> We haven't formally appointed a release manager for Xen 4.16.
> I was approached and asked if I would do the job, and said yes,
> but I think things got stuck there.  Taking this as a prima
> faciae indication of community confidence, I hereby volunteer to
> take on this role.
> 
> And, I would like to tentatively propose the following schedule and
> policies for Xen 4.16.  This is based on the 4.15 schedule with some
> tweaks, and is intended to align the ultimate date roughly with the
> 4.10 and 4.13 releases, which were also in early/mid December.
> 
> I suggest we use the Lazy Consensus procedure to decide on the Release
> Manager appointment, starting now (since we're already rather late).
> In particular, if you feel someone else would make a better release
> manager, please say so right away.
> 
> For the release schedule, assuming I'm the RM, please send comments
> ASAP and in any case by noon UTC on Wednesday the 25th of August.  I
> hope to finalise the schedule then.
> 
> ** DRAFT **
> 
>    Friday 17th September                 Last posting date
> 
>      Patches adding new features should be posted to the mailing list
>      by this cate, although perhaps not in their final version.
>      (3 weeks)
> 
>    Friday 8th October                    Feature freeze
>   
>      Patches adding new features should be committed by this date.
>      Straightforward bugfixes may continue to be accepted by
>      maintainers.
>      (3 weeks)

At the moment, on Arm, I am tracking a few big series that are candidate 
for 4.16. Unfortunately, we will have some review bandwidth shortage 
over the next month so this will impact what we can merge on time.

So far, we don't have major features merged. If we follow this schedule, 
this will be a fairly quiet release.

I think on the Arm side, an extra week could give us some slack to merge 
part of what is currently under review on the ML.

I saw your answer to Jan about splipping the release for longer. I will 
answer there directly.

Cheers,

-- 
Julien Grall

Re: Xen 4.16: Proposed release manager and schedule

[Julien Grall]

Hi Ian,

On 19/08/2021 16:36, Ian Jackson wrote:
> Jan Beulich writes ("Re: Xen 4.16: Proposed release manager and schedule"):
>> On 18.08.2021 17:09, Ian Jackson wrote:
>>> ** DRAFT **
>>>
>>>    Friday 17th September                 Last posting date
>>>
>>>      Patches adding new features should be posted to the mailing list
>>>      by this cate, although perhaps not in their final version.
>>>      (3 weeks)
>>
>> Without meaning this to express any opposition to the proposed schedule,
>> I'd like to point out that as of mid July there's been unusually little
>> changes getting committed. The above leaves less than a month (plus the
>> 3 weeks of until feature freeze); at the present rate this would mean
>> about no further (larger) changes.
> 
> Thanks for your reply.  I recognise the problems you describe.
>  From my pov as (putative) RM it is difficult to see what could
> constructively be done about this situation.
> 
> One option is to slip the whole release, in the expectation (hope!) of
> collecting more input.  In practical terms because of the impact of
> Christmas and New Year on many of our contributors, a month's slip now
> is probably more like two months' slip to the release.  So arguably
> for this to be worth it, we ought to plan to slip 2 months now and
> release in February.

A 2 months slip looks appealling to get more features. But it means we 
will end up to split all the future releases.

IIRC the timing was chosen so the release avoids major holidays in 
various part of the world. So we would either need to decide on a new 
cadence or find a way (maybe the next release will be longer as well?) 
to get back on track.

> 
> Some of the thinness of this release in particular relates to an
> unusual combination of substantial leave taken by many key
> contributors, so maybe this is a thing we should consider.
> Even my proposing this schedule was rather late, in part for those
> reasons.
> 
> I don't personally have a strong opinion about the right decision.
> But if we are intending to release in December, I think something like
> my proposed schedule is probably about right.  Perhaps it could be
> compressed a bit due to the lack of major contributions so far.

At the moment, my preference is to stick with the release in December. 
We don't have major contributions checked in yet on Arm, but there is a 
chance to have a couple of them with the current schedule on Arm.

We also have a collection of bug fixes which makes Arm running better on 
some platform.

So I think the release would still be worthwhile even in the worst case 
where nothing major is merged.

Cheers,

-- 
Julien Grall

Re: Xen 4.16: Proposed release manager and schedule

[Ian Jackson]

Julien Grall writes ("Re: Xen 4.16: Proposed release manager and schedule"):
> On 19/08/2021 16:36, Ian Jackson wrote:
> > One option is to slip the whole release, in the expectation (hope!) of
> > collecting more input.  In practical terms because of the impact of
> > Christmas and New Year on many of our contributors, a month's slip now
> > is probably more like two months' slip to the release.  So arguably
> > for this to be worth it, we ought to plan to slip 2 months now and
> > release in February.
> 
> A 2 months slip looks appealling to get more features. But it means we 
> will end up to split all the future releases.

No, it wouldn't, really.  Our usual release interval is 9 months.
9+2 = 11, so the result would be a one month longer release, if we
delayed this one by 2 months.  In practice it is always a bit of a
struggle to keep things to time (that's the way things are set up with
this system) so I think this is well within our usual tolerances.

For the avoidance of doubt, I am not arguing in favour of this option.
But I wanted to present it as a reasonable option the community might
want to consider.

Jan, do you have an opnion about this ?  Do you expect that a 2-3
month slip would help with your review backlog, bearing in mind the
people who are currently on leave and when they'll be back ?

If no-one wishes to argue in favour of the longer slip I think we
should go with the normal plan.

> At the moment, my preference is to stick with the release in December. 
> We don't have major contributions checked in yet on Arm, but there is a 
> chance to have a couple of them with the current schedule on Arm.
> 
> We also have a collection of bug fixes which makes Arm running better on 
> some platform.
> 
> So I think the release would still be worthwhile even in the worst case 
> where nothing major is merged.

Thanks for the opinion.


Can I at least get a +1 from someone for appointing me as RM
for 4.16 ? :-)

Thanks,
Ian.

Re: Xen 4.16: Proposed release manager and schedule

[Jan Beulich]

On 24.08.2021 15:38, Ian Jackson wrote:
> Julien Grall writes ("Re: Xen 4.16: Proposed release manager and schedule"):
>> On 19/08/2021 16:36, Ian Jackson wrote:
>>> One option is to slip the whole release, in the expectation (hope!) of
>>> collecting more input.  In practical terms because of the impact of
>>> Christmas and New Year on many of our contributors, a month's slip now
>>> is probably more like two months' slip to the release.  So arguably
>>> for this to be worth it, we ought to plan to slip 2 months now and
>>> release in February.
>>
>> A 2 months slip looks appealling to get more features. But it means we 
>> will end up to split all the future releases.
> 
> No, it wouldn't, really.  Our usual release interval is 9 months.
> 9+2 = 11, so the result would be a one month longer release, if we
> delayed this one by 2 months.  In practice it is always a bit of a
> struggle to keep things to time (that's the way things are set up with
> this system) so I think this is well within our usual tolerances.
> 
> For the avoidance of doubt, I am not arguing in favour of this option.
> But I wanted to present it as a reasonable option the community might
> want to consider.
> 
> Jan, do you have an opnion about this ?  Do you expect that a 2-3
> month slip would help with your review backlog, bearing in mind the
> people who are currently on leave and when they'll be back ?

It's extremely hard to predict whether 2-3 months would help, or by
how much. I wanted to make the "slim" release aspect explicit, but
I didn't mean to question the schedule unless there would have been
signals that many care about a more "feature rich" 4.16.

>> At the moment, my preference is to stick with the release in December. 
>> We don't have major contributions checked in yet on Arm, but there is a 
>> chance to have a couple of them with the current schedule on Arm.
>>
>> We also have a collection of bug fixes which makes Arm running better on 
>> some platform.
>>
>> So I think the release would still be worthwhile even in the worst case 
>> where nothing major is merged.
> 
> Thanks for the opinion.
> 
> 
> Can I at least get a +1 from someone for appointing me as RM
> for 4.16 ? :-)

Sure: +1

Jan

Re: Xen 4.16: Proposed release manager and schedule

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 2243 bytes --]

On 24.08.21 15:38, Ian Jackson wrote:
> Julien Grall writes ("Re: Xen 4.16: Proposed release manager and schedule"):
>> On 19/08/2021 16:36, Ian Jackson wrote:
>>> One option is to slip the whole release, in the expectation (hope!) of
>>> collecting more input.  In practical terms because of the impact of
>>> Christmas and New Year on many of our contributors, a month's slip now
>>> is probably more like two months' slip to the release.  So arguably
>>> for this to be worth it, we ought to plan to slip 2 months now and
>>> release in February.
>>
>> A 2 months slip looks appealling to get more features. But it means we
>> will end up to split all the future releases.
> 
> No, it wouldn't, really.  Our usual release interval is 9 months.

8 months (that's what I suggested in my time as a release manager and
what was approved and never changed "officially").

> 9+2 = 11, so the result would be a one month longer release, if we
> delayed this one by 2 months.  In practice it is always a bit of a
> struggle to keep things to time (that's the way things are set up with
> this system) so I think this is well within our usual tolerances.
> 
> For the avoidance of doubt, I am not arguing in favour of this option.
> But I wanted to present it as a reasonable option the community might
> want to consider.
> 
> Jan, do you have an opnion about this ?  Do you expect that a 2-3
> month slip would help with your review backlog, bearing in mind the
> people who are currently on leave and when they'll be back ?
> 
> If no-one wishes to argue in favour of the longer slip I think we
> should go with the normal plan.
> 
>> At the moment, my preference is to stick with the release in December.
>> We don't have major contributions checked in yet on Arm, but there is a
>> chance to have a couple of them with the current schedule on Arm.
>>
>> We also have a collection of bug fixes which makes Arm running better on
>> some platform.
>>
>> So I think the release would still be worthwhile even in the worst case
>> where nothing major is merged.
> 
> Thanks for the opinion.
> 
> 
> Can I at least get a +1 from someone for appointing me as RM
> for 4.16 ? :-)

+1 :-)


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: Xen 4.16: Proposed release manager and schedule

[Stefano Stabellini]

On Tue, 24 Aug 2021, Julien Grall wrote:
> > Some of the thinness of this release in particular relates to an
> > unusual combination of substantial leave taken by many key
> > contributors, so maybe this is a thing we should consider.
> > Even my proposing this schedule was rather late, in part for those
> > reasons.
> > 
> > I don't personally have a strong opinion about the right decision.
> > But if we are intending to release in December, I think something like
> > my proposed schedule is probably about right.  Perhaps it could be
> > compressed a bit due to the lack of major contributions so far.
> 
> At the moment, my preference is to stick with the release in December. We
> don't have major contributions checked in yet on Arm, but there is a chance to
> have a couple of them with the current schedule on Arm.

+1

Re: Xen 4.16: Proposed release manager and schedule

[Julien Grall]

Hi Ian,

On 24/08/2021 14:38, Ian Jackson wrote:
> Julien Grall writes ("Re: Xen 4.16: Proposed release manager and schedule"):
>> On 19/08/2021 16:36, Ian Jackson wrote:
> Can I at least get a +1 from someone for appointing me as RM
> for 4.16 ? :-)

+1. Sorry I forgot to mention it in the previous e-mail.

Cheers,

-- 
Julien Grall