Xen 4.16 development update; request for regression bug reports

Xen 4.16 development update; request for regression bug reports

[Ian Jackson]

The mainline development branch of Xen is has taken the first step
towards stabilisation for 4.16: new features not yet submitted, at
least in draft form, will now be deferred to the following release.

We aim to make each Xen release as good as possible, and that includes
high quality and particularly an absence of serious regressions.
To help deliver this as Release Manager for Xen 4.16, I need the help
of the Xen developer and user community

In particular, please would users and developers make sure that I know
about:

 * Regressions in the current development branch (xen-unstable)
   compared to Xen 4.15;

 * Major or important work which is targeted for 4.16, but at risk
   of missing the deadlines.

Please email me, CC xen-devel (as in the Reply-To), and of course
other interested parties. including the maintainer(s) of relevant
subsystems (if you know who that would be).

Thanks for your attention,
Ian.


Planned release schedule for Xen 4.16 (recap):

  Friday 24th September   PASSED        Last posting date

    Patches adding new features should be posted to the mailing list
    by this cate, although perhaps not in their final version.
    (3 weeks)

  Friday 15th October                   Feature freeze
 
    Patches adding new features should be committed by this date.
    Straightforward bugfixes may continue to be accepted by
    maintainers.
    (3 weeks)

  Friday 5th November **tentatve**      Code freeze

    Bugfixes only, all changes to be approved by the Release Manager,
    on the basis of a (progressively stricter[*]) risk assessment.
    (2 weeks)

  Friday 19th November **tentative**    Hard code freeze [*]

    Bugfixes for serious bugs (including regressions), and low-risk
    fixes only.
    (0.5 weeks)

  Tuesday 23rd November **tentative**   Branch off staging-4.16
                                        
    xen-unstable open again - with caveats to avoid release disruption.
    (1.5 weeks)

  Friday 3rd December **tentative**     Final commits (docs/prep only)
  Week of 6th December **tentative**    Release
    (probably Tuesday or Wednesday)

Any patches containing substantial refactoring are to treated as
new features, even if they intent is to fix bugs.

Freeze exceptions will not be routine, but may be granted in
exceptional cases for small changes (on the basis of risk assessment,
like any release-ack).  Large series will not get exceptions.
Contributors *must not* rely on, or expect, a freeze exception, or
release schedule slip.

If as a feature proponent you feel your feature is at risk and there
is something the Xen Project could do to help, please consult me or
ask here on xen-devel.  In such situations please reach out earlier
rather than later.  I will try to put you in touch with people who may
be able to help.

[*] The distinction between Code Freeze and Hard Code Freeze is a
matter of degree, not kind; the Hard Code Freeze data and associated
tighter policy text is indicative rather than normative.

Re: Xen 4.16 development update; request for regression bug reports

[Jan Beulich]

On 27.09.2021 16:24, Ian Jackson wrote:
> The mainline development branch of Xen is has taken the first step
> towards stabilisation for 4.16: new features not yet submitted, at
> least in draft form, will now be deferred to the following release.
> 
> We aim to make each Xen release as good as possible, and that includes
> high quality and particularly an absence of serious regressions.
> To help deliver this as Release Manager for Xen 4.16, I need the help
> of the Xen developer and user community
> 
> In particular, please would users and developers make sure that I know
> about:
> 
>  * Regressions in the current development branch (xen-unstable)
>    compared to Xen 4.15;
> 
>  * Major or important work which is targeted for 4.16, but at risk
>    of missing the deadlines.

Effectively all of the series that have been stuck for a more or less
long periods of time were originally intended for 4.16:

- xvmalloc() / x86 xstate area / x86 CPUID / AMX+XFD
- x86: memcpy() / memset() (non-)ERMS flavors plus fallout
- x86/mm: large parts of P2M code and struct p2m_domain are HVM-only
- x86emul: a few small steps towards disintegration
- x86: more or less log-dirty related improvements
- IOMMU: superpage support when not sharing pagetables
- AMD/IOMMU: further work split from XSA-378
- gnttab: further work from XSA-380 / -382 context
- x86/mwait-idle: updates from Linux
- x86/boot: (mostly) video mode handling adjustments

There are further individual patches, which I've omitted. The only one
that I think (or should I say hope) is on track is "x86/PVH: Dom0
building adjustments".

It would of course also be nice for Anthony's "xen: Build system
improvements, now with out-of-tree build!" to go in, but it's hard to
predict whether the remaining almost 40 patches can get reviewed in
time. I've been meaning to get through with this (and others), but
other stuff keeps popping up.

Additionally there is Marek's "x86/xstate: reset cached register values
on resume", which Andrew has been meaning to supply a superset patch for.
He has hence been objecting to this patch going in, but seeing that 1.5
months have passed now I think this wants reconsidering.

Similarly there is Roger's even older "x86/viridian: EOI MSR should
always happen in affected vCPU context".

Jan

Re: Xen 4.16 development update

[Ian Jackson]

I wrote:
> Planned release schedule for Xen 4.16 (recap):
> 
>   Friday 24th September   PASSED        Last posting date
> 
>     Patches adding new features should be posted to the mailing list
>     by this cate, although perhaps not in their final version.
>     (3 weeks)
> 
>   Friday 15th October                   Feature freeze
>  
>     Patches adding new features should be committed by this date.
>     Straightforward bugfixes may continue to be accepted by
>     maintainers.

As previously noted, we have had a lack of review bandwidth which
means this release will be light on content.  We discussed but
rejected the possibility of extending the release (beyond Christmas,
to some time in January/February).

Under the circumstances, and considering some external constraints, I
am considering trying to *compress* the release schedule by
approximately one week.  The feature freeze data would remain the same
- we need to give people as much time to get things reviewed and into
tree as we can, but I think we can probably cut a week off the freeze.

That would look roughly like this:

>   Friday 15th October                   Feature freeze

      (2 weeks)

    Friday 29th October **tentatve**      Code freeze

      Bugfixes only, all changes to be approved by the Release Manager,
      on the basis of a (progressively stricter[*]) risk assessment.
      (2 weeks)

    Friday 12th November **tentative**    Hard code freeze [*]

      Bugfixes for serious bugs (including regressions), and low-risk
      fixes only.
      (0.5 weeks)

    Tuesday 16th November **tentative**   Branch off staging-4.16

      xen-unstable open again - with caveats to avoid release disruption.
      (1.5 weeks)

    Friday 26th November **tentative**    Final commits (docs/prep only)
    Week of 29th November **tentative**   Release
      (probably Tuesday or Wednesday)

Ian.

Re: Xen 4.16 development update

[Jan Beulich]

On 06.10.2021 13:18, Ian Jackson wrote:
> I wrote:
>> Planned release schedule for Xen 4.16 (recap):
>>
>>   Friday 24th September   PASSED        Last posting date
>>
>>     Patches adding new features should be posted to the mailing list
>>     by this cate, although perhaps not in their final version.
>>     (3 weeks)
>>
>>   Friday 15th October                   Feature freeze
>>  
>>     Patches adding new features should be committed by this date.
>>     Straightforward bugfixes may continue to be accepted by
>>     maintainers.
> 
> As previously noted, we have had a lack of review bandwidth which
> means this release will be light on content.  We discussed but
> rejected the possibility of extending the release (beyond Christmas,
> to some time in January/February).
> 
> Under the circumstances, and considering some external constraints, I
> am considering trying to *compress* the release schedule by
> approximately one week.  The feature freeze data would remain the same
> - we need to give people as much time to get things reviewed and into
> tree as we can, but I think we can probably cut a week off the freeze.
> 
> That would look roughly like this:
> 
>>   Friday 15th October                   Feature freeze
> 
>       (2 weeks)

Now that we're here, I of course have given up on any of the pending
non-bugfix patches to make 4.16. But I very much hope 4.16 won't go
out with known bugs having fixes available. Therefore I've pinged (in
some cases again):

x86/CPUID: don't shrink hypervisor leaves
x86emul: de-duplicate scatters to the same linear address
x86/AMD: make HT range dynamic for Fam17 and up
x86/PV: address odd UB in I/O emulation
x86/xstate: reset cached register values on resume
x86/altp2m: don't consider "active" when enabling failed
x86/PV: '0' debug key stack dumping adjustments [at least the first 2]
x86/paging: restrict physical address width reported to guests
x86/PoD: defer nested P2M flushes

In principle "x86: more or less log-dirty related improvements" also
contains a number of bug fixes, but the way this series is stuck
made me not include it in the pings.

I've tried to remember to Cc you on these pings, but I may not have
succeeded in doing so.

Jan

>     Friday 29th October **tentatve**      Code freeze
> 
>       Bugfixes only, all changes to be approved by the Release Manager,
>       on the basis of a (progressively stricter[*]) risk assessment.
>       (2 weeks)
> 
>     Friday 12th November **tentative**    Hard code freeze [*]
> 
>       Bugfixes for serious bugs (including regressions), and low-risk
>       fixes only.
>       (0.5 weeks)
> 
>     Tuesday 16th November **tentative**   Branch off staging-4.16
> 
>       xen-unstable open again - with caveats to avoid release disruption.
>       (1.5 weeks)
> 
>     Friday 26th November **tentative**    Final commits (docs/prep only)
>     Week of 29th November **tentative**   Release
>       (probably Tuesday or Wednesday)
> 
> Ian.
> 

Re: Xen 4.16 development update

[Ian Jackson]

Jan Beulich writes ("Re: Xen 4.16 development update"):
> Now that we're here, I of course have given up on any of the pending
> non-bugfix patches to make 4.16. But I very much hope 4.16 won't go
> out with known bugs having fixes available. Therefore I've pinged (in
> some cases again):
> 
> x86/CPUID: don't shrink hypervisor leaves
> x86emul: de-duplicate scatters to the same linear address
> x86/AMD: make HT range dynamic for Fam17 and up
> x86/PV: address odd UB in I/O emulation
> x86/xstate: reset cached register values on resume
> x86/altp2m: don't consider "active" when enabling failed
> x86/PV: '0' debug key stack dumping adjustments [at least the first 2]
> x86/paging: restrict physical address width reported to guests
> x86/PoD: defer nested P2M flushes
> 
> In principle "x86: more or less log-dirty related improvements" also
> contains a number of bug fixes, but the way this series is stuck
> made me not include it in the pings.
> 
> I've tried to remember to Cc you on these pings, but I may not have
> succeeded in doing so.

Thanks.  I got at least most of them.

Roger, would you be able to review these changes ?

Thanks,
Ian.

Re: Xen 4.16 development update

[Roger Pau Monné]

On Mon, Oct 18, 2021 at 11:51:16AM +0100, Ian Jackson wrote:
> Jan Beulich writes ("Re: Xen 4.16 development update"):
> > Now that we're here, I of course have given up on any of the pending
> > non-bugfix patches to make 4.16. But I very much hope 4.16 won't go
> > out with known bugs having fixes available. Therefore I've pinged (in
> > some cases again):
> > 
> > x86/CPUID: don't shrink hypervisor leaves
> > x86emul: de-duplicate scatters to the same linear address
> > x86/AMD: make HT range dynamic for Fam17 and up
> > x86/PV: address odd UB in I/O emulation
> > x86/xstate: reset cached register values on resume
> > x86/altp2m: don't consider "active" when enabling failed
> > x86/PV: '0' debug key stack dumping adjustments [at least the first 2]
> > x86/paging: restrict physical address width reported to guests
> > x86/PoD: defer nested P2M flushes
> > 
> > In principle "x86: more or less log-dirty related improvements" also
> > contains a number of bug fixes, but the way this series is stuck
> > made me not include it in the pings.
> > 
> > I've tried to remember to Cc you on these pings, but I may not have
> > succeeded in doing so.
> 
> Thanks.  I got at least most of them.
> 
> Roger, would you be able to review these changes ?

Yes, already on it :).

Roger.