Xen 4.16 development update - tree status

Xen 4.16 development update - tree status

[Ian Jackson]

Tree status
===========

We are now in deep code freeze, during which we will try to discover
and eliminate serious bugs and regressions.

All patches other than documentation patches need a Release-Ack.
Fixes for serious bugs, and test improvements, will get such an ack.

I have decided to branch on Monday at the same time as cutting RC4.
I think the release is going reasonably well.


The current planned release schedule
====================================

    Friday 12th November                  Hard code freeze [*]

      Bugfixes for serious bugs (including regressions), and low-risk
      fixes only.
      (0.5 weeks)

    Monday 22nd November **planned**     Branch off staging-4.16

      xen-unstable open again - with caveats to avoid release disruption.
      (1.5 weeks)

    Friday 26th November **tentative**    Final commits (docs/prep only)
    Week of 29th November **tentative**   Release
      (probably Tuesday or Wednesday)


Open issues and potential blockers
==================================

Here is the list of issues I am aware of that might be blockers and/or
might need a release ack for fixes.  In general, please would people
explicitly note release-blocker issues to me, so that I can see that
they are sorted out.


I am aware of one outstanding fix that has not yet been committed:

* [PATCH for-4.16 v2] efi: fix alignment of function parameters in compat
  which is an UB bug spotted due to a build failure with recent clang
  http://beefy18.nyi.freebsd.org/data/main-amd64-default/p5718aa5f2a8f_s9b8eb1c5b6/logs/xen-kernel-4.15.0_1.log


I am aware of three issues for which I don't know the current
disposition:

* "x86/IOMMU: enabled / intremap handling"
  3/3 "AMD/IOMMU: iommu_enable vs iommu_intremap"
  Last I heard this was being reworked.

* I have a note "HPET regression".  There have been number of patches in
  this area.  Most recently,
  "x86/hpet: setup HPET even when disabled due to stopping in deep C states"

* "x86/viridian: EOI MSR should always happen in affected vCPU context"
  Jan mentioned this on IRC but I don't see any discussion of it in
  my mailbox.

The last two were mentioned in my last release update, but I don't
seem to find any definitive conclusion.  x86 maintainers, can you
please confirm ?



Issues dealt with
=================

ARM/VPCI: all relevant fixes seem to have been applied now.

I have carrioed out Pre-public-disclosure testing of embargoed XSAs
385 387 388 389, together with the today-released XSA-390.

I have run through those of the release checklist items which are to
be done before branching, notably library sonames.

We have reverted "xen/domctl: Introduce XEN_DOMCTL_CDF_vpci flag" as
discussed.


Patches applied to fix bugs
---------------------------

"x86/xstate: reset cached register values on resume"

"tools: disable building qemu-trad per default"

Fixes for FreeBSD build problems following "tools: disable building
qemu-trad per default".

"VT-d: misc (regression) fixes":
1/3 "VT-d: per-domain IOMMU bitmap needs to have dynamic size"
2/3 "VT-d: fix reduced page table levels support when sharing tables"

"xen/efi: Fix Grub2 boot on arm64"

"gnttab: allow setting max version per-domain" and fixes thereto

"tools/helpers: fix broken xenstore stubdom init"

"xen/arm: fix SBDF calculation for vPCI MMIO handlers"

"x86/IOMMU: enabled / intremap handling"
1/3 x86/IOMMU: mark IOMMU / intremap not in use when ACPI tables are missing
2/3 x86/APIC: avoid iommu_supports_x2apic() on error path

"x86/traps: Fix typo in do_entry_CP()"


Changes slated to be deferred to post-4.16
------------------------------------------

Discussion of the following issues and patches, since the last release
update, has resulted in me declining to give a release-ack:

* Revert "domctl: improve locking during domain destruction"

* "VT-d: misc (regression) fixes":
  3/3 "VT-d: don't needlessly engage the untrusted-MSI workaround"

* "x86/passthrough: Fix hvm_gsi_eoi() build with GCC 12"

* "arm/efi: Improve performance requesting filesystem handle"

* "qemu build failure on release tarball with python <= 3.5"
  (mail thread, no patch exists at this time)

* "x86/x2APIC: defer probe until after IOMMU ACPI table parsing"

* "arm/smmuv1,v2: Protect smmu master list with a lock"


Note on RM decisionmaking
=========================

Release management decisions are always subject to revision on the
basis of new information or new arguments, so the list of changes
slated to be deferred should not be taken to be necessarily final.

Release management decisions usually involve much uncertainty (even
guesswork), and sometimes involve a balance of interests.  If you
think I have made a mistake and have new points, new information, or
think you can demonstrate that I have made the wrong tradeoff, please
let me know.

However, this message reflects my current understanding.


--

Re: Xen 4.16 development update - tree status

[Andrew Cooper]

On 19/11/2021 15:15, Ian Jackson wrote:
> Open issues and potential blockers
> ==================================
>
> I am aware of three issues for which I don't know the current
> disposition:
>
> * "x86/IOMMU: enabled / intremap handling"
>   3/3 "AMD/IOMMU: iommu_enable vs iommu_intremap"
>   Last I heard this was being reworked.

Some of this has been committed, and has fixed the regression I reported.

One part is quite risky, and not a regression fix (Xen has been broken
for years and years here).  I suggest deferring that part to 4.17.

I'm not aware of further actions required on this task for 4.16.

> * I have a note "HPET regression".  There have been number of patches in
>   this area.  Most recently,
>   "x86/hpet: setup HPET even when disabled due to stopping in deep C states"

This regression has been fixed too.  The bugfix got in ahead of RC1.

> * "x86/viridian: EOI MSR should always happen in affected vCPU context"
>   Jan mentioned this on IRC but I don't see any discussion of it in
>   my mailbox.
>
> The last two were mentioned in my last release update, but I don't
> seem to find any definitive conclusion.  x86 maintainers, can you
> please confirm ?

There is not consensus on whether the change is correct, although I
admit it had fallen off my radar.

All it is doing is introducing two ASSERT_UNREACHABLE()'s.  It's not
critical for 4.16 at this point.

~Andrew

Re: Xen 4.16 development update - tree status

[Ian Jackson]

Andrew Cooper writes ("Re: Xen 4.16 development update - tree status"):
> On 19/11/2021 15:15, Ian Jackson wrote:
> > Open issues and potential blockers
> > ==================================

Thanks, Andrew, for this helpful information.

> > I am aware of three issues for which I don't know the current
> > disposition:
> >
> > * "x86/IOMMU: enabled / intremap handling"
> >   3/3 "AMD/IOMMU: iommu_enable vs iommu_intremap"
> >   Last I heard this was being reworked.
> 
> Some of this has been committed, and has fixed the regression I reported.
> 
> One part is quite risky, and not a regression fix (Xen has been broken
> for years and years here).  I suggest deferring that part to 4.17.

Right.  That's 3/.  1/ and 2/ have been committed already.

> > * I have a note "HPET regression".  There have been number of patches in
> >   this area.  Most recently,
> >   "x86/hpet: setup HPET even when disabled due to stopping in deep C states"
> 
> This regression has been fixed too.  The bugfix got in ahead of RC1.

Oh, good, thanks.

> > * "x86/viridian: EOI MSR should always happen in affected vCPU context"
> >   Jan mentioned this on IRC but I don't see any discussion of it in
> >   my mailbox.
> >
> > The last two were mentioned in my last release update, but I don't
> > seem to find any definitive conclusion.  x86 maintainers, can you
> > please confirm ?
> 
> There is not consensus on whether the change is correct, although I
> admit it had fallen off my radar.
> 
> All it is doing is introducing two ASSERT_UNREACHABLE()'s.  It's not
> critical for 4.16 at this point.

OK, thanks for the information.  It sounds like this might be debated.
So if anyone disagrees, and thinks I ought to have this on my 4.16
radar, please let me know.

Meanwhile I just committed the EFI alignment fix, which was the one
other outstanding item.  It's too quiet.  I'm expecting a monster to
jump out at any moment...

Ian.

Re: Xen 4.16 development update - tree status

[Jan Beulich]

On 19.11.2021 16:15, Ian Jackson wrote:
> Tree status
> ===========
> 
> We are now in deep code freeze, during which we will try to discover
> and eliminate serious bugs and regressions.
> 
> All patches other than documentation patches need a Release-Ack.
> Fixes for serious bugs, and test improvements, will get such an ack.
> 
> I have decided to branch on Monday at the same time as cutting RC4.

May I suggest to consider doing both branching and RC4 after the XSA
batch has got committed on Tuesday? Or perhaps you did consider and
found good reasons not to?

Jan

Commit Moratorium (was Re: Xen 4.16 development update - tree status)

[Ian Jackson]

tl;dr:

  Commit moratorium is now in force until further notice
  (which will be given tomorrow afternoon).

Jan Beulich writes ("Re: Xen 4.16 development update - tree status"):
> On 19.11.2021 16:15, Ian Jackson wrote:
> > Tree status
> > ===========
> > 
> > We are now in deep code freeze, during which we will try to discover
> > and eliminate serious bugs and regressions.
> > 
> > All patches other than documentation patches need a Release-Ack.
> > Fixes for serious bugs, and test improvements, will get such an ack.
> > 
> > I have decided to branch on Monday at the same time as cutting RC4.
> 
> May I suggest to consider doing both branching and RC4 after the XSA
> batch has got committed on Tuesday? Or perhaps you did consider and
> found good reasons not to?

No.  I didn't consider that option.  Obviously I should have done.
And I think, in fact, it would be better.  I'll do some more private
testing of what will become RC4.

In the meantime, please would no-one commit to staging.

Ian.

Xen 4.16 development update - tree status

[Ian Jackson]

Tree status
===========

We are now in feature freeze.

Until code freeze, straightforward bugfixes (and documentation
changes) may continue to be committed without a Release-Ack,
until the Code Freeze, which will be on Friday 29th October.[1]

Any patches introducing features should have been committed by now.
All patches containing substantial refactoring need a Release-Ack,
even if the intent is to fix bugs.

If in doubt please ask for a Release-Ack.


The current planned release schedule
====================================

    Friday 29th October                   Code freeze

      Bugfixes only, all changes to be approved by the Release Manager,
      on the basis of a (progressively stricter[*]) risk assessment.
      (2 weeks)

    Friday 12th November **tentative**    Hard code freeze [*]

      Bugfixes for serious bugs (including regressions), and low-risk
      fixes only.
      (0.5 weeks)

    Tuesday 16th November **tentative**   Branch off staging-4.16

      xen-unstable open again - with caveats to avoid release disruption.
      (1.5 weeks)

    Friday 26th November **tentative**    Final commits (docs/prep only)
    Week of 29th November **tentative**   Release
      (probably Tuesday or Wednesday)


Open issues and blockers
========================

I am aware of a number of oustanding bug fixes, particularly for x86.
I would appreciate attention from maintainers to these so that they
can be applied sooner rather than later.  I don't believe any of these
have been explicitly flagged as RC, or regressions, but I think my
information may be incomplete.

In general, please would people explicitly note release-blocker issues
to me, so that I can see that they are sorted out.

I'm aware of one thing that I definitely want to see fixed before
release: the patch "Enable the existing x86 virtual PCI support for
ARM" needs rework.  I believe this is in hand.


Ian.


[1] As previously discussed, this timescale has been compressed
    by one week.

[*] The distinction between Code Freeze and Hard Code Freeze is a
matter of degree, not kind; the Hard Code Freeze data and associated
tighter policy text is indicative rather than normative.