[Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[Paul Durrant]

This email only tracks big items for xen.git tree. Please reply for items
you would like to see in 4.14 so that people have an idea what
is going on and prioritise accordingly.

You're welcome to provide description and use cases of the feature you're
working on.

= Timeline =

We now adopt a fixed cut-off date scheme. We will release about every 8
 months.
The critical dates for Xen 4.14 are as follows:

---> We are here
* Last posting date: May 1st, 2020
* Hard code freeze: May 22nd, 2020
* Release: June 26th, 2020

Note that we don't have a freeze exception scheme anymore. All patches
that wish to go into 4.14 must be posted initially no later than the
last posting date and finally no later than the hard code freeze.
All patches posted after that date will be automatically queued into next
release.

RCs will be arranged immediately after freeze.

There is also a jira instance to track all the tasks (not only big)
for the project. See: https://xenproject.atlassian.net/projects/XEN/issues.

Some of the tasks tracked by this e-mail also have a corresponding jira task
referred by XEN-N.

There is a version number for patch series associated to each feature.
Can each owner send an update giving the latest version number if the
series has been re-posted? Also, can the owners of any completed items
please respond so that the item can be moved into the 'Completed' section.

= Projects =

== Hypervisor == 

*  Live-Updating Xen
  -  David Woodhouse

*  Non-Cooperative Live Migration
  -  Paul Durrant

=== x86 === 

*  Intel Processor Trace virtualization enabling (v1)
  -  Luwei Kang

*  Linux stub domains (RFC v2)
  -  Marek Marczykowski-Górecki

*  Fixes to #DB injection
  -  Andrew Cooper

*  CPUID/MSR Xen/toolstack improvements
  -  Andrew Cooper

*  Improvements to domain_crash()
  -  Andrew Cooper

*  EIBRS
  -  Andrew Cooper

*  Xen ioreq server (v3)
  -  Roger Pau Monne

=== ARM === 

== Completed == 


Paul Durrant

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[Marek Marczykowski-Górecki]

[-- Attachment #1.1: Type: text/plain, Size: 349 bytes --]

On Wed, Jan 29, 2020 at 12:36:18PM +0000, Paul Durrant wrote:
> *  Linux stub domains (RFC v2)
>   -  Marek Marczykowski-Górecki

There is v4 series already.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[Jan Beulich]

On 29.01.2020 13:36, Paul Durrant wrote:
> === x86 === 
> 
> *  Intel Processor Trace virtualization enabling (v1)
>   -  Luwei Kang
> 
> *  Linux stub domains (RFC v2)
>   -  Marek Marczykowski-Górecki
> 
> *  Fixes to #DB injection
>   -  Andrew Cooper
> 
> *  CPUID/MSR Xen/toolstack improvements
>   -  Andrew Cooper
> 
> *  Improvements to domain_crash()
>   -  Andrew Cooper
> 
> *  EIBRS
>   -  Andrew Cooper
> 
> *  Xen ioreq server (v3)
>   -  Roger Pau Monne

Do you want to add "x86/HVM: implement memory read caching"
(https://lists.xenproject.org/archives/html/xen-devel/2018-09/msg00976.html)
here? I think I now have something coming a little closer to
what Andrew wants. It has its own downsides, so there being a
v4 (after well over a year) doesn't mean this will get it any
closer to going in.

Do you also want to add the ongoing x86 insn emulator work
here? Some parts were posted, some parts are still in needed
of finding time to carry out, and some parts are still pretty
vague.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[Durrant, Paul]

> -----Original Message-----
> From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Sent: 29 January 2020 12:44
> To: xen-devel@lists.xenproject.org; Durrant, Paul <pdurrant@amazon.co.uk>
> Cc: Woodhouse, David <dwmw@amazon.co.uk>; luwei.kang@intel.com;
> andrew.cooper3@citrix.com; roger.pau@citrix.com
> Subject: Re: [ANNOUNCE] Xen 4.14 Development Update
> 
> On Wed, Jan 29, 2020 at 12:36:18PM +0000, Paul Durrant wrote:
> > *  Linux stub domains (RFC v2)
> >   -  Marek Marczykowski-Górecki
> 
> There is v4 series already.
> 

Thanks. Will update for next time.

  Paul
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[Jürgen Groß]

On 29.01.20 13:36, Paul Durrant wrote:
> This email only tracks big items for xen.git tree. Please reply for items
> you would like to see in 4.14 so that people have an idea what
> is going on and prioritise accordingly.
> 
> You're welcome to provide description and use cases of the feature you're
> working on.
> 
> = Timeline =
> 
> We now adopt a fixed cut-off date scheme. We will release about every 8
>   months.
> The critical dates for Xen 4.14 are as follows:
> 
> ---> We are here
> * Last posting date: May 1st, 2020
> * Hard code freeze: May 22nd, 2020
> * Release: June 26th, 2020
> 
> Note that we don't have a freeze exception scheme anymore. All patches
> that wish to go into 4.14 must be posted initially no later than the
> last posting date and finally no later than the hard code freeze.
> All patches posted after that date will be automatically queued into next
> release.
> 
> RCs will be arranged immediately after freeze.
> 
> There is also a jira instance to track all the tasks (not only big)
> for the project. See: https://xenproject.atlassian.net/projects/XEN/issues.
> 
> Some of the tasks tracked by this e-mail also have a corresponding jira task
> referred by XEN-N.
> 
> There is a version number for patch series associated to each feature.
> Can each owner send an update giving the latest version number if the
> series has been re-posted? Also, can the owners of any completed items
> please respond so that the item can be moved into the 'Completed' section.
> 
> = Projects =
> 
> == Hypervisor ==
> 
> *  Live-Updating Xen
>    -  David Woodhouse
> 
> *  Non-Cooperative Live Migration
>    -  Paul Durrant

Hypervisor file system (v3)
- Juergen Gross


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[Durrant, Paul]

> -----Original Message-----
> From: Jan Beulich <jbeulich@suse.com>
> Sent: 29 January 2020 12:48
> To: Durrant, Paul <pdurrant@amazon.co.uk>
> Cc: xen-devel@lists.xenproject.org
> Subject: Re: [ANNOUNCE] Xen 4.14 Development Update
> 
> On 29.01.2020 13:36, Paul Durrant wrote:
> > === x86 ===
> >
> > *  Intel Processor Trace virtualization enabling (v1)
> >   -  Luwei Kang
> >
> > *  Linux stub domains (RFC v2)
> >   -  Marek Marczykowski-Górecki
> >
> > *  Fixes to #DB injection
> >   -  Andrew Cooper
> >
> > *  CPUID/MSR Xen/toolstack improvements
> >   -  Andrew Cooper
> >
> > *  Improvements to domain_crash()
> >   -  Andrew Cooper
> >
> > *  EIBRS
> >   -  Andrew Cooper
> >
> > *  Xen ioreq server (v3)
> >   -  Roger Pau Monne
> 
> Do you want to add "x86/HVM: implement memory read caching"
> (https://lists.xenproject.org/archives/html/xen-devel/2018-
> 09/msg00976.html)
> here? I think I now have something coming a little closer to
> what Andrew wants. It has its own downsides, so there being a
> v4 (after well over a year) doesn't mean this will get it any
> closer to going in.

It sounds like something that is reasonable to track. I can add it in (starting at v3, if v4 has not been posted by that point)

> 
> Do you also want to add the ongoing x86 insn emulator work
> here? Some parts were posted, some parts are still in needed
> of finding time to carry out, and some parts are still pretty
> vague.

Ok. It's on-going work so it may as well be tracked.

  Paul

> 
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[David Woodhouse]

[-- Attachment #1.1: Type: text/plain, Size: 3329 bytes --]

On Wed, 2020-01-29 at 12:36 +0000, Paul Durrant wrote:
> This email only tracks big items for xen.git tree. Please reply for items
> you would like to see in 4.14 so that people have an idea what
> is going on and prioritise accordingly.
> 
> You're welcome to provide description and use cases of the feature you're
> working on.
> 
> = Timeline =
> 
> We now adopt a fixed cut-off date scheme. We will release about every 8
>  months.
> The critical dates for Xen 4.14 are as follows:
> 
> ---> We are here
> * Last posting date: May 1st, 2020
> * Hard code freeze: May 22nd, 2020
> * Release: June 26th, 2020
> 
> Note that we don't have a freeze exception scheme anymore. All patches
> that wish to go into 4.14 must be posted initially no later than the
> last posting date and finally no later than the hard code freeze.
> All patches posted after that date will be automatically queued into next
> release.
> 
> RCs will be arranged immediately after freeze.
> 
> There is also a jira instance to track all the tasks (not only big)
> for the project. See: https://xenproject.atlassian.net/projects/XEN/issues.
> 
> Some of the tasks tracked by this e-mail also have a corresponding jira task
> referred by XEN-N.
> 
> There is a version number for patch series associated to each feature.
> Can each owner send an update giving the latest version number if the
> series has been re-posted? Also, can the owners of any completed items
> please respond so that the item can be moved into the 'Completed' section.
> 
> = Projects =
> 
> == Hypervisor == 
> 
> *  Live-Updating Xen
>   -  David Woodhouse

Latest RFC patchset posted is [RFC v2] at
https://lists.xenproject.org/archives/html/xen-devel/2020-01/msg01764.html

The tree at https://xenbits.xen.org/gitweb/?p=people/dwmw2/xen.git;a=shortlog;h=refs/heads/lu-master
as moved on a little since then, and I'll post [RFC v3] shortly.

So far this is mostly just the basic handover of a stream of migration
data from one Xen to the next, and allowing the new Xen to vmap and
process it early enough, and reserve the pages which already contain
domain data during its "boot".

In our development tree, we have a PV Dom0 surviving the transition.
We're working on turning those hacks into something we can show in
public.

I have updated the wiki page at https://wiki.xenproject.org/wiki/Live-Updating_Xen
which includes a reference to the handover protocol documentation.

This also depends on the hypervisor side of non-cooperative live
migration, mentioned below. But as well as that, parts of the memory
management are going to intersect with the secret hiding work that you
*didn't* call out in this email AFAICT.


> *  Non-Cooperative Live Migration
>   -  Paul Durrant
> 
> === x86 === 
> 
> *  Intel Processor Trace virtualization enabling (v1)
>   -  Luwei Kang
> 
> *  Linux stub domains (RFC v2)
>   -  Marek Marczykowski-Górecki
> 
> *  Fixes to #DB injection
>   -  Andrew Cooper
> 
> *  CPUID/MSR Xen/toolstack improvements
>   -  Andrew Cooper
> 
> *  Improvements to domain_crash()
>   -  Andrew Cooper
> 
> *  EIBRS
>   -  Andrew Cooper
> 
> *  Xen ioreq server (v3)
>   -  Roger Pau Monne
> 
> === ARM === 
> 
> == Completed == 
> 
> 
> Paul Durrant


[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5174 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[Tamas K Lengyel]

> === x86 ===

VM forking series is at v6

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[Andrew Cooper]

On 29/01/2020 12:36, Paul Durrant wrote:
> = Projects =
>
> == Hypervisor == 
>
> === x86 === 
>
> *  Intel Processor Trace virtualization enabling (v1)
>   -  Luwei Kang

Hasn't seen any activity in several releases.  Probably safe to drop. 
Also at least somewhat entangled with CPUID/MSR support.

> *  Fixes to #DB injection
>   -  Andrew Cooper

Parts of this manifested unexpected as XSA-308.  Rest is still work in
progress, with a TODO of how not to break introspection while fixing
it.  Unlikely to see any movement in the short term.

> *  CPUID/MSR Xen/toolstack improvements
>   -  Andrew Cooper

Very much in progress.

https://lore.kernel.org/xen-devel/20200127143444.25538-1-andrew.cooper3@citrix.com/
is v2 of the "move data into the migration stream" aspect.

I'm working on "rework boot time CPUID handling" right now, which will
ultimately allow MSRs to be included in what gets configured.

All of this forms a base for "Virtualise MSR_ARCH_CAPS for guests" which
is (now) the logical goal of the work, and might be a better tracking name.

In reality, for now I mean only the various *_NO bits because ...

> *  eIBRS
>   -  Andrew Cooper

... it is unfortunately quite tangled on top.  Because of the overloaded
nature of IBRS, even having Xen try to use eIBRS while only supporting
the legacy IBRS interface for guests turns into a massive headache.

I'm fairly confident that eIBRS support shouldn't be attempted until
general MSR_ARCH_CAPS is in place.

> *  Improvements to domain_crash()
>   -  Andrew Cooper

This is a fairly small piece of work, but I don't have time to progress
it.  I've got a rebased branch if anyone else feels like spending the
day or so it will take to get up to quality.

Another task is AMD hardware mitigations for everything
SpeculativeStoreBypass (i.e. pre-L1TF) and later.  There are crippling
performance problems caused by the lack of MSR_VIRT_SPEC_CTRL.  Since
that series I posted for that, Rome has gained several other hardware
features which need exposing to guests suitably.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[paul]

This email only tracks big items for xen.git tree. Please reply for items
you would like to see in 4.14 so that people have an idea what
is going on and prioritise accordingly.

You're welcome to provide description and use cases of the feature you're
working on.

= Timeline =

We now adopt a fixed cut-off date scheme. We will release about every 8
 months.
The critical dates for Xen 4.14 are as follows:

---> We are here
* Last posting date: May 1st, 2020
* Hard code freeze: May 22nd, 2020
* Release: June 26th, 2020

Note that we don't have a freeze exception scheme anymore. All patches
that wish to go into 4.14 must be posted initially no later than the
last posting date and finally no later than the hard code freeze.
All patches posted after that date will be automatically queued into next
release.

RCs will be arranged immediately after freeze.

There is also a jira instance to track all the tasks (not only big)
for the project. See: https://xenproject.atlassian.net/projects/XEN/issues.

Some of the tasks tracked by this e-mail also have a corresponding jira task
referred by XEN-N.

There is a version number for patch series associated to each feature.
Can each owner send an update giving the latest version number if the
series has been re-posted? Also, can the owners of any completed items
please respond so that the item can be moved into the 'Completed' section.

= Projects =

== Hypervisor == 

*  Live-Updating Xen (RFC v2)
  -  David Woodhouse
  -  The latest code is available at https://xenbits.xen.org/gitweb/?p=people/dwmw2/xen.git;a=shortlog;h=refs/heads/lu-master
  -  Project wiki page at https://wiki.xenproject.org/wiki/Live-Updating_Xen

*  Non-Cooperative Live Migration (v7 (design docs))
  -  Paul Durrant

*  Secret Hiding (v5)
  -  Hongyan Xia

*  Hypervisor file system (v6)
  -  Juergen Gross

=== x86 === 

*  Linux stub domains (v4)
  -  Marek Marczykowski-Górecki

*  Virtualise MSR_ARCH_CAPS for guests
  -  Andrew Cooper

*  AMD hardware mitigations (Rome)
  -  Andrew Cooper

*  Xen ioreq server (v3)
  -  Roger Pau Monne

*  Memory read caching in emulation (v5)
  -  Jan Beulich

*  Instruction emulator improvements
  -  Jan Beulich

*  VM forking (v11)
  -  Tamas K Lengyel

=== ARM === 

== Completed == 


Paul Durrant

[Xen-devel] [ANNOUNCE] Xen 4.14 Development Update

[Paul Durrant]

This email only tracks big items for xen.git tree. Please reply for items
you would like to see in 4.14 so that people have an idea what
is going on and prioritise accordingly.

You're welcome to provide description and use cases of the feature you're
working on.

= Timeline =

We now adopt a fixed cut-off date scheme. We will release about every 8
 months.
The critical dates for Xen 4.14 are as follows:

---> We are here
* Last posting date: May 1st, 2020
* Hard code freeze: May 22nd, 2020
* Release: June 26th, 2020

Note that we don't have a freeze exception scheme anymore. All patches
that wish to go into 4.14 must be posted initially no later than the
last posting date and finally no later than the hard code freeze.
All patches posted after that date will be automatically queued into next
release.

RCs will be arranged immediately after freeze.

There is also a jira instance to track all the tasks (not only big)
for the project. See: https://xenproject.atlassian.net/projects/XEN/issues.

Some of the tasks tracked by this e-mail also have a corresponding jira task
referred by XEN-N.

There is a version number for patch series associated to each feature.
Can each owner send an update giving the latest version number if the
series has been re-posted? Also, can the owners of any completed items
please respond so that the item can be moved into the 'Completed' section.

= Projects =

== Hypervisor == 

*  Live-Updating Xen (RFC v2)
  -  David Woodhouse
  -  The latest code is available at https://xenbits.xen.org/gitweb/?p=people/dwmw2/xen.git;a=shortlog;h=refs/heads/lu-master
  -  Project wiki page at https://wiki.xenproject.org/wiki/Live-Updating_Xen

*  Non-Cooperative Live Migration
  -  Paul Durrant

*  Secret Hiding (v5)
  -  Hongyan Xia

*  Hypervisor file system (v3)
  -  Juergen Gross

=== x86 === 

*  Linux stub domains (v4)
  -  Marek Marczykowski-Górecki

*  Virtualise MSR_ARCH_CAPS for guests
  -  Andrew Cooper

*  AMD hardware mitigations (Rome)
  -  Andrew Cooper

*  Xen ioreq server (v3)
  -  Roger Pau Monne

*  Memory read caching in emulation (v4)
  -  Jan Beulich

*  Instruction emulator improvements
  -  Jan Beulich

*  VM forking (v10)
  -  Tamas K Lengyel

=== ARM === 

== Completed == 


Paul Durrant

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel