Xen 4.5 development update

Xen 4.5 development update

[konrad.wilk]

[-- Attachment #1: Type: text/plain, Size: 5605 bytes --]

Below is a summary of the projects / features being worked on for the 4.5
time frame that I had been gathering.

It is not complete! I would like folks input if I had missed something
or screwed up. Will also talk at Xen Hackahon about this and take a look
at  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the
items there should move over.

The tentative feature freeze is scheduled for September 10th,
which is months away.  With that in mind, I think it's time to take
stock of the development, so we know whether to ask for more help or divert
resources.

For items involving code hosted on the Xen.org site (including qemu-xen),
that means a likelihood of having the feature code-complete and mostly
working by the feature freeze.  (It's OK if there are still bugs to be
worked out.)  For items in Linux, I think it would mean having items on track
to make it into the kernel released just after the scheduled 4.5 time frame.
Not sure what that means for libvirt. :-)

For items involving code hosted on the Xen.org site (including qemu-xen),
that means a likelihood of having the feature code-complete and mostly
working by the feature freeze.  (It's OK if there are still bugs to be
worked out.)  For items in Linux, I think it would mean having items on track
to make it into the kernel released just after the scheduled 4.5 time frame.
Not sure what that means for libvirt. :-)

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:
* Feature Freeze: 10th September 2014
* First RC: 10th October
* Release: 10th December 2014

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

= Prognosis =

The prognosis is a numerical value of the likehood of the feature/code
making it in the code-base.

If folks prefer the fair, good, excellent marker I can switch over to
that.

= Open =

* Andrew Cooper  Prognosis:   100 %
    libx{c,l} error handling cleanup 
    New migration.
    cpuid leveling

* Andrew Benniest  Prognosis:   100 %
    Netback multiqueue

* Arianna Avanzini  Prognosis:   100 %
    block multiqueue
    XEN_DOMCTL_memory_mapping hypercall for ARM

* Elena Ufimtseva  Prognosis:   0 %
    vNUMA in Xen
    vNUMA in Linux

* Boris Ostrovsky  Prognosis:   100 %
    VPMU - 'perf' support in Xen
    VPMU - 'perf' support in Linux
    vAPIC in PVHVM guests

* Bob Liu  Prognosis:   100 %
    tmem cleanups/fixes
    1TB slow destruction

* Dario Faggioli  Prognosis:   0 %
    Soft affinity for vcpus (was NUMA affinity for vcpus)

* Matt Wilson  Prognosis:   0 %
    HVM guest NUMA

* Don Slutz  Prognosis:   100 %
    Bigger PCI hole in QEMU 
    Re-write of HPET

* David Vrabel:  Prognosis:   100 %
    New migration.

* Daniel Kiper  Prognosis:   100 %
    GRUB2 multiboot2
    Xen multiboot2 support
    Linux pvops of Xen EFI hypercall support
    libxl/xl - xm compatibility mode for mem-max and mem-set;
    Rearrange and cleanup installation destination directories (/var -> var/lib/xen)

* George Dunlap:  Prognosis:   100 %
    Default to credit2
    >  cpu pinning, numa affinity and cpu reservation

* Roger Pau Monné  Prognosis:   100 %
    Xen PVH dom0
    PVH FreeBSD dom0

* Konrad Rzeszutek Wilk  Prognosis:   30 %
    NUMA memory scrubbing
    Performance fixes for PCI passthrough

* Kelly Zytaruk  Prognosis:   100 %
    AMD Radeon PCI GPU passthrough

* Chen, Tiejun  Prognosis:   100 %
    Intel IGD PCI GPU passthrough

* Mukesh Rathor  Prognosis:   100 %
    Xen PVH dom0
    Linux PVH dom0

* Wei Liu  Prognosis:   100 %
    Adding missing 'xend' features in libxl
    xl list -l on a dom0-only system
    xl list -l doesn't contain tty console port
    xl: passing more defaults in configuration in xl.conf
    >  There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.

* Ian Campbel  Prognosis:   100 %
    OSSTest: libvirt
    OSSTest: upstream QEMU

* Ian Jackson  Prognosis:   100 %
    xl does not handle migrate interruption gracefully
    >  If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains

* Stefano Stabellini  Prognosis:   100 %
    <NONE>

* Julien Grall  Prognosis:   100 %
    ARM IOMMU support

* Malcolm Crossley  Prognosis:   100 %
    IOMMU ABI for guests to map their DMA regions

* Zoltan Kiss  Prognosis:   100 %
    Netback grant table manipulations
    "Short" grant copy (just header) of packets. 

* Feng Wu  Prognosis:   100 %
    SMAP
    alternative_asm in Xen

* Zhang, Yang Z  Prognosis:   100 %
    dirty vram / IOMMU bug

* Paul Durrant  Prognosis:   100 %
    ioreq-server, aka secondary emulators

* Jan Beulich  Prognosis:   100 %
    Stability

* Olaf Hering  Prognosis:   100 %
    libvirt and xl discard support, so that libvirt can start using it
    pvscsi should be targeted for 4.5, a prototype exists
    live migration knobs, there is no suitable code yet, just ideas

* Joe Doe aka not assigned to anybody  Prognosis:   0 %
    PoD fixes
    TLB flushing without locks in Xen
    xl does not support specifying virtual function for passthrough device
    >  http://bugs.xenproject.org/xen/bug/22
    PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
    >  http://bugs.xenproject.org/xen/bug/28



[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update

[Wei Huang]

Hi Konrad,

I would expect the following item to be added for Xen 4.5. The patch set 
has been reviewed for several rounds and shouldn't have any major 
technical challenge.

* Wei Huang Prognosis:   90 %
       ARM VM save/restore/live migration

Also, I don't see many ARM features, other than IOMMU and memory_map 
hypercall, in your list. There should be more. Maybe Stefano/IanC/Julien 
can gather a list for ARM?

Thanks,
-Wei

On 05/27/2014 01:06 PM, konrad.wilk@oracle.com wrote:
> Below is a summary of the projects / features being worked on for the 4.5
> time frame that I had been gathering.
>
> It is not complete! I would like folks input if I had missed something
> or screwed up. Will also talk at Xen Hackahon about this and take a look
> at  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the
> items there should move over.
>
> The tentative feature freeze is scheduled for September 10th,
> which is months away.  With that in mind, I think it's time to take
> stock of the development, so we know whether to ask for more help or divert
> resources.
>
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, I think it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
> Not sure what that means for libvirt. :-)
>
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, I think it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
> Not sure what that means for libvirt. :-)
>
> = Timeline =
>
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 10th September 2014
> * First RC: 10th October
> * Release: 10th December 2014
>
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
>
> = Prognosis =
>
> The prognosis is a numerical value of the likehood of the feature/code
> making it in the code-base.
>
> If folks prefer the fair, good, excellent marker I can switch over to
> that.
>
> = Open =
>
> * Andrew Cooper  Prognosis:   100 %
>      libx{c,l} error handling cleanup
>      New migration.
>      cpuid leveling
>
> * Andrew Benniest  Prognosis:   100 %
>      Netback multiqueue
>
> * Arianna Avanzini  Prognosis:   100 %
>      block multiqueue
>      XEN_DOMCTL_memory_mapping hypercall for ARM
>
> * Elena Ufimtseva  Prognosis:   0 %
>      vNUMA in Xen
>      vNUMA in Linux
>
> * Boris Ostrovsky  Prognosis:   100 %
>      VPMU - 'perf' support in Xen
>      VPMU - 'perf' support in Linux
>      vAPIC in PVHVM guests
>
> * Bob Liu  Prognosis:   100 %
>      tmem cleanups/fixes
>      1TB slow destruction
>
> * Dario Faggioli  Prognosis:   0 %
>      Soft affinity for vcpus (was NUMA affinity for vcpus)
>
> * Matt Wilson  Prognosis:   0 %
>      HVM guest NUMA
>
> * Don Slutz  Prognosis:   100 %
>      Bigger PCI hole in QEMU
>      Re-write of HPET
>
> * David Vrabel:  Prognosis:   100 %
>      New migration.
>
> * Daniel Kiper  Prognosis:   100 %
>      GRUB2 multiboot2
>      Xen multiboot2 support
>      Linux pvops of Xen EFI hypercall support
>      libxl/xl - xm compatibility mode for mem-max and mem-set;
>      Rearrange and cleanup installation destination directories (/var -> var/lib/xen)
>
> * George Dunlap:  Prognosis:   100 %
>      Default to credit2
>      >  cpu pinning, numa affinity and cpu reservation
>
> * Roger Pau Monn챕  Prognosis:   100 %
>      Xen PVH dom0
>      PVH FreeBSD dom0
>
> * Konrad Rzeszutek Wilk  Prognosis:   30 %
>      NUMA memory scrubbing
>      Performance fixes for PCI passthrough
>
> * Kelly Zytaruk  Prognosis:   100 %
>      AMD Radeon PCI GPU passthrough
>
> * Chen, Tiejun  Prognosis:   100 %
>      Intel IGD PCI GPU passthrough
>
> * Mukesh Rathor  Prognosis:   100 %
>      Xen PVH dom0
>      Linux PVH dom0
>
> * Wei Liu  Prognosis:   100 %
>      Adding missing 'xend' features in libxl
>      xl list -l on a dom0-only system
>      xl list -l doesn't contain tty console port
>      xl: passing more defaults in configuration in xl.conf
>      >  There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.
>
> * Ian Campbel  Prognosis:   100 %
>      OSSTest: libvirt
>      OSSTest: upstream QEMU
>
> * Ian Jackson  Prognosis:   100 %
>      xl does not handle migrate interruption gracefully
>      >  If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains
>
> * Stefano Stabellini  Prognosis:   100 %
>      <NONE>
>
> * Julien Grall  Prognosis:   100 %
>      ARM IOMMU support
>
> * Malcolm Crossley  Prognosis:   100 %
>      IOMMU ABI for guests to map their DMA regions
>
> * Zoltan Kiss  Prognosis:   100 %
>      Netback grant table manipulations
>      "Short" grant copy (just header) of packets.
>
> * Feng Wu  Prognosis:   100 %
>      SMAP
>      alternative_asm in Xen
>
> * Zhang, Yang Z  Prognosis:   100 %
>      dirty vram / IOMMU bug
>
> * Paul Durrant  Prognosis:   100 %
>      ioreq-server, aka secondary emulators
>
> * Jan Beulich  Prognosis:   100 %
>      Stability
>
> * Olaf Hering  Prognosis:   100 %
>      libvirt and xl discard support, so that libvirt can start using it
>      pvscsi should be targeted for 4.5, a prototype exists
>      live migration knobs, there is no suitable code yet, just ideas
>
> * Joe Doe aka not assigned to anybody  Prognosis:   0 %
>      PoD fixes
>      TLB flushing without locks in Xen
>      xl does not support specifying virtual function for passthrough device
>      >  http://bugs.xenproject.org/xen/bug/22
>      PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
>      >  http://bugs.xenproject.org/xen/bug/28
>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 7517 bytes --]

On Tue, 27 May 2014, Wei Huang wrote:
> Hi Konrad,
> 
> I would expect the following item to be added for Xen 4.5. The patch set has
> been reviewed for several rounds and shouldn't have any major technical
> challenge.
> 
> * Wei Huang Prognosis:   90 %
>       ARM VM save/restore/live migration
> 
> Also, I don't see many ARM features, other than IOMMU and memory_map
> hypercall, in your list. There should be more. Maybe Stefano/IanC/Julien can
> gather a list for ARM?

Yes, indeed. Please add:

Vijay Kilari:   90%
    GICv3 support

Julien Grall:   0%
    device assignment on ARM

Roy Franz:      0%
    Xen UEFI booting on ARM

Linaro (unknown): 0%
    GICv2m support



> Thanks,
> -Wei
> 
> On 05/27/2014 01:06 PM, konrad.wilk@oracle.com wrote:
> > Below is a summary of the projects / features being worked on for the 4.5
> > time frame that I had been gathering.
> > 
> > It is not complete! I would like folks input if I had missed something
> > or screwed up. Will also talk at Xen Hackahon about this and take a look
> > at  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the
> > items there should move over.
> > 
> > The tentative feature freeze is scheduled for September 10th,
> > which is months away.  With that in mind, I think it's time to take
> > stock of the development, so we know whether to ask for more help or divert
> > resources.
> > 
> > For items involving code hosted on the Xen.org site (including qemu-xen),
> > that means a likelihood of having the feature code-complete and mostly
> > working by the feature freeze.  (It's OK if there are still bugs to be
> > worked out.)  For items in Linux, I think it would mean having items on
> > track
> > to make it into the kernel released just after the scheduled 4.5 time frame.
> > Not sure what that means for libvirt. :-)
> > 
> > For items involving code hosted on the Xen.org site (including qemu-xen),
> > that means a likelihood of having the feature code-complete and mostly
> > working by the feature freeze.  (It's OK if there are still bugs to be
> > worked out.)  For items in Linux, I think it would mean having items on
> > track
> > to make it into the kernel released just after the scheduled 4.5 time frame.
> > Not sure what that means for libvirt. :-)
> > 
> > = Timeline =
> > 
> > We are planning on a 9-month release cycle.  Based on that, below are
> > our estimated dates:
> > * Feature Freeze: 10th September 2014
> > * First RC: 10th October
> > * Release: 10th December 2014
> > 
> > The RCs and release will of course depend on stability and bugs, and
> > will therefore be fairly unpredictable.  The feature freeze may be
> > slipped for especially important features which are near completion.
> > 
> > = Prognosis =
> > 
> > The prognosis is a numerical value of the likehood of the feature/code
> > making it in the code-base.
> > 
> > If folks prefer the fair, good, excellent marker I can switch over to
> > that.
> > 
> > = Open =
> > 
> > * Andrew Cooper  Prognosis:   100 %
> >      libx{c,l} error handling cleanup
> >      New migration.
> >      cpuid leveling
> > 
> > * Andrew Benniest  Prognosis:   100 %
> >      Netback multiqueue
> > 
> > * Arianna Avanzini  Prognosis:   100 %
> >      block multiqueue
> >      XEN_DOMCTL_memory_mapping hypercall for ARM
> > 
> > * Elena Ufimtseva  Prognosis:   0 %
> >      vNUMA in Xen
> >      vNUMA in Linux
> > 
> > * Boris Ostrovsky  Prognosis:   100 %
> >      VPMU - 'perf' support in Xen
> >      VPMU - 'perf' support in Linux
> >      vAPIC in PVHVM guests
> > 
> > * Bob Liu  Prognosis:   100 %
> >      tmem cleanups/fixes
> >      1TB slow destruction
> > 
> > * Dario Faggioli  Prognosis:   0 %
> >      Soft affinity for vcpus (was NUMA affinity for vcpus)
> > 
> > * Matt Wilson  Prognosis:   0 %
> >      HVM guest NUMA
> > 
> > * Don Slutz  Prognosis:   100 %
> >      Bigger PCI hole in QEMU
> >      Re-write of HPET
> > 
> > * David Vrabel:  Prognosis:   100 %
> >      New migration.
> > 
> > * Daniel Kiper  Prognosis:   100 %
> >      GRUB2 multiboot2
> >      Xen multiboot2 support
> >      Linux pvops of Xen EFI hypercall support
> >      libxl/xl - xm compatibility mode for mem-max and mem-set;
> >      Rearrange and cleanup installation destination directories (/var ->
> > var/lib/xen)
> > 
> > * George Dunlap:  Prognosis:   100 %
> >      Default to credit2
> >      >  cpu pinning, numa affinity and cpu reservation
> > 
> > * Roger Pau Monn챕  Prognosis:   100 %
> >      Xen PVH dom0
> >      PVH FreeBSD dom0
> > 
> > * Konrad Rzeszutek Wilk  Prognosis:   30 %
> >      NUMA memory scrubbing
> >      Performance fixes for PCI passthrough
> > 
> > * Kelly Zytaruk  Prognosis:   100 %
> >      AMD Radeon PCI GPU passthrough
> > 
> > * Chen, Tiejun  Prognosis:   100 %
> >      Intel IGD PCI GPU passthrough
> > 
> > * Mukesh Rathor  Prognosis:   100 %
> >      Xen PVH dom0
> >      Linux PVH dom0
> > 
> > * Wei Liu  Prognosis:   100 %
> >      Adding missing 'xend' features in libxl
> >      xl list -l on a dom0-only system
> >      xl list -l doesn't contain tty console port
> >      xl: passing more defaults in configuration in xl.conf
> >      >  There are a number of options for which it might be useful to pass a
> > default in xl.conf.  For example, if we could have a default "backend"
> > parameter for vifs, then it would be easy to switch back and forth between a
> > backend in a driver domain and a backend in dom0.
> > 
> > * Ian Campbel  Prognosis:   100 %
> >      OSSTest: libvirt
> >      OSSTest: upstream QEMU
> > 
> > * Ian Jackson  Prognosis:   100 %
> >      xl does not handle migrate interruption gracefully
> >      >  If you start a localhost migrate, and press "Ctrl-C" in the middle,
> > you get two hung domains
> > 
> > * Stefano Stabellini  Prognosis:   100 %
> >      <NONE>
> > 
> > * Julien Grall  Prognosis:   100 %
> >      ARM IOMMU support
> > 
> > * Malcolm Crossley  Prognosis:   100 %
> >      IOMMU ABI for guests to map their DMA regions
> > 
> > * Zoltan Kiss  Prognosis:   100 %
> >      Netback grant table manipulations
> >      "Short" grant copy (just header) of packets.
> > 
> > * Feng Wu  Prognosis:   100 %
> >      SMAP
> >      alternative_asm in Xen
> > 
> > * Zhang, Yang Z  Prognosis:   100 %
> >      dirty vram / IOMMU bug
> > 
> > * Paul Durrant  Prognosis:   100 %
> >      ioreq-server, aka secondary emulators
> > 
> > * Jan Beulich  Prognosis:   100 %
> >      Stability
> > 
> > * Olaf Hering  Prognosis:   100 %
> >      libvirt and xl discard support, so that libvirt can start using it
> >      pvscsi should be targeted for 4.5, a prototype exists
> >      live migration knobs, there is no suitable code yet, just ideas
> > 
> > * Joe Doe aka not assigned to anybody  Prognosis:   0 %
> >      PoD fixes
> >      TLB flushing without locks in Xen
> >      xl does not support specifying virtual function for passthrough device
> >      >  http://bugs.xenproject.org/xen/bug/22
> >      PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with
> > PCI/GPU passthrough
> >      >  http://bugs.xenproject.org/xen/bug/28
> > 
> > 
> > 
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> > 
> 

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update

[Julien Grall]

On 05/28/2014 12:43 PM, Stefano Stabellini wrote:
> On Tue, 27 May 2014, Wei Huang wrote:
>> Hi Konrad,
>>
>> I would expect the following item to be added for Xen 4.5. The patch set has
>> been reviewed for several rounds and shouldn't have any major technical
>> challenge.
>>
>> * Wei Huang Prognosis:   90 %
>>       ARM VM save/restore/live migration
>>
>> Also, I don't see many ARM features, other than IOMMU and memory_map
>> hypercall, in your list. There should be more. Maybe Stefano/IanC/Julien can
>> gather a list for ARM?
> 
> Yes, indeed. Please add:
> 
> Vijay Kilari:   90%
>     GICv3 support
> 
> Julien Grall:   0%
>     device assignment on ARM
> 
> Roy Franz:      0%
>     Xen UEFI booting on ARM
> 
> Linaro (unknown): 0%
>     GICv2m support

One missing item.

Parth Dixit:	 0%
   PSCI v0.2

Regards,

-- 
Julien Grall

Re: Xen 4.5 development update

[Wei Liu]

On Tue, May 27, 2014 at 02:06:40PM -0400, konrad.wilk@oracle.com wrote:
[...]
> 
> * Wei Liu  Prognosis:   100 %
>     Adding missing 'xend' features in libxl
>     xl list -l on a dom0-only system
>     xl list -l doesn't contain tty console port
>     xl: passing more defaults in configuration in xl.conf
>     >  There are a number of options for which it might be useful to
>     >  pass a default in xl.conf.  For example, if we could have a
>     >  default "backend" parameter for vifs, then it would be easy to
>     >  switch back and forth between a backend in a driver domain and
>     >  a backend in dom0.
> 

If you're referring to my work on libxl, then there's misunderstanding.
My work is orthogonal to all these problems; however I can see the new
infrastructure might enable us to fix the 'xl list' problems in a more
sensible way.

Wei.

Re: Xen 4.5 development update

[Andrew Cooper]

On 27/05/2014 19:06, konrad.wilk@oracle.com wrote:
>
> = Open =
>
> * Andrew Cooper  Prognosis:   100 %
>     libx{c,l} error handling cleanup 

I certainly advocated this as a critical item which has rolled over from
previous Xen releases (perhaps the biggest issue being raised during
XSA-55 where several functions were identified as being unable to
determine whether the call succeeded or failed, given the integer
returned to the caller).  We can certainly discuss this at the
hackathon, but given my other obligations (new migration first,
XenServer second) I am probably not the best person to lead this,
although I will certainly help where possible.

>     New migration.

This is certainly the main focus of work, and top priority as far as
myself, David, and XenServer is concerned.

>     cpuid leveling

While this is certainly a key issue, it has been deprioritised behind
new migration.  There is still a lot of work to do there, so I can't
make any guarantees at this point as to when I can start it, let alone
finish it, and where that will sit in the 4.5 timeframe.  Hopefully, but
no certainty.

>
> * Andrew Benniest  Prognosis:   100 %
>     Netback multiqueue

Andrew left Citrix recently.  As far as I am aware, the patches are all
acked and ready to be merged.  Baring any complications, David is
probably the best person to keep on top of them, should any further work
be required.

(I realise I am nominating David for work without his explicit consent
in this regard, but hope it will be accepted in principle)

~Andrew

Re: Xen 4.5 development update

[Konrad Rzeszutek Wilk]

On Wed, May 28, 2014 at 12:43:46AM +0100, Andrew Cooper wrote:
> On 27/05/2014 19:06, konrad.wilk@oracle.com wrote:
> >
> > = Open =
> >
> > * Andrew Cooper  Prognosis:   100 %
> >     libx{c,l} error handling cleanup 
> 
> I certainly advocated this as a critical item which has rolled over from
> previous Xen releases (perhaps the biggest issue being raised during
> XSA-55 where several functions were identified as being unable to
> determine whether the call succeeded or failed, given the integer
> returned to the caller).  We can certainly discuss this at the
> hackathon, but given my other obligations (new migration first,
> XenServer second) I am probably not the best person to lead this,
> although I will certainly help where possible.

Let me move it to the unassigned part.
> 
> >     New migration.
> 
> This is certainly the main focus of work, and top priority as far as
> myself, David, and XenServer is concerned.
> 
> >     cpuid leveling
> 
> While this is certainly a key issue, it has been deprioritised behind
> new migration.  There is still a lot of work to do there, so I can't
> make any guarantees at this point as to when I can start it, let alone
> finish it, and where that will sit in the 4.5 timeframe.  Hopefully, but
> no certainty.


Lets keep it then, and just give you a 50% chance of getting both
items in Xen 4.5.

> 
> >
> > * Andrew Benniest  Prognosis:   100 %
> >     Netback multiqueue
> 
> Andrew left Citrix recently.  As far as I am aware, the patches are all
> acked and ready to be merged.  Baring any complications, David is
> probably the best person to keep on top of them, should any further work
> be required.
> 
> (I realise I am nominating David for work without his explicit consent
> in this regard, but hope it will be accepted in principle)

I am sure he will be fine :-)

Thanks for the update!
> 
> ~Andrew
> 

Re: Xen 4.5 development update

[Aravindh Puthiyaparambil (aravindp)]

>Below is a summary of the projects / features being worked on for the 4.5
>time frame that I had been gathering.
>
>It is not complete! I would like folks input if I had missed something or
>screwed up. Will also talk at Xen Hackahon about this and take a look at
>http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the items there
>should move over.

Hi Konrad,

I am working on extending mem_access support to PV domains. I sent out an RFC and am gearing up to send out another. The aim is to have it included in the 4.5 time frame. Could you please add this to your list?

Thanks,
Aravindh

Re: Xen 4.5 development update

[Jan Beulich]

>>> On 28.05.14 at 01:56, <aravindp@cisco.com> wrote:
>> Below is a summary of the projects / features being worked on for the 4.5
>>time frame that I had been gathering.
>>
>>It is not complete! I would like folks input if I had missed something or
>>screwed up. Will also talk at Xen Hackahon about this and take a look at
>>http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the items there
>>should move over.
> 
> I am working on extending mem_access support to PV domains. I sent out an 
> RFC and am gearing up to send out another. The aim is to have it included in 
> the 4.5 time frame. Could you please add this to your list?

With, at this time, perhaps relatively low likelihood due to the
technical problems you're having that no solution is visible for so
far...

Jan

Re: Xen 4.5 development update

[Aravindh Puthiyaparambil (aravindp)]

>> I am working on extending mem_access support to PV domains. I sent out
>an
>> RFC and am gearing up to send out another. The aim is to have it included in
>> the 4.5 time frame. Could you please add this to your list?
>
>With, at this time, perhaps relatively low likelihood due to the
>technical problems you're having that no solution is visible for so
>far...

Now I am confused about the conclusion of the previous thread regarding this. I will reply to that so as to not hijack this thread and you can set me straight.

Aravindh

Re: Xen 4.5 development update

[Konrad Rzeszutek Wilk]

On Tue, May 27, 2014 at 11:56:35PM +0000, Aravindh Puthiyaparambil (aravindp) wrote:
> >Below is a summary of the projects / features being worked on for the 4.5
> >time frame that I had been gathering.
> >
> >It is not complete! I would like folks input if I had missed something or
> >screwed up. Will also talk at Xen Hackahon about this and take a look at
> >http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the items there
> >should move over.
> 
> Hi Konrad,
> 
> I am working on extending mem_access support to PV domains. I sent out an RFC and am gearing up to send out another. The aim is to have it included in the 4.5 time frame. Could you please add this to your list?

Done!

Thanks
> 
> Thanks,
> Aravindh
> 

Re: Xen 4.5 development update

[Roger Pau Monné]

On 27/05/14 19:06, konrad.wilk@oracle.com wrote:
> Below is a summary of the projects / features being worked on for the 4.5
> time frame that I had been gathering.
> 
> It is not complete! I would like folks input if I had missed something
> or screwed up. Will also talk at Xen Hackahon about this and take a look
> at  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the
> items there should move over.
> 
> The tentative feature freeze is scheduled for September 10th,
> which is months away.  With that in mind, I think it's time to take
> stock of the development, so we know whether to ask for more help or divert
> resources.
> 
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, I think it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
> Not sure what that means for libvirt. :-)
> 
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, I think it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
> Not sure what that means for libvirt. :-)
> 
> = Timeline =
> 
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 10th September 2014
> * First RC: 10th October
> * Release: 10th December 2014
> 
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
> 
> = Prognosis =

[...]

> * Roger Pau Monné  Prognosis:   100 %
>     Xen PVH dom0
>     PVH FreeBSD dom0

Hello Konrad,

Thanks for doing this list, I certainly think Xen PVH Dom0 will make it
to 4.5, most patches are already committed, and the remaining ones are
already reviewed AFAIK. So very soon we will have PVH Dom0 available on
Intel HW with EPT support, which brings the next question, what do we
think we should work on after that? There are a bunch of things still
pending for PVH, and I'm not sure we will be able to get all them done
for 4.5, here's a list on top of my head:

 - PCI passthrough for DomU.
 - Migration of PVH DomUs.
 - Migration of guests from a PVH Dom0.
 - AMD hardware support.
 - PVH working with shadow.

IMHO, I think the two most important items would be getting PVH working
on AMD hardware and shadow, but maybe others have different opinions.

Roger.

Re: Xen 4.5 development update

[Konrad Rzeszutek Wilk]

On Wed, May 28, 2014 at 11:01:43AM +0100, Roger Pau Monné wrote:
> On 27/05/14 19:06, konrad.wilk@oracle.com wrote:
> > Below is a summary of the projects / features being worked on for the 4.5
> > time frame that I had been gathering.
> > 
> > It is not complete! I would like folks input if I had missed something
> > or screwed up. Will also talk at Xen Hackahon about this and take a look
> > at  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the
> > items there should move over.
> > 
> > The tentative feature freeze is scheduled for September 10th,
> > which is months away.  With that in mind, I think it's time to take
> > stock of the development, so we know whether to ask for more help or divert
> > resources.
> > 
> > For items involving code hosted on the Xen.org site (including qemu-xen),
> > that means a likelihood of having the feature code-complete and mostly
> > working by the feature freeze.  (It's OK if there are still bugs to be
> > worked out.)  For items in Linux, I think it would mean having items on track
> > to make it into the kernel released just after the scheduled 4.5 time frame.
> > Not sure what that means for libvirt. :-)
> > 
> > For items involving code hosted on the Xen.org site (including qemu-xen),
> > that means a likelihood of having the feature code-complete and mostly
> > working by the feature freeze.  (It's OK if there are still bugs to be
> > worked out.)  For items in Linux, I think it would mean having items on track
> > to make it into the kernel released just after the scheduled 4.5 time frame.
> > Not sure what that means for libvirt. :-)
> > 
> > = Timeline =
> > 
> > We are planning on a 9-month release cycle.  Based on that, below are
> > our estimated dates:
> > * Feature Freeze: 10th September 2014
> > * First RC: 10th October
> > * Release: 10th December 2014
> > 
> > The RCs and release will of course depend on stability and bugs, and
> > will therefore be fairly unpredictable.  The feature freeze may be
> > slipped for especially important features which are near completion.
> > 
> > = Prognosis =
> 
> [...]
> 
> > * Roger Pau Monné  Prognosis:   100 %
> >     Xen PVH dom0
> >     PVH FreeBSD dom0
> 
> Hello Konrad,

Hey!
> 
> Thanks for doing this list, I certainly think Xen PVH Dom0 will make it
> to 4.5, most patches are already committed, and the remaining ones are
> already reviewed AFAIK. So very soon we will have PVH Dom0 available on
> Intel HW with EPT support, which brings the next question, what do we
> think we should work on after that? There are a bunch of things still
> pending for PVH, and I'm not sure we will be able to get all them done
> for 4.5, here's a list on top of my head:
> 
>  - PCI passthrough for DomU.
>  - Migration of PVH DomUs.
>  - Migration of guests from a PVH Dom0.
>  - AMD hardware support.
>  - PVH working with shadow.
> 
> IMHO, I think the two most important items would be getting PVH working
> on AMD hardware and shadow, but maybe others have different opinions.

I concur. I know that Mukesh had been in the time between reviews eyeballing
the AMD part, but nothing concrete yet.

I would say that the list of interesting/important items could be 
in this order:

  - Migration of PVH DomUs.
  - Migration of guests from a PVH Dom0.
  - AMD hardware support (Mukesh).
  - PVH working with shadow.
  - PCI passthrough for DomU.

But that is just my take.

> 
> Roger.
> 

Re: Xen 4.5 development update

[Stefano Stabellini]

On Tue, 27 May 2014, konrad.wilk@oracle.com wrote:
> Below is a summary of the projects / features being worked on for the 4.5
> time frame that I had been gathering.
> 
> It is not complete! I would like folks input if I had missed something
> or screwed up. Will also talk at Xen Hackahon about this and take a look
> at  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the
> items there should move over.
> 
> The tentative feature freeze is scheduled for September 10th,
> which is months away.  With that in mind, I think it's time to take
> stock of the development, so we know whether to ask for more help or divert
> resources.
> 
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, I think it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
> Not sure what that means for libvirt. :-)
> 
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, I think it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
> Not sure what that means for libvirt. :-)
> 
> = Timeline =
> 
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 10th September 2014
> * First RC: 10th October
> * Release: 10th December 2014
> 
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
> 
> = Prognosis =
> 
> The prognosis is a numerical value of the likehood of the feature/code
> making it in the code-base.
> 
> If folks prefer the fair, good, excellent marker I can switch over to
> that.
> 
> = Open =
> 
> * Andrew Cooper  Prognosis:   100 %
>     libx{c,l} error handling cleanup 
>     New migration.
>     cpuid leveling
> 
> * Andrew Benniest  Prognosis:   100 %
>     Netback multiqueue
> 
> * Arianna Avanzini  Prognosis:   100 %
>     block multiqueue
>     XEN_DOMCTL_memory_mapping hypercall for ARM
> 
> * Elena Ufimtseva  Prognosis:   0 %
>     vNUMA in Xen
>     vNUMA in Linux
> 
> * Boris Ostrovsky  Prognosis:   100 %
>     VPMU - 'perf' support in Xen
>     VPMU - 'perf' support in Linux
>     vAPIC in PVHVM guests
> 
> * Bob Liu  Prognosis:   100 %
>     tmem cleanups/fixes
>     1TB slow destruction
> 
> * Dario Faggioli  Prognosis:   0 %
>     Soft affinity for vcpus (was NUMA affinity for vcpus)
> 
> * Matt Wilson  Prognosis:   0 %
>     HVM guest NUMA
> 
> * Don Slutz  Prognosis:   100 %
>     Bigger PCI hole in QEMU 
>     Re-write of HPET
> 
> * David Vrabel:  Prognosis:   100 %
>     New migration.
> 
> * Daniel Kiper  Prognosis:   100 %
>     GRUB2 multiboot2
>     Xen multiboot2 support
>     Linux pvops of Xen EFI hypercall support
>     libxl/xl - xm compatibility mode for mem-max and mem-set;
>     Rearrange and cleanup installation destination directories (/var -> var/lib/xen)
> 
> * George Dunlap:  Prognosis:   100 %
>     Default to credit2
>     >  cpu pinning, numa affinity and cpu reservation
> 
> * Roger Pau Monn??  Prognosis:   100 %
>     Xen PVH dom0
>     PVH FreeBSD dom0
> 
> * Konrad Rzeszutek Wilk  Prognosis:   30 %
>     NUMA memory scrubbing
>     Performance fixes for PCI passthrough
> 
> * Kelly Zytaruk  Prognosis:   100 %
>     AMD Radeon PCI GPU passthrough
> 
> * Chen, Tiejun  Prognosis:   100 %
>     Intel IGD PCI GPU passthrough
> 
> * Mukesh Rathor  Prognosis:   100 %
>     Xen PVH dom0
>     Linux PVH dom0
> 
> * Wei Liu  Prognosis:   100 %
>     Adding missing 'xend' features in libxl
>     xl list -l on a dom0-only system
>     xl list -l doesn't contain tty console port
>     xl: passing more defaults in configuration in xl.conf
>     >  There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.
> 
> * Ian Campbel  Prognosis:   100 %
>     OSSTest: libvirt
>     OSSTest: upstream QEMU
> 
> * Ian Jackson  Prognosis:   100 %
>     xl does not handle migrate interruption gracefully
>     >  If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains
> 
> * Stefano Stabellini  Prognosis:   100 %
>     <NONE>

Interrupt latency reduction (no maintenance interrupts): 100%


> * Julien Grall  Prognosis:   100 %
>     ARM IOMMU support
> 
> * Malcolm Crossley  Prognosis:   100 %
>     IOMMU ABI for guests to map their DMA regions
> 
> * Zoltan Kiss  Prognosis:   100 %
>     Netback grant table manipulations
>     "Short" grant copy (just header) of packets. 
> 
> * Feng Wu  Prognosis:   100 %
>     SMAP
>     alternative_asm in Xen
> 
> * Zhang, Yang Z  Prognosis:   100 %
>     dirty vram / IOMMU bug
> 
> * Paul Durrant  Prognosis:   100 %
>     ioreq-server, aka secondary emulators
> 
> * Jan Beulich  Prognosis:   100 %
>     Stability
> 
> * Olaf Hering  Prognosis:   100 %
>     libvirt and xl discard support, so that libvirt can start using it
>     pvscsi should be targeted for 4.5, a prototype exists
>     live migration knobs, there is no suitable code yet, just ideas
> 
> * Joe Doe aka not assigned to anybody  Prognosis:   0 %
>     PoD fixes
>     TLB flushing without locks in Xen
>     xl does not support specifying virtual function for passthrough device
>     >  http://bugs.xenproject.org/xen/bug/22
>     PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
>     >  http://bugs.xenproject.org/xen/bug/28
> 

Re: Xen 4.5 development update

[Xu, Dongxiao]

> -----Original Message-----
> From: xen-devel-bounces@lists.xen.org
> [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of konrad.wilk@oracle.com
> Sent: Wednesday, May 28, 2014 2:07 AM
> To: andrew.cooper3@citrix.com; andrew.bennieston@citrix.com;
> avanzini.arianna@gmail.com; ufimtseva@gmail.com;
> boris.ostrovsky@oracle.com; bob.liu@oracle.com; msw@amazon.com;
> dslutz@verizon.com; david.vrabel@citrix.com; daniel.kiper@oracle.com;
> roger.pau@citrix.com; Kelly.Zytaruk@amd.com; Chen, Tiejun;
> mukesh.rathor@oracle.com; Wei.Liu2@citrix.com;
> stefano.stabellini@eu.citrix.com; julien.grall@linaro.org;
> malcolm.crossley@citrix.com; zoltan.kiss@citrix.com; Wu, Feng;
> konrad.wilk@oracle.com; Paul.Durrant@citrix.com; JBeulich@suse.com;
> olaf@aepfle.de; xen-devel@lists.xenproject.org
> Subject: [Xen-devel] Xen 4.5 development update
> 
> Below is a summary of the projects / features being worked on for the 4.5
> time frame that I had been gathering.
> 
> It is not complete! I would like folks input if I had missed something
> or screwed up. Will also talk at Xen Hackahon about this and take a look
> at  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the
> items there should move over.
> 
> The tentative feature freeze is scheduled for September 10th,
> which is months away.  With that in mind, I think it's time to take
> stock of the development, so we know whether to ask for more help or divert
> resources.
> 
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, I think it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
> Not sure what that means for libvirt. :-)
> 
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, I think it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
> Not sure what that means for libvirt. :-)
> 
> = Timeline =
> 
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 10th September 2014
> * First RC: 10th October
> * Release: 10th December 2014
> 
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
> 
> = Prognosis =
> 
> The prognosis is a numerical value of the likehood of the feature/code
> making it in the code-base.
> 
> If folks prefer the fair, good, excellent marker I can switch over to
> that.

Hi Konrad,

Could you please add the following?

* Dongxiao Xu  Prognosis: 50%
	Cache QoS Monitoring
* Shantong Kang	Prognosis: 50%
    Memory Bandwidth Monitoring

Thanks,
Dongxiao

> 
> = Open =
> 
> * Andrew Cooper  Prognosis:   100 %
>     libx{c,l} error handling cleanup
>     New migration.
>     cpuid leveling
> 
> * Andrew Benniest  Prognosis:   100 %
>     Netback multiqueue
> 
> * Arianna Avanzini  Prognosis:   100 %
>     block multiqueue
>     XEN_DOMCTL_memory_mapping hypercall for ARM
> 
> * Elena Ufimtseva  Prognosis:   0 %
>     vNUMA in Xen
>     vNUMA in Linux
> 
> * Boris Ostrovsky  Prognosis:   100 %
>     VPMU - 'perf' support in Xen
>     VPMU - 'perf' support in Linux
>     vAPIC in PVHVM guests
> 
> * Bob Liu  Prognosis:   100 %
>     tmem cleanups/fixes
>     1TB slow destruction
> 
> * Dario Faggioli  Prognosis:   0 %
>     Soft affinity for vcpus (was NUMA affinity for vcpus)
> 
> * Matt Wilson  Prognosis:   0 %
>     HVM guest NUMA
> 
> * Don Slutz  Prognosis:   100 %
>     Bigger PCI hole in QEMU
>     Re-write of HPET
> 
> * David Vrabel:  Prognosis:   100 %
>     New migration.
> 
> * Daniel Kiper  Prognosis:   100 %
>     GRUB2 multiboot2
>     Xen multiboot2 support
>     Linux pvops of Xen EFI hypercall support
>     libxl/xl - xm compatibility mode for mem-max and mem-set;
>     Rearrange and cleanup installation destination directories (/var ->
> var/lib/xen)
> 
> * George Dunlap:  Prognosis:   100 %
>     Default to credit2
>     >  cpu pinning, numa affinity and cpu reservation
> 
> * Roger Pau Monné  Prognosis:   100 %
>     Xen PVH dom0
>     PVH FreeBSD dom0
> 
> * Konrad Rzeszutek Wilk  Prognosis:   30 %
>     NUMA memory scrubbing
>     Performance fixes for PCI passthrough
> 
> * Kelly Zytaruk  Prognosis:   100 %
>     AMD Radeon PCI GPU passthrough
> 
> * Chen, Tiejun  Prognosis:   100 %
>     Intel IGD PCI GPU passthrough
> 
> * Mukesh Rathor  Prognosis:   100 %
>     Xen PVH dom0
>     Linux PVH dom0
> 
> * Wei Liu  Prognosis:   100 %
>     Adding missing 'xend' features in libxl
>     xl list -l on a dom0-only system
>     xl list -l doesn't contain tty console port
>     xl: passing more defaults in configuration in xl.conf
>     >  There are a number of options for which it might be useful to pass a
> default in xl.conf.  For example, if we could have a default "backend" parameter
> for vifs, then it would be easy to switch back and forth between a backend in a
> driver domain and a backend in dom0.
> 
> * Ian Campbel  Prognosis:   100 %
>     OSSTest: libvirt
>     OSSTest: upstream QEMU
> 
> * Ian Jackson  Prognosis:   100 %
>     xl does not handle migrate interruption gracefully
>     >  If you start a localhost migrate, and press "Ctrl-C" in the middle, you get
> two hung domains
> 
> * Stefano Stabellini  Prognosis:   100 %
>     <NONE>
> 
> * Julien Grall  Prognosis:   100 %
>     ARM IOMMU support
> 
> * Malcolm Crossley  Prognosis:   100 %
>     IOMMU ABI for guests to map their DMA regions
> 
> * Zoltan Kiss  Prognosis:   100 %
>     Netback grant table manipulations
>     "Short" grant copy (just header) of packets.
> 
> * Feng Wu  Prognosis:   100 %
>     SMAP
>     alternative_asm in Xen
> 
> * Zhang, Yang Z  Prognosis:   100 %
>     dirty vram / IOMMU bug
> 
> * Paul Durrant  Prognosis:   100 %
>     ioreq-server, aka secondary emulators
> 
> * Jan Beulich  Prognosis:   100 %
>     Stability
> 
> * Olaf Hering  Prognosis:   100 %
>     libvirt and xl discard support, so that libvirt can start using it
>     pvscsi should be targeted for 4.5, a prototype exists
>     live migration knobs, there is no suitable code yet, just ideas
> 
> * Joe Doe aka not assigned to anybody  Prognosis:   0 %
>     PoD fixes
>     TLB flushing without locks in Xen
>     xl does not support specifying virtual function for passthrough device
>     >  http://bugs.xenproject.org/xen/bug/22
>     PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with
> PCI/GPU passthrough
>     >  http://bugs.xenproject.org/xen/bug/28
> 

Re: Xen 4.5 development update

[Don Slutz]

On 5/27/2014 2:06 PM, konrad.wilk@oracle.com wrote:
> Below is a summary of the projects / features being worked on for the 4.5
> time frame that I had been gathering.
>


>
>
>
>
>
> * Don Slutz  Prognosis:   100 %
>      Bigger PCI hole in QEMU
>      Re-write of HPET

An additional new feature:

VMware backdoor (hypercall)



    -Don Slutz
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.5 development update

[Zytaruk, Kelly]

> 
> On 5/27/2014 2:06 PM, konrad.wilk@oracle.com wrote:
> > Below is a summary of the projects / features being worked on for the
> > 4.5 time frame that I had been gathering.
> >
> > * Don Slutz  Prognosis:   100 %
> >      Bigger PCI hole in QEMU
> >      Re-write of HPET
> 
> An additional new feature:
> 
> VMware backdoor (hypercall)
> 

Hi Don,  I'm not familiar with this feature but it intrigues me.  Can you provide a very short description of what it is?
Is this for the VMWare backdoor port at I/O address 0x5658?  Is the scenario, VMWare running in a Xen guest VM?

Thanks,
Kelly

> 
> 
>     -Don Slutz
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel

Re: Xen 4.5 development update

[Don Slutz]

On 5/30/2014 9:57 AM, Zytaruk, Kelly wrote:
>> On 5/27/2014 2:06 PM, konrad.wilk@oracle.com wrote:
>>> Below is a summary of the projects / features being worked on for the
>>> 4.5 time frame that I had been gathering.
>>>
>>> * Don Slutz  Prognosis:   100 %
>>>       Bigger PCI hole in QEMU
>>>       Re-write of HPET
>> An additional new feature:
>>
>> VMware backdoor (hypercall)
>>
> Hi Don,  I'm not familiar with this feature but it intrigues me.  Can you provide a very short description of what it is?

Support a sub-set of possible commands via port 0x5658 including VMware 
RPC channels that allow VMware tools to run in the Guest.
This also allows requesting a clean shutdown (and reboot) of the Guest 
if VMware tools is running.

> Is this for the VMWare backdoor port at I/O address 0x5658?

Yes.

>   Is the scenario, VMWare running in a Xen guest VM?

If you mean VMware tools, then yes.

    -Don Slutz

> Thanks,
> Kelly
>
>>
>>      -Don Slutz
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xen.org
>>> http://lists.xen.org/xen-devel

Re: Xen 4.5 development update

[Lars Kurth]

[-- Attachment #1.1: Type: text/plain, Size: 6214 bytes --]

Adding Artem from Globallogic to that thread on request
Lars

On 27/05/2014 19:06, konrad.wilk@oracle.com wrote:
> Below is a summary of the projects / features being worked on for the 4.5
> time frame that I had been gathering.
>
> It is not complete! I would like folks input if I had missed something
> or screwed up. Will also talk at Xen Hackahon about this and take a look
> at  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 to see which of the
> items there should move over.
>
> The tentative feature freeze is scheduled for September 10th,
> which is months away.  With that in mind, I think it's time to take
> stock of the development, so we know whether to ask for more help or divert
> resources.
>
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, I think it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
> Not sure what that means for libvirt. :-)
>
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, I think it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
> Not sure what that means for libvirt. :-)
>
> = Timeline =
>
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 10th September 2014
> * First RC: 10th October
> * Release: 10th December 2014
>
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
>
> = Prognosis =
>
> The prognosis is a numerical value of the likehood of the feature/code
> making it in the code-base.
>
> If folks prefer the fair, good, excellent marker I can switch over to
> that.
>
> = Open =
>
> * Andrew Cooper  Prognosis:   100 %
>      libx{c,l} error handling cleanup
>      New migration.
>      cpuid leveling
>
> * Andrew Benniest  Prognosis:   100 %
>      Netback multiqueue
>
> * Arianna Avanzini  Prognosis:   100 %
>      block multiqueue
>      XEN_DOMCTL_memory_mapping hypercall for ARM
>
> * Elena Ufimtseva  Prognosis:   0 %
>      vNUMA in Xen
>      vNUMA in Linux
>
> * Boris Ostrovsky  Prognosis:   100 %
>      VPMU - 'perf' support in Xen
>      VPMU - 'perf' support in Linux
>      vAPIC in PVHVM guests
>
> * Bob Liu  Prognosis:   100 %
>      tmem cleanups/fixes
>      1TB slow destruction
>
> * Dario Faggioli  Prognosis:   0 %
>      Soft affinity for vcpus (was NUMA affinity for vcpus)
>
> * Matt Wilson  Prognosis:   0 %
>      HVM guest NUMA
>
> * Don Slutz  Prognosis:   100 %
>      Bigger PCI hole in QEMU
>      Re-write of HPET
>
> * David Vrabel:  Prognosis:   100 %
>      New migration.
>
> * Daniel Kiper  Prognosis:   100 %
>      GRUB2 multiboot2
>      Xen multiboot2 support
>      Linux pvops of Xen EFI hypercall support
>      libxl/xl - xm compatibility mode for mem-max and mem-set;
>      Rearrange and cleanup installation destination directories (/var -> var/lib/xen)
>
> * George Dunlap:  Prognosis:   100 %
>      Default to credit2
>      >  cpu pinning, numa affinity and cpu reservation
>
> * Roger Pau Monné  Prognosis:   100 %
>      Xen PVH dom0
>      PVH FreeBSD dom0
>
> * Konrad Rzeszutek Wilk  Prognosis:   30 %
>      NUMA memory scrubbing
>      Performance fixes for PCI passthrough
>
> * Kelly Zytaruk  Prognosis:   100 %
>      AMD Radeon PCI GPU passthrough
>
> * Chen, Tiejun  Prognosis:   100 %
>      Intel IGD PCI GPU passthrough
>
> * Mukesh Rathor  Prognosis:   100 %
>      Xen PVH dom0
>      Linux PVH dom0
>
> * Wei Liu  Prognosis:   100 %
>      Adding missing 'xend' features in libxl
>      xl list -l on a dom0-only system
>      xl list -l doesn't contain tty console port
>      xl: passing more defaults in configuration in xl.conf
>      >  There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.
>
> * Ian Campbel  Prognosis:   100 %
>      OSSTest: libvirt
>      OSSTest: upstream QEMU
>
> * Ian Jackson  Prognosis:   100 %
>      xl does not handle migrate interruption gracefully
>      >  If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains
>
> * Stefano Stabellini  Prognosis:   100 %
>      <NONE>
>
> * Julien Grall  Prognosis:   100 %
>      ARM IOMMU support
>
> * Malcolm Crossley  Prognosis:   100 %
>      IOMMU ABI for guests to map their DMA regions
>
> * Zoltan Kiss  Prognosis:   100 %
>      Netback grant table manipulations
>      "Short" grant copy (just header) of packets.
>
> * Feng Wu  Prognosis:   100 %
>      SMAP
>      alternative_asm in Xen
>
> * Zhang, Yang Z  Prognosis:   100 %
>      dirty vram / IOMMU bug
>
> * Paul Durrant  Prognosis:   100 %
>      ioreq-server, aka secondary emulators
>
> * Jan Beulich  Prognosis:   100 %
>      Stability
>
> * Olaf Hering  Prognosis:   100 %
>      libvirt and xl discard support, so that libvirt can start using it
>      pvscsi should be targeted for 4.5, a prototype exists
>      live migration knobs, there is no suitable code yet, just ideas
>
> * Joe Doe aka not assigned to anybody  Prognosis:   0 %
>      PoD fixes
>      TLB flushing without locks in Xen
>      xl does not support specifying virtual function for passthrough device
>      >  http://bugs.xenproject.org/xen/bug/22
>      PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
>      >  http://bugs.xenproject.org/xen/bug/28
>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


[-- Attachment #1.2: Type: text/html, Size: 6869 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update

[Zytaruk, Kelly]

>
>Adding Artem from Globallogic to that thread on request
>Lars
>
>On 27/05/2014 19:06, konrad.wilk@oracle.com wrote:
>Below is a summary of the projects / features being worked on for the 4.5
>time frame that I had been gathering.
>
>* Kelly Zytaruk  Prognosis:   100 %
>   AMD Radeon PCI GPU passthrough
>

Due to a shift in focus on my side it looks like I am stuck in a Xen 4.2 spiral.  I can't guarantee any Xen 4.5 support until I get my Xen 4.2 issues out of the way.  I know that Konrad has been working on some of the issues that were identified so I am not sure how much more remains to be done for 4.5.

Thanks,
Kelly

Re: Xen 4.5 development update

[Sander Eikelenboom]

Wednesday, June 4, 2014, 3:50:12 PM, you wrote:


>>
>>Adding Artem from Globallogic to that thread on request
>>Lars
>>
>>On 27/05/2014 19:06, konrad.wilk@oracle.com wrote:
>>Below is a summary of the projects / features being worked on for the 4.5
>>time frame that I had been gathering.
>>
>>* Kelly Zytaruk  Prognosis:   100 %
>>   AMD Radeon PCI GPU passthrough
>>

> Due to a shift in focus on my side it looks like I am stuck in a Xen 4.2 spiral.  I can't guarantee any Xen 4.5 support until I get my Xen 4.2 issues out of the way.  I know that Konrad has been working on some of the issues that were identified so I am not sure how much more remains to be done for 4.5.

> Thanks,
> Kelly

I assume the target here is to passthrough and use the GPU as a primary GPU in 
the guest (and also showing boot etc from the start like as it was native) ?

Passing through AMD radeon GPU as secondary, which gets enabled somewhere during 
boot when the radeon driver loads, already works (for me) when using a HVM guest.
All seems to work for me:
   - console + X
   - opengl
   - hardware video acceleration
   - powermanagement (but only on first boot, see below)

The main issue with that setup is resetting the card when shutting down the guest
and later booting it again. That also seems to work when using "radeon.hard_reset=1"
as kernel parameter.

Accept for the powermanagement part (dpm), that still fails to initialize on a second 
or later boot (but works fine on the first boot).

I don't known if the radeon cards have some sort of full reset function 
somewhere that completely "posts" the card ?

That could be used by xen and kvm or more generally .. wired up to a 
/sysfs/bus/pci/device/<BDF>/reset entry (which can be used by both).

--
Sander

Xen 4.5 development update

[konrad.wilk]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 8809 bytes --]

Below is a summary of the projects / features being worked on for the 4.5
time frame.  The tentative feature freeze is scheduled for September 10th,
which is months away.  With that in mind, I think it's time to take
stock of the development, so we know whether to ask for more help or divert
resources.

The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
Instead of values (0->100) it is now five states:

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs


For items involving code hosted on the Xen.org site (including qemu-xen),
that means a likelihood of having the feature code-complete and mostly
working by the feature freeze.  (It's OK if there are still bugs to be
worked out.)  For items in Linux, it would mean having items on track
to make it into the kernel released just after the scheduled 4.5 time frame.

Not exactly sure what that is in terms of libvirt.

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:

* Feature Freeze: 10th September 2014
* First RC: 10th October
* Release: 10th December 2014

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

= Prognosis =

The states are: none -> fair -> ok -> good -> done

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs


= Open =

== ARM ==

*  ARM  - IOMMU support (good)
  -  Julien Grall

*  ARM - Device assigment on ARM (good)
   Xen parts done. Linux at risk
  -  Julien Grall

*  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (good)
  -  Arianna Avanzini

*  ARM Interrupt latency reduction (no maintenance interrupts) (good)
  -  Stefano Stabellini

*  ARM Xen UEFI booting on ARM (none)
   Based on Daniel's multiboot2-EFI work
  -  Roy Franz

*  ARM PSCI v0.2 (good)
  -  Parth Dixit

*  ARM GICv3 support (good)
  -  Vijay Kilari

*  ARM GICv2m support (none)
  -  Linaro (unknown)

*  ARM VM save/restore/live migration (fair)
  -  Wei Huang

== x86 ==

*  New migration. (good)
  -  Andrew Cooper & David Vrabel

*  PVH - AMD hardware support. (fair)
   Issues with FSBASE MSRs
  -  Mukesh Rathor

*  Xen multiboot2-EFI support (fair)
   Needed for SecureBoot
  -  Daniel Kiper

*  Re-write of vHPET (fair)
   Xen 4.5 blocker
  -  Don Slutz

*  HT enabled, virtualization overhead is high (Xen 4.4) (none)
   kernbench demonstrated it
  -  Dario Faggioli

*  IOMMU ABI for guests to map their DMA regions (fair)
  -  Malcolm Crossley

*  dirty vram / IOMMU bug (fair)
  -  Zhang, Yang Z

*  Convert tasklet to per-cpu tasklets (fair)
  -  Konrad Rzeszutek Wilk

*  VMware backdoor (hypercall) (ok)
   Needs to be split up and redone
  -  Don Slutz

*  VPMU - 'perf' support in Xen (good)
  -  Boris Ostrovsky

*  ioreq-server, aka secondary emulators (good)
   Chase down XSM maintainer
  -  Paul Durrant

*  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
  -  Dario Faggioli

*  Cache QoS Monitoring - hypercalls (fair)
   Just hypercalls - no toolstack changes.
  -  Dongxiao Xu and Shantong Kang

*  HVM guest NUMA (none)
  -  Matt Wilson

*  1TB slow destruction (ok)
  -  Bob Liu

*  extending mem_access support to PV domain (ok)
  -  Aravindh Puthiyaparambil (aravindp)

*  Stability fix (good)
  -  Jan Beulich

== QEMU ==

*  Rebase of QEMU 2.0 and SeaBIOS (fair)
  -  Ian Jackson

*  Using qemu-upstream in a stubdomain (fair)
  -  Ian Jackson

*  Intel IGD PCI GPU passthrough (ok)
   Maintainers of QEMU don't like it. Want a different way.
  -  Chen, Tiejun

*  AMD Radeon PCI GPU passthrough (none)
   Focusing on Xen 4.2 and qemu-traditional
  -  Kelly Zytaruk

*  Bigger PCI hole in QEMU (ok)
   Needs to be rebased
  -  Don Slutz

== lib{xc,xl} and toolstack ==

*  libxl/xl - xm compatibility mode for mem-max and mem-set; (ok)
  -  Daniel Kiper

*  Rearrange and cleanup installation destination directories (/var -> var/lib/xen) (fair)
  -  Daniel Kiper

*  libxl work - JSON to keep track of guest configs (ok)
  -  Wei Liu

*  pvscsi should be targeted for 4.5, a prototype exists (fair)
  -  Olaf Hering

*  xl list --long (and some related xl commands) have some bugs (none)
  -  Zhigang Wang

*  PVH - Migration of PVH DomUs. (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  PVH - Migration of guests from a PVH dom0  (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  xl does not handle migrate interruption gracefully (none)
   If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains
  -  Ian Jackson

*  Remus in Xen (ok)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  COLO in Xen (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

== Linux ==

*  Linux block multiqueue (fair)
  -  Arianna Avanzini

*  Netback grant table manipulations (ok)
  -  Zoltan Kiss

*  VPMU - 'perf' support in Linux (ok)
   Depends on Xen patches
  -  Boris Ostrovsky

*  vNUMA in Linux (ok)
  -  Elena Ufimtseva

*  vsyscall in Linux (fair)
  -  Konrad Rzeszutek Wilk

*  Linux pvops of Xen EFI hypercall support (good)
  -  Daniel Kiper

*  Netback multiqueue (good)
  -  Wei Liu

*  COLO Agent in Linux (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

== FreeBSD ==

*  PVH FreeBSD dom0 (ok)
   FreeBSD 11 goal. Toolstack side done in Xen 4.5
  -  Roger Pau Monné

== Other OSes (MiniOS, QNX) ==

*  PV drivers for automotive kernels (fair)
  -  Artem Mygaiev

== GRUB2 ==

*  GRUB2 multiboot2 (fair)
  -  Daniel Kiper

== OSSTEST ==

*  OSSTest: libvirt (good)
  -  Ian Campbell

== Deferred to Xen 4.6 ==

*  Default to credit2 (none)
   cpu pinning, numa affinity and cpu reservation
  -  George Dunlap

*  "Short" grant copy (just header) of packets. (none)
  -  Zoltan Kiss

*  vNUMA in Xen (ok)
  -  Elena Ufimtseva

*  cpuid leveling (none)
   http://xenbits.xen.org/people/andrewcoop/feature-levelling/feature-levelling-D.pdf
  -  Andrew Cooper

*  live migration knobs, there is no suitable code yet, just ideas (none)
    http://lists.xenproject.org/archives/html/xen-devel/2014-03/msg00785.html
  -  Olaf Hering

*  Further tmem cleanups/fixes (16TB etc) (fair)
  -  Bob Liu

== Up for grabs ==

*  PoD fixes
   if you boot with memory <= maxmem we have a size estimation bug

*  TLB flushing without locks in Xen

*  xl does not support specifying virtual function for passthrough device
   http://bugs.xenproject.org/xen/bug/22

*  PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
   http://bugs.xenproject.org/xen/bug/28

*  libx{c,l} error handling cleanup

*  Adding missing 'xend' features in libxl

*  xl list -l on a dom0-only system

*  xl list -l doesn't contain tty console port

*  xl: passing more defaults in configuration in xl.conf
   There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.

*  PVH - PVH working with shadow.
   Based on Tim's work

*  PVH - PCI passthrough for DomU.

*  AMD performance regressions

*  Performance due to hypercall preemption. More preemptions - slower. (none)

== Completed ==

*  alternative_asm in Xen (done)
  -  Feng Wu

*  SMAP (done)
  -  Feng Wu

*  Re-write of HPET (done)
  -  Don Slutz

*  vAPIC in PVHVM guests (done)
  -  Boris Ostrovsky

*  libvirt and xl discard support, so that libvirt can start using it (done)
  -  Olaf Hering

*  Xen PVH dom0 (done)
  -  Mukesh Rathor

*  Linux PVH dom0 (done)
  -  Mukesh Rathor

*  OSSTest: upstream QEMU (done)
  -  Ian Campbell

*  amd_ucode cleanups, verify patch size(enhancement) (mostly in master except one patch)

*  Data breakpoint Extension support (new-feat) (in master)

*  Feature masking MSR support (enhancement) (in master)

*  Support BRCM TruManage chip (Serial over LAN support) (new-feat) (in master)

*  fix vmce_amd* functions, unify mce_amd mcheck initialization (fixes/cleanups)
  -  Aravind and Suravee

*  NUMA memory scrubbing (done)
  -  Konrad Rzeszutek Wilk



[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update

[Julien Grall]

Hi Konrad,

On 18/06/14 21:22, konrad.wilk@oracle.com wrote:
> = Open =
>
> == ARM ==
>
> *  ARM  - IOMMU support (good)
>    -  Julien Grall

This has been fully pushed upstream a week ago.

> *  ARM - Device assigment on ARM (good)
>     Xen parts done. Linux at risk

I just sent an early RFC for the hypervisor side.

Regards,

-- 
Julien Grall

Re: Xen 4.5 development update

[Andrew Cooper]

On 18/06/2014 21:22, konrad.wilk@oracle.com wrote:
> <snip>
> The states are: none -> fair -> ok -> good -> done
>
> none - nothing yet
> fair - still working on it, patches are prototypes or RFC
> ok   - patches posted, acting on review
> good - some last minute pieces
> done - all done, might have bugs
>
>
> = Open =
> <snip>
> == x86 == 
>
> *  New migration. (good)
>   -  Andrew Cooper & David Vrabel

This is still very much "fair" given your definition above.  The libxl
work is still at the very start of planning without any real code yet,
and there are other areas still not started such as remus and tmem.  The
libxc stuff is still RFC with TODOs remaining.

> <snip>
>
> *  ioreq-server, aka secondary emulators (good)
>    Chase down XSM maintainer
>   -  Paul Durrant

This is fully committed as far as I am aware, so is done.

~Andrew

Re: Xen 4.5 development update

[Paul Durrant]

> -----Original Message-----
> From: Andrew Cooper [mailto:amc96@hermes.cam.ac.uk] On Behalf Of
> Andrew Cooper
> Sent: 18 June 2014 21:48
> To: konrad.wilk@oracle.com; julien.grall@linaro.org;
> avanzini.arianna@gmail.com; Stefano Stabellini; roy.franz@linaro.org;
> vijay.kilari@gmail.com; Vijaya.Kumar@caviumnetworks.com;
> serge.broslavsky@linaro.org; christoffer.dall@linaro.org;
> w1.huang@samsung.com; David Vrabel; mukesh.rathor@oracle.com;
> daniel.kiper@oracle.com; dslutz@verizon.com; Dario Faggioli; Malcolm
> Crossley; yang.z.zhang@intel.com; boris.ostrovsky@oracle.com; Paul
> Durrant; dongxiao.xu@intel.com; shantong.kang@intel.com;
> msw@amazon.com; bob.liu@oracle.com; aravindp@cisco.com;
> JBeulich@suse.com; Ian Jackson; tiejun.chen@intel.com;
> Kelly.Zytaruk@amd.com; Wei Liu; olaf@aepfle.de;
> zhigang.x.wang@oracle.com; Roger Pau Monne; guijianfeng@cn.fujitsu.com;
> Eddie Dong; Zoltan Kiss; ufimtseva@gmail.com;
> artem.mygaiev@globallogic.com; Ian Campbell;
> george.dunlap@eu.cit.rix.com; xen-devel@lists.xenproject.org;
> feng.wu@intel.com; Aravind.Gopalakrishnan@amd.com;
> Suravee.Suthikulpanit@amd.com
> Subject: Re: Xen 4.5 development update
> 
> On 18/06/2014 21:22, konrad.wilk@oracle.com wrote:
> > <snip>
> > The states are: none -> fair -> ok -> good -> done
> >
> > none - nothing yet
> > fair - still working on it, patches are prototypes or RFC
> > ok   - patches posted, acting on review
> > good - some last minute pieces
> > done - all done, might have bugs
> >
> >
> > = Open =
> > <snip>
> > == x86 ==
> >
> > *  New migration. (good)
> >   -  Andrew Cooper & David Vrabel
> 
> This is still very much "fair" given your definition above.  The libxl
> work is still at the very start of planning without any real code yet,
> and there are other areas still not started such as remus and tmem.  The
> libxc stuff is still RFC with TODOs remaining.
> 
> > <snip>
> >
> > *  ioreq-server, aka secondary emulators (good)
> >    Chase down XSM maintainer
> >   -  Paul Durrant
> 
> This is fully committed as far as I am aware, so is done.

Yes, it is. I do have a follow-up patch for a PCI hotplug controller that's not quite ready to go, but that's only loosely related.

  Paul

> 
> ~Andrew

Re: Xen 4.5 development update

[Aravindh Puthiyaparambil (aravindp)]

>The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
>Instead of values (0->100) it is now five states:
>
>none - nothing yet
>fair - still working on it, patches are prototypes or RFC
>ok   - patches posted, acting on review
>good - some last minute pieces
>done - all done, might have bugs
>
>*  extending mem_access support to PV domain (ok)
>  -  Aravindh Puthiyaparambil (aravindp)

This is in "fair" state given your definition above. 

Thanks,
Aravindh

Re: Xen 4.5 development update

[Roy Franz]

> *  ARM Xen UEFI booting on ARM (none)
>    Based on Daniel's multiboot2-EFI work
>   -  Roy Franz

This is 'fair'.  I have working prototype code and am preparing an RFC patchset.

Roy

Re: Xen 4.5 development update

[Vijay Kilari]

>
> *  ARM GICv3 support (good)
>   -  Vijay Kilari

   It is in 'ok' state as per your definition.

Regards
Vijay

Re: Xen 4.5 development update

[Wei Liu]

On Wed, Jun 18, 2014 at 04:22:55PM -0400, konrad.wilk@oracle.com wrote:
[...]
> *  Netback multiqueue (good)
>   -  Wei Liu

Both frontend and backend patches have been merged upstream. So this one
can be considered done.

Wei.

Re: Xen 4.5 development update

[Stefano Stabellini]

On Wed, 18 Jun 2014, konrad.wilk@oracle.com wrote:
> *  ARM Interrupt latency reduction (no maintenance interrupts) (good)
>   -  Stefano Stabellini

It is merged.

Re: Xen 4.5 development update

[Don Slutz]

On 06/18/14 16:22, konrad.wilk@oracle.com wrote:
> Below is a summary of the projects / features being worked on for the 4.5
> time frame.  The tentative feature freeze is scheduled for September 10th,
> which is months away.  With that in mind, I think it's time to take
> stock of the development, so we know whether to ask for more help or divert
> resources.
>

> *  Re-write of vHPET (fair)
>     Xen 4.5 blocker
>    -  Don Slutz
I have no idea what this is if it is not the item below.



>
> *  VMware backdoor (hypercall) (ok)
>     Needs to be split up and redone
>    -  Don Slutz

No change.


>
> *  Bigger PCI hole in QEMU (ok)
>     Needs to be rebased
>    -  Don Slutz

Getting close to done for QEMU 2.1




> *  Re-write of HPET (done)
>    -  Don Slutz

This is referring to vHPET changes as far as I know:

  22d73d1 |Don Slutz| hvm/hpet: Detect comparator values in the past

etc.


     -Don Slutz

Re: Xen 4.5 development update

[Jan Beulich]

>>> Don Slutz <dslutz@verizon.com> 06/19/14 3:24 PM >>>
>On 06/18/14 16:22, konrad.wilk@oracle.com wrote:
>> *  Re-write of vHPET (fair)
>>     Xen 4.5 blocker
>>    -  Don Slutz
>I have no idea what this is if it is not the item below.

I think this and ...

>> *  Re-write of HPET (done)
>>    -  Don Slutz
>
>This is referring to vHPET changes as far as I know:
>
>22d73d1 |Don Slutz| hvm/hpet: Detect comparator values in the past

... this got mixed up, much like also during the hackathon session. vHPET is done,
HPET is in work (backburner afair) by Andrew.

Jan

Re: Xen 4.5 development update

[Andrew Cooper]

On 19/06/14 17:37, Jan Beulich wrote:
>>>> Don Slutz <dslutz@verizon.com> 06/19/14 3:24 PM >>>
>> On 06/18/14 16:22, konrad.wilk@oracle.com wrote:
>>> *  Re-write of vHPET (fair)
>>>     Xen 4.5 blocker
>>>    -  Don Slutz
>> I have no idea what this is if it is not the item below.
> I think this and ...
>
>>> *  Re-write of HPET (done)
>>>    -  Don Slutz
>> This is referring to vHPET changes as far as I know:
>>
>> 22d73d1 |Don Slutz| hvm/hpet: Detect comparator values in the past
> ... this got mixed up, much like also during the hackathon session. vHPET is done,
> HPET is in work (backburner afair) by Andrew.
>
> Jan
>

Ah yes, how about:

Xen HPET interrupt fixes (fair, backburner behind migration v2)
 - Andrew Cooper

~Andrew

Re: Xen 4.5 development update

[Boris Ostrovsky]

On 06/18/2014 04:22 PM, konrad.wilk@oracle.com wrote:
> *  vAPIC in PVHVM guests (done)
>    -  Boris Ostrovsky

Xen part is done. I haven't started on Linux side yet.

-boris

Re: Xen 4.5 development update

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1051 bytes --]

On mer, 2014-06-18 at 16:22 -0400, konrad.wilk@oracle.com wrote:

> The states are: none -> fair -> ok -> good -> done
> 
> none - nothing yet
> fair - still working on it, patches are prototypes or RFC
> ok   - patches posted, acting on review
> good - some last minute pieces
> done - all done, might have bugs
> 
> 
> = Open =

> == x86 == 

> *  HT enabled, virtualization overhead is high (Xen 4.4) (none)
>    kernbench demonstrated it
>   -  Dario Faggioli
> 
Well, I'm looking into it, tracing, etc... 'none' is certainly accurate
in term of patches posted, though.

> *  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
>   -  Dario Faggioli
> 
Correct.

> *  HVM guest NUMA (none)
>   -  Matt Wilson
> 
Sadly true.

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update

[Lars Kurth]

On 18/06/2014 21:22, konrad.wilk@oracle.com wrote:
> [snip]
>
>
> = Open =
>
> == ARM ==
>
> *  ARM  - IOMMU support (good)
>    -  Julien Grall
>
> *  ARM - Device assigment on ARM (good)
>     Xen parts done. Linux at risk
>    -  Julien Grall
> [snip]
I put together http://wiki.xen.org/wiki/Xen_Roadmap/4.5 to explain the 
process and added links to the update mails as well as copied items in 
done and good state

Lars

Re: Xen 4.5 development update

[Ian Campbell]

Culling CC's somewhat...

On Wed, 2014-06-18 at 16:22 -0400, konrad.wilk@oracle.com wrote:
> = Timeline =
> 
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> 
> * Feature Freeze: 10th September 2014
> * First RC: 10th October
> * Release: 10th December 2014

FYI I'm on vacation Friday 1 August and from 6-18 August and at
conferences 18-31 August, 15-19 September and 13-17 October.

So it mostly lines up OK :-P

Ian.

Re: Xen 4.5 development update

[Konrad Rzeszutek Wilk]

On Thu, Jun 26, 2014 at 03:10:55PM +0100, Ian Campbell wrote:
> Culling CC's somewhat...
> 
> On Wed, 2014-06-18 at 16:22 -0400, konrad.wilk@oracle.com wrote:
> > = Timeline =
> > 
> > We are planning on a 9-month release cycle.  Based on that, below are
> > our estimated dates:
> > 
> > * Feature Freeze: 10th September 2014
> > * First RC: 10th October
> > * Release: 10th December 2014
> 
> FYI I'm on vacation Friday 1 August and from 6-18 August and at
> conferences 18-31 August, 15-19 September and 13-17 October.
> 
> So it mostly lines up OK :-P

:-)

Thanks for the update!
> 
> Ian.
> 

Xen 4.5 development update

[konrad.wilk]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 10278 bytes --]

Below is a summary of the projects / features being worked on for the 4.5
time frame.  The tentative feature freeze is scheduled for September 10th,
which is months away.  With that in mind, I think it's time to take
stock of the development, so we know whether to ask for more help or divert
resources.

The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
Instead of values (0->100) it is now five states:

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs


For items involving code hosted on the Xen.org site (including qemu-xen),
that means a likelihood of having the feature code-complete and mostly
working by the feature freeze.  (It's OK if there are still bugs to be
worked out.)  For items in Linux, it would mean having items on track
to make it into the kernel released just after the scheduled 4.5 time frame.

Not exactly sure what that is in terms of libvirt.

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:

* Feature Freeze: 10th September 2014
* First RC: 10th October
* Release: 10th December 2014

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

= Prognosis =

The states are: none -> fair -> ok -> good -> done

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs


= Open =

== ARM ==

*  ARM - Device assigment on ARM (good)
   RFC for the hypervisor side sent.
   Linux parts at risk.
  -  Julien Grall

*  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (good)
  -  Arianna Avanzini

*  ARM Xen UEFI booting on ARM (ok)
   RFC posted
  -  Roy Franz

*  ARM PSCI v0.2 (good)
   v2 posted
  -  Parth Dixit

*  ARM GICv3 support (ok)
   Split in two patchsets: v6a and v6
   v7a posted which does GIC and VGIC code refactoring
  -  Vijay Kilari

*  ARM GICv2m support (none)
  -  Linaro (unknown)

*  ARM VM save/restore/live migration (ok)
  -  Junghyun Yoo

*  ARM remote processor iommu module (GPUs + IPUs) (ok)
  -  Andrii Tseglytskyi

*  ARM: Use super pages in p2m  (ok)
   v4 posted
  -  Ian Campbell

*  ARM DRA7 support (ok)
  -  Andrii Tseglytskyi

*  ARM - MiniOS (ok)
   v5 posted
  -  Thomas Leonard

== x86 ==

*  New migration. (ok)
   tmem&remus need work
   libxc is RFC
  -  Andrew Cooper & David Vrabel

*  PVH - AMD hardware support. (fair)
   Issues with FSBASE MSRs
  -  Mukesh Rathor

*  Xen multiboot2-EFI support (fair)
   Needed for SecureBoot
  -  Daniel Kiper

*  Xen HPET interrupt fixes (fair)
   behind migration v2
  -  Andrew Cooper

*  HT enabled, virtualization overhead is high (Xen 4.4) (none)
   kernbench demonstrated it
   looking and tracing it
  -  Dario Faggioli

*  IOMMU ABI for guests to map their DMA regions (fair)
  -  Malcolm Crossley

*  dirty vram / IOMMU bug (fair)
  -  Zhang, Yang Z

*  Convert tasklet to per-cpu tasklets (fair)
  -  Konrad Rzeszutek Wilk

*  VMware backdoor (hypercall) (ok)
   Needs to be split up and redone
  -  Don Slutz

*  VPMU - 'perf' support in Xen (good)
   v5 posted
  -  Boris Ostrovsky

*  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
   v11 posted
  -  Dario Faggioli

*  Cache QoS Monitoring - hypercalls (fair)
   Just hypercalls - no toolstack changes.
   v11 posted
  -  Dongxiao Xu and Shantong Kang

*  HVM guest NUMA (none)
  -  Matt Wilson

*  1TB slow destruction (ok)
  -  Bob Liu

*  extending mem_access support to PV domain (fair)
  -  Aravindh Puthiyaparambil (aravindp)

*  Stability fix (good)
  -  Jan Beulich

*  Support controlling the max C-state sub-state (ok)
   v3 posted
  -  Ross Lagerwall

*  Repurpose SEDF Scheduler for Real-time (ok)
   RFC patch posted (v1)
  -  Joshua Whitehead, Robert VanVossen

== QEMU ==

*  Rebase of QEMU 2.0 and SeaBIOS (fair)
  -  Ian Jackson

*  Using qemu-upstream in a stubdomain (fair)
  -  Ian Jackson

*  Intel IGD PCI GPU passthrough (ok)
   v5 posted
  -  Chen, Tiejun

*  AMD Radeon PCI GPU passthrough (none)
   Focusing on Xen 4.2 and qemu-traditional
  -  Kelly Zytaruk

*  vNUMA in Xen (ok)
   git://gitorious.org/xenvnuma_v5/xenvnuma_v5.git
  -  Elena Ufimtseva

== lib{xc,xl} and toolstack ==

*  libxl/xl - xm compatibility mode for mem-max and mem-set; (ok)
  -  Daniel Kiper

*  Rearrange and cleanup installation destination directories (/var -> var/lib/xen) (fair)
  -  Daniel Kiper

*  libxl work - JSON to keep track of guest configs (ok)
  -  Wei Liu

*  pvscsi should be targeted for 4.5, a prototype exists (fair)
  -  Olaf Hering

*  xl list --long (and some related xl commands) have some bugs (none)
  -  Zhigang Wang

*  PVH - Migration of PVH DomUs. (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  PVH - Migration of guests from a PVH dom0  (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  xl does not handle migrate interruption gracefully (none)
   If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains
  -  Ian Jackson

*  Remus in Xen (ok)
   url: https://github.com/laijs/xen remus-v13
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  COLO in Xen (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  systemd support (ok)
   v6
  -  Luis R. Rodriguez

*  snapshot API extension (checkpointing disk) (ok)
  -  Bamvor Jian Zhang

*  extend the xenstore ring with a 'closing' signal
   RFC patch posted
  -  David Scott

== Linux ==

*  Linux block multiqueue (fair)
  -  Arianna Avanzini

*  Netback grant table manipulations (ok)
  -  Zoltan Kiss

*  VPMU - 'perf' support in Linux (ok)
   Depends on Xen patches
  -  Boris Ostrovsky

*  vNUMA in Linux (ok)
   git://gitorious.org/xenvnuma_v5/xenvnuma_v5.git
  -  Elena Ufimtseva

*  vsyscall in Linux (fair)
  -  Konrad Rzeszutek Wilk

*  Linux pvops of Xen EFI hypercall support (good)
  -  Daniel Kiper

*  COLO Agent in Linux (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  vAPIC in PVHVM guests (Linux side) (none)
  -  Boris Ostrovsky

*  pvSCSI (ok)
   Initial patch posted.
  -  Juergen Gross

== FreeBSD ==

*  PVH FreeBSD dom0 (ok)
   FreeBSD 11 goal. Toolstack side done in Xen 4.5
  -  Roger Pau Monné

== Other OSes (MiniOS, QNX) ==

*  PV drivers for automotive kernels (fair)
  -  Artem Mygaiev

*  mini-os: xenbus changes for rump kernels (ok)
   git://xenbits.xen.org/people/iwj/rumpuser-xen.git
   branch: base.dev-xen-xenbus.v1..dev-xen-xenbus.v1
  -  Ian Jackson

== GRUB2 ==

*  GRUB2 multiboot2 (fair)
  -  Daniel Kiper

== OSSTEST ==

*  OSSTest: libvirt (good)
  -  Ian Campbell

== Deferred to Xen 4.6 ==

*  Default to credit2 (none)
   cpu pinning, numa affinity and cpu reservation
  -  George Dunlap

*  "Short" grant copy (just header) of packets. (none)
  -  Zoltan Kiss

*  cpuid leveling (none)
   http://xenbits.xen.org/people/andrewcoop/feature-levelling/feature-levelling-D.pdf
  -  Andrew Cooper

*  live migration knobs, there is no suitable code yet, just ideas (none)
    http://lists.xenproject.org/archives/html/xen-devel/2014-03/msg00785.html
  -  Olaf Hering

*  Further tmem cleanups/fixes (16TB etc) (fair)
  -  Bob Liu

== Up for grabs ==

*  PoD fixes
   if you boot with memory <= maxmem we have a size estimation bug

*  TLB flushing without locks in Xen

*  xl does not support specifying virtual function for passthrough device
   http://bugs.xenproject.org/xen/bug/22

*  PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
   http://bugs.xenproject.org/xen/bug/28

*  libx{c,l} error handling cleanup

*  Adding missing 'xend' features in libxl

*  xl list -l on a dom0-only system

*  xl list -l doesn't contain tty console port

*  xl: passing more defaults in configuration in xl.conf
   There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.

*  PVH - PVH working with shadow.
   Based on Tim's work

*  PVH - PCI passthrough for DomU.

*  AMD performance regressions

*  Performance due to hypercall preemption. More preemptions - slower. (none)

== Completed ==

*  alternative_asm in Xen (done)
  -  Feng Wu

*  SMAP (done)
  -  Feng Wu

*  Re-write of vHPET (done)
   aka hvm/hpet: Detect comparator values in the past
  -  Don Slutz

*  vAPIC in PVHVM guests (Xen side) (done)
  -  Boris Ostrovsky

*  libvirt and xl discard support, so that libvirt can start using it (done)
  -  Olaf Hering

*  Xen PVH dom0 (done)
  -  Mukesh Rathor

*  Linux PVH dom0 (done)
  -  Mukesh Rathor

*  OSSTest: upstream QEMU (done)
  -  Ian Campbell

*  amd_ucode cleanups, verify patch size(enhancement) (mostly in master except one patch)

*  Data breakpoint Extension support (new-feat) (in master)

*  Feature masking MSR support (enhancement) (in master)

*  Support BRCM TruManage chip (Serial over LAN support) (new-feat) (in master)

*  fix vmce_amd* functions, unify mce_amd mcheck initialization (fixes/cleanups)
  -  Aravind and Suravee

*  NUMA memory scrubbing (done)
  -  Konrad Rzeszutek Wilk

*  ARM  - IOMMU support (done)
  -  Julien Grall

*  ioreq-server, aka secondary emulators (done)
  -  Paul Durrant

*  Netback multiqueue (good)
  -  Wei Liu

*  ARM Interrupt latency reduction (no maintenance interrupts) (good)
  -  Stefano Stabellini

*  Bigger PCI hole in QEMU (done)
   Needs to be rebased
  -  Don Slutz

*  rework VM Generation ID (ok)
   v7 posted
  -  David Vrabel



[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update

[George Dunlap]

On 07/01/2014 05:43 PM, konrad.wilk@oracle.com wrote:
> *  PoD fixes
>     if you boot with memory <= maxmem we have a size estimation bug

Speaking of which, did anyone do a write-up of what we discussed at the 
hackathon?

ISTR coming to some kind of conclusion, but I'm a bit hazy as to exactly 
what it was...

  -George

Re: Xen 4.5 development update

[Jan Beulich]

>>> On 02.07.14 at 13:33, <george.dunlap@eu.citrix.com> wrote:
> On 07/01/2014 05:43 PM, konrad.wilk@oracle.com wrote:
>> *  PoD fixes
>>     if you boot with memory <= maxmem we have a size estimation bug
> 
> Speaking of which, did anyone do a write-up of what we discussed at the 
> hackathon?
> 
> ISTR coming to some kind of conclusion, but I'm a bit hazy as to exactly 
> what it was...

IIRC we didn't really manage to finish up, but were at the point where
we thought XENMEM_current_reservation returning an unadjusted
value was wrong, fixing of which might be the way to go. Later I
started thinking that this is probably not correct - not just because
of the change in behavior of an existing hypercall, but also since the
current reservation of a domain _is_ d->tot_pages without any
adjustment.

Sadly I don't recall what exact adjustment we had been thinking of.

Jan

Re: Xen 4.5 development update

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1360 bytes --]

On mar, 2014-07-01 at 12:43 -0400, konrad.wilk@oracle.com wrote:

> none - nothing yet
> fair - still working on it, patches are prototypes or RFC
> ok   - patches posted, acting on review
> good - some last minute pieces
> done - all done, might have bugs

> = Open =

> == x86 == 
> 

> *  HT enabled, virtualization overhead is high (Xen 4.4) (none)
>    kernbench demonstrated it
>    looking and tracing it
>   -  Dario Faggioli
>
Right. Anyway, I shall say more about this in a new thread.

> *  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
>    v11 posted
>   -  Dario Faggioli
> 
Indeed. v12 coming today.

> *  Repurpose SEDF Scheduler for Real-time (ok)
>    RFC patch posted (v1)
>   -  Joshua Whitehead, Robert VanVossen
>
TBF, Patches are still RFC.

Also, either somehow related to this, or, if you want as a new separate
item (something like 'New Real-Time Scheduling'):

 Upstreaming bits of RT-Xen by Meng Xu, Sisu Xi
 RFC posted (v1)

Given the RFC status, I'd say this is fair, according to legend above.

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel