Xen 4.4 development update: RC0 imminent

Xen 4.4 development update: RC0 imminent

[George Dunlap]

This information will be mirrored on the Xen 4.4 Roadmap wiki page:
 http://wiki.xen.org/wiki/Xen_Roadmap/4.4

We're nearly to the completion of the code freeze, scheduled for
tomorrow.  After the code freeze, only bug fixes and features marked
as "blockers" will be considered.  At the moment, the only feature
considered a blocker is experimental PVH dom0 support.

In early RCs, most bug fixes will be accepted; but in later RCs, even
bug fixes may be rejected if they risk breaking more important
functionality than they fix.

I don't think at this point every bug fix needs a blessing from me;
committers, if there are fixes which are obviously low-risk, just go
ahead and check them in.

I was thinking that for some of our new features, it would be good to
have a blog post describing the feature and how to test it.  This
would both raise awareness of the feature, and hopefully get it more
testing before the release.  We could choose a couple to focus on for
each test day.

The code freeze will be complete tomorrow, but we will not be cutting
the first RC until PVH dom0 support gets in (hopefully in the next few
days).

= Timeline =

Here is our current timeline based on a 6-month release:

* Feature freeze: 18 October 2013
* Code freezing point: 18 November 2013 <== WE ARE HERE
* First RCs: 6 December 2013
* Release: 21 January 2014

Last updated: 5 December 2013

== Completed ==

* Event channel scalability (FIFO event channels)

* Non-udev scripts for driver domains (non-Linux driver domains)

* Multi-vector PCI MSI (Hypervisor side)

* Improved Spice support on libxl
 - Added Spice vdagent support
 - Added Spice clipboard sharing support

* PHV domU (experimental only)

* pvgrub2 checked into grub upstream (external)

* ARM64 guest

* Guest EFI booting (tianocore)

* kexec

* Testing: Xen on ARM

* Update to SeaBIOS 1.7.3.1

* Update to qemu 1.6

* SWIOTLB (in Linux 3.13)

* Disk: indirect descriptors (in 3.11)

== Resolved since last update ==

* xen_platform_pci=0 doesn't work with qemu-xen

== Open ==

* qemu-upstream not freeing pirq
 > http://www.gossamer-threads.com/lists/xen/devel/281498
 > http://marc.info/?l=xen-devel&m=137265766424502
 status: patches posted; latest patches need testing

* Race in PV shutdown between tool detection and shutdown watch
 > http://www.gossamer-threads.com/lists/xen/devel/282467
 > Nothing to do with ACPI
 status: Patches posted

* Supposed regression from a3513737 ("x86: allow guest to set/clear
 > MSI-X mask bit (try 2)"), as per
 > http://lists.xenproject.org/archives/html/xen-devel/2013-09/msg01589.html.

* qemu-traditional mis-parses host bus 8 as 0
 > http://bugs.xenproject.org/xen/bug/15

* xl does not support specifying virtual function for passthrough device
 > http://bugs.xenproject.org/xen/bug/22

* xl support for vnc and vnclisten options with PV guests
 > http://bugs.xenproject.org/xen/bug/25

* xl does not handle migrate interruption gracefully
  > If you start a localhost migrate, and press "Ctrl-C" in the middle,
  > you get two hung domains

* libxl / xl does not handle failure of remote qemu gracefully
  > Easiest way to reproduce:
  >  - set "vncunused=0" and do a local migrate
  >  - The "remote" qemu will fail because the vnc port is in use
  > The failure isn't the problem, but everything being stuck afterwards is

* xl needs to disallow PoD with PCI passthrough
  >see http://xen.1045712.n5.nabble.com/PATCH-VT-d-Dis-allow-PCI-device-assignment-if-PoD-is-enabled-td2547788.html

* HPET interrupt stack overflow (when using hpet_broadcast mode and MSI
capable HPETs)
  owner: andyh@citrix
  status: patches posted, undergoing review iteration.

* PCI hole resize support hvmloader/qemu-traditional/qemu-upstream
with PCI/GPU passthrough
  > http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
  > Where Stefano writes:
  > 2) for Xen 4.4 rework the two patches above and improve
  > i440fx_update_pci_mem_hole: resizing the pci_hole subregion is not
  > enough, it also needs to be able to resize the system memory region
  > (xen.ram) to make room for the bigger pci_hole

* qemu memory leak?
  > http://lists.xen.org/archives/html/xen-users/2013-03/msg00276.html

=== Big ticket items ===

* PVH dom0 (w/ Linux)
  owner: mukesh@oracle, george@citrix
  status (Linux): Acked, waiting for ABI to be nailed down
  status (Xen): v5 posted; considered a blocker

* libvirt/libxl integration (external)
 - owner: jfehlig@suse, dario@citrix
 - patches posted (should be released before 4.4)
  - migration
  - PCI pass-through
 - In progress
  - integration w/ libvirt's lock manager
  - improved concurrency

* libxl: Spice usbredirection support for upstream qemu
 > Includes usb2/3 support
 status: v2 acked
 prognosis: good

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On Thu, Dec 5, 2013 at 4:09 PM, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>
> We're nearly to the completion of the code freeze, scheduled for
> tomorrow.  After the code freeze, only bug fixes and features marked
> as "blockers" will be considered.  At the moment, the only feature
> considered a blocker is experimental PVH dom0 support.
>
> In early RCs, most bug fixes will be accepted; but in later RCs, even
> bug fixes may be rejected if they risk breaking more important
> functionality than they fix.
>
> I don't think at this point every bug fix needs a blessing from me;
> committers, if there are fixes which are obviously low-risk, just go
> ahead and check them in.
>
> I was thinking that for some of our new features, it would be good to
> have a blog post describing the feature and how to test it.  This
> would both raise awareness of the feature, and hopefully get it more
> testing before the release.  We could choose a couple to focus on for
> each test day.

Features which might be worth highlighting for testing in blogs:

* Non-udev scripts for driver domains (non-Linux driver domains)
 - Roger Pau Monne

* PHV domU (experimental only)

* Improved Spice support on libxl
 - Fabio Fantoni

* Event channel scalability
 - David Vrabel

* pvgrub2 checked into grub upstream (external)
 - Vladmir Servinenko

* Guest EFI booting (tianocore)
 - Wei Liu

* kexec -- is this worth testing?
 - David Vrabel

* Disk: indirect descriptors (in 3.11)
 - ?

Are people willing to step up and write a brief description of what
the feature is, as well as a quick guide for how to test it?

 -George

Re: Xen 4.4 development update: RC0 imminent

[Andrew Cooper]

On 05/12/13 16:09, George Dunlap wrote:
> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>
> We're nearly to the completion of the code freeze, scheduled for
> tomorrow.  After the code freeze, only bug fixes and features marked
> as "blockers" will be considered.  At the moment, the only feature
> considered a blocker is experimental PVH dom0 support.
>
> In early RCs, most bug fixes will be accepted; but in later RCs, even
> bug fixes may be rejected if they risk breaking more important
> functionality than they fix.
>
> I don't think at this point every bug fix needs a blessing from me;
> committers, if there are fixes which are obviously low-risk, just go
> ahead and check them in.
>
> I was thinking that for some of our new features, it would be good to
> have a blog post describing the feature and how to test it.  This
> would both raise awareness of the feature, and hopefully get it more
> testing before the release.  We could choose a couple to focus on for
> each test day.
>
> The code freeze will be complete tomorrow, but we will not be cutting
> the first RC until PVH dom0 support gets in (hopefully in the next few
> days).
>
> = Timeline =
>
> Here is our current timeline based on a 6-month release:
>
> * Feature freeze: 18 October 2013
> * Code freezing point: 18 November 2013 <== WE ARE HERE
> * First RCs: 6 December 2013
> * Release: 21 January 2014
>
> Last updated: 5 December 2013
>
> == Completed ==
>
> * Event channel scalability (FIFO event channels)
>
> * Non-udev scripts for driver domains (non-Linux driver domains)
>
> * Multi-vector PCI MSI (Hypervisor side)
>
> * Improved Spice support on libxl
>  - Added Spice vdagent support
>  - Added Spice clipboard sharing support
>
> * PHV domU (experimental only)
>
> * pvgrub2 checked into grub upstream (external)
>
> * ARM64 guest
>
> * Guest EFI booting (tianocore)
>
> * kexec
>
> * Testing: Xen on ARM
>
> * Update to SeaBIOS 1.7.3.1
>
> * Update to qemu 1.6
>
> * SWIOTLB (in Linux 3.13)
>
> * Disk: indirect descriptors (in 3.11)
>
> == Resolved since last update ==
>
> * xen_platform_pci=0 doesn't work with qemu-xen
>
> == Open ==
>
> * qemu-upstream not freeing pirq
>  > http://www.gossamer-threads.com/lists/xen/devel/281498
>  > http://marc.info/?l=xen-devel&m=137265766424502
>  status: patches posted; latest patches need testing
>
> * Race in PV shutdown between tool detection and shutdown watch
>  > http://www.gossamer-threads.com/lists/xen/devel/282467
>  > Nothing to do with ACPI
>  status: Patches posted
>
> * Supposed regression from a3513737 ("x86: allow guest to set/clear
>  > MSI-X mask bit (try 2)"), as per
>  > http://lists.xenproject.org/archives/html/xen-devel/2013-09/msg01589.html.

There was no followup on that, so is possibly stale.

There has been a more recent commit
(http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=74fd0036deb585a139b63b26db025805ecedc37a)
which fixes up a Xen-Qemu communication error when unmasking MSI-X
interrupts which might be related?

>
> * qemu-traditional mis-parses host bus 8 as 0
>  > http://bugs.xenproject.org/xen/bug/15
>
> * xl does not support specifying virtual function for passthrough device
>  > http://bugs.xenproject.org/xen/bug/22
>
> * xl support for vnc and vnclisten options with PV guests
>  > http://bugs.xenproject.org/xen/bug/25
>
> * xl does not handle migrate interruption gracefully
>   > If you start a localhost migrate, and press "Ctrl-C" in the middle,
>   > you get two hung domains
>
> * libxl / xl does not handle failure of remote qemu gracefully
>   > Easiest way to reproduce:
>   >  - set "vncunused=0" and do a local migrate
>   >  - The "remote" qemu will fail because the vnc port is in use
>   > The failure isn't the problem, but everything being stuck afterwards is
>
> * xl needs to disallow PoD with PCI passthrough
>   >see http://xen.1045712.n5.nabble.com/PATCH-VT-d-Dis-allow-PCI-device-assignment-if-PoD-is-enabled-td2547788.html
>
> * HPET interrupt stack overflow (when using hpet_broadcast mode and MSI
> capable HPETs)
>   owner: andyh@citrix
>   status: patches posted, undergoing review iteration.

* Win2k3 SP2 RTC infinite loops
   > Regression introduced late in Xen-4.3 development
   owner: andrew.cooper@citrix
   status: patches posted, undergoing review. ( v2 ID
1386241748-9617-1-git-send-email-andrew.cooper3@citrix.com )

~Andrew

>
> * PCI hole resize support hvmloader/qemu-traditional/qemu-upstream
> with PCI/GPU passthrough
>   > http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
>   > Where Stefano writes:
>   > 2) for Xen 4.4 rework the two patches above and improve
>   > i440fx_update_pci_mem_hole: resizing the pci_hole subregion is not
>   > enough, it also needs to be able to resize the system memory region
>   > (xen.ram) to make room for the bigger pci_hole
>
> * qemu memory leak?
>   > http://lists.xen.org/archives/html/xen-users/2013-03/msg00276.html
>
> === Big ticket items ===
>
> * PVH dom0 (w/ Linux)
>   owner: mukesh@oracle, george@citrix
>   status (Linux): Acked, waiting for ABI to be nailed down
>   status (Xen): v5 posted; considered a blocker
>
> * libvirt/libxl integration (external)
>  - owner: jfehlig@suse, dario@citrix
>  - patches posted (should be released before 4.4)
>   - migration
>   - PCI pass-through
>  - In progress
>   - integration w/ libvirt's lock manager
>   - improved concurrency
>
> * libxl: Spice usbredirection support for upstream qemu
>  > Includes usb2/3 support
>  status: v2 acked
>  prognosis: good
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.4 development update: RC0 imminent

[Wei Liu]

On Thu, Dec 05, 2013 at 04:29:08PM +0000, George Dunlap wrote:
> On Thu, Dec 5, 2013 at 4:09 PM, George Dunlap
> <George.Dunlap@eu.citrix.com> wrote:
> > This information will be mirrored on the Xen 4.4 Roadmap wiki page:
> >  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
> >
> > We're nearly to the completion of the code freeze, scheduled for
> > tomorrow.  After the code freeze, only bug fixes and features marked
> > as "blockers" will be considered.  At the moment, the only feature
> > considered a blocker is experimental PVH dom0 support.
> >
> > In early RCs, most bug fixes will be accepted; but in later RCs, even
> > bug fixes may be rejected if they risk breaking more important
> > functionality than they fix.
> >
> > I don't think at this point every bug fix needs a blessing from me;
> > committers, if there are fixes which are obviously low-risk, just go
> > ahead and check them in.
> >
> > I was thinking that for some of our new features, it would be good to
> > have a blog post describing the feature and how to test it.  This
> > would both raise awareness of the feature, and hopefully get it more
> > testing before the release.  We could choose a couple to focus on for
> > each test day.
> 
> Features which might be worth highlighting for testing in blogs:
> 
> * Non-udev scripts for driver domains (non-Linux driver domains)
>  - Roger Pau Monne
> 
> * PHV domU (experimental only)
> 
> * Improved Spice support on libxl
>  - Fabio Fantoni
> 
> * Event channel scalability
>  - David Vrabel
> 
> * pvgrub2 checked into grub upstream (external)
>  - Vladmir Servinenko
> 
> * Guest EFI booting (tianocore)
>  - Wei Liu
> 

A bunch of critical patches are neither applied upstream nor in our tree
(though they are already acked / reviewed by maintainers), so I don't
think we want to advertise it as "working"...

Wei.

Re: Xen 4.4 development update: RC0 imminent

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1.1: Type: text/plain, Size: 167 bytes --]

On 05.12.2013 17:29, George Dunlap wrote:
> * pvgrub2 checked into grub upstream (external)
>  - Vladmir Servinenko
Please correct 2 typos (Vladimir Serbinenko)


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 291 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On 12/05/2013 04:34 PM, Wei Liu wrote:
> On Thu, Dec 05, 2013 at 04:29:08PM +0000, George Dunlap wrote:
>> On Thu, Dec 5, 2013 at 4:09 PM, George Dunlap
>> <George.Dunlap@eu.citrix.com> wrote:
>>> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>>>   http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>>>
>>> We're nearly to the completion of the code freeze, scheduled for
>>> tomorrow.  After the code freeze, only bug fixes and features marked
>>> as "blockers" will be considered.  At the moment, the only feature
>>> considered a blocker is experimental PVH dom0 support.
>>>
>>> In early RCs, most bug fixes will be accepted; but in later RCs, even
>>> bug fixes may be rejected if they risk breaking more important
>>> functionality than they fix.
>>>
>>> I don't think at this point every bug fix needs a blessing from me;
>>> committers, if there are fixes which are obviously low-risk, just go
>>> ahead and check them in.
>>>
>>> I was thinking that for some of our new features, it would be good to
>>> have a blog post describing the feature and how to test it.  This
>>> would both raise awareness of the feature, and hopefully get it more
>>> testing before the release.  We could choose a couple to focus on for
>>> each test day.
>> Features which might be worth highlighting for testing in blogs:
>>
>> * Non-udev scripts for driver domains (non-Linux driver domains)
>>   - Roger Pau Monne
>>
>> * PHV domU (experimental only)
>>
>> * Improved Spice support on libxl
>>   - Fabio Fantoni
>>
>> * Event channel scalability
>>   - David Vrabel
>>
>> * pvgrub2 checked into grub upstream (external)
>>   - Vladmir Servinenko
>>
>> * Guest EFI booting (tianocore)
>>   - Wei Liu
>>
> A bunch of critical patches are neither applied upstream nor in our tree
> (though they are already acked / reviewed by maintainers), so I don't
> think we want to advertise it as "working"...

OK -- would the Xen side of these be something that could come in as 
"bug fixes", or should I take this off the list of 4.4 features?

  -George

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On 12/05/2013 04:39 PM, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> On 05.12.2013 17:29, George Dunlap wrote:
>> * pvgrub2 checked into grub upstream (external)
>>   - Vladmir Servinenko
> Please correct 2 typos (Vladimir Serbinenko)

Sorry for the typos -- but in any case this is not a list to be 
re-published anywhere; it's a question. :-)

  -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update: RC0 imminent

[Roger Pau Monné]

On 05/12/13 17:29, George Dunlap wrote:
> On Thu, Dec 5, 2013 at 4:09 PM, George Dunlap
> <George.Dunlap@eu.citrix.com> wrote:
>> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>>
>> We're nearly to the completion of the code freeze, scheduled for
>> tomorrow.  After the code freeze, only bug fixes and features marked
>> as "blockers" will be considered.  At the moment, the only feature
>> considered a blocker is experimental PVH dom0 support.
>>
>> In early RCs, most bug fixes will be accepted; but in later RCs, even
>> bug fixes may be rejected if they risk breaking more important
>> functionality than they fix.
>>
>> I don't think at this point every bug fix needs a blessing from me;
>> committers, if there are fixes which are obviously low-risk, just go
>> ahead and check them in.
>>
>> I was thinking that for some of our new features, it would be good to
>> have a blog post describing the feature and how to test it.  This
>> would both raise awareness of the feature, and hopefully get it more
>> testing before the release.  We could choose a couple to focus on for
>> each test day.
> 
> Features which might be worth highlighting for testing in blogs:
> 
> * Non-udev scripts for driver domains (non-Linux driver domains)
>  - Roger Pau Monne
> 
> * PHV domU (experimental only)
> 
> * Improved Spice support on libxl
>  - Fabio Fantoni
> 
> * Event channel scalability
>  - David Vrabel
> 
> * pvgrub2 checked into grub upstream (external)
>  - Vladmir Servinenko
> 
> * Guest EFI booting (tianocore)
>  - Wei Liu
> 
> * kexec -- is this worth testing?
>  - David Vrabel
> 
> * Disk: indirect descriptors (in 3.11)
>  - ?

I did the implementation for this, but it's not directly related to Xen,
just to the Linux kernel. I'm not sure it's fair to announce this as a
new feature of Xen 4.4, when it completely depends on the Linux kernel
version the user is running.

Re: Xen 4.4 development update: RC0 imminent

[Wei Liu]

On Thu, Dec 05, 2013 at 04:39:27PM +0000, George Dunlap wrote:
> On 12/05/2013 04:34 PM, Wei Liu wrote:
> >On Thu, Dec 05, 2013 at 04:29:08PM +0000, George Dunlap wrote:
> >>On Thu, Dec 5, 2013 at 4:09 PM, George Dunlap
> >><George.Dunlap@eu.citrix.com> wrote:
> >>>This information will be mirrored on the Xen 4.4 Roadmap wiki page:
> >>>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
> >>>
> >>>We're nearly to the completion of the code freeze, scheduled for
> >>>tomorrow.  After the code freeze, only bug fixes and features marked
> >>>as "blockers" will be considered.  At the moment, the only feature
> >>>considered a blocker is experimental PVH dom0 support.
> >>>
> >>>In early RCs, most bug fixes will be accepted; but in later RCs, even
> >>>bug fixes may be rejected if they risk breaking more important
> >>>functionality than they fix.
> >>>
> >>>I don't think at this point every bug fix needs a blessing from me;
> >>>committers, if there are fixes which are obviously low-risk, just go
> >>>ahead and check them in.
> >>>
> >>>I was thinking that for some of our new features, it would be good to
> >>>have a blog post describing the feature and how to test it.  This
> >>>would both raise awareness of the feature, and hopefully get it more
> >>>testing before the release.  We could choose a couple to focus on for
> >>>each test day.
> >>Features which might be worth highlighting for testing in blogs:
> >>
> >>* Non-udev scripts for driver domains (non-Linux driver domains)
> >>  - Roger Pau Monne
> >>
> >>* PHV domU (experimental only)
> >>
> >>* Improved Spice support on libxl
> >>  - Fabio Fantoni
> >>
> >>* Event channel scalability
> >>  - David Vrabel
> >>
> >>* pvgrub2 checked into grub upstream (external)
> >>  - Vladmir Servinenko
> >>
> >>* Guest EFI booting (tianocore)
> >>  - Wei Liu
> >>
> >A bunch of critical patches are neither applied upstream nor in our tree
> >(though they are already acked / reviewed by maintainers), so I don't
> >think we want to advertise it as "working"...
> 
> OK -- would the Xen side of these be something that could come in as
> "bug fixes", or should I take this off the list of 4.4 features?
>

I pinged maintainer this morning, let's see his response later. He's in
GMT -8 timezone.

If we cannot get it merged within next week I would rather take it off
our list.

Wei.

>  -George

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On 12/05/2013 04:42 PM, Roger Pau Monné wrote:
> On 05/12/13 17:29, George Dunlap wrote:
>> On Thu, Dec 5, 2013 at 4:09 PM, George Dunlap
>> <George.Dunlap@eu.citrix.com> wrote:
>>> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>>>   http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>>>
>>> We're nearly to the completion of the code freeze, scheduled for
>>> tomorrow.  After the code freeze, only bug fixes and features marked
>>> as "blockers" will be considered.  At the moment, the only feature
>>> considered a blocker is experimental PVH dom0 support.
>>>
>>> In early RCs, most bug fixes will be accepted; but in later RCs, even
>>> bug fixes may be rejected if they risk breaking more important
>>> functionality than they fix.
>>>
>>> I don't think at this point every bug fix needs a blessing from me;
>>> committers, if there are fixes which are obviously low-risk, just go
>>> ahead and check them in.
>>>
>>> I was thinking that for some of our new features, it would be good to
>>> have a blog post describing the feature and how to test it.  This
>>> would both raise awareness of the feature, and hopefully get it more
>>> testing before the release.  We could choose a couple to focus on for
>>> each test day.
>> Features which might be worth highlighting for testing in blogs:
>>
>> * Non-udev scripts for driver domains (non-Linux driver domains)
>>   - Roger Pau Monne
>>
>> * PHV domU (experimental only)
>>
>> * Improved Spice support on libxl
>>   - Fabio Fantoni
>>
>> * Event channel scalability
>>   - David Vrabel
>>
>> * pvgrub2 checked into grub upstream (external)
>>   - Vladmir Servinenko
>>
>> * Guest EFI booting (tianocore)
>>   - Wei Liu
>>
>> * kexec -- is this worth testing?
>>   - David Vrabel
>>
>> * Disk: indirect descriptors (in 3.11)
>>   - ?
> I did the implementation for this, but it's not directly related to Xen,
> just to the Linux kernel. I'm not sure it's fair to announce this as a
> new feature of Xen 4.4, when it completely depends on the Linux kernel
> version the user is running.

It's part of the "Xen ecosystem", and so it should be tested and 
announced.  It makes sense to me to announce developments that happen 
all together with Xen releases; we can make it clear that these are 
features in *external projects* that happened during the Xen 4.4 
*timeframe*.  The alternative would be to announce them when the other 
projects had a release; but I think that may be diluting the messaging a 
bit.

In any case, if we don't announce them on Xen releases, we ought to pay 
attention and announce them when the projects do their release.

Lars / Russ, any opinions here?

  -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On Thu, Dec 5, 2013 at 4:09 PM, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>
> We're nearly to the completion of the code freeze, scheduled for
> tomorrow.  After the code freeze, only bug fixes and features marked
> as "blockers" will be considered.  At the moment, the only feature
> considered a blocker is experimental PVH dom0 support.
>
> In early RCs, most bug fixes will be accepted; but in later RCs, even
> bug fixes may be rejected if they risk breaking more important
> functionality than they fix.
>
> I don't think at this point every bug fix needs a blessing from me;
> committers, if there are fixes which are obviously low-risk, just go
> ahead and check them in.
>
> I was thinking that for some of our new features, it would be good to
> have a blog post describing the feature and how to test it.  This
> would both raise awareness of the feature, and hopefully get it more
> testing before the release.  We could choose a couple to focus on for
> each test day.

Lars / Russ,

We should be able to cut an RC0 sometime next week.  Do you want to
plan a test day for 16 December, and then maybe another one for 6
January?

 -George

Re: Xen 4.4 development update: RC0 imminent

[Ian Campbell]

On Thu, 2013-12-05 at 16:48 +0000, Wei Liu wrote:

> > >>* Guest EFI booting (tianocore)
> > >>  - Wei Liu
> > >>
> > >A bunch of critical patches are neither applied upstream nor in our tree
> > >(though they are already acked / reviewed by maintainers), so I don't
> > >think we want to advertise it as "working"...
> > 
> > OK -- would the Xen side of these be something that could come in as
> > "bug fixes", or should I take this off the list of 4.4 features?
> >
> 
> I pinged maintainer this morning, let's see his response later. He's in
> GMT -8 timezone.
> 
> If we cannot get it merged within next week I would rather take it off
> our list.

IIRC the Xen side patches were pretty small and straight forward and the
only issue was agreeing the ABI for the struct passed from hvmloader to
ovmf.

Other than that lack of ABI agreement is there anything else which would
block taking the patches on our side. In particular do they break what
little functionality there is with the existing ovmf code base which we
pull in?

Ian.

Re: Xen 4.4 development update: RC0 imminent

[David Vrabel]

On 05/12/13 16:29, George Dunlap wrote:
>  
> Features which might be worth highlighting for testing in blogs:
> 
> * Event channel scalability
>  - David Vrabel

I have a pending critical bug fix which I am running through some
automated tests right now.  There's not much point in testing this
without the fix.

I was also going to schedule some more extensive testing when a slot
becomes available, but if RC0 is imminent I will post the fix tomorrow
rather than waiting for further testing.

> * kexec -- is this worth testing?
>  - David Vrabel

Everything is worth testing...  This has already seen a fair amount of
testing by various people already (as well as extensive testing in
XenServer's automated test system) so I don't think it needs to be
called out specifically for more testing.

David

Re: Xen 4.4 development update: RC0 imminent

[Olaf Hering]

Because its almost Friday:

On Thu, Dec 05, George Dunlap wrote:

> * kexec -- is this worth testing?

Its at least worth noting what this is all about ;-)
kexec Xen?
kexec dom0?
kexec PV?
kexec PVH?
kexec PVonHVM?
kexec HVM?

At least the latter should already work without changes.

Olaf

Re: Xen 4.4 development update: RC0 imminent

[Lars Kurth]

>> * Disk: indirect descriptors (in 3.11)
>>   - ?
>
> It's part of the "Xen ecosystem", and so it should be tested and announced.  It makes sense to me to announce 
> developments that happen all together with Xen releases; we can make it clear that these are features in *external 
> projects* that happened during the Xen 4.4 *timeframe*.  The alternative would be to announce them when the 
> other projects had a release; but I think that may be diluting the messaging a bit.
>
> In any case, if we don't announce them on Xen releases, we ought to pay attention and announce them when the projects do their release.
> 
> Lars / Russ, any opinions here?

That works

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On 12/05/2013 05:01 PM, Lars Kurth wrote:
>
>>> * Disk: indirect descriptors (in 3.11)
>>>    - ?
>> It's part of the "Xen ecosystem", and so it should be tested and announced.  It makes sense to me to announce
>> developments that happen all together with Xen releases; we can make it clear that these are features in *external
>> projects* that happened during the Xen 4.4 *timeframe*.  The alternative would be to announce them when the
>> other projects had a release; but I think that may be diluting the messaging a bit.
>>
>> In any case, if we don't announce them on Xen releases, we ought to pay attention and announce them when the projects do their release.
>>
>> Lars / Russ, any opinions here?
> That works

That wasn't a yes/no question; it is a, "which do you think is best" 
question:

* Announcing features for external projects (linux, libvirt) in the main 
Xen release.  Advantage: Longer list of features, more concentrated coverage

* Announcing features for external projects when the external projects 
release.  Advantage: More frequent coverage, may build good-will between 
projects to announce their releases

   -George

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On 12/05/2013 04:59 PM, David Vrabel wrote:
> On 05/12/13 16:29, George Dunlap wrote:
>>   
>> Features which might be worth highlighting for testing in blogs:
>>
>> * Event channel scalability
>>   - David Vrabel
> I have a pending critical bug fix which I am running through some
> automated tests right now.  There's not much point in testing this
> without the fix.
>
> I was also going to schedule some more extensive testing when a slot
> becomes available, but if RC0 is imminent I will post the fix tomorrow
> rather than waiting for further testing.
>
>> * kexec -- is this worth testing?
>>   - David Vrabel
> Everything is worth testing...  This has already seen a fair amount of
> testing by various people already (as well as extensive testing in
> XenServer's automated test system) so I don't think it needs to be
> called out specifically for more testing.

OK -- but is it something that is worthwhile calling out specifically as 
a cool new feature?  Is it the kind of thing for which it would be 
helpful to raise awareness?

  -George

Re: Xen 4.4 development update: RC0 imminent

[David Vrabel]

On 05/12/13 17:01, Olaf Hering wrote:
> Because its almost Friday:
> 
> On Thu, Dec 05, George Dunlap wrote:
> 
>> * kexec -- is this worth testing?
> 
> Its at least worth noting what this is all about ;-)
> kexec Xen?

This one. i.e., executing an image that replaces Xen.  It's not a new
feature, but a new, more reliable implementation.

David

Re: Xen 4.4 development update: RC0 imminent

[Wei Liu]

On Thu, Dec 05, 2013 at 04:59:20PM +0000, Ian Campbell wrote:
> On Thu, 2013-12-05 at 16:48 +0000, Wei Liu wrote:
> 
> > > >>* Guest EFI booting (tianocore)
> > > >>  - Wei Liu
> > > >>
> > > >A bunch of critical patches are neither applied upstream nor in our tree
> > > >(though they are already acked / reviewed by maintainers), so I don't
> > > >think we want to advertise it as "working"...
> > > 
> > > OK -- would the Xen side of these be something that could come in as
> > > "bug fixes", or should I take this off the list of 4.4 features?
> > >
> > 
> > I pinged maintainer this morning, let's see his response later. He's in
> > GMT -8 timezone.
> > 
> > If we cannot get it merged within next week I would rather take it off
> > our list.
> 
> IIRC the Xen side patches were pretty small and straight forward and the
> only issue was agreeing the ABI for the struct passed from hvmloader to
> ovmf.
> 

Yes, that's the main missing bit. However the interface has been acked
on OVMF side, does that mean it is safe to go in now?

> Other than that lack of ABI agreement is there anything else which would
> block taking the patches on our side. In particular do they break what
> little functionality there is with the existing ovmf code base which we
> pull in?
> 

No, it only makes OVMF work better than what we have now. ;-)

On the other hand I have two more patches for our build system. Shall I
send them for review now?

Wei.

> Ian.

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On 12/05/2013 04:59 PM, David Vrabel wrote:
> On 05/12/13 16:29, George Dunlap wrote:
>>   
>> Features which might be worth highlighting for testing in blogs:
>>
>> * Event channel scalability
>>   - David Vrabel
> I have a pending critical bug fix which I am running through some
> automated tests right now.  There's not much point in testing this
> without the fix.
>
> I was also going to schedule some more extensive testing when a slot
> becomes available, but if RC0 is imminent I will post the fix tomorrow
> rather than waiting for further testing.

RC0 will have to wait until the PVH dom0 series is in; probably won't 
happen until Tuesday.

Also, this was meant to get the pipeline started; we can easily put the 
blog for this sometime in January.

  -George

Re: Xen 4.4 development update: RC0 imminent

[Ian Campbell]

On Thu, 2013-12-05 at 17:06 +0000, Wei Liu wrote:
> On Thu, Dec 05, 2013 at 04:59:20PM +0000, Ian Campbell wrote:
> > On Thu, 2013-12-05 at 16:48 +0000, Wei Liu wrote:
> > 
> > > > >>* Guest EFI booting (tianocore)
> > > > >>  - Wei Liu
> > > > >>
> > > > >A bunch of critical patches are neither applied upstream nor in our tree
> > > > >(though they are already acked / reviewed by maintainers), so I don't
> > > > >think we want to advertise it as "working"...
> > > > 
> > > > OK -- would the Xen side of these be something that could come in as
> > > > "bug fixes", or should I take this off the list of 4.4 features?
> > > >
> > > 
> > > I pinged maintainer this morning, let's see his response later. He's in
> > > GMT -8 timezone.
> > > 
> > > If we cannot get it merged within next week I would rather take it off
> > > our list.
> > 
> > IIRC the Xen side patches were pretty small and straight forward and the
> > only issue was agreeing the ABI for the struct passed from hvmloader to
> > ovmf.
> > 
> 
> Yes, that's the main missing bit. However the interface has been acked
> on OVMF side, does that mean it is safe to go in now?

I think so long as the risk that it might change is small we could take
it. The danger would be that we take it and release it and then someone
changes their mind or whatever. If they just spotted an issue then we'll
have to live with it -- just like we would if we committed to both ovmf
and Xen right now.

There's also an inherent danger in taking something into Xen without the
3rd party code which exercises it.

But the advantage is that when OVMF is updated it will be possible to
use it with Xen.

On the other hand if you need to build an OVMF yourself a few patches to
hvmloader are probably not a big deal either.

OK, so I'm clearly in two minds about this ;-)

George -- what do you think?

> > Other than that lack of ABI agreement is there anything else which would
> > block taking the patches on our side. In particular do they break what
> > little functionality there is with the existing ovmf code base which we
> > pull in?
> > 
> 
> No, it only makes OVMF work better than what we have now. ;-)
> 
> On the other hand I have two more patches for our build system. Shall I
> send them for review now?

It's probably a good idea to resend the whole lot anyhow.

Ian.

Re: Xen 4.4 development update: RC0 imminent

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2656 bytes --]

[Adding @publicity, Mukesh and Konrad]

On gio, 2013-12-05 at 16:29 +0000, George Dunlap wrote:
> On Thu, Dec 5, 2013 at 4:09 PM, George Dunlap
> > I was thinking that for some of our new features, it would be good to
> > have a blog post describing the feature and how to test it.  This
> > would both raise awareness of the feature, and hopefully get it more
> > testing before the release.  We could choose a couple to focus on for
> > each test day.
> 
That's a really cool idea. I was also thinking about something like
that, but more as something 'post-release'.

However, if we can get some, or all, of these posts in time for test
days, that would be even better.

> Features which might be worth highlighting for testing in blogs:
> 
> * Non-udev scripts for driver domains (non-Linux driver domains)
>  - Roger Pau Monne
> 
That would make a very cool blog post indeed. Roger?

> * PHV domU (experimental only)
> 
We definitely should have something about PVH being finally available on
the blog. Mukesh? Konrad? Or perhaps George?

> * Improved Spice support on libxl
>  - Fabio Fantoni
> 
Same here.

> * Event channel scalability
>  - David Vrabel
> 
There has been something already, I think about the design/proposal.
Still, it would be good to have a follow-up. However, among this an
kexec, in case David's time is limited (;-P) I probably think kexec is a
more fancy feature to do some fuss about. :-)

> * pvgrub2 checked into grub upstream (external)
>  - Vladmir Servinenko
> 
Yep, and I explicitly asked Vladimir a while back if he'd be
interested...

> * Guest EFI booting (tianocore)
>  - Wei Liu
> 
Sure.

> * kexec -- is this worth testing?
>  - David Vrabel
> 
Ditto.

> * Disk: indirect descriptors (in 3.11)
>  - ?
> 
This also has been covered. So, unless there is something really new
here, I'd rather have Roger blogging about non-Linux driver domains.

> Are people willing to step up and write a brief description of what
> the feature is, as well as a quick guide for how to test it?
> 
And, anyone interested to do that in the form of a blog post, namely,
write that description directly in a blog article, please, contact me
for the details.

Of course, whatever form you want to provide the above is fine, we can
do the work of putting it on the blog.

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update: RC0 imminent

[Wei Liu]

On Thu, Dec 05, 2013 at 05:16:55PM +0000, Ian Campbell wrote:
[...]
> > > > I pinged maintainer this morning, let's see his response later. He's in
> > > > GMT -8 timezone.
> > > > 
> > > > If we cannot get it merged within next week I would rather take it off
> > > > our list.
> > > 
> > > IIRC the Xen side patches were pretty small and straight forward and the
> > > only issue was agreeing the ABI for the struct passed from hvmloader to
> > > ovmf.
> > > 
> > 
> > Yes, that's the main missing bit. However the interface has been acked
> > on OVMF side, does that mean it is safe to go in now?
> 
> I think so long as the risk that it might change is small we could take
> it. The danger would be that we take it and release it and then someone
> changes their mind or whatever. If they just spotted an issue then we'll
> have to live with it -- just like we would if we committed to both ovmf
> and Xen right now.
> 
> There's also an inherent danger in taking something into Xen without the
> 3rd party code which exercises it.
> 
> But the advantage is that when OVMF is updated it will be possible to
> use it with Xen.
> 
> On the other hand if you need to build an OVMF yourself a few patches to
> hvmloader are probably not a big deal either.
> 
> OK, so I'm clearly in two minds about this ;-)
> 
> George -- what do you think?
> 

My two cents would be let's see if we can upstream OVMF side patches in
time (it looks quite close now but I cannot say for sure, it's not
controlled by us ;-)), then release it in 4.4 and mark this feature as
experimental. Otherwise there's no need to merge Xen side patches.

Wei.

Re: Xen 4.4 development update: RC0 imminent

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1149 bytes --]

On gio, 2013-12-05 at 17:05 +0000, George Dunlap wrote:
> On 12/05/2013 04:59 PM, David Vrabel wrote:
> >> * kexec -- is this worth testing?
> >>   - David Vrabel
> > Everything is worth testing...  This has already seen a fair amount of
> > testing by various people already (as well as extensive testing in
> > XenServer's automated test system) so I don't think it needs to be
> > called out specifically for more testing.
> 
> OK -- but is it something that is worthwhile calling out specifically as 
> a cool new feature?  Is it the kind of thing for which it would be 
> helpful to raise awareness?
> 
I think that is the case. As far as I remember, there hasn't been much
about this on the blog, so we may well take the opportunity that we've
reworked it, to shout out laud what it is and what it could be useful
for, no matter since when it's in. :-)

Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On 12/05/2013 05:34 PM, Wei Liu wrote:
> On Thu, Dec 05, 2013 at 05:16:55PM +0000, Ian Campbell wrote:
> [...]
>>>>> I pinged maintainer this morning, let's see his response later. He's in
>>>>> GMT -8 timezone.
>>>>>
>>>>> If we cannot get it merged within next week I would rather take it off
>>>>> our list.
>>>> IIRC the Xen side patches were pretty small and straight forward and the
>>>> only issue was agreeing the ABI for the struct passed from hvmloader to
>>>> ovmf.
>>>>
>>> Yes, that's the main missing bit. However the interface has been acked
>>> on OVMF side, does that mean it is safe to go in now?
>> I think so long as the risk that it might change is small we could take
>> it. The danger would be that we take it and release it and then someone
>> changes their mind or whatever. If they just spotted an issue then we'll
>> have to live with it -- just like we would if we committed to both ovmf
>> and Xen right now.
>>
>> There's also an inherent danger in taking something into Xen without the
>> 3rd party code which exercises it.
>>
>> But the advantage is that when OVMF is updated it will be possible to
>> use it with Xen.
>>
>> On the other hand if you need to build an OVMF yourself a few patches to
>> hvmloader are probably not a big deal either.

I think for the common user, applying patches is a much higher barrier 
than just downloading a repo.

>>
>> OK, so I'm clearly in two minds about this ;-)
>>
>> George -- what do you think?
>>
> My two cents would be let's see if we can upstream OVMF side patches in
> time (it looks quite close now but I cannot say for sure, it's not
> controlled by us ;-)), then release it in 4.4 and mark this feature as
> experimental. Otherwise there's no need to merge Xen side patches.

In time for what?

Like I said, I think downloading a repo / tarball and building it is 
much lower than applying patches and then building.  And changing 
editing the one file in the Xen tree that has the commit hash is easier 
yet.  So I think there's an advantage to checking them in even if OVMF 
isn't upstream by the time we release.

And since OVMF is broken now anyway, we can't really break it; so 
there's little risk. :-)

  -George

Re: Xen 4.4 development update: RC0 imminent

[Wei Liu]

On Thu, Dec 05, 2013 at 05:53:21PM +0000, George Dunlap wrote:
[...]
> >>OK, so I'm clearly in two minds about this ;-)
> >>
> >>George -- what do you think?
> >>
> >My two cents would be let's see if we can upstream OVMF side patches in
> >time (it looks quite close now but I cannot say for sure, it's not
> >controlled by us ;-)), then release it in 4.4 and mark this feature as
> >experimental. Otherwise there's no need to merge Xen side patches.
> 
> In time for what?
> 

I mean to have those OVMF side patches applied to upstream tree within
certain time frame then we can push those changes to our tree and
publish a hash in Config.mk in time for 4.4.

> Like I said, I think downloading a repo / tarball and building it is
> much lower than applying patches and then building.  And changing
> editing the one file in the Xen tree that has the commit hash is
> easier yet.  So I think there's an advantage to checking them in
> even if OVMF isn't upstream by the time we release.
> 

I guess this would work as well.

> And since OVMF is broken now anyway, we can't really break it; so
> there's little risk. :-)
> 

Right. I certainly cannot make it more broken than what we had in 4.3.
:-P

Wei.

>  -George

Re: Xen 4.4 development update: RC0 imminent

[Russell Pavlicek]

>That wasn't a yes/no question; it is a, "which do you think is best" question:

>* Announcing features for external projects (linux, libvirt) in the main Xen release.  Advantage: Longer list of features, more concentrated coverage

>* Announcing features for external projects when the external projects release.  Advantage: More frequent coverage, may build good-will between projects to announce their releases

My gut: do both.  Make noise in the Xen release, then make more noise when the external project releases.  Preface the second one with "as we previously indicated during the release of XXXX...".

News is news until people know about it.  We reach a very limited subset of the Open Source world with any piece of PR.  Exercising an opportunity to announce & announce again (under the guise of a reminder to people) is prudent, IMO.  We need to make lots of positive noise continually to overcome the existing market misconception that KVM is the future.

It's kind of like the old adage of voting in Chicago: "Vote early; vote often."  ;)

Russ Pavlicek
Xen Project Evangelist, Citrix Systems
Home Office: +1-301-829-5327
Mobile: +1-240-397-0199
UK VoIP: +44 1223 852 894

-----Original Message-----
From: George Dunlap [mailto:george.dunlap@eu.citrix.com] 
Sent: Thursday, December 05, 2013 12:05 PM
To: Lars Kurth; George Dunlap; Roger Pau Monne; xen-devel
Cc: Russell Pavlicek
Subject: Re: Xen 4.4 development update: RC0 imminent

On 12/05/2013 05:01 PM, Lars Kurth wrote:
>
>>> * Disk: indirect descriptors (in 3.11)
>>>    - ?
>> It's part of the "Xen ecosystem", and so it should be tested and 
>> announced.  It makes sense to me to announce developments that happen 
>> all together with Xen releases; we can make it clear that these are 
>> features in *external
>> projects* that happened during the Xen 4.4 *timeframe*.  The 
>> alternative would be to announce them when the other projects had a release; but I think that may be diluting the messaging a bit.
>>
>> In any case, if we don't announce them on Xen releases, we ought to pay attention and announce them when the projects do their release.
>>
>> Lars / Russ, any opinions here?
> That works

That wasn't a yes/no question; it is a, "which do you think is best" 
question:

* Announcing features for external projects (linux, libvirt) in the main Xen release.  Advantage: Longer list of features, more concentrated coverage

* Announcing features for external projects when the external projects release.  Advantage: More frequent coverage, may build good-will between projects to announce their releases

   -George

Re: Xen 4.4 development update: RC0 imminent

[Sander Eikelenboom]

Thursday, December 5, 2013, 9:06:16 PM, you wrote:

>>That wasn't a yes/no question; it is a, "which do you think is best" question:

>>* Announcing features for external projects (linux, libvirt) in the main Xen release.  Advantage: Longer list of features, more concentrated coverage

>>* Announcing features for external projects when the external projects release.  Advantage: More frequent coverage, may build good-will between projects to announce their releases

> My gut: do both.  Make noise in the Xen release, then make more noise when the external project releases.  Preface the second one with "as we previously indicated during the release of XXXX...".

I guess in practice it would be the other way around i think .. at least Linux and Qemu seem to have a faster paced development cycle.
So if such a project gets a feature that is related to Xen and the present or next version of Xen is going to take advantage of that, that news could be shared on a blog.

The release announcement could have a summary or a link to a summary article which gives an overview of features in external projects the new Xen release can now take advantage off and that were introduced in the external project in the timeframe of the Xen release.
Although that could lead to misconceptions as will i guess, say f.e.:
- As a external feature in linux, pv ticketlocks could be mentioned.
- It's xen related, it went into the kernel during the timeframe of the coming release.
- But by mentioning it specifically at the release it could be misinterpreted as being tied to the Xen version of the release (which it is not, in this case)

So it's probably important to at least make very clear:
- if it is tied to this release, or also applicable to the previous stable versions of Xen.
- in which version of the external project it was introduced

--
Sander

> News is news until people know about it.  We reach a very limited subset of the Open Source world with any piece of PR.  Exercising an opportunity to announce & announce again (under the guise of a reminder to people) is prudent, IMO.  We need to make lots of positive noise continually to overcome the existing market misconception that KVM is the future.

> It's kind of like the old adage of voting in Chicago: "Vote early; vote often."  ;)

> Russ Pavlicek
> Xen Project Evangelist, Citrix Systems
> Home Office: +1-301-829-5327
> Mobile: +1-240-397-0199
> UK VoIP: +44 1223 852 894

> -----Original Message-----
> From: George Dunlap [mailto:george.dunlap@eu.citrix.com] 
> Sent: Thursday, December 05, 2013 12:05 PM
> To: Lars Kurth; George Dunlap; Roger Pau Monne; xen-devel
> Cc: Russell Pavlicek
> Subject: Re: Xen 4.4 development update: RC0 imminent

> On 12/05/2013 05:01 PM, Lars Kurth wrote:
>>
>>>> * Disk: indirect descriptors (in 3.11)
>>>>    - ?
>>> It's part of the "Xen ecosystem", and so it should be tested and 
>>> announced.  It makes sense to me to announce developments that happen 
>>> all together with Xen releases; we can make it clear that these are 
>>> features in *external
>>> projects* that happened during the Xen 4.4 *timeframe*.  The 
>>> alternative would be to announce them when the other projects had a release; but I think that may be diluting the messaging a bit.
>>>
>>> In any case, if we don't announce them on Xen releases, we ought to pay attention and announce them when the projects do their release.
>>>
>>> Lars / Russ, any opinions here?
>> That works

> That wasn't a yes/no question; it is a, "which do you think is best" 
> question:

> * Announcing features for external projects (linux, libvirt) in the main Xen release.  Advantage: Longer list of features, more concentrated coverage

> * Announcing features for external projects when the external projects release.  Advantage: More frequent coverage, may build good-will between projects to announce their releases

>    -George

Re: Xen 4.4 development update: RC0 imminent

[Jan Beulich]

>>> On 05.12.13 at 17:34, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> On 05/12/13 16:09, George Dunlap wrote:
>> * Supposed regression from a3513737 ("x86: allow guest to set/clear
>>  > MSI-X mask bit (try 2)"), as per
>>  > http://lists.xenproject.org/archives/html/xen-devel/2013-09/msg01589.html.
> 
> There was no followup on that, so is possibly stale.
> 
> There has been a more recent commit
> (http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=74fd0036deb585a139b 
> 63b26db025805ecedc37a)
> which fixes up a Xen-Qemu communication error when unmasking MSI-X
> interrupts which might be related?

That's not just related, that _is_ the one fixing the regression.

Jan

Re: Xen 4.4 development update: RC0 imminent

[Ian Campbell]

On Thu, 2013-12-05 at 18:03 +0000, Wei Liu wrote:
> On Thu, Dec 05, 2013 at 05:53:21PM +0000, George Dunlap wrote:
> [...]
> > >>OK, so I'm clearly in two minds about this ;-)
> > >>
> > >>George -- what do you think?
> > >>
> > >My two cents would be let's see if we can upstream OVMF side patches in
> > >time (it looks quite close now but I cannot say for sure, it's not
> > >controlled by us ;-)), then release it in 4.4 and mark this feature as
> > >experimental. Otherwise there's no need to merge Xen side patches.
> > 
> > In time for what?
> > 
> 
> I mean to have those OVMF side patches applied to upstream tree within
> certain time frame then we can push those changes to our tree and
> publish a hash in Config.mk in time for 4.4.
> 
> > Like I said, I think downloading a repo / tarball and building it is
> > much lower than applying patches and then building.  And changing
> > editing the one file in the Xen tree that has the commit hash is
> > easier yet.  So I think there's an advantage to checking them in
> > even if OVMF isn't upstream by the time we release.
> > 
> 
> I guess this would work as well.
> 
> > And since OVMF is broken now anyway, we can't really break it; so
> > there's little risk. :-)
> > 
> 
> Right. I certainly cannot make it more broken than what we had in 4.3.
> :-P

I think that if we are going to commit now without waiting for the OVMF
side commit first we should omit "enable OVMF build for Linux by
default".

Ian.

Re: Xen 4.4 development update: RC0 imminent

[Lars Kurth]

> My gut: do both.  Make noise in the Xen release, then make more noise when the external project 
> releases.  Preface the second one with "as we previously indicated during the release of XXXX...".
I would definitely include features which impact/improve/... Xen in the information we give to the PR folks for the Xen 4.4 announcement, but be clear about this. The PR folks, then have all the information and can make the most of it.

If they are covered elsewhere again : that increases the impact

Lars

________________________________________
From: Russell Pavlicek
Sent: 05 December 2013 20:06
To: George Dunlap; Lars Kurth; George Dunlap; Roger Pau Monne; xen-devel
Subject: RE: Xen 4.4 development update: RC0 imminent

>That wasn't a yes/no question; it is a, "which do you think is best" question:

>* Announcing features for external projects (linux, libvirt) in the main Xen release.  Advantage: Longer list of features, more concentrated coverage

>* Announcing features for external projects when the external projects release.  Advantage: More frequent coverage, may build good-will between projects to announce their releases

My gut: do both.  Make noise in the Xen release, then make more noise when the external project releases.  Preface the second one with "as we previously indicated during the release of XXXX...".

News is news until people know about it.  We reach a very limited subset of the Open Source world with any piece of PR.  Exercising an opportunity to announce & announce again (under the guise of a reminder to people) is prudent, IMO.  We need to make lots of positive noise continually to overcome the existing market misconception that KVM is the future.

It's kind of like the old adage of voting in Chicago: "Vote early; vote often."  ;)

Russ Pavlicek
Xen Project Evangelist, Citrix Systems
Home Office: +1-301-829-5327
Mobile: +1-240-397-0199
UK VoIP: +44 1223 852 894

-----Original Message-----
From: George Dunlap [mailto:george.dunlap@eu.citrix.com]
Sent: Thursday, December 05, 2013 12:05 PM
To: Lars Kurth; George Dunlap; Roger Pau Monne; xen-devel
Cc: Russell Pavlicek
Subject: Re: Xen 4.4 development update: RC0 imminent

On 12/05/2013 05:01 PM, Lars Kurth wrote:
>
>>> * Disk: indirect descriptors (in 3.11)
>>>    - ?
>> It's part of the "Xen ecosystem", and so it should be tested and
>> announced.  It makes sense to me to announce developments that happen
>> all together with Xen releases; we can make it clear that these are
>> features in *external
>> projects* that happened during the Xen 4.4 *timeframe*.  The
>> alternative would be to announce them when the other projects had a release; but I think that may be diluting the messaging a bit.
>>
>> In any case, if we don't announce them on Xen releases, we ought to pay attention and announce them when the projects do their release.
>>
>> Lars / Russ, any opinions here?
> That works

That wasn't a yes/no question; it is a, "which do you think is best"
question:

* Announcing features for external projects (linux, libvirt) in the main Xen release.  Advantage: Longer list of features, more concentrated coverage

* Announcing features for external projects when the external projects release.  Advantage: More frequent coverage, may build good-will between projects to announce their releases

   -George

Re: Xen 4.4 development update: RC0 imminent

[Wei Liu]

On Fri, Dec 06, 2013 at 09:27:21AM +0000, Ian Campbell wrote:
> On Thu, 2013-12-05 at 18:03 +0000, Wei Liu wrote:
> > On Thu, Dec 05, 2013 at 05:53:21PM +0000, George Dunlap wrote:
> > [...]
> > > >>OK, so I'm clearly in two minds about this ;-)
> > > >>
> > > >>George -- what do you think?
> > > >>
> > > >My two cents would be let's see if we can upstream OVMF side patches in
> > > >time (it looks quite close now but I cannot say for sure, it's not
> > > >controlled by us ;-)), then release it in 4.4 and mark this feature as
> > > >experimental. Otherwise there's no need to merge Xen side patches.
> > > 
> > > In time for what?
> > > 
> > 
> > I mean to have those OVMF side patches applied to upstream tree within
> > certain time frame then we can push those changes to our tree and
> > publish a hash in Config.mk in time for 4.4.
> > 
> > > Like I said, I think downloading a repo / tarball and building it is
> > > much lower than applying patches and then building.  And changing
> > > editing the one file in the Xen tree that has the commit hash is
> > > easier yet.  So I think there's an advantage to checking them in
> > > even if OVMF isn't upstream by the time we release.
> > > 
> > 
> > I guess this would work as well.
> > 
> > > And since OVMF is broken now anyway, we can't really break it; so
> > > there's little risk. :-)
> > > 
> > 
> > Right. I certainly cannot make it more broken than what we had in 4.3.
> > :-P
> 
> I think that if we are going to commit now without waiting for the OVMF
> side commit first we should omit "enable OVMF build for Linux by
> default".
> 

Yes. We can wait until OVMF side it is upstreamed for this one.

Wei.

> Ian.

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On 12/05/2013 08:06 PM, Russell Pavlicek wrote:
>> That wasn't a yes/no question; it is a, "which do you think is best" question:
>> * Announcing features for external projects (linux, libvirt) in the main Xen release.  Advantage: Longer list of features, more concentrated coverage
>> * Announcing features for external projects when the external projects release.  Advantage: More frequent coverage, may build good-will between projects to announce their releases
> My gut: do both.  Make noise in the Xen release, then make more noise when the external project releases.  Preface the second one with "as we previously indicated during the release of XXXX..."

I hadn't considered that, but I suppose yes, that's even better. :-)

  -George

Re: Xen 4.4 development update: RC0 imminent

[George Dunlap]

On 12/06/2013 09:07 AM, Jan Beulich wrote:
>>>> On 05.12.13 at 17:34, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> On 05/12/13 16:09, George Dunlap wrote:
>>> * Supposed regression from a3513737 ("x86: allow guest to set/clear
>>>   > MSI-X mask bit (try 2)"), as per
>>>   > http://lists.xenproject.org/archives/html/xen-devel/2013-09/msg01589.html.
>> There was no followup on that, so is possibly stale.
>>
>> There has been a more recent commit
>> (http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=74fd0036deb585a139b
>> 63b26db025805ecedc37a)
>> which fixes up a Xen-Qemu communication error when unmasking MSI-X
>> interrupts which might be related?
> That's not just related, that _is_ the one fixing the regression.

So I can mark this one as fixed then?

  -George

Re: Xen 4.4 development update: RC0 imminent

[Fabio Fantoni]

Il 05/12/2013 17:29, George Dunlap ha scritto:
> On Thu, Dec 5, 2013 at 4:09 PM, George Dunlap
> <George.Dunlap@eu.citrix.com> wrote:
>> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>>   http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>>
>> We're nearly to the completion of the code freeze, scheduled for
>> tomorrow.  After the code freeze, only bug fixes and features marked
>> as "blockers" will be considered.  At the moment, the only feature
>> considered a blocker is experimental PVH dom0 support.
>>
>> In early RCs, most bug fixes will be accepted; but in later RCs, even
>> bug fixes may be rejected if they risk breaking more important
>> functionality than they fix.
>>
>> I don't think at this point every bug fix needs a blessing from me;
>> committers, if there are fixes which are obviously low-risk, just go
>> ahead and check them in.
>>
>> I was thinking that for some of our new features, it would be good to
>> have a blog post describing the feature and how to test it.  This
>> would both raise awareness of the feature, and hopefully get it more
>> testing before the release.  We could choose a couple to focus on for
>> each test day.
> Features which might be worth highlighting for testing in blogs:
>
> * Non-udev scripts for driver domains (non-Linux driver domains)
>   - Roger Pau Monne
>
> * PHV domU (experimental only)
>
> * Improved Spice support on libxl
>   - Fabio Fantoni

libxl: Spice vdagent support for upstream qemu (on upstream git)

Usage:
- spicevdagent=1|0 (default=0)
Enables spice vdagent. The Spice vdagent is an optional component for
enhancing user experience and performing guest-oriented management
tasks. Its features includes: client mouse mode (no need to grab mouse
by client, no mouse lag), automatic adjustment of screen resolution,
copy and paste (text and image) between client and domU. It also
requires vdagent service installed on domU o.s. to work.

- spice_clipboard_sharing=1|0 (default=0)
Enables Spice clipboard sharing (copy/paste). It requires spicevdagent
enabled.

I used it since 2 year on windows domUs without problem.
About linux hvm domUs there is a problem with virtio-serial port create 
under /dev, to have it working pci=nomsi must be added on kernel boot line.
Latest post about it below:
http://lists.xen.org/archives/html/xen-devel/2013-12/msg01059.html

---

  libxl: spice usbredirection support for upstream qemu

Awaiting upstream approval latest version here is available here:
https://github.com/Fantu/Xen/commits/rebase/m2r-next
Require also usbversion patch.

Usage: spiceusbredirection=NUMBER (default=0)

Enables spice usbredirection. Creates NUMBER usbredirection channels
for redirection of up to 4 usb devices from spice client to domU's qemu.
It requires an usb controller and if not defined will automatically adds
an usb2 controller.

I used it since 2 year on hvm domUs (windows and linux) without problem, 
used mainly with usb2 controller.
In the latest tests made with  xen 4.4 and qemu 1.6 it is working with 
both usb 1,2,3 controllers.
It is also working even with new usb passthrough (from dom0) hotplug by 
George Dunlap patches.


>
> * Event channel scalability
>   - David Vrabel
>
> * pvgrub2 checked into grub upstream (external)
>   - Vladmir Servinenko
>
> * Guest EFI booting (tianocore)
>   - Wei Liu
>
> * kexec -- is this worth testing?
>   - David Vrabel
>
> * Disk: indirect descriptors (in 3.11)
>   - ?
>
> Are people willing to step up and write a brief description of what
> the feature is, as well as a quick guide for how to test it?
>
>   -George
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.4 development update: RC0 imminent

[Jan Beulich]

>>> On 06.12.13 at 14:07, George Dunlap <george.dunlap@eu.citrix.com> wrote:
> On 12/06/2013 09:07 AM, Jan Beulich wrote:
>>>>> On 05.12.13 at 17:34, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>> On 05/12/13 16:09, George Dunlap wrote:
>>>> * Supposed regression from a3513737 ("x86: allow guest to set/clear
>>>>   > MSI-X mask bit (try 2)"), as per
>>>>   > http://lists.xenproject.org/archives/html/xen-devel/2013-09/msg01589.html.
>>> There was no followup on that, so is possibly stale.
>>>
>>> There has been a more recent commit
>>> (http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=74fd0036deb585a139b 
>>> 63b26db025805ecedc37a)
>>> which fixes up a Xen-Qemu communication error when unmasking MSI-X
>>> interrupts which might be related?
>> That's not just related, that _is_ the one fixing the regression.
> 
> So I can mark this one as fixed then?

Yes.

Jan

Re: Xen 4.4 development update: RC0 imminent

[Lars Kurth]

On 05/12/2013 16:54, George Dunlap wrote:
> On Thu, Dec 5, 2013 at 4:09 PM, George Dunlap
> <George.Dunlap@eu.citrix.com> wrote:
>> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>>   http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>>
>> We're nearly to the completion of the code freeze, scheduled for
>> tomorrow.  After the code freeze, only bug fixes and features marked
>> as "blockers" will be considered.  At the moment, the only feature
>> considered a blocker is experimental PVH dom0 support.
>>
>> In early RCs, most bug fixes will be accepted; but in later RCs, even
>> bug fixes may be rejected if they risk breaking more important
>> functionality than they fix.
>>
>> I don't think at this point every bug fix needs a blessing from me;
>> committers, if there are fixes which are obviously low-risk, just go
>> ahead and check them in.
>>
>> I was thinking that for some of our new features, it would be good to
>> have a blog post describing the feature and how to test it.  This
>> would both raise awareness of the feature, and hopefully get it more
>> testing before the release.  We could choose a couple to focus on for
>> each test day.
> Lars / Russ,
>
> We should be able to cut an RC0 sometime next week.  Do you want to
> plan a test day for 16 December, and then maybe another one for 6
> January?
The RC0 has not happened last week. I guess we are a little late for a 
test day. Also, I will be struggling organizing a Test Day due to 
holidays and a planned trip to China. Unless Russ can pick it up, and we 
do have a schedule for test days after RC0, which we can stick to. Also, 
we need volunteers to prepare decent test instructions.
Lars