Re: Xen 4.4 development update -- RFC for feature freeze timeline

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[유재용]

> On Thu, 26 Sep 2013, George Dunlap wrote:
> > Additional things which are likely to make it if we extend a month (18
> > Nov):
> 
> > * NUMA Memory migration
> > * Per-vcpu NUMA affinity
> > * PV NUMA interface
> > * ARM guest migration
> > * USB hotplug for libxl
> > 
> * new hypercalls for SWIOTLB on ARM

I have a question regarding the candidate features for feature freeze.
Can anyone declare a new feature or does it only for Citrix employees and maintainers?
For instance, if I want to upstream live migration feature in ARM, what am I going to do?

> > 
> > 
> > > Nonetheless, it does seem likely that delaying for a month may allow a
> > significant number of important features to get in.
> > 
> > Any thoughts?
> 
> + 1 for delaying by one month
> 

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[Jan Beulich]

>>> On 07.10.13 at 06:59, 유재용<jaeyong.yoo@samsung.com> wrote:
>>  On Thu, 26 Sep 2013, George Dunlap wrote:
>> > Additional things which are likely to make it if we extend a month (18
>> > Nov):
>> 
>> > * NUMA Memory migration
>> > * Per-vcpu NUMA affinity
>> > * PV NUMA interface
>> > * ARM guest migration
>> > * USB hotplug for libxl
>> > 
>> * new hypercalls for SWIOTLB on ARM
> 
> I have a question regarding the candidate features for feature freeze.
> Can anyone declare a new feature or does it only for Citrix employees and 
> maintainers?
> For instance, if I want to upstream live migration feature in ARM, what am I 
> going to do?

You'd announce (ideally in the feature _planning_ phase) that you
intend to work on a particular feature, and if possible give at least
a rough estimate when you expect to have things in shape to go in.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[George Dunlap]

On 07/10/13 05:59, 유재용 wrote:
>> On Thu, 26 Sep 2013, George Dunlap wrote:
>>> Additional things which are likely to make it if we extend a month (18
>>> Nov):
>>> * NUMA Memory migration
>>> * Per-vcpu NUMA affinity
>>> * PV NUMA interface
>>> * ARM guest migration
>>> * USB hotplug for libxl
>>>
>> * new hypercalls for SWIOTLB on ARM
> I have a question regarding the candidate features for feature freeze.
> Can anyone declare a new feature or does it only for Citrix employees and maintainers?
> For instance, if I want to upstream live migration feature in ARM, what am I going to do?

So first of all, "Citrix employee" is absolutely irrelevant to any
discussion. Committers are chosen based on their merit, contribution to
the community, and need, according to the Xen governance proceedures
(http://www.xenproject.org/governance.html). Everything else is just a
matter of influence. If you want something to happen, just try to
convince people that it would be a good idea. The more you have
contributed, and the higher the quality of your contribution, the more
weight your preferences and judgement will have; but if your idea is
good and your argument is persuasive, you will be able to convince
people even with little prior involvement. Citrix employees are on the
same ground as everyone else -- if they haven't contributed much, they
won't have much influence.

Secondly, the long list that I send out is just an attempt to track what
people are doing, so that we have an idea who is doing what, and what
state the different features are in, so that we can make good decisions
about when to make releases. It's really just book-keeping. Anyone can
submit features to be tracked; you just have to respond to one of the
e-mails, and then respond again with updates when I send it out again.

The list above is an attempt to *predict* what features *will be
accepted*, not an attempt to *decide* what features are accepted.
Features are accepted when the maintainers / committers think that they
are ready. What I am trying to do is help the community decide when to
do a release. If you think a specific feature worth delaying the release
for, just try to make a case.

Finally, it sounds like the feature you are talking about is already on
my list of things which may be worth delaying the feature freeze for a
month: "ARM guest migration". :-)

-George


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[Ian Campbell]

On Mon, 2013-10-07 at 10:49 +0000, 유재용 wrote:
> > ------- Original Message -------
> > Sender : Jan Beulich<JBeulich@suse.com>
> > Date : 2013-10-07 15:55 (GMT+09:00)
> > Title : Re: [Xen-devel] Xen 4.4 development update -- RFC for feature freeze timeline
> > 
> >>>> On 07.10.13 at 06:59, 유재용 wrote:
> >>>  On Thu, 26 Sep 2013, George Dunlap wrote:
> >>> > Additional things which are likely to make it if we extend a month (18
> >>> > Nov):
> >>> 
> >>> > * NUMA Memory migration
> >>> > * Per-vcpu NUMA affinity
> >>> > * PV NUMA interface
> >>> > * ARM guest migration
> >>> > * USB hotplug for libxl
> >>> > 
> >>> * new hypercalls for SWIOTLB on ARM
> >> 
> >> I have a question regarding the candidate features for feature freeze.
> >> Can anyone declare a new feature or does it only for Citrix employees and 
> >> maintainers?
> >> For instance, if I want to upstream live migration feature in ARM, what am I 
> >> going to do?
> >
> >You'd announce (ideally in the feature _planning_ phase) that you
> >intend to work on a particular feature, and if possible give at least
> >a rough estimate when you expect to have things in shape to go in.
> 
> Aparently, I have missed the announcement, my mistake.

It's no problem, these things are mostly guidelines, not rules and as
George says the list is just a prediction not a decision, it is entirely
possible for things to go in without having been listed there.

I have your migration patches in my queue to look at, hopefully this
week.

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[유재용]

> ------- Original Message -------
> Sender : Jan Beulich<JBeulich@suse.com>
> Date : 2013-10-07 15:55 (GMT+09:00)
> Title : Re: [Xen-devel] Xen 4.4 development update -- RFC for feature freeze timeline
> 
>>>> On 07.10.13 at 06:59, 유재용 wrote:
>>>  On Thu, 26 Sep 2013, George Dunlap wrote:
>>> > Additional things which are likely to make it if we extend a month (18
>>> > Nov):
>>> 
>>> > * NUMA Memory migration
>>> > * Per-vcpu NUMA affinity
>>> > * PV NUMA interface
>>> > * ARM guest migration
>>> > * USB hotplug for libxl
>>> > 
>>> * new hypercalls for SWIOTLB on ARM
>> 
>> I have a question regarding the candidate features for feature freeze.
>> Can anyone declare a new feature or does it only for Citrix employees and 
>> maintainers?
>> For instance, if I want to upstream live migration feature in ARM, what am I 
>> going to do?
>
>You'd announce (ideally in the feature _planning_ phase) that you
>intend to work on a particular feature, and if possible give at least
>a rough estimate when you expect to have things in shape to go in.

Aparently, I have missed the announcement, my mistake.
I think I have to be more careful catching up the mailing list.

Jaeyong
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[Jaeyong Yoo]

>------- Original Message -------
>Sender : George Dunlap<george.dunlap@eu.citrix.com>
>Date : 2013-10-07 18:53 (GMT+09:00)
>Title : Re: [Xen-devel] Xen 4.4 development update -- RFC for feature freeze timeline
>
>On 07/10/13 05:59, 유재용 wrote:
>>> On Thu, 26 Sep 2013, George Dunlap wrote:
>>>> Additional things which are likely to make it if we extend a month (18
>>>> Nov):
>>>> * NUMA Memory migration
>>>> * Per-vcpu NUMA affinity
>>>> * PV NUMA interface
>>>> * ARM guest migration
>>>> * USB hotplug for libxl
>>>>
>>> * new hypercalls for SWIOTLB on ARM
>> I have a question regarding the candidate features for feature freeze.
>> Can anyone declare a new feature or does it only for Citrix employees and maintainers?
>> For instance, if I want to upstream live migration feature in ARM, what am I going to do?
>
>So first of all, "Citrix employee" is absolutely irrelevant to any
>discussion. Committers are chosen based on their merit, contribution to
>the community, and need, according to the Xen governance proceedures
>(http://www.xenproject.org/governance.html). Everything else is just a
>matter of influence. If you want something to happen, just try to
>convince people that it would be a good idea. The more you have
>contributed, and the higher the quality of your contribution, the more
>weight your preferences and judgement will have; but if your idea is
>good and your argument is persuasive, you will be able to convince
>people even with little prior involvement. Citrix employees are on the
>same ground as everyone else -- if they haven't contributed much, they
>won't have much influence.
>
>Secondly, the long list that I send out is just an attempt to track what
>people are doing, so that we have an idea who is doing what, and what
>state the different features are in, so that we can make good decisions
>about when to make releases. It's really just book-keeping. Anyone can
>submit features to be tracked; you just have to respond to one of the
>e-mails, and then respond again with updates when I send it out again.
>
>The list above is an attempt to *predict* what features *will be
>accepted*, not an attempt to *decide* what features are accepted.
>Features are accepted when the maintainers / committers think that they
>are ready. What I am trying to do is help the community decide when to
>do a release. If you think a specific feature worth delaying the release
>for, just try to make a case.
>
>Finally, it sounds like the feature you are talking about is already on
>my list of things which may be worth delaying the feature freeze for a
>month: "ARM guest migration". :-)

Thanks a lot for the explanation.
Now, I've got the clear idea how do I make a feature contribution.
I was worried that I may missed something for upstream the feature,
and I'm relieved that the feature is on your list. :-)

Thanks again,
Jaeyong
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[George Dunlap]

On 27/09/13 08:41, Jan Beulich wrote:
>>>> On 26.09.13 at 18:47, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
>> Nonetheless, it does seem likely that delaying for a month may allow a
>> significant number of important features to get in.
>>
>> Any thoughts?
> I'm in favor of pushing back by a month as long as this allows at
> least a fair share of the listed pending things to go in.

Well it's really hard to say -- back in January I said that the USB 
hot-plug series was basically ready to go in, but it wasn't ready by 
April when we had the feature freeze.  Sometimes I feel like I'm reading 
tea leaves here. :-)  All we can do is make our best stab at things, and 
then go back and see how we did.

> An
> alternative would be a weak feature freeze (no new features
> except for a well defined set) on the original date, but that would
> certainly undermine the stabilizing phase to some degree.

Well in theory it would allow "frozen" parts of the code (those not 
touched by the well-defined set of features) to start stabilizing while 
we are still working on non-frozen parts.  But the non-frozen parts 
still need to be stabilized, and the features we are talking about 
including are pretty big and will need a decent amount of time for 
stabilization; so I don't really see how doing a partial freeze is going 
to really help that much.

  -George

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[George Dunlap]

On 27/09/13 12:52, Boris Ostrovsky wrote:
> ----- George.Dunlap@eu.citrix.com wrote:
>
>> * xenperf
>>    prognosis: ?
>>    Owner: Boris Otovsky
>>    status: v2 patches posted
> I expect to have v3 for hypervisor before Oct. 18. I should also be able to post v2 for Linux by then although given the churn in ABI I am not sure it is worth it.
>
> (and Otovsky -> Ostrovsky)

Oops -- not sure how I got that mixed up... Sorry!

  -George

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[Boris Ostrovsky]

----- George.Dunlap@eu.citrix.com wrote:

> 
> * xenperf
>   prognosis: ?
>   Owner: Boris Otovsky
>   status: v2 patches posted

I expect to have v3 for hypervisor before Oct. 18. I should also be able to post v2 for Linux by then although given the churn in ABI I am not sure it is worth it. 

(and Otovsky -> Ostrovsky)

-boris

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[Stefano Stabellini]

On Thu, 26 Sep 2013, George Dunlap wrote:
> Additional things which are likely to make it if we extend a month (18
> Nov):
> 
> * NUMA Memory migration
> * Per-vcpu NUMA affinity
> * PV NUMA interface
> * ARM guest migration
> * USB hotplug for libxl

* new hypercalls for SWIOTLB on ARM


> Nonetheless, it does seem likely that delaying for a month may allow a
> significant number of important features to get in.
> 
> Any thoughts?

+ 1 for delaying by one month

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[Jan Beulich]

>>> On 27.09.13 at 11:37, David Vrabel <david.vrabel@citrix.com> wrote:
> On 27/09/13 08:38, Jan Beulich wrote:
>>>>> On 26.09.13 at 18:47, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
>>> * Event channel scalability
>> 
>> I don't recall there having been significant change requests to the
>> last posting - David, what's your estimation here?
> 
> The hypervisor and toolstack changes are now done.  There's one
> remaining issue with the Linux side.
> 
> Should be easily done before Oct 18.
> 
> I can repost the Xen series now if people would prefer an earlier review.

Unless the Linux side issue is risking to need an interface change
to deal with, I'd recommend doing so, as in that case the
hypervisor/tools side could then already go in (be sure to Cc
Keir, as he'll need to ack most of the hypervisor side; not sure
whether you got tools side acks already).

Jan

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[David Vrabel]

On 27/09/13 08:38, Jan Beulich wrote:
>>>> On 26.09.13 at 18:47, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
>> * Event channel scalability
> 
> I don't recall there having been significant change requests to the
> last posting - David, what's your estimation here?

The hypervisor and toolstack changes are now done.  There's one
remaining issue with the Linux side.

Should be easily done before Oct 18.

I can repost the Xen series now if people would prefer an earlier review.

David

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[Jan Beulich]

>>> On 26.09.13 at 18:47, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
> Nonetheless, it does seem likely that delaying for a month may allow a
> significant number of important features to get in.
> 
> Any thoughts?

I'm in favor of pushing back by a month as long as this allows at
least a fair share of the listed pending things to go in. An
alternative would be a weak feature freeze (no new features
except for a well defined set) on the original date, but that would
certainly undermine the stabilizing phase to some degree.

Jan

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[Jan Beulich]

>>> On 26.09.13 at 18:47, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
> * Event channel scalability

I don't recall there having been significant change requests to the
last posting - David, what's your estimation here?

Jan

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[Elena Ufimtseva]

On Thu, Sep 26, 2013 at 1:24 PM, Dario Faggioli
<dario.faggioli@citrix.com> wrote:
> On gio, 2013-09-26 at 17:47 +0100, George Dunlap wrote:
>> Additional things which are likely to make it if we extend a month (18
>> Nov):
>>
>> * NUMA Memory migration
>>
> Yep, I confirm: 18 Nov => probable. 18 Oct => no way. :-(
>
>> * Per-vcpu NUMA affinity
>>
> This may make it by 18 Oct too. 18 Nov, I'm positive it'll be in.
>
>> * PV NUMA interface
>>
> It's worth wait for Elena to comment on this, but (as said on IRC) I'm
> quite sure there's no way this can make it by 18 Oct.

I agree, taking into account all other things on the plate.. Nov 18 is
a feasible date.

>
>> Other things that are a possibility if we slip a month but definitely
>> not if we stick with the original schedule:
>>
>> * qemu-xen stubdomains (either Linux or rump kernel / BSD libc)
>>
> Which would be something really cool to have on the release notes,
> especially the rump kernel variant. ;-P
>
>> Nonetheless, it does seem likely that delaying for a month may allow a
>> significant number of important features to get in.
>>
>> Any thoughts?
>>
> +1 for delaying 1 month.
>
> I think it's probably worth to mention that we have LinuxConEU and
> XenSummit coming (and just had LinuxConNA), for which people need to
> prepare and attend, which would probably mean even less
> developing/reviewing time for getting stuff in by 18 Oct.
>
> Regards,
> Dario
>
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>



-- 
Elena

Re: Xen 4.4 development update -- RFC for feature freeze timeline

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1513 bytes --]

On gio, 2013-09-26 at 17:47 +0100, George Dunlap wrote:
> Additional things which are likely to make it if we extend a month (18
> Nov):
> 
> * NUMA Memory migration
>
Yep, I confirm: 18 Nov => probable. 18 Oct => no way. :-(

> * Per-vcpu NUMA affinity
>
This may make it by 18 Oct too. 18 Nov, I'm positive it'll be in.

> * PV NUMA interface
>
It's worth wait for Elena to comment on this, but (as said on IRC) I'm
quite sure there's no way this can make it by 18 Oct.

> Other things that are a possibility if we slip a month but definitely
> not if we stick with the original schedule:
> 
> * qemu-xen stubdomains (either Linux or rump kernel / BSD libc)
>
Which would be something really cool to have on the release notes,
especially the rump kernel variant. ;-P

> Nonetheless, it does seem likely that delaying for a month may allow a
> significant number of important features to get in.
>
> Any thoughts?
> 
+1 for delaying 1 month.

I think it's probably worth to mention that we have LinuxConEU and
XenSummit coming (and just had LinuxConNA), for which people need to
prepare and attend, which would probably mean even less
developing/reviewing time for getting stuff in by 18 Oct.

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Xen 4.4 development update -- RFC for feature freeze timeline

[George Dunlap]

This information will be mirrored on the Xen 4.4 Roadmap wiki page:
 http://wiki.xen.org/wiki/Xen_Roadmap/4.4

= Timeline (RFC!!) =

Here is our current timeline based on a 6-month release:

* Feature freeze: 18 October 2013
* Code freezing point: 8 November 2013
* First RC: 26 November 2013
* Release: 21 January 2014

I've been going through the list and chatting with different
developers, and it seems like there are a number of features in
development that are close to being done, but not quite: they have a
fairly high probability of missing the 18 Oct window, but a pretty
decent chance of making the window if it were extended for a month.

Here are the lists:

Things which will probably make it by 18 Oct:

* Multi-vector MSI (done)
* Improved spice support in libxl (done)
* PVH for domUs
* Non-udev scripts for driver domains
* A raft of fixes from Coverity reports

Additional things which are likely to make it if we extend a month (18
Nov):

* NUMA Memory migration
* Per-vcpu NUMA affinity
* PV NUMA interface
* ARM guest migration
* USB hotplug for libxl

Other things that are a possibility if we slip a month but definitely
not if we stick with the original schedule:

* qemu-xen stubdomains (either Linux or rump kernel / BSD libc)
* Event channel scalability

Obviously there is no guarantee for any of these that a month will
suffice; developers are notoriously bad at estimation and generally
think they can do absolutely anything in 2-3 months.

Nonetheless, it does seem likely that delaying for a month may allow a
significant number of important features to get in.

Any thoughts?

Last updated: 26 September 2016

== Completed  ==

* Multi-vector PCI MSI (Hypervisor side)

* Improved Spice support on libxl
 - Added Spice vdagent support
 - Added Spice clipboard sharing support

== Open ==

* xend still in tree (x)
 - xl list -l on a dom0-only system
 - xl list -l doesn't contain tty console port
 - xl Alternate transport support for migration
 - xl PVSCSI support
 - xl PVUSB support

* qemu-upstream not freeing pirq
 > http://www.gossamer-threads.com/lists/xen/devel/281498
 status: patches posted, tested-and-acked

* Race in PV shutdown between tool detection and shutdown watch
 > http://www.gossamer-threads.com/lists/xen/devel/282467
 > Nothing to do with ACPI
 status: Probably a bug in Linux xen drivers

* credit scheduler doesn't update other fields when tslice updated from sysctl
 > Reported by Luwei Cheng <lwcheng@cs.hku.hk>
 > http://bugs.xenproject.org/xen/bug/16

* Supposed regression from a3513737 ("x86: allow guest to set/clear
 > MSI-X mask bit (try 2)"), as per
 > http://lists.xenproject.org/archives/html/xen-devel/2013-09/msg01589.html.

== Backlog ==

=== Testing coverage ===

* new libxl w/ previous versions of xl
 @IanJ

* Host S3 suspend
 @bguthro, @dariof

* Default [example] XSM policy
 @Stefano to ask Daniel D

* Xen on ARM
 # hardware
  @ianc
   emulator: @stefano to think about it

* Storage driver domains
 @roger

* HVM pci passthrough
 @anthony

* PV pce passthrough
 @konrad (or @george if he gets to it first)

* Network driver domains
 @George

* Nested virt?
 @intel (chased by George)

* Fix SRIOV test (chase intel)
 @ianj

* Fix bisector to e-mail blame-worthy parties
 @ianj

* Fix xl shutdown
  @ianj

* stub domains
  @athony

* performance benchmarks
  @dario

=== Clean-ups ===

* Polish up xenbugtool
  owner: wei.liu2@citrix.com

* ACPI WAET table vs RTC emulation mode
  owner: jan@suse
  prognosis: ?
 > An overly simplified fix was posted a while ago
 > (http://lists.xenproject.org/archives/html/xen-devel/2013-07/msg00122.html),
 > but Tim's objection is rather valid. I can't, however, estimate
 > if/when I would find time to learn what tools side changes are
 > necessary to accommodate a new HVM param, and hence this is
 > currently stalled.  The current solution (as of 3fa7fb8b ["x86/HVM:
 > RTC code must be in line with WAET flags passed by hvmloader"])
 > isn't desirable to be kept for 4.4.

* Sort out better memory / ballooning / dom0 autoballooning thing
 > Don't forget NUMA angle
 - Inaccurate / incomplete info from HV

* Implement Xen hypervisor dmesg log entry timestamps
 > https://xenorg.uservoice.com/forums/172169-xen-development/suggestions/3924048-implement-xen-hypervisor-dmesg-log-entry-timestamp
 > Request seems to be for a shorter stamp (seconds-only, rather than full date)

* Make network driver domains easier to set up / more useful
 - Make it easy to make a device assignable (in discussion)
 - Automatically start/shutdown (xendomains?)
 - Pause booting of other domains until network driver domain is up (necessary?)

* libxl: More fine-grained control over when to pass through a device
 > Some IOMMUs are secure; some are merely functional, some are not present.
 > Allow the adminitrator to set the default

* xl does not handle migrate interruption gracefully
  > If you start a localhost migrate, and press "Ctrl-C" in the middle,
  > you get two hung domains

* libxl / xl does not handle failure of remote qemu gracefully
  > Easiest way to reproduce:
  >  - set "vncunused=0" and do a local migrate
  >  - The "remote" qemu will fail because the vnc port is in use
  > The failure isn't the problem, but everything being stuck afterwards is

* mac address changes on reboot if not specified in config file
  > Needs a robust way to "add" to the config

* qxl
  > http://bugs.xenproject.org/xen/bug/11
  - Uninitialized struct element in qemu
  - Revert 5479961 to re-enable qxl in xl,libxl
  - Option in Xen top-level to enable qxl support in qemu tree
  - Fix sse2 MMIO issue
   - make word size arbitrary

* libxl config file

* libxl: Don't use RAW format for "URL"-based qdisks (e.g., rbd:rbd/foo.img)
  - Figure out whether to use a generic URL or have a specific type for each one
  - Check existence of disk file for all RAW

* acpi-related xenstore entries not propagated on migrate
 > http://www.gossamer-threads.com/lists/xen/devel/282466
 > Only used by hvmloader; only a clean-up, not a bug.

=== Big ticket items ===

* NUMA Memory migration
  owner: dario@citrix
  status: in progress

* Event channel scalability (FIFO event channels)
  owner: david@citrix
  status: RFC v3 posted
  Increase limit on event channels (currently 1024 for 32-bit guests,
  4096 for 64-bit guests)

* PVH mode (w/ Linux)
  owner: mukesh@oracle, george@citrix
  status (Linux): Acked, waiting for ABI to be nailed down
  status (Xen): v12 posted

* ARM stuff: ??

* Meta: PVIO NUMA improvements
 - NUMA affinity for vcpus
    owner: Dario
    status: in progress
 - PV guest NUMA interface
    owner: Elena
    status: v2 posted
 - Sensible dom0 NUMA layout
 - Toolstack pinning backend thread / virq to appropraite d0 vcpu
 - NUMA-aware ballooning
    owner: Li Yechen
    status: in progress

* xl migrate transport improvements
 owner: None
 > See discussion here: http://bugs.xenproject.org/xen/bug/19
 - Option to connect over a plain TCP socket rather than ssh
 - xl-migrate-recieve suitable for running in inetd
 - option for above to redirect log output somewhere useful
 - Documentation for setting up alternate transports

* HVM guest NUMA
  owner: Matt Wilson@amazon
  status: in progress (?)

* qemu-upstream stubdom, Linux
   owner: anthony@citrix
   status: in progress
   prognosis: ?
   qemu-upstream needs a more fully-featured libc than exists in
   mini-os.  Either work on a minimalist linux-based stubdom with
   glibc, or port one of the BSD libcs to minios.

* qemu-upstream stubdom, BSD libc
  prognosis: ?
  owner: ianj@citrix

* Network performance improvements
  owner: wei@citrix

* Disk performance improvements

* Xen EFI feature: Xen can boot from grub.efi
 owner: Daniel Kiper
 status: in progress

* libvirt/libxl integration (external)
 > need a status update
 - owner: jfehlig@suse, dario@citrix

* Default to credit2
 status: Probably not for 4.4
 - cpu pinning
 - NUMA affinity
 - cpu "reservation"

* xenperf
  prognosis: ?
  Owner: Boris Otovsky
  status: v2 patches posted

* blktap3
  owner: thanos@citrix
  status: on hold pending XenServer decision

* Nested virtualization on Intel

* Nested virtualization on AMD

* xl USB pass-through for HVM guests using Qemu USB emulation
  prognosis: Good if extended
  owner: George
  status: v6 patch series posted

* libxl: Spice usbredirection support for upstream qemu
 owner: fabio@M2R
 status: I'll post new patch version shortly

* libxl: usb2 and usb3 controller support for upstream qemu
 owner: fabio@M2R
 status: patch v5 posted, tested and working, awaiting reviews

* Rationalized backend scripts
  owner: roger@citrix
  status: patches posted

* Scripts for driver domains (depends on backend scripts)
  owner: roger@citrix
  status: Patches posted (?)
  prognosis: good

* Multi-vector PCI MSI (upstream Linux)
  owner: konrad@oracle

* xl: passing more defaults in configuration in xl.conf
  owner: ?
  There are a number of options for which it might be useful to pass a
  default in xl.conf.  For example, if we could have a default
  "backend" parameter for vifs, then it would be easy to switch back
  and forth between a backend in a driver domain and a backend in dom0.

* xl PVUSB pass-through for PV guests
* xl PVUSB pass-through for HVM guests
  owner: George
  status: ?
  xm/xend supports PVUSB pass-through to guests with PVUSB drivers
(both PV and HVM guests).
  - port the xm/xend functionality to xl.
  - this PVUSB feature does not require support or emulation from Qemu.
  - upstream the Linux frontend/backend drivers. Current
work-in-progress versions are in Konrad's git tree.
  - James Harper's GPLPV drivers for Windows include PVUSB frontend drivers.

* Guest EFI booting
 - status: tianocore in-tree, some build problems.
   Needs new owner.

* Xen EFI feature: pvops dom0 able to make use of EFI run-time
services (external)
 owner: Daniel Kiper
 status: Just begun

* New kexec implementation
  owner: dvrabel@citrix
  status: v7 posted, only minor updates expected

=== Wishlist / someday ===

* Make storage migration possible
  owner: ?
  status: none
  There needs to be a way, either via command-line or via some hooks,
  that someone can build a "storage migration" feature on top of libxl
  or xl.

* Full-VM snapshotting
  owner: ?
  status: none
  Have a way of coordinating the taking and restoring of VM memory and
  disk snapshots.  This would involve some investigation into the best
  way to accomplish this.

* VM Cloning
  owner: ?
  status: none
  Again, a way of coordinating the memory and disk aspects.  Research
  into the best way to do this would probably go along with the
  snapshotting feature.

* xl vm-{export,import}
  owner: ?
  status: none
  Allow xl to import and export VMs to other formats; particularly
  ovf, perhaps the XenServer format, or more.

* Memory: Replace PoD with paging mechanism
  owner: george@citrix
  status: none

* PV audio (audio for stubdom qemu)
  owner: stefano.panella@citrix
  status: ?

* Wait queues for mm
 > Needed for more advanced paging schemes
  owner: ?
  status: Draft posted Feb 2012; more work to do.

* V4V: Inter-domain communication
  owner (Xen): dominic.curran@citrix.com
  status (Xen): patches submitted
  owner (Linux driver):  stefano.panella@citrix
  status (Linux driver): in progress

* Serial console improvements
  owner: ?
  status: Stalled (see below)
  -xHCI debug port (Needs hardware)
  -Firewire (needs hardware)