Xen 4.3 release planning proposal

Xen 4.3 release planning proposal

[George Dunlap]

Hello everyone!  With the completion of our first few release candidates
for 4.2, it's time to look forward and start planning for the 4.3
release.  I've volunteered to step up and help coordinate the release
for this cycle.

The 4.2 release cycle this time has been nearly a year and a half.
One of the problems with having such a long release is that people who
get in features early have to wait a long time for that feature to be
in a published version; they then have to wait even longer for it to
be part of a released distribution.  Historically the cycle has been
around 9 months, but this has not been made explicit.  Many people
(including myself) think that the 9 month release cycle was a good
cadence that we should aim for.

So I propose that we move to a time-based release schedule.  Rather
than aiming for a release date, I propose that we aim to do a "feature
freeze" six months after the 4.2 release -- that would be around March
1, 2013.  That way we'll probably end up releasing in 9 months' time,
around June 2013.  This is one of the things we can discuss at the Dev
Meeting before the Xen Summit next week.  If you have other opinions,
please let us know.

I will also be tracking ahead of time many of the features and
improvements that we want to try to get into 4.3.  Below is a list of
high-level features and improvements that the Citrix Xen.org team are
either planning on working on ourselves, or are aware of other people
working on, and that we should reasonably be able to get into the 4.3
release.  Most of them have owners, but many do not yet; volunteers
are welcome.

If you are planning on working any features not listed that you would like
to have tracked, please let me know.

I will be sending tracking updates similar to Ian Campbell's 4.2
release updates.  I think to begin with, weekly may be a bit
excessive.  I'll probably go for bi-weekly, and switch to weekly after
the feature freeze.

It should be noted this is not an exhaustive list, nor an immutable
one.  Our main priority will be to release within 9 months; only a
very important feature indeed would cause us to slip the release.

Features and improvements not on this list are of course welcome at
any time before the feature freeze.

Any questions and feedback are welcome!

Your 4.3 release coordinator,
 George Dunlap

* Event channel scalability
  owner: attilio@citrix
  Increase limit on event channels (currently 1024 for 32-bit guests,
  4096 for 64-bit guests)

* NUMA scheduler affinity
  owner: dario@citrix

* NUMA Memory migration
  owner: dario@citrix

* PVH mode, domU (w/ Linux)
  owner: mukesh@oracle

* PVH mode, dom0 (w/ Linux)
  owner: mukesh@oracle

* ARM server port
  owner: @citrix

* blktap3
  owner: @citrix

* Default to QEMU upstream
 - qemu-based stubdom (Linux or BSD libc)
    owner: anthony@citrix
    qemu-upstream needs a more fully-featured libc than exists in
    minios.  Either work on a minimalist linux-based stubdom with
    glibc, or port one of the BSD libcs to minios.

 - pci pass-thru
    owner: anthony@citrix

* Persistent grants
  owner: @citrix

* Multi-page blk rings
 - blkback in kernel (@intel)
 - qemu blkback

* Multi-page net protocol
  owner: ?
  expand the network ring protocol to allow multiple pages for
  increased throughput

* xl vm-{export,import}
  owner: ?
  Allow xl to import and export VMs to other formats; particularly
  ovf, perhaps the XenServer format, or more.


* xl USB pass-through for PV guests
  owner: ?
  Port the xend PV pass-through functionality to xl.

* openvswitch toostack integration
  owner: roger@citrix

* Rationalized backend scripts (incl. driver domains)
  owner: roger@citrix

* Full-VM snapshotting
  owner: ?
  Have a way of coordinating the taking and restoring of VM memory and
  disk snapshots.  This would involve some investigation into the best
  way to accomplish this.

* VM Cloning
  owner: ?
  Again, a way of coordinating the memory and disk aspects.  Research
  into the best way to do this would probably go along with the
  snapshotting feature.

* Make storage migration possible
  owner: ?
  There needs to be a way, either via command-line or via some hooks,
  that someone can build a "storage migration" feature on top of libxl
  or xl.

* PV audio (audio for stubdom qemu)
  owner: stefano.panella@citrix

* Memory: Replace PoD with paging mechanism
  owner: george@citrix

* Managed domains?

Re: Xen 4.3 release planning proposal

[Pasi Kärkkäinen]

On Mon, Aug 20, 2012 at 05:46:59PM +0100, George Dunlap wrote:
> 
> Features and improvements not on this list are of course welcome at
> any time before the feature freeze.
> 
> Any questions and feedback are welcome!
> 
> Your 4.3 release coordinator,
>  George Dunlap
> 

<snip>

> 
> * xl USB pass-through for PV guests
>   owner: ?
>   Port the xend PV pass-through functionality to xl.
> 

xm/xend PVUSB works for both PV and HVM guests, so xl should support PVUSB for both PV and HVM guests aswell.
James Harper's GPLPV drivers actually do have PVUSB frontend driver for Windows.

Also Suse's xenlinux forward-ported patches have PVUSB support in unmodified_drivers for HVM guests.


Another USB item:

* xl support for USB device passthru using QEMU emulated USB for HVM guests (no need for PVUSB drivers in the HVM guest).
  This works today in xm/xend with qemu-traditional, but is limited to USB 1.1, probably because 
  the old version of Qemu-dm-traditional which lacks USB 2.0/3.0.
  So xl support for emulated USB device passthru for both qemu-upstream and qemu-traditional.


More wishlist items:

* Nested hardware virtualization. Important for easier testing and development of Xen (Xen-on-Xen),
  and for running other hypervisors in Xen VMs. Interesting for labs, POCs, etc.

* VGA/GPU passthru support for AMD/NVIDIA; lots of patches on xen-devel archives, 
  but noone has yet stepped up to clean up and get them merged. 
  Currently Intel gfx passthru patches are merged to Xen, but primary ATI/NVIDIA require extra patches.
  This is actually something that a LOT of users ask often, it's discussed almost every day on ##xen on IRC.
  I wonder if XenClient folks could help here? 

* Dom0 Keyboard/mouse sharing to HVM guests; mainly needed by VGA/GPU passthru users.
  Fujitsu guys posted some patches for this in 2010, and XenClient guys in 2009 (iirc),
  but nothing got further developed and merged to upstream Xen.

* QXL virtual GPU support for SPICE. Someone was already developing this, 
  and posted patches earlier during 4.2 development cycle to xen-devel. 
  Upstream Qemu includes QXL support.

* PVSCSI support in XL. James Harper was (semi) interested in working with this,
  because he has a PVSCSI frontend driver in Windows GPLPV drivers, and he's using PVSCSI for tape backups himself.

* libvirt libxl driver improvements; support more Xen features. 
  Allows better using the Ubuntu/Debian/Fedora/RHEL/CentOS "default" virtualization GUI also with Xen.


Hopefully we'll find interested developers for these items :)


-- Pasi

Re: Xen 4.3 release planning proposal

[Konrad Rzeszutek Wilk]

On Mon, Aug 20, 2012 at 05:46:59PM +0100, George Dunlap wrote:
> Hello everyone!  With the completion of our first few release candidates
> for 4.2, it's time to look forward and start planning for the 4.3
> release.  I've volunteered to step up and help coordinate the release
> for this cycle.
> 
> The 4.2 release cycle this time has been nearly a year and a half.
> One of the problems with having such a long release is that people who
> get in features early have to wait a long time for that feature to be
> in a published version; they then have to wait even longer for it to
> be part of a released distribution.  Historically the cycle has been
> around 9 months, but this has not been made explicit.  Many people
> (including myself) think that the 9 month release cycle was a good
> cadence that we should aim for.
> 
> So I propose that we move to a time-based release schedule.  Rather
> than aiming for a release date, I propose that we aim to do a "feature
> freeze" six months after the 4.2 release -- that would be around March
> 1, 2013.  That way we'll probably end up releasing in 9 months' time,
> around June 2013.  This is one of the things we can discuss at the Dev
> Meeting before the Xen Summit next week.  If you have other opinions,
> please let us know.
> 
> I will also be tracking ahead of time many of the features and
> improvements that we want to try to get into 4.3.  Below is a list of
> high-level features and improvements that the Citrix Xen.org team are
> either planning on working on ourselves, or are aware of other people
> working on, and that we should reasonably be able to get into the 4.3
> release.  Most of them have owners, but many do not yet; volunteers
> are welcome.
> 
> If you are planning on working any features not listed that you would like
> to have tracked, please let me know.
> 
> I will be sending tracking updates similar to Ian Campbell's 4.2
> release updates.  I think to begin with, weekly may be a bit
> excessive.  I'll probably go for bi-weekly, and switch to weekly after
> the feature freeze.
> 
> It should be noted this is not an exhaustive list, nor an immutable
> one.  Our main priority will be to release within 9 months; only a
> very important feature indeed would cause us to slip the release.
> 
> Features and improvements not on this list are of course welcome at
> any time before the feature freeze.
> 
> Any questions and feedback are welcome!
> 
> Your 4.3 release coordinator,
>  George Dunlap
> 
> * Event channel scalability
>   owner: attilio@citrix
>   Increase limit on event channels (currently 1024 for 32-bit guests,
>   4096 for 64-bit guests)
> 
> * NUMA scheduler affinity
>   owner: dario@citrix
> 
> * NUMA Memory migration
>   owner: dario@citrix
> 
> * PVH mode, domU (w/ Linux)
>   owner: mukesh@oracle
> 
> * PVH mode, dom0 (w/ Linux)
>   owner: mukesh@oracle
> 
> * ARM server port
>   owner: @citrix
> 
> * blktap3
>   owner: @citrix
> 
> * Default to QEMU upstream
>  - qemu-based stubdom (Linux or BSD libc)
>     owner: anthony@citrix
>     qemu-upstream needs a more fully-featured libc than exists in
>     minios.  Either work on a minimalist linux-based stubdom with
>     glibc, or port one of the BSD libcs to minios.
> 
>  - pci pass-thru
>     owner: anthony@citrix
> 
> * Persistent grants
>   owner: @citrix
> 
> * Multi-page blk rings
>  - blkback in kernel (@intel)

.. and me as well.

>  - qemu blkback
> 
> * Multi-page net protocol
>   owner: ?
>   expand the network ring protocol to allow multiple pages for
>   increased throughput

Multiple people working on this. Ian, me, Annie, and Wei I believe.
> 
> * xl vm-{export,import}
>   owner: ?
>   Allow xl to import and export VMs to other formats; particularly
>   ovf, perhaps the XenServer format, or more.
> 
> 
> * xl USB pass-through for PV guests
>   owner: ?
>   Port the xend PV pass-through functionality to xl.

Well, what about the Linux side? The frontend/backend drivers haven't
really been proposed for upstream.

> 
> * openvswitch toostack integration
>   owner: roger@citrix
> 
> * Rationalized backend scripts (incl. driver domains)
>   owner: roger@citrix
> 
> * Full-VM snapshotting
>   owner: ?
>   Have a way of coordinating the taking and restoring of VM memory and
>   disk snapshots.  This would involve some investigation into the best
>   way to accomplish this.
> 
> * VM Cloning
>   owner: ?
>   Again, a way of coordinating the memory and disk aspects.  Research
>   into the best way to do this would probably go along with the
>   snapshotting feature.
> 
> * Make storage migration possible
>   owner: ?
>   There needs to be a way, either via command-line or via some hooks,
>   that someone can build a "storage migration" feature on top of libxl
>   or xl.
> 
> * PV audio (audio for stubdom qemu)
>   owner: stefano.panella@citrix

Is this a new person? Anyhow, there was a PV audio in pulseaudio as part
of the GSOC.
> 
> * Memory: Replace PoD with paging mechanism
>   owner: george@citrix
> 
> * Managed domains?
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

Re: Xen 4.3 release planning proposal

[George Dunlap]

On 20/08/12 21:28, Konrad Rzeszutek Wilk wrote:
> On Mon, Aug 20, 2012 at 05:46:59PM +0100, George Dunlap wrote:
>> * Multi-page blk rings
>>   - blkback in kernel (@intel)
> .. and me as well.
OK -- I think what I really need is just one person to be a 
point-of-contact, who can give updates on progress.  Shall I put you 
down for that, or should I find out who at Intel is working on it?
>>   - qemu blkback
>>
>> * Multi-page net protocol
>>    owner: ?
>>    expand the network ring protocol to allow multiple pages for
>>    increased throughput
> Multiple people working on this. Ian, me, Annie, and Wei I believe.
OK -- Ian C said he'd be the primary point of contact for this one until 
Wei gets back; if you or Annie want to be the contact directly, let me 
know. :-)
>> * xl USB pass-through for PV guests
>>    owner: ?
>>    Port the xend PV pass-through functionality to xl.
> Well, what about the Linux side? The frontend/backend drivers haven't
> really been proposed for upstream.
OK, I'll put that down as well.
>> * PV audio (audio for stubdom qemu)
>>    owner: stefano.panella@citrix
> Is this a new person? Anyhow, there was a PV audio in pulseaudio as part
> of the GSOC.
Stefano Panella is on the XenClient team at Citrix.  I think he probably 
knows about the previous work, but I'll make sure.  One thing I know 
he's been particularly concerned about is audio latency, particularly 
with using GoToMeeting or Skype inside a VM.

Thanks,
  -George

Re: Xen 4.3 release planning proposal

[George Dunlap]

On 20/08/12 20:14, Pasi Kärkkäinen wrote:
> Another USB item:
>
> * xl support for USB device passthru using QEMU emulated USB for HVM guests (no need for PVUSB drivers in the HVM guest).
>    This works today in xm/xend with qemu-traditional, but is limited to USB 1.1, probably because
>    the old version of Qemu-dm-traditional which lacks USB 2.0/3.0.
>    So xl support for emulated USB device passthru for both qemu-upstream and qemu-traditional.

OK, I'll put that on the list.  Thanks.

> More wishlist items:

I'm going to be experimenting with a website designed for user feedback, 
to both suggest features but also to help prioritize what you think are 
the important features. Expect an e-mail in a day or two with the 
announcement.

I think the Citrix team is probably mostly full for this release cycle; 
so anything that requires significant development work but doesn't 
already have developers working on it will probably have to be put off 
until later, unless you can convince the regular devs it's something to 
prioritize, or you can bring in more developers to work on it. :-)

> * Nested hardware virtualization. Important for easier testing and development of Xen (Xen-on-Xen),
>    and for running other hypervisors in Xen VMs. Interesting for labs, POCs, etc.

The ball is really in Intel / AMD's court on this one.  It's on our list 
of "things that might be nice", but given the other things we really do 
want to try to make into 4.3, wasn't that big of a priority for us.  If 
Intel or AMD want to make this a priority for them for 4.3, I can track it.

> * VGA/GPU passthru support for AMD/NVIDIA; lots of patches on xen-devel archives,
>    but noone has yet stepped up to clean up and get them merged.
>    Currently Intel gfx passthru patches are merged to Xen, but primary ATI/NVIDIA require extra patches.
>    This is actually something that a LOT of users ask often, it's discussed almost every day on ##xen on IRC.
>    I wonder if XenClient folks could help here?

What kind of patches are these? Are they mostly to pvops Linux?

> * Dom0 Keyboard/mouse sharing to HVM guests; mainly needed by VGA/GPU passthru users.
>    Fujitsu guys posted some patches for this in 2010, and XenClient guys in 2009 (iirc),
>    but nothing got further developed and merged to upstream Xen.

What exactly is involved here? If the work is mostly done, but just 
needs someone to dust it off and submit it, then it's probably something 
we can add to the list.

> * QXL virtual GPU support for SPICE. Someone was already developing this,
>    and posted patches earlier during 4.2 development cycle to xen-devel.
>    Upstream Qemu includes QXL support.

If "someone" wants to step up and claim responsibility, I'll put it on 
the list of things to track. :-)

> * PVSCSI support in XL. James Harper was (semi) interested in working with this,
>    because he has a PVSCSI frontend driver in Windows GPLPV drivers, and he's using PVSCSI for tape backups himself.
>
> * libvirt libxl driver improvements; support more Xen features.
>    Allows better using the Ubuntu/Debian/Fedora/RHEL/CentOS "default" virtualization GUI also with Xen.

This is pretty important.  Looking at feature parity between libvirt+KVM 
and libvirt+Xen on the Citrix xen.org to-do list, but to begin with it's 
more of a research than a feature, so wasn't on this particular list.  
If you happen to know a specific list of missing features, that might be 
something we could try to fit in for 4.3.

  -George

Re: Xen 4.3 release planning proposal

[Konrad Rzeszutek Wilk]

On Tue, Aug 21, 2012 at 11:06:07AM +0100, George Dunlap wrote:
> On 20/08/12 21:28, Konrad Rzeszutek Wilk wrote:
> >On Mon, Aug 20, 2012 at 05:46:59PM +0100, George Dunlap wrote:
> >>* Multi-page blk rings
> >>  - blkback in kernel (@intel)
> >.. and me as well.
> OK -- I think what I really need is just one person to be a
> point-of-contact, who can give updates on progress.  Shall I put you
> down for that, or should I find out who at Intel is working on it?

You can put me as contact person and I will keep track of
status.
> >>  - qemu blkback
> >>
> >>* Multi-page net protocol
> >>   owner: ?
> >>   expand the network ring protocol to allow multiple pages for
> >>   increased throughput
> >Multiple people working on this. Ian, me, Annie, and Wei I believe.
> OK -- Ian C said he'd be the primary point of contact for this one
> until Wei gets back; if you or Annie want to be the contact
> directly, let me know. :-)

Ian would be a better choice.

> >>* xl USB pass-through for PV guests
> >>   owner: ?
> >>   Port the xend PV pass-through functionality to xl.
> >Well, what about the Linux side? The frontend/backend drivers haven't
> >really been proposed for upstream.
> OK, I'll put that down as well.
> >>* PV audio (audio for stubdom qemu)
> >>   owner: stefano.panella@citrix
> >Is this a new person? Anyhow, there was a PV audio in pulseaudio as part
> >of the GSOC.
> Stefano Panella is on the XenClient team at Citrix.  I think he
> probably knows about the previous work, but I'll make sure.  One
> thing I know he's been particularly concerned about is audio
> latency, particularly with using GoToMeeting or Skype inside a VM.
> 
> Thanks,
>  -George
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

Re: Xen 4.3 release planning proposal

[Attilio Rao]

On 21/08/12 15:43, Jan Beulich wrote:
>>>> On 20.08.12 at 18:46, George Dunlap<George.Dunlap@eu.citrix.com>  wrote:
>>>>          
>> So I propose that we move to a time-based release schedule.  Rather
>> than aiming for a release date, I propose that we aim to do a "feature
>> freeze" six months after the 4.2 release -- that would be around March
>> 1, 2013.  That way we'll probably end up releasing in 9 months' time,
>> around June 2013.  This is one of the things we can discuss at the Dev
>> Meeting before the Xen Summit next week.  If you have other opinions,
>> please let us know.
>>      
> That would make for a scheduled 3 month feature freeze. I don't
> recall for how long we've been in feature freeze for 4.2 now, but
> from my perspective this is already definitely too long. Over that
> time I accumulated around 50 patches (not counting the bug fixes
> that I keep posting), and I'm sure it was never that bad during
> earlier feature freeze periods.
>
>    
>> * Event channel scalability
>>    owner: attilio@citrix
>>    Increase limit on event channels (currently 1024 for 32-bit guests,
>>    4096 for 64-bit guests)
>>      
> This one I have on my todo list as well.
>
>    

Do you mean that you already have a patch for that?
I'm starting making a complete plan for it. I discussed with Ian 
Campbell some aspects and I plan to send a detailed e-mail in the next 
few days.

Attilio

Re: Xen 4.3 release planning proposal

[Jan Beulich]

>>> On 20.08.12 at 18:46, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
> So I propose that we move to a time-based release schedule.  Rather
> than aiming for a release date, I propose that we aim to do a "feature
> freeze" six months after the 4.2 release -- that would be around March
> 1, 2013.  That way we'll probably end up releasing in 9 months' time,
> around June 2013.  This is one of the things we can discuss at the Dev
> Meeting before the Xen Summit next week.  If you have other opinions,
> please let us know.

That would make for a scheduled 3 month feature freeze. I don't
recall for how long we've been in feature freeze for 4.2 now, but
from my perspective this is already definitely too long. Over that
time I accumulated around 50 patches (not counting the bug fixes
that I keep posting), and I'm sure it was never that bad during
earlier feature freeze periods.

> * Event channel scalability
>   owner: attilio@citrix
>   Increase limit on event channels (currently 1024 for 32-bit guests,
>   4096 for 64-bit guests)

This one I have on my todo list as well.

Bigger things that I have ready to be submitted (and that aren't
cleanup in a more or less strong sense) are an EHCI debug port
driver for the console and a port of the CPUID based (i.e. not
requiring ACPI data to be passed from Dom0) idle driver from
Linux.

Larger things I have on my todo list and didn't see in yours are
- breaking the 5Tb memory barrier (for the moment aiming at
  16Tb due to certain implementation details)
- an xHCI debug port driver for the console (no Linux driver to
  clone from so far, so needs someone with good knowledge of
  the device and access to hardware, neither of which is the
  case for me, or finding out whether this is already in the works
  on the Linux side)
- if possible, a FireWire based console driver (but I've never done
  anything with FireWire, so it would depend on me finding ample
  time to first play with and then work on this)
- getting the public headers x32-clean

Plus there's one tools side item that I was asked to raise in the
course of 4.3 planning/development, but which I'm unlikely to
work on myself - removal of the hard code modprobe-s from
xencommons, properly loading the needed modules on demand
from the tool stack.

Jan

Re: Xen 4.3 release planning proposal

[Jan Beulich]

>>> On 21.08.12 at 16:36, Attilio Rao <attilio.rao@citrix.com> wrote:
> On 21/08/12 15:43, Jan Beulich wrote:
>>>>> On 20.08.12 at 18:46, George Dunlap<George.Dunlap@eu.citrix.com>  wrote:
>>> * Event channel scalability
>>>    owner: attilio@citrix
>>>    Increase limit on event channels (currently 1024 for 32-bit guests,
>>>    4096 for 64-bit guests)
>>>      
>> This one I have on my todo list as well.
> 
> Do you mean that you already have a patch for that?

Oh, no, I didn't mean to say that.

Jan

Re: Xen 4.3 release planning proposal

[George Dunlap]

On Tue, Aug 21, 2012 at 3:43 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 20.08.12 at 18:46, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
>> So I propose that we move to a time-based release schedule.  Rather
>> than aiming for a release date, I propose that we aim to do a "feature
>> freeze" six months after the 4.2 release -- that would be around March
>> 1, 2013.  That way we'll probably end up releasing in 9 months' time,
>> around June 2013.  This is one of the things we can discuss at the Dev
>> Meeting before the Xen Summit next week.  If you have other opinions,
>> please let us know.
>
> That would make for a scheduled 3 month feature freeze. I don't
> recall for how long we've been in feature freeze for 4.2 now, but
> from my perspective this is already definitely too long. Over that
> time I accumulated around 50 patches (not counting the bug fixes
> that I keep posting), and I'm sure it was never that bad during
> earlier feature freeze periods.

I think it's been nearly 5 months.  The main reason for the long
feature freeze, as far as I can tell, is that we did the freeze too
early; there were several major components (namely a stable libxl api)
that were nowhere near ready.  I'm hoping to avoid that this time by
having a clearer idea what we want from the release; features that
will be blockers must be in a reasonable state before we freeze.

A 3-month freeze is basically 6 weeks of clean-up and bug-fixing, and
6 weeks of RCs, which seems pretty realistic to me.  Feel free to
suggest another break-down if you want. :-)

>> * Event channel scalability
>>   owner: attilio@citrix
>>   Increase limit on event channels (currently 1024 for 32-bit guests,
>>   4096 for 64-bit guests)
>
> This one I have on my todo list as well.

I would say, "You should coordinate with Attilio", but I see Attilio
has already responded. :-)

>
> Bigger things that I have ready to be submitted (and that aren't
> cleanup in a more or less strong sense) are an EHCI debug port
> driver for the console and a port of the CPUID based (i.e. not
> requiring ACPI data to be passed from Dom0) idle driver from
> Linux.
>
> Larger things I have on my todo list and didn't see in yours are
> - breaking the 5Tb memory barrier (for the moment aiming at
>   16Tb due to certain implementation details)
> - an xHCI debug port driver for the console (no Linux driver to
>   clone from so far, so needs someone with good knowledge of
>   the device and access to hardware, neither of which is the
>   case for me, or finding out whether this is already in the works
>   on the Linux side)
> - if possible, a FireWire based console driver (but I've never done
>   anything with FireWire, so it would depend on me finding ample
>   time to first play with and then work on this)
> - getting the public headers x32-clean
>
> Plus there's one tools side item that I was asked to raise in the
> course of 4.3 planning/development, but which I'm unlikely to
> work on myself - removal of the hard code modprobe-s from
> xencommons, properly loading the needed modules on demand
> from the tool stack.

Great, thanks Jan -- I've put all of these on my list.

 -Geoge

Re: Xen 4.3 release planning proposal

[Pasi Kärkkäinen]

On Tue, Aug 21, 2012 at 01:56:27PM +0100, George Dunlap wrote:
> On 20/08/12 20:14, Pasi Kärkkäinen wrote:
> >Another USB item:
> >
> >* xl support for USB device passthru using QEMU emulated USB for HVM guests (no need for PVUSB drivers in the HVM guest).
> >   This works today in xm/xend with qemu-traditional, but is limited to USB 1.1, probably because
> >   the old version of Qemu-dm-traditional which lacks USB 2.0/3.0.
> >   So xl support for emulated USB device passthru for both qemu-upstream and qemu-traditional.
> 
> OK, I'll put that on the list.  Thanks.
> 

Thanks!

> >More wishlist items:
> 
> I'm going to be experimenting with a website designed for user
> feedback, to both suggest features but also to help prioritize what
> you think are the important features. Expect an e-mail in a day or
> two with the announcement.
> 

Good idea.

> I think the Citrix team is probably mostly full for this release
> cycle; so anything that requires significant development work but
> doesn't already have developers working on it will probably have to
> be put off until later, unless you can convince the regular devs
> it's something to prioritize, or you can bring in more developers to
> work on it. :-)
> 

Yeah I can see that :) If we aren't able to get these done for 4.3,
some of these might be good GSoC projects aswell.. 


> >* Nested hardware virtualization. Important for easier testing and development of Xen (Xen-on-Xen),
> >   and for running other hypervisors in Xen VMs. Interesting for labs, POCs, etc.
> 
> The ball is really in Intel / AMD's court on this one.  It's on our
> list of "things that might be nice", but given the other things we
> really do want to try to make into 4.3, wasn't that big of a
> priority for us.  If Intel or AMD want to make this a priority for
> them for 4.3, I can track it.
> 

I think AMD nested svm is in pretty good shape already,
but Intel nested VMX is not there yet.. 

I think we'll know more about this after XenSummit.

> >* VGA/GPU passthru support for AMD/NVIDIA; lots of patches on xen-devel archives,
> >   but noone has yet stepped up to clean up and get them merged.
> >   Currently Intel gfx passthru patches are merged to Xen, but primary ATI/NVIDIA require extra patches.
> >   This is actually something that a LOT of users ask often, it's discussed almost every day on ##xen on IRC.
> >   I wonder if XenClient folks could help here?
> 
> What kind of patches are these? Are they mostly to pvops Linux?
> 

I think mostly Qemu-dm patches.. some "tricks" are required to 
fetch the VBIOS from the physical gfx card so it can be copied to the HVM guest 
(some vendor specific hacks are needed to get a properly working copy of the VBIOS).
Also legacy IO ports, legacy vga memory ranges, MMIOs, VESA/VBE need to be passed thru etc.

Intel-specific tweaks are already in xen-unstable, but AMD/NVIDIA specific ones aren't.


> >* Dom0 Keyboard/mouse sharing to HVM guests; mainly needed by VGA/GPU passthru users.
> >   Fujitsu guys posted some patches for this in 2010, and XenClient guys in 2009 (iirc),
> >   but nothing got further developed and merged to upstream Xen.
> 
> What exactly is involved here? If the work is mostly done, but just
> needs someone to dust it off and submit it, then it's probably
> something we can add to the list.
> 

Here's the patch from Dietmar Hahn / Fujitsu:
http://lists.xen.org/archives/html/xen-devel/2010-03/msg01292.html

And here's some discussion about the XenClient keyboard/mouse sharing patch from Jean Guyader:

http://old-list-archives.xen.org/archives/html/xen-devel/2010-03/msg00979.html
and
http://lists.xen.org/archives/html/xen-devel/2012-01/msg00585.html


> >* QXL virtual GPU support for SPICE. Someone was already developing this,
> >   and posted patches earlier during 4.2 development cycle to xen-devel.
> >   Upstream Qemu includes QXL support.
> 
> If "someone" wants to step up and claim responsibility, I'll put it
> on the list of things to track. :-)
> 

Hehe, hopefully the "someone" notices this thread.. if not, we can go through xen-devel archives.


> >* PVSCSI support in XL. James Harper was (semi) interested in working with this,
> >   because he has a PVSCSI frontend driver in Windows GPLPV drivers, and he's using PVSCSI for tape backups himself.
> >


Hopefully James is still interested in this :) 


> >* libvirt libxl driver improvements; support more Xen features.
> >   Allows better using the Ubuntu/Debian/Fedora/RHEL/CentOS "default" virtualization GUI also with Xen.
> 
> This is pretty important.  Looking at feature parity between
> libvirt+KVM and libvirt+Xen on the Citrix xen.org to-do list, but to
> begin with it's more of a research than a feature, so wasn't on this
> particular list.  If you happen to know a specific list of missing
> features, that might be something we could try to fit in for 4.3.
> 

Yeah, I've been planning to test the latest libvirt libxl driver and 
see what works and what doesn't, but haven't gotten there yet.

-- Pasi

Re: Xen 4.3 release planning proposal

[Dan Magenheimer]

> From: George Dunlap [mailto:George.Dunlap@eu.citrix.com]
> Subject: [Xen-devel] Xen 4.3 release planning proposal
> 
> Hello everyone!  With the completion of our first few release candidates
> for 4.2, it's time to look forward and start planning for the 4.3
> release.  I've volunteered to step up and help coordinate the release
> for this cycle.
> 
> The 4.2 release cycle this time has been nearly a year and a half.
> One of the problems with having such a long release is that people who
> get in features early have to wait a long time for that feature to be
> in a published version; they then have to wait even longer for it to
> be part of a released distribution.  Historically the cycle has been
> around 9 months, but this has not been made explicit.  Many people
> (including myself) think that the 9 month release cycle was a good
> cadence that we should aim for.
> 
> So I propose that we move to a time-based release schedule.  Rather
> than aiming for a release date, I propose that we aim to do a "feature
> freeze" six months after the 4.2 release -- that would be around March
> 1, 2013.  That way we'll probably end up releasing in 9 months' time,
> around June 2013.  This is one of the things we can discuss at the Dev
> Meeting before the Xen Summit next week.  If you have other opinions,
> please let us know.

Hi George --

(Sorry if I missed relevant discussion on this... I'm way behind
on xen-devel.)

Just a thought...

Maybe it is time to move to match the well-known highly-greased
Linux kernel release process?  This would include, for example, a short
window for new functionality and a xen-next for pre-window shaking
out and merging (of new functionality) and testing.  As has
been pointed out, xen-unstable is, well, unstable for far too long.

It may not be necessary to aggressively match Linus' 8-9 week release
cycle or weekly rcN releases, but the core process is known to
work very well, is reasonably well documented, and will be familiar
to many in the open source community.

Dan

Re: Xen 4.3 release planning proposal

[Andrew Cooper]

On 29/08/12 21:53, Dan Magenheimer wrote:
>> From: George Dunlap [mailto:George.Dunlap@eu.citrix.com]
>> Subject: [Xen-devel] Xen 4.3 release planning proposal
>>
>> Hello everyone!  With the completion of our first few release candidates
>> for 4.2, it's time to look forward and start planning for the 4.3
>> release.  I've volunteered to step up and help coordinate the release
>> for this cycle.
>>
>> The 4.2 release cycle this time has been nearly a year and a half.
>> One of the problems with having such a long release is that people who
>> get in features early have to wait a long time for that feature to be
>> in a published version; they then have to wait even longer for it to
>> be part of a released distribution.  Historically the cycle has been
>> around 9 months, but this has not been made explicit.  Many people
>> (including myself) think that the 9 month release cycle was a good
>> cadence that we should aim for.
>>
>> So I propose that we move to a time-based release schedule.  Rather
>> than aiming for a release date, I propose that we aim to do a "feature
>> freeze" six months after the 4.2 release -- that would be around March
>> 1, 2013.  That way we'll probably end up releasing in 9 months' time,
>> around June 2013.  This is one of the things we can discuss at the Dev
>> Meeting before the Xen Summit next week.  If you have other opinions,
>> please let us know.
> Hi George --
>
> (Sorry if I missed relevant discussion on this... I'm way behind
> on xen-devel.)
>
> Just a thought...
>
> Maybe it is time to move to match the well-known highly-greased
> Linux kernel release process?  This would include, for example, a short
> window for new functionality and a xen-next for pre-window shaking
> out and merging (of new functionality) and testing.  As has
> been pointed out, xen-unstable is, well, unstable for far too long.
>
> It may not be necessary to aggressively match Linus' 8-9 week release
> cycle or weekly rcN releases, but the core process is known to
> work very well, is reasonably well documented, and will be familiar
> to many in the open source community.
>
> Dan

And in line with this, having a more formal approach/system for
backporting bugfixes would be fantastic.

~Andrew

>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com

Re: Xen 4.3 release planning proposal

[David Vrabel]

On 29/08/12 21:53, Dan Magenheimer wrote:
> 
> Maybe it is time to move to match the well-known highly-greased
> Linux kernel release process?  This would include, for example, a short
> window for new functionality and a xen-next for pre-window shaking
> out and merging (of new functionality) and testing.  As has
> been pointed out, xen-unstable is, well, unstable for far too long.
>
> It may not be necessary to aggressively match Linus' 8-9 week release
> cycle or weekly rcN releases, but the core process is known to
> work very well, is reasonably well documented, and will be familiar
> to many in the open source community.

I think such a system only works if you have a short release cycle.  If
the only time to merge new features is two weeks in every 6/9 months
then that is just far too long and is not very contributor-friendly.

Xen doesn't have the number of contributors or changes that make a Linux
kernel style process necessary.

David

Re: Xen 4.3 release planning proposal

[Dan Magenheimer]

> From: David Vrabel [mailto:dvrabel@cantab.net]
> Sent: Thursday, August 30, 2012 4:54 AM
> To: Dan Magenheimer
> Cc: George Dunlap; xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] Xen 4.3 release planning proposal
> 
> On 29/08/12 21:53, Dan Magenheimer wrote:
> >
> > Maybe it is time to move to match the well-known highly-greased
> > Linux kernel release process?  This would include, for example, a short
> > window for new functionality and a xen-next for pre-window shaking
> > out and merging (of new functionality) and testing.  As has
> > been pointed out, xen-unstable is, well, unstable for far too long.
> >
> > It may not be necessary to aggressively match Linus' 8-9 week release
> > cycle or weekly rcN releases, but the core process is known to
> > work very well, is reasonably well documented, and will be familiar
> > to many in the open source community.
> 
> I think such a system only works if you have a short release cycle.  If
> the only time to merge new features is two weeks in every 6/9 months
> then that is just far too long and is not very contributor-friendly.

That wasn't my point (though I see I wasn't very clear).

I meant that the part of the release cycle where new functionality
is accepted (the "window") should be a smaller _percentage_ of the
release cycle.  With Linux, it is about 20-25%.  For Xen it is probably
closer to 60-80%.  Once the window closes, the RC's start and new
functionality is put into "xen-next".  At the next window, the
release-Linus (George in this case) decides which functionality
in xen-next is stable enough to be pulled in during the window.

Of course, 18 months is far too long a release cycle for this approach,
and 9 months may be too long as well.  I think a target cycle
of 6 months with a "window" of 6 weeks would be a step in
the right direction
 
> Xen doesn't have the number of contributors or changes that make a Linux
> kernel style process necessary.

Personally, I think that's a self-fulfilling prophecy.  It is too hard
to use or develop on xen-unstable in part because too much is thrown in
(which, as George pointed out, is a result of developers learning that
if it doesn't go in xen-unstable, it will wait for many months).

Re: Xen 4.3 release planning proposal

[Jan Beulich]

>>> On 30.08.12 at 18:11, Dan Magenheimer <dan.magenheimer@oracle.com> wrote:
> Of course, 18 months is far too long a release cycle for this approach,
> and 9 months may be too long as well.  I think a target cycle
> of 6 months with a "window" of 6 weeks would be a step in
> the right direction

I disagree. Even the fix months of a freeze we're having right
now already is way too long. Nor do I personally consider the
two-weeks-out-of-ten (approximately) model on the Linux side
too nice. Having larger development windows, and shorter
stabilization periods is pretty desirable imo, the more on Xen
where, despite its name, -unstable really normally isn't that
unstable.

Jan

Re: Xen 4.3 release planning proposal

[Dan Magenheimer]

> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Friday, August 31, 2012 5:01 AM
> To: Dan Magenheimer
> Cc: David Vrabel; George Dunlap; xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] Xen 4.3 release planning proposal
> 
> >>> On 30.08.12 at 18:11, Dan Magenheimer <dan.magenheimer@oracle.com> wrote:
> > Of course, 18 months is far too long a release cycle for this approach,
> > and 9 months may be too long as well.  I think a target cycle
> > of 6 months with a "window" of 6 weeks would be a step in
> > the right direction
> 
> I disagree. Even the fix months of a freeze we're having right
> now already is way too long. Nor do I personally consider the
> two-weeks-out-of-ten (approximately) model on the Linux side
> too nice. Having larger development windows, and shorter
> stabilization periods is pretty desirable imo, the more on Xen
> where, despite its name, -unstable really normally isn't that
> unstable.

I apparently still haven't made my point clear.

With the current Xen model, there is a functionality freeze for
MONTHS during the rc cycles.  This guarantees a stampede
when the freeze ends, which almost certainly guarantees a long
period of instability in xen-unstable.  I agree that "-unstable
really isn't that unstable" but IMHO that's mostly because of the
very long release cycle.

With the Linux model, the stampede still occurs but it gets
sorted out in linux-next and the cream that rises to the top
is merged into the next release at the next (brief) window.

(Did you know that Linus now mostly refuses any new functionality
that wasn't already in linux-next at least for a week or so
before the window?)

So on Linux the "development window" is 100% _minus_ "the window",
i.e. the stabilization period and the "development window"
happen concurrently and the only time new functionality cannot
be taken is during the "window" (during which linux-next is
unavailable).

While the root cause of the difference between Xen and Linux
release cycles may indeed be that Linux has more developers,
I think everyone agrees the current Xen model (18 months since
last release) is broken, so it may be worth re-examining the
process rather than just saying "we'll try to do better this
time and maybe get it down to 9 months"... even though that
was the goal last time and it didn't work.  See classic
definition of insanity.

Just my opinion though...

Re: Xen 4.3 release planning proposal

[Martinx - ジェームズ]

[-- Attachment #1.1: Type: text/plain, Size: 3305 bytes --]

Hi Pasi!

 Can you tell me if it will be possible to use Xen like this:

 dom0 -> ATI GPU Passthrough as primary -> HVM domU with Catalyst -> Spice
-> Spice-Client ?

 I do not want to use Spice "alone" and, I do not want to use my domU with
my ATI without SPICE...  That makes sense?

Thanks!
Thiago

On 20 August 2012 16:14, Pasi Kärkkäinen <pasik@iki.fi> wrote:

> On Mon, Aug 20, 2012 at 05:46:59PM +0100, George Dunlap wrote:
> >
> > Features and improvements not on this list are of course welcome at
> > any time before the feature freeze.
> >
> > Any questions and feedback are welcome!
> >
> > Your 4.3 release coordinator,
> >  George Dunlap
> >
>
> <snip>
>
> >
> > * xl USB pass-through for PV guests
> >   owner: ?
> >   Port the xend PV pass-through functionality to xl.
> >
>
> xm/xend PVUSB works for both PV and HVM guests, so xl should support PVUSB
> for both PV and HVM guests aswell.
> James Harper's GPLPV drivers actually do have PVUSB frontend driver for
> Windows.
>
> Also Suse's xenlinux forward-ported patches have PVUSB support in
> unmodified_drivers for HVM guests.
>
>
> Another USB item:
>
> * xl support for USB device passthru using QEMU emulated USB for HVM
> guests (no need for PVUSB drivers in the HVM guest).
>   This works today in xm/xend with qemu-traditional, but is limited to USB
> 1.1, probably because
>   the old version of Qemu-dm-traditional which lacks USB 2.0/3.0.
>   So xl support for emulated USB device passthru for both qemu-upstream
> and qemu-traditional.
>
>
> More wishlist items:
>
> * Nested hardware virtualization. Important for easier testing and
> development of Xen (Xen-on-Xen),
>   and for running other hypervisors in Xen VMs. Interesting for labs,
> POCs, etc.
>
> * VGA/GPU passthru support for AMD/NVIDIA; lots of patches on xen-devel
> archives,
>   but noone has yet stepped up to clean up and get them merged.
>   Currently Intel gfx passthru patches are merged to Xen, but primary
> ATI/NVIDIA require extra patches.
>   This is actually something that a LOT of users ask often, it's discussed
> almost every day on ##xen on IRC.
>   I wonder if XenClient folks could help here?
>
> * Dom0 Keyboard/mouse sharing to HVM guests; mainly needed by VGA/GPU
> passthru users.
>   Fujitsu guys posted some patches for this in 2010, and XenClient guys in
> 2009 (iirc),
>   but nothing got further developed and merged to upstream Xen.
>
> * QXL virtual GPU support for SPICE. Someone was already developing this,
>   and posted patches earlier during 4.2 development cycle to xen-devel.
>   Upstream Qemu includes QXL support.
>
> * PVSCSI support in XL. James Harper was (semi) interested in working with
> this,
>   because he has a PVSCSI frontend driver in Windows GPLPV drivers, and
> he's using PVSCSI for tape backups himself.
>
> * libvirt libxl driver improvements; support more Xen features.
>   Allows better using the Ubuntu/Debian/Fedora/RHEL/CentOS "default"
> virtualization GUI also with Xen.
>
>
> Hopefully we'll find interested developers for these items :)
>
>
> -- Pasi
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

[-- Attachment #1.2: Type: text/html, Size: 4063 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.3 release planning proposal

[Pasi Kärkkäinen]

On Mon, Dec 17, 2012 at 09:57:59PM -0200, Martinx - ?$B%8%'!<%`%: wrote:
>    Hi Pasi!
>     Can you tell me if it will be possible to use Xen like this:
>     dom0 -> ATI GPU Passthrough as primary -> HVM domU with Catalyst -> Spice
>    -> Spice-Client ?
>     I do not want to use Spice "alone" and, I do not want to use my domU with
>    my ATI without SPICE...  That makes sense?
>    Thanks!

Afaik SPICE uses and requires the virtual QXL GPU for efficient operation, 
so it doesn't work with physical GPUs.

-- Pasi

>    Thiago
> 
>    On 20 August 2012 16:14, Pasi Kärkkäinen <[1]pasik@iki.fi> wrote:
> 
>      On Mon, Aug 20, 2012 at 05:46:59PM +0100, George Dunlap wrote:
>      >
>      > Features and improvements not on this list are of course welcome at
>      > any time before the feature freeze.
>      >
>      > Any questions and feedback are welcome!
>      >
>      > Your 4.3 release coordinator,
>      >  George Dunlap
>      >
> 
>      <snip>
>      >
>      > * xl USB pass-through for PV guests
>      >   owner: ?
>      >   Port the xend PV pass-through functionality to xl.
>      >
> 
>      xm/xend PVUSB works for both PV and HVM guests, so xl should support
>      PVUSB for both PV and HVM guests aswell.
>      James Harper's GPLPV drivers actually do have PVUSB frontend driver for
>      Windows.
> 
>      Also Suse's xenlinux forward-ported patches have PVUSB support in
>      unmodified_drivers for HVM guests.
> 
>      Another USB item:
> 
>      * xl support for USB device passthru using QEMU emulated USB for HVM
>      guests (no need for PVUSB drivers in the HVM guest).
>        This works today in xm/xend with qemu-traditional, but is limited to
>      USB 1.1, probably because
>        the old version of Qemu-dm-traditional which lacks USB 2.0/3.0.
>        So xl support for emulated USB device passthru for both qemu-upstream
>      and qemu-traditional.
> 
>      More wishlist items:
> 
>      * Nested hardware virtualization. Important for easier testing and
>      development of Xen (Xen-on-Xen),
>        and for running other hypervisors in Xen VMs. Interesting for labs,
>      POCs, etc.
> 
>      * VGA/GPU passthru support for AMD/NVIDIA; lots of patches on xen-devel
>      archives,
>        but noone has yet stepped up to clean up and get them merged.
>        Currently Intel gfx passthru patches are merged to Xen, but primary
>      ATI/NVIDIA require extra patches.
>        This is actually something that a LOT of users ask often, it's
>      discussed almost every day on ##xen on IRC.
>        I wonder if XenClient folks could help here?
> 
>      * Dom0 Keyboard/mouse sharing to HVM guests; mainly needed by VGA/GPU
>      passthru users.
>        Fujitsu guys posted some patches for this in 2010, and XenClient guys
>      in 2009 (iirc),
>        but nothing got further developed and merged to upstream Xen.
> 
>      * QXL virtual GPU support for SPICE. Someone was already developing
>      this,
>        and posted patches earlier during 4.2 development cycle to xen-devel.
>        Upstream Qemu includes QXL support.
> 
>      * PVSCSI support in XL. James Harper was (semi) interested in working
>      with this,
>        because he has a PVSCSI frontend driver in Windows GPLPV drivers, and
>      he's using PVSCSI for tape backups himself.
> 
>      * libvirt libxl driver improvements; support more Xen features.
>        Allows better using the Ubuntu/Debian/Fedora/RHEL/CentOS "default"
>      virtualization GUI also with Xen.
> 
>      Hopefully we'll find interested developers for these items :)
> 
>      -- Pasi
> 
>      _______________________________________________
>      Xen-devel mailing list
>      [2]Xen-devel@lists.xen.org
>      [3]http://lists.xen.org/xen-devel
> 
> References
> 
>    Visible links
>    1. mailto:pasik@iki.fi
>    2. mailto:Xen-devel@lists.xen.org
>    3. http://lists.xen.org/xen-devel

Re: Xen 4.3 release planning proposal

[Martinx - ジェームズ]

[-- Attachment #1.1: Type: text/plain, Size: 4533 bytes --]

Pasi,

 How can I take full advantage of a remote HVM DomU with a Radeon within
it, without SPICE?

 VNC is enough? Or there is something else better that I don't know?

Thank you!
Thiago

On 18 December 2012 05:03, Pasi Kärkkäinen <pasik@iki.fi> wrote:

> On Mon, Dec 17, 2012 at 09:57:59PM -0200, Martinx - ?$B%8%'!<%`%: wrote:
> >    Hi Pasi!
> >     Can you tell me if it will be possible to use Xen like this:
> >     dom0 -> ATI GPU Passthrough as primary -> HVM domU with Catalyst ->
> Spice
> >    -> Spice-Client ?
> >     I do not want to use Spice "alone" and, I do not want to use my domU
> with
> >    my ATI without SPICE...  That makes sense?
> >    Thanks!
>
> Afaik SPICE uses and requires the virtual QXL GPU for efficient operation,
> so it doesn't work with physical GPUs.
>
> -- Pasi
>
> >    Thiago
> >
> >    On 20 August 2012 16:14, Pasi Kärkkäinen <[1]pasik@iki.fi> wrote:
> >
> >      On Mon, Aug 20, 2012 at 05:46:59PM +0100, George Dunlap wrote:
> >      >
> >      > Features and improvements not on this list are of course welcome
> at
> >      > any time before the feature freeze.
> >      >
> >      > Any questions and feedback are welcome!
> >      >
> >      > Your 4.3 release coordinator,
> >      >  George Dunlap
> >      >
> >
> >      <snip>
> >      >
> >      > * xl USB pass-through for PV guests
> >      >   owner: ?
> >      >   Port the xend PV pass-through functionality to xl.
> >      >
> >
> >      xm/xend PVUSB works for both PV and HVM guests, so xl should support
> >      PVUSB for both PV and HVM guests aswell.
> >      James Harper's GPLPV drivers actually do have PVUSB frontend driver
> for
> >      Windows.
> >
> >      Also Suse's xenlinux forward-ported patches have PVUSB support in
> >      unmodified_drivers for HVM guests.
> >
> >      Another USB item:
> >
> >      * xl support for USB device passthru using QEMU emulated USB for HVM
> >      guests (no need for PVUSB drivers in the HVM guest).
> >        This works today in xm/xend with qemu-traditional, but is limited
> to
> >      USB 1.1, probably because
> >        the old version of Qemu-dm-traditional which lacks USB 2.0/3.0.
> >        So xl support for emulated USB device passthru for both
> qemu-upstream
> >      and qemu-traditional.
> >
> >      More wishlist items:
> >
> >      * Nested hardware virtualization. Important for easier testing and
> >      development of Xen (Xen-on-Xen),
> >        and for running other hypervisors in Xen VMs. Interesting for
> labs,
> >      POCs, etc.
> >
> >      * VGA/GPU passthru support for AMD/NVIDIA; lots of patches on
> xen-devel
> >      archives,
> >        but noone has yet stepped up to clean up and get them merged.
> >        Currently Intel gfx passthru patches are merged to Xen, but
> primary
> >      ATI/NVIDIA require extra patches.
> >        This is actually something that a LOT of users ask often, it's
> >      discussed almost every day on ##xen on IRC.
> >        I wonder if XenClient folks could help here?
> >
> >      * Dom0 Keyboard/mouse sharing to HVM guests; mainly needed by
> VGA/GPU
> >      passthru users.
> >        Fujitsu guys posted some patches for this in 2010, and XenClient
> guys
> >      in 2009 (iirc),
> >        but nothing got further developed and merged to upstream Xen.
> >
> >      * QXL virtual GPU support for SPICE. Someone was already developing
> >      this,
> >        and posted patches earlier during 4.2 development cycle to
> xen-devel.
> >        Upstream Qemu includes QXL support.
> >
> >      * PVSCSI support in XL. James Harper was (semi) interested in
> working
> >      with this,
> >        because he has a PVSCSI frontend driver in Windows GPLPV drivers,
> and
> >      he's using PVSCSI for tape backups himself.
> >
> >      * libvirt libxl driver improvements; support more Xen features.
> >        Allows better using the Ubuntu/Debian/Fedora/RHEL/CentOS "default"
> >      virtualization GUI also with Xen.
> >
> >      Hopefully we'll find interested developers for these items :)
> >
> >      -- Pasi
> >
> >      _______________________________________________
> >      Xen-devel mailing list
> >      [2]Xen-devel@lists.xen.org
> >      [3]http://lists.xen.org/xen-devel
> >
> > References
> >
> >    Visible links
> >    1. mailto:pasik@iki.fi
> >    2. mailto:Xen-devel@lists.xen.org
> >    3. http://lists.xen.org/xen-devel
>

[-- Attachment #1.2: Type: text/html, Size: 5814 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.3 release planning proposal

[Jan Beulich]

>>> On 18.12.12 at 14:37, Martinx - ジェームズ<thiagocmartinsc@gmail.com> wrote:
>  How can I take full advantage of a remote HVM DomU with a Radeon within
> it, without SPICE?
> 
>  VNC is enough? Or there is something else better that I don't know?

Please stop hijacking an unrelated thread.

Thanks, Jan

> On 18 December 2012 05:03, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> 
>> On Mon, Dec 17, 2012 at 09:57:59PM -0200, Martinx - ?$B%8%'!<%`%: wrote:
>> >    Hi Pasi!
>> >     Can you tell me if it will be possible to use Xen like this:
>> >     dom0 -> ATI GPU Passthrough as primary -> HVM domU with Catalyst ->
>> Spice
>> >    -> Spice-Client ?
>> >     I do not want to use Spice "alone" and, I do not want to use my domU
>> with
>> >    my ATI without SPICE...  That makes sense?
>> >    Thanks!
>>
>> Afaik SPICE uses and requires the virtual QXL GPU for efficient operation,
>> so it doesn't work with physical GPUs.
>>
>> -- Pasi
>>
>> >    Thiago
>> >
>> >    On 20 August 2012 16:14, Pasi Kärkkäinen <[1]pasik@iki.fi> wrote:
>> >
>> >      On Mon, Aug 20, 2012 at 05:46:59PM +0100, George Dunlap wrote:
>> >      >
>> >      > Features and improvements not on this list are of course welcome
>> at
>> >      > any time before the feature freeze.
>> >      >
>> >      > Any questions and feedback are welcome!
>> >      >
>> >      > Your 4.3 release coordinator,
>> >      >  George Dunlap
>> >      >
>> >
>> >      <snip>
>> >      >
>> >      > * xl USB pass-through for PV guests
>> >      >   owner: ?
>> >      >   Port the xend PV pass-through functionality to xl.
>> >      >
>> >
>> >      xm/xend PVUSB works for both PV and HVM guests, so xl should support
>> >      PVUSB for both PV and HVM guests aswell.
>> >      James Harper's GPLPV drivers actually do have PVUSB frontend driver
>> for
>> >      Windows.
>> >
>> >      Also Suse's xenlinux forward-ported patches have PVUSB support in
>> >      unmodified_drivers for HVM guests.
>> >
>> >      Another USB item:
>> >
>> >      * xl support for USB device passthru using QEMU emulated USB for HVM
>> >      guests (no need for PVUSB drivers in the HVM guest).
>> >        This works today in xm/xend with qemu-traditional, but is limited
>> to
>> >      USB 1.1, probably because
>> >        the old version of Qemu-dm-traditional which lacks USB 2.0/3.0.
>> >        So xl support for emulated USB device passthru for both
>> qemu-upstream
>> >      and qemu-traditional.
>> >
>> >      More wishlist items:
>> >
>> >      * Nested hardware virtualization. Important for easier testing and
>> >      development of Xen (Xen-on-Xen),
>> >        and for running other hypervisors in Xen VMs. Interesting for
>> labs,
>> >      POCs, etc.
>> >
>> >      * VGA/GPU passthru support for AMD/NVIDIA; lots of patches on
>> xen-devel
>> >      archives,
>> >        but noone has yet stepped up to clean up and get them merged.
>> >        Currently Intel gfx passthru patches are merged to Xen, but
>> primary
>> >      ATI/NVIDIA require extra patches.
>> >        This is actually something that a LOT of users ask often, it's
>> >      discussed almost every day on ##xen on IRC.
>> >        I wonder if XenClient folks could help here?
>> >
>> >      * Dom0 Keyboard/mouse sharing to HVM guests; mainly needed by
>> VGA/GPU
>> >      passthru users.
>> >        Fujitsu guys posted some patches for this in 2010, and XenClient
>> guys
>> >      in 2009 (iirc),
>> >        but nothing got further developed and merged to upstream Xen.
>> >
>> >      * QXL virtual GPU support for SPICE. Someone was already developing
>> >      this,
>> >        and posted patches earlier during 4.2 development cycle to
>> xen-devel.
>> >        Upstream Qemu includes QXL support.
>> >
>> >      * PVSCSI support in XL. James Harper was (semi) interested in
>> working
>> >      with this,
>> >        because he has a PVSCSI frontend driver in Windows GPLPV drivers,
>> and
>> >      he's using PVSCSI for tape backups himself.
>> >
>> >      * libvirt libxl driver improvements; support more Xen features.
>> >        Allows better using the Ubuntu/Debian/Fedora/RHEL/CentOS "default"
>> >      virtualization GUI also with Xen.
>> >
>> >      Hopefully we'll find interested developers for these items :)
>> >
>> >      -- Pasi
>> >
>> >      _______________________________________________
>> >      Xen-devel mailing list
>> >      [2]Xen-devel@lists.xen.org 
>> >      [3]http://lists.xen.org/xen-devel 
>> >
>> > References
>> >
>> >    Visible links
>> >    1. mailto:pasik@iki.fi 
>> >    2. mailto:Xen-devel@lists.xen.org 
>> >    3. http://lists.xen.org/xen-devel 
>>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.3 release planning proposal

[Pasi Kärkkäinen]

On Tue, Aug 21, 2012 at 07:50:53AM -0700, Andres Lagar-Cavilla wrote:
> >
> > Hello everyone!  With the completion of our first few release candidates
> > for 4.2, it's time to look forward and start planning for the 4.3
> > release.  I've volunteered to step up and help coordinate the release
> > for this cycle.
> Hi George. Great idea. Cutting to the chase below
> 
> >
> An observation: the three below really sound like xapi or libvirt tasks.
>

libxl probably needs to provide the necessary "hooks" so that snapshots 
can have synchronized cpu/mem/disk states.. or what do you think? 

.. and the necessary bits for being able to do storage live migration..
this is an often asked feature. 

If you have thoughts how to best implement these, let us know! 
It'd be good to have these available also in xl, not only on the higher level toolstacks.


> >
> > * Full-VM snapshotting
> >   owner: ?
> >   Have a way of coordinating the taking and restoring of VM memory and
> >   disk snapshots.  This would involve some investigation into the best
> >   way to accomplish this.
> >
> > * VM Cloning
> >   owner: ?
> >   Again, a way of coordinating the memory and disk aspects.  Research
> >   into the best way to do this would probably go along with the
> >   snapshotting feature.
> >
> > * Make storage migration possible
> >   owner: ?
> >   There needs to be a way, either via command-line or via some hooks,
> >   that someone can build a "storage migration" feature on top of libxl
> >   or xl.
> >


-- Pasi

Re: Xen 4.3 release planning proposal

[Andres Lagar-Cavilla]

>
> Hello everyone!  With the completion of our first few release candidates
> for 4.2, it's time to look forward and start planning for the 4.3
> release.  I've volunteered to step up and help coordinate the release
> for this cycle.
Hi George. Great idea. Cutting to the chase below

>
An observation: the three below really sound like xapi or libvirt tasks.
>
> * Full-VM snapshotting
>   owner: ?
>   Have a way of coordinating the taking and restoring of VM memory and
>   disk snapshots.  This would involve some investigation into the best
>   way to accomplish this.
>
> * VM Cloning
>   owner: ?
>   Again, a way of coordinating the memory and disk aspects.  Research
>   into the best way to do this would probably go along with the
>   snapshotting feature.
>
> * Make storage migration possible
>   owner: ?
>   There needs to be a way, either via command-line or via some hooks,
>   that someone can build a "storage migration" feature on top of libxl
>   or xl.
>
> * PV audio (audio for stubdom qemu)
>   owner: stefano.panella@citrix
>
> * Memory: Replace PoD with paging mechanism
>   owner: george@citrix
This is one visible tip of the paging iceberg.

More generally, we need full wait-queue support for gfn translation
resolution. Working on this might include any of the folks who touched x86
mm code during 4.2.

Due to wait-queue locking limitations we decided not to tackle in the 4.2
time-frame, the hypervisor is unable to put a vcpu on a wait-queue in
hypervisor context at will. This means that right now, gfn->mfn
translations don't automagically hide all fixup conditions that require
helper intervention (paged out, unshare enomem). Fixing this will make the
p2m code a lot more self-contained and easier to parse, paging will be
completely transparent to guests (right now you can crash a guest with a
skillfully chosen paged out gfn), and will help you towards your goal of
s/pod/paging/

Andres
>
> * Managed domains?