4.2 TODO update

4.2 TODO update

[Ian Campbell]

This update covers two weeks since I was away last week.

As usual please ping me with any updates/corrections. There are several
things below which are in need of a volunteer to take care of them...

hypervisor, blockers:
      * domctls / sysctls set up to modify scheduler parameters, like
        the credit1 timeslice and schedule rate. (George Dunlap, DONE)
 
tools, blockers:
      * libxl stable API -- we would like 4.2 to define a stable API
        which downstream's can start to rely on not changing. Aspects of
        this are:
              * add libxl_defbool and generally try and arrange that
                memset(foo,0,...) requests the defaults (Ian Campbell,
                DONE)
              * Safe fork vs. fd handling hooks. This is an API
                addition, so maybe not required fro stable API, bit need
                to have for 4.2? (Ian J, patches posted)
      * xl feature parity with xend wrt driver domain support (George
        Dunlap)
      * More formally deprecate xm/xend. Manpage patches already in
        tree. Needs release noting and communication around -rc1 to
        remind people to test xl.
      * Correct paging/sharing tools buffer mlocking (Tim, Andres, DONE)
      * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
      * Domain 0 block attach & general hotplug when using qdisk backend
        (need to start qemu as necessary etc)
      * file:// backend performance. qemu-xen-tradition's qdisk is quite
        slow & blktap2 not available in upstream kernels. Need to
        consider our options:
              * qemu-xen's qdisk is thought to be well performing but
                qemu-xen is not yet the default. Complexity arising from
                splitting qemu-for-qdisk out from qemu-for-dm and
                running N qemu's.
              * potentially fully userspace blktap could be ready for
                4.2
              * use /dev/loop+blkback. This requires loop driver AIO and
                O_DIRECT patches which are not (AFAIK) yet upstream.
              * Leverage XCP's blktap2 DKMS work.
              * Other ideas?

hypervisor, nice to have:
      * solid implementation of sharing/paging/mem-events (using work
        queues) (Tim Deegan, Olaf Herring et al -- is this happening? is
        it DONE even?)

tools, nice to have:
      * Hotplug script stuff -- internal to libxl (I think, therefore I
        didn't put this under stable API above) but still good to have
        for 4.2? (Roger Pau Monné, patches posted)
      * Block script support -- follows on from hotplug script (Roger
        Pau Monné)
      * Configure/control paging via xl/libxl (Olaf Herring, lots of
        discussion around interface)
      * Upstream qemu feature patches:
              * Upstream qemu PCI passthrough support (Anthony Perard,
                patches sent)
              * Upstream qemu save restore (Anthony Perard, Stefano
                Stabellini, patches sent, waiting for upstream ack)
      * Nested-virtualisation (currently should be marked experimental,
        likely to release that way? Consider nested-svm separate to
        nested-vmx. Nested-svm is in better shape)
      * Initial xl support for Remus (memory checkpoint, blackholing)
        (Shriram, patches posted, blocked behind qemu save restore
        patches)
      * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
        (Nobody AFAICT)
      * support for vif "rate" parameter (No one, AFAICT)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> [...] 
> tools, blockers:
> [...]
>       * Correct paging/sharing tools buffer mlocking (Tim, Andres, DONE)
> [...]
> hypervisor, nice to have:
>       * solid implementation of sharing/paging/mem-events (using work
>         queues) (Tim Deegan, Olaf Herring et al -- is this happening? is
>         it DONE even?)
> 
> tools, nice to have:
> [...]
>       * Configure/control paging via xl/libxl (Olaf Herring, lots of
>         discussion around interface)

Is this a reasonable summary of the state of the paging/sharing &
friends world for 4.2? Is there anything missing?

Ian.

Volunteers required for 4.2 TODO items (Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> There are several things below which are in need of a volunteer to take care of them...

Specifically:

> tools, blockers:
> [...]
>       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
>       * Domain 0 block attach & general hotplug when using qdisk backend
>         (need to start qemu as necessary etc)
> tools, nice to have:
> [...]
>       * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
>         (Nobody AFAICT)
>       * support for vif "rate" parameter (No one, AFAICT)

#1,#3 & #4 are reasonably simple libxl/xl features possibly suitable for
a newcomer who is interested in getting started with toolstack level
stuff.

#2 is a bit more involved.

Any takers?

Ian.

Re: Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Olaf Hering]

On Mon, Mar 12, Ian Campbell wrote:

> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> > [...] 
> > tools, blockers:
> > [...]
> >       * Correct paging/sharing tools buffer mlocking (Tim, Andres, DONE)
> > [...]
> > hypervisor, nice to have:
> >       * solid implementation of sharing/paging/mem-events (using work
> >         queues) (Tim Deegan, Olaf Herring et al -- is this happening? is
> >         it DONE even?)

The last patch to use a waitqueue in __get_gfn_type_access() from Tim
works.  However, there are a few users who call __get_gfn_type_access
with the domain_lock held. This part needs to be addressed in some way.

> > tools, nice to have:
> > [...]
> >       * Configure/control paging via xl/libxl (Olaf Herring, lots of
> >         discussion around interface)
> 
> Is this a reasonable summary of the state of the paging/sharing &
> friends world for 4.2? Is there anything missing?


There are indeed alot of ideas regarding the interface, and its
currently not clear to me if there is an agreement yet. Was there any
outcome where to proceed?

Olaf

Xen 4.2 release plan (Was: Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
[4.2 TODO list]

So the list is getting to be quite manageable. I think it is time we
started discussing plans for a 4.2, timelines etc.

As a starting point for discussion how about the following:

After next week's TODO list update (19 March) I start taking a much
harder line on what goes on the TODO list, especially the blocker list.
A strong case will have to be made for why any particular addition is
critical to Xen 4.2. This means that posting additions this week means
you will have a much greater chance of that item making it onto the list
for 4.2!

We then continue to focus on reducing the TODO list with a view to doing
a feature freeze on, lets say, 2 April (3 weeks from today, 2 weeks from
the "TODO list freeze").

At that point we become strict about not taking new features which are
not already on the TODO list, becoming increasingly strict WRT
nice-to-have vs blocker.

Aim to have -rc0 mid to late April, continuing weekly "Until Its
Ready(tm)", expecting the release itself to land in May or June.

Any thoughts? Too Aggressive? Too Pessimistic?

Ian.

Re: Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 13:04 +0000, Olaf Hering wrote:
> On Mon, Mar 12, Ian Campbell wrote:
> 
> > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> > > [...] 
> > > tools, blockers:
> > > [...]
> > >       * Correct paging/sharing tools buffer mlocking (Tim, Andres, DONE)
> > > [...]
> > > hypervisor, nice to have:
> > >       * solid implementation of sharing/paging/mem-events (using work
> > >         queues) (Tim Deegan, Olaf Herring et al -- is this happening? is
> > >         it DONE even?)
> 
> The last patch to use a waitqueue in __get_gfn_type_access() from Tim
> works.  However, there are a few users who call __get_gfn_type_access
> with the domain_lock held.

Thanks. 

>  This part needs to be addressed in some way.

This is happening?

> > > tools, nice to have:
> > > [...]
> > >       * Configure/control paging via xl/libxl (Olaf Herring, lots of
> > >         discussion around interface)
> > 
> > Is this a reasonable summary of the state of the paging/sharing &
> > friends world for 4.2? Is there anything missing?
> 
> 
> There are indeed alot of ideas regarding the interface, and its
> currently not clear to me if there is an agreement yet. Was there any
> outcome where to proceed?

I've mostly swapped it out while I was away so I need to go back to the
thread and figure out where we got to. I don't think we had discussed
the libxl interface, only xl. I think the proposal Andres made sense in
this respect when he described it to me (although I've not read the mail
yet).

Ian.

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> tools, nice to have:
>       * Hotplug script stuff -- internal to libxl (I think, therefore I
>         didn't put this under stable API above) but still good to have
>         for 4.2? (Roger Pau Monné, patches posted)
>       * Block script support -- follows on from hotplug script (Roger
>         Pau Monné)

Any opinions on whether this, and in particular the block script bits,
should be blockers for 4.2?

I don't use block-script functionality myself so I'm not sure how
critical it is.

I guess there is always the workaround of doing whatever the script
would do outside of xl which is why I think (IIRC) I initially put it
under nice to have.

Opinions?

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 release plan (Was: Re: 4.2 TODO update)

[Keir Fraser]

On 12/03/2012 13:37, "Ian Campbell" <Ian.Campbell@citrix.com> wrote:

> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> [4.2 TODO list]
> 
> So the list is getting to be quite manageable. I think it is time we
> started discussing plans for a 4.2, timelines etc.
> 
> As a starting point for discussion how about the following:
> 
> After next week's TODO list update (19 March) I start taking a much
> harder line on what goes on the TODO list, especially the blocker list.
> A strong case will have to be made for why any particular addition is
> critical to Xen 4.2. This means that posting additions this week means
> you will have a much greater chance of that item making it onto the list
> for 4.2!
> 
> We then continue to focus on reducing the TODO list with a view to doing
> a feature freeze on, lets say, 2 April (3 weeks from today, 2 weeks from
> the "TODO list freeze").
> 
> At that point we become strict about not taking new features which are
> not already on the TODO list, becoming increasingly strict WRT
> nice-to-have vs blocker.
> 
> Aim to have -rc0 mid to late April, continuing weekly "Until Its
> Ready(tm)", expecting the release itself to land in May or June.
> 
> Any thoughts? Too Aggressive? Too Pessimistic?

Sounds about right.

 -- Keir

> Ian.
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[Jan Beulich]

>>> On 12.03.12 at 14:42, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
>> tools, nice to have:
>>       * Hotplug script stuff -- internal to libxl (I think, therefore I
>>         didn't put this under stable API above) but still good to have
>>         for 4.2? (Roger Pau Monné, patches posted)
>>       * Block script support -- follows on from hotplug script (Roger
>>         Pau Monné)
> 
> Any opinions on whether this, and in particular the block script bits,
> should be blockers for 4.2?
> 
> I don't use block-script functionality myself so I'm not sure how
> critical it is.
> 
> I guess there is always the workaround of doing whatever the script
> would do outside of xl which is why I think (IIRC) I initially put it
> under nice to have.
> 
> Opinions?

If it's this feature that you referred to when discussing the "xl
block-attach 0 ..." shortcomings (including the blkback-over-loop
that's currently unavailable with xl), then count this as a "+1"
for this being a blocker - if we want 4.2 to push forward the
deprecation status of xend, then (known) regressions like this
should get eliminated.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[Roger Pau Monné]

2012/3/12 Ian Campbell <Ian.Campbell@citrix.com>:
> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
>> tools, nice to have:
>>       * Hotplug script stuff -- internal to libxl (I think, therefore I
>>         didn't put this under stable API above) but still good to have
>>         for 4.2? (Roger Pau Monné, patches posted)
>>       * Block script support -- follows on from hotplug script (Roger
>>         Pau Monné)
>
> Any opinions on whether this, and in particular the block script bits,
> should be blockers for 4.2?
>
> I don't use block-script functionality myself so I'm not sure how
> critical it is.
>
> I guess there is always the workaround of doing whatever the script
> would do outside of xl which is why I think (IIRC) I initially put it
> under nice to have.
>
> Opinions?

I would really like to be able to send the updated driver-domain
series (using Ian Jackson new event interface), but I'm not sure if I
will be able to do so until the next week, sorry.

> Ian.
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 release plan (Was: Re: 4.2 TODO update)

[Sander Eikelenboom]

Hello Ian,

Monday, March 12, 2012, 2:37:35 PM, you wrote:

> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> [4.2 TODO list]

> So the list is getting to be quite manageable. I think it is time we
> started discussing plans for a 4.2, timelines etc.

> As a starting point for discussion how about the following:

> After next week's TODO list update (19 March) I start taking a much
> harder line on what goes on the TODO list, especially the blocker list.
> A strong case will have to be made for why any particular addition is
> critical to Xen 4.2. This means that posting additions this week means
> you will have a much greater chance of that item making it onto the list
> for 4.2!

> We then continue to focus on reducing the TODO list with a view to doing
> a feature freeze on, lets say, 2 April (3 weeks from today, 2 weeks from
> the "TODO list freeze").

> At that point we become strict about not taking new features which are
> not already on the TODO list, becoming increasingly strict WRT
> nice-to-have vs blocker.

> Aim to have -rc0 mid to late April, continuing weekly "Until Its
> Ready(tm)", expecting the release itself to land in May or June.

> Any thoughts? Too Aggressive? Too Pessimistic?

> Ian.

I think there could be quite some fallout when users start testing the RC's (announce them well on xen-users ?), since there a quite some important changes (xl instead of xm) that don't have had such a wide spread testing yet.
I haven't tested xen-unstable myself since i'm a bit on a time constrain right now and some major changes are still in progress, but i'm planning on switching when the rc's are there.

--
Sander

Re: Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Andres Lagar-Cavilla]

> On Mon, Mar 12, Ian Campbell wrote:
>
>> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
>> > [...]
>> > tools, blockers:
>> > [...]
>> >       * Correct paging/sharing tools buffer mlocking (Tim, Andres,
>> DONE)
>> > [...]
>> > hypervisor, nice to have:
>> >       * solid implementation of sharing/paging/mem-events (using work
>> >         queues) (Tim Deegan, Olaf Herring et al -- is this happening?
>> is
>> >         it DONE even?)
>
> The last patch to use a waitqueue in __get_gfn_type_access() from Tim
> works.  However, there are a few users who call __get_gfn_type_access
> with the domain_lock held. This part needs to be addressed in some way.

So, Tim posted a 6-patch series
http://lists.xen.org/archives/html/xen-devel/2012-02/msg02133.html

The first four are fixes that should go in. I acked them pending some
nits. Tim, I can re-package them and submit them myself if that would save
you some time.

But, the core patch in that series, #5, breaks pv-on-hvm windows 7. So I'm
afraid it's a solid nack from my end for the time being. In fact, it would
break not just windows but any other os, should they start invoking XENMEM
hypercalls (and who knows what else is still lurking!) more liberally.

Apart from that I have 4 hypervisor patches that deal with some
correctness corner cases, so I'll float them out there in short order.

There is work going on with AMD to get all these features working on their
processors. Let's hope for a breakthrough, but if it doesn't happen, let's
keep going forward.

I cannot really think of anything else on the hypervisor side.
Andres

>
>> > tools, nice to have:
>> > [...]
>> >       * Configure/control paging via xl/libxl (Olaf Herring, lots of
>> >         discussion around interface)
>>
>> Is this a reasonable summary of the state of the paging/sharing &
>> friends world for 4.2? Is there anything missing?
>
>
> There are indeed alot of ideas regarding the interface, and its
> currently not clear to me if there is an agreement yet. Was there any
> outcome where to proceed?
>
> Olaf
>

Re: Xen 4.2 release plan (Was: Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 14:12 +0000, Sander Eikelenboom wrote:
> Hello Ian,
> 
> Monday, March 12, 2012, 2:37:35 PM, you wrote:
> 
> > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> > [4.2 TODO list]
> 
> > So the list is getting to be quite manageable. I think it is time we
> > started discussing plans for a 4.2, timelines etc.
> 
> > As a starting point for discussion how about the following:
> 
> > After next week's TODO list update (19 March) I start taking a much
> > harder line on what goes on the TODO list, especially the blocker list.
> > A strong case will have to be made for why any particular addition is
> > critical to Xen 4.2. This means that posting additions this week means
> > you will have a much greater chance of that item making it onto the list
> > for 4.2!
> 
> > We then continue to focus on reducing the TODO list with a view to doing
> > a feature freeze on, lets say, 2 April (3 weeks from today, 2 weeks from
> > the "TODO list freeze").
> 
> > At that point we become strict about not taking new features which are
> > not already on the TODO list, becoming increasingly strict WRT
> > nice-to-have vs blocker.
> 
> > Aim to have -rc0 mid to late April, continuing weekly "Until Its
> > Ready(tm)", expecting the release itself to land in May or June.
> 
> > Any thoughts? Too Aggressive? Too Pessimistic?
> 
> > Ian.
> 
> I think there could be quite some fallout when users start testing the
> RC's (announce them well on xen-users ?), since there a quite some
> important changes (xl instead of xm) that don't have had such a wide
> spread testing yet.

Right, I think this is where the "Until Its Ready" bit comes in ;-0

> I haven't tested xen-unstable myself since i'm a bit on a time
> constrain right now and some major changes are still in progress, but
> i'm planning on switching when the rc's are there.

Great!

Ian.

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-03-12 at 13:51 +0000, Jan Beulich wrote:
> >>> On 12.03.12 at 14:42, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> >> tools, nice to have:
> >>       * Hotplug script stuff -- internal to libxl (I think, therefore I
> >>         didn't put this under stable API above) but still good to have
> >>         for 4.2? (Roger Pau Monné, patches posted)
> >>       * Block script support -- follows on from hotplug script (Roger
> >>         Pau Monné)
> > 
> > Any opinions on whether this, and in particular the block script bits,
> > should be blockers for 4.2?
> > 
> > I don't use block-script functionality myself so I'm not sure how
> > critical it is.
> > 
> > I guess there is always the workaround of doing whatever the script
> > would do outside of xl which is why I think (IIRC) I initially put it
> > under nice to have.
> > 
> > Opinions?
> 
> If it's this feature that you referred to when discussing the "xl
> block-attach 0 ..." shortcomings (including the blkback-over-loop
> that's currently unavailable with xl), then count this as a "+1"
> for this being a blocker - if we want 4.2 to push forward the
> deprecation status of xend, then (known) regressions like this
> should get eliminated.

I actually included the specific use case of block-attach vs qdisk on
the TODO in its own right and separately the handling of file:// too --
it turns out that loopback may not be the best answer (see Joseph
Glanville's posts about loop and AIO/O_DIRECT) so block script may not
actually be all that helpful there.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Volunteers required for 4.2 TODO items (Re: 4.2 TODO update)

[Mathieu Gagné]

On 3/12/12 8:22 AM, Ian Campbell wrote:
> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
>> There are several things below which are in need of a volunteer to take care of them...
>
> Specifically:
>
>> tools, nice to have:
>>        * support for vif "rate" parameter (No one, AFAICT)
>

I'm working on a patch to add support for vif rate limiting in libxl.

I don't have much experience with C and my knowledge is limited. 
(although I develop in other languages)

At this moment, I only have a "proof of concept" with a main function so 
I can play and do tests to verify the parsing/results. 
(bytes_per_interval,interval_usecs)

I would have a couple of questions about how to integrate the 
feature/code within libxl/xl.

The rate syntax requires relatively complex parsing which would require 
at least 3 new functions which could all be combined in one if required. 
(parse_vif_rate, parse_vif_rate_bytes_per_sec and 
parse_vif_rate_interval_usecs)

Where should those functions be added? xl_cmdimpl.c?

What should be the names of the new struct members of libxl_device_nic? 
I same some "suggestions" in the previous rejected patch [1]. Are those 
names good? Should they be closer to the concept of "rate" instead of "qos"?

Also, should more information be provided to the user about the rate 
syntax? rate=<rate> isn't that useful.

Thanks!

[1] http://lists.xen.org/archives/html/xen-devel/2011-03/msg01963.html


-- 
Mathieu

Re: Volunteers required for 4.2 TODO items (Re: 4.2 TODO update)

[Lin Ming]

On Mon, Mar 12, 2012 at 8:22 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
>> There are several things below which are in need of a volunteer to take care of them...

Hi,

I'm a newcomer.
Could you share more details about the requirements?

>
> Specifically:
>
>> tools, blockers:
>> [...]
>>       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)

Are these parameters to be added for some xl subcommand?
What are they used for?

>>       * Domain 0 block attach & general hotplug when using qdisk backend
>>         (need to start qemu as necessary etc)
>> tools, nice to have:
>> [...]
>>       * xl support for autospawning vncviewer (vncviewer=1 or otherwise)

Could you tell more detail?

>>         (Nobody AFAICT)
>>       * support for vif "rate" parameter (No one, AFAICT)

It's a kernel module parameter(xen-4.1.2/tools/vnet/vnet-module), right?

>
> #1,#3 & #4 are reasonably simple libxl/xl features possibly suitable for
> a newcomer who is interested in getting started with toolstack level
> stuff.
>
> #2 is a bit more involved.
>
> Any takers?

I'm a new comer and I'd like to get start from here.

Thanks,
Lin Ming

>
> Ian.

Re: 4.2 TODO update

[Stefano Stabellini]

On Mon, 12 Mar 2012, Ian Campbell wrote:
> This update covers two weeks since I was away last week.
> 
> As usual please ping me with any updates/corrections. There are several
> things below which are in need of a volunteer to take care of them...
> 
> hypervisor, blockers:
>       * domctls / sysctls set up to modify scheduler parameters, like
>         the credit1 timeslice and schedule rate. (George Dunlap, DONE)
>  
> tools, blockers:
>       * libxl stable API -- we would like 4.2 to define a stable API
>         which downstream's can start to rely on not changing. Aspects of
>         this are:
>               * add libxl_defbool and generally try and arrange that
>                 memset(foo,0,...) requests the defaults (Ian Campbell,
>                 DONE)
>               * Safe fork vs. fd handling hooks. This is an API
>                 addition, so maybe not required fro stable API, bit need
>                 to have for 4.2? (Ian J, patches posted)
>       * xl feature parity with xend wrt driver domain support (George
>         Dunlap)
>       * More formally deprecate xm/xend. Manpage patches already in
>         tree. Needs release noting and communication around -rc1 to
>         remind people to test xl.
>       * Correct paging/sharing tools buffer mlocking (Tim, Andres, DONE)
>       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
>       * Domain 0 block attach & general hotplug when using qdisk backend
>         (need to start qemu as necessary etc)

I should be able to look into this

>       * file:// backend performance. qemu-xen-tradition's qdisk is quite
>         slow & blktap2 not available in upstream kernels. Need to
>         consider our options:
>               * qemu-xen's qdisk is thought to be well performing but
>                 qemu-xen is not yet the default. Complexity arising from
>                 splitting qemu-for-qdisk out from qemu-for-dm and
>                 running N qemu's.
>               * potentially fully userspace blktap could be ready for
>                 4.2
>               * use /dev/loop+blkback. This requires loop driver AIO and
>                 O_DIRECT patches which are not (AFAIK) yet upstream.
>               * Leverage XCP's blktap2 DKMS work.
>               * Other ideas?

Considering how long these patches have been outstanding (years), it
might be wise to design a solution that does not require them.

In particular I think we should either go with userblktap or qdisk.
Qdisk could be a nice fallback, not too hard to code, if we don't
get userblktap in time.

Re: Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Tim Deegan]

At 08:05 -0700 on 12 Mar (1331539511), Andres Lagar-Cavilla wrote:
> So, Tim posted a 6-patch series
> http://lists.xen.org/archives/html/xen-devel/2012-02/msg02133.html
> 
> The first four are fixes that should go in. I acked them pending some
> nits. Tim, I can re-package them and submit them myself if that would save
> you some time.
> 
> But, the core patch in that series, #5, breaks pv-on-hvm windows 7. So I'm
> afraid it's a solid nack from my end for the time being. In fact, it would
> break not just windows but any other os, should they start invoking XENMEM
> hypercalls (and who knows what else is still lurking!) more liberally.

Agreed.  I'll repost them this week, broken into the preliminary fixes
and the wait-queue change itself (which I think has to wait until after
4.2 now). 

> Apart from that I have 4 hypervisor patches that deal with some
> correctness corner cases, so I'll float them out there in short order.
> 
> There is work going on with AMD to get all these features working on their
> processors. Let's hope for a breakthrough, but if it doesn't happen, let's
> keep going forward.

Yep - Can AMD support go in as a separate "nice to have" on the list?

Tim.

Re: Volunteers required for 4.2 TODO items (Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 16:00 +0000, Mathieu Gagné wrote:
> On 3/12/12 8:22 AM, Ian Campbell wrote:
> > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> >> There are several things below which are in need of a volunteer to take care of them...
> >
> > Specifically:
> >
> >> tools, nice to have:
> >>        * support for vif "rate" parameter (No one, AFAICT)
> >
> 
> I'm working on a patch to add support for vif rate limiting in libxl.
> 
> I don't have much experience with C and my knowledge is limited. 
> (although I develop in other languages)
> 
> At this moment, I only have a "proof of concept" with a main function so 
> I can play and do tests to verify the parsing/results. 
> (bytes_per_interval,interval_usecs)
> 
> I would have a couple of questions about how to integrate the 
> feature/code within libxl/xl.
> 
> The rate syntax

Are you copying the xm/xend syntax here?

>  requires relatively complex parsing which would require 
> at least 3 new functions which could all be combined in one if required. 
> (parse_vif_rate, parse_vif_rate_bytes_per_sec

Aren't these the same function but with handling for units?

> and parse_vif_rate_interval_usecs)

I think externally a single function to parse a rate and initialise the
relevant fields of libxl_device_nic is the way to go. Perhaps internal
to the implementation of that there might be additional helpers etc.

> Where should those functions be added? xl_cmdimpl.c?

How about libxlu_vif.c (new file, similar to e.g. libxlu_disk.c)? libxlu
is a library of stuff which is part of xl which might be useful to other
toolstacks -- code for parsing vif rate's seems to fall into that
category, much like parsing disk configure strings does.

> What should be the names of the new struct members of libxl_device_nic? 
> I same some "suggestions" in the previous rejected patch [1]. Are those 
> names good? Should they be closer to the concept of "rate" instead of "qos"?

I suppose it depends what their actual semantics are. Something like
max_<unit>_per_<thing> and <thing>_per_second/<thing>_<time-unit> should
be ok I guess?

> Also, should more information be provided to the user about the rate 
> syntax? rate=<rate> isn't that useful.

The new option/syntax should be fully documented in
docs/misc/xl-network-configuration.markdown which is referenced by
docs/man/xl.cfg.pod.5.

> Thanks!

Thanks for doing this -- looking forward to the patches!

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[George Dunlap]

On Mon, Mar 12, 2012 at 12:11 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> This update covers two weeks since I was away last week.
>
> As usual please ping me with any updates/corrections. There are several
> things below which are in need of a volunteer to take care of them...
>
> hypervisor, blockers:
>      * domctls / sysctls set up to modify scheduler parameters, like
>        the credit1 timeslice and schedule rate. (George Dunlap, DONE)

I recently discovered a set of patches updating the PoD code in the
XenServer patchqueue which I was meant to have upstreamed but never
actually got to.  Can I add this to the "blockers" list?  It involves
changes based on some experimentation; and while I think it may be a
few days worth of work to port (with all the p2m reorganization done
after the 4.1 release), but it should be a relatively known quantity.

>
> tools, blockers:
>      * libxl stable API -- we would like 4.2 to define a stable API
>        which downstream's can start to rely on not changing. Aspects of
>        this are:
>              * add libxl_defbool and generally try and arrange that
>                memset(foo,0,...) requests the defaults (Ian Campbell,
>                DONE)
>              * Safe fork vs. fd handling hooks. This is an API
>                addition, so maybe not required fro stable API, bit need
>                to have for 4.2? (Ian J, patches posted)
>      * xl feature parity with xend wrt driver domain support (George
>        Dunlap)
>      * More formally deprecate xm/xend. Manpage patches already in
>        tree. Needs release noting and communication around -rc1 to
>        remind people to test xl.
>      * Correct paging/sharing tools buffer mlocking (Tim, Andres, DONE)
>      * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
>      * Domain 0 block attach & general hotplug when using qdisk backend
>        (need to start qemu as necessary etc)
>      * file:// backend performance. qemu-xen-tradition's qdisk is quite
>        slow & blktap2 not available in upstream kernels. Need to
>        consider our options:
>              * qemu-xen's qdisk is thought to be well performing but
>                qemu-xen is not yet the default. Complexity arising from
>                splitting qemu-for-qdisk out from qemu-for-dm and
>                running N qemu's.
>              * potentially fully userspace blktap could be ready for
>                4.2
>              * use /dev/loop+blkback. This requires loop driver AIO and
>                O_DIRECT patches which are not (AFAIK) yet upstream.
>              * Leverage XCP's blktap2 DKMS work.
>              * Other ideas?
>
> hypervisor, nice to have:
>      * solid implementation of sharing/paging/mem-events (using work
>        queues) (Tim Deegan, Olaf Herring et al -- is this happening? is
>        it DONE even?)
>
> tools, nice to have:
>      * Hotplug script stuff -- internal to libxl (I think, therefore I
>        didn't put this under stable API above) but still good to have
>        for 4.2? (Roger Pau Monné, patches posted)
>      * Block script support -- follows on from hotplug script (Roger
>        Pau Monné)
>      * Configure/control paging via xl/libxl (Olaf Herring, lots of
>        discussion around interface)
>      * Upstream qemu feature patches:
>              * Upstream qemu PCI passthrough support (Anthony Perard,
>                patches sent)
>              * Upstream qemu save restore (Anthony Perard, Stefano
>                Stabellini, patches sent, waiting for upstream ack)
>      * Nested-virtualisation (currently should be marked experimental,
>        likely to release that way? Consider nested-svm separate to
>        nested-vmx. Nested-svm is in better shape)
>      * Initial xl support for Remus (memory checkpoint, blackholing)
>        (Shriram, patches posted, blocked behind qemu save restore
>        patches)
>      * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
>        (Nobody AFAICT)
>      * support for vif "rate" parameter (No one, AFAICT)
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Volunteers required for 4.2 TODO items (Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 16:00 +0000, Lin Ming wrote:
> On Mon, Mar 12, 2012 at 8:22 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> >> There are several things below which are in need of a volunteer to take care of them...
> 
> Hi,
> 
> I'm a newcomer.
> Could you share more details about the requirements?
> 
> >
> > Specifically:
> >
> >> tools, blockers:
> >> [...]
> >>       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
> 
> Are these parameters to be added for some xl subcommand?
> What are they used for?

These should be options which can be specified in the domain
configuration file for HVM guests. They control the offset of the
guest-visible emulated RTC from the host time -- IOW they control the
time which the guest sees.

These options are supported by xm/xend and need to be implemented for xl
in a compatible way. This means exposing suitable fields in the libxl
API (probably as fields in libxl_domain_build_info) and arranging for xl
to parse the configuration file options.

Some amount of reverse engineering of xend will be required to figure
out what the actual meaning of those options in order to implement in a
compatible way but AFAIK rtc_timeoffset is the offset between host time
and guest time. localtime I think means to specify whether the emulted
RTC appears as UTC or is offset by the host.

Having figured out the meaning libxl needs to arrange to call
xc_domain_set_time_offset with the right values.

I think much of the related xend code is in
tools/python/xen/xend/image.py but you might need to grep about a bit.

> >>       * Domain 0 block attach & general hotplug when using qdisk backend
> >>         (need to start qemu as necessary etc)
> >> tools, nice to have:
> >> [...]
> >>       * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
> 
> Could you tell more detail?

xm/xend supports a configuration file option "vncviewer=1" which causes
xend to automatically spawn vncviewer with the correct options (e.g.
port number) when doing "xm create". We would like the same for "xl
create".

As well as a config file option I think a command line option to "xl
create" would also be useful, similar to '-c' which automatically
connects to the virtual serial console. e.g '-V' perhaps.

I guess we would want a libxl interface similar to
libxl_primary_console_exec and xl support to call it when appropriate.
overall it should look a lot like the console spawning stuff.

> >>         (Nobody AFAICT)
> >>       * support for vif "rate" parameter (No one, AFAICT)
> 
> It's a kernel module parameter(xen-4.1.2/tools/vnet/vnet-module), right?

I don't think so, this is network rate limiting as implemented by
netback. Mathieu Gagné is already taking care of this so I won't go into
any further detail (which is lucky, because I'm fuzzy about the details
myself).

> 
> >
> > #1,#3 & #4 are reasonably simple libxl/xl features possibly suitable for
> > a newcomer who is interested in getting started with toolstack level
> > stuff.
> >
> > #2 is a bit more involved.
> >
> > Any takers?
> 
> I'm a new comer and I'd like to get start from here.

Please do let us know if you decide to take on one of the tasks.

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-03-12 at 16:36 +0000, George Dunlap wrote:
> On Mon, Mar 12, 2012 at 12:11 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > This update covers two weeks since I was away last week.
> >
> > As usual please ping me with any updates/corrections. There are several
> > things below which are in need of a volunteer to take care of them...
> >
> > hypervisor, blockers:
> >      * domctls / sysctls set up to modify scheduler parameters, like
> >        the credit1 timeslice and schedule rate. (George Dunlap, DONE)
> 
> I recently discovered a set of patches updating the PoD code in the
> XenServer patchqueue which I was meant to have upstreamed but never
> actually got to.  Can I add this to the "blockers" list?

If we've been living without them are they really blockers? Adding them
to the nice to have list seems like a no brainer but a few words on why
they are blockers could easily upgrade my assessment ;-)

Ian.

Re: Volunteers required for 4.2 TODO items (Re: 4.2 TODO update)

[Lin Ming]

On Tue, Mar 13, 2012 at 12:40 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-03-12 at 16:00 +0000, Lin Ming wrote:
>> On Mon, Mar 12, 2012 at 8:22 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
>> >> There are several things below which are in need of a volunteer to take care of them...
>>
>> Hi,
>>
>> I'm a newcomer.
>> Could you share more details about the requirements?
>>
>> >
>> > Specifically:
>> >
>> >> tools, blockers:
>> >> [...]
>> >>       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
>>
>> Are these parameters to be added for some xl subcommand?
>> What are they used for?
>
> These should be options which can be specified in the domain
> configuration file for HVM guests. They control the offset of the
> guest-visible emulated RTC from the host time -- IOW they control the
> time which the guest sees.
>
> These options are supported by xm/xend and need to be implemented for xl
> in a compatible way. This means exposing suitable fields in the libxl
> API (probably as fields in libxl_domain_build_info) and arranging for xl
> to parse the configuration file options.
>
> Some amount of reverse engineering of xend will be required to figure
> out what the actual meaning of those options in order to implement in a
> compatible way but AFAIK rtc_timeoffset is the offset between host time
> and guest time. localtime I think means to specify whether the emulted
> RTC appears as UTC or is offset by the host.
>
> Having figured out the meaning libxl needs to arrange to call
> xc_domain_set_time_offset with the right values.
>
> I think much of the related xend code is in
> tools/python/xen/xend/image.py but you might need to grep about a bit.
>
>> >>       * Domain 0 block attach & general hotplug when using qdisk backend
>> >>         (need to start qemu as necessary etc)
>> >> tools, nice to have:
>> >> [...]
>> >>       * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
>>
>> Could you tell more detail?
>
> xm/xend supports a configuration file option "vncviewer=1" which causes
> xend to automatically spawn vncviewer with the correct options (e.g.
> port number) when doing "xm create". We would like the same for "xl
> create".
>
> As well as a config file option I think a command line option to "xl
> create" would also be useful, similar to '-c' which automatically
> connects to the virtual serial console. e.g '-V' perhaps.
>
> I guess we would want a libxl interface similar to
> libxl_primary_console_exec and xl support to call it when appropriate.
> overall it should look a lot like the console spawning stuff.
>
>> >>         (Nobody AFAICT)
>> >>       * support for vif "rate" parameter (No one, AFAICT)
>>
>> It's a kernel module parameter(xen-4.1.2/tools/vnet/vnet-module), right?
>
> I don't think so, this is network rate limiting as implemented by
> netback. Mathieu Gagné is already taking care of this so I won't go into
> any further detail (which is lucky, because I'm fuzzy about the details
> myself).
>
>>
>> >
>> > #1,#3 & #4 are reasonably simple libxl/xl features possibly suitable for
>> > a newcomer who is interested in getting started with toolstack level
>> > stuff.
>> >
>> > #2 is a bit more involved.
>> >
>> > Any takers?
>>
>> I'm a new comer and I'd like to get start from here.
>
> Please do let us know if you decide to take on one of the tasks.

Thanks for the detail explanation.
I take #1.

Do we have a deadline for the patch?
I'll need time to ramp up the code.

Thanks,
Lin Ming

>
> Ian.

Re: Volunteers required for 4.2 TODO items (Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 16:59 +0000, Lin Ming wrote:
> On Tue, Mar 13, 2012 at 12:40 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Mon, 2012-03-12 at 16:00 +0000, Lin Ming wrote:
> >> On Mon, Mar 12, 2012 at 8:22 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >> > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> >> >> There are several things below which are in need of a volunteer to take care of them...
> >>
> >> Hi,
> >>
> >> I'm a newcomer.
> >> Could you share more details about the requirements?
> >>
> >> >
> >> > Specifically:
> >> >
> >> >> tools, blockers:
> >> >> [...]
> >> >>       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
> >>
> >> Are these parameters to be added for some xl subcommand?
> >> What are they used for?
> >
> > These should be options which can be specified in the domain
> > configuration file for HVM guests. They control the offset of the
> > guest-visible emulated RTC from the host time -- IOW they control the
> > time which the guest sees.
> >
> > These options are supported by xm/xend and need to be implemented for xl
> > in a compatible way. This means exposing suitable fields in the libxl
> > API (probably as fields in libxl_domain_build_info) and arranging for xl
> > to parse the configuration file options.
> >
> > Some amount of reverse engineering of xend will be required to figure
> > out what the actual meaning of those options in order to implement in a
> > compatible way but AFAIK rtc_timeoffset is the offset between host time
> > and guest time. localtime I think means to specify whether the emulted
> > RTC appears as UTC or is offset by the host.
> >
> > Having figured out the meaning libxl needs to arrange to call
> > xc_domain_set_time_offset with the right values.
> >
> > I think much of the related xend code is in
> > tools/python/xen/xend/image.py but you might need to grep about a bit.
> >
> >> >>       * Domain 0 block attach & general hotplug when using qdisk backend
> >> >>         (need to start qemu as necessary etc)
> >> >> tools, nice to have:
> >> >> [...]
> >> >>       * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
> >>
> >> Could you tell more detail?
> >
> > xm/xend supports a configuration file option "vncviewer=1" which causes
> > xend to automatically spawn vncviewer with the correct options (e.g.
> > port number) when doing "xm create". We would like the same for "xl
> > create".
> >
> > As well as a config file option I think a command line option to "xl
> > create" would also be useful, similar to '-c' which automatically
> > connects to the virtual serial console. e.g '-V' perhaps.
> >
> > I guess we would want a libxl interface similar to
> > libxl_primary_console_exec and xl support to call it when appropriate.
> > overall it should look a lot like the console spawning stuff.
> >
> >> >>         (Nobody AFAICT)
> >> >>       * support for vif "rate" parameter (No one, AFAICT)
> >>
> >> It's a kernel module parameter(xen-4.1.2/tools/vnet/vnet-module), right?
> >
> > I don't think so, this is network rate limiting as implemented by
> > netback. Mathieu Gagné is already taking care of this so I won't go into
> > any further detail (which is lucky, because I'm fuzzy about the details
> > myself).
> >
> >>
> >> >
> >> > #1,#3 & #4 are reasonably simple libxl/xl features possibly suitable for
> >> > a newcomer who is interested in getting started with toolstack level
> >> > stuff.
> >> >
> >> > #2 is a bit more involved.
> >> >
> >> > Any takers?
> >>
> >> I'm a new comer and I'd like to get start from here.
> >
> > Please do let us know if you decide to take on one of the tasks.
> 
> Thanks for the detail explanation.
> I take #1.

Great!

> Do we have a deadline for the patch?

See the "Xen 4.2 release plan" email which I sent to the list this
morning for a broad outline

Since these are in the "nice to have" category and are pretty small and
self contained I think that even if they miss 4.2.0 they can go in to
unstable in the 4.3 dev cycle and be obvious candidates for backporting
to 4.2.1.

Ian.

> I'll need time to ramp up the code.
> 
> Thanks,
> Lin Ming
> 
> >
> > Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Volunteers required for 4.2 TODO items (Re: 4.2 TODO update)

[Goncalo Gomes]

On Mon, 12 Mar 2012, Ian Campbell wrote:

> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> > There are several things below which are in need of a volunteer to take care of them...
> 
> Specifically:
> 
> > tools, blockers:
> > [...]
> >       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
> >       * Domain 0 block attach & general hotplug when using qdisk backend
> >         (need to start qemu as necessary etc)
> > tools, nice to have:
> > [...]
> >       * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
> >         (Nobody AFAICT)
> >       * support for vif "rate" parameter (No one, AFAICT)
> 
> #1,#3 & #4 are reasonably simple libxl/xl features possibly suitable for
> a newcomer who is interested in getting started with toolstack level
> stuff.
> 
> #2 is a bit more involved.
> 
> Any takers?

I'd be keen to work on 

> >       * Domain 0 block attach & general hotplug when using qdisk backend
> >         (need to start qemu as necessary etc)

Cheers,
Goncalo

 
> Ian.
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-03-12 at 16:01 +0000, Stefano Stabellini wrote:
> On Mon, 12 Mar 2012, Ian Campbell wrote:
> >>       * Domain 0 block attach & general hotplug when using qdisk backend
> >         (need to start qemu as necessary etc)
> 
> I should be able to look into this

Thanks!

Ian

Re: 4.2 TODO update

[George Dunlap]

On Mon, Mar 12, 2012 at 4:42 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-03-12 at 16:36 +0000, George Dunlap wrote:
>> On Mon, Mar 12, 2012 at 12:11 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > This update covers two weeks since I was away last week.
>> >
>> > As usual please ping me with any updates/corrections. There are several
>> > things below which are in need of a volunteer to take care of them...
>> >
>> > hypervisor, blockers:
>> >      * domctls / sysctls set up to modify scheduler parameters, like
>> >        the credit1 timeslice and schedule rate. (George Dunlap, DONE)
>>
>> I recently discovered a set of patches updating the PoD code in the
>> XenServer patchqueue which I was meant to have upstreamed but never
>> actually got to.  Can I add this to the "blockers" list?
>
> If we've been living without them are they really blockers? Adding them
> to the nice to have list seems like a no brainer but a few words on why
> they are blockers could easily upgrade my assessment ;-)

As you wish; I think it is the case that at the moment they're just
for performance, and they're just for booting a VM anyway.  But it's
certainly *my* goal to get them out before 4.2. :-)

 -George

Re: Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Olaf Hering]

On Mon, Mar 12, Ian Campbell wrote:

> On Mon, 2012-03-12 at 13:04 +0000, Olaf Hering wrote:
> > On Mon, Mar 12, Ian Campbell wrote:
> > 
> > > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> > > > hypervisor, nice to have:
> > > >       * solid implementation of sharing/paging/mem-events (using work
> > > >         queues) (Tim Deegan, Olaf Herring et al -- is this happening? is
> > > >         it DONE even?)
> > 
> > The last patch to use a waitqueue in __get_gfn_type_access() from Tim
> > works.  However, there are a few users who call __get_gfn_type_access
> > with the domain_lock held.
> 
> Thanks. 
> 
> >  This part needs to be addressed in some way.
> 
> This is happening?

To me its not clear what the domain_lock() protects:

Just a guest: XENMEM_remove_from_physmap should be ok with the gfn_lock
taken in guest_physmap_remove_page(), so the domain_lock() could be
removed there.

I'm not sure about the domain_lock in xenmem_add_to_physmap_once() and
emulate_privileged_op().


> > > > tools, nice to have:
> > > > [...]
> > > >       * Configure/control paging via xl/libxl (Olaf Herring, lots of
> > > >         discussion around interface)
> > > 
> > > Is this a reasonable summary of the state of the paging/sharing &
> > > friends world for 4.2? Is there anything missing?
> > 
> > There are indeed alot of ideas regarding the interface, and its
> > currently not clear to me if there is an agreement yet. Was there any
> > outcome where to proceed?
> 
> I've mostly swapped it out while I was away so I need to go back to the
> thread and figure out where we got to. I don't think we had discussed
> the libxl interface, only xl. I think the proposal Andres made sense in
> this respect when he described it to me (although I've not read the mail
> yet).

So is this something we should work out for the 4.2 release?

If I understand it right, there should be a .cfg option
mem_policy=<string>, where string could be "paging" or "balloon".
In case of balloon the target value would be whats in use now
(memory+maxmem). In case of paging these values could be reused in the
same way, just that maxmem+memory would not create a PoD guest.
If mem_policy= is not given it will default to balloon to retain current
behaviour.


Olaf

Re: Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Ian Campbell]

On Tue, 2012-03-13 at 10:54 +0000, Olaf Hering wrote:
> On Mon, Mar 12, Ian Campbell wrote:
> > > > > tools, nice to have:
> > > > > [...]
> > > > >       * Configure/control paging via xl/libxl (Olaf Herring, lots of
> > > > >         discussion around interface)
> > > > 
> > > > Is this a reasonable summary of the state of the paging/sharing &
> > > > friends world for 4.2? Is there anything missing?
> > > 
> > > There are indeed alot of ideas regarding the interface, and its
> > > currently not clear to me if there is an agreement yet. Was there any
> > > outcome where to proceed?
> > 
> > I've mostly swapped it out while I was away so I need to go back to the
> > thread and figure out where we got to. I don't think we had discussed
> > the libxl interface, only xl. I think the proposal Andres made sense in
> > this respect when he described it to me (although I've not read the mail
> > yet).
> 
> So is this something we should work out for the 4.2 release?

I think it was on the nice to have list which I think is an accurate
place.

> If I understand it right, there should be a .cfg option
> mem_policy=<string>, where string could be "paging" or "balloon".

The string names the "actor" which will implement the policy (so perhaps
the cfg option name should be "mem_actor="? Seems clumsy). So the
default for xl would be "xl". An existing alternative actor would be
"squeezed" which should cause the system to use XCP's squeezed (this
would require updates to squeezed to actually use these interfaces). You
can imagine that others might want to implement other more complex
actors in the future (e.g. which combine sharing, paging, and tmem in an
interesting way).

The "xl" actor should implement the "paging=auto" balloon, delay, then
page mechanism (or just ballooning for PV guests) we discussed
previously (I think most recent proposal was in
<1330078304.8557.157.camel@zakaz.uk.xensource.com>),
iff /local/domain/X/memory-policy/actor == "xl". We can ignore sharing
with this new scheme and leave it to whomever implements the sharing
memory policy actor.

I expect that the xl daemon would be the entity which actually
implements this actor for the guest which it is monitoring. We should
probably supply a helpful libxl event triggered when the policy
parameters (under /local/domain/X/memory-policy/) change.

I suppose there is also scope for an actor specific arbitrary string,
e.g. mem_policy_args or so.

I think that in the normal case we would not support mixing and matching
actors on a system, so in the case of xl I would expect to normally find
mem_policy in /etc/xen/xl.conf rather than in the guest configuration
file. It is reasonable for an actor implementation to consist either a
per-host daemon (like squeezed) or a per-domain daemon (like xl).

libxl_set_memory_target would be changed to set the
"/local/domain/X/memory-policy/target", this is used by "xl mem-set".

libxl should also expose methods to set the balloon and paging targets,
these would be used by the code in xl which implements the "xl" policy
described above.

> In case of balloon the target value would be whats in use now
> (memory+maxmem). In case of paging these values could be reused in the
> same way, just that maxmem+memory would not create a PoD guest.
> If mem_policy= is not given it will default to balloon to retain current
> behaviour.

I think the libxl default should be to immediately set the balloon
target. This would retain the historical behaviour for toolstacks which
don't say differently and would also work as expected for dom0 (which
may not have the necessary /local/domain/X/memory-policy/actor key).

The default set by xl should be "xl" or whatever is provided in the
config.

The other option for the default provided by libxl will be to do nothing
I don't think that is as helpful/useful as a default though.

There should probably be an option to set the actor to "none", meaning
that the toolstack is taking direct responsibility for this domains
memory targets. This would be used when "xl mem-paging-set domain
manual" was called allowing xl to implement the "xl mem-paging-set
domain N" in manual mode as described in
<1330078304.8557.157.camel@zakaz.uk.xensource.com>. Or maybe this
corresponds to using "xl-auto" and "xl-manual" as the policies?

Thoughts?

I suppose I ought to go back to
<1330078304.8557.157.camel@zakaz.uk.xensource.com> and update the
descriptions to account for this "actor" scheme and also flesh out the
underlying libxl interface (which we previously have ignored in that
discussion). Would that be useful?

Ian.

Nested SVM (was: Re: 4.2 TODO update)

[Christoph Egger]

On 03/12/12 13:11, Ian Campbell wrote:
>        * Nested-virtualisation (currently should be marked experimental,
>          likely to release that way? Consider nested-svm separate to
>          nested-vmx. Nested-svm is in better shape)

I tested the following scenarios:

- Linux on Xen on Xen
- NetBSD on Xen on Xen
- KVM (+ KVM-guest) on Xen
- KVM (+ KVM-guest) on Xen on Xen (yeah, l3 guest works!)
- Windows 7 (both 32bit and 64bit) on Xen on Xen
- Win7 XP mode on Windows 7 (both 32bit and 64bit) on Xen
- Hyper-V (64bit) on Xen
- VMware ESX (64bit) on Xen

Issues:
- Hyper-V SMP not tested, only UP
- Win7 XP mode often quits with BSOD, but when it boots
   playing Solitair makes fun :-)
- L2 guest screen doesn't refresh in graphical mode
   (VGA text mode works),
   workaround: dis- and re-establish the VNC connection,
   between two VNC reconnects you can work like with your
   monitor powered off

Christoph


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85689 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: Nested SVM (was: Re: 4.2 TODO update)

[Ian Campbell]

On Tue, 2012-03-13 at 11:40 +0000, Christoph Egger wrote:
> On 03/12/12 13:11, Ian Campbell wrote:
> >        * Nested-virtualisation (currently should be marked experimental,
> >          likely to release that way? Consider nested-svm separate to
> >          nested-vmx. Nested-svm is in better shape)
> 
> I tested the following scenarios:
> 
> - Linux on Xen on Xen
> - NetBSD on Xen on Xen
> - KVM (+ KVM-guest) on Xen
> - KVM (+ KVM-guest) on Xen on Xen (yeah, l3 guest works!)
> - Windows 7 (both 32bit and 64bit) on Xen on Xen
> - Win7 XP mode on Windows 7 (both 32bit and 64bit) on Xen
> - Hyper-V (64bit) on Xen
> - VMware ESX (64bit) on Xen

Wow, that's far more combinations than I expected.

I wonder where we can document this, is there a nested-HVM page on the
wiki perhaps?

> Issues:
> - Hyper-V SMP not tested, only UP
> - Win7 XP mode often quits with BSOD, 

Hrm, this one sounds quite important? I expect Win7 XP mode is the
primary real world use case for Nested-HVM?

> but when it boots playing Solitair makes fun :-)

;-)

> - L2 guest screen doesn't refresh in graphical mode
>    (VGA text mode works),
>    workaround: dis- and re-establish the VNC connection,
>    between two VNC reconnects you can work like with your
>    monitor powered off

Uh, that also sounds pretty critical. Does it also effect the Win7 XP
mode usecase?

Between those two issues I think removing the experimental tag sounds
premature?

Ian.

Re: Nested SVM

[Christoph Egger]

On 03/13/12 12:56, Ian Campbell wrote:
> On Tue, 2012-03-13 at 11:40 +0000, Christoph Egger wrote:
>> On 03/12/12 13:11, Ian Campbell wrote:
>>>         * Nested-virtualisation (currently should be marked experimental,
>>>           likely to release that way? Consider nested-svm separate to
>>>           nested-vmx. Nested-svm is in better shape)
>>
>> I tested the following scenarios:
>>
>> - Linux on Xen on Xen
>> - NetBSD on Xen on Xen
>> - KVM (+ KVM-guest) on Xen
>> - KVM (+ KVM-guest) on Xen on Xen (yeah, l3 guest works!)
>> - Windows 7 (both 32bit and 64bit) on Xen on Xen
>> - Win7 XP mode on Windows 7 (both 32bit and 64bit) on Xen
>> - Hyper-V (64bit) on Xen
>> - VMware ESX (64bit) on Xen
>
> Wow, that's far more combinations than I expected.
>
> I wonder where we can document this, is there a nested-HVM page on the
> wiki perhaps?
>
>> Issues:
>> - Hyper-V SMP not tested, only UP
>> - Win7 XP mode often quits with BSOD,
>
> Hrm, this one sounds quite important? I expect Win7 XP mode is the
> primary real world use case for Nested-HVM?

yes, but I have no idea what might be wrong.

>> but when it boots playing Solitair makes fun :-)
>
> ;-)

Playing Solitair gives a good impression on the end-user feeling
in how fast it response. It is also a good test case if interrupts
come in fast enough. If that is not the case a double click is
interpreted as two single clicks.

Playing Solitair in Win7 XP mode runs even faster than in Win7
thus makes more fun to play in Win7 XP mode.

>> - L2 guest screen doesn't refresh in graphical mode
>>     (VGA text mode works),
>>     workaround: dis- and re-establish the VNC connection,
>>     between two VNC reconnects you can work like with your
>>     monitor powered off
>
> Uh, that also sounds pretty critical. Does it also effect the Win7 XP
> mode usecase?

No, because in the Win7 XP mode usecase the graphic output is done by
the L1 guest (Windows 7) and that works. This problem only occurs when
the L2 guest runs in graphic mode.

I think, this issue is neither AMD nor Intel specific.
I think, this has something to do with the non-buffered io mode.
IIRC, VGA text mode uses a buffered io mode.

> Between those two issues I think removing the experimental tag sounds
> premature?

Yes.

Christoph


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85689 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Olaf Hering]

On Tue, Mar 13, Ian Campbell wrote:

> The string names the "actor" which will implement the policy (so perhaps
> the cfg option name should be "mem_actor="? Seems clumsy). So the
> default for xl would be "xl". An existing alternative actor would be
> "squeezed" which should cause the system to use XCP's squeezed (this
> would require updates to squeezed to actually use these interfaces). You
> can imagine that others might want to implement other more complex
> actors in the future (e.g. which combine sharing, paging, and tmem in an
> interesting way).

Ok, makes sense.

> The "xl" actor should implement the "paging=auto" balloon, delay, then
> page mechanism (or just ballooning for PV guests) we discussed
> previously (I think most recent proposal was in
> <1330078304.8557.157.camel@zakaz.uk.xensource.com>),
> iff /local/domain/X/memory-policy/actor == "xl". We can ignore sharing
> with this new scheme and leave it to whomever implements the sharing
> memory policy actor.

Yes.

> I think that in the normal case we would not support mixing and matching
> actors on a system, so in the case of xl I would expect to normally find
> mem_policy in /etc/xen/xl.conf rather than in the guest configuration
> file. It is reasonable for an actor implementation to consist either a
> per-host daemon (like squeezed) or a per-domain daemon (like xl).

Sounds good.

> libxl should also expose methods to set the balloon and paging targets,
> these would be used by the code in xl which implements the "xl" policy
> described above.

Yes.


> I think the libxl default should be to immediately set the balloon
> target. This would retain the historical behaviour for toolstacks which
> don't say differently and would also work as expected for dom0 (which
> may not have the necessary /local/domain/X/memory-policy/actor key).
> 
> The default set by xl should be "xl" or whatever is provided in the
> config.
> 
> The other option for the default provided by libxl will be to do nothing
> I don't think that is as helpful/useful as a default though.

I think that a default of "none" would change behaviour. So having "xl"
as default which makes the guests behave like before will remove
surprises during upgrade to 4.2.

> There should probably be an option to set the actor to "none", meaning
> that the toolstack is taking direct responsibility for this domains
> memory targets. This would be used when "xl mem-paging-set domain
> manual" was called allowing xl to implement the "xl mem-paging-set
> domain N" in manual mode as described in
> <1330078304.8557.157.camel@zakaz.uk.xensource.com>. Or maybe this
> corresponds to using "xl-auto" and "xl-manual" as the policies?

I'm not sure about the manual mode. If one calls mem-paging-set or
mem-balloon-set to change the target value, why not do it right away?


> Thoughts?

Thanks for the writeup, Ian!

> I suppose I ought to go back to
> <1330078304.8557.157.camel@zakaz.uk.xensource.com> and update the
> descriptions to account for this "actor" scheme and also flesh out the
> underlying libxl interface (which we previously have ignored in that
> discussion). Would that be useful?

Yes, an updated description/proposal is useful IMHO.

Olaf
> 

Re: Xen 4.2 release plan (Was: Re: 4.2 TODO update)

[Ross Philipson]

> -----Original Message-----
> From: xen-devel-bounces@lists.xen.org [mailto:xen-devel-
> bounces@lists.xen.org] On Behalf Of Ian Campbell
> Sent: Monday, March 12, 2012 9:38 AM
> To: xen-devel@lists.xen.org
> Subject: [Xen-devel] Xen 4.2 release plan (Was: Re: 4.2 TODO update)
> 
> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> [4.2 TODO list]
> 
> So the list is getting to be quite manageable. I think it is time we
> started discussing plans for a 4.2, timelines etc.
> 
> As a starting point for discussion how about the following:
> 
> After next week's TODO list update (19 March) I start taking a much
> harder line on what goes on the TODO list, especially the blocker list.
> A strong case will have to be made for why any particular addition is
> critical to Xen 4.2. This means that posting additions this week means
> you will have a much greater chance of that item making it onto the list
> for 4.2!
> 
> We then continue to focus on reducing the TODO list with a view to doing
> a feature freeze on, lets say, 2 April (3 weeks from today, 2 weeks from
> the "TODO list freeze").
> 
> At that point we become strict about not taking new features which are
> not already on the TODO list, becoming increasingly strict WRT nice-to-
> have vs blocker.
> 
> Aim to have -rc0 mid to late April, continuing weekly "Until Its
> Ready(tm)", expecting the release itself to land in May or June.
> 
> Any thoughts? Too Aggressive? Too Pessimistic?
> 
> Ian.
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

I am still working on a patch set for the BIOS module pass-through. I did not receive any responses to my RFC so I just started putting it together. I am working at getting the rest of it done today and spending tomorrow testing. Is it going to be too late for this to be considered?

Thanks
Ross

Re: Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Ian Campbell]

On Tue, 2012-03-13 at 13:36 +0000, Olaf Hering wrote:
> On Tue, Mar 13, Ian Campbell wrote:

> > I think the libxl default should be to immediately set the balloon
> > target. This would retain the historical behaviour for toolstacks which
> > don't say differently and would also work as expected for dom0 (which
> > may not have the necessary /local/domain/X/memory-policy/actor key).
> > 
> > The default set by xl should be "xl" or whatever is provided in the
> > config.
> > 
> > The other option for the default provided by libxl will be to do nothing
> > I don't think that is as helpful/useful as a default though.
> 
> I think that a default of "none" would change behaviour. So having "xl"
> as default which makes the guests behave like before will remove
> surprises during upgrade to 4.2.

The default for _libxl_ cannot be "xl", since the toolstack may not be
xl.

I think my first proposal for libxl's default makes sense, that is that
libxl should set things directly unless
libxl_domain_set_memory_policy_actor (or whatever the function ends up
being called) has been called.

> > There should probably be an option to set the actor to "none", meaning
> > that the toolstack is taking direct responsibility for this domains
> > memory targets. This would be used when "xl mem-paging-set domain
> > manual" was called allowing xl to implement the "xl mem-paging-set
> > domain N" in manual mode as described in
> > <1330078304.8557.157.camel@zakaz.uk.xensource.com>. Or maybe this
> > corresponds to using "xl-auto" and "xl-manual" as the policies?
> 
> I'm not sure about the manual mode. If one calls mem-paging-set or
> mem-balloon-set to change the target value, why not do it right away?

You would also need to change the actor to something which would not
conflict with such a manual change.

> > Thoughts?
> 
> Thanks for the writeup, Ian!
> 
> > I suppose I ought to go back to
> > <1330078304.8557.157.camel@zakaz.uk.xensource.com> and update the
> > descriptions to account for this "actor" scheme and also flesh out the
> > underlying libxl interface (which we previously have ignored in that
> > discussion). Would that be useful?
> 
> Yes, an updated description/proposal is useful IMHO.

I'll try and get to it in the next couple of days.

Ian.

Re: Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Olaf Hering]

On Tue, Mar 13, Ian Campbell wrote:

> The default for _libxl_ cannot be "xl", since the toolstack may not be
> xl.

Oh, yes you are right. I mixed libxl and xl.

> > I'm not sure about the manual mode. If one calls mem-paging-set or
> > mem-balloon-set to change the target value, why not do it right away?
> 
> You would also need to change the actor to something which would not
> conflict with such a manual change.

Yes, that should be done. Good point.


Olaf

Re: Xen 4.2 release plan (Was: Re: 4.2 TODO update)

[Ian Campbell]

On Tue, 2012-03-13 at 13:43 +0000, Ross Philipson wrote:
[...]
> > We then continue to focus on reducing the TODO list with a view to doing
> > a feature freeze on, lets say, 2 April (3 weeks from today, 2 weeks from
> > the "TODO list freeze").
> > [...]
> 
> I am still working on a patch set for the BIOS module pass-through. I
> did not receive any responses to my RFC so I just started putting it
> together. I am working at getting the rest of it done today and
> spending tomorrow testing. Is it going to be too late for this to be
> considered?

Not at all, my proposed date for a feature freeze was 2 April which is
weeks away.

Ian.

Re: Xen 4.2 release plan (Was: Re: 4.2 TODO update)

[Ross Philipson]

Super thanks. I just wanted to be clear. I should be able to post the series this week (barring any serious hurdles which I do not anticipate).

Ross

> -----Original Message-----
> From: Ian Campbell
> Sent: Tuesday, March 13, 2012 9:54 AM
> To: Ross Philipson
> Cc: xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] Xen 4.2 release plan (Was: Re: 4.2 TODO update)
> 
> On Tue, 2012-03-13 at 13:43 +0000, Ross Philipson wrote:
> [...]
> > > We then continue to focus on reducing the TODO list with a view to
> > > doing a feature freeze on, lets say, 2 April (3 weeks from today, 2
> > > weeks from the "TODO list freeze").
> > > [...]
> >
> > I am still working on a patch set for the BIOS module pass-through. I
> > did not receive any responses to my RFC so I just started putting it
> > together. I am working at getting the rest of it done today and
> > spending tomorrow testing. Is it going to be too late for this to be
> > considered?
> 
> Not at all, my proposed date for a feature freeze was 2 April which is
> weeks away.
> 
> Ian.
> 

Re: Paging/sharing in 4.2 (Was: Re: 4.2 TODO update)

[Andres Lagar-Cavilla]

>> There should probably be an option to set the actor to "none", meaning
>> that the toolstack is taking direct responsibility for this domains
>> memory targets. This would be used when "xl mem-paging-set domain
>> manual" was called allowing xl to implement the "xl mem-paging-set
>> domain N" in manual mode as described in
>> <1330078304.8557.157.camel@zakaz.uk.xensource.com>. Or maybe this
>> corresponds to using "xl-auto" and "xl-manual" as the policies?
>
> I'm not sure about the manual mode. If one calls mem-paging-set or
> mem-balloon-set to change the target value, why not do it right away?
>
>
>> Thoughts?
>
> Thanks for the writeup, Ian!

Ditto, thanks for sorting this out
Andres
>
>> I suppose I ought to go back to
>> <1330078304.8557.157.camel@zakaz.uk.xensource.com> and update the
>> descriptions to account for this "actor" scheme and also flesh out the
>> underlying libxl interface (which we previously have ignored in that
>> discussion). Would that be useful?
>
> Yes, an updated description/proposal is useful IMHO.
>
> Olaf
>>
>

Re: Volunteers required for 4.2 TODO items (Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 18:18 +0000, Goncalo Gomes wrote:
> On Mon, 12 Mar 2012, Ian Campbell wrote:
> 
> > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> > > There are several things below which are in need of a volunteer to take care of them...
> > 
> > Specifically:
> > 
> > > tools, blockers:
> > > [...]
> > >       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
> > >       * Domain 0 block attach & general hotplug when using qdisk backend
> > >         (need to start qemu as necessary etc)
> > > tools, nice to have:
> > > [...]
> > >       * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
> > >         (Nobody AFAICT)
> > >       * support for vif "rate" parameter (No one, AFAICT)
> > 
> > #1,#3 & #4 are reasonably simple libxl/xl features possibly suitable for
> > a newcomer who is interested in getting started with toolstack level
> > stuff.
> > 
> > #2 is a bit more involved.
> > 
> > Any takers?
> 
> I'd be keen to work on 
> 
> > >       * Domain 0 block attach & general hotplug when using qdisk backend
> > >         (need to start qemu as necessary etc)

I'm afraid Stefano has picked this one up.

I don't think anyone has picked up:
> * xl support for autospawning vncviewer (vncviewer=1 or otherwise)

I described it in a bit more detail in a mail to Lin Ming earlier in the
thread.

Ian.

Re: Volunteers required for 4.2 TODO items (Re: 4.2 TODO update)

[Goncalo Gomes]

On Tue, 13 Mar 2012, Ian Campbell wrote:

> On Mon, 2012-03-12 at 18:18 +0000, Goncalo Gomes wrote:
>
> I don't think anyone has picked up:
> > * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
> 
> I described it in a bit more detail in a mail to Lin Ming earlier in the
> thread.

I've just read the email on it and seeing as there isn't any 
confirmation from Lin Ming, please add my name and I'll be happy to 
take it.

Goncalo
 
> Ian.
> 

libxl stable API (Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> [...] 
> tools, blockers:
>       * libxl stable API -- we would like 4.2 to define a stable API
>         which downstream's can start to rely on not changing. Aspects of
>         this are:

So now would be a good time for potential users of libxl in 4.2 onwards
to do a quick sanity check of the interfaces such that any omissions can
be added to the 4.2 list.

Now is also a good time to decide on the policy and mechanisms which we
will use to make life easy for our downstreams (some of whom I have
CC'd).

The goal and default assumption should obviously be that a downstream
which builds against libxl from 4.2 will continue to build (and work!)
against future versions of libxl without source modification.

In order to help us achieve this we can define some interfaces /
mechanisms now which will make everyone's lives easier in the future. I
propose that downstreams who want to be exposed to a particular API
should be required to #define LIBXL_API_VERSION before including
libxl.h. This will allow libxl.h to introduce the necessary compat/shim
layer. e.g. if they want the 4.2.0 interface they must:
	#define LIBXL_API_VERSION 0x040200

Assuming that 4.3.0 eventually releases with an identical API then users
would be expected to continue to use 0x040200 (i.e. we wouldn't
explicitly duplicate up the #ifdef's for compatible releases) but if in
4.4.0 changes are required then users could either continue to use
0x040200 (and expect libxl.h to provide suitable impedance matching to
make that work) or switch to 0x040400 and make the necessary source
level changes. Lack of LIBXL_API_VERSION would be taken to mean "the
latest". Specifying an unknown LIBXL_API_VERSION would result in #error
(e.g. in the above example 0x040300 and 0x123456 would both be an
error).

We should try especially hard to avoid changing the API during a stable
series, i.e. it should be unusual for the last byte to be non-zero.

If there is broad agreement with this scheme I will write up a patch to
add it to some documentation / header somewhere.

Do we need to define a horizon for how far we are willing to support
this level of compatibility? A new major release seems like the obvious
watershed -- i.e. at Xen 5.0.0 we may decide to drop support for
LIBXL_API_VERSION 0x04xxxx and earlier (although we don't have to).

Much as I hate to admit it I expect there will eventually/inevitably be
changes which cannot be papered over with this scheme. In order to make
it possible for downstreams to support the use of a single code base
across such changes I suggest that we always add a #define
LIBXL_HAVE_foo_interface for any such change.

The assumption should be that the barrier for backporting to a stable
branch will be very high for any change of this type.

Lastly we also to decide what we want to do about ABI (as opposed to
API) compatibility. Obviously if we change the ABI then we should change
the SONAME, but is this something we want to commit to avoiding? i.e.
should it be possible to build a downstream against libxl.so.2.0 (the
current libxl soname) from 4.2 and expect that dropping in libxl.so.2.0
from 4.3 will Just Work against the new hypervisor interfaces? Or do we
expect that 4.3 will provide libxl.so.2.1 and that the same downstream
source can be built twice? (e.g. with the correct one selected via some
plugin mechanism). Obviously avoiding ABI changes is a lot harder and
I'm not sure if that is something we are able to commit to at this stage
(and I'm not sure how good we would be at it even if we did try and make
that commitment).

>               * add libxl_defbool and generally try and arrange that
>                 memset(foo,0,...) requests the defaults (Ian Campbell,
>                 DONE)
>               * Safe fork vs. fd handling hooks. This is an API
>                 addition, so maybe not required fro stable API, bit need
>                 to have for 4.2? (Ian J, patches posted)

Ian.

libxl stable API (Re: 4.2 TODO update)

[Ian Jackson]

Ian Campbell writes ("[Xen-devel] libxl stable API (Re:  4.2 TODO update)"):
> Lastly we also to decide what we want to do about ABI (as opposed to
> API) compatibility. Obviously if we change the ABI then we should change
> the SONAME, but is this something we want to commit to avoiding? i.e.
> should it be possible to build a downstream against libxl.so.2.0 (the
> current libxl soname) from 4.2 and expect that dropping in libxl.so.2.0
> >from 4.3 will Just Work against the new hypervisor interfaces? Or do we
> expect that 4.3 will provide libxl.so.2.1 and that the same downstream
> source can be built twice? (e.g. with the correct one selected via some
> plugin mechanism). Obviously avoiding ABI changes is a lot harder and
> I'm not sure if that is something we are able to commit to at this stage
> (and I'm not sure how good we would be at it even if we did try and make
> that commitment).

I agree with most of what you say.

I don't think we can commit to not making ABI changes.  It is thus an
unfortunate fact that the whole stack, from libxl caller on down to
the hypervisor, will have to change together when the version of Xen
is updated.

> >               * Safe fork vs. fd handling hooks. This is an API
> >                 addition, so maybe not required fro stable API, bit need
> >                 to have for 4.2? (Ian J, patches posted)

I think this is important to have.  It will imply changes to quite a
few ordinary api functions.

Ian.

Re: libxl stable API (Re: 4.2 TODO update)

[Ian Campbell]

On Wed, 2012-03-14 at 11:16 +0000, Ian Jackson wrote:
> Ian Campbell writes ("[Xen-devel] libxl stable API (Re:  4.2 TODO update)"):
> > Lastly we also to decide what we want to do about ABI (as opposed to
> > API) compatibility. Obviously if we change the ABI then we should change
> > the SONAME, but is this something we want to commit to avoiding? i.e.
> > should it be possible to build a downstream against libxl.so.2.0 (the
> > current libxl soname) from 4.2 and expect that dropping in libxl.so.2.0
> > >from 4.3 will Just Work against the new hypervisor interfaces? Or do we
> > expect that 4.3 will provide libxl.so.2.1 and that the same downstream
> > source can be built twice? (e.g. with the correct one selected via some
> > plugin mechanism). Obviously avoiding ABI changes is a lot harder and
> > I'm not sure if that is something we are able to commit to at this stage
> > (and I'm not sure how good we would be at it even if we did try and make
> > that commitment).
> 
> I agree with most of what you say.

Thanks. I shall add "agree & document compatibility guarantees +
associated technical measures" to the TODO list.

> I don't think we can commit to not making ABI changes.  It is thus an
> unfortunate fact that the whole stack, from libxl caller on down to
> the hypervisor, will have to change together when the version of Xen
> is updated.

Right. We can strive to make this a straight rebuild with no source
changes (somewhat akin to a binNMU in Debian-speak).

> 
> > >               * Safe fork vs. fd handling hooks. This is an API
> > >                 addition, so maybe not required fro stable API, bit need
> > >                 to have for 4.2? (Ian J, patches posted)
> 
> I think this is important to have.  It will imply changes to quite a
> few ordinary api functions.

Thanks, I will update the commentary accordingly in the next iteration.

Ian.

Re: 4.2 TODO update

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1738 bytes --]

On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> This update covers two weeks since I was away last week.
> 
Hi,

I think the below should be considered too, and I'm proposing it as a
(tools) blocker, as I really think it affects API stability.

I can take care of it, provided we have some preliminary
design-discussion. :-D
 
> tools, blockers:
>       * libxl stable API -- we would like 4.2 to define a stable API
>         which downstream's can start to rely on not changing. Aspects of
>         this are:
>               * add libxl_defbool and generally try and arrange that
>                 memset(foo,0,...) requests the defaults (Ian Campbell,
>                 DONE)
>               * Safe fork vs. fd handling hooks. This is an API
>                 addition, so maybe not required fro stable API, bit need
>                 to have for 4.2? (Ian J, patches posted)
>
  * locking/serialization, e.g., for domain creation. As of now, 
    nothing is provided for this purpose, and downstream toolstacks have
    to put their own mechanisms in place. E.g., xl uses a fcntl() lock  
    around the whole process of domain creation. It should OTOH be 
    libxl job to offer a set of hooks --properly placed within the
    domain creation process-- for the downstreams to fill with the
    proper callbacks. (Dario Faggioli)

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa (Italy)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[Ian Campbell]

On Wed, 2012-03-14 at 16:48 +0000, Dario Faggioli wrote:
> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> > This update covers two weeks since I was away last week.
> > 
> Hi,
> 
> I think the below should be considered too, and I'm proposing it as a
> (tools) blocker, as I really think it affects API stability.

Sure.

> I can take care of it, provided we have some preliminary
> design-discussion. :-D

A good starting point for discussion would be a proposed patch to
libxl.h with associated inline documentation.

>  
> > tools, blockers:
> >       * libxl stable API -- we would like 4.2 to define a stable API
> >         which downstream's can start to rely on not changing. Aspects of
> >         this are:
> >               * add libxl_defbool and generally try and arrange that
> >                 memset(foo,0,...) requests the defaults (Ian Campbell,
> >                 DONE)
> >               * Safe fork vs. fd handling hooks. This is an API
> >                 addition, so maybe not required fro stable API, bit need
> >                 to have for 4.2? (Ian J, patches posted)
> >
>   * locking/serialization, e.g., for domain creation. As of now, 
>     nothing is provided for this purpose, and downstream toolstacks have
>     to put their own mechanisms in place. E.g., xl uses a fcntl() lock  
>     around the whole process of domain creation. It should OTOH be 
>     libxl job to offer a set of hooks --properly placed within the
>     domain creation process-- for the downstreams to fill with the
>     proper callbacks. (Dario Faggioli)
> 
> Thanks and Regards,
> Dario
> 

Re: Xen 4.2 release plan (Was: Re: 4.2 TODO update)

[Ian Campbell]

On Mon, 2012-03-12 at 13:45 +0000, Keir Fraser wrote:
> On 12/03/2012 13:37, "Ian Campbell" <Ian.Campbell@citrix.com> wrote:
> 
> > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> > [4.2 TODO list]
> > 
> > So the list is getting to be quite manageable. I think it is time we
> > started discussing plans for a 4.2, timelines etc.
> > 
> > As a starting point for discussion how about the following:
> > 
> > After next week's TODO list update (19 March) I start taking a much
> > harder line on what goes on the TODO list, especially the blocker list.
> > A strong case will have to be made for why any particular addition is
> > critical to Xen 4.2. This means that posting additions this week means
> > you will have a much greater chance of that item making it onto the list
> > for 4.2!
> > 
> > We then continue to focus on reducing the TODO list with a view to doing
> > a feature freeze on, lets say, 2 April (3 weeks from today, 2 weeks from
> > the "TODO list freeze").
> > 
> > At that point we become strict about not taking new features which are
> > not already on the TODO list, becoming increasingly strict WRT
> > nice-to-have vs blocker.
> > 
> > Aim to have -rc0 mid to late April, continuing weekly "Until Its
> > Ready(tm)", expecting the release itself to land in May or June.
> > 
> > Any thoughts? Too Aggressive? Too Pessimistic?
> 
> Sounds about right.

Thanks. So in the absence of any objections lets call this The Plan.

Ian.

Re: 4.2 TODO update

[Ian Campbell]

On Tue, 2012-02-21 at 02:38 +0000, Michael A. Collins wrote:
> Is it my understanding that currently there is no way to pass the -vga
> option to qemu using the xl cfg file?  This seems in my mind a very trivial
> thing to do, I don't want to reinvent the wheel if I'm just missing
> something.

It seems you can arrange for "-std-vga" or "-vga std" (depending on if
you are using traditional or upstream qemu) or nothing but nothing more
complex than that.

What is it that you are trying to do by passing such an option?

Do you know if this this option was exposed in xend? If so then how?

(as a workaround there is an device_model_args{,_pv,_hvm} trio of
options in xl which allows arbitrary option to be passed to qemu)

Ian.

Re: 4.2 TODO update

[Michael A. Collins]

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> bounces@lists.xensource.com] On Behalf Of Ian Campbell
> Sent: Monday, February 13, 2012 6:56 AM
> To: Anthony PERARD
> Cc: xen-devel@lists.xensource.com; Fantu
> Subject: Re: [Xen-devel] 4.2 TODO update
> 
> On Mon, 2012-02-13 at 11:50 +0000, Anthony PERARD wrote:
> > On Mon, Feb 13, 2012 at 11:05, Fantu <fantonifabio@tiscali.it> wrote:
> > >         * Spice full support and build enabled by default
> > >
> > > About Spice support on qemu upstream, now is not built by default and
> on xl
> > > configuration files is minimal. Probably there are also some required
> missed
> > > options about qxl graphic needed by Spice on xl configuration files.
> > > I search on tools/libxl/libxl_dm.c but I have found nothing about qxl
now.
> >
> > About the build, QEMU should include spice support if the necessary
> > library are installed. But on Debian stable (Squeeze), the spice
> > library are too old, so forcing QEMU to have spice support will just
> > break the build.
> 
> Agreed, spice support should be conditional on the necessary support
> libraries being available. It would however be useful to document (e.g.
> in the top-level README) exactly what those requirements are.
> 
> Ian.
> 

Is it my understanding that currently there is no way to pass the -vga
option to qemu using the xl cfg file?  This seems in my mind a very trivial
thing to do, I don't want to reinvent the wheel if I'm just missing
something.
Mike

Re: 4.2 TODO update

[Andres Lagar-Cavilla]

> Date: Mon, 20 Feb 2012 15:52:01 +0000
> From: Ian Campbell <Ian.Campbell@citrix.com>
> To: xen-devel <xen-devel@lists.xensource.com>
> Subject: [Xen-devel] 4.2 TODO update
> Message-ID: <1329753121.3990.67.camel@zakaz.uk.xensource.com>
> Content-Type: text/plain; charset="UTF-8"
>
> This weeks update. As usual please ping me with any updates.
>
> hypervisor, blockers:
>       * round-up of the closing of the security hole in MSI-X
>         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>         access to MSI-X table pages). (Jan Beulich, DONE)
>       * domctls / sysctls set up to modify scheduler parameters, like
>         the credit1 timeslice and schedule rate. (George Dunlap)
>       * get the interface changes for sharing/paging/mem-events done and
>         dusted so that 4.2 is a stable API that we hold to. (Tim Deegan,
>         Andres Lagar-Cavilla et al)
>               * sharing patches posted (DONE)
>
> tools, blockers:
>
>       * libxl stable API -- we would like 4.2 to define a stable API
>         which downstream's can start to rely on not changing. Aspects of
>         this are:
>               * add libxl_defbool and generally try and arrange that
>                 memset(foo,0,...) requests the defaults (Ian Campbell,
>                 patches reposted)
>               * Safe fork vs. fd handling hooks. This is an API
>                 addition, so maybe not required fro stable API, bit need
>                 to have for 4.2? (Ian J)
>               * xl feature parity with xend wrt driver domain support
>                 (George Dunlap)
>               * More formally deprecate xm/xend. Manpage patches already
>                 in tree. Needs release noting and communication around
>                 -rc1 to remind people to test xl.
>       * Correct paging/sharing tools buffer mlocking (Tim, Andres)
Will post later in the week.

>       * Autoconf (Roger Pau Monn? & Ian J, blocked on test
>         infrastructure changes, Roger to respin patch when test system
>         ready for new features)
>       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
>
> hypervisor, nice to have:
>
>       * solid implementation of sharing/paging/mem-events (using work
>         queues) (Tim Deegan, Olaf Herring et al)
>       * A long standing issue is a fully synchronized p2m (locking
>         lookups) (Andres Lagar-Cavilla, patches posted?)
This one is DONE

Thanks,
Andres
>
> tools, nice to have:
>
>       * Hotplug script stuff -- internal to libxl (I think, therefore I
>         didn't put this under stable API above) but still good to have
>         for 4.2? (Roger Pau Monn?, patches posted)
>       * Block script support -- follows on from hotplug script (Roger
>         Pau Monn?)
>       * Configure/control paging via xl/libxl (Olaf Herring)
>       * Upstream qemu feature patches:
>               * Upstream qemu PCI passthrough support (Anthony Perard,
>                 patches sent)
>               * Upstream qemu save restore (Anthony Perard, Stefano
>                 Stabellini, patches sent, waiting for upstream ack)
>       * Nested-virtualisation (currently should be marked experimental,
>         likely to release that way? Consider nested-svm separate to
>         nested-vmx. Nested-svm is in better shape)
>       * Initial xl support for Remus (memory checkpoint, blackholing)
>         (Shriram, patches posted)
>       * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
>         (Nobody AFAICT)
>
>

4.2 TODO update

[Ian Campbell]

This weeks update. As usual please ping me with any updates.

hypervisor, blockers:
      * round-up of the closing of the security hole in MSI-X
        passthrough (uniformly - i.e. even for Dom0 - disallowing write
        access to MSI-X table pages). (Jan Beulich, DONE)
      * domctls / sysctls set up to modify scheduler parameters, like
        the credit1 timeslice and schedule rate. (George Dunlap)
      * get the interface changes for sharing/paging/mem-events done and
        dusted so that 4.2 is a stable API that we hold to. (Tim Deegan,
        Andres Lagar-Cavilla et al)
              * sharing patches posted (DONE)

tools, blockers:

      * libxl stable API -- we would like 4.2 to define a stable API
        which downstream's can start to rely on not changing. Aspects of
        this are:
              * add libxl_defbool and generally try and arrange that
                memset(foo,0,...) requests the defaults (Ian Campbell,
                patches reposted)
              * Safe fork vs. fd handling hooks. This is an API
                addition, so maybe not required fro stable API, bit need
                to have for 4.2? (Ian J)
              * xl feature parity with xend wrt driver domain support
                (George Dunlap)
              * More formally deprecate xm/xend. Manpage patches already
                in tree. Needs release noting and communication around
                -rc1 to remind people to test xl.
      * Correct paging/sharing tools buffer mlocking (Tim, Andres)
      * Autoconf (Roger Pau Monné & Ian J, blocked on test
        infrastructure changes, Roger to respin patch when test system
        ready for new features)
      * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)

hypervisor, nice to have:

      * solid implementation of sharing/paging/mem-events (using work
        queues) (Tim Deegan, Olaf Herring et al)
      * A long standing issue is a fully synchronized p2m (locking
        lookups) (Andres Lagar-Cavilla, patches posted?)

tools, nice to have:

      * Hotplug script stuff -- internal to libxl (I think, therefore I
        didn't put this under stable API above) but still good to have
        for 4.2? (Roger Pau Monné, patches posted)
      * Block script support -- follows on from hotplug script (Roger
        Pau Monné)
      * Configure/control paging via xl/libxl (Olaf Herring)
      * Upstream qemu feature patches:
              * Upstream qemu PCI passthrough support (Anthony Perard,
                patches sent)
              * Upstream qemu save restore (Anthony Perard, Stefano
                Stabellini, patches sent, waiting for upstream ack)
      * Nested-virtualisation (currently should be marked experimental,
        likely to release that way? Consider nested-svm separate to
        nested-vmx. Nested-svm is in better shape)
      * Initial xl support for Remus (memory checkpoint, blackholing)
        (Shriram, patches posted)
      * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
        (Nobody AFAICT)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-02-13 at 10:17 +0000, Ian Campbell wrote:
> 
> tools, nice to have: 

Jim Burns reports that

	vncviewer=1 used to auto spawn a vnc viewer for you

and that

	localtime and rtc_timeoffset do not work with xl.

Both seem like worthwhile things to have in some form.

Ian.

Re: 4.2 TODO update

[Tim Deegan]

At 08:22 -0800 on 15 Feb (1329294161), Andres Lagar-Cavilla wrote:
> > Maybe we can arrange that instead of bugging out if the cpu is
> > in_atomic() it gdprintk()s a big ol' warning and crashes the guest?  It
> > seems no worse than the current failure modes.
> 
> How about judiciously adding the following
> 
> get_gfn_sleep(d, gfn, type)
> {
>   if (d == current_domain && !in_atomic())
>   {
>     printk("Naughty");
>     crash_domain(d);
>     return INVALID_MFN;
>   }

Yes, that's the sort of thing I had in mind (though the in_atomic() test
shouldn't be inverted).

I'll dig out Olaf's most recent patch tomorrow and see how that would
work; I'm travelling so my access to test hardware is a bit limited but
I'll try to at least make a draft patch.

Tim.

Re: 4.2 TODO update

[Andres Lagar-Cavilla]

> At 08:47 -0800 on 14 Feb (1329209245), Andres Lagar-Cavilla wrote:
>> > On Tue, Feb 14, Andres Lagar-Cavilla wrote:
>> >
>> >> Why? Because it's really really hard to guarantee we'll go to sleep
>> in
>> >> an
>> >> atomic context. The main use for wait queues (imho) is in hvm_copy,
>> and
>> >> there's a zillion paths going into hvm_copy (copy_from/to_user!) with
>> >> all
>> >> ways of bumping the preemption count.
>> >
>> > If the guests pagetable is paged out this code path will trigger, then
>> > one of the hypercalls returns an error and the guest runs into a
>> BUG().
>> > I think it was decrease_reservation, or similar.
>>
>> Unlikely to be something specific about decrease_reservation. If the
>> guest
>> page table is paged out, then copy_from_user for any hypercall, or,
>> "virtual address to gfn" for any emulation will run into this.
>>
>> Now, even an innocent-looking rcu lock anywhere in this code path will
>> crash the host if we go into a wait queue. Hence my concern.
>
> OK, so the current code breaks the guest and you're worried about it
> crashing the host instead.   That seems fair.
>
> Maybe we can arrange that instead of bugging out if the cpu is
> in_atomic() it gdprintk()s a big ol' warning and crashes the guest?  It
> seems no worse than the current failure modes.

How about judiciously adding the following

get_gfn_sleep(d, gfn, type)
{
  if (d == current_domain && !in_atomic())
  {
    printk("Naughty");
    crash_domain(d);
    return INVALID_MFN;
  }

retry:
  mfn rval = get_gfn(d, gfn, type)
  if (d == current->domain && type == paging && !mfn_valid(mfn))
  {
    put_gfn(d, gfn);
    go to sleep on wait queue;
    goto retry;
  }

  return rval;
}

Then we could toss it in before vmx_load_pdptrs gets to do evil things,
for example.

Andres

>
>> > What other way exist to make paging 100% transparent to the guest?
>>
>> Don't page out page table pages? I know you were not expecting that...
>
> Sadly, it's not possible to reliably detect which pages are pagetables
> (or the shadow pagetable code would be more straightforward!).
>
> Cheers,
>
> Tim.
>

Re: 4.2 TODO update

[Tim Deegan]

At 08:47 -0800 on 14 Feb (1329209245), Andres Lagar-Cavilla wrote:
> > On Tue, Feb 14, Andres Lagar-Cavilla wrote:
> >
> >> Why? Because it's really really hard to guarantee we'll go to sleep in
> >> an
> >> atomic context. The main use for wait queues (imho) is in hvm_copy, and
> >> there's a zillion paths going into hvm_copy (copy_from/to_user!) with
> >> all
> >> ways of bumping the preemption count.
> >
> > If the guests pagetable is paged out this code path will trigger, then
> > one of the hypercalls returns an error and the guest runs into a BUG().
> > I think it was decrease_reservation, or similar.
> 
> Unlikely to be something specific about decrease_reservation. If the guest
> page table is paged out, then copy_from_user for any hypercall, or,
> "virtual address to gfn" for any emulation will run into this.
> 
> Now, even an innocent-looking rcu lock anywhere in this code path will
> crash the host if we go into a wait queue. Hence my concern.

OK, so the current code breaks the guest and you're worried about it
crashing the host instead.   That seems fair.

Maybe we can arrange that instead of bugging out if the cpu is
in_atomic() it gdprintk()s a big ol' warning and crashes the guest?  It
seems no worse than the current failure modes.

> > What other way exist to make paging 100% transparent to the guest?
> 
> Don't page out page table pages? I know you were not expecting that...

Sadly, it's not possible to reliably detect which pages are pagetables
(or the shadow pagetable code would be more straightforward!).

Cheers,

Tim.

Re: 4.2 TODO update

[Andres Lagar-Cavilla]

> On Tue, Feb 14, Andres Lagar-Cavilla wrote:
>
>> > On Tue, Feb 14, Andres Lagar-Cavilla wrote:
>> >
>> >> Why? Because it's really really hard to guarantee we'll go to sleep
>> in
>> >> an
>> >> atomic context. The main use for wait queues (imho) is in hvm_copy,
>> and
>> >> there's a zillion paths going into hvm_copy (copy_from/to_user!) with
>> >> all
>> >> ways of bumping the preemption count.
>> >
>> > If the guests pagetable is paged out this code path will trigger, then
>> > one of the hypercalls returns an error and the guest runs into a
>> BUG().
>> > I think it was decrease_reservation, or similar.
>>
>> Unlikely to be something specific about decrease_reservation. If the
>> guest
>> page table is paged out, then copy_from_user for any hypercall, or,
>> "virtual address to gfn" for any emulation will run into this.
>>
>> Now, even an innocent-looking rcu lock anywhere in this code path will
>> crash the host if we go into a wait queue. Hence my concern.
>
> The workaround for the guest crash I were seeing:
> http://lists.xen.org/archives/html/xen-devel/2010-11/msg01609.html

This makes sense. It's basically what emulation does (X86_EMUL_RETRY). Great!

>
> I once modified xenpaging to keep the pagetables paged out as much as it
> could (and still allowed the guest to make at least a little bit
> progress) and run into no appearent issue.
>
>> > Another thing reported by Huawei on this list was somewhere in the
>> > emulation code where a gfn_to_mfn() failed.
>>
>> Can you point to the original report? Is there anything more specific?
>
> It was vmx_load_pdptrs().
> http://lists.xen.org/archives/html/xen-devel/2011-09/msg01336.html

I have a patch queued up for cleaning up vmx_load_pdptrs so that it
doesn't crash the host if the cr3 is paged out for a PAE guest.

You can't really do what they propose (mdelay, retry). You're in a
hypervisor context holding locks, nothing is getting scheduled!

One option worth considering is testing the cr3 "early enough", when it's
safe to sleep.

>
>> > What other way exist to make paging 100% transparent to the guest?
>> >
>>
>> Don't page out page table pages? I know you were not expecting that...
>
> How can xenpaging know what gfns are pagetables?

There's a bunch of heuristics one can try. It boils down to a lengthy
discussion about what the pager is trying to do. Say, you can
statistically sample cr3's, map candidates and look for page table-like
structures, exploit knowledge about the guest OS.

I'll agree though that the hypervisor support should be good enough that
the pager should not care. But it isn't and it won't get there quickly,
and I don't want us to be cavalier about these wait queues.

Andres

>
> Olaf
>

Re: 4.2 TODO update

[Olaf Hering]

On Tue, Feb 14, Andres Lagar-Cavilla wrote:

> > On Tue, Feb 14, Andres Lagar-Cavilla wrote:
> >
> >> Why? Because it's really really hard to guarantee we'll go to sleep in
> >> an
> >> atomic context. The main use for wait queues (imho) is in hvm_copy, and
> >> there's a zillion paths going into hvm_copy (copy_from/to_user!) with
> >> all
> >> ways of bumping the preemption count.
> >
> > If the guests pagetable is paged out this code path will trigger, then
> > one of the hypercalls returns an error and the guest runs into a BUG().
> > I think it was decrease_reservation, or similar.
> 
> Unlikely to be something specific about decrease_reservation. If the guest
> page table is paged out, then copy_from_user for any hypercall, or,
> "virtual address to gfn" for any emulation will run into this.
> 
> Now, even an innocent-looking rcu lock anywhere in this code path will
> crash the host if we go into a wait queue. Hence my concern.

The workaround for the guest crash I were seeing:
http://lists.xen.org/archives/html/xen-devel/2010-11/msg01609.html

I once modified xenpaging to keep the pagetables paged out as much as it
could (and still allowed the guest to make at least a little bit
progress) and run into no appearent issue.

> > Another thing reported by Huawei on this list was somewhere in the
> > emulation code where a gfn_to_mfn() failed.
> 
> Can you point to the original report? Is there anything more specific?

It was vmx_load_pdptrs().
http://lists.xen.org/archives/html/xen-devel/2011-09/msg01336.html

> > What other way exist to make paging 100% transparent to the guest?
> >
> 
> Don't page out page table pages? I know you were not expecting that...

How can xenpaging know what gfns are pagetables?

Olaf

Re: 4.2 TODO update

[Andres Lagar-Cavilla]

> On Tue, Feb 14, Andres Lagar-Cavilla wrote:
>
>> Why? Because it's really really hard to guarantee we'll go to sleep in
>> an
>> atomic context. The main use for wait queues (imho) is in hvm_copy, and
>> there's a zillion paths going into hvm_copy (copy_from/to_user!) with
>> all
>> ways of bumping the preemption count.
>
> If the guests pagetable is paged out this code path will trigger, then
> one of the hypercalls returns an error and the guest runs into a BUG().
> I think it was decrease_reservation, or similar.

Unlikely to be something specific about decrease_reservation. If the guest
page table is paged out, then copy_from_user for any hypercall, or,
"virtual address to gfn" for any emulation will run into this.

Now, even an innocent-looking rcu lock anywhere in this code path will
crash the host if we go into a wait queue. Hence my concern.

> Another thing reported by Huawei on this list was somewhere in the
> emulation code where a gfn_to_mfn() failed.

Can you point to the original report? Is there anything more specific?

>
> What other way exist to make paging 100% transparent to the guest?
>

Don't page out page table pages? I know you were not expecting that...

Andres

> Olaf
>

Re: 4.2 TODO update

[Olaf Hering]

On Tue, Feb 14, Andres Lagar-Cavilla wrote:

> Why? Because it's really really hard to guarantee we'll go to sleep in an
> atomic context. The main use for wait queues (imho) is in hvm_copy, and
> there's a zillion paths going into hvm_copy (copy_from/to_user!) with all
> ways of bumping the preemption count.

If the guests pagetable is paged out this code path will trigger, then
one of the hypercalls returns an error and the guest runs into a BUG().
I think it was decrease_reservation, or similar.
Another thing reported by Huawei on this list was somewhere in the
emulation code where a gfn_to_mfn() failed.

What other way exist to make paging 100% transparent to the guest?

Olaf

Re: 4.2 TODO update

[Andres Lagar-Cavilla]

> Date: Mon, 13 Feb 2012 10:24:29 +0000
> From: Tim Deegan <tim@xen.org>
> To: Ian Campbell <Ian.Campbell@citrix.com>
> Cc: xen-devel <xen-devel@lists.xensource.com>
> Subject: Re: [Xen-devel] 4.2 TODO update
> Message-ID: <20120213102429.GA88765@ocelot.phlegethon.org>
> Content-Type: text/plain; charset=iso-8859-1
>
> At 10:17 +0000 on 13 Feb (1329128265), Ian Campbell wrote:
>> This weeks update. I was away last Monday so this covers two weeks worth
>> of updates. I have dropped things which were marked as DONE in the last
>> update and added a bunch more DONE (so the list is shrinking!). Please
>> send me corrections (especially "done").
>>
>> hypervisor, blockers:
>>       * round-up of the closing of the security hole in MSI-X
>>         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>>         access to MSI-X table pages). (Jan Beulich -- more fixes
>>         required than first thought, patches posted)
>>       * domctls / sysctls set up to modify scheduler parameters, like
>>         the credit1 timeslice and schedule rate. (George Dunlap)
>>       * get the interface changes for sharing/paging/mem-events done and
>>         dusted so that 4.2 is a stable API that we hold to. (Tim Deegan,
>>         Andres Lagar-Cavilla et al)
>>               * sharing patches posted
>
> The hypervisor interface changes for sharing and events have gone in;
> there's a libxc-level change (to do with mlock()ing buffers) that ought
> to go in before 4.2, if at all.

Ian, it's probably OK to mark the paging/sharing as DONE.

There is the outstanding issue of properly mapping the event rings, which
ties into the mlock()ing buffers bit Tim mentions. I think we all want it
Done Right for 4.2. And that will likely not be the patch I last posted.
Perhaps best to rename the bullet point?

In the "nice to have" category, "properly synchronized p2m lookups" have
also gone into the tree.

>
> There's a paging+waitqueues patch that's been posted but needs a bit more
> work still; I'll try and look at it on Thursday.
>

This one is thorny. Right now we seem to have fairly stable paging. (Dare
I say bug-free? I dare not, lest I be smitten) I surmise other consumers
of the interface would agree with this (Olaf? Huawei?)

Wait queues have a significant potential for damage, unless we use them in
very specific contexts. I worry we'll drop from A- to C in terms of
stability. And right now I have not crashed hosts with paging in a long
while, whereas wait queues will definitely introduce that.

Why? Because it's really really hard to guarantee we'll go to sleep in an
atomic context. The main use for wait queues (imho) is in hvm_copy, and
there's a zillion paths going into hvm_copy (copy_from/to_user!) with all
ways of bumping the preemption count.

I'm dealing with this at the moment in a different context: running out of
memory to unshare pages when writing to them in the hypervisor. And we
simply cannot throw a wait queue sleep in there carelessly.

I'm not saying this is impossible, I'm saying there's a long path ahead,
and since it's all *internal* to the hypervisor, my vote is to be very
conservative for the 4.2 window.

Thanks!
Andres

> Tim.
>

Re: 4.2 TODO update

[Jan Beulich]

>>> On 13.02.12 at 12:32, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-02-13 at 11:29 +0000, Jan Beulich wrote:
>> >>> On 13.02.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > hypervisor, blockers:
>> >       * round-up of the closing of the security hole in MSI-X
>> >         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>> >         access to MSI-X table pages). (Jan Beulich -- more fixes
>> >         required than first thought, patches posted)
>> 
>> The only one currently open is the one removing write permission for
>> Dom0.
> 
> Oh, I thought you had found another issue which required further
> patches. Did I misunderstand or did they go in already?
> 
>>  The intention was to get this in after the qemu usptream pass
>> through patch series got adjusted along the lines of what was done
>> to qemu traditional, and while this was promise to happen soon after
>> New Year I didn't hear back anything from Anthony or Stefano.
>> 
>> Question is whether, given that the patch series in question isn't in
>> anything that is or will soon be released, it makes sense to push the
>> hypervisor change without waiting for that fixup.
> 
> I don't think it is necessary to wait for the qemu-upstream patch series
> to be updated for this, is it?

Patch just sent out. Once that's in, the issue should be fully addressed
for HVM guests (the situation with PV guests is explained in the patch).

Jan

Re: 4.2 TODO update

[Shriram Rajagopalan]

[-- Attachment #1.1: Type: text/plain, Size: 1499 bytes --]

On Mon, Feb 13, 2012 at 2:17 AM, Ian Campbell <Ian.Campbell@citrix.com>wrote:

>
> tools, nice to have:
>
>      * Hotplug script stuff -- internal to libxl (I think, therefore I
>        didn't put this under stable API above) but still good to have
>        for 4.2? (Roger Pau Monet, patches posted)
>      * Block script support -- follows on from hotplug script (Roger
>        Pau Monet)
>      * libyajl v2 support (Roger Pau Monet, DONE)
>      * Configure/control paging via xl/libxl (Olaf Herring)
>      * Upstream qemu feature patches:
>              * Upstream qemu PCI passthrough support (Anthony Perard)
>              * Upstream qemu save restore (Anthony Perard, Stefano
>                Stabellini)
>      * Nested-virtualisation (currently should be marked
>        experimental,likely to release that way? Consider nested-svm
>        separate to nested-vmx. Nested-svm is in better shape)
>      * Initial xl support for Remus (memory checkpoint, blackholing)
>        (Shriram, patches posted)
>
>


FYI, the network buffering module which was part of 2.6.32 pvops tree has
been accepted into upstream net-next tree. I have also posted patches for
the
userspace libnl code that talks to the qdiscs via netlink.

I am just waiting for the initial xl patches to go in, after which I could
take a stab
at adding network buffering support. Depending on whether Roger's hotplug
patches
go in or not, I could add some rudimentary disk replication support, time
permitting.

shriram

[-- Attachment #1.2: Type: text/html, Size: 1876 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 4.2 TODO update

[Stefano Stabellini]

On Mon, 13 Feb 2012, Jan Beulich wrote:
> >>  The intention was to get this in after the qemu usptream pass
> >> through patch series got adjusted along the lines of what was done
> >> to qemu traditional, and while this was promise to happen soon after
> >> New Year I didn't hear back anything from Anthony or Stefano.
> >> 
> >> Question is whether, given that the patch series in question isn't in
> >> anything that is or will soon be released, it makes sense to push the
> >> hypervisor change without waiting for that fixup.
> > 
> > I don't think it is necessary to wait for the qemu-upstream patch series
> > to be updated for this, is it?
> 
> Let's wait at least a day or two to give Stefano and Anthony a chance
> to respond.

I don't think we need to wait for the upstream PCI passthrough series to
be accepted in order for the hypervisor changes to go in, however it
would be nice if you could point out what needs to be changed on the
QEMU side, when Anthony posts the next version of the series.

Re: 4.2 TODO update

[Jan Beulich]

>>> On 13.02.12 at 12:32, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-02-13 at 11:29 +0000, Jan Beulich wrote:
>> >>> On 13.02.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > hypervisor, blockers:
>> >       * round-up of the closing of the security hole in MSI-X
>> >         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>> >         access to MSI-X table pages). (Jan Beulich -- more fixes
>> >         required than first thought, patches posted)
>> 
>> The only one currently open is the one removing write permission for
>> Dom0.
> 
> Oh, I thought you had found another issue which required further
> patches. Did I misunderstand or did they go in already?

They went in already.

>>  The intention was to get this in after the qemu usptream pass
>> through patch series got adjusted along the lines of what was done
>> to qemu traditional, and while this was promise to happen soon after
>> New Year I didn't hear back anything from Anthony or Stefano.
>> 
>> Question is whether, given that the patch series in question isn't in
>> anything that is or will soon be released, it makes sense to push the
>> hypervisor change without waiting for that fixup.
> 
> I don't think it is necessary to wait for the qemu-upstream patch series
> to be updated for this, is it?

Let's wait at least a day or two to give Stefano and Anthony a chance
to respond.

Jan

Re: 4.2 TODO update

[Stefano Stabellini]

On Mon, 13 Feb 2012, Ian Campbell wrote:
>       * Upstream qemu feature patches:
>               * Upstream qemu PCI passthrough support (Anthony Perard)
>               * Upstream qemu save restore (Anthony Perard, Stefano
>                 Stabellini)

I have sent patches for both QEMU and xl/libxl, I have addressed all
comments but I haven't received any acks yet. I'll ping the maintainers
again.

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-02-13 at 11:50 +0000, Anthony PERARD wrote:
> On Mon, Feb 13, 2012 at 11:05, Fantu <fantonifabio@tiscali.it> wrote:
> >         * Spice full support and build enabled by default
> >
> > About Spice support on qemu upstream, now is not built by default and on xl
> > configuration files is minimal. Probably there are also some required missed
> > options about qxl graphic needed by Spice on xl configuration files.
> > I search on tools/libxl/libxl_dm.c but I have found nothing about qxl now.
> 
> About the build, QEMU should include spice support if the necessary
> library are installed. But on Debian stable (Squeeze), the spice
> library are too old, so forcing QEMU to have spice support will just
> break the build.

Agreed, spice support should be conditional on the necessary support
libraries being available. It would however be useful to document (e.g.
in the top-level README) exactly what those requirements are.

Ian.

Re: 4.2 TODO update

[Anthony PERARD]

On Mon, Feb 13, 2012 at 11:05, Fantu <fantonifabio@tiscali.it> wrote:
>         * Spice full support and build enabled by default
>
> About Spice support on qemu upstream, now is not built by default and on xl
> configuration files is minimal. Probably there are also some required missed
> options about qxl graphic needed by Spice on xl configuration files.
> I search on tools/libxl/libxl_dm.c but I have found nothing about qxl now.

About the build, QEMU should include spice support if the necessary
library are installed. But on Debian stable (Squeeze), the spice
library are too old, so forcing QEMU to have spice support will just
break the build.

Regards,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 4.2 TODO update

[Anthony PERARD]

On Mon, Feb 13, 2012 at 10:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>      * Upstream qemu feature patches:
>              * Upstream qemu PCI passthrough support (Anthony Perard)

I'll send a new patch set soon, but the power management capability
will be missing as it's need a bit more work.
Also, this patch series does not longuer include the msitranslate code.

>              * Upstream qemu save restore (Anthony Perard, Stefano
>                Stabellini)

Still need some ack from qemu's developper.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-02-13 at 11:29 +0000, Jan Beulich wrote:
> >>> On 13.02.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > hypervisor, blockers:
> >       * round-up of the closing of the security hole in MSI-X
> >         passthrough (uniformly - i.e. even for Dom0 - disallowing write
> >         access to MSI-X table pages). (Jan Beulich -- more fixes
> >         required than first thought, patches posted)
> 
> The only one currently open is the one removing write permission for
> Dom0.

Oh, I thought you had found another issue which required further
patches. Did I misunderstand or did they go in already?

>  The intention was to get this in after the qemu usptream pass
> through patch series got adjusted along the lines of what was done
> to qemu traditional, and while this was promise to happen soon after
> New Year I didn't hear back anything from Anthony or Stefano.
> 
> Question is whether, given that the patch series in question isn't in
> anything that is or will soon be released, it makes sense to push the
> hypervisor change without waiting for that fixup.

I don't think it is necessary to wait for the qemu-upstream patch series
to be updated for this, is it?

Ian.

Re: 4.2 TODO update

[Jan Beulich]

>>> On 13.02.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> hypervisor, blockers:
>       * round-up of the closing of the security hole in MSI-X
>         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>         access to MSI-X table pages). (Jan Beulich -- more fixes
>         required than first thought, patches posted)

The only one currently open is the one removing write permission for
Dom0. The intention was to get this in after the qemu usptream pass
through patch series got adjusted along the lines of what was done
to qemu traditional, and while this was promise to happen soon after
New Year I didn't hear back anything from Anthony or Stefano.

Question is whether, given that the patch series in question isn't in
anything that is or will soon be released, it makes sense to push the
hypervisor change without waiting for that fixup.

Jan

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-02-13 at 11:05 +0000, Fantu wrote:
> Thanks to all developer for good work, I think it would be good to add these:
> 
> tools, nice to have: 
> ...
>       * Add network script stuff for Open Vswitch support

I made a stab at this a way back but fell into the exact sorts of issues
that the "Hotplug script stuff" (already on the TODO) is supposed to
address (mainly the lack of a suitable teardown hook which vswitch
needs). However once that stuff is done I reckon vswitch support would
be reasonably easy to add.

>       * Upstream qemu feature patches:
>          ...
>          * Spice full support and build enabled by default
> 
> 
> About Spice support on qemu upstream, now is not built by default and on xl
> configuration files is minimal. Probably there are also some required missed
> options about qxl graphic needed by Spice on xl configuration files.
> I search on tools/libxl/libxl_dm.c but I have found nothing about qxl now.
> 
> In these days I'll try to build and use Spice, and eventually I'll post
> patches about to improve Spice support.

Thanks, I'm looking forward to your patches.

> Anyone got the above stuff already working? Any patches out there?

Presumably the contributor who added the spice support to libxl had but
they had probably built qemu upstream by hand with the appropriate
options. I'm not aware of any outstanding SPICE patches.

Ian.

Re: 4.2 TODO update

[Fantu]

Thanks to all developer for good work, I think it would be good to add these:

tools, nice to have: 
...
      * Add network script stuff for Open Vswitch support
      * Upstream qemu feature patches:
         ...
         * Spice full support and build enabled by default


About Spice support on qemu upstream, now is not built by default and on xl
configuration files is minimal. Probably there are also some required missed
options about qxl graphic needed by Spice on xl configuration files.
I search on tools/libxl/libxl_dm.c but I have found nothing about qxl now.

In these days I'll try to build and use Spice, and eventually I'll post
patches about to improve Spice support.

Anyone got the above stuff already working? Any patches out there?

Thanks for any reply and sorry for bad english.

--
View this message in context: http://xen.1045712.n5.nabble.com/4-2-TODO-update-tp5478696p5478798.html
Sent from the Xen - Dev mailing list archive at Nabble.com.

Re: 4.2 TODO update

[George Dunlap]

On Mon, Feb 13, 2012 at 10:17 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>      * domctls / sysctls set up to modify scheduler parameters, like
>        the credit1 timeslice and schedule rate. (George Dunlap)

Not done; I'll try to get this fixed up this week.

>              * xl feature parity with xend wrt driver domain support
>                (George Dunlap)

I probably won't have time to start on this for another two weeks or
so; it's not clear ATM how long it might take to get working after
that (since I haven't even taken a cursory look at it).  If anyone is
sufficiently motivated to take a look, they're welcome to do so. :-)

Re blocker status: I think blockers aren't exactly binary, but more
like "How long can this reasonably block the release"?  I think this
should be probably allowed to delay the release for 1 month, but if
it's more than that (say, 6 months), it's probably best to release
without it.

 -George

Re: 4.2 TODO update

[Tim Deegan]

At 10:17 +0000 on 13 Feb (1329128265), Ian Campbell wrote:
> This weeks update. I was away last Monday so this covers two weeks worth
> of updates. I have dropped things which were marked as DONE in the last
> update and added a bunch more DONE (so the list is shrinking!). Please
> send me corrections (especially "done").
> 
> hypervisor, blockers:
>       * round-up of the closing of the security hole in MSI-X
>         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>         access to MSI-X table pages). (Jan Beulich -- more fixes
>         required than first thought, patches posted)
>       * domctls / sysctls set up to modify scheduler parameters, like
>         the credit1 timeslice and schedule rate. (George Dunlap)
>       * get the interface changes for sharing/paging/mem-events done and
>         dusted so that 4.2 is a stable API that we hold to. (Tim Deegan,
>         Andres Lagar-Cavilla et al)
>               * sharing patches posted

The hypervisor interface changes for sharing and events have gone in;
there's a libxc-level change (to do with mlock()ing buffers) that ought
to go in before 4.2, if at all. 

There's a paging+waitqueues patch that's been posted but needs a bit more
work still; I'll try and look at it on Thursday.

Tim.

4.2 TODO update

[Ian Campbell]

This weeks update. I was away last Monday so this covers two weeks worth
of updates. I have dropped things which were marked as DONE in the last
update and added a bunch more DONE (so the list is shrinking!). Please
send me corrections (especially "done").

hypervisor, blockers:
      * round-up of the closing of the security hole in MSI-X
        passthrough (uniformly - i.e. even for Dom0 - disallowing write
        access to MSI-X table pages). (Jan Beulich -- more fixes
        required than first thought, patches posted)
      * domctls / sysctls set up to modify scheduler parameters, like
        the credit1 timeslice and schedule rate. (George Dunlap)
      * get the interface changes for sharing/paging/mem-events done and
        dusted so that 4.2 is a stable API that we hold to. (Tim Deegan,
        Andres Lagar-Cavilla et al)
              * sharing patches posted

tools, blockers:

      * libxl stable API -- we would like 4.2 to define a stable API
        which downstream's can start to rely on not changing. Aspects of
        this are:
              * drop libxl_device_model_info (move bits to build_info or
                elsewhere as appropriate). (Ian Campbell, DONE).
              * add libxl_defbool and generally try and arrange that
                memset(foo,0,...) requests the defaults (Ian Campbell,
                repost pending)
              * topologyinfo datastructure should be a list of tuples,
                not a tuple of lists. (Ian Campbell, DONE)
              * xl to use json for machine readable output instead of
                sexp by default (Ian Campbell, DONE)
              * xl feature parity with xend wrt driver domain support
                (George Dunlap)
              * More formally deprecate xm/xend. Manpage patches already
                in tree. Needs release noting and communication around
                -rc1 to remind people to test xl.

hypervisor, nice to have:

      * solid implementation of sharing/paging/mem-events (using work
        queues) (Tim Deegan, Olaf Herring et al)
      * A long standing issue is a fully synchronized p2m (locking
        lookups) (Andres Lagar-Cavilla, patches posted?)

tools, nice to have:

      * Hotplug script stuff -- internal to libxl (I think, therefore I
        didn't put this under stable API above) but still good to have
        for 4.2? (Roger Pau Monet, patches posted)
      * Block script support -- follows on from hotplug script (Roger
        Pau Monet)
      * libyajl v2 support (Roger Pau Monet, DONE)
      * Configure/control paging via xl/libxl (Olaf Herring)
      * Upstream qemu feature patches:
              * Upstream qemu PCI passthrough support (Anthony Perard)
              * Upstream qemu save restore (Anthony Perard, Stefano
                Stabellini)
      * Nested-virtualisation (currently should be marked
        experimental,likely to release that way? Consider nested-svm
        separate to nested-vmx. Nested-svm is in better shape)
      * Initial xl support for Remus (memory checkpoint, blackholing)
        (Shriram, patches posted)

Tools, need to decide if pre- or post-4.2 feature:
              * Autoconf (Roger Pau Monet posted a patch, we'll probably
                take this for 4.2)