4.2 TODO update

4.2 TODO update

[Ian Campbell]

This weeks update. I was away last Monday so this covers two weeks worth
of updates. I have dropped things which were marked as DONE in the last
update and added a bunch more DONE (so the list is shrinking!). Please
send me corrections (especially "done").

hypervisor, blockers:
      * round-up of the closing of the security hole in MSI-X
        passthrough (uniformly - i.e. even for Dom0 - disallowing write
        access to MSI-X table pages). (Jan Beulich -- more fixes
        required than first thought, patches posted)
      * domctls / sysctls set up to modify scheduler parameters, like
        the credit1 timeslice and schedule rate. (George Dunlap)
      * get the interface changes for sharing/paging/mem-events done and
        dusted so that 4.2 is a stable API that we hold to. (Tim Deegan,
        Andres Lagar-Cavilla et al)
              * sharing patches posted

tools, blockers:

      * libxl stable API -- we would like 4.2 to define a stable API
        which downstream's can start to rely on not changing. Aspects of
        this are:
              * drop libxl_device_model_info (move bits to build_info or
                elsewhere as appropriate). (Ian Campbell, DONE).
              * add libxl_defbool and generally try and arrange that
                memset(foo,0,...) requests the defaults (Ian Campbell,
                repost pending)
              * topologyinfo datastructure should be a list of tuples,
                not a tuple of lists. (Ian Campbell, DONE)
              * xl to use json for machine readable output instead of
                sexp by default (Ian Campbell, DONE)
              * xl feature parity with xend wrt driver domain support
                (George Dunlap)
              * More formally deprecate xm/xend. Manpage patches already
                in tree. Needs release noting and communication around
                -rc1 to remind people to test xl.

hypervisor, nice to have:

      * solid implementation of sharing/paging/mem-events (using work
        queues) (Tim Deegan, Olaf Herring et al)
      * A long standing issue is a fully synchronized p2m (locking
        lookups) (Andres Lagar-Cavilla, patches posted?)

tools, nice to have:

      * Hotplug script stuff -- internal to libxl (I think, therefore I
        didn't put this under stable API above) but still good to have
        for 4.2? (Roger Pau Monet, patches posted)
      * Block script support -- follows on from hotplug script (Roger
        Pau Monet)
      * libyajl v2 support (Roger Pau Monet, DONE)
      * Configure/control paging via xl/libxl (Olaf Herring)
      * Upstream qemu feature patches:
              * Upstream qemu PCI passthrough support (Anthony Perard)
              * Upstream qemu save restore (Anthony Perard, Stefano
                Stabellini)
      * Nested-virtualisation (currently should be marked
        experimental,likely to release that way? Consider nested-svm
        separate to nested-vmx. Nested-svm is in better shape)
      * Initial xl support for Remus (memory checkpoint, blackholing)
        (Shriram, patches posted)

Tools, need to decide if pre- or post-4.2 feature:
              * Autoconf (Roger Pau Monet posted a patch, we'll probably
                take this for 4.2)

Re: 4.2 TODO update

[Tim Deegan]

At 10:17 +0000 on 13 Feb (1329128265), Ian Campbell wrote:
> This weeks update. I was away last Monday so this covers two weeks worth
> of updates. I have dropped things which were marked as DONE in the last
> update and added a bunch more DONE (so the list is shrinking!). Please
> send me corrections (especially "done").
> 
> hypervisor, blockers:
>       * round-up of the closing of the security hole in MSI-X
>         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>         access to MSI-X table pages). (Jan Beulich -- more fixes
>         required than first thought, patches posted)
>       * domctls / sysctls set up to modify scheduler parameters, like
>         the credit1 timeslice and schedule rate. (George Dunlap)
>       * get the interface changes for sharing/paging/mem-events done and
>         dusted so that 4.2 is a stable API that we hold to. (Tim Deegan,
>         Andres Lagar-Cavilla et al)
>               * sharing patches posted

The hypervisor interface changes for sharing and events have gone in;
there's a libxc-level change (to do with mlock()ing buffers) that ought
to go in before 4.2, if at all. 

There's a paging+waitqueues patch that's been posted but needs a bit more
work still; I'll try and look at it on Thursday.

Tim.

Re: 4.2 TODO update

[George Dunlap]

On Mon, Feb 13, 2012 at 10:17 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>      * domctls / sysctls set up to modify scheduler parameters, like
>        the credit1 timeslice and schedule rate. (George Dunlap)

Not done; I'll try to get this fixed up this week.

>              * xl feature parity with xend wrt driver domain support
>                (George Dunlap)

I probably won't have time to start on this for another two weeks or
so; it's not clear ATM how long it might take to get working after
that (since I haven't even taken a cursory look at it).  If anyone is
sufficiently motivated to take a look, they're welcome to do so. :-)

Re blocker status: I think blockers aren't exactly binary, but more
like "How long can this reasonably block the release"?  I think this
should be probably allowed to delay the release for 1 month, but if
it's more than that (say, 6 months), it's probably best to release
without it.

 -George

Re: 4.2 TODO update

[Fantu]

Thanks to all developer for good work, I think it would be good to add these:

tools, nice to have: 
...
      * Add network script stuff for Open Vswitch support
      * Upstream qemu feature patches:
         ...
         * Spice full support and build enabled by default


About Spice support on qemu upstream, now is not built by default and on xl
configuration files is minimal. Probably there are also some required missed
options about qxl graphic needed by Spice on xl configuration files.
I search on tools/libxl/libxl_dm.c but I have found nothing about qxl now.

In these days I'll try to build and use Spice, and eventually I'll post
patches about to improve Spice support.

Anyone got the above stuff already working? Any patches out there?

Thanks for any reply and sorry for bad english.

--
View this message in context: http://xen.1045712.n5.nabble.com/4-2-TODO-update-tp5478696p5478798.html
Sent from the Xen - Dev mailing list archive at Nabble.com.

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-02-13 at 11:05 +0000, Fantu wrote:
> Thanks to all developer for good work, I think it would be good to add these:
> 
> tools, nice to have: 
> ...
>       * Add network script stuff for Open Vswitch support

I made a stab at this a way back but fell into the exact sorts of issues
that the "Hotplug script stuff" (already on the TODO) is supposed to
address (mainly the lack of a suitable teardown hook which vswitch
needs). However once that stuff is done I reckon vswitch support would
be reasonably easy to add.

>       * Upstream qemu feature patches:
>          ...
>          * Spice full support and build enabled by default
> 
> 
> About Spice support on qemu upstream, now is not built by default and on xl
> configuration files is minimal. Probably there are also some required missed
> options about qxl graphic needed by Spice on xl configuration files.
> I search on tools/libxl/libxl_dm.c but I have found nothing about qxl now.
> 
> In these days I'll try to build and use Spice, and eventually I'll post
> patches about to improve Spice support.

Thanks, I'm looking forward to your patches.

> Anyone got the above stuff already working? Any patches out there?

Presumably the contributor who added the spice support to libxl had but
they had probably built qemu upstream by hand with the appropriate
options. I'm not aware of any outstanding SPICE patches.

Ian.

Re: 4.2 TODO update

[Jan Beulich]

>>> On 13.02.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> hypervisor, blockers:
>       * round-up of the closing of the security hole in MSI-X
>         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>         access to MSI-X table pages). (Jan Beulich -- more fixes
>         required than first thought, patches posted)

The only one currently open is the one removing write permission for
Dom0. The intention was to get this in after the qemu usptream pass
through patch series got adjusted along the lines of what was done
to qemu traditional, and while this was promise to happen soon after
New Year I didn't hear back anything from Anthony or Stefano.

Question is whether, given that the patch series in question isn't in
anything that is or will soon be released, it makes sense to push the
hypervisor change without waiting for that fixup.

Jan

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-02-13 at 11:29 +0000, Jan Beulich wrote:
> >>> On 13.02.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > hypervisor, blockers:
> >       * round-up of the closing of the security hole in MSI-X
> >         passthrough (uniformly - i.e. even for Dom0 - disallowing write
> >         access to MSI-X table pages). (Jan Beulich -- more fixes
> >         required than first thought, patches posted)
> 
> The only one currently open is the one removing write permission for
> Dom0.

Oh, I thought you had found another issue which required further
patches. Did I misunderstand or did they go in already?

>  The intention was to get this in after the qemu usptream pass
> through patch series got adjusted along the lines of what was done
> to qemu traditional, and while this was promise to happen soon after
> New Year I didn't hear back anything from Anthony or Stefano.
> 
> Question is whether, given that the patch series in question isn't in
> anything that is or will soon be released, it makes sense to push the
> hypervisor change without waiting for that fixup.

I don't think it is necessary to wait for the qemu-upstream patch series
to be updated for this, is it?

Ian.

Re: 4.2 TODO update

[Anthony PERARD]

On Mon, Feb 13, 2012 at 10:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>      * Upstream qemu feature patches:
>              * Upstream qemu PCI passthrough support (Anthony Perard)

I'll send a new patch set soon, but the power management capability
will be missing as it's need a bit more work.
Also, this patch series does not longuer include the msitranslate code.

>              * Upstream qemu save restore (Anthony Perard, Stefano
>                Stabellini)

Still need some ack from qemu's developper.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 4.2 TODO update

[Anthony PERARD]

On Mon, Feb 13, 2012 at 11:05, Fantu <fantonifabio@tiscali.it> wrote:
>         * Spice full support and build enabled by default
>
> About Spice support on qemu upstream, now is not built by default and on xl
> configuration files is minimal. Probably there are also some required missed
> options about qxl graphic needed by Spice on xl configuration files.
> I search on tools/libxl/libxl_dm.c but I have found nothing about qxl now.

About the build, QEMU should include spice support if the necessary
library are installed. But on Debian stable (Squeeze), the spice
library are too old, so forcing QEMU to have spice support will just
break the build.

Regards,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-02-13 at 11:50 +0000, Anthony PERARD wrote:
> On Mon, Feb 13, 2012 at 11:05, Fantu <fantonifabio@tiscali.it> wrote:
> >         * Spice full support and build enabled by default
> >
> > About Spice support on qemu upstream, now is not built by default and on xl
> > configuration files is minimal. Probably there are also some required missed
> > options about qxl graphic needed by Spice on xl configuration files.
> > I search on tools/libxl/libxl_dm.c but I have found nothing about qxl now.
> 
> About the build, QEMU should include spice support if the necessary
> library are installed. But on Debian stable (Squeeze), the spice
> library are too old, so forcing QEMU to have spice support will just
> break the build.

Agreed, spice support should be conditional on the necessary support
libraries being available. It would however be useful to document (e.g.
in the top-level README) exactly what those requirements are.

Ian.

Re: 4.2 TODO update

[Stefano Stabellini]

On Mon, 13 Feb 2012, Ian Campbell wrote:
>       * Upstream qemu feature patches:
>               * Upstream qemu PCI passthrough support (Anthony Perard)
>               * Upstream qemu save restore (Anthony Perard, Stefano
>                 Stabellini)

I have sent patches for both QEMU and xl/libxl, I have addressed all
comments but I haven't received any acks yet. I'll ping the maintainers
again.

Re: 4.2 TODO update

[Jan Beulich]

>>> On 13.02.12 at 12:32, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-02-13 at 11:29 +0000, Jan Beulich wrote:
>> >>> On 13.02.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > hypervisor, blockers:
>> >       * round-up of the closing of the security hole in MSI-X
>> >         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>> >         access to MSI-X table pages). (Jan Beulich -- more fixes
>> >         required than first thought, patches posted)
>> 
>> The only one currently open is the one removing write permission for
>> Dom0.
> 
> Oh, I thought you had found another issue which required further
> patches. Did I misunderstand or did they go in already?

They went in already.

>>  The intention was to get this in after the qemu usptream pass
>> through patch series got adjusted along the lines of what was done
>> to qemu traditional, and while this was promise to happen soon after
>> New Year I didn't hear back anything from Anthony or Stefano.
>> 
>> Question is whether, given that the patch series in question isn't in
>> anything that is or will soon be released, it makes sense to push the
>> hypervisor change without waiting for that fixup.
> 
> I don't think it is necessary to wait for the qemu-upstream patch series
> to be updated for this, is it?

Let's wait at least a day or two to give Stefano and Anthony a chance
to respond.

Jan

Re: 4.2 TODO update

[Stefano Stabellini]

On Mon, 13 Feb 2012, Jan Beulich wrote:
> >>  The intention was to get this in after the qemu usptream pass
> >> through patch series got adjusted along the lines of what was done
> >> to qemu traditional, and while this was promise to happen soon after
> >> New Year I didn't hear back anything from Anthony or Stefano.
> >> 
> >> Question is whether, given that the patch series in question isn't in
> >> anything that is or will soon be released, it makes sense to push the
> >> hypervisor change without waiting for that fixup.
> > 
> > I don't think it is necessary to wait for the qemu-upstream patch series
> > to be updated for this, is it?
> 
> Let's wait at least a day or two to give Stefano and Anthony a chance
> to respond.

I don't think we need to wait for the upstream PCI passthrough series to
be accepted in order for the hypervisor changes to go in, however it
would be nice if you could point out what needs to be changed on the
QEMU side, when Anthony posts the next version of the series.

Re: 4.2 TODO update

[Shriram Rajagopalan]

[-- Attachment #1.1: Type: text/plain, Size: 1499 bytes --]

On Mon, Feb 13, 2012 at 2:17 AM, Ian Campbell <Ian.Campbell@citrix.com>wrote:

>
> tools, nice to have:
>
>      * Hotplug script stuff -- internal to libxl (I think, therefore I
>        didn't put this under stable API above) but still good to have
>        for 4.2? (Roger Pau Monet, patches posted)
>      * Block script support -- follows on from hotplug script (Roger
>        Pau Monet)
>      * libyajl v2 support (Roger Pau Monet, DONE)
>      * Configure/control paging via xl/libxl (Olaf Herring)
>      * Upstream qemu feature patches:
>              * Upstream qemu PCI passthrough support (Anthony Perard)
>              * Upstream qemu save restore (Anthony Perard, Stefano
>                Stabellini)
>      * Nested-virtualisation (currently should be marked
>        experimental,likely to release that way? Consider nested-svm
>        separate to nested-vmx. Nested-svm is in better shape)
>      * Initial xl support for Remus (memory checkpoint, blackholing)
>        (Shriram, patches posted)
>
>


FYI, the network buffering module which was part of 2.6.32 pvops tree has
been accepted into upstream net-next tree. I have also posted patches for
the
userspace libnl code that talks to the qdiscs via netlink.

I am just waiting for the initial xl patches to go in, after which I could
take a stab
at adding network buffering support. Depending on whether Roger's hotplug
patches
go in or not, I could add some rudimentary disk replication support, time
permitting.

shriram

[-- Attachment #1.2: Type: text/html, Size: 1876 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 4.2 TODO update

[Jan Beulich]

>>> On 13.02.12 at 12:32, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-02-13 at 11:29 +0000, Jan Beulich wrote:
>> >>> On 13.02.12 at 11:17, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > hypervisor, blockers:
>> >       * round-up of the closing of the security hole in MSI-X
>> >         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>> >         access to MSI-X table pages). (Jan Beulich -- more fixes
>> >         required than first thought, patches posted)
>> 
>> The only one currently open is the one removing write permission for
>> Dom0.
> 
> Oh, I thought you had found another issue which required further
> patches. Did I misunderstand or did they go in already?
> 
>>  The intention was to get this in after the qemu usptream pass
>> through patch series got adjusted along the lines of what was done
>> to qemu traditional, and while this was promise to happen soon after
>> New Year I didn't hear back anything from Anthony or Stefano.
>> 
>> Question is whether, given that the patch series in question isn't in
>> anything that is or will soon be released, it makes sense to push the
>> hypervisor change without waiting for that fixup.
> 
> I don't think it is necessary to wait for the qemu-upstream patch series
> to be updated for this, is it?

Patch just sent out. Once that's in, the issue should be fully addressed
for HVM guests (the situation with PV guests is explained in the patch).

Jan

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-02-13 at 10:17 +0000, Ian Campbell wrote:
> 
> tools, nice to have: 

Jim Burns reports that

	vncviewer=1 used to auto spawn a vnc viewer for you

and that

	localtime and rtc_timeoffset do not work with xl.

Both seem like worthwhile things to have in some form.

Ian.

Re: 4.2 TODO update

[Michael A. Collins]

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> bounces@lists.xensource.com] On Behalf Of Ian Campbell
> Sent: Monday, February 13, 2012 6:56 AM
> To: Anthony PERARD
> Cc: xen-devel@lists.xensource.com; Fantu
> Subject: Re: [Xen-devel] 4.2 TODO update
> 
> On Mon, 2012-02-13 at 11:50 +0000, Anthony PERARD wrote:
> > On Mon, Feb 13, 2012 at 11:05, Fantu <fantonifabio@tiscali.it> wrote:
> > >         * Spice full support and build enabled by default
> > >
> > > About Spice support on qemu upstream, now is not built by default and
> on xl
> > > configuration files is minimal. Probably there are also some required
> missed
> > > options about qxl graphic needed by Spice on xl configuration files.
> > > I search on tools/libxl/libxl_dm.c but I have found nothing about qxl
now.
> >
> > About the build, QEMU should include spice support if the necessary
> > library are installed. But on Debian stable (Squeeze), the spice
> > library are too old, so forcing QEMU to have spice support will just
> > break the build.
> 
> Agreed, spice support should be conditional on the necessary support
> libraries being available. It would however be useful to document (e.g.
> in the top-level README) exactly what those requirements are.
> 
> Ian.
> 

Is it my understanding that currently there is no way to pass the -vga
option to qemu using the xl cfg file?  This seems in my mind a very trivial
thing to do, I don't want to reinvent the wheel if I'm just missing
something.
Mike

Re: 4.2 TODO update

[Ian Campbell]

On Tue, 2012-02-21 at 02:38 +0000, Michael A. Collins wrote:
> Is it my understanding that currently there is no way to pass the -vga
> option to qemu using the xl cfg file?  This seems in my mind a very trivial
> thing to do, I don't want to reinvent the wheel if I'm just missing
> something.

It seems you can arrange for "-std-vga" or "-vga std" (depending on if
you are using traditional or upstream qemu) or nothing but nothing more
complex than that.

What is it that you are trying to do by passing such an option?

Do you know if this this option was exposed in xend? If so then how?

(as a workaround there is an device_model_args{,_pv,_hvm} trio of
options in xl which allows arbitrary option to be passed to qemu)

Ian.

Re: 4.2 TODO update

[Ian Campbell]

On Wed, 2012-03-14 at 16:48 +0000, Dario Faggioli wrote:
> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> > This update covers two weeks since I was away last week.
> > 
> Hi,
> 
> I think the below should be considered too, and I'm proposing it as a
> (tools) blocker, as I really think it affects API stability.

Sure.

> I can take care of it, provided we have some preliminary
> design-discussion. :-D

A good starting point for discussion would be a proposed patch to
libxl.h with associated inline documentation.

>  
> > tools, blockers:
> >       * libxl stable API -- we would like 4.2 to define a stable API
> >         which downstream's can start to rely on not changing. Aspects of
> >         this are:
> >               * add libxl_defbool and generally try and arrange that
> >                 memset(foo,0,...) requests the defaults (Ian Campbell,
> >                 DONE)
> >               * Safe fork vs. fd handling hooks. This is an API
> >                 addition, so maybe not required fro stable API, bit need
> >                 to have for 4.2? (Ian J, patches posted)
> >
>   * locking/serialization, e.g., for domain creation. As of now, 
>     nothing is provided for this purpose, and downstream toolstacks have
>     to put their own mechanisms in place. E.g., xl uses a fcntl() lock  
>     around the whole process of domain creation. It should OTOH be 
>     libxl job to offer a set of hooks --properly placed within the
>     domain creation process-- for the downstreams to fill with the
>     proper callbacks. (Dario Faggioli)
> 
> Thanks and Regards,
> Dario
> 

Re: 4.2 TODO update

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1738 bytes --]

On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> This update covers two weeks since I was away last week.
> 
Hi,

I think the below should be considered too, and I'm proposing it as a
(tools) blocker, as I really think it affects API stability.

I can take care of it, provided we have some preliminary
design-discussion. :-D
 
> tools, blockers:
>       * libxl stable API -- we would like 4.2 to define a stable API
>         which downstream's can start to rely on not changing. Aspects of
>         this are:
>               * add libxl_defbool and generally try and arrange that
>                 memset(foo,0,...) requests the defaults (Ian Campbell,
>                 DONE)
>               * Safe fork vs. fd handling hooks. This is an API
>                 addition, so maybe not required fro stable API, bit need
>                 to have for 4.2? (Ian J, patches posted)
>
  * locking/serialization, e.g., for domain creation. As of now, 
    nothing is provided for this purpose, and downstream toolstacks have
    to put their own mechanisms in place. E.g., xl uses a fcntl() lock  
    around the whole process of domain creation. It should OTOH be 
    libxl job to offer a set of hooks --properly placed within the
    domain creation process-- for the downstreams to fill with the
    proper callbacks. (Dario Faggioli)

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa (Italy)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[George Dunlap]

On Mon, Mar 12, 2012 at 4:42 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-03-12 at 16:36 +0000, George Dunlap wrote:
>> On Mon, Mar 12, 2012 at 12:11 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > This update covers two weeks since I was away last week.
>> >
>> > As usual please ping me with any updates/corrections. There are several
>> > things below which are in need of a volunteer to take care of them...
>> >
>> > hypervisor, blockers:
>> >      * domctls / sysctls set up to modify scheduler parameters, like
>> >        the credit1 timeslice and schedule rate. (George Dunlap, DONE)
>>
>> I recently discovered a set of patches updating the PoD code in the
>> XenServer patchqueue which I was meant to have upstreamed but never
>> actually got to.  Can I add this to the "blockers" list?
>
> If we've been living without them are they really blockers? Adding them
> to the nice to have list seems like a no brainer but a few words on why
> they are blockers could easily upgrade my assessment ;-)

As you wish; I think it is the case that at the moment they're just
for performance, and they're just for booting a VM anyway.  But it's
certainly *my* goal to get them out before 4.2. :-)

 -George

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-03-12 at 16:01 +0000, Stefano Stabellini wrote:
> On Mon, 12 Mar 2012, Ian Campbell wrote:
> >>       * Domain 0 block attach & general hotplug when using qdisk backend
> >         (need to start qemu as necessary etc)
> 
> I should be able to look into this

Thanks!

Ian

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-03-12 at 16:36 +0000, George Dunlap wrote:
> On Mon, Mar 12, 2012 at 12:11 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > This update covers two weeks since I was away last week.
> >
> > As usual please ping me with any updates/corrections. There are several
> > things below which are in need of a volunteer to take care of them...
> >
> > hypervisor, blockers:
> >      * domctls / sysctls set up to modify scheduler parameters, like
> >        the credit1 timeslice and schedule rate. (George Dunlap, DONE)
> 
> I recently discovered a set of patches updating the PoD code in the
> XenServer patchqueue which I was meant to have upstreamed but never
> actually got to.  Can I add this to the "blockers" list?

If we've been living without them are they really blockers? Adding them
to the nice to have list seems like a no brainer but a few words on why
they are blockers could easily upgrade my assessment ;-)

Ian.

Re: 4.2 TODO update

[George Dunlap]

On Mon, Mar 12, 2012 at 12:11 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> This update covers two weeks since I was away last week.
>
> As usual please ping me with any updates/corrections. There are several
> things below which are in need of a volunteer to take care of them...
>
> hypervisor, blockers:
>      * domctls / sysctls set up to modify scheduler parameters, like
>        the credit1 timeslice and schedule rate. (George Dunlap, DONE)

I recently discovered a set of patches updating the PoD code in the
XenServer patchqueue which I was meant to have upstreamed but never
actually got to.  Can I add this to the "blockers" list?  It involves
changes based on some experimentation; and while I think it may be a
few days worth of work to port (with all the p2m reorganization done
after the 4.1 release), but it should be a relatively known quantity.

>
> tools, blockers:
>      * libxl stable API -- we would like 4.2 to define a stable API
>        which downstream's can start to rely on not changing. Aspects of
>        this are:
>              * add libxl_defbool and generally try and arrange that
>                memset(foo,0,...) requests the defaults (Ian Campbell,
>                DONE)
>              * Safe fork vs. fd handling hooks. This is an API
>                addition, so maybe not required fro stable API, bit need
>                to have for 4.2? (Ian J, patches posted)
>      * xl feature parity with xend wrt driver domain support (George
>        Dunlap)
>      * More formally deprecate xm/xend. Manpage patches already in
>        tree. Needs release noting and communication around -rc1 to
>        remind people to test xl.
>      * Correct paging/sharing tools buffer mlocking (Tim, Andres, DONE)
>      * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
>      * Domain 0 block attach & general hotplug when using qdisk backend
>        (need to start qemu as necessary etc)
>      * file:// backend performance. qemu-xen-tradition's qdisk is quite
>        slow & blktap2 not available in upstream kernels. Need to
>        consider our options:
>              * qemu-xen's qdisk is thought to be well performing but
>                qemu-xen is not yet the default. Complexity arising from
>                splitting qemu-for-qdisk out from qemu-for-dm and
>                running N qemu's.
>              * potentially fully userspace blktap could be ready for
>                4.2
>              * use /dev/loop+blkback. This requires loop driver AIO and
>                O_DIRECT patches which are not (AFAIK) yet upstream.
>              * Leverage XCP's blktap2 DKMS work.
>              * Other ideas?
>
> hypervisor, nice to have:
>      * solid implementation of sharing/paging/mem-events (using work
>        queues) (Tim Deegan, Olaf Herring et al -- is this happening? is
>        it DONE even?)
>
> tools, nice to have:
>      * Hotplug script stuff -- internal to libxl (I think, therefore I
>        didn't put this under stable API above) but still good to have
>        for 4.2? (Roger Pau Monné, patches posted)
>      * Block script support -- follows on from hotplug script (Roger
>        Pau Monné)
>      * Configure/control paging via xl/libxl (Olaf Herring, lots of
>        discussion around interface)
>      * Upstream qemu feature patches:
>              * Upstream qemu PCI passthrough support (Anthony Perard,
>                patches sent)
>              * Upstream qemu save restore (Anthony Perard, Stefano
>                Stabellini, patches sent, waiting for upstream ack)
>      * Nested-virtualisation (currently should be marked experimental,
>        likely to release that way? Consider nested-svm separate to
>        nested-vmx. Nested-svm is in better shape)
>      * Initial xl support for Remus (memory checkpoint, blackholing)
>        (Shriram, patches posted, blocked behind qemu save restore
>        patches)
>      * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
>        (Nobody AFAICT)
>      * support for vif "rate" parameter (No one, AFAICT)
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[Stefano Stabellini]

On Mon, 12 Mar 2012, Ian Campbell wrote:
> This update covers two weeks since I was away last week.
> 
> As usual please ping me with any updates/corrections. There are several
> things below which are in need of a volunteer to take care of them...
> 
> hypervisor, blockers:
>       * domctls / sysctls set up to modify scheduler parameters, like
>         the credit1 timeslice and schedule rate. (George Dunlap, DONE)
>  
> tools, blockers:
>       * libxl stable API -- we would like 4.2 to define a stable API
>         which downstream's can start to rely on not changing. Aspects of
>         this are:
>               * add libxl_defbool and generally try and arrange that
>                 memset(foo,0,...) requests the defaults (Ian Campbell,
>                 DONE)
>               * Safe fork vs. fd handling hooks. This is an API
>                 addition, so maybe not required fro stable API, bit need
>                 to have for 4.2? (Ian J, patches posted)
>       * xl feature parity with xend wrt driver domain support (George
>         Dunlap)
>       * More formally deprecate xm/xend. Manpage patches already in
>         tree. Needs release noting and communication around -rc1 to
>         remind people to test xl.
>       * Correct paging/sharing tools buffer mlocking (Tim, Andres, DONE)
>       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
>       * Domain 0 block attach & general hotplug when using qdisk backend
>         (need to start qemu as necessary etc)

I should be able to look into this

>       * file:// backend performance. qemu-xen-tradition's qdisk is quite
>         slow & blktap2 not available in upstream kernels. Need to
>         consider our options:
>               * qemu-xen's qdisk is thought to be well performing but
>                 qemu-xen is not yet the default. Complexity arising from
>                 splitting qemu-for-qdisk out from qemu-for-dm and
>                 running N qemu's.
>               * potentially fully userspace blktap could be ready for
>                 4.2
>               * use /dev/loop+blkback. This requires loop driver AIO and
>                 O_DIRECT patches which are not (AFAIK) yet upstream.
>               * Leverage XCP's blktap2 DKMS work.
>               * Other ideas?

Considering how long these patches have been outstanding (years), it
might be wise to design a solution that does not require them.

In particular I think we should either go with userblktap or qdisk.
Qdisk could be a nice fallback, not too hard to code, if we don't
get userblktap in time.

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-03-12 at 13:51 +0000, Jan Beulich wrote:
> >>> On 12.03.12 at 14:42, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> >> tools, nice to have:
> >>       * Hotplug script stuff -- internal to libxl (I think, therefore I
> >>         didn't put this under stable API above) but still good to have
> >>         for 4.2? (Roger Pau Monné, patches posted)
> >>       * Block script support -- follows on from hotplug script (Roger
> >>         Pau Monné)
> > 
> > Any opinions on whether this, and in particular the block script bits,
> > should be blockers for 4.2?
> > 
> > I don't use block-script functionality myself so I'm not sure how
> > critical it is.
> > 
> > I guess there is always the workaround of doing whatever the script
> > would do outside of xl which is why I think (IIRC) I initially put it
> > under nice to have.
> > 
> > Opinions?
> 
> If it's this feature that you referred to when discussing the "xl
> block-attach 0 ..." shortcomings (including the blkback-over-loop
> that's currently unavailable with xl), then count this as a "+1"
> for this being a blocker - if we want 4.2 to push forward the
> deprecation status of xend, then (known) regressions like this
> should get eliminated.

I actually included the specific use case of block-attach vs qdisk on
the TODO in its own right and separately the handling of file:// too --
it turns out that loopback may not be the best answer (see Joseph
Glanville's posts about loop and AIO/O_DIRECT) so block script may not
actually be all that helpful there.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[Roger Pau Monné]

2012/3/12 Ian Campbell <Ian.Campbell@citrix.com>:
> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
>> tools, nice to have:
>>       * Hotplug script stuff -- internal to libxl (I think, therefore I
>>         didn't put this under stable API above) but still good to have
>>         for 4.2? (Roger Pau Monné, patches posted)
>>       * Block script support -- follows on from hotplug script (Roger
>>         Pau Monné)
>
> Any opinions on whether this, and in particular the block script bits,
> should be blockers for 4.2?
>
> I don't use block-script functionality myself so I'm not sure how
> critical it is.
>
> I guess there is always the workaround of doing whatever the script
> would do outside of xl which is why I think (IIRC) I initially put it
> under nice to have.
>
> Opinions?

I would really like to be able to send the updated driver-domain
series (using Ian Jackson new event interface), but I'm not sure if I
will be able to do so until the next week, sorry.

> Ian.
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[Jan Beulich]

>>> On 12.03.12 at 14:42, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
>> tools, nice to have:
>>       * Hotplug script stuff -- internal to libxl (I think, therefore I
>>         didn't put this under stable API above) but still good to have
>>         for 4.2? (Roger Pau Monné, patches posted)
>>       * Block script support -- follows on from hotplug script (Roger
>>         Pau Monné)
> 
> Any opinions on whether this, and in particular the block script bits,
> should be blockers for 4.2?
> 
> I don't use block-script functionality myself so I'm not sure how
> critical it is.
> 
> I guess there is always the workaround of doing whatever the script
> would do outside of xl which is why I think (IIRC) I initially put it
> under nice to have.
> 
> Opinions?

If it's this feature that you referred to when discussing the "xl
block-attach 0 ..." shortcomings (including the blkback-over-loop
that's currently unavailable with xl), then count this as a "+1"
for this being a blocker - if we want 4.2 to push forward the
deprecation status of xend, then (known) regressions like this
should get eliminated.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[Ian Campbell]

On Mon, 2012-03-12 at 12:11 +0000, Ian Campbell wrote:
> tools, nice to have:
>       * Hotplug script stuff -- internal to libxl (I think, therefore I
>         didn't put this under stable API above) but still good to have
>         for 4.2? (Roger Pau Monné, patches posted)
>       * Block script support -- follows on from hotplug script (Roger
>         Pau Monné)

Any opinions on whether this, and in particular the block script bits,
should be blockers for 4.2?

I don't use block-script functionality myself so I'm not sure how
critical it is.

I guess there is always the workaround of doing whatever the script
would do outside of xl which is why I think (IIRC) I initially put it
under nice to have.

Opinions?

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

4.2 TODO update

[Ian Campbell]

This update covers two weeks since I was away last week.

As usual please ping me with any updates/corrections. There are several
things below which are in need of a volunteer to take care of them...

hypervisor, blockers:
      * domctls / sysctls set up to modify scheduler parameters, like
        the credit1 timeslice and schedule rate. (George Dunlap, DONE)
 
tools, blockers:
      * libxl stable API -- we would like 4.2 to define a stable API
        which downstream's can start to rely on not changing. Aspects of
        this are:
              * add libxl_defbool and generally try and arrange that
                memset(foo,0,...) requests the defaults (Ian Campbell,
                DONE)
              * Safe fork vs. fd handling hooks. This is an API
                addition, so maybe not required fro stable API, bit need
                to have for 4.2? (Ian J, patches posted)
      * xl feature parity with xend wrt driver domain support (George
        Dunlap)
      * More formally deprecate xm/xend. Manpage patches already in
        tree. Needs release noting and communication around -rc1 to
        remind people to test xl.
      * Correct paging/sharing tools buffer mlocking (Tim, Andres, DONE)
      * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
      * Domain 0 block attach & general hotplug when using qdisk backend
        (need to start qemu as necessary etc)
      * file:// backend performance. qemu-xen-tradition's qdisk is quite
        slow & blktap2 not available in upstream kernels. Need to
        consider our options:
              * qemu-xen's qdisk is thought to be well performing but
                qemu-xen is not yet the default. Complexity arising from
                splitting qemu-for-qdisk out from qemu-for-dm and
                running N qemu's.
              * potentially fully userspace blktap could be ready for
                4.2
              * use /dev/loop+blkback. This requires loop driver AIO and
                O_DIRECT patches which are not (AFAIK) yet upstream.
              * Leverage XCP's blktap2 DKMS work.
              * Other ideas?

hypervisor, nice to have:
      * solid implementation of sharing/paging/mem-events (using work
        queues) (Tim Deegan, Olaf Herring et al -- is this happening? is
        it DONE even?)

tools, nice to have:
      * Hotplug script stuff -- internal to libxl (I think, therefore I
        didn't put this under stable API above) but still good to have
        for 4.2? (Roger Pau Monné, patches posted)
      * Block script support -- follows on from hotplug script (Roger
        Pau Monné)
      * Configure/control paging via xl/libxl (Olaf Herring, lots of
        discussion around interface)
      * Upstream qemu feature patches:
              * Upstream qemu PCI passthrough support (Anthony Perard,
                patches sent)
              * Upstream qemu save restore (Anthony Perard, Stefano
                Stabellini, patches sent, waiting for upstream ack)
      * Nested-virtualisation (currently should be marked experimental,
        likely to release that way? Consider nested-svm separate to
        nested-vmx. Nested-svm is in better shape)
      * Initial xl support for Remus (memory checkpoint, blackholing)
        (Shriram, patches posted, blocked behind qemu save restore
        patches)
      * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
        (Nobody AFAICT)
      * support for vif "rate" parameter (No one, AFAICT)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 4.2 TODO update

[Andres Lagar-Cavilla]

> Date: Mon, 20 Feb 2012 15:52:01 +0000
> From: Ian Campbell <Ian.Campbell@citrix.com>
> To: xen-devel <xen-devel@lists.xensource.com>
> Subject: [Xen-devel] 4.2 TODO update
> Message-ID: <1329753121.3990.67.camel@zakaz.uk.xensource.com>
> Content-Type: text/plain; charset="UTF-8"
>
> This weeks update. As usual please ping me with any updates.
>
> hypervisor, blockers:
>       * round-up of the closing of the security hole in MSI-X
>         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>         access to MSI-X table pages). (Jan Beulich, DONE)
>       * domctls / sysctls set up to modify scheduler parameters, like
>         the credit1 timeslice and schedule rate. (George Dunlap)
>       * get the interface changes for sharing/paging/mem-events done and
>         dusted so that 4.2 is a stable API that we hold to. (Tim Deegan,
>         Andres Lagar-Cavilla et al)
>               * sharing patches posted (DONE)
>
> tools, blockers:
>
>       * libxl stable API -- we would like 4.2 to define a stable API
>         which downstream's can start to rely on not changing. Aspects of
>         this are:
>               * add libxl_defbool and generally try and arrange that
>                 memset(foo,0,...) requests the defaults (Ian Campbell,
>                 patches reposted)
>               * Safe fork vs. fd handling hooks. This is an API
>                 addition, so maybe not required fro stable API, bit need
>                 to have for 4.2? (Ian J)
>               * xl feature parity with xend wrt driver domain support
>                 (George Dunlap)
>               * More formally deprecate xm/xend. Manpage patches already
>                 in tree. Needs release noting and communication around
>                 -rc1 to remind people to test xl.
>       * Correct paging/sharing tools buffer mlocking (Tim, Andres)
Will post later in the week.

>       * Autoconf (Roger Pau Monn? & Ian J, blocked on test
>         infrastructure changes, Roger to respin patch when test system
>         ready for new features)
>       * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)
>
> hypervisor, nice to have:
>
>       * solid implementation of sharing/paging/mem-events (using work
>         queues) (Tim Deegan, Olaf Herring et al)
>       * A long standing issue is a fully synchronized p2m (locking
>         lookups) (Andres Lagar-Cavilla, patches posted?)
This one is DONE

Thanks,
Andres
>
> tools, nice to have:
>
>       * Hotplug script stuff -- internal to libxl (I think, therefore I
>         didn't put this under stable API above) but still good to have
>         for 4.2? (Roger Pau Monn?, patches posted)
>       * Block script support -- follows on from hotplug script (Roger
>         Pau Monn?)
>       * Configure/control paging via xl/libxl (Olaf Herring)
>       * Upstream qemu feature patches:
>               * Upstream qemu PCI passthrough support (Anthony Perard,
>                 patches sent)
>               * Upstream qemu save restore (Anthony Perard, Stefano
>                 Stabellini, patches sent, waiting for upstream ack)
>       * Nested-virtualisation (currently should be marked experimental,
>         likely to release that way? Consider nested-svm separate to
>         nested-vmx. Nested-svm is in better shape)
>       * Initial xl support for Remus (memory checkpoint, blackholing)
>         (Shriram, patches posted)
>       * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
>         (Nobody AFAICT)
>
>

4.2 TODO update

[Ian Campbell]

This weeks update. As usual please ping me with any updates.

hypervisor, blockers:
      * round-up of the closing of the security hole in MSI-X
        passthrough (uniformly - i.e. even for Dom0 - disallowing write
        access to MSI-X table pages). (Jan Beulich, DONE)
      * domctls / sysctls set up to modify scheduler parameters, like
        the credit1 timeslice and schedule rate. (George Dunlap)
      * get the interface changes for sharing/paging/mem-events done and
        dusted so that 4.2 is a stable API that we hold to. (Tim Deegan,
        Andres Lagar-Cavilla et al)
              * sharing patches posted (DONE)

tools, blockers:

      * libxl stable API -- we would like 4.2 to define a stable API
        which downstream's can start to rely on not changing. Aspects of
        this are:
              * add libxl_defbool and generally try and arrange that
                memset(foo,0,...) requests the defaults (Ian Campbell,
                patches reposted)
              * Safe fork vs. fd handling hooks. This is an API
                addition, so maybe not required fro stable API, bit need
                to have for 4.2? (Ian J)
              * xl feature parity with xend wrt driver domain support
                (George Dunlap)
              * More formally deprecate xm/xend. Manpage patches already
                in tree. Needs release noting and communication around
                -rc1 to remind people to test xl.
      * Correct paging/sharing tools buffer mlocking (Tim, Andres)
      * Autoconf (Roger Pau Monné & Ian J, blocked on test
        infrastructure changes, Roger to respin patch when test system
        ready for new features)
      * xl support for "rtc_timeoffset" and "localtime" (Nobody AFAICT)

hypervisor, nice to have:

      * solid implementation of sharing/paging/mem-events (using work
        queues) (Tim Deegan, Olaf Herring et al)
      * A long standing issue is a fully synchronized p2m (locking
        lookups) (Andres Lagar-Cavilla, patches posted?)

tools, nice to have:

      * Hotplug script stuff -- internal to libxl (I think, therefore I
        didn't put this under stable API above) but still good to have
        for 4.2? (Roger Pau Monné, patches posted)
      * Block script support -- follows on from hotplug script (Roger
        Pau Monné)
      * Configure/control paging via xl/libxl (Olaf Herring)
      * Upstream qemu feature patches:
              * Upstream qemu PCI passthrough support (Anthony Perard,
                patches sent)
              * Upstream qemu save restore (Anthony Perard, Stefano
                Stabellini, patches sent, waiting for upstream ack)
      * Nested-virtualisation (currently should be marked experimental,
        likely to release that way? Consider nested-svm separate to
        nested-vmx. Nested-svm is in better shape)
      * Initial xl support for Remus (memory checkpoint, blackholing)
        (Shriram, patches posted)
      * xl support for autospawning vncviewer (vncviewer=1 or otherwise)
        (Nobody AFAICT)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 4.2 TODO update

[Tim Deegan]

At 08:22 -0800 on 15 Feb (1329294161), Andres Lagar-Cavilla wrote:
> > Maybe we can arrange that instead of bugging out if the cpu is
> > in_atomic() it gdprintk()s a big ol' warning and crashes the guest?  It
> > seems no worse than the current failure modes.
> 
> How about judiciously adding the following
> 
> get_gfn_sleep(d, gfn, type)
> {
>   if (d == current_domain && !in_atomic())
>   {
>     printk("Naughty");
>     crash_domain(d);
>     return INVALID_MFN;
>   }

Yes, that's the sort of thing I had in mind (though the in_atomic() test
shouldn't be inverted).

I'll dig out Olaf's most recent patch tomorrow and see how that would
work; I'm travelling so my access to test hardware is a bit limited but
I'll try to at least make a draft patch.

Tim.

Re: 4.2 TODO update

[Andres Lagar-Cavilla]

> At 08:47 -0800 on 14 Feb (1329209245), Andres Lagar-Cavilla wrote:
>> > On Tue, Feb 14, Andres Lagar-Cavilla wrote:
>> >
>> >> Why? Because it's really really hard to guarantee we'll go to sleep
>> in
>> >> an
>> >> atomic context. The main use for wait queues (imho) is in hvm_copy,
>> and
>> >> there's a zillion paths going into hvm_copy (copy_from/to_user!) with
>> >> all
>> >> ways of bumping the preemption count.
>> >
>> > If the guests pagetable is paged out this code path will trigger, then
>> > one of the hypercalls returns an error and the guest runs into a
>> BUG().
>> > I think it was decrease_reservation, or similar.
>>
>> Unlikely to be something specific about decrease_reservation. If the
>> guest
>> page table is paged out, then copy_from_user for any hypercall, or,
>> "virtual address to gfn" for any emulation will run into this.
>>
>> Now, even an innocent-looking rcu lock anywhere in this code path will
>> crash the host if we go into a wait queue. Hence my concern.
>
> OK, so the current code breaks the guest and you're worried about it
> crashing the host instead.   That seems fair.
>
> Maybe we can arrange that instead of bugging out if the cpu is
> in_atomic() it gdprintk()s a big ol' warning and crashes the guest?  It
> seems no worse than the current failure modes.

How about judiciously adding the following

get_gfn_sleep(d, gfn, type)
{
  if (d == current_domain && !in_atomic())
  {
    printk("Naughty");
    crash_domain(d);
    return INVALID_MFN;
  }

retry:
  mfn rval = get_gfn(d, gfn, type)
  if (d == current->domain && type == paging && !mfn_valid(mfn))
  {
    put_gfn(d, gfn);
    go to sleep on wait queue;
    goto retry;
  }

  return rval;
}

Then we could toss it in before vmx_load_pdptrs gets to do evil things,
for example.

Andres

>
>> > What other way exist to make paging 100% transparent to the guest?
>>
>> Don't page out page table pages? I know you were not expecting that...
>
> Sadly, it's not possible to reliably detect which pages are pagetables
> (or the shadow pagetable code would be more straightforward!).
>
> Cheers,
>
> Tim.
>

Re: 4.2 TODO update

[Tim Deegan]

At 08:47 -0800 on 14 Feb (1329209245), Andres Lagar-Cavilla wrote:
> > On Tue, Feb 14, Andres Lagar-Cavilla wrote:
> >
> >> Why? Because it's really really hard to guarantee we'll go to sleep in
> >> an
> >> atomic context. The main use for wait queues (imho) is in hvm_copy, and
> >> there's a zillion paths going into hvm_copy (copy_from/to_user!) with
> >> all
> >> ways of bumping the preemption count.
> >
> > If the guests pagetable is paged out this code path will trigger, then
> > one of the hypercalls returns an error and the guest runs into a BUG().
> > I think it was decrease_reservation, or similar.
> 
> Unlikely to be something specific about decrease_reservation. If the guest
> page table is paged out, then copy_from_user for any hypercall, or,
> "virtual address to gfn" for any emulation will run into this.
> 
> Now, even an innocent-looking rcu lock anywhere in this code path will
> crash the host if we go into a wait queue. Hence my concern.

OK, so the current code breaks the guest and you're worried about it
crashing the host instead.   That seems fair.

Maybe we can arrange that instead of bugging out if the cpu is
in_atomic() it gdprintk()s a big ol' warning and crashes the guest?  It
seems no worse than the current failure modes.

> > What other way exist to make paging 100% transparent to the guest?
> 
> Don't page out page table pages? I know you were not expecting that...

Sadly, it's not possible to reliably detect which pages are pagetables
(or the shadow pagetable code would be more straightforward!).

Cheers,

Tim.

Re: 4.2 TODO update

[Andres Lagar-Cavilla]

> On Tue, Feb 14, Andres Lagar-Cavilla wrote:
>
>> > On Tue, Feb 14, Andres Lagar-Cavilla wrote:
>> >
>> >> Why? Because it's really really hard to guarantee we'll go to sleep
>> in
>> >> an
>> >> atomic context. The main use for wait queues (imho) is in hvm_copy,
>> and
>> >> there's a zillion paths going into hvm_copy (copy_from/to_user!) with
>> >> all
>> >> ways of bumping the preemption count.
>> >
>> > If the guests pagetable is paged out this code path will trigger, then
>> > one of the hypercalls returns an error and the guest runs into a
>> BUG().
>> > I think it was decrease_reservation, or similar.
>>
>> Unlikely to be something specific about decrease_reservation. If the
>> guest
>> page table is paged out, then copy_from_user for any hypercall, or,
>> "virtual address to gfn" for any emulation will run into this.
>>
>> Now, even an innocent-looking rcu lock anywhere in this code path will
>> crash the host if we go into a wait queue. Hence my concern.
>
> The workaround for the guest crash I were seeing:
> http://lists.xen.org/archives/html/xen-devel/2010-11/msg01609.html

This makes sense. It's basically what emulation does (X86_EMUL_RETRY). Great!

>
> I once modified xenpaging to keep the pagetables paged out as much as it
> could (and still allowed the guest to make at least a little bit
> progress) and run into no appearent issue.
>
>> > Another thing reported by Huawei on this list was somewhere in the
>> > emulation code where a gfn_to_mfn() failed.
>>
>> Can you point to the original report? Is there anything more specific?
>
> It was vmx_load_pdptrs().
> http://lists.xen.org/archives/html/xen-devel/2011-09/msg01336.html

I have a patch queued up for cleaning up vmx_load_pdptrs so that it
doesn't crash the host if the cr3 is paged out for a PAE guest.

You can't really do what they propose (mdelay, retry). You're in a
hypervisor context holding locks, nothing is getting scheduled!

One option worth considering is testing the cr3 "early enough", when it's
safe to sleep.

>
>> > What other way exist to make paging 100% transparent to the guest?
>> >
>>
>> Don't page out page table pages? I know you were not expecting that...
>
> How can xenpaging know what gfns are pagetables?

There's a bunch of heuristics one can try. It boils down to a lengthy
discussion about what the pager is trying to do. Say, you can
statistically sample cr3's, map candidates and look for page table-like
structures, exploit knowledge about the guest OS.

I'll agree though that the hypervisor support should be good enough that
the pager should not care. But it isn't and it won't get there quickly,
and I don't want us to be cavalier about these wait queues.

Andres

>
> Olaf
>

Re: 4.2 TODO update

[Olaf Hering]

On Tue, Feb 14, Andres Lagar-Cavilla wrote:

> > On Tue, Feb 14, Andres Lagar-Cavilla wrote:
> >
> >> Why? Because it's really really hard to guarantee we'll go to sleep in
> >> an
> >> atomic context. The main use for wait queues (imho) is in hvm_copy, and
> >> there's a zillion paths going into hvm_copy (copy_from/to_user!) with
> >> all
> >> ways of bumping the preemption count.
> >
> > If the guests pagetable is paged out this code path will trigger, then
> > one of the hypercalls returns an error and the guest runs into a BUG().
> > I think it was decrease_reservation, or similar.
> 
> Unlikely to be something specific about decrease_reservation. If the guest
> page table is paged out, then copy_from_user for any hypercall, or,
> "virtual address to gfn" for any emulation will run into this.
> 
> Now, even an innocent-looking rcu lock anywhere in this code path will
> crash the host if we go into a wait queue. Hence my concern.

The workaround for the guest crash I were seeing:
http://lists.xen.org/archives/html/xen-devel/2010-11/msg01609.html

I once modified xenpaging to keep the pagetables paged out as much as it
could (and still allowed the guest to make at least a little bit
progress) and run into no appearent issue.

> > Another thing reported by Huawei on this list was somewhere in the
> > emulation code where a gfn_to_mfn() failed.
> 
> Can you point to the original report? Is there anything more specific?

It was vmx_load_pdptrs().
http://lists.xen.org/archives/html/xen-devel/2011-09/msg01336.html

> > What other way exist to make paging 100% transparent to the guest?
> >
> 
> Don't page out page table pages? I know you were not expecting that...

How can xenpaging know what gfns are pagetables?

Olaf

Re: 4.2 TODO update

[Andres Lagar-Cavilla]

> On Tue, Feb 14, Andres Lagar-Cavilla wrote:
>
>> Why? Because it's really really hard to guarantee we'll go to sleep in
>> an
>> atomic context. The main use for wait queues (imho) is in hvm_copy, and
>> there's a zillion paths going into hvm_copy (copy_from/to_user!) with
>> all
>> ways of bumping the preemption count.
>
> If the guests pagetable is paged out this code path will trigger, then
> one of the hypercalls returns an error and the guest runs into a BUG().
> I think it was decrease_reservation, or similar.

Unlikely to be something specific about decrease_reservation. If the guest
page table is paged out, then copy_from_user for any hypercall, or,
"virtual address to gfn" for any emulation will run into this.

Now, even an innocent-looking rcu lock anywhere in this code path will
crash the host if we go into a wait queue. Hence my concern.

> Another thing reported by Huawei on this list was somewhere in the
> emulation code where a gfn_to_mfn() failed.

Can you point to the original report? Is there anything more specific?

>
> What other way exist to make paging 100% transparent to the guest?
>

Don't page out page table pages? I know you were not expecting that...

Andres

> Olaf
>

Re: 4.2 TODO update

[Olaf Hering]

On Tue, Feb 14, Andres Lagar-Cavilla wrote:

> Why? Because it's really really hard to guarantee we'll go to sleep in an
> atomic context. The main use for wait queues (imho) is in hvm_copy, and
> there's a zillion paths going into hvm_copy (copy_from/to_user!) with all
> ways of bumping the preemption count.

If the guests pagetable is paged out this code path will trigger, then
one of the hypercalls returns an error and the guest runs into a BUG().
I think it was decrease_reservation, or similar.
Another thing reported by Huawei on this list was somewhere in the
emulation code where a gfn_to_mfn() failed.

What other way exist to make paging 100% transparent to the guest?

Olaf

Re: 4.2 TODO update

[Andres Lagar-Cavilla]

> Date: Mon, 13 Feb 2012 10:24:29 +0000
> From: Tim Deegan <tim@xen.org>
> To: Ian Campbell <Ian.Campbell@citrix.com>
> Cc: xen-devel <xen-devel@lists.xensource.com>
> Subject: Re: [Xen-devel] 4.2 TODO update
> Message-ID: <20120213102429.GA88765@ocelot.phlegethon.org>
> Content-Type: text/plain; charset=iso-8859-1
>
> At 10:17 +0000 on 13 Feb (1329128265), Ian Campbell wrote:
>> This weeks update. I was away last Monday so this covers two weeks worth
>> of updates. I have dropped things which were marked as DONE in the last
>> update and added a bunch more DONE (so the list is shrinking!). Please
>> send me corrections (especially "done").
>>
>> hypervisor, blockers:
>>       * round-up of the closing of the security hole in MSI-X
>>         passthrough (uniformly - i.e. even for Dom0 - disallowing write
>>         access to MSI-X table pages). (Jan Beulich -- more fixes
>>         required than first thought, patches posted)
>>       * domctls / sysctls set up to modify scheduler parameters, like
>>         the credit1 timeslice and schedule rate. (George Dunlap)
>>       * get the interface changes for sharing/paging/mem-events done and
>>         dusted so that 4.2 is a stable API that we hold to. (Tim Deegan,
>>         Andres Lagar-Cavilla et al)
>>               * sharing patches posted
>
> The hypervisor interface changes for sharing and events have gone in;
> there's a libxc-level change (to do with mlock()ing buffers) that ought
> to go in before 4.2, if at all.

Ian, it's probably OK to mark the paging/sharing as DONE.

There is the outstanding issue of properly mapping the event rings, which
ties into the mlock()ing buffers bit Tim mentions. I think we all want it
Done Right for 4.2. And that will likely not be the patch I last posted.
Perhaps best to rename the bullet point?

In the "nice to have" category, "properly synchronized p2m lookups" have
also gone into the tree.

>
> There's a paging+waitqueues patch that's been posted but needs a bit more
> work still; I'll try and look at it on Thursday.
>

This one is thorny. Right now we seem to have fairly stable paging. (Dare
I say bug-free? I dare not, lest I be smitten) I surmise other consumers
of the interface would agree with this (Olaf? Huawei?)

Wait queues have a significant potential for damage, unless we use them in
very specific contexts. I worry we'll drop from A- to C in terms of
stability. And right now I have not crashed hosts with paging in a long
while, whereas wait queues will definitely introduce that.

Why? Because it's really really hard to guarantee we'll go to sleep in an
atomic context. The main use for wait queues (imho) is in hvm_copy, and
there's a zillion paths going into hvm_copy (copy_from/to_user!) with all
ways of bumping the preemption count.

I'm dealing with this at the moment in a different context: running out of
memory to unshare pages when writing to them in the hypervisor. And we
simply cannot throw a wait queue sleep in there carelessly.

I'm not saying this is impossible, I'm saying there's a long path ahead,
and since it's all *internal* to the hypervisor, my vote is to be very
conservative for the 4.2 window.

Thanks!
Andres

> Tim.
>