Xen 4.2 Release Plan / TODO

Xen 4.2 Release Plan / TODO

[Ian Campbell]

So as discussed last week we now have a plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html

The time line is as follows:

19 March	-- TODO list locked down	<< WE ARE HERE
2 April		-- Feature Freeze
Mid/Late April	-- First release candidate
Weekly		-- RCN+1 until it is ready

The updated TODO list follows. Per the release plan a strong case will
now have to be made as to why new items should be added to the list,
especially as a blocker, rather than deferred to 4.3.

Things look pretty good on the hypervisor side of things.

The tools list is quite long but most stuff is known to be in progress
and in many cases preliminary patches are available. I think there is a
name next to everything.

I have gather various items under a top level "xl compatibility with xm"
heading under both blocker and nice-to-have. I expect this is the area
where most bugs will be reported once we hit -rc and users start testing
this stuff in anger.

Ian.

hypervisor, blockers:
      * NONE?
 
tools, blockers:
      * libxl stable API -- we would like 4.2 to define a stable API
        which downstream's can start to rely on not changing. Aspects of
        this are:
              * Safe fork vs. fd handling hooks. Involves API changes
                (Ian J, patches posted)
              * locking/serialization, e.g., for domain creation. As of
                now,  nothing is provided for this purpose, and
                downstream toolstacks have to put their own mechanisms
                in place. E.g., xl uses a fcntl() lock around the whole
                process of domain creation. It should OTOH be libxl job
                to offer a proper set of hooks --placed at the proper
                spots during the domain creation process-- for the
                downstreams to  fill with the proper callbacks. (Dario
                Faggioli)
              * agree & document compatibility guarantees + associated
                technical measures (Ian C)
      * xl compatibility with xm:
              * feature parity wrt driver domain support (George Dunlap)
              * xl support for "rtc_timeoffset" and "localtime" (Lin
                Ming, Patches posted)
      * More formally deprecate xm/xend. Manpage patches already in
        tree. Needs release noting and communication around -rc1 to
        remind people to test xl.
      * Domain 0 block attach & general hotplug when using qdisk backend
        (need to start qemu as necessary etc) (Stefano S)
      * file:// backend performance. qemu-xen-tradition's qdisk is quite
        slow & blktap2 not available in upstream kernels. Need to
        consider our options:
              * qemu-xen's qdisk is thought to be well performing but
                qemu-xen is not yet the default. Complexity arising from
                splitting qemu-for-qdisk out from qemu-for-dm and
                running N qemu's.
              * potentially fully userspace blktap could be ready for
                4.2
              * use /dev/loop+blkback. This requires loop driver AIO and
                O_DIRECT patches which are not (AFAIK) yet upstream.
              * Leverage XCP's blktap2 DKMS work.
              * Other ideas?
      * Improved Hotplug script support (Roger Pau Monné, patches
        posted)
      * Block script support -- follows on from hotplug script (Roger
        Pau Monné)

hypervisor, nice to have:
      * solid implementation of sharing/paging/mem-events (using work
        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
              * "The last patch to use a waitqueue in
                __get_gfn_type_access() from Tim works.  However, there
                are a few users who call __get_gfn_type_access with the
                domain_lock held. This part needs to be addressed in
                some way."
      * Sharing support for AMD (Tim, Andres).
      * PoD performance improvements (George Dunlap)

tools, nice to have:
      * Configure/control paging via xl/libxl (Olaf Herring, lots of
        discussion around interface, general consensus reached on what
        it should look like)
      * Upstream qemu feature patches:
              * Upstream qemu PCI passthrough support (Anthony Perard,
                patches sent)
              * Upstream qemu save restore (Anthony Perard, Stefano
                Stabellini, patches sent, waiting for upstream ack)
      * Nested-virtualisation. Currently "experimental". Likely to
        release that way.
              * Nested SVM. Tested in a variety of configurations but
                still some issues with the most important use case (w7
                XP mode) [0]  (Christoph Egger)
              * Nested VMX. Needs nested EPT to be genuinely useful.
                Need more data on testedness etc (Intel)
      * Initial xl support for Remus (memory checkpoint, blackholing)
        (Shriram, patches posted, blocked behind qemu save restore
        patches)
      * xl compatibility with xm:
              * xl support for autospawning vncviewer (vncviewer=1 or
                otherwise) (Goncalo Gomes)
              * support for vif "rate" parameter (Mathieu Gagné)

[0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Jan Beulich]

>>> On 19.03.12 at 11:57, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>       * file:// backend performance. qemu-xen-tradition's qdisk is quite
>         slow & blktap2 not available in upstream kernels. Need to
>         consider our options:
>               * qemu-xen's qdisk is thought to be well performing but
>                 qemu-xen is not yet the default. Complexity arising from
>                 splitting qemu-for-qdisk out from qemu-for-dm and
>                 running N qemu's.
>               * potentially fully userspace blktap could be ready for
>                 4.2
>               * use /dev/loop+blkback. This requires loop driver AIO and
>                 O_DIRECT patches which are not (AFAIK) yet upstream.

I meant to ask already when this was first mentioned: What's the
reason for this requirement? Didn't we have blkback over loop running
fine for years? Or is this just a performance consideration (in which
case "requires" might be too strong a term)?

Jan

>               * Leverage XCP's blktap2 DKMS work.
>               * Other ideas?

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Mon, 2012-03-19 at 11:25 +0000, Jan Beulich wrote:
> >>> On 19.03.12 at 11:57, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >       * file:// backend performance. qemu-xen-tradition's qdisk is quite
> >         slow & blktap2 not available in upstream kernels. Need to
> >         consider our options:
> >               * qemu-xen's qdisk is thought to be well performing but
> >                 qemu-xen is not yet the default. Complexity arising from
> >                 splitting qemu-for-qdisk out from qemu-for-dm and
> >                 running N qemu's.
> >               * potentially fully userspace blktap could be ready for
> >                 4.2
> >               * use /dev/loop+blkback. This requires loop driver AIO and
> >                 O_DIRECT patches which are not (AFAIK) yet upstream.
> 
> I meant to ask already when this was first mentioned: What's the
> reason for this requirement? Didn't we have blkback over loop running
> fine for years? Or is this just a performance consideration (in which
> case "requires" might be too strong a term)?

My understanding (which could well be totally bogus) was that the use
of /dev/loop in this way was unsafe since pages were only committed to
the dom0 page cache and not to the actual platter when success was
reported to the guest. I think that is why many people used tap:aio:
instead of file: (personally I use phy: almost exclusively so I could be
talking rubbish).

Unless there are some loop patches in the classic-Xen patchset? I don't
think there are though.

I don't know so much about the performance aspect. Stefano might be able
to comment.

Ian.

> 
> Jan
> 
> >               * Leverage XCP's blktap2 DKMS work.
> >               * Other ideas?
> 
> 

Re: Xen 4.2 Release Plan / TODO

[Jan Beulich]

>>> On 19.03.12 at 12:33, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2012-03-19 at 11:25 +0000, Jan Beulich wrote:
>> >>> On 19.03.12 at 11:57, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> >       * file:// backend performance. qemu-xen-tradition's qdisk is quite
>> >         slow & blktap2 not available in upstream kernels. Need to
>> >         consider our options:
>> >               * qemu-xen's qdisk is thought to be well performing but
>> >                 qemu-xen is not yet the default. Complexity arising from
>> >                 splitting qemu-for-qdisk out from qemu-for-dm and
>> >                 running N qemu's.
>> >               * potentially fully userspace blktap could be ready for
>> >                 4.2
>> >               * use /dev/loop+blkback. This requires loop driver AIO and
>> >                 O_DIRECT patches which are not (AFAIK) yet upstream.
>> 
>> I meant to ask already when this was first mentioned: What's the
>> reason for this requirement? Didn't we have blkback over loop running
>> fine for years? Or is this just a performance consideration (in which
>> case "requires" might be too strong a term)?
> 
> My understanding (which could well be totally bogus) was that the use
> of /dev/loop in this way was unsafe since pages were only committed to
> the dom0 page cache and not to the actual platter when success was
> reported to the guest. I think that is why many people used tap:aio:
> instead of file: (personally I use phy: almost exclusively so I could be
> talking rubbish).
> 
> Unless there are some loop patches in the classic-Xen patchset? I don't
> think there are though.

I know of none either.

Jan

> I don't know so much about the performance aspect. Stefano might be able
> to comment.
> 
> Ian.
> 
>> 
>> Jan
>> 
>> >               * Leverage XCP's blktap2 DKMS work.
>> >               * Other ideas?
>> 
>> 

Re: Xen 4.2 Release Plan / TODO

[Stefano Stabellini]

On Mon, 19 Mar 2012, Jan Beulich wrote:
> >>> On 19.03.12 at 11:57, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >       * file:// backend performance. qemu-xen-tradition's qdisk is quite
> >         slow & blktap2 not available in upstream kernels. Need to
> >         consider our options:
> >               * qemu-xen's qdisk is thought to be well performing but
> >                 qemu-xen is not yet the default. Complexity arising from
> >                 splitting qemu-for-qdisk out from qemu-for-dm and
> >                 running N qemu's.
> >               * potentially fully userspace blktap could be ready for
> >                 4.2
> >               * use /dev/loop+blkback. This requires loop driver AIO and
> >                 O_DIRECT patches which are not (AFAIK) yet upstream.
> 
> I meant to ask already when this was first mentioned: What's the
> reason for this requirement? Didn't we have blkback over loop running
> fine for years? Or is this just a performance consideration (in which
> case "requires" might be too strong a term)?

There are several problems here:

- the usage of loop with blkback is unsafe because loop doesn't support
O_DIRECT, at least the version of loop present in upstream Linux. Also
this means that the very good performance results that we get with loop
are actually inflated, the real numbers could be very low.

- Loop with blkback doesn't work with anything but raw files, so it
won't work for qcow, qcow2 or vhd.

- Using qdisk as backend, with or without AIO, is possible but from the
performance measurements I have run so far is very slow. In theory this
should be the best option, but in practice I cannot explain why I am
getting 1/10 of the performances I am expecting.

Re: Xen 4.2 Release Plan / TODO

[George Dunlap]

On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> So as discussed last week we now have a plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
>
> The time line is as follows:
>
> 19 March        -- TODO list locked down        << WE ARE HERE
> 2 April         -- Feature Freeze
> Mid/Late April  -- First release candidate
> Weekly          -- RCN+1 until it is ready
>
> The updated TODO list follows. Per the release plan a strong case will
> now have to be made as to why new items should be added to the list,
> especially as a blocker, rather than deferred to 4.3.
>
> Things look pretty good on the hypervisor side of things.
>
> The tools list is quite long but most stuff is known to be in progress
> and in many cases preliminary patches are available. I think there is a
> name next to everything.
>
> I have gather various items under a top level "xl compatibility with xm"
> heading under both blocker and nice-to-have. I expect this is the area
> where most bugs will be reported once we hit -rc and users start testing
> this stuff in anger.
>
> Ian.
>
> hypervisor, blockers:
>      * NONE?
>
> tools, blockers:
>      * libxl stable API -- we would like 4.2 to define a stable API
>        which downstream's can start to rely on not changing. Aspects of
>        this are:
>              * Safe fork vs. fd handling hooks. Involves API changes
>                (Ian J, patches posted)
>              * locking/serialization, e.g., for domain creation. As of
>                now,  nothing is provided for this purpose, and
>                downstream toolstacks have to put their own mechanisms
>                in place. E.g., xl uses a fcntl() lock around the whole
>                process of domain creation. It should OTOH be libxl job
>                to offer a proper set of hooks --placed at the proper
>                spots during the domain creation process-- for the
>                downstreams to  fill with the proper callbacks. (Dario
>                Faggioli)
>              * agree & document compatibility guarantees + associated
>                technical measures (Ian C)
>      * xl compatibility with xm:
>              * feature parity wrt driver domain support (George Dunlap)
>              * xl support for "rtc_timeoffset" and "localtime" (Lin
>                Ming, Patches posted)
>      * More formally deprecate xm/xend. Manpage patches already in
>        tree. Needs release noting and communication around -rc1 to
>        remind people to test xl.

Nonetheless, we should still make basic functionality of xm a
requirement for this release, right?  I've just created a VM with a
config file that worked in xl, and it complains that the hotplug
scripts are not working, for example.  Does that go in this todo list,
or are we keeping track of bugs somewhere else?

>      * Domain 0 block attach & general hotplug when using qdisk backend
>        (need to start qemu as necessary etc) (Stefano S)
>      * file:// backend performance. qemu-xen-tradition's qdisk is quite
>        slow & blktap2 not available in upstream kernels. Need to
>        consider our options:
>              * qemu-xen's qdisk is thought to be well performing but
>                qemu-xen is not yet the default. Complexity arising from
>                splitting qemu-for-qdisk out from qemu-for-dm and
>                running N qemu's.
>              * potentially fully userspace blktap could be ready for
>                4.2
>              * use /dev/loop+blkback. This requires loop driver AIO and
>                O_DIRECT patches which are not (AFAIK) yet upstream.
>              * Leverage XCP's blktap2 DKMS work.
>              * Other ideas?
>      * Improved Hotplug script support (Roger Pau Monné, patches
>        posted)
>      * Block script support -- follows on from hotplug script (Roger
>        Pau Monné)
>
> hypervisor, nice to have:
>      * solid implementation of sharing/paging/mem-events (using work
>        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
>              * "The last patch to use a waitqueue in
>                __get_gfn_type_access() from Tim works.  However, there
>                are a few users who call __get_gfn_type_access with the
>                domain_lock held. This part needs to be addressed in
>                some way."
>      * Sharing support for AMD (Tim, Andres).
>      * PoD performance improvements (George Dunlap)
>
> tools, nice to have:
>      * Configure/control paging via xl/libxl (Olaf Herring, lots of
>        discussion around interface, general consensus reached on what
>        it should look like)
>      * Upstream qemu feature patches:
>              * Upstream qemu PCI passthrough support (Anthony Perard,
>                patches sent)
>              * Upstream qemu save restore (Anthony Perard, Stefano
>                Stabellini, patches sent, waiting for upstream ack)
>      * Nested-virtualisation. Currently "experimental". Likely to
>        release that way.
>              * Nested SVM. Tested in a variety of configurations but
>                still some issues with the most important use case (w7
>                XP mode) [0]  (Christoph Egger)
>              * Nested VMX. Needs nested EPT to be genuinely useful.
>                Need more data on testedness etc (Intel)
>      * Initial xl support for Remus (memory checkpoint, blackholing)
>        (Shriram, patches posted, blocked behind qemu save restore
>        patches)
>      * xl compatibility with xm:
>              * xl support for autospawning vncviewer (vncviewer=1 or
>                otherwise) (Goncalo Gomes)
>              * support for vif "rate" parameter (Mathieu Gagné)
>
> [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Mon, 2012-03-19 at 12:13 +0000, George Dunlap wrote:
> On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >      * More formally deprecate xm/xend. Manpage patches already in
> >        tree. Needs release noting and communication around -rc1 to
> >        remind people to test xl.
> 
> Nonetheless, we should still make basic functionality of xm a
> requirement for this release, right?

Right, deprecated != known broken.

>   I've just created a VM with a
> config file that worked in xl, and it complains that the hotplug
> scripts are not working, for example.  Does that go in this todo list,
> or are we keeping track of bugs somewhere else?

I've not been tracking bugs as such here, I suppose I could. I'd be
inclined to wait until we get a bit further along the process (e.g. past
feature freeze) before starting to keep a close eye on the bugs. Of
course I would be more than happy (indeed, very appreciative) if someone
wanted to independently take on that task.

However xend is still tested by the automated test and appears to be
passing -- are you sure you have a bug and not a configuration issue?

Ian.

Re: Xen 4.2 Release Plan / TODO

[Matt Wilson]

On Mon, Mar 19, 2012 at 03:57:25AM -0700, Ian Campbell wrote:
> 
> The updated TODO list follows. Per the release plan a strong case will
> now have to be made as to why new items should be added to the list,
> especially as a blocker, rather than deferred to 4.3.

The patches to PV-GRUB for ext4 [1] and btrfs [2] support are very low
risk changes. I'll happily repost them rebased against tip
xen-unstable (and with no rejects created in the case of the btrfs
patch), if that helps.

Matt

[1] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00651.html
[2] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00649.html

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Tue, 2012-03-20 at 05:19 +0000, Matt Wilson wrote:
> On Mon, Mar 19, 2012 at 03:57:25AM -0700, Ian Campbell wrote:
> > 
> > The updated TODO list follows. Per the release plan a strong case will
> > now have to be made as to why new items should be added to the list,
> > especially as a blocker, rather than deferred to 4.3.
> 
> The patches to PV-GRUB for ext4 [1] and btrfs [2] support are very low
> risk changes. I'll happily repost them rebased against tip
> xen-unstable (and with no rejects created in the case of the btrfs
> patch), if that helps.

I think these would be good to have in, and it certainly isn't too late
for 4.2.

I think these originally came in while Ian J (tools maintainer) was away
so I expect they just fell through the cracks. Reposting would likely be
very useful, thanks.

Ian.

> 
> Matt
> 
> [1] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00651.html
> [2] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00649.html

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

Someone pointed out that some people may not know that their name was on
this list, so I've CC'd everyone who is mentioned by name. Please let me
know if there is a problem with your entry.

On Mon, 2012-03-19 at 10:57 +0000, Ian Campbell wrote:
> So as discussed last week we now have a plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
> 
> The time line is as follows:
> 
> 19 March	-- TODO list locked down	<< WE ARE HERE
> 2 April		-- Feature Freeze
> Mid/Late April	-- First release candidate
> Weekly		-- RCN+1 until it is ready
> 
> The updated TODO list follows. Per the release plan a strong case will
> now have to be made as to why new items should be added to the list,
> especially as a blocker, rather than deferred to 4.3.
> 
> Things look pretty good on the hypervisor side of things.
> 
> The tools list is quite long but most stuff is known to be in progress
> and in many cases preliminary patches are available. I think there is a
> name next to everything.
> 
> I have gather various items under a top level "xl compatibility with xm"
> heading under both blocker and nice-to-have. I expect this is the area
> where most bugs will be reported once we hit -rc and users start testing
> this stuff in anger.
> 
> Ian.
> 
> hypervisor, blockers:
>       * NONE?
>  
> tools, blockers:
>       * libxl stable API -- we would like 4.2 to define a stable API
>         which downstream's can start to rely on not changing. Aspects of
>         this are:
>               * Safe fork vs. fd handling hooks. Involves API changes
>                 (Ian J, patches posted)
>               * locking/serialization, e.g., for domain creation. As of
>                 now,  nothing is provided for this purpose, and
>                 downstream toolstacks have to put their own mechanisms
>                 in place. E.g., xl uses a fcntl() lock around the whole
>                 process of domain creation. It should OTOH be libxl job
>                 to offer a proper set of hooks --placed at the proper
>                 spots during the domain creation process-- for the
>                 downstreams to  fill with the proper callbacks. (Dario
>                 Faggioli)
>               * agree & document compatibility guarantees + associated
>                 technical measures (Ian C)
>       * xl compatibility with xm:
>               * feature parity wrt driver domain support (George Dunlap)
>               * xl support for "rtc_timeoffset" and "localtime" (Lin
>                 Ming, Patches posted)
>       * More formally deprecate xm/xend. Manpage patches already in
>         tree. Needs release noting and communication around -rc1 to
>         remind people to test xl.
>       * Domain 0 block attach & general hotplug when using qdisk backend
>         (need to start qemu as necessary etc) (Stefano S)
>       * file:// backend performance. qemu-xen-tradition's qdisk is quite
>         slow & blktap2 not available in upstream kernels. Need to
>         consider our options:
>               * qemu-xen's qdisk is thought to be well performing but
>                 qemu-xen is not yet the default. Complexity arising from
>                 splitting qemu-for-qdisk out from qemu-for-dm and
>                 running N qemu's.
>               * potentially fully userspace blktap could be ready for
>                 4.2
>               * use /dev/loop+blkback. This requires loop driver AIO and
>                 O_DIRECT patches which are not (AFAIK) yet upstream.
>               * Leverage XCP's blktap2 DKMS work.
>               * Other ideas?
>       * Improved Hotplug script support (Roger Pau Monné, patches
>         posted)
>       * Block script support -- follows on from hotplug script (Roger
>         Pau Monné)
> 
> hypervisor, nice to have:
>       * solid implementation of sharing/paging/mem-events (using work
>         queues) (Tim Deegan, Olaf Herring et al -- patches posted)
>               * "The last patch to use a waitqueue in
>                 __get_gfn_type_access() from Tim works.  However, there
>                 are a few users who call __get_gfn_type_access with the
>                 domain_lock held. This part needs to be addressed in
>                 some way."
>       * Sharing support for AMD (Tim, Andres).
>       * PoD performance improvements (George Dunlap)
> 
> tools, nice to have:
>       * Configure/control paging via xl/libxl (Olaf Herring, lots of
>         discussion around interface, general consensus reached on what
>         it should look like)
>       * Upstream qemu feature patches:
>               * Upstream qemu PCI passthrough support (Anthony Perard,
>                 patches sent)
>               * Upstream qemu save restore (Anthony Perard, Stefano
>                 Stabellini, patches sent, waiting for upstream ack)
>       * Nested-virtualisation. Currently "experimental". Likely to
>         release that way.
>               * Nested SVM. Tested in a variety of configurations but
>                 still some issues with the most important use case (w7
>                 XP mode) [0]  (Christoph Egger)
>               * Nested VMX. Needs nested EPT to be genuinely useful.
>                 Need more data on testedness etc (Intel)
>       * Initial xl support for Remus (memory checkpoint, blackholing)
>         (Shriram, patches posted, blocked behind qemu save restore
>         patches)
>       * xl compatibility with xm:
>               * xl support for autospawning vncviewer (vncviewer=1 or
>                 otherwise) (Goncalo Gomes)
>               * support for vif "rate" parameter (Mathieu Gagné)
> 
> [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Mon, 2012-03-19 at 10:57 +0000, Ian Campbell wrote:

> tools, nice to have:
[...]
>       * Initial xl support for Remus (memory checkpoint, blackholing)
>         (Shriram, patches posted, blocked behind qemu save restore
>         patches)

AIUI the qemu save/restore patches have now been accepted. Not sure of
the status of the xl/libxl side of that, perhaps time for a repost of
both that series and the Remus one?


Ian.

Re: Xen 4.2 Release Plan / TODO

[Stefano Stabellini]

On Thu, 22 Mar 2012, Ian Campbell wrote:
> On Mon, 2012-03-19 at 10:57 +0000, Ian Campbell wrote:
> 
> > tools, nice to have:
> [...]
> >       * Initial xl support for Remus (memory checkpoint, blackholing)
> >         (Shriram, patches posted, blocked behind qemu save restore
> >         patches)
> 
> AIUI the qemu save/restore patches have now been accepted. Not sure of
> the status of the xl/libxl side of that, perhaps time for a repost of
> both that series and the Remus one?

I have reposted my series already:

http://marc.info/?l=xen-devel&m=133224577008104

Re: Xen 4.2 Release Plan / TODO

[George Dunlap]

On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>      * xl compatibility with xm:
>              * feature parity wrt driver domain support (George Dunlap)
I just discovered (while playing with driver domains) that xl is
missing one bit of feature parity with xm for pci passthrough for PV
guests -- and that's the "pci quirk" config file support.  I'm going
to ask Intel if they have an interest in porting it over; I think it
should at least be a "nice-to-have", and it may be a low-level
blocker, as a lot of devices won't work passed through without it.

>              * xl support for "rtc_timeoffset" and "localtime" (Lin
>                Ming, Patches posted)
>      * More formally deprecate xm/xend. Manpage patches already in
>        tree. Needs release noting and communication around -rc1 to
>        remind people to test xl.
>      * Domain 0 block attach & general hotplug when using qdisk backend
>        (need to start qemu as necessary etc) (Stefano S)
>      * file:// backend performance. qemu-xen-tradition's qdisk is quite
>        slow & blktap2 not available in upstream kernels. Need to
>        consider our options:
>              * qemu-xen's qdisk is thought to be well performing but
>                qemu-xen is not yet the default. Complexity arising from
>                splitting qemu-for-qdisk out from qemu-for-dm and
>                running N qemu's.
>              * potentially fully userspace blktap could be ready for
>                4.2
>              * use /dev/loop+blkback. This requires loop driver AIO and
>                O_DIRECT patches which are not (AFAIK) yet upstream.
>              * Leverage XCP's blktap2 DKMS work.
>              * Other ideas?
>      * Improved Hotplug script support (Roger Pau Monné, patches
>        posted)
>      * Block script support -- follows on from hotplug script (Roger
>        Pau Monné)
>
> hypervisor, nice to have:
>      * solid implementation of sharing/paging/mem-events (using work
>        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
>              * "The last patch to use a waitqueue in
>                __get_gfn_type_access() from Tim works.  However, there
>                are a few users who call __get_gfn_type_access with the
>                domain_lock held. This part needs to be addressed in
>                some way."
>      * Sharing support for AMD (Tim, Andres).
>      * PoD performance improvements (George Dunlap)
>
> tools, nice to have:
>      * Configure/control paging via xl/libxl (Olaf Herring, lots of
>        discussion around interface, general consensus reached on what
>        it should look like)
>      * Upstream qemu feature patches:
>              * Upstream qemu PCI passthrough support (Anthony Perard,
>                patches sent)
>              * Upstream qemu save restore (Anthony Perard, Stefano
>                Stabellini, patches sent, waiting for upstream ack)
>      * Nested-virtualisation. Currently "experimental". Likely to
>        release that way.
>              * Nested SVM. Tested in a variety of configurations but
>                still some issues with the most important use case (w7
>                XP mode) [0]  (Christoph Egger)
>              * Nested VMX. Needs nested EPT to be genuinely useful.
>                Need more data on testedness etc (Intel)
>      * Initial xl support for Remus (memory checkpoint, blackholing)
>        (Shriram, patches posted, blocked behind qemu save restore
>        patches)
>      * xl compatibility with xm:
>              * xl support for autospawning vncviewer (vncviewer=1 or
>                otherwise) (Goncalo Gomes)
>              * support for vif "rate" parameter (Mathieu Gagné)
>
> [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Thu, 2012-03-22 at 09:35 +0000, George Dunlap wrote:
> On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >      * xl compatibility with xm:
> >              * feature parity wrt driver domain support (George Dunlap)
> I just discovered (while playing with driver domains) that xl is
> missing one bit of feature parity with xm for pci passthrough for PV
> guests -- and that's the "pci quirk" config file support.  I'm going
> to ask Intel if they have an interest in porting it over; I think it
> should at least be a "nice-to-have", and it may be a low-level
> blocker, as a lot of devices won't work passed through without it.

This is the stuff in tools/python/xen/xend/server/pciquirk.py ?

pciback in upstream doesn't mention "quirk" which suggests no support
for the necessary sysfs node either?

tools/examples/xend-pci-quirks.sxp  seems to only have a quirk for a
single card?

I don't think we want to implement an SXP parser for xl/libxl so if this
is reimplemented I think a different format should be used.

Anyway, I'll put this onto the list.

Ian

> 
> >              * xl support for "rtc_timeoffset" and "localtime" (Lin
> >                Ming, Patches posted)
> >      * More formally deprecate xm/xend. Manpage patches already in
> >        tree. Needs release noting and communication around -rc1 to
> >        remind people to test xl.
> >      * Domain 0 block attach & general hotplug when using qdisk backend
> >        (need to start qemu as necessary etc) (Stefano S)
> >      * file:// backend performance. qemu-xen-tradition's qdisk is quite
> >        slow & blktap2 not available in upstream kernels. Need to
> >        consider our options:
> >              * qemu-xen's qdisk is thought to be well performing but
> >                qemu-xen is not yet the default. Complexity arising from
> >                splitting qemu-for-qdisk out from qemu-for-dm and
> >                running N qemu's.
> >              * potentially fully userspace blktap could be ready for
> >                4.2
> >              * use /dev/loop+blkback. This requires loop driver AIO and
> >                O_DIRECT patches which are not (AFAIK) yet upstream.
> >              * Leverage XCP's blktap2 DKMS work.
> >              * Other ideas?
> >      * Improved Hotplug script support (Roger Pau Monné, patches
> >        posted)
> >      * Block script support -- follows on from hotplug script (Roger
> >        Pau Monné)
> >
> > hypervisor, nice to have:
> >      * solid implementation of sharing/paging/mem-events (using work
> >        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
> >              * "The last patch to use a waitqueue in
> >                __get_gfn_type_access() from Tim works.  However, there
> >                are a few users who call __get_gfn_type_access with the
> >                domain_lock held. This part needs to be addressed in
> >                some way."
> >      * Sharing support for AMD (Tim, Andres).
> >      * PoD performance improvements (George Dunlap)
> >
> > tools, nice to have:
> >      * Configure/control paging via xl/libxl (Olaf Herring, lots of
> >        discussion around interface, general consensus reached on what
> >        it should look like)
> >      * Upstream qemu feature patches:
> >              * Upstream qemu PCI passthrough support (Anthony Perard,
> >                patches sent)
> >              * Upstream qemu save restore (Anthony Perard, Stefano
> >                Stabellini, patches sent, waiting for upstream ack)
> >      * Nested-virtualisation. Currently "experimental". Likely to
> >        release that way.
> >              * Nested SVM. Tested in a variety of configurations but
> >                still some issues with the most important use case (w7
> >                XP mode) [0]  (Christoph Egger)
> >              * Nested VMX. Needs nested EPT to be genuinely useful.
> >                Need more data on testedness etc (Intel)
> >      * Initial xl support for Remus (memory checkpoint, blackholing)
> >        (Shriram, patches posted, blocked behind qemu save restore
> >        patches)
> >      * xl compatibility with xm:
> >              * xl support for autospawning vncviewer (vncviewer=1 or
> >                otherwise) (Goncalo Gomes)
> >              * support for vif "rate" parameter (Mathieu Gagné)
> >
> > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[George Dunlap]

On Thu, Mar 22, 2012 at 9:53 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Thu, 2012-03-22 at 09:35 +0000, George Dunlap wrote:
>> On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> >      * xl compatibility with xm:
>> >              * feature parity wrt driver domain support (George Dunlap)
>> I just discovered (while playing with driver domains) that xl is
>> missing one bit of feature parity with xm for pci passthrough for PV
>> guests -- and that's the "pci quirk" config file support.  I'm going
>> to ask Intel if they have an interest in porting it over; I think it
>> should at least be a "nice-to-have", and it may be a low-level
>> blocker, as a lot of devices won't work passed through without it.
>
> This is the stuff in tools/python/xen/xend/server/pciquirk.py ?
>
> pciback in upstream doesn't mention "quirk" which suggests no support
> for the necessary sysfs node either?

Ah, interesting -- that's worth tracking down.  Maybe there's a better
way to deal with quirks?  Or maybe it just hasn't been upstreamed yet
(or perhaps even implemented in pvops?).  I'm using the Debian squeeze
2.6.32-5-xen-686 kernel.

> tools/examples/xend-pci-quirks.sxp  seems to only have a quirk for a
> single card?

Yes, well I could add two more cards just from experience w/ one of my
test boxen. :-)

> I don't think we want to implement an SXP parser for xl/libxl so if this
> is reimplemented I think a different format should be used.

Since we're using yajl anyway, JSON might not be a bad option.

Anyway, I'll ping the Intel guy who recently posted a patch to libxl_pci.c.

 -George

>
> Anyway, I'll put this onto the list.
>
> Ian
>
>>
>> >              * xl support for "rtc_timeoffset" and "localtime" (Lin
>> >                Ming, Patches posted)
>> >      * More formally deprecate xm/xend. Manpage patches already in
>> >        tree. Needs release noting and communication around -rc1 to
>> >        remind people to test xl.
>> >      * Domain 0 block attach & general hotplug when using qdisk backend
>> >        (need to start qemu as necessary etc) (Stefano S)
>> >      * file:// backend performance. qemu-xen-tradition's qdisk is quite
>> >        slow & blktap2 not available in upstream kernels. Need to
>> >        consider our options:
>> >              * qemu-xen's qdisk is thought to be well performing but
>> >                qemu-xen is not yet the default. Complexity arising from
>> >                splitting qemu-for-qdisk out from qemu-for-dm and
>> >                running N qemu's.
>> >              * potentially fully userspace blktap could be ready for
>> >                4.2
>> >              * use /dev/loop+blkback. This requires loop driver AIO and
>> >                O_DIRECT patches which are not (AFAIK) yet upstream.
>> >              * Leverage XCP's blktap2 DKMS work.
>> >              * Other ideas?
>> >      * Improved Hotplug script support (Roger Pau Monné, patches
>> >        posted)
>> >      * Block script support -- follows on from hotplug script (Roger
>> >        Pau Monné)
>> >
>> > hypervisor, nice to have:
>> >      * solid implementation of sharing/paging/mem-events (using work
>> >        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
>> >              * "The last patch to use a waitqueue in
>> >                __get_gfn_type_access() from Tim works.  However, there
>> >                are a few users who call __get_gfn_type_access with the
>> >                domain_lock held. This part needs to be addressed in
>> >                some way."
>> >      * Sharing support for AMD (Tim, Andres).
>> >      * PoD performance improvements (George Dunlap)
>> >
>> > tools, nice to have:
>> >      * Configure/control paging via xl/libxl (Olaf Herring, lots of
>> >        discussion around interface, general consensus reached on what
>> >        it should look like)
>> >      * Upstream qemu feature patches:
>> >              * Upstream qemu PCI passthrough support (Anthony Perard,
>> >                patches sent)
>> >              * Upstream qemu save restore (Anthony Perard, Stefano
>> >                Stabellini, patches sent, waiting for upstream ack)
>> >      * Nested-virtualisation. Currently "experimental". Likely to
>> >        release that way.
>> >              * Nested SVM. Tested in a variety of configurations but
>> >                still some issues with the most important use case (w7
>> >                XP mode) [0]  (Christoph Egger)
>> >              * Nested VMX. Needs nested EPT to be genuinely useful.
>> >                Need more data on testedness etc (Intel)
>> >      * Initial xl support for Remus (memory checkpoint, blackholing)
>> >        (Shriram, patches posted, blocked behind qemu save restore
>> >        patches)
>> >      * xl compatibility with xm:
>> >              * xl support for autospawning vncviewer (vncviewer=1 or
>> >                otherwise) (Goncalo Gomes)
>> >              * support for vif "rate" parameter (Mathieu Gagné)
>> >
>> > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
>> >
>> >
>> > _______________________________________________
>> > Xen-devel mailing list
>> > Xen-devel@lists.xen.org
>> > http://lists.xen.org/xen-devel
>
>

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Thu, 2012-03-22 at 10:08 +0000, George Dunlap wrote:
> On Thu, Mar 22, 2012 at 9:53 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Thu, 2012-03-22 at 09:35 +0000, George Dunlap wrote:
> >> On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >> >      * xl compatibility with xm:
> >> >              * feature parity wrt driver domain support (George Dunlap)
> >> I just discovered (while playing with driver domains) that xl is
> >> missing one bit of feature parity with xm for pci passthrough for PV
> >> guests -- and that's the "pci quirk" config file support.  I'm going
> >> to ask Intel if they have an interest in porting it over; I think it
> >> should at least be a "nice-to-have", and it may be a low-level
> >> blocker, as a lot of devices won't work passed through without it.
> >
> > This is the stuff in tools/python/xen/xend/server/pciquirk.py ?
> >
> > pciback in upstream doesn't mention "quirk" which suggests no support
> > for the necessary sysfs node either?
> 
> Ah, interesting -- that's worth tracking down.  Maybe there's a better
> way to deal with quirks?  Or maybe it just hasn't been upstreamed yet
> (or perhaps even implemented in pvops?).  I'm using the Debian squeeze
> 2.6.32-5-xen-686 kernel.

I told a lie -- the code does seem to be there in mainline
(drivers/xen/xen-pciback/conf_space_quirks.c et al). Not sure how grep
missed it.

Does anyone know what the actual purpose/function of the single defined
quirk is? 10845:df80de098d15 which introduces it doesn't really say,
it's just a bunch of magic register frobbing as far as I'm concerned.

I guess you have a tg3 and are suffering from this exact quirk?

It's an awful lot of scaffolding on both the userspace and kernel side
to support a generic quirks system which has had exactly one quirk since
it was introduced in mid 2006. Perhaps we should just address the
specific tg3 issue directly?

> 
> > tools/examples/xend-pci-quirks.sxp  seems to only have a quirk for a
> > single card?
> 
> Yes, well I could add two more cards just from experience w/ one of my
> test boxen. :-)
> 
> > I don't think we want to implement an SXP parser for xl/libxl so if this
> > is reimplemented I think a different format should be used.
> 
> Since we're using yajl anyway, JSON might not be a bad option.
> 
> Anyway, I'll ping the Intel guy who recently posted a patch to libxl_pci.c.
> 
>  -George
> 
> >
> > Anyway, I'll put this onto the list.
> >
> > Ian
> >
> >>
> >> >              * xl support for "rtc_timeoffset" and "localtime" (Lin
> >> >                Ming, Patches posted)
> >> >      * More formally deprecate xm/xend. Manpage patches already in
> >> >        tree. Needs release noting and communication around -rc1 to
> >> >        remind people to test xl.
> >> >      * Domain 0 block attach & general hotplug when using qdisk backend
> >> >        (need to start qemu as necessary etc) (Stefano S)
> >> >      * file:// backend performance. qemu-xen-tradition's qdisk is quite
> >> >        slow & blktap2 not available in upstream kernels. Need to
> >> >        consider our options:
> >> >              * qemu-xen's qdisk is thought to be well performing but
> >> >                qemu-xen is not yet the default. Complexity arising from
> >> >                splitting qemu-for-qdisk out from qemu-for-dm and
> >> >                running N qemu's.
> >> >              * potentially fully userspace blktap could be ready for
> >> >                4.2
> >> >              * use /dev/loop+blkback. This requires loop driver AIO and
> >> >                O_DIRECT patches which are not (AFAIK) yet upstream.
> >> >              * Leverage XCP's blktap2 DKMS work.
> >> >              * Other ideas?
> >> >      * Improved Hotplug script support (Roger Pau Monné, patches
> >> >        posted)
> >> >      * Block script support -- follows on from hotplug script (Roger
> >> >        Pau Monné)
> >> >
> >> > hypervisor, nice to have:
> >> >      * solid implementation of sharing/paging/mem-events (using work
> >> >        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
> >> >              * "The last patch to use a waitqueue in
> >> >                __get_gfn_type_access() from Tim works.  However, there
> >> >                are a few users who call __get_gfn_type_access with the
> >> >                domain_lock held. This part needs to be addressed in
> >> >                some way."
> >> >      * Sharing support for AMD (Tim, Andres).
> >> >      * PoD performance improvements (George Dunlap)
> >> >
> >> > tools, nice to have:
> >> >      * Configure/control paging via xl/libxl (Olaf Herring, lots of
> >> >        discussion around interface, general consensus reached on what
> >> >        it should look like)
> >> >      * Upstream qemu feature patches:
> >> >              * Upstream qemu PCI passthrough support (Anthony Perard,
> >> >                patches sent)
> >> >              * Upstream qemu save restore (Anthony Perard, Stefano
> >> >                Stabellini, patches sent, waiting for upstream ack)
> >> >      * Nested-virtualisation. Currently "experimental". Likely to
> >> >        release that way.
> >> >              * Nested SVM. Tested in a variety of configurations but
> >> >                still some issues with the most important use case (w7
> >> >                XP mode) [0]  (Christoph Egger)
> >> >              * Nested VMX. Needs nested EPT to be genuinely useful.
> >> >                Need more data on testedness etc (Intel)
> >> >      * Initial xl support for Remus (memory checkpoint, blackholing)
> >> >        (Shriram, patches posted, blocked behind qemu save restore
> >> >        patches)
> >> >      * xl compatibility with xm:
> >> >              * xl support for autospawning vncviewer (vncviewer=1 or
> >> >                otherwise) (Goncalo Gomes)
> >> >              * support for vif "rate" parameter (Mathieu Gagné)
> >> >
> >> > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
> >> >
> >> >
> >> > _______________________________________________
> >> > Xen-devel mailing list
> >> > Xen-devel@lists.xen.org
> >> > http://lists.xen.org/xen-devel
> >
> >



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Keir Fraser]

On 22/03/2012 10:19, "Ian Campbell" <Ian.Campbell@citrix.com> wrote:

> I told a lie -- the code does seem to be there in
> mainline
(drivers/xen/xen-pciback/conf_space_quirks.c et al). Not sure how
> grep
missed it.

Does anyone know what the actual purpose/function of the
> single defined
quirk is? 10845:df80de098d15 which introduces it doesn't really
> say,
it's just a bunch of magic register frobbing as far as I'm concerned.

I
> guess you have a tg3 and are suffering from this exact quirk?

It's an awful
> lot of scaffolding on both the userspace and kernel side
to support a generic
> quirks system which has had exactly one quirk since
it was introduced in mid
> 2006. Perhaps we should just address the
specific tg3 issue directly?


The issue is that 'quirk' is something of a misnomer. Many devices may have
device-specific registers exposed via their PCI config space. The default
pciback policy is set to allow only known generic PCI config space registers
to be accessed by a guest. This will fail for likely a fair few devices, and
the most common solution is to set the pciback.permissive flag and just
allow guest access to all of the device's PCI config space. This is far less
hassle than actually setting up a proper 'quirk' and frankly most people
trust the device drivers they run, even if they are in a domU. I wonder how
many people really give a crap about the protection of !pciback.permissive.

 -- Keir

Re: Xen 4.2 Release Plan / TODO

[George Dunlap]

On Thu, Mar 22, 2012 at 10:19 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Thu, 2012-03-22 at 10:08 +0000, George Dunlap wrote:
>> On Thu, Mar 22, 2012 at 9:53 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > On Thu, 2012-03-22 at 09:35 +0000, George Dunlap wrote:
>> >> On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> >> >      * xl compatibility with xm:
>> >> >              * feature parity wrt driver domain support (George Dunlap)
>> >> I just discovered (while playing with driver domains) that xl is
>> >> missing one bit of feature parity with xm for pci passthrough for PV
>> >> guests -- and that's the "pci quirk" config file support.  I'm going
>> >> to ask Intel if they have an interest in porting it over; I think it
>> >> should at least be a "nice-to-have", and it may be a low-level
>> >> blocker, as a lot of devices won't work passed through without it.
>> >
>> > This is the stuff in tools/python/xen/xend/server/pciquirk.py ?
>> >
>> > pciback in upstream doesn't mention "quirk" which suggests no support
>> > for the necessary sysfs node either?
>>
>> Ah, interesting -- that's worth tracking down.  Maybe there's a better
>> way to deal with quirks?  Or maybe it just hasn't been upstreamed yet
>> (or perhaps even implemented in pvops?).  I'm using the Debian squeeze
>> 2.6.32-5-xen-686 kernel.
>
> I told a lie -- the code does seem to be there in mainline
> (drivers/xen/xen-pciback/conf_space_quirks.c et al). Not sure how grep
> missed it.
>
> Does anyone know what the actual purpose/function of the single defined
> quirk is? 10845:df80de098d15 which introduces it doesn't really say,
> it's just a bunch of magic register frobbing as far as I'm concerned.
>
> I guess you have a tg3 and are suffering from this exact quirk?
>
> It's an awful lot of scaffolding on both the userspace and kernel side
> to support a generic quirks system which has had exactly one quirk since
> it was introduced in mid 2006. Perhaps we should just address the
> specific tg3 issue directly?

On the contrary, I don't have a tg3, but an Intel nic that uses
Linux's igd driver, and another that uses the bnx2 driver.  When I
pass those through to a PV guest, I get the following messages printed
from dom0's pciback, respectively:

(for igd)
[   77.619293] pciback 0000:07:00.0: PCI INT A -> GSI 45 (level, low)
-> IRQ 45^M
[   77.626683] pciback 0000:07:00.0: Driver tried to write to a
read-only configuration space field at offset 0xa8, size 2. This may
be harmless, but if you have problems with your device:^M
[   77.626685] 1) see permissive attribute in sysfs^M
[   77.626687] 2) report problems to the xen-devel mailing list along
with details of your device obtained from lspci.^M

(for bnx2)
[  363.582059] pciback 0000:02:00.0: PCI INT A -> GSI 32 (level, low)
-> IRQ 32^M
[  363.590050] pciback 0000:02:00.0: Driver tried to write to a
read-only configuration space field at offset 0x68, size 4. This may
be harmless, but if you have problems with your device:^M
[  363.590054] 1) see permissive attribute in sysfs^M
[  363.590055] 2) report problems to the xen-devel mailing list along
with details of your device obtained from lspci.^M

And at least one person has solved this by adding something to the
"quirks" file (search for "PCI permissions"):
http://technical.bestgrid.org/index.php/Xen:_assigning_PCI_devices_to_a_domain

The devices in fact don't work quite right, AFAICT.  So I'm gathering
that the "quirks" lists areas of PCI configuration space which it is
safe for pciback to allow the guest to modify.  (I'm going to hack
libxl to set "permissive" to test this theory.)

 -George

>
>>
>> > tools/examples/xend-pci-quirks.sxp  seems to only have a quirk for a
>> > single card?
>>
>> Yes, well I could add two more cards just from experience w/ one of my
>> test boxen. :-)
>>
>> > I don't think we want to implement an SXP parser for xl/libxl so if this
>> > is reimplemented I think a different format should be used.
>>
>> Since we're using yajl anyway, JSON might not be a bad option.
>>
>> Anyway, I'll ping the Intel guy who recently posted a patch to libxl_pci.c.
>>
>>  -George
>>
>> >
>> > Anyway, I'll put this onto the list.
>> >
>> > Ian
>> >
>> >>
>> >> >              * xl support for "rtc_timeoffset" and "localtime" (Lin
>> >> >                Ming, Patches posted)
>> >> >      * More formally deprecate xm/xend. Manpage patches already in
>> >> >        tree. Needs release noting and communication around -rc1 to
>> >> >        remind people to test xl.
>> >> >      * Domain 0 block attach & general hotplug when using qdisk backend
>> >> >        (need to start qemu as necessary etc) (Stefano S)
>> >> >      * file:// backend performance. qemu-xen-tradition's qdisk is quite
>> >> >        slow & blktap2 not available in upstream kernels. Need to
>> >> >        consider our options:
>> >> >              * qemu-xen's qdisk is thought to be well performing but
>> >> >                qemu-xen is not yet the default. Complexity arising from
>> >> >                splitting qemu-for-qdisk out from qemu-for-dm and
>> >> >                running N qemu's.
>> >> >              * potentially fully userspace blktap could be ready for
>> >> >                4.2
>> >> >              * use /dev/loop+blkback. This requires loop driver AIO and
>> >> >                O_DIRECT patches which are not (AFAIK) yet upstream.
>> >> >              * Leverage XCP's blktap2 DKMS work.
>> >> >              * Other ideas?
>> >> >      * Improved Hotplug script support (Roger Pau Monné, patches
>> >> >        posted)
>> >> >      * Block script support -- follows on from hotplug script (Roger
>> >> >        Pau Monné)
>> >> >
>> >> > hypervisor, nice to have:
>> >> >      * solid implementation of sharing/paging/mem-events (using work
>> >> >        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
>> >> >              * "The last patch to use a waitqueue in
>> >> >                __get_gfn_type_access() from Tim works.  However, there
>> >> >                are a few users who call __get_gfn_type_access with the
>> >> >                domain_lock held. This part needs to be addressed in
>> >> >                some way."
>> >> >      * Sharing support for AMD (Tim, Andres).
>> >> >      * PoD performance improvements (George Dunlap)
>> >> >
>> >> > tools, nice to have:
>> >> >      * Configure/control paging via xl/libxl (Olaf Herring, lots of
>> >> >        discussion around interface, general consensus reached on what
>> >> >        it should look like)
>> >> >      * Upstream qemu feature patches:
>> >> >              * Upstream qemu PCI passthrough support (Anthony Perard,
>> >> >                patches sent)
>> >> >              * Upstream qemu save restore (Anthony Perard, Stefano
>> >> >                Stabellini, patches sent, waiting for upstream ack)
>> >> >      * Nested-virtualisation. Currently "experimental". Likely to
>> >> >        release that way.
>> >> >              * Nested SVM. Tested in a variety of configurations but
>> >> >                still some issues with the most important use case (w7
>> >> >                XP mode) [0]  (Christoph Egger)
>> >> >              * Nested VMX. Needs nested EPT to be genuinely useful.
>> >> >                Need more data on testedness etc (Intel)
>> >> >      * Initial xl support for Remus (memory checkpoint, blackholing)
>> >> >        (Shriram, patches posted, blocked behind qemu save restore
>> >> >        patches)
>> >> >      * xl compatibility with xm:
>> >> >              * xl support for autospawning vncviewer (vncviewer=1 or
>> >> >                otherwise) (Goncalo Gomes)
>> >> >              * support for vif "rate" parameter (Mathieu Gagné)
>> >> >
>> >> > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Xen-devel mailing list
>> >> > Xen-devel@lists.xen.org
>> >> > http://lists.xen.org/xen-devel
>> >
>> >
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Thu, 2012-03-22 at 10:34 +0000, George Dunlap wrote:
> The devices in fact don't work quite right, AFAICT.  So I'm gathering
> that the "quirks" lists areas of PCI configuration space which it is
> safe for pciback to allow the guest to modify.  (I'm going to hack
> libxl to set "permissive" to test this theory.)

I think that based on what Keir says we should just expose the
permissive setting in xl/libxl for the time being, that's a far simpler
patch. No need for the full power of per-register whitelisting

Ian.

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Thu, 2012-03-22 at 10:38 +0000, Ian Campbell wrote:
> On Thu, 2012-03-22 at 10:34 +0000, George Dunlap wrote:
> > The devices in fact don't work quite right, AFAICT.  So I'm gathering
> > that the "quirks" lists areas of PCI configuration space which it is
> > safe for pciback to allow the guest to modify.  (I'm going to hack
> > libxl to set "permissive" to test this theory.)
> 
> I think that based on what Keir says we should just expose the
> permissive setting in xl/libxl for the time being, that's a far simpler
> patch. No need for the full power of per-register whitelisting

Looks like this is just a case of writing the BDF of a device to a sysfs
node or booting with xen-pciback.permissive=1 to turn it on globally.
Possibly this could be "fixed" purely by documentation? I've added it to
the TODO as a nice to have with George's name next to it, hope that's
OK.

Ian.

Re: Xen 4.2 Release Plan / TODO

[George Dunlap]

On Tue, Mar 27, 2012 at 10:33 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Thu, 2012-03-22 at 10:38 +0000, Ian Campbell wrote:
>> On Thu, 2012-03-22 at 10:34 +0000, George Dunlap wrote:
>> > The devices in fact don't work quite right, AFAICT.  So I'm gathering
>> > that the "quirks" lists areas of PCI configuration space which it is
>> > safe for pciback to allow the guest to modify.  (I'm going to hack
>> > libxl to set "permissive" to test this theory.)
>>
>> I think that based on what Keir says we should just expose the
>> permissive setting in xl/libxl for the time being, that's a far simpler
>> patch. No need for the full power of per-register whitelisting
>
> Looks like this is just a case of writing the BDF of a device to a sysfs
> node or booting with xen-pciback.permissive=1 to turn it on globally.
> Possibly this could be "fixed" purely by documentation? I've added it to
> the TODO as a nice to have with George's name next to it, hope that's
> OK.

It seems like having xl/libxl capable of doing the sysfs writing is a
better thing, and shouldn't be too hard really.  I'll try to hack that
up this week.

Going forward, it would be nice if xl could do hot unplug and pciback
binding, so people wouldn't have to muck about with rmmod and manual
sysfs twiddling; but that will have to be a 4.3 thing I guess.

 -George

>
> Ian.
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Xen 4.2 Release Plan / TODO

[Ian Campbell]

A day late due to the document day yesterday.

Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html

The time line is as follows:

19 March        -- TODO list locked down
					        << WE ARE HERE
2 April         -- Feature Freeze		<< THIS IS NEXT WEEK
Mid/Late April  -- First release candidate
Weekly          -- RCN+1 until it is ready

The updated TODO list follows. Per the release plan a strong case will
now have to be made as to why new items should be added to the list,
especially as a blocker, rather than deferred to 4.3.

Feature Freeze is this Monday. At this point we will begin deferring
patches which have not previously been posted and which are not on the
list below until Xen 4.3.

Ian.

hypervisor, blockers:
      * NONE?
 
tools, blockers:
      * libxl stable API -- we would like 4.2 to define a stable API
        which downstream's can start to rely on not changing. Aspects of
        this are:
              * Safe fork vs. fd handling hooks. Involves API changes
                (Ian J, patches posted)
              * locking/serialization, e.g., for domain creation. As of
                now,  nothing is provided for this purpose, and
                downstream toolstacks have to put their own mechanisms
                in place. E.g., xl uses a fcntl() lock around the whole
                process of domain creation. It should OTOH be libxl job
                to offer a proper set of hooks --placed at the proper
                spots during the domain creation process-- for the
                downstreams to  fill with the proper callbacks. (Dario
                Faggioli)
              * agree & document compatibility guarantees + associated
                technical measures (Ian C, patches posted)
      * xl compatibility with xm:
              * feature parity wrt driver domain support (George Dunlap)
              * xl support for "rtc_timeoffset" and "localtime" (Lin
                Ming, Patches posted)
      * More formally deprecate xm/xend. Manpage patches already in
        tree. Needs release noting and communication around -rc1 to
        remind people to test xl.
      * Domain 0 block attach & general hotplug when using qdisk backend
        (need to start qemu as necessary etc) (Stefano S)
      * file:// backend performance. qemu-xen-tradition's qdisk is quite
        slow & blktap2 not available in upstream kernels. Need to
        consider our options:
                      * qemu-xen's qdisk is thought to be well
                        performing but qemu-xen is not yet the default.
                        Complexity arising from splitting qemu-for-qdisk
                        out from qemu-for-dm and running N qemu's.
                      * potentially fully userspace blktap could be
                        ready for 4.2
                      * use /dev/loop+blkback. This requires loop driver
                        AIO and O_DIRECT patches which are not (AFAIK)
                        yet upstream.
                      * Leverage XCP's blktap2 DKMS work.
                      * Other ideas?
                              * In general we should recommend blkback
                                (e.g. with LVM) since it always out
                                performs other solutions, although at
                                the expense of flexibility regarding
                                image formats.
        Stefano has done a lot of work here and has proposed some
        performance improvements for qdisk in both qemu-xen and
        qemu-xen-traditional which make them a plausible alternative for
        Xen 4.2.
      * Improved Hotplug script support (Roger Pau Monné, patches
        posted)
      * Block script support -- follows on from hotplug script (Roger
        Pau Monné)

hypervisor, nice to have:
      * solid implementation of sharing/paging/mem-events (using work
        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
              * "The last patch to use a waitqueue in
                __get_gfn_type_access() from Tim works.  However, there
                are a few users who call __get_gfn_type_access with the
                domain_lock held. This part needs to be addressed in
                some way."
      * Sharing support for AMD (Tim, Andres).
      * PoD performance improvements (George Dunlap)

tools, nice to have:
      * Configure/control paging via xl/libxl (Olaf Herring, lots of
        discussion around interface, general consensus reached on what
        it should look like. Olaf has let me know that he is probably
        not going to have time to do this for 4.2, will likely slip to
        4.3 unless someone else want to pick up that work)
      * Upstream qemu feature patches:
              * Upstream qemu PCI passthrough support (Anthony Perard,
                patches sent)
              * Upstream qemu save restore (Anthony Perard, Stefano
                Stabellini, qemu patches applied, waiting for libxl etc
                side which has been recently reposted)
      * Nested-virtualisation. Currently "experimental". Likely to
        release that way.
              * Nested SVM. Tested in a variety of configurations but
                still some issues with the most important use case (w7
                XP mode) [0]  (Christoph Egger)
              * Nested VMX. Needs nested EPT to be genuinely useful.
                Need more data on testedness etc (Intel)
      * Initial xl support for Remus (memory checkpoint, blackholing)
        (Shriram, patches reposted recently, was blocked behind qemu
        save restore patches which are now in)
      * xl compatibility with xm:
              * xl support for autospawning vncviewer (vncviewer=1 or
                otherwise) (Goncalo Gomes, patches posted)
              * support for vif "rate" parameter (Mathieu Gagné, patches
                posted)
              * expose PCI back "permissive" parameter (George Dunlap)

[0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Shriram Rajagopalan]

[-- Attachment #1.1: Type: text/plain, Size: 1422 bytes --]

On Tue, Mar 27, 2012 at 2:34 AM, Ian Campbell <Ian.Campbell@citrix.com>wrote:

> A day late due to the document day yesterday.
>
> Plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
>
> The time line is as follows:
>
> 19 March        -- TODO list locked down
>                                                << WE ARE HERE
> 2 April         -- Feature Freeze               << THIS IS NEXT WEEK
> Mid/Late April  -- First release candidate
> Weekly          -- RCN+1 until it is ready
>      * Initial xl support for Remus (memory checkpoint, blackholing)
>         (Shriram, patches reposted recently, was blocked behind qemu
>        save restore patches which are now in)
>

I reposted the patches but I dont see any of them in the staging tree
(nor do I see stefano's patches).




>       * xl compatibility with xm:
>              * xl support for autospawning vncviewer (vncviewer=1 or
>                 otherwise) (Goncalo Gomes, patches posted)
>              * support for vif "rate" parameter (Mathieu Gagné, patches
>                posted)
>              * expose PCI back "permissive" parameter (George Dunlap)
>
> [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

[-- Attachment #1.2: Type: text/html, Size: 2323 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Xen 4.2 Release Plan / TODO

[Ian Campbell]

Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html

The time line is as follows:

19 March        -- TODO list locked down
2 April         -- Feature Freeze               << WE ARE HERE
Mid/Late April  -- First release candidate
Weekly          -- RCN+1 until it is ready

The updated TODO list follows. Per the release plan a strong case will
now have to be made as to why new items should be added to the list,
especially as a blocker, rather than deferred to 4.3.

We have now entered Feature Freeze. Patches which have been posted
before or which address something on the list below are still acceptable
(for now, we will gradually be getting stricter about this), everything
else will be deferred until 4.3 by default.

hypervisor, blockers:
      * NONE?
 
tools, blockers:
      * libxl stable API -- we would like 4.2 to define a stable API
        which downstream's can start to rely on not changing. Aspects of
        this are:
              * Safe fork vs. fd handling hooks. Involves API changes
                (Ian J, patches posted)
              * locking/serialization, e.g., for domain creation. As of
                now,  nothing is provided for this purpose, and
                downstream toolstacks have to put their own mechanisms
                in place. E.g., xl uses a fcntl() lock around the whole
                process of domain creation. It should OTOH be libxl job
                to offer a proper set of hooks --placed at the proper
                spots during the domain creation process-- for the
                downstreams to  fill with the proper callbacks. (Dario
                Faggioli)
              * agree & document compatibility guarantees + associated
                technical measures (Ian C, patches posted)
      * xl compatibility with xm:
              * feature parity wrt driver domain support (George Dunlap)
              * xl support for "rtc_timeoffset" and "localtime" (Lin
                Ming, Patches posted)
      * More formally deprecate xm/xend. Manpage patches already in
        tree. Needs release noting and communication around -rc1 to
        remind people to test xl.
      * Domain 0 block attach & general hotplug when using qdisk backend
        (need to start qemu as necessary etc) (Stefano S)
      * file:// backend performance. qemu-xen-tradition's qdisk is quite
        slow & blktap2 not available in upstream kernels. Need to
        consider our options:
              * qemu-xen's qdisk is thought to be well performing but
                qemu-xen is not yet the default. Complexity arising from
                splitting qemu-for-qdisk out from qemu-for-dm and
                running N qemu's.
              * potentially fully userspace blktap could be ready for
                4.2
              * use /dev/loop+blkback. This requires loop driver AIO and
                O_DIRECT patches which are not (AFAIK) yet upstream.
              * Leverage XCP's blktap2 DKMS work.
              * Other ideas?
                      * In general we should recommend blkback (e.g.
                        with LVM) since it always out performs other
                        solutions, although at the expense of
                        flexibility regarding image formats.
        Stefano has done a lot of work here and has proposed some
        performance improvements for qdisk in both qemu-xen and
        qemu-xen-traditional which make them a plausible alternative for
        Xen 4.2.
      * Improved Hotplug script support (Roger Pau Monné, patches
        posted)
      * Block script support -- follows on from hotplug script (Roger
        Pau Monné)

hypervisor, nice to have:
      * solid implementation of sharing/paging/mem-events (using work
        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
              * "The last patch to use a waitqueue in
                __get_gfn_type_access() from Tim works.  However, there
                are a few users who call __get_gfn_type_access with the
                domain_lock held. This part needs to be addressed in
                some way."
      * Sharing support for AMD (Tim, Andres).
      * PoD performance improvements (George Dunlap)

tools, nice to have:
      * Configure/control paging via xl/libxl (Olaf Herring, lots of
        discussion around interface, general consensus reached on what
        it should look like. Olaf has let me know that he is probably
        not going to have time to do this for 4.2, will likely slip to
        4.3 unless someone else want to pick up that work)
      * Upstream qemu feature patches:
              * Upstream qemu PCI passthrough support (Anthony Perard,
                patches sent)
              * Upstream qemu save restore (Anthony Perard, Stefano
                Stabellini, qemu patches applied, waiting for libxl etc
                side which has been recently reposted)
      * Nested-virtualisation. Currently "experimental". Likely to
        release that way.
              * Nested SVM. Tested in a variety of configurations but
                still some issues with the most important use case (w7
                XP mode) [0]  (Christoph Egger)
              * Nested VMX. Needs nested EPT to be genuinely useful.
                Need more data on testedness etc (Intel)
      * Initial xl support for Remus (memory checkpoint, blackholing)
        (Shriram, patches reposted recently, was blocked behind qemu
        save restore patches which are now in)
      * xl compatibility with xm:
              * xl support for autospawning vncviewer (vncviewer=1 or
                otherwise) (Goncalo Gomes, patches posted)
              * support for vif "rate" parameter (Mathieu Gagné, patches
                posted)
              * expose PCI back "permissive" parameter (George Dunlap)

[0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[David Vrabel]

On 02/04/12 11:26, Ian Campbell wrote:
> 
> We have now entered Feature Freeze. Patches which have been posted
> before or which address something on the list below are still acceptable
> (for now, we will gradually be getting stricter about this), everything
> else will be deferred until 4.3 by default.

How does this affect ARM patches?  Are they similarly restricted?

David

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Mon, 2012-04-02 at 11:39 +0100, David Vrabel wrote:
> On 02/04/12 11:26, Ian Campbell wrote:
> > 
> > We have now entered Feature Freeze. Patches which have been posted
> > before or which address something on the list below are still acceptable
> > (for now, we will gradually be getting stricter about this), everything
> > else will be deferred until 4.3 by default.
> 
> How does this affect ARM patches?  Are they similarly restricted?

Given that ARM support in 4.2 is going to be experimental in nature I
suggest that, at least for the time being, ARM patches which do not have
an impact on generic or non-ARM code continue to be accepted.

Any objections?

Ian.

Re: Xen 4.2 Release Plan / TODO

[George Dunlap]

On Mon, Apr 2, 2012 at 11:26 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> Plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
>
> The time line is as follows:
>
> 19 March        -- TODO list locked down
> 2 April         -- Feature Freeze               << WE ARE HERE
> Mid/Late April  -- First release candidate
> Weekly          -- RCN+1 until it is ready
>
> The updated TODO list follows. Per the release plan a strong case will
> now have to be made as to why new items should be added to the list,
> especially as a blocker, rather than deferred to 4.3.
>
> We have now entered Feature Freeze. Patches which have been posted
> before or which address something on the list below are still acceptable
> (for now, we will gradually be getting stricter about this), everything
> else will be deferred until 4.3 by default.
>
> hypervisor, blockers:
>      * NONE?
>
> tools, blockers:
>      * libxl stable API -- we would like 4.2 to define a stable API
>        which downstream's can start to rely on not changing. Aspects of
>        this are:
>              * Safe fork vs. fd handling hooks. Involves API changes
>                (Ian J, patches posted)
>              * locking/serialization, e.g., for domain creation. As of
>                now,  nothing is provided for this purpose, and
>                downstream toolstacks have to put their own mechanisms
>                in place. E.g., xl uses a fcntl() lock around the whole
>                process of domain creation. It should OTOH be libxl job
>                to offer a proper set of hooks --placed at the proper
>                spots during the domain creation process-- for the
>                downstreams to  fill with the proper callbacks. (Dario
>                Faggioli)
>              * agree & document compatibility guarantees + associated
>                technical measures (Ian C, patches posted)
>      * xl compatibility with xm:
>              * feature parity wrt driver domain support (George Dunlap)

The only thing missing is the pci "permissive" flag.  Patches posted.

>              * xl support for "rtc_timeoffset" and "localtime" (Lin
>                Ming, Patches posted)
>      * More formally deprecate xm/xend. Manpage patches already in
>        tree. Needs release noting and communication around -rc1 to
>        remind people to test xl.
>      * Domain 0 block attach & general hotplug when using qdisk backend
>        (need to start qemu as necessary etc) (Stefano S)
>      * file:// backend performance. qemu-xen-tradition's qdisk is quite
>        slow & blktap2 not available in upstream kernels. Need to
>        consider our options:
>              * qemu-xen's qdisk is thought to be well performing but
>                qemu-xen is not yet the default. Complexity arising from
>                splitting qemu-for-qdisk out from qemu-for-dm and
>                running N qemu's.
>              * potentially fully userspace blktap could be ready for
>                4.2
>              * use /dev/loop+blkback. This requires loop driver AIO and
>                O_DIRECT patches which are not (AFAIK) yet upstream.
>              * Leverage XCP's blktap2 DKMS work.
>              * Other ideas?
>                      * In general we should recommend blkback (e.g.
>                        with LVM) since it always out performs other
>                        solutions, although at the expense of
>                        flexibility regarding image formats.
>        Stefano has done a lot of work here and has proposed some
>        performance improvements for qdisk in both qemu-xen and
>        qemu-xen-traditional which make them a plausible alternative for
>        Xen 4.2.
>      * Improved Hotplug script support (Roger Pau Monné, patches
>        posted)
>      * Block script support -- follows on from hotplug script (Roger
>        Pau Monné)
>
> hypervisor, nice to have:
>      * solid implementation of sharing/paging/mem-events (using work
>        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
>              * "The last patch to use a waitqueue in
>                __get_gfn_type_access() from Tim works.  However, there
>                are a few users who call __get_gfn_type_access with the
>                domain_lock held. This part needs to be addressed in
>                some way."
>      * Sharing support for AMD (Tim, Andres).
>      * PoD performance improvements (George Dunlap)
>
> tools, nice to have:
>      * Configure/control paging via xl/libxl (Olaf Herring, lots of
>        discussion around interface, general consensus reached on what
>        it should look like. Olaf has let me know that he is probably
>        not going to have time to do this for 4.2, will likely slip to
>        4.3 unless someone else want to pick up that work)
>      * Upstream qemu feature patches:
>              * Upstream qemu PCI passthrough support (Anthony Perard,
>                patches sent)
>              * Upstream qemu save restore (Anthony Perard, Stefano
>                Stabellini, qemu patches applied, waiting for libxl etc
>                side which has been recently reposted)
>      * Nested-virtualisation. Currently "experimental". Likely to
>        release that way.
>              * Nested SVM. Tested in a variety of configurations but
>                still some issues with the most important use case (w7
>                XP mode) [0]  (Christoph Egger)
>              * Nested VMX. Needs nested EPT to be genuinely useful.
>                Need more data on testedness etc (Intel)
>      * Initial xl support for Remus (memory checkpoint, blackholing)
>        (Shriram, patches reposted recently, was blocked behind qemu
>        save restore patches which are now in)
>      * xl compatibility with xm:
>              * xl support for autospawning vncviewer (vncviewer=1 or
>                otherwise) (Goncalo Gomes, patches posted)
>              * support for vif "rate" parameter (Mathieu Gagné, patches
>                posted)
>              * expose PCI back "permissive" parameter (George Dunlap)
>
> [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Stefano Stabellini]

On Mon, 2 Apr 2012, Ian Campbell wrote:
>       * file:// backend performance. qemu-xen-tradition's qdisk is quite
>         slow & blktap2 not available in upstream kernels. Need to
>         consider our options:
>               * qemu-xen's qdisk is thought to be well performing but
>                 qemu-xen is not yet the default. Complexity arising from
>                 splitting qemu-for-qdisk out from qemu-for-dm and
>                 running N qemu's.
>               * potentially fully userspace blktap could be ready for
>                 4.2
>               * use /dev/loop+blkback. This requires loop driver AIO and
>                 O_DIRECT patches which are not (AFAIK) yet upstream.
>               * Leverage XCP's blktap2 DKMS work.
>               * Other ideas?
>                       * In general we should recommend blkback (e.g.
>                         with LVM) since it always out performs other
>                         solutions, although at the expense of
>                         flexibility regarding image formats.
>         Stefano has done a lot of work here and has proposed some
>         performance improvements for qdisk in both qemu-xen and
>         qemu-xen-traditional which make them a plausible alternative for
>         Xen 4.2.

Using O_DIRECT rather than the default write-through cache policy solves
the performance problem in QEMU.
I sent patches to xen-devel to enable O_DIRECT for QDISK on
qemu-xen-traditional. I also sent patches to qemu-devel to enable
O_DIRECT and native AIO for QDISK on upstream QEMU.

Upstream QEMU PV backends are as good as the ones in
qemu-xen-traditional, but upstream QDISK performs better because it can
use native AIO. Thus I sent a patch to xen-devel to enable upstream QEMU
as default to provide userspace backends to PV guests.

I wrote and sent a patch series to fix the m2p override in Linux in case
the frontend and backend are both in the same domain.
Then I sent a patch series to xen-devel to make
libxl__device_disk_local_attach work with QDISK: the implementation uses
a frontend/backend pair in dom0.

As a result, with all these patches applied, the disk performances with
file based disk images are good and one can use a qcow2 file to store
a PV guest disk image and boot from it using pygrub.

Re: Xen 4.2 Release Plan / TODO

[Ian Jackson]

Ian Campbell writes ("Xen 4.2 Release Plan / TODO"):
> Plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
...
> tools, blockers:
>       * libxl stable API -- we would like 4.2 to define a stable API
>         which downstream's can start to rely on not changing. Aspects of
>         this are:

I took a look at libxl.h and came up with the comments below.  Firstly
general things I tripped over, and then the list of things which need
converting to the new event system.

Ian.


Other:
======

]   int libxl_wait_for_free_memory(libxl_ctx *ctx, uint32_t domid, uint32_t memory_kb, int wait_secs);
]   /* wait for the memory target of a domain to be reached */
]   int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid, int wait_secs);

This whole memory interface is a bit of a dog's breakfast.

]   int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass);
]   int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num, libxl_console_type type);
]   /* libxl_primary_console_exec finds the domid and console number
]    * corresponding to the primary console of the given vm, then calls
]    * libxl_console_exec with the right arguments (domid might be different
]    * if the guest is using stubdoms).
]    * This function can be called after creating the device model, in
]    * case of HVM guests, and before libxl_run_bootloader in case of PV
]    * guests using pygrub. */
]   int libxl_primary_console_exec(libxl_ctx *ctx, uint32_t domid_vm);

These functions should not exec things for you; they should tell you
the parameters.  Any execing helpers should be in libxlu.

]   /* common paths */
]   const char *libxl_sbindir_path(void);
]   const char *libxl_bindir_path(void);
]   const char *libxl_libexec_path(void);
]   const char *libxl_libdir_path(void);
]   const char *libxl_sharedir_path(void);
]   const char *libxl_private_bindir_path(void);
]   const char *libxl_xenfirmwaredir_path(void);
]   const char *libxl_xen_config_dir_path(void);
]   const char *libxl_xen_script_dir_path(void);
]   const char *libxl_lock_dir_path(void);
]   const char *libxl_run_dir_path(void);
]   const char *libxl_xenpaging_dir_path(void);

Surely these should be private ?


Need to be ao/eventified:
=========================

]   typedef struct {
]   #define XL_SUSPEND_DEBUG 1
]   #define XL_SUSPEND_LIVE 2
]       int flags;
]       int (*suspend_callback)(void *, int);
]   } libxl_domain_suspend_info;
...
]   int libxl_domain_suspend(libxl_ctx *ctx, libxl_domain_suspend_info *info,
]                            uint32_t domid, int fd);

]   typedef int (*libxl_console_ready)(libxl_ctx *ctx, uint32_t domid, void *priv);
]   int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, libxl_console_ready cb, void *priv, uint32_t *domid);
]   int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, libxl_console_ready cb, void *priv, uint32_t *domid, int restore_fd);

]   int libxl_domain_shutdown(libxl_ctx *ctx, uint32_t domid);
]   int libxl_domain_reboot(libxl_ctx *ctx, uint32_t domid);

Are these now merely initiations ?

]   int libxl_domain_core_dump(libxl_ctx *ctx, uint32_t domid, const char *filename);

Might become long-running in the future.

]   int libxl_device_disk_add(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk);

]   /*
]    * Insert a CD-ROM device. A device corresponding to disk must already
]    * be attached to the guest.
]    */
]   int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk);

]   /*
]    * Make a disk available in this (the control) domain. Returns path to
]    * a device.
]    */
]   char * libxl_device_disk_local_attach(libxl_ctx *ctx, libxl_device_disk *disk);
]   int libxl_device_disk_local_detach(libxl_ctx *ctx, libxl_device_disk *disk);

Does this even need to be public at this stage ?

]   /* Network Interfaces */
]   int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_device_nic *nic);

]   /* Keyboard */
]   int libxl_device_vkb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_vkb *vkb);

]   /* Framebuffer */
]   int libxl_device_vfb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_vfb *vfb);

]   /* PCI Passthrough */
]   int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid, libxl_device_pci *pcidev);
]   int libxl_device_pci_remove(libxl_ctx *ctx, uint32_t domid, libxl_device_pci *pcidev);

]   typedef struct libxl__xen_console_reader libxl_xen_console_reader;
]
]   libxl_xen_console_reader *
]       libxl_xen_console_read_start(libxl_ctx *ctx, int clear);
]   int libxl_xen_console_read_line(libxl_ctx *ctx,
]                                   libxl_xen_console_reader *cr,
]                                   char **line_r);
]   void libxl_xen_console_read_finish(libxl_ctx *ctx,
]                                      libxl_xen_console_reader *cr);

]   char *libxl_tmem_list(libxl_ctx *ctx, uint32_t domid, int use_long);
]   int libxl_tmem_freeze(libxl_ctx *ctx, uint32_t domid);
]   int libxl_tmem_destroy(libxl_ctx *ctx, uint32_t domid);
]   int libxl_tmem_thaw(libxl_ctx *ctx, uint32_t domid);
]   int libxl_tmem_set(libxl_ctx *ctx, uint32_t domid, char* name,
]                      uint32_t set);
]   int libxl_tmem_shared_auth(libxl_ctx *ctx, uint32_t domid, char* uuid,
]                              int auth);
]   int libxl_tmem_freeable(libxl_ctx *ctx);

Not sure about the tmem calls.

And from libxl_utils.h:

]   pid_t libxl_fork(libxl_ctx *ctx);

This function is going to have to go away.

-- 

Re: Xen 4.2 Release Plan / TODO

[Ian Jackson]

Ian Jackson writes ("Re: Xen 4.2 Release Plan / TODO"):
> Ian Campbell writes ("Xen 4.2 Release Plan / TODO"):
> > Plan for a 4.2 release:
> > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
> ...
> > tools, blockers:
> >       * libxl stable API -- we would like 4.2 to define a stable API
> >         which downstream's can start to rely on not changing. Aspects of
> >         this are:
> 
> I took a look at libxl.h and came up with the comments below.  Firstly
> general things I tripped over, and then the list of things which need
> converting to the new event system.

And I should mention that in my event series I have two
as-yet-unposted half-backed patches to rewrite libxl__spawn_* and
libxl_domain_create_*.

It may be that we can add dummy ao_hows to these functions which might
be good enough for now, although particularly for domain creation
(which includes spawning) it might complicate the efforts of users to
use the new API.

Currently any use of subprocesses inside libxl which is not dealt with
by the new event machinery will experience problems if the event loop
is also used, because the event loop will reap the children.

Ian.

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Wed, 2012-04-11 at 17:13 +0100, Ian Jackson wrote:
> Ian Jackson writes ("Re: Xen 4.2 Release Plan / TODO"):
> > Ian Campbell writes ("Xen 4.2 Release Plan / TODO"):
> > > Plan for a 4.2 release:
> > > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
> > ...
> > > tools, blockers:
> > >       * libxl stable API -- we would like 4.2 to define a stable API
> > >         which downstream's can start to rely on not changing. Aspects of
> > >         this are:
> > 
> > I took a look at libxl.h and came up with the comments below.  Firstly
> > general things I tripped over, and then the list of things which need
> > converting to the new event system.
> 
> And I should mention that in my event series I have two
> as-yet-unposted half-backed patches to rewrite libxl__spawn_* and
> libxl_domain_create_*.
> 
> It may be that we can add dummy ao_hows to these functions which might
> be good enough for now, although particularly for domain creation
> (which includes spawning) it might complicate the efforts of users to
> use the new API.

How close is your half baked series to do it properly?

> Currently any use of subprocesses inside libxl which is not dealt with
> by the new event machinery will experience problems if the event loop
> is also used, because the event loop will reap the children.

You've covered the bootloader one in existing patches and mentioned the
libxl_$CONSOLE_exec style ones in your last mail. The only other one is
the DM fork which is covered by your rewrite of libxl__spawn?

Ian.

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Wed, 2012-04-11 at 17:11 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Xen 4.2 Release Plan / TODO"):
> > Plan for a 4.2 release:
> > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
> ...
> > tools, blockers:
> >       * libxl stable API -- we would like 4.2 to define a stable API
> >         which downstream's can start to rely on not changing. Aspects of
> >         this are:
> 
> I took a look at libxl.h and came up with the comments below.  Firstly
> general things I tripped over, and then the list of things which need
> converting to the new event system.

A slightly worrying list at this stage in the game.

> 
> Ian.
> 
> 
> Other:
> ======
> 
> ]   int libxl_wait_for_free_memory(libxl_ctx *ctx, uint32_t domid, uint32_t memory_kb, int wait_secs);
> ]   /* wait for the memory target of a domain to be reached */
> ]   int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid, int wait_secs);
> 
> This whole memory interface is a bit of a dog's breakfast.

I think we can defer this to 4.3? The existing interface may be pants
but at least the name is pretty explicit that it will block. I think
this should then be easy enough to sort out in a backward compatible
manner in 4.3 since I expect the name of the function would change and
we could retain the old name in terms of the new for compat.

> ]   int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass);
> ]   int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num, libxl_console_type type);
> ]   /* libxl_primary_console_exec finds the domid and console number
> ]    * corresponding to the primary console of the given vm, then calls
> ]    * libxl_console_exec with the right arguments (domid might be different
> ]    * if the guest is using stubdoms).
> ]    * This function can be called after creating the device model, in
> ]    * case of HVM guests, and before libxl_run_bootloader in case of PV
> ]    * guests using pygrub. */
> ]   int libxl_primary_console_exec(libxl_ctx *ctx, uint32_t domid_vm);
> 
> These functions should not exec things for you; they should tell you
> the parameters.  Any execing helpers should be in libxlu.

It's not enough for them to just use libxl__exec?

It'd be reasonably easy to make this return a libxl_string_list or
similar and to write a libxlu function which takes one of those.

> ]   /* common paths */
> ]   const char *libxl_sbindir_path(void);
> ]   const char *libxl_bindir_path(void);
> ]   const char *libxl_libexec_path(void);
> ]   const char *libxl_libdir_path(void);
> ]   const char *libxl_sharedir_path(void);
> ]   const char *libxl_private_bindir_path(void);
> ]   const char *libxl_xenfirmwaredir_path(void);
> ]   const char *libxl_xen_config_dir_path(void);
> ]   const char *libxl_xen_script_dir_path(void);
> ]   const char *libxl_lock_dir_path(void);
> ]   const char *libxl_run_dir_path(void);
> ]   const char *libxl_xenpaging_dir_path(void);
> 
> Surely these should be private ?

As far as I can grep, yes.

> Need to be ao/eventified:
> =========================
> 
> ]   typedef struct {
> ]   #define XL_SUSPEND_DEBUG 1
> ]   #define XL_SUSPEND_LIVE 2
> ]       int flags;
> ]       int (*suspend_callback)(void *, int);
> ]   } libxl_domain_suspend_info;
> ...
> ]   int libxl_domain_suspend(libxl_ctx *ctx, libxl_domain_suspend_info *info,
> ]                            uint32_t domid, int fd);
> 
> ]   typedef int (*libxl_console_ready)(libxl_ctx *ctx, uint32_t domid, void *priv);
> ]   int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, libxl_console_ready cb, void *priv, uint32_t *domid);
> ]   int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, libxl_console_ready cb, void *priv, uint32_t *domid, int restore_fd);

You are on the case with these?

> ]   int libxl_domain_shutdown(libxl_ctx *ctx, uint32_t domid);
> ]   int libxl_domain_reboot(libxl_ctx *ctx, uint32_t domid);
> 
> Are these now merely initiations ?

I think so yes.

Does a non-transaction write make a function "slow"? That's all these
actually do. If they are currently "fast" then we could likely get away
with a dummy ao_how. (I think it is valid for a function which is "fast"
to take an ao_how and always run sync?)

> ]   int libxl_domain_core_dump(libxl_ctx *ctx, uint32_t domid, const char *filename);
> 
> Might become long-running in the future.

But is currently fast? Dummy ao_how?

> ]   int libxl_device_disk_add(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk);

Roger makes this async in his hotplug series.

> ]   /*
> ]    * Insert a CD-ROM device. A device corresponding to disk must already
> ]    * be attached to the guest.
> ]    */
> ]   int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk);

Were you looking at this one? I know you mentioned it at one point.

> ]   /*
> ]    * Make a disk available in this (the control) domain. Returns path to
> ]    * a device.
> ]    */
> ]   char * libxl_device_disk_local_attach(libxl_ctx *ctx, libxl_device_disk *disk);
> ]   int libxl_device_disk_local_detach(libxl_ctx *ctx, libxl_device_disk *disk);
> 
> Does this even need to be public at this stage ?

I think Stefano internalises them in his qdisk/dom0-attach series.

> ]   /* Network Interfaces */
> ]   int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_device_nic *nic);
> 
> ]   /* Keyboard */
> ]   int libxl_device_vkb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_vkb *vkb);
> 
> ]   /* Framebuffer */
> ]   int libxl_device_vfb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_vfb *vfb);

These ought to be pretty trivial conversions?

> ]   /* PCI Passthrough */
> ]   int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid, libxl_device_pci *pcidev);
> ]   int libxl_device_pci_remove(libxl_ctx *ctx, uint32_t domid, libxl_device_pci *pcidev);

I'm less confident that this one will be trivial to make async :-(

> ]   typedef struct libxl__xen_console_reader libxl_xen_console_reader;
> ]
> ]   libxl_xen_console_reader *
> ]       libxl_xen_console_read_start(libxl_ctx *ctx, int clear);
> ]   int libxl_xen_console_read_line(libxl_ctx *ctx,
> ]                                   libxl_xen_console_reader *cr,
> ]                                   char **line_r);
> ]   void libxl_xen_console_read_finish(libxl_ctx *ctx,
> ]                                      libxl_xen_console_reader *cr);

This is basically "xl dmesg". I'm not sure what interface makes sense
here, really it is just dumping the current ring, so each call is
"fast".

I'm not sure there is a need for an event driven "new-line-in-log"
callback style thing, i.e. a need to implement a "tail -f" style thing. 
Firstly I'm not sure that Xen actually produces an event which would
allow this to be implemented without polling and secondly if you want
that you can configure xenconsoled to log the hypervisor output and then
tail the logfile.

Perhaps this interface is OK?

> ]   char *libxl_tmem_list(libxl_ctx *ctx, uint32_t domid, int use_long);
> ]   int libxl_tmem_freeze(libxl_ctx *ctx, uint32_t domid);
> ]   int libxl_tmem_destroy(libxl_ctx *ctx, uint32_t domid);
> ]   int libxl_tmem_thaw(libxl_ctx *ctx, uint32_t domid);
> ]   int libxl_tmem_set(libxl_ctx *ctx, uint32_t domid, char* name,
> ]                      uint32_t set);
> ]   int libxl_tmem_shared_auth(libxl_ctx *ctx, uint32_t domid, char* uuid,
> ]                              int auth);
> ]   int libxl_tmem_freeable(libxl_ctx *ctx);
> 
> Not sure about the tmem calls.

Me neither.

> And from libxl_utils.h:
> 
> ]   pid_t libxl_fork(libxl_ctx *ctx);
> 
> This function is going to have to go away.

Great.

Maybe things aren't as bad as I feared.

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Thu, 2012-04-12 at 08:35 +0100, Ian Campbell wrote:
> 
> > ]   char *libxl_tmem_list(libxl_ctx *ctx, uint32_t domid, int
> use_long);
> > ]   int libxl_tmem_freeze(libxl_ctx *ctx, uint32_t domid);
> > ]   int libxl_tmem_destroy(libxl_ctx *ctx, uint32_t domid);
> > ]   int libxl_tmem_thaw(libxl_ctx *ctx, uint32_t domid);
> > ]   int libxl_tmem_set(libxl_ctx *ctx, uint32_t domid, char* name,
> > ]                      uint32_t set);
> > ]   int libxl_tmem_shared_auth(libxl_ctx *ctx, uint32_t domid, char*
> uuid,
> > ]                              int auth);
> > ]   int libxl_tmem_freeable(libxl_ctx *ctx);
> > 
> > Not sure about the tmem calls.
> 
> Me neither. 

Dan,

We want to declare the libxl 4.2 API as "stable" so we are trying to
determine whether any of these functions need to be made potentially
asynchronous or not, i.e. if they may be "slow" per the definition under
the comment "Machinery for asynchronous operations ("ao")" in
tools/libxl/libxl_internal.h, effectively if they may block for extended
periods.

If they were then we would possibly want to change the API to take an
"ao_how" as described under "Some libxl operations can take a long time"
in tools/libxl/libxl.h

If they are "fast" today but could potentially be slow in the future
then we may be able to make the trivial API change but keep the
synchronous implementation (depending on the specifics). It's quite late
in the day so if the functions are "slow" then this would be the
preferred option at this stage.

Otherwise the alternative is that we have to maintain these interfaces
going forward (for compat) and perhaps be forced introduce a new
parallel async interface in the future. Annoying but not the end of the
world.

Ian.

Re: Xen 4.2 Release Plan / TODO

[Dan Magenheimer]

> From: Ian Campbell [mailto:Ian.Campbell@citrix.com]
> Sent: Thursday, April 12, 2012 1:59 AM
> To: Ian Jackson; Dan Magenheimer
> Cc: xen-devel
> Subject: Re: [Xen-devel] Xen 4.2 Release Plan / TODO
> 
> On Thu, 2012-04-12 at 08:35 +0100, Ian Campbell wrote:
> >
> > > ]   char *libxl_tmem_list(libxl_ctx *ctx, uint32_t domid, int
> > use_long);
> > > ]   int libxl_tmem_freeze(libxl_ctx *ctx, uint32_t domid);
> > > ]   int libxl_tmem_destroy(libxl_ctx *ctx, uint32_t domid);
> > > ]   int libxl_tmem_thaw(libxl_ctx *ctx, uint32_t domid);
> > > ]   int libxl_tmem_set(libxl_ctx *ctx, uint32_t domid, char* name,
> > > ]                      uint32_t set);
> > > ]   int libxl_tmem_shared_auth(libxl_ctx *ctx, uint32_t domid, char*
> > uuid,
> > > ]                              int auth);
> > > ]   int libxl_tmem_freeable(libxl_ctx *ctx);
> > >
> > > Not sure about the tmem calls.
> >
> > Me neither.
> 
> Dan,
> 
> We want to declare the libxl 4.2 API as "stable" so we are trying to
> determine whether any of these functions need to be made potentially
> asynchronous or not, i.e. if they may be "slow" per the definition under
> the comment "Machinery for asynchronous operations ("ao")" in
> tools/libxl/libxl_internal.h, effectively if they may block for extended
> periods.
> 
> If they were then we would possibly want to change the API to take an
> "ao_how" as described under "Some libxl operations can take a long time"
> in tools/libxl/libxl.h
> 
> If they are "fast" today but could potentially be slow in the future
> then we may be able to make the trivial API change but keep the
> synchronous implementation (depending on the specifics). It's quite late
> in the day so if the functions are "slow" then this would be the
> preferred option at this stage.
> 
> Otherwise the alternative is that we have to maintain these interfaces
> going forward (for compat) and perhaps be forced introduce a new
> parallel async interface in the future. Annoying but not the end of the
> world.

Hi Ian(s) --

After reading libxl.h, I'm not absolutely positive I understand
all the conditions that would cause you to label a function as
"slow" but I believe all the libxl_tmem_* functions are "fast".
All of them are strictly "call the hypervisor, wait for it to
return" and none of the hypercalls (actually which are variations of
the one tmem hypercall) require a callback to dom0 or to the
calling guest... they all complete entirely inside the hypervisor.

Libxl_tmem_destroy may take a long time as it has to walk
through and free some potentially very large data structures,
but it is only used at domain destruction.

Libxl_tmem_list does allocate some memory in userland that the
hypercall fills synchronously (with ascii-formatted statistics/counters
maintained entirely by the tmem code in the hypervisor).

If any of the above raises any alarms/concerns, let me know,
else no need to asynchronizify any of the tmem functions in libxl.

(Zhigang cc'ed since he's more familiar with the libxl layer than I.)

Dan

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Thu, 2012-04-12 at 17:37 +0100, Dan Magenheimer wrote:
> > From: Ian Campbell [mailto:Ian.Campbell@citrix.com]
> > Sent: Thursday, April 12, 2012 1:59 AM
> > To: Ian Jackson; Dan Magenheimer
> > Cc: xen-devel
> > Subject: Re: [Xen-devel] Xen 4.2 Release Plan / TODO
> > 
> > On Thu, 2012-04-12 at 08:35 +0100, Ian Campbell wrote:
> > >
> > > > ]   char *libxl_tmem_list(libxl_ctx *ctx, uint32_t domid, int
> > > use_long);
> > > > ]   int libxl_tmem_freeze(libxl_ctx *ctx, uint32_t domid);
> > > > ]   int libxl_tmem_destroy(libxl_ctx *ctx, uint32_t domid);
> > > > ]   int libxl_tmem_thaw(libxl_ctx *ctx, uint32_t domid);
> > > > ]   int libxl_tmem_set(libxl_ctx *ctx, uint32_t domid, char* name,
> > > > ]                      uint32_t set);
> > > > ]   int libxl_tmem_shared_auth(libxl_ctx *ctx, uint32_t domid, char*
> > > uuid,
> > > > ]                              int auth);
> > > > ]   int libxl_tmem_freeable(libxl_ctx *ctx);
> > > >
> > > > Not sure about the tmem calls.
> > >
> > > Me neither.
> > 
> > Dan,
> > 
> > We want to declare the libxl 4.2 API as "stable" so we are trying to
> > determine whether any of these functions need to be made potentially
> > asynchronous or not, i.e. if they may be "slow" per the definition under
> > the comment "Machinery for asynchronous operations ("ao")" in
> > tools/libxl/libxl_internal.h, effectively if they may block for extended
> > periods.
> > 
> > If they were then we would possibly want to change the API to take an
> > "ao_how" as described under "Some libxl operations can take a long time"
> > in tools/libxl/libxl.h
> > 
> > If they are "fast" today but could potentially be slow in the future
> > then we may be able to make the trivial API change but keep the
> > synchronous implementation (depending on the specifics). It's quite late
> > in the day so if the functions are "slow" then this would be the
> > preferred option at this stage.
> > 
> > Otherwise the alternative is that we have to maintain these interfaces
> > going forward (for compat) and perhaps be forced introduce a new
> > parallel async interface in the future. Annoying but not the end of the
> > world.
> 
> Hi Ian(s) --
> 
> After reading libxl.h, I'm not absolutely positive I understand
> all the conditions that would cause you to label a function as
> "slow" but I believe all the libxl_tmem_* functions are "fast".
> All of them are strictly "call the hypervisor, wait for it to
> return" and none of the hypercalls (actually which are variations of
> the one tmem hypercall) require a callback to dom0 or to the
> calling guest... they all complete entirely inside the hypervisor.

OK, this sounds good, thanks.

> Libxl_tmem_destroy may take a long time as it has to walk
> through and free some potentially very large data structures,
> but it is only used at domain destruction.

Should libxl_tmem_destroy be a public function or should it solely be
called from libxl_domain_destroy? I notice that there is an xl
tmem-destroy so I presume the former?

We might consider adding a dummy ao_how to this one I suppose.

> Libxl_tmem_list does allocate some memory in userland that the
> hypercall fills synchronously (with ascii-formatted statistics/counters
> maintained entirely by the tmem code in the hypervisor).
> 
> If any of the above raises any alarms/concerns, let me know,
> else no need to asynchronizify any of the tmem functions in libxl.

It all sounds fine, I more curious about tmem_destroy than worried.

Ian.

> 
> (Zhigang cc'ed since he's more familiar with the libxl layer than I.)
> 
> Dan

Re: Xen 4.2 Release Plan / TODO

[Dan Magenheimer]

> From: Ian Campbell [mailto:Ian.Campbell@citrix.com]
> > >
> 
> > Libxl_tmem_destroy may take a long time as it has to walk
> > through and free some potentially very large data structures,
> > but it is only used at domain destruction.
> 
> Should libxl_tmem_destroy be a public function or should it solely be
> called from libxl_domain_destroy? I notice that there is an xl
> tmem-destroy so I presume the former?
> 
> We might consider adding a dummy ao_how to this one I suppose.

Bearing in mind that I haven't poked around in this
code for over 2 years, I *think* the only reason tmem_destroy
needs to exist at all is if tmem is broken.  The normal domain
termination process destroys any persistent tmem pools and
any ephemeral tmem pools will eventually "age out" as the
last of the pools' pages fall off the end of the LRU.

I think the tmem_destroy functionality pre-dates the
existence of tmem "freeable" memory* and was a way for
a toolset to force the hypervisor to free up the hypervisor
memory used by some or all ephemeral tmem pools.  Once the
tmem allocation/free process was directly linked into
alloc_heap_pages() in the hypervisor (see call to
tmem_relinquish_pages()), this forcing function was
no longer needed.

So, bottom line, I *think* it can be ripped out, or at least
for now removed from the definition of the stable xl API/UI.
The libxl.c routine libxl_tmem_destroy() could also be
removed if you like, but I guess I'd prefer to leave the
lower level droppings in xc.c and in the hypervisor in case
I am misremembering.

* http://xenbits.xensource.com/hg/xen-unstable.hg/rev/03063e309356

Re: Xen 4.2 Release Plan / TODO

[Ian Jackson]

Dan Magenheimer writes ("RE: [Xen-devel] Xen 4.2 Release Plan / TODO"):
> After reading libxl.h, I'm not absolutely positive I understand
> all the conditions that would cause you to label a function as
> "slow" but I believe all the libxl_tmem_* functions are "fast".

Thanks for your reply.

There are a few operations that make a function necessarily have to be
slow in the libxl api sense.  These are: xenstore watches; spawning
subprocesses; anything with a timeout.

More broadly any function which is sufficiently slow that a caller
might reasonably want to initiate it, and then carry on doing
something else while the function completes.  So this includes any
operation which a toolstack might want to parallelise.

> All of them are strictly "call the hypervisor, wait for it to
> return" and none of the hypercalls (actually which are variations of
> the one tmem hypercall) require a callback to dom0 or to the
> calling guest... they all complete entirely inside the hypervisor.

Right, that sounds good.  I guess you also mean that this will always
be the case.

> Libxl_tmem_destroy may take a long time as it has to walk
> through and free some potentially very large data structures,
> but it is only used at domain destruction.

How long a time are we talking about ?  Would it be a scalability or
performance problem if an entire host's management toolstack had to
block, and no other management operations could be performed on any
domain for any reason, while the tmem destroy takes place ?

> Libxl_tmem_list does allocate some memory in userland that the
> hypercall fills synchronously (with ascii-formatted statistics/counters
> maintained entirely by the tmem code in the hypervisor).

Memory allocation in userland is fine.  I guess we're not talking
about megabytes here.

Ian.

Re: Xen 4.2 Release Plan / TODO

[Dan Magenheimer]

> From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com]
> Subject: RE: [Xen-devel] Xen 4.2 Release Plan / TODO
> 
> Dan Magenheimer writes ("RE: [Xen-devel] Xen 4.2 Release Plan / TODO"):
> > After reading libxl.h, I'm not absolutely positive I understand
> > all the conditions that would cause you to label a function as
> > "slow" but I believe all the libxl_tmem_* functions are "fast".
> 
> There are a few operations that make a function necessarily have to be
> slow in the libxl api sense.  These are: xenstore watches; spawning
> subprocesses; anything with a timeout.
> 
> More broadly any function which is sufficiently slow that a caller
> might reasonably want to initiate it, and then carry on doing
> something else while the function completes.  So this includes any
> operation which a toolstack might want to parallelise.

Got it.  Thanks.  This is a bit clearer than the comment in libxl.h.

> > All of them are strictly "call the hypervisor, wait for it to
> > return" and none of the hypercalls (actually which are variations of
> > the one tmem hypercall) require a callback to dom0 or to the
> > calling guest... they all complete entirely inside the hypervisor.
> 
> Right, that sounds good.  I guess you also mean that this will always
> be the case.

Yes AFAICT.

> > Libxl_tmem_destroy may take a long time as it has to walk
> > through and free some potentially very large data structures,
> > but it is only used at domain destruction.
> 
> How long a time are we talking about ?  Would it be a scalability or
> performance problem if an entire host's management toolstack had to
> block, and no other management operations could be performed on any
> domain for any reason, while the tmem destroy takes place ?

See previous reply to IanC... this is moot since (I think)
tmem_destroy will go away.

> > Libxl_tmem_list does allocate some memory in userland that the
> > hypercall fills synchronously (with ascii-formatted statistics/counters
> > maintained entirely by the tmem code in the hypervisor).
> 
> Memory allocation in userland is fine.  I guess we're not talking
> about megabytes here.

A reasonable bound would be on the order of 1K per tmem-enabled guest.
The current code in pyxc_tmem_control enforces a 32K buffer limit.

Dan

Re: Xen 4.2 Release Plan / TODO

[Ian Jackson]

Dan Magenheimer writes ("RE: [Xen-devel] Xen 4.2 Release Plan / TODO"):
> > From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com]
...
> > There are a few operations that make a function necessarily have to be
> > slow in the libxl api sense.  These are: xenstore watches; spawning
> > subprocesses; anything with a timeout.
> > 
> > More broadly any function which is sufficiently slow that a caller
> > might reasonably want to initiate it, and then carry on doing
> > something else while the function completes.  So this includes any
> > operation which a toolstack might want to parallelise.
> 
> Got it.  Thanks.  This is a bit clearer than the comment in libxl.h.

The internal-facing documentation, which is for people who might be
implementing libxl facilities and API designers, is in
libxl_internal.h.  But the comment there is less comprehensive.  I
will prepare a patch to update it.

Ian.

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Thu, 2012-04-12 at 08:35 +0100, Ian Campbell wrote:
> > ]   /* common paths */
> > ]   const char *libxl_sbindir_path(void);
> > ]   const char *libxl_bindir_path(void);
> > ]   const char *libxl_libexec_path(void);
> > ]   const char *libxl_libdir_path(void);
> > ]   const char *libxl_sharedir_path(void);
> > ]   const char *libxl_private_bindir_path(void);
> > ]   const char *libxl_xenfirmwaredir_path(void);
> > ]   const char *libxl_xen_config_dir_path(void);
> > ]   const char *libxl_xen_script_dir_path(void);
> > ]   const char *libxl_lock_dir_path(void);
> > ]   const char *libxl_run_dir_path(void);
> > ]   const char *libxl_xenpaging_dir_path(void);
> > 
> > Surely these should be private ?
> 
> As far as I can grep, yes. 

All but two it turns out. Not sure about those. The following patch
moves the rest.

libxl_lock_dir_path seems like it ought to be part of xl not libxl, or
at a stretch libxlu.

config_dir_path is only actually used by xl but an argument could be
made that it is more generally useful?

8<-----------------------------------------------

# HG changeset patch
# User Ian Campbell <ian.campbell@citrix.com>
# Date 1334218378 -3600
# Node ID 14cac8170e122115ef090cb390775b8c356ae643
# Parent  86b4ff3e201dc81c76ac91f8a9f134669294b3ba
libxl: make most libxl_FOO_path() functions internal.

Only libxl_xen_config_dir_path and libxl_lock_dir_path are used outside the
library. Also bindir, sbindir, sharedir and xenpagingdir appeared to be
completely unused so nuke them.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl.c
--- a/tools/libxl/libxl.c	Thu Apr 12 09:02:05 2012 +0100
+++ b/tools/libxl/libxl.c	Thu Apr 12 09:12:58 2012 +0100
@@ -1126,7 +1126,7 @@ out:
 int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num, libxl_console_type type)
 {
     GC_INIT(ctx);
-    char *p = libxl__sprintf(gc, "%s/xenconsole", libxl_private_bindir_path());
+    char *p = libxl__sprintf(gc, "%s/xenconsole", libxl__private_bindir_path());
     char *domid_s = libxl__sprintf(gc, "%d", domid);
     char *cons_num_s = libxl__sprintf(gc, "%d", cons_num);
     char *cons_type_s;
@@ -1767,7 +1767,7 @@ int libxl__device_nic_setdefault(libxl__
         if (!nic->bridge) return ERROR_NOMEM;
     }
     if ( !nic->script && asprintf(&nic->script, "%s/vif-bridge",
-                                  libxl_xen_script_dir_path()) < 0 )
+                                  libxl__xen_script_dir_path()) < 0 )
         return ERROR_FAIL;
     if (!nic->nictype)
         nic->nictype = LIBXL_NIC_TYPE_IOEMU;
@@ -1837,7 +1837,7 @@ int libxl_device_nic_add(libxl_ctx *ctx,
         flexarray_append(back, "script");
         flexarray_append(back, nic->script[0]=='/' ? nic->script
                          : libxl__sprintf(gc, "%s/%s",
-                                          libxl_xen_script_dir_path(),
+                                          libxl__xen_script_dir_path(),
                                           nic->script));
     }
 
diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl.h
--- a/tools/libxl/libxl.h	Thu Apr 12 09:02:05 2012 +0100
+++ b/tools/libxl/libxl.h	Thu Apr 12 09:12:58 2012 +0100
@@ -787,18 +787,8 @@ int libxl_flask_setenforce(libxl_ctx *ct
 int libxl_flask_loadpolicy(libxl_ctx *ctx, void *policy, uint32_t size);
 
 /* common paths */
-const char *libxl_sbindir_path(void);
-const char *libxl_bindir_path(void);
-const char *libxl_libexec_path(void);
-const char *libxl_libdir_path(void);
-const char *libxl_sharedir_path(void);
-const char *libxl_private_bindir_path(void);
-const char *libxl_xenfirmwaredir_path(void);
 const char *libxl_xen_config_dir_path(void);
-const char *libxl_xen_script_dir_path(void);
 const char *libxl_lock_dir_path(void);
-const char *libxl_run_dir_path(void);
-const char *libxl_xenpaging_dir_path(void);
 
 /* misc */
 
diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl_dm.c
--- a/tools/libxl/libxl_dm.c	Thu Apr 12 09:02:05 2012 +0100
+++ b/tools/libxl/libxl_dm.c	Thu Apr 12 09:12:58 2012 +0100
@@ -24,7 +24,7 @@ static const char *libxl_tapif_script(li
 #ifdef __linux__
     return libxl__strdup(gc, "no");
 #else
-    return libxl__sprintf(gc, "%s/qemu-ifup", libxl_xen_script_dir_path());
+    return libxl__sprintf(gc, "%s/qemu-ifup", libxl__xen_script_dir_path());
 #endif
 }
 
@@ -47,10 +47,10 @@ const char *libxl__domain_device_model(l
     } else {
         switch (info->device_model_version) {
         case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL:
-            dm = libxl__abs_path(gc, "qemu-dm", libxl_libexec_path());
+            dm = libxl__abs_path(gc, "qemu-dm", libxl__libexec_path());
             break;
         case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
-            dm = libxl__abs_path(gc, "qemu-system-i386", libxl_libexec_path());
+            dm = libxl__abs_path(gc, "qemu-system-i386", libxl__libexec_path());
             break;
         default:
             LIBXL__LOG(ctx, LIBXL__LOG_ERROR,
@@ -337,7 +337,7 @@ static char ** libxl__build_device_model
     flexarray_append(dm_args,
                      libxl__sprintf(gc, "socket,id=libxl-cmd,"
                                     "path=%s/qmp-libxl-%d,server,nowait",
-                                    libxl_run_dir_path(), guest_domid));
+                                    libxl__run_dir_path(), guest_domid));
 
     flexarray_append(dm_args, "-mon");
     flexarray_append(dm_args, "chardev=libxl-cmd,mode=control");
@@ -708,7 +708,7 @@ static int libxl__create_stubdom(libxl__
     dm_config.b_info.target_memkb = dm_config.b_info.max_memkb;
 
     dm_config.b_info.u.pv.kernel.path = libxl__abs_path(gc, "ioemu-stubdom.gz",
-                                              libxl_xenfirmwaredir_path());
+                                              libxl__xenfirmwaredir_path());
     dm_config.b_info.u.pv.cmdline = libxl__sprintf(gc, " -d %d", guest_domid);
     dm_config.b_info.u.pv.ramdisk.path = "";
     dm_config.b_info.u.pv.features = "";
diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl_dom.c
--- a/tools/libxl/libxl_dom.c	Thu Apr 12 09:02:05 2012 +0100
+++ b/tools/libxl/libxl_dom.c	Thu Apr 12 09:12:58 2012 +0100
@@ -349,7 +349,7 @@ static const char *libxl__domain_firmwar
             break;
         }
     }
-    return libxl__abs_path(gc, firmware, libxl_xenfirmwaredir_path());
+    return libxl__abs_path(gc, firmware, libxl__xenfirmwaredir_path());
 }
 
 int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl_internal.h
--- a/tools/libxl/libxl_internal.h	Thu Apr 12 09:02:05 2012 +0100
+++ b/tools/libxl/libxl_internal.h	Thu Apr 12 09:12:58 2012 +0100
@@ -1374,6 +1374,14 @@ int libxl__carefd_close(libxl__carefd*);
 /* You may pass NULL in which case the answer is -1. */
 int libxl__carefd_fd(const libxl__carefd*);
 
+/* common paths */
+_hidden const char *libxl__libexec_path(void);
+_hidden const char *libxl__private_bindir_path(void);
+_hidden const char *libxl__xenfirmwaredir_path(void);
+_hidden const char *libxl__xen_config_dir_path(void);
+_hidden const char *libxl__xen_script_dir_path(void);
+_hidden const char *libxl__lock_dir_path(void);
+_hidden const char *libxl__run_dir_path(void);
 
 /*
  * Convenience macros.
diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl_paths.c
--- a/tools/libxl/libxl_paths.c	Thu Apr 12 09:02:05 2012 +0100
+++ b/tools/libxl/libxl_paths.c	Thu Apr 12 09:12:58 2012 +0100
@@ -15,37 +15,17 @@
 #include "libxl_osdeps.h" /* must come before any other headers */
 #include "libxl_internal.h"
 
-const char *libxl_sbindir_path(void)
-{
-    return SBINDIR;
-}
-
-const char *libxl_bindir_path(void)
-{
-    return BINDIR;
-}
-
-const char *libxl_libexec_path(void)
+const char *libxl__libexec_path(void)
 {
     return LIBEXEC;
 }
 
-const char *libxl_libdir_path(void)
-{
-    return LIBDIR;
-}
-
-const char *libxl_sharedir_path(void)
-{
-    return SHAREDIR;
-}
-
-const char *libxl_private_bindir_path(void)
+const char *libxl__private_bindir_path(void)
 {
     return PRIVATE_BINDIR;
 }
 
-const char *libxl_xenfirmwaredir_path(void)
+const char *libxl__xenfirmwaredir_path(void)
 {
     return XENFIRMWAREDIR;
 }
@@ -55,7 +35,7 @@ const char *libxl_xen_config_dir_path(vo
     return XEN_CONFIG_DIR;
 }
 
-const char *libxl_xen_script_dir_path(void)
+const char *libxl__xen_script_dir_path(void)
 {
     return XEN_SCRIPT_DIR;
 }
@@ -65,16 +45,11 @@ const char *libxl_lock_dir_path(void)
     return XEN_LOCK_DIR;
 }
 
-const char *libxl_run_dir_path(void)
+const char *libxl__run_dir_path(void)
 {
     return XEN_RUN_DIR;
 }
 
-const char *libxl_xenpaging_dir_path(void)
-{
-    return XEN_PAGING_DIR;
-}
-
 /*
  * Local variables:
  * mode: C
diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl_qmp.c
--- a/tools/libxl/libxl_qmp.c	Thu Apr 12 09:02:05 2012 +0100
+++ b/tools/libxl/libxl_qmp.c	Thu Apr 12 09:12:58 2012 +0100
@@ -633,7 +633,7 @@ libxl__qmp_handler *libxl__qmp_initializ
     qmp = qmp_init_handler(gc, domid);
 
     qmp_socket = libxl__sprintf(gc, "%s/qmp-libxl-%d",
-                                libxl_run_dir_path(), domid);
+                                libxl__run_dir_path(), domid);
     if ((ret = qmp_open(qmp, qmp_socket, QMP_SOCKET_CONNECT_TIMEOUT)) < 0) {
         LIBXL__LOG_ERRNO(qmp->ctx, LIBXL__LOG_ERROR, "Connection error");
         qmp_free_handler(qmp);
@@ -671,7 +671,7 @@ void libxl__qmp_cleanup(libxl__gc *gc, u
     char *qmp_socket;
 
     qmp_socket = libxl__sprintf(gc, "%s/qmp-libxl-%d",
-                                libxl_run_dir_path(), domid);
+                                libxl__run_dir_path(), domid);
     if (unlink(qmp_socket) == -1) {
         if (errno != ENOENT) {
             LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR,

Re: Xen 4.2 Release Plan / TODO

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] Xen 4.2 Release Plan / TODO"):
> libxl: make most libxl_FOO_path() functions internal.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>

Xen 4.2 Release Plan / TODO

[Ian Campbell]

Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html

The time line is as follows:

19 March        -- TODO list locked down
2 April         -- Feature Freeze
					               << WE ARE HERE
Mid/Late April  -- First release candidate
Weekly          -- RCN+1 until it is ready

The updated TODO list follows. Per the release plan a strong case will
now have to be made as to why new items should be added to the list,
especially as a blocker, rather than deferred to 4.3.

We have now entered Feature Freeze. Patches which have been posted
before or which address something on the list below are still acceptable
(for now, we will gradually be getting stricter about this), everything
else will be deferred until 4.3 by default.

Lots more DONE this week, still a few big ticket items or items with no
progress (that I've noticed) please let me know if the status of
anything has changed.

>From next week I think I will start also tracking bugs on these lists.
Please let me know if you have something which you feel should be listed
here, as well as your justification for it being a blocker rather than
"nice to fix".

hypervisor, blockers:
      * NONE?
 
tools, blockers:
      * libxl stable API -- we would like 4.2 to define a stable API
        which downstream's can start to rely on not changing. Aspects of
        this are:
              * Safe fork vs. fd handling hooks. Involves API changes
                (Ian J, patches posted)
              * locking/serialization, e.g., for domain creation. As of
                now,  nothing is provided for this purpose, and
                downstream toolstacks have to put their own mechanisms
                in place. E.g., xl uses a fcntl() lock around the whole
                process of domain creation. It should OTOH be libxl job
                to offer a proper set of hooks --placed at the proper
                spots during the domain creation process-- for the
                downstreams to fill with the proper callbacks. (Dario
                Faggioli)
                      * Thinking to defer this until 4.3?
              * agree & document compatibility guarantees + associated
                technical measures (Ian C, DONE)
      * xl compatibility with xm:
              * feature parity wrt driver domain support (George Dunlap,
                DONE)
              * xl support for "rtc_timeoffset" and "localtime" (Lin
                Ming, DONE)
      * More formally deprecate xm/xend. Manpage patches already in
        tree. Needs release noting and communication around -rc1 to
        remind people to test xl.
      * Domain 0 block attach & general hotplug when using qdisk backend
        (need to start qemu as necessary etc) (Stefano S, patches
        posted)
      * file:// backend performance. qemu-xen-tradition's qdisk is quite
        slow & blktap2 not available in upstream kernels. Need to
        consider our options:
              * qemu-xen's qdisk is thought to be well performing but
                qemu-xen is not yet the default. Complexity arising from
                splitting qemu-for-qdisk out from qemu-for-dm and
                running N qemu's.
              * potentially fully userspace blktap could be ready for
                4.2 (unlikely to be available in 4.2)
              * use /dev/loop+blkback. This requires loop driver AIO and
                O_DIRECT patches which are not (AFAIK) yet upstream.
              * Leverage XCP's blktap2 DKMS work. (Useful fallback for
                some usecases
        Stefano has done a lot of work here and has proposed some
        performance improvements for qdisk in both qemu-xen and
        qemu-xen-traditional which make them a plausible alternative for
        Xen 4.2. This is likely the route we will now take. Need to
        mention in release notes?
      * Improved Hotplug script support (Roger Pau Monné, patches
        posted)
      * Block script support -- follows on from hotplug script (Roger
        Pau Monné)

hypervisor, nice to have:
      * solid implementation of sharing/paging/mem-events (using work
        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
              * "The last patch to use a waitqueue in
                __get_gfn_type_access() from Tim works.  However, there
                are a few users who call __get_gfn_type_access with the
                domain_lock held. This part needs to be addressed in
                some way."
      * Sharing support for AMD (Tim, Andres).
      * PoD performance improvements (George Dunlap)

tools, nice to have:
      * Configure/control paging via xl/libxl (Olaf Herring, lots of
        discussion around interface, general consensus reached on what
        it should look like. Olaf has let me know that he is probably
        not going to have time to do this for 4.2, will likely slip to
        4.3 unless someone else want to pick up that work)
      * Upstream qemu feature patches:
              * Upstream qemu PCI passthrough support (Anthony Perard,
                patches sent)
              * Upstream qemu save restore (Anthony Perard, Stefano
                Stabellini, qemu patches applied, waiting for libxl etc
                side which has been recently reposted)
      * Nested-virtualisation. Currently "experimental". Likely to
        release that way.
              * Nested SVM. Tested in a variety of configurations but
                still some issues with the most important use case (w7
                XP mode) [0]  (Christoph Egger)
              * Nested VMX. Needs nested EPT to be genuinely useful.
                Need more data on testedness etc (Intel)
      * Initial xl support for Remus (memory checkpoint, blackholing)
        (Shriram, patches reposted recently, was blocked behind qemu
        save restore patches which are now in)
      * xl compatibility with xm:
              * xl support for autospawning vncviewer (vncviewer=1 or
                otherwise) (Goncalo Gomes, patches posted)
              * support for vif "rate" parameter (Mathieu Gagné, patches
                posted)
              * expose PCI back "permissive" parameter (George Dunlap)

[0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html

Re: Xen 4.2 Release Plan / TODO

[George Dunlap]

On Tue, Apr 10, 2012 at 11:24 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> Plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
>
> The time line is as follows:
>
> 19 March        -- TODO list locked down
> 2 April         -- Feature Freeze
>                                                       << WE ARE HERE
> Mid/Late April  -- First release candidate
> Weekly          -- RCN+1 until it is ready
>
> The updated TODO list follows. Per the release plan a strong case will
> now have to be made as to why new items should be added to the list,
> especially as a blocker, rather than deferred to 4.3.
>
> We have now entered Feature Freeze. Patches which have been posted
> before or which address something on the list below are still acceptable
> (for now, we will gradually be getting stricter about this), everything
> else will be deferred until 4.3 by default.
>
> Lots more DONE this week, still a few big ticket items or items with no
> progress (that I've noticed) please let me know if the status of
> anything has changed.
>
> From next week I think I will start also tracking bugs on these lists.
> Please let me know if you have something which you feel should be listed
> here, as well as your justification for it being a blocker rather than
> "nice to fix".

Here are bugs I've found:

* Zombie driver domains.  There's a bug where PV guests with devices
passed through with xl hang around as "zombies", with one outstanding
page reference.  I think this should really be a blocker.  I'm working
on this at the moment.

* Manually ballooning dom0.  xl mem-set 0 [foo] fails with "libxl:
error: libxl.c:2569:libxl_set_memory_target: cannot get memory info
from /local/domain/0/memory/static-max: No such file or directory".
I'd make this a blocker just because it should be easy to fix and it's
pretty embarassing.  This might be suitable for an enthusiastic
on-looker.

Also, since there have been other situations / reports of zombie
domains, it would be good if we could get a zombie domain detector
into the testing system to see how widespread they are.

 -George

>
> hypervisor, blockers:
>      * NONE?
>
> tools, blockers:
>      * libxl stable API -- we would like 4.2 to define a stable API
>        which downstream's can start to rely on not changing. Aspects of
>        this are:
>              * Safe fork vs. fd handling hooks. Involves API changes
>                (Ian J, patches posted)
>              * locking/serialization, e.g., for domain creation. As of
>                now,  nothing is provided for this purpose, and
>                downstream toolstacks have to put their own mechanisms
>                in place. E.g., xl uses a fcntl() lock around the whole
>                process of domain creation. It should OTOH be libxl job
>                to offer a proper set of hooks --placed at the proper
>                spots during the domain creation process-- for the
>                downstreams to fill with the proper callbacks. (Dario
>                Faggioli)
>                      * Thinking to defer this until 4.3?
>              * agree & document compatibility guarantees + associated
>                technical measures (Ian C, DONE)
>      * xl compatibility with xm:
>              * feature parity wrt driver domain support (George Dunlap,
>                DONE)
>              * xl support for "rtc_timeoffset" and "localtime" (Lin
>                Ming, DONE)
>      * More formally deprecate xm/xend. Manpage patches already in
>        tree. Needs release noting and communication around -rc1 to
>        remind people to test xl.
>      * Domain 0 block attach & general hotplug when using qdisk backend
>        (need to start qemu as necessary etc) (Stefano S, patches
>        posted)
>      * file:// backend performance. qemu-xen-tradition's qdisk is quite
>        slow & blktap2 not available in upstream kernels. Need to
>        consider our options:
>              * qemu-xen's qdisk is thought to be well performing but
>                qemu-xen is not yet the default. Complexity arising from
>                splitting qemu-for-qdisk out from qemu-for-dm and
>                running N qemu's.
>              * potentially fully userspace blktap could be ready for
>                4.2 (unlikely to be available in 4.2)
>              * use /dev/loop+blkback. This requires loop driver AIO and
>                O_DIRECT patches which are not (AFAIK) yet upstream.
>              * Leverage XCP's blktap2 DKMS work. (Useful fallback for
>                some usecases
>        Stefano has done a lot of work here and has proposed some
>        performance improvements for qdisk in both qemu-xen and
>        qemu-xen-traditional which make them a plausible alternative for
>        Xen 4.2. This is likely the route we will now take. Need to
>        mention in release notes?
>      * Improved Hotplug script support (Roger Pau Monné, patches
>        posted)
>      * Block script support -- follows on from hotplug script (Roger
>        Pau Monné)
>
> hypervisor, nice to have:
>      * solid implementation of sharing/paging/mem-events (using work
>        queues) (Tim Deegan, Olaf Herring et al -- patches posted)
>              * "The last patch to use a waitqueue in
>                __get_gfn_type_access() from Tim works.  However, there
>                are a few users who call __get_gfn_type_access with the
>                domain_lock held. This part needs to be addressed in
>                some way."
>      * Sharing support for AMD (Tim, Andres).
>      * PoD performance improvements (George Dunlap)
>
> tools, nice to have:
>      * Configure/control paging via xl/libxl (Olaf Herring, lots of
>        discussion around interface, general consensus reached on what
>        it should look like. Olaf has let me know that he is probably
>        not going to have time to do this for 4.2, will likely slip to
>        4.3 unless someone else want to pick up that work)
>      * Upstream qemu feature patches:
>              * Upstream qemu PCI passthrough support (Anthony Perard,
>                patches sent)
>              * Upstream qemu save restore (Anthony Perard, Stefano
>                Stabellini, qemu patches applied, waiting for libxl etc
>                side which has been recently reposted)
>      * Nested-virtualisation. Currently "experimental". Likely to
>        release that way.
>              * Nested SVM. Tested in a variety of configurations but
>                still some issues with the most important use case (w7
>                XP mode) [0]  (Christoph Egger)
>              * Nested VMX. Needs nested EPT to be genuinely useful.
>                Need more data on testedness etc (Intel)
>      * Initial xl support for Remus (memory checkpoint, blackholing)
>        (Shriram, patches reposted recently, was blocked behind qemu
>        save restore patches which are now in)
>      * xl compatibility with xm:
>              * xl support for autospawning vncviewer (vncviewer=1 or
>                otherwise) (Goncalo Gomes, patches posted)
>              * support for vif "rate" parameter (Mathieu Gagné, patches
>                posted)
>              * expose PCI back "permissive" parameter (George Dunlap)
>
> [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2166 bytes --]

On Tue, 2012-04-10 at 10:24 +0000, Ian Campbell wrote: 
> Lots more DONE this week, still a few big ticket items or items with no
> progress (that I've noticed) please let me know if the status of
> anything has changed.
>
Here I am. :-)

> tools, blockers:
>       * libxl stable API -- we would like 4.2 to define a stable API
>         which downstream's can start to rely on not changing. Aspects of
>         this are:
>               * Safe fork vs. fd handling hooks. Involves API changes
>                 (Ian J, patches posted)
>               * locking/serialization, e.g., for domain creation. As of
>                 now,  nothing is provided for this purpose, and
>                 downstream toolstacks have to put their own mechanisms
>                 in place. E.g., xl uses a fcntl() lock around the whole
>                 process of domain creation. It should OTOH be libxl job
>                 to offer a proper set of hooks --placed at the proper
>                 spots during the domain creation process-- for the
>                 downstreams to fill with the proper callbacks. (Dario
>                 Faggioli)
>                       * Thinking to defer this until 4.3?
>
Here's the point. This was raised for the following main reasons:
1. xl holds a lock during domain creation for way too long
2. checking for free memory in xl when it is Xen that will actually 
    allocate it after a while is prone to races (the classical TOCTTOU 
    condition)

We thought both these things to be addressable by messing around with
locks, but a more careful analysis revealed we can't solve 2. in a sane
enough way by just doing that (i.e., messing with lock, moving it
around, etc.).

That being the case, yes, I think we can move this forward toward 4.3
without much problems, and I propose doing so. Thoughts?

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)




[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Thu, 2012-04-12 at 11:24 +0100, Dario Faggioli wrote:
> On Tue, 2012-04-10 at 10:24 +0000, Ian Campbell wrote: 
> > Lots more DONE this week, still a few big ticket items or items with no
> > progress (that I've noticed) please let me know if the status of
> > anything has changed.
> >
> Here I am. :-)
> 
> > tools, blockers:
> >       * libxl stable API -- we would like 4.2 to define a stable API
> >         which downstream's can start to rely on not changing. Aspects of
> >         this are:
> >               * Safe fork vs. fd handling hooks. Involves API changes
> >                 (Ian J, patches posted)
> >               * locking/serialization, e.g., for domain creation. As of
> >                 now,  nothing is provided for this purpose, and
> >                 downstream toolstacks have to put their own mechanisms
> >                 in place. E.g., xl uses a fcntl() lock around the whole
> >                 process of domain creation. It should OTOH be libxl job
> >                 to offer a proper set of hooks --placed at the proper
> >                 spots during the domain creation process-- for the
> >                 downstreams to fill with the proper callbacks. (Dario
> >                 Faggioli)
> >                       * Thinking to defer this until 4.3?
> >
> Here's the point. This was raised for the following main reasons:
> 1. xl holds a lock during domain creation for way too long
> 2. checking for free memory in xl when it is Xen that will actually 
>     allocate it after a while is prone to races (the classical TOCTTOU 
>     condition)
> 
> We thought both these things to be addressable by messing around with
> locks, but a more careful analysis revealed we can't solve 2. in a sane
> enough way by just doing that (i.e., messing with lock, moving it
> around, etc.).
> 
> That being the case, yes, I think we can move this forward toward 4.3
> without much problems, and I propose doing so. Thoughts?

I think this is OK, it will effectively be a new API in 4.3 so it should
be reasonably easy to do in a compatible manner.

Ian.

Xen 4.2 Release Plan / TODO

[Ian Campbell]

A bit late this week, due to my being on vacation. Normal Monday
service should be resumed next week.

Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html

The time line is as follows:

19 March        -- TODO list locked down
2 April         -- Feature Freeze
                                                << WE ARE HERE
When It's Ready -- First release candidate
Weekly          -- RCN+1 until release

The updated TODO list follows.

If you are aware of any bugs which must/should be fixed for 4.2 then
please reply to this thread (otherwise I may not remember to pick them
up next week)

As well as [BUG]s I've also started tracking things to [CHECK] before
the release. These are basically for things which we ought to confirm
during the RC cycles e.g. things which are not covered by automated
testing.

Per the release plan a strong case will now have to be made as to why
new items should be added to the list, especially as a blocker, rather
than deferred to 4.3.

hypervisor, blockers:

    * None
 
tools, blockers:

    * libxl stable API -- we would like 4.2 to define a stable API
      which downstream's can start to rely on not changing. Aspects of
      this are:

        * Interfaces which may need to be async:

            * libxl_domain_suspend. Move xc_domain_save/restore into a
              separate process (Ian Jackson, patches reviews and reposted).

            * libxl_device_{disk,nic,vkb,add,pci}_add (and
              remove). (Roger Pau Monné, patches posted for disk & nic, vkb
              trivial, not looked at pci yet)

        * LIBXL_NIC_TYPE enum names are confusing. (Roger, included in
          calling hotplug scripts series)

        * use libxl_cpumap for b_info.avail_cpus instead of an int,
          this allows setting more than 31 CPUS (Yang Z Zhang, patches
          posted, awaiting a repost)

        * use an enum for VGA interface type (e.g. Cirrus,
          StdVGA). Allows for QXL support (in 4.3). (Zhou Peng,
          awaiting repost)

    * xl compatibility with xm:

        * [BUG] cannot create an empty CD-ROM drive on HVM guest,
          reported by Fabio Fantoni in <4F9672DD.2080902@tiscali.it>

        * does not automatically try to select a (set of) node(s) on
          which to create a VM and pin its vcpus there. (Dario
          Faggioli, v2 posted)

    * More formally deprecate xm/xend. Manpage patches already in
      tree. Needs release noting and communication around -rc1 to
      remind people to test xl.

    * calling hotplug scripts from xl (Linux and NetBSD) (Roger Pau
      Monné, v6 posted, awaiting review)

    * Block script support -- follows on from hotplug script (Roger
      Pau Monné, "just be a matter of removing an "if"")

    * Adjustments needed for qdisk backend to work on non-pvops Linux.
      "qemu/xendisk: set maximum number of grants to be used" (Jan
      Beulich, patch committed to qemu-xen-upstream, pending for
      qemu-xen-traditional)

    * [CHECK] Confirm that migration from Xen 4.1 -> 4.2 works.

    * [BUG] LIBLEAFDIR et al and libfsimage do the wrong thing on
      modern Debian/Ubuntu w/ multiarch capabilities (Matt Wilson,
      patch posted)

    * [CHECK] Test stub domains work with xl.

hypervisor, nice to have:

    * PoD performance improvements (George Dunlap, and reviewed
      awaiting repost)

tools, nice to have:

    * xl compatibility with xm:

        * Accept "xl cr /dev/null param=value" to provide all config
          on the command line (W. Michael Petullo, patch posted)

    * libxl stable API

        * libxl_wait_for_free_memory/libxl_wait_for_memory_target.
          Interface needs an overhaul, related to
          locking/serialization over domain create. IanJ to add note
          about this interface being substandard but otherwise defer
          to 4.3.

        * Interfaces which may need to be async:

            * libxl_cdrom_insert. Should be easy once
              disk_{add,remove} done. This is basically a helper
              function and its functionality can be implemented in
              terms of the libxl_disk_* interfaces. If this is not
              done in time we should document as a substandard
              interface which is subject to change post 4.2.

    * xl.cfg(5) documentation patch for qemu-upstream
      videoram/videomem support:
      http://lists.xen.org/archives/html/xen-devel/2012-05/msg00250.html
      qemu-upstream doesn't support specifying videomem size for the
      HVM guest cirrus/stdvga.  (but this works with
      qemu-xen-traditional). (Pasi Kärkkäinen)

    * qemu-upstream for NetBSD (Roger, patch commited to NetBSD
      kernel, awaiting approval, DONE as far as Xen 4.2 is concerned)

    * [BUG] long stop during the guest boot process with qcow image,
      reported by Intel: http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1821

    * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
      http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822

    * Load blktap driver from xencommons initscript if available, thread at:
      <db614e92faf743e20b3f.1337096977@kodo2>. To be fixed more
      properly in 4.3. (Patch posted, discussion, plan to take simple
      xencommons patch for 4.2 and revist for 4.3)




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Andrew Cooper]

On 20/06/12 12:29, Ian Campbell wrote:
> A bit late this week, due to my being on vacation. Normal Monday
> service should be resumed next week.
>
> Plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
>
> The time line is as follows:
>
> 19 March        -- TODO list locked down
> 2 April         -- Feature Freeze
>                                                 << WE ARE HERE
> When It's Ready -- First release candidate
> Weekly          -- RCN+1 until release
>
> The updated TODO list follows.
>
> If you are aware of any bugs which must/should be fixed for 4.2 then
> please reply to this thread (otherwise I may not remember to pick them
> up next week)
>
> As well as [BUG]s I've also started tracking things to [CHECK] before
> the release. These are basically for things which we ought to confirm
> during the RC cycles e.g. things which are not covered by automated
> testing.
>
> Per the release plan a strong case will now have to be made as to why
> new items should be added to the list, especially as a blocker, rather
> than deferred to 4.3.
>
> hypervisor, blockers:
>
>     * None

Not certain if this is a blocker or nice-to-have, but we have identified
a regression with Xen's ability to boot, suppectedly due to c/s
25336:edd7c7ad1ad2 "x86: adjust handling of interrupts coming in via
legacy vectors" on AMD hardware with an MP-BIOS bug claiming that the
PIT is not connected through the IO-APIC.  Fixing this is next on my
todo list, and I hope to have a solution available by the end of the week.

~Andrew

>  
> tools, blockers:
>
>     * libxl stable API -- we would like 4.2 to define a stable API
>       which downstream's can start to rely on not changing. Aspects of
>       this are:
>
>         * Interfaces which may need to be async:
>
>             * libxl_domain_suspend. Move xc_domain_save/restore into a
>               separate process (Ian Jackson, patches reviews and reposted).
>
>             * libxl_device_{disk,nic,vkb,add,pci}_add (and
>               remove). (Roger Pau Monné, patches posted for disk & nic, vkb
>               trivial, not looked at pci yet)
>
>         * LIBXL_NIC_TYPE enum names are confusing. (Roger, included in
>           calling hotplug scripts series)
>
>         * use libxl_cpumap for b_info.avail_cpus instead of an int,
>           this allows setting more than 31 CPUS (Yang Z Zhang, patches
>           posted, awaiting a repost)
>
>         * use an enum for VGA interface type (e.g. Cirrus,
>           StdVGA). Allows for QXL support (in 4.3). (Zhou Peng,
>           awaiting repost)
>
>     * xl compatibility with xm:
>
>         * [BUG] cannot create an empty CD-ROM drive on HVM guest,
>           reported by Fabio Fantoni in <4F9672DD.2080902@tiscali.it>
>
>         * does not automatically try to select a (set of) node(s) on
>           which to create a VM and pin its vcpus there. (Dario
>           Faggioli, v2 posted)
>
>     * More formally deprecate xm/xend. Manpage patches already in
>       tree. Needs release noting and communication around -rc1 to
>       remind people to test xl.
>
>     * calling hotplug scripts from xl (Linux and NetBSD) (Roger Pau
>       Monné, v6 posted, awaiting review)
>
>     * Block script support -- follows on from hotplug script (Roger
>       Pau Monné, "just be a matter of removing an "if"")
>
>     * Adjustments needed for qdisk backend to work on non-pvops Linux.
>       "qemu/xendisk: set maximum number of grants to be used" (Jan
>       Beulich, patch committed to qemu-xen-upstream, pending for
>       qemu-xen-traditional)
>
>     * [CHECK] Confirm that migration from Xen 4.1 -> 4.2 works.
>
>     * [BUG] LIBLEAFDIR et al and libfsimage do the wrong thing on
>       modern Debian/Ubuntu w/ multiarch capabilities (Matt Wilson,
>       patch posted)
>
>     * [CHECK] Test stub domains work with xl.
>
> hypervisor, nice to have:
>
>     * PoD performance improvements (George Dunlap, and reviewed
>       awaiting repost)
>
> tools, nice to have:
>
>     * xl compatibility with xm:
>
>         * Accept "xl cr /dev/null param=value" to provide all config
>           on the command line (W. Michael Petullo, patch posted)
>
>     * libxl stable API
>
>         * libxl_wait_for_free_memory/libxl_wait_for_memory_target.
>           Interface needs an overhaul, related to
>           locking/serialization over domain create. IanJ to add note
>           about this interface being substandard but otherwise defer
>           to 4.3.
>
>         * Interfaces which may need to be async:
>
>             * libxl_cdrom_insert. Should be easy once
>               disk_{add,remove} done. This is basically a helper
>               function and its functionality can be implemented in
>               terms of the libxl_disk_* interfaces. If this is not
>               done in time we should document as a substandard
>               interface which is subject to change post 4.2.
>
>     * xl.cfg(5) documentation patch for qemu-upstream
>       videoram/videomem support:
>       http://lists.xen.org/archives/html/xen-devel/2012-05/msg00250.html
>       qemu-upstream doesn't support specifying videomem size for the
>       HVM guest cirrus/stdvga.  (but this works with
>       qemu-xen-traditional). (Pasi Kärkkäinen)
>
>     * qemu-upstream for NetBSD (Roger, patch commited to NetBSD
>       kernel, awaiting approval, DONE as far as Xen 4.2 is concerned)
>
>     * [BUG] long stop during the guest boot process with qcow image,
>       reported by Intel: http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1821
>
>     * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
>       http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822
>
>     * Load blktap driver from xencommons initscript if available, thread at:
>       <db614e92faf743e20b3f.1337096977@kodo2>. To be fixed more
>       properly in 4.3. (Patch posted, discussion, plan to take simple
>       xencommons patch for 4.2 and revist for 4.3)
>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Jan Beulich]

>>> On 20.06.12 at 13:43, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> On 20/06/12 12:29, Ian Campbell wrote:
>> hypervisor, blockers:
>>
>>     * None
> 
> Not certain if this is a blocker or nice-to-have, but we have identified
> a regression with Xen's ability to boot, suppectedly due to c/s
> 25336:edd7c7ad1ad2 "x86: adjust handling of interrupts coming in via
> legacy vectors" on AMD hardware with an MP-BIOS bug claiming that the
> PIT is not connected through the IO-APIC.  Fixing this is next on my
> todo list, and I hope to have a solution available by the end of the week.

Sure this (being a regression) is a blocker. Care to share any
details, e.g. a hypervisor logs with "apic_verbosity=debug" without
and with that c/s?

Jan

Re: Xen 4.2 Release Plan / TODO

[Andrew Cooper]

On 20/06/12 14:07, Jan Beulich wrote:
>>>> On 20.06.12 at 13:43, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> On 20/06/12 12:29, Ian Campbell wrote:
>>> hypervisor, blockers:
>>>
>>>     * None
>> Not certain if this is a blocker or nice-to-have, but we have identified
>> a regression with Xen's ability to boot, suppectedly due to c/s
>> 25336:edd7c7ad1ad2 "x86: adjust handling of interrupts coming in via
>> legacy vectors" on AMD hardware with an MP-BIOS bug claiming that the
>> PIT is not connected through the IO-APIC.  Fixing this is next on my
>> todo list, and I hope to have a solution available by the end of the week.
> Sure this (being a regression) is a blocker. Care to share any
> details, e.g. a hypervisor logs with "apic_verbosity=debug" without
> and with that c/s?
>
> Jan
>

I have not investigated yet as I am just wrapping up a more important
issue, but the issue was a constant console spam saying no handler for
vector 0xe5, which was causing the server to run like treacle.  The
issue started occurring shortly after I backported c/s 25336 to
XenServer, which is why I suspect it as the culprit.  Having said that,
the patch appears to work fine on every other server we have, so I
suspect its some motherboard quirk; it is a slightly old AMD box we
have, and is 100% reproducible.  I am hoping to have time to start
looking at the problem this afternoon.

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com

Re: Xen 4.2 Release Plan / TODO

[Andrew Cooper]

> I have not investigated yet as I am just wrapping up a more important
> issue, but the issue was a constant console spam saying no handler for
> vector 0xe5, which was causing the server to run like treacle.  The
> issue started occurring shortly after I backported c/s 25336 to
> XenServer, which is why I suspect it as the culprit.  Having said that,
> the patch appears to work fine on every other server we have, so I
> suspect its some motherboard quirk; it is a slightly old AMD box we
> have, and is 100% reproducible.  I am hoping to have time to start
> looking at the problem this afternoon.
>

After some investigation, it appears that the server in question is not
old (as the bug report I received said).  It appears to be a fairly-new
evaluation dual socket AMD box, with a beta-looking BIOS, which can't
reliably boot Xen 3.4 or Xen 4.1 (with or without the changeset), can't
reliably reboot via ACPI, and cant reliably turn back on after you cut
its power.

As such, I would say that the server is rather more suspect than Xen
4.2, so this IRQ issue should probably not be considered a blocker.

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Wed, 2012-06-20 at 20:29 +0100, Andrew Cooper wrote:
> > I have not investigated yet as I am just wrapping up a more important
> > issue, but the issue was a constant console spam saying no handler for
> > vector 0xe5, which was causing the server to run like treacle.  The
> > issue started occurring shortly after I backported c/s 25336 to
> > XenServer, which is why I suspect it as the culprit.  Having said that,
> > the patch appears to work fine on every other server we have, so I
> > suspect its some motherboard quirk; it is a slightly old AMD box we
> > have, and is 100% reproducible.  I am hoping to have time to start
> > looking at the problem this afternoon.
> >
> 
> After some investigation, it appears that the server in question is not
> old (as the bug report I received said).  It appears to be a fairly-new
> evaluation dual socket AMD box, with a beta-looking BIOS, which can't
> reliably boot Xen 3.4 or Xen 4.1 (with or without the changeset), can't
> reliably reboot via ACPI, and cant reliably turn back on after you cut
> its power.
> 
> As such, I would say that the server is rather more suspect than Xen
> 4.2, so this IRQ issue should probably not be considered a blocker.

Thanks, I won't include this issue on the list this week then.

Ian.

Xen 4.2 Release Plan / TODO

[Ian Campbell]

A day late due to the Xen.org docs day yesterday. Thanks to all who
took part -- next one is July 30!

Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html

The time line is as follows:

19 March        -- TODO list locked down
2 April         -- Feature Freeze
                                                << WE ARE HERE
When It's Ready -- First release candidate
Weekly          -- RCN+1 until release

The updated TODO list follows.

If you are aware of any bugs which must/should be fixed for 4.2 then
please reply to this thread (otherwise I may not remember to pick them
up next week)

Per the release plan a strong case will now have to be made as to why
new items should be added to the list, especially as a blocker, rather
than deferred to 4.3.

hypervisor, blockers:

    * None
 
tools, blockers:

    * libxl stable API -- we would like 4.2 to define a stable API
      which downstream's can start to rely on not changing. Aspects of
      this are:

        * Interfaces which may need to be async:

            * libxl_domain_suspend. Move xc_domain_save/restore into a
              separate process (Ian Jackson, patches reviewed and reposted).

            * libxl_device_{disk,nic,vkb,add,pci}_add (and
              remove). (Roger Pau Monné, patches posted for disk & nic, vkb
              trivial, not looked at pci yet)

        * LIBXL_NIC_TYPE enum names are confusing. (Roger, included in
          calling hotplug scripts series)

        * use libxl_cpumap for b_info.avail_cpus instead of an int,
          this allows setting more than 31 CPUS (Yang Z Zhang, patches
          posted, awaiting a repost)

        * use an enum for VGA interface type (e.g. Cirrus,
          StdVGA). Allows for QXL support (in 4.3). (Zhou Peng,
          awaiting repost)

    * xl compatibility with xm:

        * [BUG] cannot create an empty CD-ROM drive on HVM guest,
          reported by Fabio Fantoni in <4F9672DD.2080902@tiscali.it>

        * does not automatically try to select a (set of) node(s) on
          which to create a VM and pin its vcpus there. (Dario
          Faggioli, posted and reviewed, awaiting a repost)

    * [CHECK] More formally deprecate xm/xend. Manpage patches already
      in tree. Needs release noting and communication around -rc1 to
      remind people to test xl.

    * calling hotplug scripts from xl (Linux and NetBSD) (Roger Pau
      Monné, posted, awaiting review)

    * Block script support -- follows on from hotplug script (Roger
      Pau Monné, "just be a matter of removing an "if"")

    * Adjustments needed for qdisk backend to work on non-pvops Linux.
      "qemu/xendisk: set maximum number of grants to be used" (Jan
      Beulich, patch committed to qemu-xen-upstream, pending for
      qemu-xen-traditional)

    * [CHECK] Confirm that migration from Xen 4.1 -> 4.2 works.

    * [BUG] LIBLEAFDIR et al and libfsimage do the wrong thing on
      modern Debian/Ubuntu w/ multiarch capabilities (Matt Wilson,
      patch posted)

    * [BUG] stub domains broken. HVM_PARAM_DM_DOMAIN needs to reset
      HVM_PARAM_BUFIOREQ_EVTCHN (Ian Campbell, Anthony Perard)

hypervisor, nice to have:

    * PoD performance improvements (George Dunlap, and reviewed
      awaiting repost)

tools, nice to have:

    * xl compatibility with xm:

        * Accept "xl cr /dev/null param=value" to provide all config
          on the command line (W. Michael Petullo, patch posted)

    * libxl stable API

        * libxl_wait_for_free_memory/libxl_wait_for_memory_target.
          Interface needs an overhaul, related to
          locking/serialization over domain create. IanJ to add note
          about this interface being substandard but otherwise defer
          to 4.3.

        * Interfaces which may need to be async:

            * libxl_cdrom_insert. Should be easy once
              disk_{add,remove} done. This is basically a helper
              function and its functionality can be implemented in
              terms of the libxl_disk_* interfaces. If this is not
              done in time we should document as a substandard
              interface which is subject to change post 4.2.

    * xl.cfg(5) documentation patch for qemu-upstream
      videoram/videomem support:
      http://lists.xen.org/archives/html/xen-devel/2012-05/msg00250.html
      qemu-upstream doesn't support specifying videomem size for the
      HVM guest cirrus/stdvga.  (but this works with
      qemu-xen-traditional). (Pasi Kärkkäinen)

    * [BUG] long stop during the guest boot process with qcow image,
      reported by Intel: http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1821

    * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
      http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822

    * Load blktap driver from xencommons initscript if available, thread at:
      <db614e92faf743e20b3f.1337096977@kodo2>. To be fixed more
      properly in 4.3. (Patch posted, discussion, plan to take simple
      xencommons patch for 4.2 and revist for 4.3)




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Matt Wilson]

On Tue, Jun 26, 2012 at 01:39:42AM -0700, Ian Campbell wrote:
>     * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
>       http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822

This may be a problem with the guest, rather than a problem with Xen
or the toolstack. I tested on a domU running a 3.2.20-based Linux
kernel. The domU kernel didn't automatically enable the vCPUs after
calling "xl vcpu-set" to increase the allocation.

# xl list 2
Name                                        ID   Mem VCPUs      State Time(s)
test                                         2 15360     1     -b---- 6617.6
# xl vcpu-set 2 4
# xl list 2
Name                                        ID   Mem VCPUs      State Time(s)
test                                         2 15360     1     -b---- 6617.7

But when I hotplugged the CPUs on the domU side with

  for online in /sys/devices/system/cpu/cpu*/online; do
      echo 1 > $online
  done

Now we see:

# xl list 2
Name                                        ID   Mem VCPUs      State Time(s)
test                                         2 15360     4     -b---- 6617.8

I've added this as a comment to the bugzilla.

Matt

Re: Xen 4.2 Release Plan / TODO

[Konrad Rzeszutek Wilk]

On Tue, Jun 26, 2012 at 01:31:51PM -0700, Matt Wilson wrote:
> On Tue, Jun 26, 2012 at 01:39:42AM -0700, Ian Campbell wrote:
> >     * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
> >       http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822
> 
> This may be a problem with the guest, rather than a problem with Xen
> or the toolstack. I tested on a domU running a 3.2.20-based Linux
> kernel. The domU kernel didn't automatically enable the vCPUs after
> calling "xl vcpu-set" to increase the allocation.
> 
> # xl list 2
> Name                                        ID   Mem VCPUs      State Time(s)
> test                                         2 15360     1     -b---- 6617.6
> # xl vcpu-set 2 4
> # xl list 2
> Name                                        ID   Mem VCPUs      State Time(s)
> test                                         2 15360     1     -b---- 6617.7
> 
> But when I hotplugged the CPUs on the domU side with
> 
>   for online in /sys/devices/system/cpu/cpu*/online; do
>       echo 1 > $online
>   done
> 
> Now we see:
> 
> # xl list 2
> Name                                        ID   Mem VCPUs      State Time(s)
> test                                         2 15360     4     -b---- 6617.8
> 
> I've added this as a comment to the bugzilla.

That is a generic bug - its a race in the generic code where
the cpuX directory is created; then udev kicks off and tries to
echo 1 > cpuX/online (but online hasn't been created yet); the cpu-hotplug
code creates 'online' directory.

I've asked Greg KH about it, and here is his take:
https://lkml.org/lkml/2012/4/30/198

But I never got to prep a patch. If somebody gets to do it before me,
I owe them a beer!

Re: Xen 4.2 Release Plan / TODO

[Matt Wilson]

On Tue, Jun 26, 2012 at 02:09:06PM -0700, Konrad Rzeszutek Wilk wrote:
> On Tue, Jun 26, 2012 at 01:31:51PM -0700, Matt Wilson wrote:
> > On Tue, Jun 26, 2012 at 01:39:42AM -0700, Ian Campbell wrote:
> > >     * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
> > >       http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822
> > 
> > This may be a problem with the guest, rather than a problem with Xen
> > or the toolstack. I tested on a domU running a 3.2.20-based Linux
> > kernel. The domU kernel didn't automatically enable the vCPUs after
> > calling "xl vcpu-set" to increase the allocation.
> > [...]
> > 
> > I've added this as a comment to the bugzilla.
> 
> That is a generic bug - its a race in the generic code where
> the cpuX directory is created; then udev kicks off and tries to
> echo 1 > cpuX/online (but online hasn't been created yet); the cpu-hotplug
> code creates 'online' directory.
> 
> I've asked Greg KH about it, and here is his take:
> https://lkml.org/lkml/2012/4/30/198
> 
> But I never got to prep a patch. If somebody gets to do it before me,
> I owe them a beer!

My problem was that I didn't even have a udev rule to auto-online
hotplugged CPUs. Everything works as expected after adding the rule
suggested on the wiki: http://wiki.xen.org/wiki/Paravirt_Linux_CPU_Hotplug

I'm thinking that the bug report is just this well hashed out
problem. For historical reference, here's the thread discussing the
behavior change in pv-ops kernels:
  http://lists.xen.org/archives/html/xen-devel/2010-05/msg00516.html

Matt

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Tue, 2012-06-26 at 23:57 +0100, Matt Wilson wrote:
> On Tue, Jun 26, 2012 at 02:09:06PM -0700, Konrad Rzeszutek Wilk wrote:
> > On Tue, Jun 26, 2012 at 01:31:51PM -0700, Matt Wilson wrote:
> > > On Tue, Jun 26, 2012 at 01:39:42AM -0700, Ian Campbell wrote:
> > > >     * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
> > > >       http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822
> > > 
> > > This may be a problem with the guest, rather than a problem with Xen
> > > or the toolstack. I tested on a domU running a 3.2.20-based Linux
> > > kernel. The domU kernel didn't automatically enable the vCPUs after
> > > calling "xl vcpu-set" to increase the allocation.
> > > [...]
> > > 
> > > I've added this as a comment to the bugzilla.
> > 
> > That is a generic bug - its a race in the generic code where
> > the cpuX directory is created; then udev kicks off and tries to
> > echo 1 > cpuX/online (but online hasn't been created yet); the cpu-hotplug
> > code creates 'online' directory.
> > 
> > I've asked Greg KH about it, and here is his take:
> > https://lkml.org/lkml/2012/4/30/198
> > 
> > But I never got to prep a patch. If somebody gets to do it before me,
> > I owe them a beer!
> 
> My problem was that I didn't even have a udev rule to auto-online
> hotplugged CPUs. Everything works as expected after adding the rule
> suggested on the wiki: http://wiki.xen.org/wiki/Paravirt_Linux_CPU_Hotplug
> 
> I'm thinking that the bug report is just this well hashed out
> problem. For historical reference, here's the thread discussing the
> behavior change in pv-ops kernels:
>   http://lists.xen.org/archives/html/xen-devel/2010-05/msg00516.html

I've added a link to that thread to the wiki page.

Re: Xen 4.2 Release Plan / TODO

[Ren, Yongjie]

> -----Original Message-----
> From: xen-devel-bounces@lists.xen.org
> [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Matt Wilson
> Sent: Wednesday, June 27, 2012 6:57 AM
> To: Konrad Rzeszutek Wilk
> Cc: Ian Campbell; xen-devel
> Subject: Re: [Xen-devel] Xen 4.2 Release Plan / TODO
> 
> On Tue, Jun 26, 2012 at 02:09:06PM -0700, Konrad Rzeszutek Wilk wrote:
> > On Tue, Jun 26, 2012 at 01:31:51PM -0700, Matt Wilson wrote:
> > > On Tue, Jun 26, 2012 at 01:39:42AM -0700, Ian Campbell wrote:
> > > >     * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
> > > >       http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822
> > >
> > > This may be a problem with the guest, rather than a problem with Xen
> > > or the toolstack. I tested on a domU running a 3.2.20-based Linux
> > > kernel. The domU kernel didn't automatically enable the vCPUs after
> > > calling "xl vcpu-set" to increase the allocation.
> > > [...]
> > >
> > > I've added this as a comment to the bugzilla.
> >
> > That is a generic bug - its a race in the generic code where
> > the cpuX directory is created; then udev kicks off and tries to
> > echo 1 > cpuX/online (but online hasn't been created yet); the
> cpu-hotplug
> > code creates 'online' directory.
> >
> > I've asked Greg KH about it, and here is his take:
> > https://lkml.org/lkml/2012/4/30/198
> >
> > But I never got to prep a patch. If somebody gets to do it before me,
> > I owe them a beer!
> 
> My problem was that I didn't even have a udev rule to auto-online
> hotplugged CPUs. Everything works as expected after adding the rule
> suggested on the wiki:
> http://wiki.xen.org/wiki/Paravirt_Linux_CPU_Hotplug
> 
> I'm thinking that the bug report is just this well hashed out
> problem. For historical reference, here's the thread discussing the
> behavior change in pv-ops kernels:
>   http://lists.xen.org/archives/html/xen-devel/2010-05/msg00516.html
> 
Hi Matt, thanks for the useful status.
According to my testing on xen-unstable(#25517) with linux3.4.3 dom0 and RHEL6.2
hvm guest, 'xl vcpu-set' can increase the number of the vCPU for guest. But it
can't decrease the number of vCPU. After trying to decrease the number of vCPU,
the number of the onlined CPU in /sys/devices/system/cpu/ or /proc/cpuinfo is not
changed.

But I can do "echo 0 > /sys/devices/system/cpu/cpu3/online" to offline the CPU3
in the guest even if I didn't use any command line like 'xl vcpu-set'.

Also, I can do "echo 1 > /sys/devices/LNXSYSTM:00/LNXSYBUS:00/LNXCPU:03/eject"
to remove CPU3 in the guest.

Note that 'CPU3' can be any CPU except for CPU0.

Also add a comment in the bugzilla.

Re: Xen 4.2 Release Plan / TODO

[Jan Beulich]

>>> On 26.06.12 at 10:39, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> hypervisor, blockers:
> 
>     * None

We should evaluate whether getting the vMCE interface into a
sane state ought to be treated as a blocker (see Jinsong's
recent posting about their design work). Otherwise we'll run
into yet another round of compatibility issues when moving on
to 4.3, particularly with respect to migration.

Jan

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Wed, 2012-06-27 at 14:12 +0100, Jan Beulich wrote:
> >>> On 26.06.12 at 10:39, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > hypervisor, blockers:
> > 
> >     * None
> 
> We should evaluate whether getting the vMCE interface into a
> sane state ought to be treated as a blocker (see Jinsong's
> recent posting about their design work). Otherwise we'll run
> into yet another round of compatibility issues when moving on
> to 4.3, particularly with respect to migration.

Isn't this way to late for 4.2 if it's only at the design stage right
now?

Does vMCE exist in Xen unstable today? Did it exist in 4.1?

I think we'll need to just suck it up and accept that migration needs
handling for 4.3.

Ian.

Re: Xen 4.2 Release Plan / TODO

[Jan Beulich]

>>> On 27.06.12 at 16:52, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Wed, 2012-06-27 at 14:12 +0100, Jan Beulich wrote:
>> >>> On 26.06.12 at 10:39, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > hypervisor, blockers:
>> > 
>> >     * None
>> 
>> We should evaluate whether getting the vMCE interface into a
>> sane state ought to be treated as a blocker (see Jinsong's
>> recent posting about their design work). Otherwise we'll run
>> into yet another round of compatibility issues when moving on
>> to 4.3, particularly with respect to migration.
> 
> Isn't this way to late for 4.2 if it's only at the design stage right
> now?

That's why I'm bringing this up.

> Does vMCE exist in Xen unstable today? Did it exist in 4.1?

Yes. And what information get migrated is different between
4.1 and 4.2 already.

> I think we'll need to just suck it up and accept that migration needs
> handling for 4.3.

I was hoping to have at least the HVM save records in 4.2 in a
state suitable for 4.3.

Jan

Re: Xen 4.2 Release Plan / TODO

[Ian Campbell]

On Wed, 2012-06-27 at 15:57 +0100, Jan Beulich wrote:
> >>> On 27.06.12 at 16:52, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Wed, 2012-06-27 at 14:12 +0100, Jan Beulich wrote:
> >> >>> On 26.06.12 at 10:39, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >> > hypervisor, blockers:
> >> > 
> >> >     * None
> >> 
> >> We should evaluate whether getting the vMCE interface into a
> >> sane state ought to be treated as a blocker (see Jinsong's
> >> recent posting about their design work). Otherwise we'll run
> >> into yet another round of compatibility issues when moving on
> >> to 4.3, particularly with respect to migration.
> > 
> > Isn't this way to late for 4.2 if it's only at the design stage right
> > now?
> 
> That's why I'm bringing this up.
> 
> > Does vMCE exist in Xen unstable today? Did it exist in 4.1?
> 
> Yes. And what information get migrated is different between
> 4.1 and 4.2 already.
> 
> > I think we'll need to just suck it up and accept that migration needs
> > handling for 4.3.
> 
> I was hoping to have at least the HVM save records in 4.2 in a
> state suitable for 4.3.

That might be reasonable, I guess it depends what that looks like and
how much real code impact as opposed to just save record it has?

Ian.

Re: Xen 4.2 Release Plan / TODO

[Jan Beulich]

>>> On 27.06.12 at 17:01, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Wed, 2012-06-27 at 15:57 +0100, Jan Beulich wrote:
>> >>> On 27.06.12 at 16:52, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > On Wed, 2012-06-27 at 14:12 +0100, Jan Beulich wrote:
>> >> >>> On 26.06.12 at 10:39, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> >> > hypervisor, blockers:
>> >> > 
>> >> >     * None
>> >> 
>> >> We should evaluate whether getting the vMCE interface into a
>> >> sane state ought to be treated as a blocker (see Jinsong's
>> >> recent posting about their design work). Otherwise we'll run
>> >> into yet another round of compatibility issues when moving on
>> >> to 4.3, particularly with respect to migration.
>> > 
>> > Isn't this way to late for 4.2 if it's only at the design stage right
>> > now?
>> 
>> That's why I'm bringing this up.
>> 
>> > Does vMCE exist in Xen unstable today? Did it exist in 4.1?
>> 
>> Yes. And what information get migrated is different between
>> 4.1 and 4.2 already.
>> 
>> > I think we'll need to just suck it up and accept that migration needs
>> > handling for 4.3.
>> 
>> I was hoping to have at least the HVM save records in 4.2 in a
>> state suitable for 4.3.
> 
> That might be reasonable, I guess it depends what that looks like and
> how much real code impact as opposed to just save record it has?

Certainly. Minimally, code handling the expected to be extended
save record would need to be added though.

In any case, I'd appreciate if you could add this at least to the
nice-to-have section.

Jan

Re: Xen 4.2 Release Plan / TODO

[Tim Deegan]

At 09:39 +0100 on 26 Jun (1340703582), Ian Campbell wrote:
> hypervisor, nice to have:
> 
>     * PoD performance improvements (George Dunlap, and reviewed
>       awaiting repost)

Reposted, and applied: DONE.

Tim.

Xen 4.2 Release Plan / TODO

[Ian Campbell]

Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html

The time line is as follows:

19 March        -- TODO list locked down
2 April         -- Feature Freeze
                                                << WE ARE HERE
When It's Ready -- First release candidate
Weekly          -- RCN+1 until release

Lots of DONE this week which is good, including one of the big three
remaining issues (libxl_domain_suspend asyncification) has gone in
now. Still to come are Roger's hotplug script changes (which acutally
cover multiple items on the list) and Dario's basic NUMA support, both
of which I think are almost there.

The updated TODO list follows.

If you are aware of any bugs which must/should be fixed for 4.2 then
please reply to this thread (otherwise I may not remember to pick them
up next week)

Per the release plan a strong case will now have to be made as to why
new items should be added to the list, especially as a blocker, rather
than deferred to 4.3.

hypervisor, blockers:

    * [BUG] stub domains broken. HVM_PARAM_DM_DOMAIN needs to reset
      HVM_PARAM_BUFIOREQ_EVTCHN (Anthony Perard, patch sent, reviewed,
      awaiting repost)

tools, blockers:

    * libxl stable API -- we would like 4.2 to define a stable API
      which downstream's can start to rely on not changing. Aspects of
      this are:

        * Interfaces which may need to be async:

            * libxl_domain_suspend. Move xc_domain_save/restore into a
              separate process (Ian Jackson, DONE).

            * libxl_device_{disk,nic,vkb,add,pci}_add (and
              remove). (Roger Pau Monné, disk & nic included in
              calling hotplug scripts from xl, vkb
              trivial, not looked at pci yet)

        * LIBXL_NIC_TYPE enum names are confusing. (Roger, included in
          calling hotplug scripts series)

        * use libxl_cpumap for b_info.avail_cpus instead of an int,
          this allows setting more than 31 CPUS (Yang Z Zhang, DONE)

        * use an enum for VGA interface type (e.g. Cirrus,
          StdVGA). Allows for QXL support (in 4.3). (Zhou Peng, DONE)

    * xl compatibility with xm:

        * [BUG] cannot create an empty CD-ROM drive on HVM guest,
          reported by Fabio Fantoni in <4F9672DD.2080902@tiscali.it>

        * does not automatically try to select a (set of) node(s) on
          which to create a VM and pin its vcpus there. (Dario
          Faggioli, posted and reviewed, awaiting a repost)

    * [CHECK] More formally deprecate xm/xend. Manpage patches already
      in tree. Needs release noting and communication around -rc1 to
      remind people to test xl.

    * calling hotplug scripts from xl (Linux and NetBSD) (Roger Pau
      Monné, waiting on another iteration)

    * Block script support -- follows on from hotplug script (Roger
      Pau Monné, "just be a matter of removing an "if"")

    * Adjustments needed for qdisk backend to work on non-pvops Linux.
      "qemu/xendisk: set maximum number of grants to be used" (Jan
      Beulich, DONE for both qemu-xen-upstream and
      qemu-xen-traditional)

    * [CHECK] Confirm that migration from Xen 4.1 -> 4.2 works.

    * [BUG] LIBLEAFDIR et al and libfsimage do the wrong thing on
      modern Debian/Ubuntu w/ multiarch capabilities (Matt Wilson,
      patch posted)

    * libxl to reject attempts to migrate a domain using upstream
      qemu, due to lack of log dirty support (Ian C, patch sent,
      reviewed, awaiting repost)

hypervisor, nice to have:

    * PoD performance improvements (George Dunlap, DONE)

    * vMCE save/restore changes, to simplify migration 4.2->4.3
     (Jinsong Liu)

tools, nice to have:

    * xl compatibility with xm:

        * Accept "xl cr /dev/null param=value" to provide all config
          on the command line (W. Michael Petullo, patch posted)

    * libxl stable API

        * libxl_wait_for_free_memory/libxl_wait_for_memory_target.
          Interface needs an overhaul, related to
          locking/serialization over domain create. IanJ to add note
          about this interface being substandard but otherwise defer
          to 4.3.

        * Interfaces which may need to be async:

            * libxl_cdrom_insert. Should be easy once
              disk_{add,remove} done. This is basically a helper
              function and its functionality can be implemented in
              terms of the libxl_disk_* interfaces. If this is not
              done in time we should document as a substandard
              interface which is subject to change post 4.2.

    * xl.cfg(5) documentation patch for qemu-upstream
      videoram/videomem support:
      http://lists.xen.org/archives/html/xen-devel/2012-05/msg00250.html
      qemu-upstream doesn't support specifying videomem size for the
      HVM guest cirrus/stdvga.  (but this works with
      qemu-xen-traditional). (Pasi Kärkkäinen)

    * [BUG] long stop during the guest boot process with qcow image,
      reported by Intel: http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1821

    * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
      http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822

    * Load blktap driver from xencommons initscript if available, thread at:
      <db614e92faf743e20b3f.1337096977@kodo2>. To be fixed more
      properly in 4.3. (Patch posted, discussion, plan to take simple
      xencommons patch for 4.2 and revist for 4.3)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Jan Beulich]

>>> On 02.07.12 at 13:02, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> hypervisor, blockers:
> 
>     * [BUG] stub domains broken. HVM_PARAM_DM_DOMAIN needs to reset
>       HVM_PARAM_BUFIOREQ_EVTCHN (Anthony Perard, patch sent, reviewed,
>       awaiting repost)

Committed (with minor adjustments, but Anthony, please double
check).

Jan

Re: Xen 4.2 Release Plan / TODO

[Anthony PERARD]

On Tue, Jul 3, 2012 at 8:52 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 02.07.12 at 13:02, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> hypervisor, blockers:
>>
>>     * [BUG] stub domains broken. HVM_PARAM_DM_DOMAIN needs to reset
>>       HVM_PARAM_BUFIOREQ_EVTCHN (Anthony Perard, patch sent, reviewed,
>>       awaiting repost)
>
> Committed (with minor adjustments, but Anthony, please double
> check).

Look good to me. Thanks.

-- 
Anthony PERARD

Re: Xen 4.2 Release Plan / TODO

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 814 bytes --]

On Mon, 2012-07-02 at 12:02 +0100, Ian Campbell wrote:
> tools, blockers:
> 
> [...]
>
>     * xl compatibility with xm:
> 
>         * [BUG] cannot create an empty CD-ROM drive on HVM guest,
>           reported by Fabio Fantoni in <4F9672DD.2080902@tiscali.it>
> 
>         * does not automatically try to select a (set of) node(s) on
>           which to create a VM and pin its vcpus there. (Dario
>           Faggioli, posted and reviewed, awaiting a repost)
>
Just reposted, awaiting review.

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Roger Pau Monne]

Ian Campbell wrote:
> Plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
>
> The time line is as follows:
>
> 19 March        -- TODO list locked down
> 2 April         -- Feature Freeze
>                                                  <<  WE ARE HERE
> When It's Ready -- First release candidate
> Weekly          -- RCN+1 until release
>
> Lots of DONE this week which is good, including one of the big three
> remaining issues (libxl_domain_suspend asyncification) has gone in
> now. Still to come are Roger's hotplug script changes (which acutally
> cover multiple items on the list) and Dario's basic NUMA support, both
> of which I think are almost there.
>
> The updated TODO list follows.
>
> If you are aware of any bugs which must/should be fixed for 4.2 then
> please reply to this thread (otherwise I may not remember to pick them
> up next week)
>
> Per the release plan a strong case will now have to be made as to why
> new items should be added to the list, especially as a blocker, rather
> than deferred to 4.3.
>
> hypervisor, blockers:
>
>      * [BUG] stub domains broken. HVM_PARAM_DM_DOMAIN needs to reset
>        HVM_PARAM_BUFIOREQ_EVTCHN (Anthony Perard, patch sent, reviewed,
>        awaiting repost)
>
> tools, blockers:
>
>      * libxl stable API -- we would like 4.2 to define a stable API
>        which downstream's can start to rely on not changing. Aspects of
>        this are:
>
>          * Interfaces which may need to be async:
>
>              * libxl_domain_suspend. Move xc_domain_save/restore into a
>                separate process (Ian Jackson, DONE).
>
>              * libxl_device_{disk,nic,vkb,add,pci}_add (and
>                remove). (Roger Pau Monné, disk&  nic included in
>                calling hotplug scripts from xl, vkb
>                trivial, not looked at pci yet)
>
>          * LIBXL_NIC_TYPE enum names are confusing. (Roger, included in
>            calling hotplug scripts series)
>
>          * use libxl_cpumap for b_info.avail_cpus instead of an int,
>            this allows setting more than 31 CPUS (Yang Z Zhang, DONE)
>
>          * use an enum for VGA interface type (e.g. Cirrus,
>            StdVGA). Allows for QXL support (in 4.3). (Zhou Peng, DONE)
>
>      * xl compatibility with xm:
>
>          * [BUG] cannot create an empty CD-ROM drive on HVM guest,
>            reported by Fabio Fantoni in<4F9672DD.2080902@tiscali.it>
>
>          * does not automatically try to select a (set of) node(s) on
>            which to create a VM and pin its vcpus there. (Dario
>            Faggioli, posted and reviewed, awaiting a repost)
>
>      * [CHECK] More formally deprecate xm/xend. Manpage patches already
>        in tree. Needs release noting and communication around -rc1 to
>        remind people to test xl.
>
>      * calling hotplug scripts from xl (Linux and NetBSD) (Roger Pau
>        Monné, waiting on another iteration)

Posted v8.

>
>      * Block script support -- follows on from hotplug script (Roger
>        Pau Monné, "just be a matter of removing an "if"")
>
>      * Adjustments needed for qdisk backend to work on non-pvops Linux.
>        "qemu/xendisk: set maximum number of grants to be used" (Jan
>        Beulich, DONE for both qemu-xen-upstream and
>        qemu-xen-traditional)
>
>      * [CHECK] Confirm that migration from Xen 4.1 ->  4.2 works.
>
>      * [BUG] LIBLEAFDIR et al and libfsimage do the wrong thing on
>        modern Debian/Ubuntu w/ multiarch capabilities (Matt Wilson,
>        patch posted)
>
>      * libxl to reject attempts to migrate a domain using upstream
>        qemu, due to lack of log dirty support (Ian C, patch sent,
>        reviewed, awaiting repost)
>
> hypervisor, nice to have:
>
>      * PoD performance improvements (George Dunlap, DONE)
>
>      * vMCE save/restore changes, to simplify migration 4.2->4.3
>       (Jinsong Liu)
>
> tools, nice to have:
>
>      * xl compatibility with xm:
>
>          * Accept "xl cr /dev/null param=value" to provide all config
>            on the command line (W. Michael Petullo, patch posted)
>
>      * libxl stable API
>
>          * libxl_wait_for_free_memory/libxl_wait_for_memory_target.
>            Interface needs an overhaul, related to
>            locking/serialization over domain create. IanJ to add note
>            about this interface being substandard but otherwise defer
>            to 4.3.
>
>          * Interfaces which may need to be async:
>
>              * libxl_cdrom_insert. Should be easy once
>                disk_{add,remove} done. This is basically a helper
>                function and its functionality can be implemented in
>                terms of the libxl_disk_* interfaces. If this is not
>                done in time we should document as a substandard
>                interface which is subject to change post 4.2.
>
>      * xl.cfg(5) documentation patch for qemu-upstream
>        videoram/videomem support:
>        http://lists.xen.org/archives/html/xen-devel/2012-05/msg00250.html
>        qemu-upstream doesn't support specifying videomem size for the
>        HVM guest cirrus/stdvga.  (but this works with
>        qemu-xen-traditional). (Pasi Kärkkäinen)
>
>      * [BUG] long stop during the guest boot process with qcow image,
>        reported by Intel: http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1821
>
>      * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
>        http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822
>
>      * Load blktap driver from xencommons initscript if available, thread at:
>        <db614e92faf743e20b3f.1337096977@kodo2>. To be fixed more
>        properly in 4.3. (Patch posted, discussion, plan to take simple
>        xencommons patch for 4.2 and revist for 4.3)
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.2 Release Plan / TODO

[Roger Pau Monne]

Roger Pau Monne wrote:
> Ian Campbell wrote:
>> Plan for a 4.2 release:
>> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
>>
>> The time line is as follows:
>>
>> 19 March        -- TODO list locked down
>> 2 April         -- Feature Freeze
>>                                                  <<  WE ARE HERE
>> When It's Ready -- First release candidate
>> Weekly          -- RCN+1 until release
>>
>> Lots of DONE this week which is good, including one of the big three
>> remaining issues (libxl_domain_suspend asyncification) has gone in
>> now. Still to come are Roger's hotplug script changes (which acutally
>> cover multiple items on the list) and Dario's basic NUMA support, both
>> of which I think are almost there.
>>
>> The updated TODO list follows.
>>
>> If you are aware of any bugs which must/should be fixed for 4.2 then
>> please reply to this thread (otherwise I may not remember to pick them
>> up next week)
>>
>> Per the release plan a strong case will now have to be made as to why
>> new items should be added to the list, especially as a blocker, rather
>> than deferred to 4.3.
>>
>> hypervisor, blockers:
>>
>>      * [BUG] stub domains broken. HVM_PARAM_DM_DOMAIN needs to reset
>>        HVM_PARAM_BUFIOREQ_EVTCHN (Anthony Perard, patch sent, reviewed,
>>        awaiting repost)
>>
>> tools, blockers:
>>
>>      * libxl stable API -- we would like 4.2 to define a stable API
>>        which downstream's can start to rely on not changing. Aspects of
>>        this are:
>>
>>          * Interfaces which may need to be async:
>>
>>              * libxl_domain_suspend. Move xc_domain_save/restore into a
>>                separate process (Ian Jackson, DONE).
>>
>>              * libxl_device_{disk,nic,vkb,add,pci}_add (and
>>                remove). (Roger Pau Monné, disk&  nic included in
>>                calling hotplug scripts from xl, vkb
>>                trivial, not looked at pci yet)

vkb and vfvb posted as part of my v9 hotplug series. I've given a quick
look at PCI and it looks tricky, but I think some parts of the code are
no longer needed (mainly the wait for the device model, because we
already know Qemu is up and running when we attach PCI devices now), so
it should simplify the conversion a bit.

>>
>>          * LIBXL_NIC_TYPE enum names are confusing. (Roger, included in
>>            calling hotplug scripts series)
>>
>>          * use libxl_cpumap for b_info.avail_cpus instead of an int,
>>            this allows setting more than 31 CPUS (Yang Z Zhang, DONE)
>>
>>          * use an enum for VGA interface type (e.g. Cirrus,
>>            StdVGA). Allows for QXL support (in 4.3). (Zhou Peng, DONE)
>>
>>      * xl compatibility with xm:
>>
>>          * [BUG] cannot create an empty CD-ROM drive on HVM guest,
>>            reported by Fabio Fantoni in<4F9672DD.2080902@tiscali.it>
>>
>>          * does not automatically try to select a (set of) node(s) on
>>            which to create a VM and pin its vcpus there. (Dario
>>            Faggioli, posted and reviewed, awaiting a repost)
>>
>>      * [CHECK] More formally deprecate xm/xend. Manpage patches already
>>        in tree. Needs release noting and communication around -rc1 to
>>        remind people to test xl.
>>
>>      * calling hotplug scripts from xl (Linux and NetBSD) (Roger Pau
>>        Monné, waiting on another iteration)
> 
> Posted v8.

v9 now.

> 
>>      * Block script support -- follows on from hotplug script (Roger
>>        Pau Monné, "just be a matter of removing an "if"")
>>
>>      * Adjustments needed for qdisk backend to work on non-pvops Linux.
>>        "qemu/xendisk: set maximum number of grants to be used" (Jan
>>        Beulich, DONE for both qemu-xen-upstream and
>>        qemu-xen-traditional)
>>
>>      * [CHECK] Confirm that migration from Xen 4.1 ->  4.2 works.
>>
>>      * [BUG] LIBLEAFDIR et al and libfsimage do the wrong thing on
>>        modern Debian/Ubuntu w/ multiarch capabilities (Matt Wilson,
>>        patch posted)
>>
>>      * libxl to reject attempts to migrate a domain using upstream
>>        qemu, due to lack of log dirty support (Ian C, patch sent,
>>        reviewed, awaiting repost)
>>
>> hypervisor, nice to have:
>>
>>      * PoD performance improvements (George Dunlap, DONE)
>>
>>      * vMCE save/restore changes, to simplify migration 4.2->4.3
>>       (Jinsong Liu)
>>
>> tools, nice to have:
>>
>>      * xl compatibility with xm:
>>
>>          * Accept "xl cr /dev/null param=value" to provide all config
>>            on the command line (W. Michael Petullo, patch posted)
>>
>>      * libxl stable API
>>
>>          * libxl_wait_for_free_memory/libxl_wait_for_memory_target.
>>            Interface needs an overhaul, related to
>>            locking/serialization over domain create. IanJ to add note
>>            about this interface being substandard but otherwise defer
>>            to 4.3.
>>
>>          * Interfaces which may need to be async:
>>
>>              * libxl_cdrom_insert. Should be easy once
>>                disk_{add,remove} done. This is basically a helper
>>                function and its functionality can be implemented in
>>                terms of the libxl_disk_* interfaces. If this is not
>>                done in time we should document as a substandard
>>                interface which is subject to change post 4.2.
>>
>>      * xl.cfg(5) documentation patch for qemu-upstream
>>        videoram/videomem support:
>>        http://lists.xen.org/archives/html/xen-devel/2012-05/msg00250.html
>>        qemu-upstream doesn't support specifying videomem size for the
>>        HVM guest cirrus/stdvga.  (but this works with
>>        qemu-xen-traditional). (Pasi Kärkkäinen)
>>
>>      * [BUG] long stop during the guest boot process with qcow image,
>>        reported by Intel: http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1821
>>
>>      * [BUG] vcpu-set doesn't take effect on guest, reported by Intel:
>>        http://bugzilla.xen.org/bugzilla/show_bug.cgi?id=1822
>>
>>      * Load blktap driver from xencommons initscript if available, thread at:
>>        <db614e92faf743e20b3f.1337096977@kodo2>. To be fixed more
>>        properly in 4.3. (Patch posted, discussion, plan to take simple
>>        xencommons patch for 4.2 and revist for 4.3)
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel