* Re: QEMU 2.2.0 in Xen 4.6
@ 2015-01-22 12:10 Eric Shelton
2015-01-22 14:38 ` Stefano Stabellini
0 siblings, 1 reply; 11+ messages in thread
From: Eric Shelton @ 2015-01-22 12:10 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: Anthony PERARD, Don Slutz, xen-devel
On Fri, 16 Jan 2015, Stefano Stabellini wrote:
> On Thu, 15 Jan 2015, Don Slutz wrote:
> > Now that Xen 4.5 has been released, I would like to see at least QEMU
> > 2.2.0 in qemu-upstream-stable on the master branch.
>
> Agreed
Is there any concrete plan regarding if and how stub domains will be
implemented for qemu-upstream in 4.6? If not, would you entertain the
idea of someone continuing Anthony's earlier work with a Linux-based
stub domain?
Eric
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: QEMU 2.2.0 in Xen 4.6 2015-01-22 12:10 QEMU 2.2.0 in Xen 4.6 Eric Shelton @ 2015-01-22 14:38 ` Stefano Stabellini [not found] ` <CAPQw5r=MHVW_Suy49dn3ckkjBVSYOHOxsXtdTAU+uxfhvapr1g@mail.gmail.com> 0 siblings, 1 reply; 11+ messages in thread From: Stefano Stabellini @ 2015-01-22 14:38 UTC (permalink / raw) To: Eric Shelton; +Cc: Anthony PERARD, xen-devel, Don Slutz, Stefano Stabellini On Thu, 22 Jan 2015, Eric Shelton wrote: > On Fri, 16 Jan 2015, Stefano Stabellini wrote: > > On Thu, 15 Jan 2015, Don Slutz wrote: > > > Now that Xen 4.5 has been released, I would like to see at least QEMU > > > 2.2.0 in qemu-upstream-stable on the master branch. > > > > Agreed > > Is there any concrete plan regarding if and how stub domains will be > implemented for qemu-upstream in 4.6? If not, would you entertain the > idea of someone continuing Anthony's earlier work with a Linux-based > stub domain? I think it would be fantastic if you could pick up Anthony's work on Linux based stubdoms and complete it! I would really like to have upstream QEMU based stubdoms in Xen 4.6! FYI there have been some efforts toward getting upstream QEMU building on rumpkernel (http://wiki.netbsd.org/rumpkernel/), but the work isn't completed, it is difficult to estimate when it is going to be finished and it won't make 4.6. So I think that Linux-based stubdoms would be a great start. ^ permalink raw reply [flat|nested] 11+ messages in thread
[parent not found: <CAPQw5r=MHVW_Suy49dn3ckkjBVSYOHOxsXtdTAU+uxfhvapr1g@mail.gmail.com>]
* Re: QEMU 2.2.0 in Xen 4.6 [not found] ` <CAPQw5r=MHVW_Suy49dn3ckkjBVSYOHOxsXtdTAU+uxfhvapr1g@mail.gmail.com> @ 2015-01-23 14:42 ` Stefano Stabellini 2015-01-23 14:55 ` Ian Campbell 0 siblings, 1 reply; 11+ messages in thread From: Stefano Stabellini @ 2015-01-23 14:42 UTC (permalink / raw) To: Eric Shelton; +Cc: xen-devel, Stefano Stabellini [-- Attachment #1: Type: text/plain, Size: 4631 bytes --] On Thu, 22 Jan 2015, Eric Shelton wrote: > On Jan 22, 2015 9:39 AM, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com> wrote: > > > > On Thu, 22 Jan 2015, Eric Shelton wrote: > > > On Fri, 16 Jan 2015, Stefano Stabellini wrote: > > > > On Thu, 15 Jan 2015, Don Slutz wrote: > > > > > Now that Xen 4.5 has been released, I would like to see at least QEMU > > > > > 2.2.0 in qemu-upstream-stable on the master branch. > > > > > > > > Agreed > > > > > > Is there any concrete plan regarding if and how stub domains will be > > > implemented for qemu-upstream in 4.6? If not, would you entertain the > > > idea of someone continuing Anthony's earlier work with a Linux-based > > > stub domain? > > > > I think it would be fantastic if you could pick up Anthony's work on > > Linux based stubdoms and complete it! I would really like to have > > upstream QEMU based stubdoms in Xen 4.6! > > > > FYI there have been some efforts toward getting upstream QEMU building > > on rumpkernel (http://wiki.netbsd.org/rumpkernel/), but the work isn't > > completed, it is difficult to estimate when it is going to be finished > > and it won't make 4.6. So I think that Linux-based stubdoms would be a > > great start. > > Stefano, > > Just a brief question, that I did not think needed to go via xen-devel I think that it is a very reasonable question and might be useful to others, so I am CC'ing xen-devel back. > There are several components interacting with each other in the current qemu stubdom: > - dom0 qemu process > - stub domain with MiniOS > - qemu device model running under MiniOS > - hvm domain with guest OS > > Is there something, or would you mind taking a minute to write up something, that briefly describes this architecture, and how, > for example, the two qemu instances cooperate and the two domains must cooperate with each other? I can go about figuring this > out, but a brief primer would speed things up. Sure. Like you wrote, device emulation is done by QEMU running on top of MiniOS within the stubdom. The stubdom itself is a PV guest and as such it connects to PV backends in Dom0 to get access to disk, network and other PV protocols. But in the stubdom case, they are the disk and network of the guest. The stubdom doesn't actually have disk or network for itself. (However it has a couple of other PV interfaces for special purposes, I'll talk about them later.) As usual the PV backends might be running in the Linux kernel or in userspace in QEMU. Therefore you might have a second QEMU instance running in Dom0 to provide the PV backends. HVM guest -> QEMU Stubdom (device emulation) -> QEMU in Dom0 and Linux (PV backends for stubdom) For example when the HVM guest tries to read from the disk via the emulated IDE interface, it will trap in Xen and the IO request is going to be forwarded to QEMU running in the stubdom. QEMU will do the emulation and eventually will issue a PV read request via MiniOS to the PV block backend to read from the guest disk. If the PV block backend is in QEMU, for example because the guest disk is a QCOW2 file, then the read request will reach the QEMU instance running in Dom0. In the Linux stubdom case, QEMU running in the stubdom would simply use the POSIX read syscall to read from the guest disk as usual, then the Linux kernel in the stubdom would issue the PV block read request to the backend via xen-blkfront. From QEMU's point of view there should be no differences between running on a Linux stubdom or in Dom0. Similarly it is business as usual for the Linux kernel: it is running just as a regular PV guest. In addition the stubdom is provided with 3 "special" PV console interfaces, the first one is used for logging the stubdom's own output, the second one is used to save the state of emulated devices and the third one if used to read the state of emulated devices. The last two are only used for save/restore purposes. See tools/libxl/libxl_dm.c:spawn_stub_launch_dm. To complicate matter further, the HVM guest could be running PV frontends of its own. For example the Windows PV Drivers. They would be connecting to a separate set of PV backends in Dom0. Again if one or more PV backends are running in QEMU, there is going to a third QEMU instance to provide PV backends to the guest. HVM guest ---------(PV)----------> QEMU in Dom0 for guest | --(emulation)--> QEMU Stubdom-(syscall)->Linux Stubdom---(PV)--> QEMU Dom0 for stubdom I hope I managed to make things clearer for you. Cheers, Stefano [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: QEMU 2.2.0 in Xen 4.6 2015-01-23 14:42 ` Stefano Stabellini @ 2015-01-23 14:55 ` Ian Campbell 2015-01-23 15:00 ` Stefano Stabellini 0 siblings, 1 reply; 11+ messages in thread From: Ian Campbell @ 2015-01-23 14:55 UTC (permalink / raw) To: Stefano Stabellini; +Cc: xen-devel, Eric Shelton On Fri, 2015-01-23 at 14:42 +0000, Stefano Stabellini wrote: > HVM guest ---------(PV)----------> QEMU in Dom0 for guest > | > --(emulation)--> QEMU Stubdom-(syscall)->Linux Stubdom---(PV)--> QEMU Dom0 for stubdom Here, and throughout what you said I think, "QEMU in Dom0 for guest" could equally well be e.g. "blkback in driver domain for guest" likewise the "... for stubdom" too. i.e. the PV backend for the stubdom or the guest doesn't necessarily need to be QEMU and doesn't necessarily need to be in dom0. Ian. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: QEMU 2.2.0 in Xen 4.6 2015-01-23 14:55 ` Ian Campbell @ 2015-01-23 15:00 ` Stefano Stabellini 2015-01-23 16:37 ` Eric Shelton 0 siblings, 1 reply; 11+ messages in thread From: Stefano Stabellini @ 2015-01-23 15:00 UTC (permalink / raw) To: Ian Campbell; +Cc: Eric Shelton, xen-devel, Stefano Stabellini On Fri, 23 Jan 2015, Ian Campbell wrote: > On Fri, 2015-01-23 at 14:42 +0000, Stefano Stabellini wrote: > > > HVM guest ---------(PV)----------> QEMU in Dom0 for guest > > | > > --(emulation)--> QEMU Stubdom-(syscall)->Linux Stubdom---(PV)--> QEMU Dom0 for stubdom > > Here, and throughout what you said I think, "QEMU in Dom0 for guest" > could equally well be e.g. "blkback in driver domain for guest" likewise > the "... for stubdom" too. > > i.e. the PV backend for the stubdom or the guest doesn't necessarily > need to be QEMU and doesn't necessarily need to be in dom0. Indeed ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: QEMU 2.2.0 in Xen 4.6 2015-01-23 15:00 ` Stefano Stabellini @ 2015-01-23 16:37 ` Eric Shelton 2015-01-26 14:37 ` Stefano Stabellini 0 siblings, 1 reply; 11+ messages in thread From: Eric Shelton @ 2015-01-23 16:37 UTC (permalink / raw) To: Stefano Stabellini; +Cc: xen-devel, Ian Campbell [-- Attachment #1.1: Type: text/plain, Size: 1136 bytes --] On Jan 23, 2015 10:10 AM, "Stefano Stabellini" < stefano.stabellini@eu.citrix.com> wrote: > > On Fri, 23 Jan 2015, Ian Campbell wrote: > > On Fri, 2015-01-23 at 14:42 +0000, Stefano Stabellini wrote: > > > > > HVM guest ---------(PV)----------> QEMU in Dom0 for guest > > > | > > > --(emulation)--> QEMU Stubdom-(syscall)->Linux Stubdom---(PV)--> QEMU Dom0 for stubdom > > > > Here, and throughout what you said I think, "QEMU in Dom0 for guest" > > could equally well be e.g. "blkback in driver domain for guest" likewise > > the "... for stubdom" too. > > > > i.e. the PV backend for the stubdom or the guest doesn't necessarily > > need to be QEMU and doesn't necessarily need to be in dom0. > > Indeed > Thank you both. There is one other thing that would be helpful to understand. Anthony had to patch the Linux kernel running in the stubdom to allow memory mapping. What mappings are needed between dom0 and the stub domain, and between the stub domain and the HVM guest domain? I am guessing this comes into play for the display, as it seems that the dom0 qemu instance is running the VNC server. Thanks again, Eric [-- Attachment #1.2: Type: text/html, Size: 1511 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: QEMU 2.2.0 in Xen 4.6 2015-01-23 16:37 ` Eric Shelton @ 2015-01-26 14:37 ` Stefano Stabellini 2015-01-26 23:02 ` Eric Shelton 0 siblings, 1 reply; 11+ messages in thread From: Stefano Stabellini @ 2015-01-26 14:37 UTC (permalink / raw) To: Eric Shelton; +Cc: Anthony Perard, xen-devel, Ian Campbell, Stefano Stabellini [-- Attachment #1: Type: text/plain, Size: 1331 bytes --] On Fri, 23 Jan 2015, Eric Shelton wrote: > On Jan 23, 2015 10:10 AM, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com> wrote: > > > > On Fri, 23 Jan 2015, Ian Campbell wrote: > > > On Fri, 2015-01-23 at 14:42 +0000, Stefano Stabellini wrote: > > > > > > > HVM guest ---------(PV)----------> QEMU in Dom0 for guest > > > > | > > > > --(emulation)--> QEMU Stubdom-(syscall)->Linux Stubdom---(PV)--> QEMU Dom0 for stubdom > > > > > > Here, and throughout what you said I think, "QEMU in Dom0 for guest" > > > could equally well be e.g. "blkback in driver domain for guest" likewise > > > the "... for stubdom" too. > > > > > > i.e. the PV backend for the stubdom or the guest doesn't necessarily > > > need to be QEMU and doesn't necessarily need to be in dom0. > > > > Indeed > > > > Thank you both. > > There is one other thing that would be helpful to understand. Anthony had to patch the Linux kernel running in the stubdom to > allow memory mapping. What mappings are needed between dom0 and the stub domain, and between the stub domain and the HVM guest > domain? I am guessing this comes into play for the display, as it seems that the dom0 qemu instance is running the VNC server. I don't remember the details of it anymore. CC'ing Anthony that might have a better idea. [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: QEMU 2.2.0 in Xen 4.6 2015-01-26 14:37 ` Stefano Stabellini @ 2015-01-26 23:02 ` Eric Shelton 2015-01-27 10:47 ` Stefano Stabellini 0 siblings, 1 reply; 11+ messages in thread From: Eric Shelton @ 2015-01-26 23:02 UTC (permalink / raw) To: Stefano Stabellini; +Cc: Anthony Perard, xen-devel, Ian Campbell On Mon, Jan 26, 2015 at 9:37 AM, Stefano Stabellini <stefano.stabellini@eu.citrix.com> wrote: > On Fri, 23 Jan 2015, Eric Shelton wrote: >> On Jan 23, 2015 10:10 AM, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com> wrote: >> > >> > On Fri, 23 Jan 2015, Ian Campbell wrote: >> > > On Fri, 2015-01-23 at 14:42 +0000, Stefano Stabellini wrote: >> > > >> > > > HVM guest ---------(PV)----------> QEMU in Dom0 for guest >> > > > | >> > > > --(emulation)--> QEMU Stubdom-(syscall)->Linux Stubdom---(PV)--> QEMU Dom0 for stubdom >> > > >> > > Here, and throughout what you said I think, "QEMU in Dom0 for guest" >> > > could equally well be e.g. "blkback in driver domain for guest" likewise >> > > the "... for stubdom" too. >> > > >> > > i.e. the PV backend for the stubdom or the guest doesn't necessarily >> > > need to be QEMU and doesn't necessarily need to be in dom0. >> > >> > Indeed >> > >> >> Thank you both. >> >> There is one other thing that would be helpful to understand. Anthony had to patch the Linux kernel running in the stubdom to >> allow memory mapping. What mappings are needed between dom0 and the stub domain, and between the stub domain and the HVM guest >> domain? I am guessing this comes into play for the display, as it seems that the dom0 qemu instance is running the VNC server. > > I don't remember the details of it anymore. CC'ing Anthony that might > have a better idea. Actually, it looks like what I need to understand is how the startup of (1) QEMU in Dom0, (2) QEMU in stubdom, and (3) the HVM domain are synchronized (or supposed to be synchronized). I assume the desired order is (2) -> (1) -> (3). It looks like /local/domain/0/device-model/{hvm-domid}/state is used to notify xl that QEMU in Dom0 is running, which prompts xl to unpause the HVM domain, making sure that (1) occurs before (3). The problem I am running into is that, at least with QEMU 2.0, nothing ensures that QEMU in stubdom is up and running before unpausing the HVM domain. This causes hvmloader to fail, as (3) occurs before (2), and there is no device model in place yet. So, what mechanism is being used with qemu-traditional to ensure QEMU in stubdom is running before the HVM domain is unpaused? Thanks, Eric ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: QEMU 2.2.0 in Xen 4.6 2015-01-26 23:02 ` Eric Shelton @ 2015-01-27 10:47 ` Stefano Stabellini 0 siblings, 0 replies; 11+ messages in thread From: Stefano Stabellini @ 2015-01-27 10:47 UTC (permalink / raw) To: Eric Shelton; +Cc: Anthony Perard, xen-devel, Ian Campbell, Stefano Stabellini On Mon, 26 Jan 2015, Eric Shelton wrote: > On Mon, Jan 26, 2015 at 9:37 AM, Stefano Stabellini > <stefano.stabellini@eu.citrix.com> wrote: > > On Fri, 23 Jan 2015, Eric Shelton wrote: > >> On Jan 23, 2015 10:10 AM, "Stefano Stabellini" <stefano.stabellini@eu.citrix.com> wrote: > >> > > >> > On Fri, 23 Jan 2015, Ian Campbell wrote: > >> > > On Fri, 2015-01-23 at 14:42 +0000, Stefano Stabellini wrote: > >> > > > >> > > > HVM guest ---------(PV)----------> QEMU in Dom0 for guest > >> > > > | > >> > > > --(emulation)--> QEMU Stubdom-(syscall)->Linux Stubdom---(PV)--> QEMU Dom0 for stubdom > >> > > > >> > > Here, and throughout what you said I think, "QEMU in Dom0 for guest" > >> > > could equally well be e.g. "blkback in driver domain for guest" likewise > >> > > the "... for stubdom" too. > >> > > > >> > > i.e. the PV backend for the stubdom or the guest doesn't necessarily > >> > > need to be QEMU and doesn't necessarily need to be in dom0. > >> > > >> > Indeed > >> > > >> > >> Thank you both. > >> > >> There is one other thing that would be helpful to understand. Anthony had to patch the Linux kernel running in the stubdom to > >> allow memory mapping. What mappings are needed between dom0 and the stub domain, and between the stub domain and the HVM guest > >> domain? I am guessing this comes into play for the display, as it seems that the dom0 qemu instance is running the VNC server. > > > > I don't remember the details of it anymore. CC'ing Anthony that might > > have a better idea. > > Actually, it looks like what I need to understand is how the startup > of (1) QEMU in Dom0, (2) QEMU in stubdom, and (3) the HVM domain are > synchronized (or supposed to be synchronized). I assume the desired > order is (2) -> (1) -> (3). We have two QEMUs in Dom0, one to provide backends to the stubdoms and one to provide backends to the HVM guest. Which one is (1)? If (1) is the QEMU providing the backends for the stubdomain, then you want (1)->(2)->(3) > It looks like > /local/domain/0/device-model/{hvm-domid}/state is used to notify xl > that QEMU in Dom0 is running, which prompts xl to unpause the HVM > domain, making sure that (1) occurs before (3). The problem I am > running into is that, at least with QEMU 2.0, nothing ensures that > QEMU in stubdom is up and running before unpausing the HVM domain. > This causes hvmloader to fail, as (3) occurs before (2), and there is > no device model in place yet. > > So, what mechanism is being used with qemu-traditional to ensure QEMU > in stubdom is running before the HVM domain is unpaused? Actually I know that is confusing but I am pretty sure that /local/domain/0/device-model/{hvm-domid}/state is to notify that QEMU in the stubdom is up and running. The QEMU writing to that path, is the one running in the stubdom. I think that we are probably missing a comparable mechanism for upstream QEMU. The toolstack (xl and libxl) talks to qemu-traditional via xenstore, and talks to upstream QEMU via QMP over a unix socket. I think that you need to get QMP working in a stubdom first, maybe by using a PV console as transport for it. Afterward you could use QMP to check the state of QEMU and pause and unpause the domain. ^ permalink raw reply [flat|nested] 11+ messages in thread
* QEMU 2.2.0 in Xen 4.6 @ 2015-01-15 19:41 Don Slutz 2015-01-16 10:30 ` Stefano Stabellini 0 siblings, 1 reply; 11+ messages in thread From: Don Slutz @ 2015-01-15 19:41 UTC (permalink / raw) To: xen-devel; +Cc: anthony PERARD, Stefano Stabellini Now that Xen 4.5 has been released, I would like to see at least QEMU 2.2.0 in qemu-upstream-stable on the master branch. -Don Slutz ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: QEMU 2.2.0 in Xen 4.6 2015-01-15 19:41 Don Slutz @ 2015-01-16 10:30 ` Stefano Stabellini 0 siblings, 0 replies; 11+ messages in thread From: Stefano Stabellini @ 2015-01-16 10:30 UTC (permalink / raw) To: Don Slutz; +Cc: anthony PERARD, Stefano Stabellini, xen-devel On Thu, 15 Jan 2015, Don Slutz wrote: > Now that Xen 4.5 has been released, I would like to see at least QEMU > 2.2.0 in qemu-upstream-stable on the master branch. Agreed ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2015-01-27 10:47 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-01-22 12:10 QEMU 2.2.0 in Xen 4.6 Eric Shelton 2015-01-22 14:38 ` Stefano Stabellini [not found] ` <CAPQw5r=MHVW_Suy49dn3ckkjBVSYOHOxsXtdTAU+uxfhvapr1g@mail.gmail.com> 2015-01-23 14:42 ` Stefano Stabellini 2015-01-23 14:55 ` Ian Campbell 2015-01-23 15:00 ` Stefano Stabellini 2015-01-23 16:37 ` Eric Shelton 2015-01-26 14:37 ` Stefano Stabellini 2015-01-26 23:02 ` Eric Shelton 2015-01-27 10:47 ` Stefano Stabellini -- strict thread matches above, loose matches on Subject: below -- 2015-01-15 19:41 Don Slutz 2015-01-16 10:30 ` Stefano Stabellini
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.