qemu device model question

qemu device model question

[Juergen Gross]

Hi,

while trying to build a pvusb backend in qemu I think I've found a
general issue in xl: qemu for pv-domains is started only at domain
creation and only if there is at least one backend in qemu required.

If there is no qemu process started for the domain at creation time
it will be impossible to successfully add such a device later while
the domain is running.

Are there any plans to remove that restriction? Or have I missed
some mechanism in xl to start qemu at a later time?

Related to this problem is the question whether it would be desirable
to have e.g. qdisk and pvusb backends in the same process possibly
influencing each others throughput.

Any thoughts?


Juergen

Re: qemu device model question

[Ian Campbell]

On Wed, 2015-05-13 at 10:54 +0200, Juergen Gross wrote:
> Hi,
> 
> while trying to build a pvusb backend in qemu I think I've found a
> general issue in xl: qemu for pv-domains is started only at domain
> creation and only if there is at least one backend in qemu required.
> 
> If there is no qemu process started for the domain at creation time
> it will be impossible to successfully add such a device later while
> the domain is running.
> 
> Are there any plans to remove that restriction? Or have I missed
> some mechanism in xl to start qemu at a later time?

I think it would be reasonable to have some way to indicate that pvusb
support is desired even if there are no such devices on boot, and for
libxl to start the necessary backend in that case.

s/pvusb/whatever/

> Related to this problem is the question whether it would be desirable
> to have e.g. qdisk and pvusb backends in the same process possibly
> influencing each others throughput.

Dissagregation of qemu's PV functionality would be nice, but I'm not
sure how easy/hard it would be.

Ian.

Re: qemu device model question

[Wei Liu]

On Wed, May 13, 2015 at 10:30:40AM +0100, Ian Campbell wrote:
> On Wed, 2015-05-13 at 10:54 +0200, Juergen Gross wrote:
> > Hi,
> > 
> > while trying to build a pvusb backend in qemu I think I've found a
> > general issue in xl: qemu for pv-domains is started only at domain
> > creation and only if there is at least one backend in qemu required.
> > 
> > If there is no qemu process started for the domain at creation time
> > it will be impossible to successfully add such a device later while
> > the domain is running.
> > 
> > Are there any plans to remove that restriction? Or have I missed
> > some mechanism in xl to start qemu at a later time?
> 
> I think it would be reasonable to have some way to indicate that pvusb
> support is desired even if there are no such devices on boot, and for
> libxl to start the necessary backend in that case.
> 
> s/pvusb/whatever/

libxl__need_xenpv_qemu comes to mind.

Also QEMU has a mode called attach mode. Never use it personally
though.

> 
> > Related to this problem is the question whether it would be desirable
> > to have e.g. qdisk and pvusb backends in the same process possibly
> > influencing each others throughput.
> 
> Dissagregation of qemu's PV functionality would be nice, but I'm not
> sure how easy/hard it would be.
> 

At least we would need to invent a new xenstore protocol to support
multiple emulator.

Wei.

> Ian.
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: qemu device model question

[Ian Campbell]

On Wed, 2015-05-13 at 14:06 +0100, Wei Liu wrote:
> On Wed, May 13, 2015 at 10:30:40AM +0100, Ian Campbell wrote:
> > On Wed, 2015-05-13 at 10:54 +0200, Juergen Gross wrote:
> > > Hi,
> > > 
> > > while trying to build a pvusb backend in qemu I think I've found a
> > > general issue in xl: qemu for pv-domains is started only at domain
> > > creation and only if there is at least one backend in qemu required.
> > > 
> > > If there is no qemu process started for the domain at creation time
> > > it will be impossible to successfully add such a device later while
> > > the domain is running.
> > > 
> > > Are there any plans to remove that restriction? Or have I missed
> > > some mechanism in xl to start qemu at a later time?
> > 
> > I think it would be reasonable to have some way to indicate that pvusb
> > support is desired even if there are no such devices on boot, and for
> > libxl to start the necessary backend in that case.
> > 
> > s/pvusb/whatever/
> 
> libxl__need_xenpv_qemu comes to mind.
> 
> Also QEMU has a mode called attach mode. Never use it personally
> though.
> 
> > 
> > > Related to this problem is the question whether it would be desirable
> > > to have e.g. qdisk and pvusb backends in the same process possibly
> > > influencing each others throughput.
> > 
> > Dissagregation of qemu's PV functionality would be nice, but I'm not
> > sure how easy/hard it would be.
> > 
> 
> At least we would need to invent a new xenstore protocol to support
> multiple emulator.

I didn't think we were talking about emulators, just pv backend (so
qdisk, qpvusb etc), those all have existing xenstore protocols based on
the backend dirs, all which would be needed would be a way to tell qemu
which backends a given process should take care of.

Ian.

Re: qemu device model question

[Juergen Gross]

On 05/13/2015 03:06 PM, Wei Liu wrote:
> On Wed, May 13, 2015 at 10:30:40AM +0100, Ian Campbell wrote:
>> On Wed, 2015-05-13 at 10:54 +0200, Juergen Gross wrote:
>>> Hi,
>>>
>>> while trying to build a pvusb backend in qemu I think I've found a
>>> general issue in xl: qemu for pv-domains is started only at domain
>>> creation and only if there is at least one backend in qemu required.
>>>
>>> If there is no qemu process started for the domain at creation time
>>> it will be impossible to successfully add such a device later while
>>> the domain is running.
>>>
>>> Are there any plans to remove that restriction? Or have I missed
>>> some mechanism in xl to start qemu at a later time?
>>
>> I think it would be reasonable to have some way to indicate that pvusb
>> support is desired even if there are no such devices on boot, and for
>> libxl to start the necessary backend in that case.
>>
>> s/pvusb/whatever/
>
> libxl__need_xenpv_qemu comes to mind.

Indeed.

I think it would be easy to add a configuration item in the domain's
config file to start qemu unconditionally. Another add-on could be an
item in xl.conf to configure the default behaviour.

> Also QEMU has a mode called attach mode. Never use it personally
> though.

A quick search of the qemu docs didn't show anything. What is it and how
could it help?

>
>>
>>> Related to this problem is the question whether it would be desirable
>>> to have e.g. qdisk and pvusb backends in the same process possibly
>>> influencing each others throughput.
>>
>> Dissagregation of qemu's PV functionality would be nice, but I'm not
>> sure how easy/hard it would be.
>>
>
> At least we would need to invent a new xenstore protocol to support
> multiple emulator.

We could either specify the qemu pid for each backend in xenstore or we
could start qemu with a parameter indicating which backend it should
provide.

In case we are doing something like this we should construct an
interface capable of splitting up even per device, in case we need more
than one process e.g. for the qdisks of one domain.


Juergen

Re: qemu device model question

[Wei Liu]

On Wed, May 13, 2015 at 02:26:33PM +0100, Ian Campbell wrote:
> On Wed, 2015-05-13 at 14:06 +0100, Wei Liu wrote:
> > On Wed, May 13, 2015 at 10:30:40AM +0100, Ian Campbell wrote:
> > > On Wed, 2015-05-13 at 10:54 +0200, Juergen Gross wrote:
> > > > Hi,
> > > > 
> > > > while trying to build a pvusb backend in qemu I think I've found a
> > > > general issue in xl: qemu for pv-domains is started only at domain
> > > > creation and only if there is at least one backend in qemu required.
> > > > 
> > > > If there is no qemu process started for the domain at creation time
> > > > it will be impossible to successfully add such a device later while
> > > > the domain is running.
> > > > 
> > > > Are there any plans to remove that restriction? Or have I missed
> > > > some mechanism in xl to start qemu at a later time?
> > > 
> > > I think it would be reasonable to have some way to indicate that pvusb
> > > support is desired even if there are no such devices on boot, and for
> > > libxl to start the necessary backend in that case.
> > > 
> > > s/pvusb/whatever/
> > 
> > libxl__need_xenpv_qemu comes to mind.
> > 
> > Also QEMU has a mode called attach mode. Never use it personally
> > though.
> > 
> > > 
> > > > Related to this problem is the question whether it would be desirable
> > > > to have e.g. qdisk and pvusb backends in the same process possibly
> > > > influencing each others throughput.
> > > 
> > > Dissagregation of qemu's PV functionality would be nice, but I'm not
> > > sure how easy/hard it would be.
> > > 
> > 
> > At least we would need to invent a new xenstore protocol to support
> > multiple emulator.
> 
> I didn't think we were talking about emulators, just pv backend (so
> qdisk, qpvusb etc), those all have existing xenstore protocols based on
> the backend dirs, all which would be needed would be a way to tell qemu
> which backends a given process should take care of.
> 

Yes, you're right.

Wei.

> Ian.
> 

Re: qemu device model question

[Wei Liu]

On Wed, May 13, 2015 at 04:44:03PM +0200, Juergen Gross wrote:
> On 05/13/2015 03:06 PM, Wei Liu wrote:
> >On Wed, May 13, 2015 at 10:30:40AM +0100, Ian Campbell wrote:
> >>On Wed, 2015-05-13 at 10:54 +0200, Juergen Gross wrote:
> >>>Hi,
> >>>
> >>>while trying to build a pvusb backend in qemu I think I've found a
> >>>general issue in xl: qemu for pv-domains is started only at domain
> >>>creation and only if there is at least one backend in qemu required.
> >>>
> >>>If there is no qemu process started for the domain at creation time
> >>>it will be impossible to successfully add such a device later while
> >>>the domain is running.
> >>>
> >>>Are there any plans to remove that restriction? Or have I missed
> >>>some mechanism in xl to start qemu at a later time?
> >>
> >>I think it would be reasonable to have some way to indicate that pvusb
> >>support is desired even if there are no such devices on boot, and for
> >>libxl to start the necessary backend in that case.
> >>
> >>s/pvusb/whatever/
> >
> >libxl__need_xenpv_qemu comes to mind.
> 
> Indeed.
> 
> I think it would be easy to add a configuration item in the domain's
> config file to start qemu unconditionally. Another add-on could be an
> item in xl.conf to configure the default behaviour.
> 
> >Also QEMU has a mode called attach mode. Never use it personally
> >though.
> 
> A quick search of the qemu docs didn't show anything. What is it and how
> could it help?
> 

All I know is there is a QEMU option called -xen-attach. QEMU will
attach to an existing domain. Again, I never use that personally so I'm
not sure if it does what you need or the appropriate infrastructure
to start QEMU in that mode is in place. I just mentioned it for
completeness.

Wei.

> >
> >>
> >>>Related to this problem is the question whether it would be desirable
> >>>to have e.g. qdisk and pvusb backends in the same process possibly
> >>>influencing each others throughput.
> >>
> >>Dissagregation of qemu's PV functionality would be nice, but I'm not
> >>sure how easy/hard it would be.
> >>
> >
> >At least we would need to invent a new xenstore protocol to support
> >multiple emulator.
> 
> We could either specify the qemu pid for each backend in xenstore or we
> could start qemu with a parameter indicating which backend it should
> provide.
> 
> In case we are doing something like this we should construct an
> interface capable of splitting up even per device, in case we need more
> than one process e.g. for the qdisks of one domain.
> 
> 
> Juergen

Re: qemu device model question

[George Dunlap]

On Wed, May 13, 2015 at 10:30 AM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Wed, 2015-05-13 at 10:54 +0200, Juergen Gross wrote:
>> Hi,
>>
>> while trying to build a pvusb backend in qemu I think I've found a
>> general issue in xl: qemu for pv-domains is started only at domain
>> creation and only if there is at least one backend in qemu required.
>>
>> If there is no qemu process started for the domain at creation time
>> it will be impossible to successfully add such a device later while
>> the domain is running.
>>
>> Are there any plans to remove that restriction? Or have I missed
>> some mechanism in xl to start qemu at a later time?
>
> I think it would be reasonable to have some way to indicate that pvusb
> support is desired even if there are no such devices on boot, and for
> libxl to start the necessary backend in that case.
>
> s/pvusb/whatever/

In the pvusb case, I think it's the plan to have an option to specify
a pvusb *bus*, even with no devices attached.

But we should probably solve the more general problem. :-)  We should
have a way to plug in disks / other devices supplied by qemu after the
VM is already running.

Specifying an "I might want something later" option is suboptimal from
a UI point of view: if the admin forgets to add it when it's needed,
she'll have to reboot the VM; if she forgets to remove it when it's
not needed, she'll have a useless process lying around, potentially
increasing the surface of attack unnecessarily.  Starting a qdisk
(qemu-pv?) process on-demand would automatically give you the most
efficient option without additional UI baggage.

Is there really no way to start up a qdisk process after the domain is
created?  The qdisk process doesn't actually need to do any emulation,
after all -- it's just acting as a backend, right?

 -George

Re: qemu device model question

[Ian Campbell]

On Mon, 2015-05-18 at 10:49 +0100, George Dunlap wrote:
> Is there really no way to start up a qdisk process after the domain is
> created?  The qdisk process doesn't actually need to do any emulation,
> after all -- it's just acting as a backend, right?

FWIW xencommons starts a qemu for dom0 using:
$QEMU_XEN -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -daemonize \
                -monitor /dev/null -serial /dev/null -parallel /dev/null \
                -pidfile $QEMU_PIDFILE

I'd be surprised if this didn't also work for an arbitrary PV domain and
I assume it wouldn't be needed for an HVM one.

Ian.

Re: qemu device model question

[Stefano Stabellini]

On Mon, 18 May 2015, Ian Campbell wrote:
> On Mon, 2015-05-18 at 10:49 +0100, George Dunlap wrote:
> > Is there really no way to start up a qdisk process after the domain is
> > created?  The qdisk process doesn't actually need to do any emulation,
> > after all -- it's just acting as a backend, right?
> 
> FWIW xencommons starts a qemu for dom0 using:
> $QEMU_XEN -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -daemonize \
>                 -monitor /dev/null -serial /dev/null -parallel /dev/null \
>                 -pidfile $QEMU_PIDFILE
> 
> I'd be surprised if this didn't also work for an arbitrary PV domain and
> I assume it wouldn't be needed for an HVM one.

It doesn't work for HVM guests because the QEMU device model also runs a
qdisk backend instance. However it could work for PV/PVH guests.

FYI I am working on a set of patches to remove the handling of PV
backends from QEMU device model. it requires both QEMU and libxl
changes. Once they are in, you'll be able to do this for HVM guests too.

Re: qemu device model question

[George Dunlap]

On Tue, May 19, 2015 at 12:36 AM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Mon, 18 May 2015, Ian Campbell wrote:
>> On Mon, 2015-05-18 at 10:49 +0100, George Dunlap wrote:
>> > Is there really no way to start up a qdisk process after the domain is
>> > created?  The qdisk process doesn't actually need to do any emulation,
>> > after all -- it's just acting as a backend, right?
>>
>> FWIW xencommons starts a qemu for dom0 using:
>> $QEMU_XEN -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -daemonize \
>>                 -monitor /dev/null -serial /dev/null -parallel /dev/null \
>>                 -pidfile $QEMU_PIDFILE
>>
>> I'd be surprised if this didn't also work for an arbitrary PV domain and
>> I assume it wouldn't be needed for an HVM one.
>
> It doesn't work for HVM guests because the QEMU device model also runs a
> qdisk backend instance. However it could work for PV/PVH guests.

...but it wouldn't be needed because the toolstack can talk to the
existing DM and tell it to start providing new PV block devices or
pvusb busses.

The discussion here is between starting a "dummy" qemu at start of day
if there are no qdisk or pvusb, or starting a qemu on-demand as it's
needed.  There's obviously no such choice for an HVM domain at the
moment, as it always needs qemu for a device model.

> FYI I am working on a set of patches to remove the handling of PV
> backends from QEMU device model. it requires both QEMU and libxl
> changes. Once they are in, you'll be able to do this for HVM guests too.

If you do this, then we'll need to solve the same problem for HVM
domains as Juergen is trying to solve for PV domains. :-)

But it sounds like it shouldn't be too hard.

 -George