Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Andrew Jones]

On Tue, Jan 12, 2016 at 04:48:27PM +0100, Andrea Bolognani wrote:
> Would providing a QMP command that returns the list of GIC versions
> available on the current host for the current machine type be
> acceptable from QEMU's point of view?
>

Adding qemu-devel. Peter any opinion on this? If it sounds like a
reasonable idea then either I or Wei can look into sending the
patches.

Thanks,
drew 

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Peter Maydell]

On 19 January 2016 at 16:46, Andrew Jones <drjones@redhat.com> wrote:
> On Tue, Jan 12, 2016 at 04:48:27PM +0100, Andrea Bolognani wrote:
>> Would providing a QMP command that returns the list of GIC versions
>> available on the current host for the current machine type be
>> acceptable from QEMU's point of view?
>>
>
> Adding qemu-devel. Peter any opinion on this? If it sounds like a
> reasonable idea then either I or Wei can look into sending the
> patches.

I don't have a strong opinion, really. We should do whatever
works for libvirt and fits with whatever we do currently
(eg whatever we do at the moment for telling libvirt
"yes this QEMU supports KVM", if we're not using some ancient
silly way to do that).

thanks
-- PMM

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Laszlo Ersek]

On 01/19/16 17:53, Peter Maydell wrote:
> On 19 January 2016 at 16:46, Andrew Jones <drjones@redhat.com> wrote:
>> On Tue, Jan 12, 2016 at 04:48:27PM +0100, Andrea Bolognani wrote:
>>> Would providing a QMP command that returns the list of GIC versions
>>> available on the current host for the current machine type be
>>> acceptable from QEMU's point of view?
>>>
>>
>> Adding qemu-devel. Peter any opinion on this? If it sounds like a
>> reasonable idea then either I or Wei can look into sending the
>> patches.
> 
> I don't have a strong opinion, really. We should do whatever
> works for libvirt and fits with whatever we do currently
> (eg whatever we do at the moment for telling libvirt
> "yes this QEMU supports KVM", if we're not using some ancient
> silly way to do that).

Would this kind of info fit the capabilities list that libvirtd fetches
from QEMU at startup (as far as I recall)?

Thanks
Laszlo

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Daniel P. Berrange]

On Wed, Jan 06, 2016 at 01:30:16PM +0000, Peter Maydell wrote:
> On 6 January 2016 at 12:49, Andrea Bolognani <abologna@redhat.com> wrote:
> > That's correct, having a QMP command that lists the values gic-version
> > can have on the current host would be just great.
> >
> > If we had that, we could validate the GIC version chosen for a guest,
> > and expose it in the capabilities XML so that higher-level tools can
> > provide a list of choices to the user.
> >
> > Please note that this QMP command would have to work regardless of the
> > machine type selected on QEMU's command line, because libvirt always
> > runs a QEMU binary with '-M none' when probing its capabilities.
> 
> On the other hand, if you don't tell us the machine type you care
> about then we can't tell you:
>  (a) "this machine type doesn't support setting this property at all"
>      (which applies to machines like vexpress-a15 which you can use with
>      KVM on 32-bit hosts, and of course also to all the non-KVM models)

We have just recently merged support for registering properties against
classes instead of object instances. There is also a proposed command
to allow querying the list of properties against a class

  https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg04348.html

So if we now update the machine types to register their properties against
the class instead of object, then we can query what properties are present
against each machine type, while still using '-M none'.

>  (b) "this machine type only supports GIC versions X and Y even if the
>       host supports more" (this is currently only hypothetical, though,
>       since we only have the property on 'virt'. it would only happen
>       if in the future we needed something other than '2' or '3' or
>       'host' I think.)

Our introspection support in QOM only allows us to say that a property
is a particular type (int / enum / str / whatever). We don't have any
way to expose info about what subset of possible values for a type are
permitted. So I don't see any near term way to inform apps that the
gic property accepts values x, y and but not z.

IMHO, we shouldn't try to overthink this. Libvirt can query the host
to find out what GIC versions are supported and default to using the
most recent version, on the basis that people are likely to have a
matching QEMU. We can just rely on QEMU to report error if we pass
it a version it doesn't support and not try to pre-emptively check
before launch. The key is just getting the default right IMHO.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Christoffer Dall]

On Fri, Jan 22, 2016 at 02:44:32PM +0000, Daniel P. Berrange wrote:
> On Wed, Jan 06, 2016 at 01:30:16PM +0000, Peter Maydell wrote:
> > On 6 January 2016 at 12:49, Andrea Bolognani <abologna@redhat.com> wrote:
> > > That's correct, having a QMP command that lists the values gic-version
> > > can have on the current host would be just great.
> > >
> > > If we had that, we could validate the GIC version chosen for a guest,
> > > and expose it in the capabilities XML so that higher-level tools can
> > > provide a list of choices to the user.
> > >
> > > Please note that this QMP command would have to work regardless of the
> > > machine type selected on QEMU's command line, because libvirt always
> > > runs a QEMU binary with '-M none' when probing its capabilities.
> > 
> > On the other hand, if you don't tell us the machine type you care
> > about then we can't tell you:
> >  (a) "this machine type doesn't support setting this property at all"
> >      (which applies to machines like vexpress-a15 which you can use with
> >      KVM on 32-bit hosts, and of course also to all the non-KVM models)
> 
> We have just recently merged support for registering properties against
> classes instead of object instances. There is also a proposed command
> to allow querying the list of properties against a class
> 
>   https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg04348.html
> 
> So if we now update the machine types to register their properties against
> the class instead of object, then we can query what properties are present
> against each machine type, while still using '-M none'.
> 
> >  (b) "this machine type only supports GIC versions X and Y even if the
> >       host supports more" (this is currently only hypothetical, though,
> >       since we only have the property on 'virt'. it would only happen
> >       if in the future we needed something other than '2' or '3' or
> >       'host' I think.)
> 
> Our introspection support in QOM only allows us to say that a property
> is a particular type (int / enum / str / whatever). We don't have any
> way to expose info about what subset of possible values for a type are
> permitted. So I don't see any near term way to inform apps that the
> gic property accepts values x, y and but not z.
> 
> IMHO, we shouldn't try to overthink this. Libvirt can query the host
> to find out what GIC versions are supported and default to using the
> most recent version, on the basis that people are likely to have a
> matching QEMU. We can just rely on QEMU to report error if we pass
> it a version it doesn't support and not try to pre-emptively check
> before launch. The key is just getting the default right IMHO.
> 
This sounds fine to me.

However, not being familiar with the internals of libvirt I really can't
just what the right implementation approach here is.

As I understand we need to either:

  1) Add a QMP command that lets you ask for -M virt, if GIC version X
     is supported

  2) Just implement something in libvirt that checks what the kernel
     supports directly via the well-defined KVM interface and chooses
     the highest supported version per default.

To me it sounds like we should just go ahead with (2) and document
somehwere that if you get an error, you need to either manually change
the gic version setting in your VM definition file or upgrade QEMU.

Can someone with libvirt authority please advice whether (1) or (2) is
what we need to do?

Thanks,
-Christoffer

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Daniel P. Berrange]

On Tue, Feb 02, 2016 at 12:49:33PM +0100, Christoffer Dall wrote:
> On Fri, Jan 22, 2016 at 02:44:32PM +0000, Daniel P. Berrange wrote:
> > On Wed, Jan 06, 2016 at 01:30:16PM +0000, Peter Maydell wrote:
> > > On 6 January 2016 at 12:49, Andrea Bolognani <abologna@redhat.com> wrote:
> > > > That's correct, having a QMP command that lists the values gic-version
> > > > can have on the current host would be just great.
> > > >
> > > > If we had that, we could validate the GIC version chosen for a guest,
> > > > and expose it in the capabilities XML so that higher-level tools can
> > > > provide a list of choices to the user.
> > > >
> > > > Please note that this QMP command would have to work regardless of the
> > > > machine type selected on QEMU's command line, because libvirt always
> > > > runs a QEMU binary with '-M none' when probing its capabilities.
> > > 
> > > On the other hand, if you don't tell us the machine type you care
> > > about then we can't tell you:
> > >  (a) "this machine type doesn't support setting this property at all"
> > >      (which applies to machines like vexpress-a15 which you can use with
> > >      KVM on 32-bit hosts, and of course also to all the non-KVM models)
> > 
> > We have just recently merged support for registering properties against
> > classes instead of object instances. There is also a proposed command
> > to allow querying the list of properties against a class
> > 
> >   https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg04348.html
> > 
> > So if we now update the machine types to register their properties against
> > the class instead of object, then we can query what properties are present
> > against each machine type, while still using '-M none'.
> > 
> > >  (b) "this machine type only supports GIC versions X and Y even if the
> > >       host supports more" (this is currently only hypothetical, though,
> > >       since we only have the property on 'virt'. it would only happen
> > >       if in the future we needed something other than '2' or '3' or
> > >       'host' I think.)
> > 
> > Our introspection support in QOM only allows us to say that a property
> > is a particular type (int / enum / str / whatever). We don't have any
> > way to expose info about what subset of possible values for a type are
> > permitted. So I don't see any near term way to inform apps that the
> > gic property accepts values x, y and but not z.
> > 
> > IMHO, we shouldn't try to overthink this. Libvirt can query the host
> > to find out what GIC versions are supported and default to using the
> > most recent version, on the basis that people are likely to have a
> > matching QEMU. We can just rely on QEMU to report error if we pass
> > it a version it doesn't support and not try to pre-emptively check
> > before launch. The key is just getting the default right IMHO.
> > 
> This sounds fine to me.
> 
> However, not being familiar with the internals of libvirt I really can't
> just what the right implementation approach here is.
> 
> As I understand we need to either:
> 
>   1) Add a QMP command that lets you ask for -M virt, if GIC version X
>      is supported
> 
>   2) Just implement something in libvirt that checks what the kernel
>      supports directly via the well-defined KVM interface and chooses
>      the highest supported version per default.
> 
> To me it sounds like we should just go ahead with (2) and document
> somehwere that if you get an error, you need to either manually change
> the gic version setting in your VM definition file or upgrade QEMU.
> 
> Can someone with libvirt authority please advice whether (1) or (2) is
> what we need to do?

IMHO we should just go for 2.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Andrew Jones]

On Tue, Feb 02, 2016 at 12:10:10PM +0000, Daniel P. Berrange wrote:
> On Tue, Feb 02, 2016 at 12:49:33PM +0100, Christoffer Dall wrote:
> > On Fri, Jan 22, 2016 at 02:44:32PM +0000, Daniel P. Berrange wrote:
> > > On Wed, Jan 06, 2016 at 01:30:16PM +0000, Peter Maydell wrote:
> > > > On 6 January 2016 at 12:49, Andrea Bolognani <abologna@redhat.com> wrote:
> > > > > That's correct, having a QMP command that lists the values gic-version
> > > > > can have on the current host would be just great.
> > > > >
> > > > > If we had that, we could validate the GIC version chosen for a guest,
> > > > > and expose it in the capabilities XML so that higher-level tools can
> > > > > provide a list of choices to the user.
> > > > >
> > > > > Please note that this QMP command would have to work regardless of the
> > > > > machine type selected on QEMU's command line, because libvirt always
> > > > > runs a QEMU binary with '-M none' when probing its capabilities.
> > > > 
> > > > On the other hand, if you don't tell us the machine type you care
> > > > about then we can't tell you:
> > > >  (a) "this machine type doesn't support setting this property at all"
> > > >      (which applies to machines like vexpress-a15 which you can use with
> > > >      KVM on 32-bit hosts, and of course also to all the non-KVM models)
> > > 
> > > We have just recently merged support for registering properties against
> > > classes instead of object instances. There is also a proposed command
> > > to allow querying the list of properties against a class
> > > 
> > >   https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg04348.html
> > > 
> > > So if we now update the machine types to register their properties against
> > > the class instead of object, then we can query what properties are present
> > > against each machine type, while still using '-M none'.
> > > 
> > > >  (b) "this machine type only supports GIC versions X and Y even if the
> > > >       host supports more" (this is currently only hypothetical, though,
> > > >       since we only have the property on 'virt'. it would only happen
> > > >       if in the future we needed something other than '2' or '3' or
> > > >       'host' I think.)
> > > 
> > > Our introspection support in QOM only allows us to say that a property
> > > is a particular type (int / enum / str / whatever). We don't have any
> > > way to expose info about what subset of possible values for a type are
> > > permitted. So I don't see any near term way to inform apps that the
> > > gic property accepts values x, y and but not z.

This actually doesn't matter for the v2 vs. v3 case. The gic-version property
doesn't exist at all for v2-only QEMU. Although maybe the gic-version property
should be reworked. Instead of gic-version=<highest-supported>, we could create
one boolean property per version supported, i.e. gicv2, gicv3, gicv4...

> > > 
> > > IMHO, we shouldn't try to overthink this. Libvirt can query the host
> > > to find out what GIC versions are supported and default to using the
> > > most recent version, on the basis that people are likely to have a
> > > matching QEMU. We can just rely on QEMU to report error if we pass
> > > it a version it doesn't support and not try to pre-emptively check
> > > before launch. The key is just getting the default right IMHO.
> > > 
> > This sounds fine to me.
> > 
> > However, not being familiar with the internals of libvirt I really can't
> > just what the right implementation approach here is.
> > 
> > As I understand we need to either:
> > 
> >   1) Add a QMP command that lets you ask for -M virt, if GIC version X
> >      is supported
> > 
> >   2) Just implement something in libvirt that checks what the kernel
> >      supports directly via the well-defined KVM interface and chooses
> >      the highest supported version per default.
> > 
> > To me it sounds like we should just go ahead with (2) and document
> > somehwere that if you get an error, you need to either manually change
> > the gic version setting in your VM definition file or upgrade QEMU.
> > 
> > Can someone with libvirt authority please advice whether (1) or (2) is
> > what we need to do?
> 
> IMHO we should just go for 2.

I'm not familiar enough with libvirt, nor the use of QMP, to really argue
one way or another, but I find it a bit strange that we'd prefer libvirt
to query two entities over one. And, why should the libvirt installed on
a particular host prefer gicv3 as the default, just because KVM supports
it, even when QEMU does not? The default will fail every time until QEMU
is upgraded, which may not be necessary/desired. Finally, I thought we
were trying to get away from relying on QEMU's error messages to make any
sort of decisions.

I don't know what else libvirt queries directly from KVM, but IMO, it
should be a long-term goal to stop doing it, not add to more. Besides
libvirt then properly selecting defaults that both KVM and QEMU support,
it would allow /dev/kvm to have QEMU-only selinux labels applied.

Thanks,
drew

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Peter Maydell]

On 2 February 2016 at 12:59, Andrew Jones <drjones@redhat.com> wrote:
> This actually doesn't matter for the v2 vs. v3 case. The gic-version property
> doesn't exist at all for v2-only QEMU. Although maybe the gic-version property
> should be reworked. Instead of gic-version=<highest-supported>, we could create
> one boolean property per version supported, i.e. gicv2, gicv3, gicv4...

QEMU 2.5 released with the gic-version property, so we can't just drop
it now; that would break users' command lines. We'd need a strong
argument for why it's worth adding new properties that duplicate
the legacy syntax.

thanks
- PMM

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Andrew Jones]

On Tue, Feb 02, 2016 at 01:15:49PM +0000, Peter Maydell wrote:
> On 2 February 2016 at 12:59, Andrew Jones <drjones@redhat.com> wrote:
> > This actually doesn't matter for the v2 vs. v3 case. The gic-version property
> > doesn't exist at all for v2-only QEMU. Although maybe the gic-version property
> > should be reworked. Instead of gic-version=<highest-supported>, we could create
> > one boolean property per version supported, i.e. gicv2, gicv3, gicv4...
> 
> QEMU 2.5 released with the gic-version property, so we can't just drop
> it now; that would break users' command lines. We'd need a strong
> argument for why it's worth adding new properties that duplicate
> the legacy syntax.

I agree we should have a good argument to justify messing with it, but
until we start versioning mach-virt, then I don't think anybody should
depend on mach-virt command lines working everywhere. Every time we add
a new property, and a user adds it to their command line, then they've
just broken that command line for older QEMU. Yes, I know the upgrade
path is a bit different, but still... Anyway, properties are changeable
with versioned machines.

drew

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Christoffer Dall]

On Tue, Feb 02, 2016 at 01:59:26PM +0100, Andrew Jones wrote:
> On Tue, Feb 02, 2016 at 12:10:10PM +0000, Daniel P. Berrange wrote:
> > On Tue, Feb 02, 2016 at 12:49:33PM +0100, Christoffer Dall wrote:
> > > On Fri, Jan 22, 2016 at 02:44:32PM +0000, Daniel P. Berrange wrote:
> > > > On Wed, Jan 06, 2016 at 01:30:16PM +0000, Peter Maydell wrote:
> > > > > On 6 January 2016 at 12:49, Andrea Bolognani <abologna@redhat.com> wrote:
> > > > > > That's correct, having a QMP command that lists the values gic-version
> > > > > > can have on the current host would be just great.
> > > > > >
> > > > > > If we had that, we could validate the GIC version chosen for a guest,
> > > > > > and expose it in the capabilities XML so that higher-level tools can
> > > > > > provide a list of choices to the user.
> > > > > >
> > > > > > Please note that this QMP command would have to work regardless of the
> > > > > > machine type selected on QEMU's command line, because libvirt always
> > > > > > runs a QEMU binary with '-M none' when probing its capabilities.
> > > > > 
> > > > > On the other hand, if you don't tell us the machine type you care
> > > > > about then we can't tell you:
> > > > >  (a) "this machine type doesn't support setting this property at all"
> > > > >      (which applies to machines like vexpress-a15 which you can use with
> > > > >      KVM on 32-bit hosts, and of course also to all the non-KVM models)
> > > > 
> > > > We have just recently merged support for registering properties against
> > > > classes instead of object instances. There is also a proposed command
> > > > to allow querying the list of properties against a class
> > > > 
> > > >   https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg04348.html
> > > > 
> > > > So if we now update the machine types to register their properties against
> > > > the class instead of object, then we can query what properties are present
> > > > against each machine type, while still using '-M none'.
> > > > 
> > > > >  (b) "this machine type only supports GIC versions X and Y even if the
> > > > >       host supports more" (this is currently only hypothetical, though,
> > > > >       since we only have the property on 'virt'. it would only happen
> > > > >       if in the future we needed something other than '2' or '3' or
> > > > >       'host' I think.)
> > > > 
> > > > Our introspection support in QOM only allows us to say that a property
> > > > is a particular type (int / enum / str / whatever). We don't have any
> > > > way to expose info about what subset of possible values for a type are
> > > > permitted. So I don't see any near term way to inform apps that the
> > > > gic property accepts values x, y and but not z.
> 
> This actually doesn't matter for the v2 vs. v3 case. The gic-version property
> doesn't exist at all for v2-only QEMU. Although maybe the gic-version property
> should be reworked. Instead of gic-version=<highest-supported>, we could create
> one boolean property per version supported, i.e. gicv2, gicv3, gicv4...
> 
> > > > 
> > > > IMHO, we shouldn't try to overthink this. Libvirt can query the host
> > > > to find out what GIC versions are supported and default to using the
> > > > most recent version, on the basis that people are likely to have a
> > > > matching QEMU. We can just rely on QEMU to report error if we pass
> > > > it a version it doesn't support and not try to pre-emptively check
> > > > before launch. The key is just getting the default right IMHO.
> > > > 
> > > This sounds fine to me.
> > > 
> > > However, not being familiar with the internals of libvirt I really can't
> > > just what the right implementation approach here is.
> > > 
> > > As I understand we need to either:
> > > 
> > >   1) Add a QMP command that lets you ask for -M virt, if GIC version X
> > >      is supported
> > > 
> > >   2) Just implement something in libvirt that checks what the kernel
> > >      supports directly via the well-defined KVM interface and chooses
> > >      the highest supported version per default.
> > > 
> > > To me it sounds like we should just go ahead with (2) and document
> > > somehwere that if you get an error, you need to either manually change
> > > the gic version setting in your VM definition file or upgrade QEMU.
> > > 
> > > Can someone with libvirt authority please advice whether (1) or (2) is
> > > what we need to do?
> > 
> > IMHO we should just go for 2.
> 
> I'm not familiar enough with libvirt, nor the use of QMP, to really argue
> one way or another, but I find it a bit strange that we'd prefer libvirt
> to query two entities over one. And, why should the libvirt installed on
> a particular host prefer gicv3 as the default, just because KVM supports
> it, even when QEMU does not? 

I think the assumption here is that if you install a recent libvirt you
also install a recent QEMU.  You always have the risk of things not
working if you have too old a QEMU, right?

> The default will fail every time until QEMU
> is upgraded, which may not be necessary/desired. Finally, I thought we
> were trying to get away from relying on QEMU's error messages to make any
> sort of decisions.

This I have no idea, if libvirt wants to go in the direction of doing
anything humanly possible to avoid QEMU errors, then (2) is probably not
the right approach.

On the other hand, there's always the possibility that something fails,
so you probably always have to deal with QEMU's errors.  I don't think
anyone here suggested that libvirt should try to run QEMU with
something, catch an error, and then run QEMU with other options, if
that's what you meant?

> 
> I don't know what else libvirt queries directly from KVM, but IMO, it
> should be a long-term goal to stop doing it, not add to more. Besides
> libvirt then properly selecting defaults that both KVM and QEMU support,
> it would allow /dev/kvm to have QEMU-only selinux labels applied.
> 
That's really for the libvirt community and maintainers to say, which
direction they're going in.

As I've tried to indicate before, we don't mind doing some work here as
a bunch of Linaro members care about this, but we just want to know from
the libvirt people what they want...?

Thanks,
-Christoffer

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Peter Maydell]

On 2 February 2016 at 14:04, Andrew Jones <drjones@redhat.com> wrote:
> I agree we should have a good argument to justify messing with it, but
> until we start versioning mach-virt, then I don't think anybody should
> depend on mach-virt command lines working everywhere. Every time we add
> a new property, and a user adds it to their command line, then they've
> just broken that command line for older QEMU. Yes, I know the upgrade
> path is a bit different, but still... Anyway, properties are changeable
> with versioned machines.

Clearly command lines that use options added in newer QEMU won't
work on older QEMU, but we do make a serious effort to continue
to support older commandlines on newer QEMU, even if there is no
versioned migration support. Upgrading QEMU and expecting your old
configs to keep working is a reasonable user expectation.

thanks
-- PMM

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Andrew Jones]

On Tue, Feb 02, 2016 at 03:05:28PM +0100, Christoffer Dall wrote:
> On Tue, Feb 02, 2016 at 01:59:26PM +0100, Andrew Jones wrote:
> > On Tue, Feb 02, 2016 at 12:10:10PM +0000, Daniel P. Berrange wrote:
> > > On Tue, Feb 02, 2016 at 12:49:33PM +0100, Christoffer Dall wrote:
> > > > On Fri, Jan 22, 2016 at 02:44:32PM +0000, Daniel P. Berrange wrote:
> > > > > On Wed, Jan 06, 2016 at 01:30:16PM +0000, Peter Maydell wrote:
> > > > > > On 6 January 2016 at 12:49, Andrea Bolognani <abologna@redhat.com> wrote:
> > > > > > > That's correct, having a QMP command that lists the values gic-version
> > > > > > > can have on the current host would be just great.
> > > > > > >
> > > > > > > If we had that, we could validate the GIC version chosen for a guest,
> > > > > > > and expose it in the capabilities XML so that higher-level tools can
> > > > > > > provide a list of choices to the user.
> > > > > > >
> > > > > > > Please note that this QMP command would have to work regardless of the
> > > > > > > machine type selected on QEMU's command line, because libvirt always
> > > > > > > runs a QEMU binary with '-M none' when probing its capabilities.
> > > > > > 
> > > > > > On the other hand, if you don't tell us the machine type you care
> > > > > > about then we can't tell you:
> > > > > >  (a) "this machine type doesn't support setting this property at all"
> > > > > >      (which applies to machines like vexpress-a15 which you can use with
> > > > > >      KVM on 32-bit hosts, and of course also to all the non-KVM models)
> > > > > 
> > > > > We have just recently merged support for registering properties against
> > > > > classes instead of object instances. There is also a proposed command
> > > > > to allow querying the list of properties against a class
> > > > > 
> > > > >   https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg04348.html
> > > > > 
> > > > > So if we now update the machine types to register their properties against
> > > > > the class instead of object, then we can query what properties are present
> > > > > against each machine type, while still using '-M none'.
> > > > > 
> > > > > >  (b) "this machine type only supports GIC versions X and Y even if the
> > > > > >       host supports more" (this is currently only hypothetical, though,
> > > > > >       since we only have the property on 'virt'. it would only happen
> > > > > >       if in the future we needed something other than '2' or '3' or
> > > > > >       'host' I think.)
> > > > > 
> > > > > Our introspection support in QOM only allows us to say that a property
> > > > > is a particular type (int / enum / str / whatever). We don't have any
> > > > > way to expose info about what subset of possible values for a type are
> > > > > permitted. So I don't see any near term way to inform apps that the
> > > > > gic property accepts values x, y and but not z.
> > 
> > This actually doesn't matter for the v2 vs. v3 case. The gic-version property
> > doesn't exist at all for v2-only QEMU. Although maybe the gic-version property
> > should be reworked. Instead of gic-version=<highest-supported>, we could create
> > one boolean property per version supported, i.e. gicv2, gicv3, gicv4...
> > 
> > > > > 
> > > > > IMHO, we shouldn't try to overthink this. Libvirt can query the host
> > > > > to find out what GIC versions are supported and default to using the
> > > > > most recent version, on the basis that people are likely to have a
> > > > > matching QEMU. We can just rely on QEMU to report error if we pass
> > > > > it a version it doesn't support and not try to pre-emptively check
> > > > > before launch. The key is just getting the default right IMHO.
> > > > > 
> > > > This sounds fine to me.
> > > > 
> > > > However, not being familiar with the internals of libvirt I really can't
> > > > just what the right implementation approach here is.
> > > > 
> > > > As I understand we need to either:
> > > > 
> > > >   1) Add a QMP command that lets you ask for -M virt, if GIC version X
> > > >      is supported
> > > > 
> > > >   2) Just implement something in libvirt that checks what the kernel
> > > >      supports directly via the well-defined KVM interface and chooses
> > > >      the highest supported version per default.
> > > > 
> > > > To me it sounds like we should just go ahead with (2) and document
> > > > somehwere that if you get an error, you need to either manually change
> > > > the gic version setting in your VM definition file or upgrade QEMU.
> > > > 
> > > > Can someone with libvirt authority please advice whether (1) or (2) is
> > > > what we need to do?
> > > 
> > > IMHO we should just go for 2.
> > 
> > I'm not familiar enough with libvirt, nor the use of QMP, to really argue
> > one way or another, but I find it a bit strange that we'd prefer libvirt
> > to query two entities over one. And, why should the libvirt installed on
> > a particular host prefer gicv3 as the default, just because KVM supports
> > it, even when QEMU does not? 
> 
> I think the assumption here is that if you install a recent libvirt you
> also install a recent QEMU.  You always have the risk of things not
> working if you have too old a QEMU, right?

I'll grant that recent libvirt probably does imply recent QEMU. If we
need to make assumptions for this solution, that's probably not a
horrible one.

> 
> > The default will fail every time until QEMU
> > is upgraded, which may not be necessary/desired. Finally, I thought we
> > were trying to get away from relying on QEMU's error messages to make any
> > sort of decisions.
> 
> This I have no idea, if libvirt wants to go in the direction of doing
> anything humanly possible to avoid QEMU errors, then (2) is probably not
> the right approach.
> 
> On the other hand, there's always the possibility that something fails,
> so you probably always have to deal with QEMU's errors.  I don't think
> anyone here suggested that libvirt should try to run QEMU with
> something, catch an error, and then run QEMU with other options, if
> that's what you meant?

I was thinking that, but now, after rereading, I see the proposal was
just to propagate the error. So any error results in the same decision,
which is just to quit, and that's reasonable.

> 
> > 
> > I don't know what else libvirt queries directly from KVM, but IMO, it
> > should be a long-term goal to stop doing it, not add to more. Besides
> > libvirt then properly selecting defaults that both KVM and QEMU support,
> > it would allow /dev/kvm to have QEMU-only selinux labels applied.
> > 
> That's really for the libvirt community and maintainers to say, which
> direction they're going in.

Agreed. I'm just thinking out loud, perhaps too noisily...

> 
> As I've tried to indicate before, we don't mind doing some work here as
> a bunch of Linaro members care about this, but we just want to know from
> the libvirt people what they want...?

Yup. I look forward to more of them chiming in. I believe Andrea has
been doing some experimenting.

Thanks,
drew

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Andrea Bolognani]

On Tue, 2016-02-02 at 13:59 +0100, Andrew Jones wrote:
> > > > Our introspection support in QOM only allows us to say
that a property
> > > > is a particular type (int / enum / str / whatever). We don't have any
> > > > way to expose info
about what subset of possible values for a type are
> > > > permitted. So I don't see any near term way to inform apps
that the
> > > > gic property accepts values x, y and but not z.
> 
> This actually doesn't matter for the v2 vs. v3 case.
The gic-version property
> doesn't exist at all for v2-only QEMU. Although maybe the gic-version property
> should be
reworked. Instead of gic-version=<highest-supported>, we could create
> one boolean property per version supported, i.e.
gicv2, gicv3, gicv4...

Hold on, so "gic-version=3" means "support all GIC versions up to 3",
not "support GIC version 3 only"? I thought the latter.

> > >   2) Just implement something in libvirt that checks what the kernel
> > >      supports directly via the well-defined KVM interface and chooses
> > >      the highest supported version per default.
[...]
> I'm not familiar enough with libvirt, nor the use of QMP, to really argue
> one way or another, but I find it a bit strange that we'd prefer libvirt
> to query two entities over one. And, why should the libvirt installed on
> a particular host prefer gicv3 as the default, just because KVM supports
> it, even when QEMU does not? The default will fail every time until QEMU
> is upgraded, which may not be necessary/desired.

Shouldn't the default be "host", to mean "whatever the host supports",
rather than a specific version based either on host or QEMU probing?

That should work for every QEMU version, right?

> Finally, I thought we
> were trying to get away from relying on QEMU's error messages to make any
> sort of decisions.

We wouldn't be looking at the error message and base our decision on
that, we would just display it to the user if QEMU fails to run.

That already happens for a bunch of other situations, so I don't think
it's really a problem, especially because libvirt can't possibly be
expected to catch every possible QEMU failure and sugar-coat it before
reporting it to the user.

> I don't know what else libvirt queries directly from KVM, but IMO, it
> should be a long-term goal to stop doing it, not add to more. Besides
> libvirt then properly selecting defaults that both KVM and QEMU support,
> it would allow /dev/kvm to have QEMU-only selinux labels applied.

One thing that comes to mind is the number of threads per subcores on
ppc64 hosts, and I don't think that's the kind of information QEMU
would provide via QMP.

In the short term we definitely need libvirt to be able to pass
"gic-version=host" to QEMU, and I'm working on a patch that enables
that feature.

As I've already voiced in this thread, my feeling is that the probing
should happen in one place only (QEMU) and libvirt should merely
query that information and report it back to the user, to avoid any
possible disagreement.

On the other hand, Dan has plenty more experience and also knowledge
that spans the whole stack, so in general I trust his opinion :)

One other way to handle this would be to simply report the GIC
versions *libvirt* supports, and let the user pick either the
default ("host", which should work anywhere) or a specific version,
which might or might not actually be accepted by QEMU. I think
there are other places in libvirt where this approach is used,
even though is probably not the most user friendly...

Cheers.

-- 
Andrea Bolognani
Software Engineer - Virtualization Team

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Pavel Fedin]

 Hello!

> Shouldn't the default be "host", to mean "whatever the host supports",
> rather than a specific version based either on host or QEMU probing?

 No, because:
1) Older qemu, which does not support this option, uses v2.
2) It also depends on whether KVM or TCG is used. Currently we can have v3 only with KVM, but when software emulation is implemented, it can be used in all cases, regardless of host KVM restrictions.

> That should work for every QEMU version, right?

 Wrong. See (1) above.

Kind regards,
Pavel Fedin
Senior Engineer
Samsung Electronics Research center Russia

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Peter Maydell]

On 2 February 2016 at 15:09, Andrea Bolognani <abologna@redhat.com> wrote:
> Hold on, so "gic-version=3" means "support all GIC versions up to 3",
> not "support GIC version 3 only"? I thought the latter.

In the current implemented syntax, it means "expose a version 3 GIC
to the guest". You can't simultaneously give the guest a GICv3 and a GICv2.
Whether the host actually supports our doing this is the thing we
can test at runtime (and currently handle by making vm creation fail
if you asked for something the host can't do, just as we do if you
say '-cpu cortex-a57' on a non-A57 host).

>> > >   2) Just implement something in libvirt that checks what the kernel
>> > >      supports directly via the well-defined KVM interface and chooses
>> > >      the highest supported version per default.
> [...]
>> I'm not familiar enough with libvirt, nor the use of QMP, to really argue
>> one way or another, but I find it a bit strange that we'd prefer libvirt
>> to query two entities over one. And, why should the libvirt installed on
>> a particular host prefer gicv3 as the default, just because KVM supports
>> it, even when QEMU does not? The default will fail every time until QEMU
>> is upgraded, which may not be necessary/desired.
>
> Shouldn't the default be "host", to mean "whatever the host supports",
> rather than a specific version based either on host or QEMU probing?
>
> That should work for every QEMU version, right?

For the existing default, the horse has already bolted. The default
(from QEMU's POV) is "2", because this retains command line compatibility
with old command lines, which didn't specify because the option didn't
exist and which assume the guest will be given a GICv2.
(This also makes the handling of -gic-version match -cpu : if you
don't specify you get a default which you might or might not want
but which is the same thing it's always been, ie A15 and GICv2;
if you want 'host' you must specify it; 'host' doesn't work for
TCG, only KVM.)

thanks
-- PMM

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 987 bytes --]

On 02/02/2016 07:05 AM, Christoffer Dall wrote:

>>
>> I'm not familiar enough with libvirt, nor the use of QMP, to really argue
>> one way or another, but I find it a bit strange that we'd prefer libvirt
>> to query two entities over one. And, why should the libvirt installed on
>> a particular host prefer gicv3 as the default, just because KVM supports
>> it, even when QEMU does not? 
> 
> I think the assumption here is that if you install a recent libvirt you
> also install a recent QEMU.  You always have the risk of things not
> working if you have too old a QEMU, right?

Libvirt exists for providing back-compat glue.  The following
combinations are supported:

old libvirt, old qemu
new libvirt, old qemu
new libvirt, new qemu

and it is only this combination that might require a libvirt upgrade to
work correctly:

old libvirt, new qemu

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Christoffer Dall]

On Tue, Feb 2, 2016 at 4:52 PM, Eric Blake <eblake@redhat.com> wrote:
> On 02/02/2016 07:05 AM, Christoffer Dall wrote:
>
>>>
>>> I'm not familiar enough with libvirt, nor the use of QMP, to really argue
>>> one way or another, but I find it a bit strange that we'd prefer libvirt
>>> to query two entities over one. And, why should the libvirt installed on
>>> a particular host prefer gicv3 as the default, just because KVM supports
>>> it, even when QEMU does not?
>>
>> I think the assumption here is that if you install a recent libvirt you
>> also install a recent QEMU.  You always have the risk of things not
>> working if you have too old a QEMU, right?
>
> Libvirt exists for providing back-compat glue.  The following
> combinations are supported:
>
> old libvirt, old qemu
> new libvirt, old qemu
> new libvirt, new qemu
>
> and it is only this combination that might require a libvirt upgrade to
> work correctly:
>
> old libvirt, new qemu
>

ok, so that would mean we need to implement a QMP command to tell us
which gic versions are supported for a given machine.  Current
possible responses are "2", "3" and "2,3"

and we also need to add code to libvirt to try that QMP command, and
if it doesn't exist, fall back to not specifying gic-version, using
the old-qemu compatible default of providing a gicv2 to guests, and if
the QMP command exists, use the newest gic-version.

users can then always override this behavior by directly specifying a
gic version "host", "2", or "3" in their xml file.

any objections?

-Christoffer

Re: [Qemu-devel] [libvirt] ARM KVM GICv3 Support

[Andrea Bolognani]

On Tue, 2016-02-02 at 16:58 +0100, Christoffer Dall wrote:
> ok, so that would mean we need to implement a QMP command to tell us
> which gic versions are supported for a given machine.  Current
> possible responses are "2", "3" and "2,3"
> 
> and we also need to add code to libvirt to try that QMP command, and
> if it doesn't exist, fall back to not specifying gic-version, using
> the old-qemu compatible default of providing a gicv2 to guests, and if
> the QMP command exists, use the newest gic-version.
> 
> users can then always override this behavior by directly specifying a
> gic version "host", "2", or "3" in their xml file.
> 
> any objections?

Dan voiced his preference for probing the host GIC versions from
libvirt and just passing that to QEMU dealing with any failure later
on, but I think that was mostly to keep things simple and not
because the QMP command approach was wrong?

IOW Dan, if we went ahead with the QMP command approach, would you
oppose it? Peter Xu has posted some RFC QEMU patches yesterday...

Cheers.

-- 
Andrea Bolognani
Software Engineer - Virtualization Team