[Qemu-devel] QOM vs QAPI for QMP APIs

[Qemu-devel] QOM vs QAPI for QMP APIs

[Stefan Hajnoczi]

I need to add a QMP API that lists dataplane threads.  This is similar
to "query-cpus" where the thread IDs are reported.  It allows the client
to bind threads to host CPUs.

I'm inclined to add a "query-iothreads" QMP command:
 * It's easy to implement using QAPI
 * We've developed best practices for QMP APIs
 * We know how to version and make QMP APIs extensible
 * Clients (including libvirt) are used to QMP JSON RPC

But maybe I should use QOM instead:
 * Add a "qom-find-objects-by-class" QMP command (Paolo's idea)
 * Client does "qom-find-objects-by-class IOThread /objects"
 * Client then uses "qom-get" to fetch the thread_id property on each
   IOThread object
 * But we haven't really established how QOM APIs will work

So my question is: should we use QOM as the external API or continue
using QAPI?

I don't think we gain much by switching to QOM other than opening a
whole new design space that we've yet to master.  We'll make plenty of
mistakes just like we did with QMP and QAPI.

Although QOM eliminates the need to implement dedicated QMP commands, it
exposes a more complex model to the client.  Instead of a JSON
command/response model we now expose a general object-oriented namespace
with properties, links, etc.  The client has to make sense of all that
and has to perform multiple qom-list/qom-get/etc commands for something
that would take a single dedicated QMP command.

Maybe I just need some convincing but it seems that QAPI is the simplest
and cleanest way to define external APIs.

Disagree?  Tell me why :).

Stefan

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Anthony Liguori]

On Fri, Feb 21, 2014 at 1:16 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> I need to add a QMP API that lists dataplane threads.  This is similar
> to "query-cpus" where the thread IDs are reported.  It allows the client
> to bind threads to host CPUs.
>
> I'm inclined to add a "query-iothreads" QMP command:
>  * It's easy to implement using QAPI
>  * We've developed best practices for QMP APIs
>  * We know how to version and make QMP APIs extensible
>  * Clients (including libvirt) are used to QMP JSON RPC
>
> But maybe I should use QOM instead:
>  * Add a "qom-find-objects-by-class" QMP command (Paolo's idea)
>  * Client does "qom-find-objects-by-class IOThread /objects"
>  * Client then uses "qom-get" to fetch the thread_id property on each
>    IOThread object
>  * But we haven't really established how QOM APIs will work

I have no objection to introducing a QMP command.

I think qom-find-objects-by-class is a reasonable approach but I would
also consider just grouping all of the IOThreads in a well known path
instead of just having them live in /objects.  So something like
/objects/threads/thread0/pid.

It ends up being very similar to working with sysfs at that point.

Regards,

Anthony Liguori

> So my question is: should we use QOM as the external API or continue
> using QAPI?
>
> I don't think we gain much by switching to QOM other than opening a
> whole new design space that we've yet to master.  We'll make plenty of
> mistakes just like we did with QMP and QAPI.
>
> Although QOM eliminates the need to implement dedicated QMP commands, it
> exposes a more complex model to the client.  Instead of a JSON
> command/response model we now expose a general object-oriented namespace
> with properties, links, etc.  The client has to make sense of all that
> and has to perform multiple qom-list/qom-get/etc commands for something
> that would take a single dedicated QMP command.
>
> Maybe I just need some convincing but it seems that QAPI is the simplest
> and cleanest way to define external APIs.
>
> Disagree?  Tell me why :).
>
> Stefan
>

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Stefan Hajnoczi]

On Fri, Feb 21, 2014 at 10:16 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> Maybe I just need some convincing but it seems that QAPI is the simplest
> and cleanest way to define external APIs.
>
> Disagree?  Tell me why :).

I'm replying to myself because we had an interesting discussion on
IRC.  Thanks Paolo, Markus, and Peter!

I'm biased, but here are the points collected from the discussion:

Why QOM *should* be QEMU's external API:
 * QOM eliminates hand-written QMP API code.  We get query-foo APIs
for free using qom-list, qom-get, qom-set, and friends

Why QOM should *not* be QEMU's external API:
 * Internal objects will diverge from the external QOM objects over
time, we'll have to maintain a backwards compatible QOM object layer
on top of QEMU's internal objects - that defeats the whole point of
getting APIs for free from QOM
 * We lose the easy-to-review qapi-schema.json.  Would need new
documentation and review tools for QOM model changes.
 * query-foo APIs are simple to implement, not a huge win to get them via QOM
 * QAPI is mature and well-understood, QMP is incomplete
 * qom-list, qom-get, qom-set are too low-level, need more powerful
APIs to save clients from elaborate back-and-forth conversations

Stefan

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Paolo Bonzini]

Il 21/02/2014 15:29, Anthony Liguori ha scritto:
> On Fri, Feb 21, 2014 at 1:16 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>> I need to add a QMP API that lists dataplane threads.  This is similar
>> to "query-cpus" where the thread IDs are reported.  It allows the client
>> to bind threads to host CPUs.
>>
>> I'm inclined to add a "query-iothreads" QMP command:
>>  * It's easy to implement using QAPI
>>  * We've developed best practices for QMP APIs
>>  * We know how to version and make QMP APIs extensible
>>  * Clients (including libvirt) are used to QMP JSON RPC
>>
>> But maybe I should use QOM instead:
>>  * Add a "qom-find-objects-by-class" QMP command (Paolo's idea)
>>  * Client does "qom-find-objects-by-class IOThread /objects"
>>  * Client then uses "qom-get" to fetch the thread_id property on each
>>    IOThread object
>>  * But we haven't really established how QOM APIs will work
>
> I have no objection to introducing a QMP command.
>
> I think qom-find-objects-by-class is a reasonable approach but I would
> also consider just grouping all of the IOThreads in a well known path
> instead of just having them live in /objects.  So something like
> /objects/threads/thread0/pid.

/objects is the namespace for -object, but a similar idea is that 
objects could create links of themselves under other paths.  So you 
would have /threads where you can list iothread objects or /backend/rng 
for RNG backends.

Still Stefan doesn't like the idea of sending O(n) commands to query the 
thread ID of n iothread objects.

Paolo

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2047 bytes --]

On 02/21/2014 07:37 AM, Paolo Bonzini wrote:
>> I have no objection to introducing a QMP command.
>>
>> I think qom-find-objects-by-class is a reasonable approach but I would
>> also consider just grouping all of the IOThreads in a well known path
>> instead of just having them live in /objects.  So something like
>> /objects/threads/thread0/pid.
> 
> /objects is the namespace for -object, but a similar idea is that
> objects could create links of themselves under other paths.  So you
> would have /threads where you can list iothread objects or /backend/rng
> for RNG backends.
> 
> Still Stefan doesn't like the idea of sending O(n) commands to query the
> thread ID of n iothread objects.

The other burden is documenting what QOM paths to be queried, and
knowing where to find that documentation.  That is, it's another layer
of complexity, but it's also a more powerful expression.

I'm comparing this situation somewhat to libvirt's 'virsh
qemu-monitor-command' vs. other libvirt commands.  qemu-monitor-command
is a more powerful interface (via libvirt, you can issue ANY qmp
command), and is therefore great for development for testing something
that libvirt has not yet supported; but not so nice to the end user
(it's use is explicitly unsupported).  What happens is that as people
say "I had to use qemu-monitor-command to do task A", it is a hint to
libvirt development to say "oh, we need to add an API to make task A
easier to do".

Thus, having qom-find-objects-by-class is a good idea, even if it is
more awkward to use than a dedicated qmp command.  But meanwhile, we
should watch what common patterns it gets used for, and add dedicated
QMP commands for those patterns.  It's much faster to get a chunk of
information in one QMP call already formatted into desired structs than
it is to make a series of QMP calls to learn about the lower-level qom
model one piece at a time.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/21/2014 07:37 AM, Paolo Bonzini wrote:
>>> I have no objection to introducing a QMP command.
>>>
>>> I think qom-find-objects-by-class is a reasonable approach but I would
>>> also consider just grouping all of the IOThreads in a well known path
>>> instead of just having them live in /objects.  So something like
>>> /objects/threads/thread0/pid.
>> 
>> /objects is the namespace for -object, but a similar idea is that
>> objects could create links of themselves under other paths.  So you
>> would have /threads where you can list iothread objects or /backend/rng
>> for RNG backends.
>> 
>> Still Stefan doesn't like the idea of sending O(n) commands to query the
>> thread ID of n iothread objects.
>
> The other burden is documenting what QOM paths to be queried, and
> knowing where to find that documentation.  That is, it's another layer
> of complexity, but it's also a more powerful expression.
>
> I'm comparing this situation somewhat to libvirt's 'virsh
> qemu-monitor-command' vs. other libvirt commands.  qemu-monitor-command
> is a more powerful interface (via libvirt, you can issue ANY qmp
> command), and is therefore great for development for testing something
> that libvirt has not yet supported; but not so nice to the end user
> (it's use is explicitly unsupported).  What happens is that as people
> say "I had to use qemu-monitor-command to do task A", it is a hint to
> libvirt development to say "oh, we need to add an API to make task A
> easier to do".
>
> Thus, having qom-find-objects-by-class is a good idea, even if it is
> more awkward to use than a dedicated qmp command.  But meanwhile, we
> should watch what common patterns it gets used for, and add dedicated
> QMP commands for those patterns.  It's much faster to get a chunk of
> information in one QMP call already formatted into desired structs than
> it is to make a series of QMP calls to learn about the lower-level qom
> model one piece at a time.

You didn't spell out the ABI promises here.  Do you argue for providing
QOM interfaces as unstable low-level interfaces?

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1792 bytes --]

On 02/24/2014 01:29 AM, Markus Armbruster wrote:

>>
>> The other burden is documenting what QOM paths to be queried, and
>> knowing where to find that documentation.  That is, it's another layer
>> of complexity, but it's also a more powerful expression.
>>
>> I'm comparing this situation somewhat to libvirt's 'virsh
>> qemu-monitor-command' vs. other libvirt commands.  qemu-monitor-command
>> is a more powerful interface (via libvirt, you can issue ANY qmp
>> command), and is therefore great for development for testing something
>> that libvirt has not yet supported; but not so nice to the end user
>> (it's use is explicitly unsupported).  What happens is that as people
>> say "I had to use qemu-monitor-command to do task A", it is a hint to
>> libvirt development to say "oh, we need to add an API to make task A
>> easier to do".
>>
>> Thus, having qom-find-objects-by-class is a good idea, even if it is
>> more awkward to use than a dedicated qmp command.  But meanwhile, we
>> should watch what common patterns it gets used for, and add dedicated
>> QMP commands for those patterns.  It's much faster to get a chunk of
>> information in one QMP call already formatted into desired structs than
>> it is to make a series of QMP calls to learn about the lower-level qom
>> model one piece at a time.
> 
> You didn't spell out the ABI promises here.  Do you argue for providing
> QOM interfaces as unstable low-level interfaces?

Haven't we already done that in the past?  For example, object-add
currently takes an unspecified dictionary of options, where you would
have to consult QOM documentation to learn what makes sense to send.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 02/24/2014 01:29 AM, Markus Armbruster wrote:
>
>>>
>>> The other burden is documenting what QOM paths to be queried, and
>>> knowing where to find that documentation.  That is, it's another layer
>>> of complexity, but it's also a more powerful expression.
>>>
>>> I'm comparing this situation somewhat to libvirt's 'virsh
>>> qemu-monitor-command' vs. other libvirt commands.  qemu-monitor-command
>>> is a more powerful interface (via libvirt, you can issue ANY qmp
>>> command), and is therefore great for development for testing something
>>> that libvirt has not yet supported; but not so nice to the end user
>>> (it's use is explicitly unsupported).  What happens is that as people
>>> say "I had to use qemu-monitor-command to do task A", it is a hint to
>>> libvirt development to say "oh, we need to add an API to make task A
>>> easier to do".
>>>
>>> Thus, having qom-find-objects-by-class is a good idea, even if it is
>>> more awkward to use than a dedicated qmp command.  But meanwhile, we
>>> should watch what common patterns it gets used for, and add dedicated
>>> QMP commands for those patterns.  It's much faster to get a chunk of
>>> information in one QMP call already formatted into desired structs than
>>> it is to make a series of QMP calls to learn about the lower-level qom
>>> model one piece at a time.
>> 
>> You didn't spell out the ABI promises here.  Do you argue for providing
>> QOM interfaces as unstable low-level interfaces?
>
> Haven't we already done that in the past?  For example, object-add
> currently takes an unspecified dictionary of options, where you would
> have to consult QOM documentation to learn what makes sense to send.

My question isn't about where the command details are documented, or
even whether they are documented.  It's about ABI promises, or lack
thereof.  The general promise for QMP is we treat it as stable ABI.  If
we add QMP commands to examine and manipulate QOM, doesn't that move all
of QOM under the QMP ABI promise, unless we explicitly excempt it?

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Andreas Färber]

Am 25.02.2014 09:25, schrieb Markus Armbruster:
> Eric Blake <eblake@redhat.com> writes:
> 
>> On 02/24/2014 01:29 AM, Markus Armbruster wrote:
>>
>>>>
>>>> The other burden is documenting what QOM paths to be queried, and
>>>> knowing where to find that documentation.  That is, it's another layer
>>>> of complexity, but it's also a more powerful expression.
>>>>
>>>> I'm comparing this situation somewhat to libvirt's 'virsh
>>>> qemu-monitor-command' vs. other libvirt commands.  qemu-monitor-command
>>>> is a more powerful interface (via libvirt, you can issue ANY qmp
>>>> command), and is therefore great for development for testing something
>>>> that libvirt has not yet supported; but not so nice to the end user
>>>> (it's use is explicitly unsupported).  What happens is that as people
>>>> say "I had to use qemu-monitor-command to do task A", it is a hint to
>>>> libvirt development to say "oh, we need to add an API to make task A
>>>> easier to do".
>>>>
>>>> Thus, having qom-find-objects-by-class is a good idea, even if it is
>>>> more awkward to use than a dedicated qmp command.  But meanwhile, we
>>>> should watch what common patterns it gets used for, and add dedicated
>>>> QMP commands for those patterns.  It's much faster to get a chunk of
>>>> information in one QMP call already formatted into desired structs than
>>>> it is to make a series of QMP calls to learn about the lower-level qom
>>>> model one piece at a time.
>>>
>>> You didn't spell out the ABI promises here.  Do you argue for providing
>>> QOM interfaces as unstable low-level interfaces?
>>
>> Haven't we already done that in the past?  For example, object-add
>> currently takes an unspecified dictionary of options, where you would
>> have to consult QOM documentation to learn what makes sense to send.
> 
> My question isn't about where the command details are documented, or
> even whether they are documented.  It's about ABI promises, or lack
> thereof.  The general promise for QMP is we treat it as stable ABI.  If
> we add QMP commands to examine and manipulate QOM, doesn't that move all
> of QOM under the QMP ABI promise, unless we explicitly excempt it?

Anthony has clearly stated and I have been repeating that QOM follows
strict ABI stability rules for its properties, and that therefore
applies to what some of you still call "qdev" as well. Therefore my
nagging to get underscores vs. dashes right for new properties and
questioning attempted type changes without changing the property name.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Paolo Bonzini]

Il 25/02/2014 09:25, Markus Armbruster ha scritto:
>> > Haven't we already done that in the past?  For example, object-add
>> > currently takes an unspecified dictionary of options, where you would
>> > have to consult QOM documentation to learn what makes sense to send.
> My question isn't about where the command details are documented, or
> even whether they are documented.  It's about ABI promises, or lack
> thereof.  The general promise for QMP is we treat it as stable ABI.  If
> we add QMP commands to examine and manipulate QOM, doesn't that move all
> of QOM under the QMP ABI promise, unless we explicitly excempt it?

We did already exempt it, in general.  QOM is not declared stable.

However, there are parts (-object, and in 2.0 object_add/object-add) 
that are part of the QEMU API/ABI.  Now that we have a way to restrict 
which classes will be compatible (only classes that are UserCreatable 
can be created and thus will require backwards compatibility), we need 
to document those closses.

Paolo

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Andreas Färber]

Am 25.02.2014 09:30, schrieb Paolo Bonzini:
> Il 25/02/2014 09:25, Markus Armbruster ha scritto:
>>> > Haven't we already done that in the past?  For example, object-add
>>> > currently takes an unspecified dictionary of options, where you would
>>> > have to consult QOM documentation to learn what makes sense to send.
>> My question isn't about where the command details are documented, or
>> even whether they are documented.  It's about ABI promises, or lack
>> thereof.  The general promise for QMP is we treat it as stable ABI.  If
>> we add QMP commands to examine and manipulate QOM, doesn't that move all
>> of QOM under the QMP ABI promise, unless we explicitly excempt it?
> 
> We did already exempt it, in general.  QOM is not declared stable.

Negative, QOM is stable in that properties may not change their type
incompatibly. Properties may be dropped/renamed, since that is
discoverable via qom-list, so it is less stable than the command line
interface.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Kevin Wolf]

Am 21.02.2014 um 15:32 hat Stefan Hajnoczi geschrieben:
> On Fri, Feb 21, 2014 at 10:16 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > Maybe I just need some convincing but it seems that QAPI is the simplest
> > and cleanest way to define external APIs.
> >
> > Disagree?  Tell me why :).
> 
> I'm replying to myself because we had an interesting discussion on
> IRC.  Thanks Paolo, Markus, and Peter!
> 
> I'm biased, but here are the points collected from the discussion:
> 
> Why QOM *should* be QEMU's external API:
>  * QOM eliminates hand-written QMP API code.  We get query-foo APIs
> for free using qom-list, qom-get, qom-set, and friends

I don't think this is valid point. We already have a QAPI code
generator. There's no reason why it couldn't automatically map things to
QOM for certain commands.

This means that you would internally use QOM in order to get rid of
hand-written code, but it would be an implementation detail and the
external API would still be as high-level as it is today.

> Why QOM should *not* be QEMU's external API:
>  * Internal objects will diverge from the external QOM objects over
> time, we'll have to maintain a backwards compatible QOM object layer
> on top of QEMU's internal objects - that defeats the whole point of
> getting APIs for free from QOM

This is a very important point. We already managed to mess up existing
QMP commands so that we're jumping through hoops to provide an old
interface, which doesn't really match the design we want to have.
Sometimes it prevents us from making changes.

With QOM, we potentially expose everything, every little detail of our
internal implementation. If we don't say that there is no (or at least
very weak) compatibility promise, then we're forever stuck with whatever
broken design we first came up with.

If QOM comes with stability requirements, I'll veto any conversion patch
of any part of the block layer to QOM.

>  * We lose the easy-to-review qapi-schema.json.  Would need new
> documentation and review tools for QOM model changes.
>  * query-foo APIs are simple to implement, not a huge win to get them via QOM
>  * QAPI is mature and well-understood, QMP is incomplete
>  * qom-list, qom-get, qom-set are too low-level, need more powerful
> APIs to save clients from elaborate back-and-forth conversations

Right, QOM is simple in the way that programming in assembly is simple.
The building blocks are really simple and easy to explain, but building
something higher level from it is hard and easy to get wrong.

Try expressing a 'transaction' QMP command with multiple newly created
snapshots and a backup job in QOM... (without creating a transaction
object that contains the current QAPI arguments as properties, because
that wouldn't really be using QOM but tunneling QMP through QOM)

External QOM interfaces have their place, especially for debugging or
trying out things before there is a proper API, but it's not as the
primary external interface.

Kevin

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Stefan Hajnoczi]

On Tue, Feb 25, 2014 at 10:46 AM, Kevin Wolf <kwolf@redhat.com> wrote:
> External QOM interfaces have their place, especially for debugging or
> trying out things before there is a proper API, but it's not as the
> primary external interface.

Yes, I agree.

Stefan

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Paolo Bonzini]

Il 25/02/2014 11:15, Stefan Hajnoczi ha scritto:
> On Tue, Feb 25, 2014 at 10:46 AM, Kevin Wolf <kwolf@redhat.com> wrote:
>> > External QOM interfaces have their place, especially for debugging or
>> > trying out things before there is a proper API, but it's not as the
>> > primary external interface.
> Yes, I agree.

-object is an exception to this though, isn't it?

Paolo

Re: [Qemu-devel] QOM vs QAPI for QMP APIs

[Kevin Wolf]

Am 25.02.2014 um 11:47 hat Paolo Bonzini geschrieben:
> Il 25/02/2014 11:15, Stefan Hajnoczi ha scritto:
> >On Tue, Feb 25, 2014 at 10:46 AM, Kevin Wolf <kwolf@redhat.com> wrote:
> >>> External QOM interfaces have their place, especially for debugging or
> >>> trying out things before there is a proper API, but it's not as the
> >>> primary external interface.
> >Yes, I agree.
> 
> -object is an exception to this though, isn't it?

I'd prefer if -object weren't the primary interface for anything.
We could define a few stable types that are meant to be created,
queried and manipulated using QOM, but it would blur the line.

Kevin