QEMU-Devel Archive on lore.kernel.org
 help / color / Atom feed
* Making QEMU easier for management tools and applications
@ 2019-12-20 16:13 Stefan Hajnoczi
  2019-12-20 21:07 ` Richard W.M. Jones
                   ` (4 more replies)
  0 siblings, 5 replies; 34+ messages in thread
From: Stefan Hajnoczi @ 2019-12-20 16:13 UTC (permalink / raw)
  To: qemu-devel
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Markus Armbruster, Marc-André Lureau, Paolo Bonzini,
	John Snow, Dominik Csapak

Hi,
QEMU presents a command-line interface and QMP monitor for
applications to interact with.  Applications actually need API
bindings in their programming language.  Bindings avoid reimplementing
code to spawn a QEMU process and interact with QMP.  QEMU is kind of
lazy and de facto relies on libvirt for API bindings.

Is it time for better QEMU APIs?

1. We have qapi-schema.json.  Let's render to HTML and publish
versioned documentation online.

2. scripts/qmp/ contains command-line tools for QMP communication.
They could use some polish and then be shipped.

3. python/qemu/ contains Python modules for managing a QEMU process
and QMP communication.  This should be packaged in distros and
available on PyPI.

4. Go and Rust bindings would also be useful.  There is
https://github.com/intel/govmm but I think it makes sense to keep it
in qemu.git and provide an interface similar to our Python modules.

5. A jailer is needed to isolate the QEMU process and vhost-user
device backends using seccomp, Linux namespaces, and maybe
SELinux/AppArmor.  We used to be able to rely on libvirt for QEMU
security, but it's becoming a common task for any device backend and
IMO should be its own launcher tool.

6. A configuration file format is sorely needed so that guest
configuration can be persisted and easily launched.  Today you have to
create a shell script that executes a QEMU command-line, but this is
suboptimal because sharing executable scripts is dangerous from a
security perspective and is hard to parse or modify programmatically.

In many of these areas we already have a partial solution.  It just
needs more work.  I think it would be worth the effort and the mental
shift to really providing APIs that are easy to use by applications.

What do you think?

Have I missed things that are needed?

Have I included things that are unnecessary?

Stefan


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-20 16:13 Making QEMU easier for management tools and applications Stefan Hajnoczi
@ 2019-12-20 21:07 ` Richard W.M. Jones
  2020-01-02 11:26   ` Stefan Hajnoczi
  2019-12-21  9:02 ` Markus Armbruster
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 34+ messages in thread
From: Richard W.M. Jones @ 2019-12-20 21:07 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	qemu-devel, Markus Armbruster, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

On Fri, Dec 20, 2019 at 04:13:59PM +0000, Stefan Hajnoczi wrote:
> Hi,
> QEMU presents a command-line interface and QMP monitor for
> applications to interact with.  Applications actually need API
> bindings in their programming language.  Bindings avoid reimplementing
> code to spawn a QEMU process and interact with QMP.  QEMU is kind of
> lazy and de facto relies on libvirt for API bindings.

Just wanted to add that for a concrete example you might want to look
at how libguestfs launches qemu, either directly:

  https://github.com/libguestfs/libguestfs/blob/master/lib/launch-direct.c

or via libvirt:

  https://github.com/libguestfs/libguestfs/blob/master/lib/launch-libvirt.c

Obviously I support anything that makes that easier, especially
the direct case.

Some points that you didn't cover:

- Any APIs we use need to be callable from a C library.  This has a
  few weird and wonderful beartraps to be aware of, such as being
  unable to predict the state of signal masks, not knowing what file
  descriptors will be open, and being religious about cleaning up any
  resources or subprocesses that you allocate / fork.

- To be usable, the new APIs need to cover everything that libguestfs
  needs, otherwise we literally cannot switch over to them.  (I guess
  the same applies to libvirt to an even greater extent.)

> 6. A configuration file format is sorely needed so that guest
> configuration can be persisted and easily launched. 

Actually qemu already has that, but it's really half-assed, barely
documented, and doesn't cover major features that qemu provides :-)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-20 16:13 Making QEMU easier for management tools and applications Stefan Hajnoczi
  2019-12-20 21:07 ` Richard W.M. Jones
@ 2019-12-21  9:02 ` Markus Armbruster
  2019-12-23 15:04   ` Michal Prívozník
                     ` (3 more replies)
  2019-12-24 13:00 ` Daniel P. Berrangé
                   ` (2 subsequent siblings)
  4 siblings, 4 replies; 34+ messages in thread
From: Markus Armbruster @ 2019-12-21  9:02 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Cleber Rosa, qemu-devel, Eduardo Habkost, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

Stefan Hajnoczi <stefanha@gmail.com> writes:

> Hi,
> QEMU presents a command-line interface and QMP monitor for
> applications to interact with.  Applications actually need API
> bindings in their programming language.  Bindings avoid reimplementing
> code to spawn a QEMU process and interact with QMP.  QEMU is kind of
> lazy and de facto relies on libvirt for API bindings.
>
> Is it time for better QEMU APIs?
>
> 1. We have qapi-schema.json.  Let's render to HTML and publish
> versioned documentation online.

Make can build docs/interop/qemu-qmp-ref.{7,html,info,pdf,txt}.  Grab
the .html and go for it.  There's also qmp-ga-ref.

Sadly, this documentation is not nearly as good as it could be.  The doc
comment format is an ad hoc solution to the problem of enabling doc
generation with minimal churn to existing comments.  The doc generator
makes an effort to play that hand (but the hand sucks).  Finally and
most seriously, the doc comments are simply not good enough.

> 2. scripts/qmp/ contains command-line tools for QMP communication.
> They could use some polish and then be shipped.

MAINTAINERS blames them on me, but they're effectively unmaintained.
Prerequisite for shipping: having a maintainer who actually gives a
damn.

Let's review them:

* scripts/qmp/qemu-ga-client

  Is anybody using this?

  I think it should be unter "QEMU Guest Agent", not under "QMP".

* scripts/qmp/qmp

  Half-hearted attempt at a CLI tool for sending a QMP command.  Is
  anybody using this?
 
* scripts/qmp/qmp-shell

  Half-hearted attempt at a human-friendly wrapper around the JSON
  syntax.  I have no use for this myself.

* scripts/qmp/qom-fuse

  I find this occasionally useful for QOM-spelunking.  The need for this
  indicates our QMP interface to QOM is too primitive for such uses.

* scripts/qmp/qom-get
  scripts/qmp/qom-list
  scripts/qmp/qom-set
  scripts/qmp/qom-tree

  Rather specialized.  Is anybody using them?

> 3. python/qemu/ contains Python modules for managing a QEMU process
> and QMP communication.  This should be packaged in distros and
> available on PyPI.

Currently maintained by Eduardo and Cleber (cc'ed) under "Python
scripts".

> 4. Go and Rust bindings would also be useful.  There is
> https://github.com/intel/govmm but I think it makes sense to keep it
> in qemu.git and provide an interface similar to our Python modules.

Mapping QAPI/QMP commands and events to function signatures isn't hard
(the QAPI code generator does).  Two problems (at least):

1. Leads to some pretty ridiculous functions.  Here's one:

    void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
                             const char *device,
                             const char *target,
                             bool has_replaces, const char *replaces,
                             MirrorSyncMode sync,
                             bool has_speed, int64_t speed,
                             bool has_granularity, uint32_t granularity,
                             bool has_buf_size, int64_t buf_size,
                             bool has_on_source_error,
                             BlockdevOnError on_source_error,
                             bool has_on_target_error, BlockdevOnError on_target_error,
                             bool has_filter_node_name, const char *filter_node_name,
                             bool has_copy_mode, MirrorCopyMode copy_mode, 
                             bool has_auto_finalize, bool auto_finalize,
                             bool has_auto_dismiss, bool auto_dismiss,
                             Error **errp);

  We commonly use 'boxed': true for such beasts, which results in
  functions like this one:

    void qmp_blockdev_add(BlockdevOptions *arg, Error **errp);

2. Many schema changes that are nicely backward compatible in QMP are
   anything but in such an "obvious" C API.  Adding optional arguments,
   for instance, or changing integer type width.  The former is less of
   an issue with 'boxed': true.

Perhaps less of an issue with dynamic languages.

I figure a static language would need much more expressive oomph than C
to be a good target.  No idea how well Go or Rust bindings can work.

> 5. A jailer is needed to isolate the QEMU process and vhost-user
> device backends using seccomp, Linux namespaces, and maybe
> SELinux/AppArmor.  We used to be able to rely on libvirt for QEMU
> security, but it's becoming a common task for any device backend and
> IMO should be its own launcher tool.

Perhaps the libvirt refactoring effort can give us one.

> 6. A configuration file format is sorely needed so that guest
> configuration can be persisted and easily launched.  Today you have to
> create a shell script that executes a QEMU command-line, but this is
> suboptimal because sharing executable scripts is dangerous from a
> security perspective and is hard to parse or modify programmatically.

No argument.  There is -readconfig, but it falls way short.

With command line QAPIfication, a real configuration file will be quite
feasible.

The main reason for the lack of progress there is our dedication to
backward compatibility.  A functional replacement of our CLI is a huge
task already.  Throwing in backward compatibility makes it a daunting
one.  Not least because nobody fully understands all the quirks.

> In many of these areas we already have a partial solution.  It just
> needs more work.  I think it would be worth the effort and the mental
> shift to really providing APIs that are easy to use by applications.
>
> What do you think?
>
> Have I missed things that are needed?
>
> Have I included things that are unnecessary?

I feel we need to make up our minds what kind of interface(s) we want to
provide.

We provide a low-level interface for management applications.  I feel we
need to nail this one.  Its QMP part is okay, I think, the CLI part is
not.

We also try to provide friendlier interfaces for human users.  Not only
do we suck at it, we're also prone to create liabilities for later: much
of the compatibility woes that have been holding us back are rooted in
"convenience" features.  And don't get me started on the folly of us
dabbling in GUI.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-21  9:02 ` Markus Armbruster
@ 2019-12-23 15:04   ` Michal Prívozník
  2020-01-07  9:36     ` Kevin Wolf
  2019-12-24 13:41   ` Daniel P. Berrangé
                     ` (2 subsequent siblings)
  3 siblings, 1 reply; 34+ messages in thread
From: Michal Prívozník @ 2019-12-23 15:04 UTC (permalink / raw)
  To: Markus Armbruster, Stefan Hajnoczi
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Cleber Rosa, qemu-devel, Marc-André Lureau, Paolo Bonzini,
	Dominik Csapak, John Snow, Eduardo Habkost

On 12/21/19 10:02 AM, Markus Armbruster wrote:
> Stefan Hajnoczi <stefanha@gmail.com> writes:
> 


>> 4. Go and Rust bindings would also be useful.  There is
>> https://github.com/intel/govmm but I think it makes sense to keep it
>> in qemu.git and provide an interface similar to our Python modules.
> 
> Mapping QAPI/QMP commands and events to function signatures isn't hard
> (the QAPI code generator does).  Two problems (at least):
> 
> 1. Leads to some pretty ridiculous functions.  Here's one:
> 
>     void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
>                              const char *device,
>                              const char *target,
>                              bool has_replaces, const char *replaces,
>                              MirrorSyncMode sync,
>                              bool has_speed, int64_t speed,
>                              bool has_granularity, uint32_t granularity,
>                              bool has_buf_size, int64_t buf_size,
>                              bool has_on_source_error,
>                              BlockdevOnError on_source_error,
>                              bool has_on_target_error, BlockdevOnError on_target_error,
>                              bool has_filter_node_name, const char *filter_node_name,
>                              bool has_copy_mode, MirrorCopyMode copy_mode, 
>                              bool has_auto_finalize, bool auto_finalize,
>                              bool has_auto_dismiss, bool auto_dismiss,
>                              Error **errp);
> 
>   We commonly use 'boxed': true for such beasts, which results in
>   functions like this one:
> 
>     void qmp_blockdev_add(BlockdevOptions *arg, Error **errp);
> 
> 2. Many schema changes that are nicely backward compatible in QMP are
>    anything but in such an "obvious" C API.  Adding optional arguments,
>    for instance, or changing integer type width.  The former is less of
>    an issue with 'boxed': true.
> 
> Perhaps less of an issue with dynamic languages.
> 
> I figure a static language would need much more expressive oomph than C
> to be a good target.  No idea how well Go or Rust bindings can work.

This is something that bothered me for a while now. Even though it's not
as bad as it used to be because we are not adding so much wrappers for
monitor commands as we used to. I mean, in libvirt the wrapper for a
monitor command has to be written by hand. Worse, whenever I'm adding a
wrapper I look at the QMP schema of it and let my muscle memory write
the wrapper.

However, it's not only what Markus already mentioned. Even if we
generated wrappers by a script, we need to be able to generate wrappers
for every single supported version of qemu.

For instance, if qemu version X has a command that accepts some set of
arguments and this set changes in version X+1 then libvirt needs both
wrappers and decides at runtime (depending on what version it is talking
to) what wrapper to use.

Unfortunately, I don't see any easy way out.

Michal



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-20 16:13 Making QEMU easier for management tools and applications Stefan Hajnoczi
  2019-12-20 21:07 ` Richard W.M. Jones
  2019-12-21  9:02 ` Markus Armbruster
@ 2019-12-24 13:00 ` Daniel P. Berrangé
  2020-01-02 14:22   ` Stefan Hajnoczi
  2020-01-02 15:10 ` Dr. David Alan Gilbert
  2020-01-07 17:11 ` Christophe de Dinechin
  4 siblings, 1 reply; 34+ messages in thread
From: Daniel P. Berrangé @ 2019-12-24 13:00 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Kevin Wolf, Peter Maydell, Denis V. Lunev, qemu-devel,
	Markus Armbruster, Paolo Bonzini, Marc-André Lureau,
	John Snow, Dominik Csapak

On Fri, Dec 20, 2019 at 04:13:59PM +0000, Stefan Hajnoczi wrote:
> Hi,
> QEMU presents a command-line interface and QMP monitor for
> applications to interact with.  Applications actually need API
> bindings in their programming language.  Bindings avoid reimplementing
> code to spawn a QEMU process and interact with QMP.  QEMU is kind of
> lazy and de facto relies on libvirt for API bindings.
> 
> Is it time for better QEMU APIs?
> 
> 1. We have qapi-schema.json.  Let's render to HTML and publish
> versioned documentation online.

You have patches to publish this to the website already which is
good for git master. We still need to get this published for
stable releases, ideally we would have up to publish it for our
pre-existing stable releases back a year or two too.

At the same time though this is insufficient. It is *really* hard
to understand how QAPI schema maps to either JSON or CLI args for
either QMP or the CLI. Most docs examples just talk in terms of
-drive. I think the qapi schema docs could benefit from more
work to provide real world example usage inline.


> 2. scripts/qmp/ contains command-line tools for QMP communication.
> They could use some polish and then be shipped.

The qmp-shell in particular is interesting. Libvirt has a similar
need for a simpler way to deal with QMP and "virsh qemu-monitor-command"
has notable overlap with functionality of qmp-shell.

I wonder if there's a way to spin this off so that we have have a
standard QMP client shell that can either talk direct to a UNIX
socket, or indirect via libvirt's QMP command passthrough APIs.

> 3. python/qemu/ contains Python modules for managing a QEMU process
> and QMP communication.  This should be packaged in distros and
> available on PyPI.
>
> 4. Go and Rust bindings would also be useful.  There is
> https://github.com/intel/govmm but I think it makes sense to keep it
> in qemu.git and provide an interface similar to our Python modules.

Presumably you mean bindings for the QMP commands ?  This will
quickly expand to mean bindings for every aspect of QEMU configuration
that's currently QAPI-ified. It will quickly become a very big job
to design, develop & maintain, especially if you want to support
multiple languages in a natural way.

> 5. A jailer is needed to isolate the QEMU process and vhost-user
> device backends using seccomp, Linux namespaces, and maybe
> SELinux/AppArmor.  We used to be able to rely on libvirt for QEMU
> security, but it's becoming a common task for any device backend and
> IMO should be its own launcher tool.

Based on experiance in libvirt, this is an even larger job than (4),
as the feature set here is huge.  Much of it directly ties into the
config problem, as to deal with SELinux / namespace setup the code
needs to understand what resources to provide access to. This
requires a way to express 100% coverage of all QEMU configuration
in use & analyse it to determine what resources it implies. So this
ties strongly into QAPI-ification completion.


On the libvirt side, we've a strong long term desire to move our
codebase from C to either Go or Rust. In doing this refactoring
and rewriting, we would quite likely end up with self-contained
packages / modules / APIs that can be used independantly of the
main libvirt API. Thus there's probably scope here to collaborate
between libvirt & QEMU communities to develop new APIs.

Some of this code could be directly QEMU related, but other parts
of the code are likely to be generic & not tied to QEMU at all.
For example libvirt's SELinux, namespace, cgroup setup code is used
by LXC too. Thus from the libvirt POV, some libraries might be
deliverable by QEMU project, but others might be deliverable by
libvirt project. Or might be scope for a indepent new project to
deliver pieces.

Essentially we can consider a stack

     +--------------+
     |  Libvirt API |
     +--------------+
           |
	   V
     +--------------+
     | Helper APIs  |
     +--------------+
           |
	   V
     +--------------+
     |  QEMU        |
     +--------------+

Right now, everything in "Helper APIs" is hidden inside libvirt.
Some of that conceptually ought to be deliverable by QEMU. Some
of that ought to be spun out as separate deliverables by libvirt
or another independant project.

> 6. A configuration file format is sorely needed so that guest
> configuration can be persisted and easily launched.  Today you have to
> create a shell script that executes a QEMU command-line, but this is
> suboptimal because sharing executable scripts is dangerous from a
> security perspective and is hard to parse or modify programmatically.

A configuration file is just one aspect of our broader configuration
problem.

Our CLI in particular suffers from trying to be "all things to all people"
and IMHO is a direct cause of the perception of QEMU as being overly
complex, hard to understand & burdened by legacy support. Of course we
didn't set out intentionally on this path, things just grew organically
we were not succesfully enough at tackling technical debt over time.

To pick on blockdev, as it has had the most growth, IIUC we have

  $QEMU FILENAME
  $QEMU -hda FILENAME
  $QEMU -drive if=ide,file=FILENAME
  $QEMU -drive if=none,file=FILENAME,id=disk0 -device virtio-blk,drive=disk0
  $QEMU -blockdev driver=raw,file.filename=FILENAME,id=disk0 -device virtio-blk,drive=disk0

IIUC, in (all/most) cases "FILENAME" can be either a plain filename,
or a "json:{....}" string, so that's giving us possibly as many as
10 ways to configure disks, with increasing generality.

If you were designing QEMU from a clean slate it is unlikely we'd
plan to have 10 syntaxes. A clean slate would probably provide
2 different syntaxes at most - one simple but convenient, and one
complex but fully expressive.

For every one there's going to be some set of users who think
their chosen syntax is ideal, and will thus be unhappy if we
cull one choice.  For the health of QEMU long term though, I
think we're going to have to accept the need to make some people
temporarily unhappy in order to become sustaininable over the
long term.


This ultimately all stems from the fact that the CLI design has
aimed to provide 100% coverage of all QEMU features for mgmt apps,
while at the same time as providing convenient syntax for humans.
Except it has actually failed in the former, because there's some
stuff you can only configured after launching QEMU via QMP commands.

We can reduce some of the maint burden by mapping "convenient syntax"
onto the more expressive (usually QAPI modelled) syntax internally,
but I think that alone is insufficient.

We should step back & consider the bigger question of what we want
QEMU's approach to configuration to be.

With my libvirt hat on, we have to support JSON for a large portion
of config because we need to use QMP hotplug. Thus if we could eliminate
the need to use the CLI syntax entirely, it would remove / simplify
libvirt's code base to work exclusively in terms of JSON.

With this in mind, I think we could reasonably declare a high level
goal that

 - CLI arg syntax is *exclusively* for human targetted convenient syntax
 - CLI arg syntax will *not* aim for 100% feature coverage
 - CLI arg syntax will not guarantee strict back compat but will not
   gratituitously break human usage
 - JSON syntax will provide 100% feature coverage for mmgmt apps and humans
 - JSON syntax will not be exposed via ARGV
 - JSON syntax will be usable via a config file
 - JSON syntax will be usable via QMP
 - JSON syntax will provide strict back compat, but with deprecations
   policy followed

This is more or less what our long term goal with QAPI has always
been. The sticking point has always been the tension with CLI arg
syntax back-compat. We need to decide how to cut the knot so that
we can complete the QAPI-ification in a forseeable amount of time.

> In many of these areas we already have a partial solution.  It just
> needs more work.  I think it would be worth the effort and the mental
> shift to really providing APIs that are easy to use by applications.
> 
> What do you think?

Personally I think QEMU configuration is the single biggest pain point
for dealing with QEMU that we're not investing enough resources in.

> Have I missed things that are needed?
> 
> Have I included things that are unnecessary?

Many of the things you list above are conceptually nice, but imply a
lot of extra work for maintainers, as they are expanding the scope of
the QEMU project. There's a risk this could divert resources away from
tackling technical debt in things we're already maintaining. The flip
side of course is that some of the things might attract new contributors
to the project, because new concepts are inevitably more interesting to
some people than tackling technical debt. IOW both are important and
we need to find a balance between them.

One of the serious long term issues we've had in QEMU is our inability
to finish jobs that we start. The configuration problem is the biggest
example or something we started JSON-ifying 10 years ago, but are still
an unknowable amount of time away from finishing.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-21  9:02 ` Markus Armbruster
  2019-12-23 15:04   ` Michal Prívozník
@ 2019-12-24 13:41   ` Daniel P. Berrangé
  2020-01-02 14:47   ` Stefan Hajnoczi
  2020-01-02 15:05   ` Dr. David Alan Gilbert
  3 siblings, 0 replies; 34+ messages in thread
From: Daniel P. Berrangé @ 2019-12-24 13:41 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Kevin Wolf, Peter Maydell, Denis V. Lunev, Cleber Rosa,
	Stefan Hajnoczi, qemu-devel, Marc-André Lureau,
	Paolo Bonzini, Dominik Csapak, John Snow, Eduardo Habkost

On Sat, Dec 21, 2019 at 10:02:23AM +0100, Markus Armbruster wrote:
> Stefan Hajnoczi <stefanha@gmail.com> writes:
> 
> > Hi,
> > QEMU presents a command-line interface and QMP monitor for
> > applications to interact with.  Applications actually need API
> > bindings in their programming language.  Bindings avoid reimplementing
> > code to spawn a QEMU process and interact with QMP.  QEMU is kind of
> > lazy and de facto relies on libvirt for API bindings.
> >
> > Is it time for better QEMU APIs?

> * scripts/qmp/qmp-shell
> 
>   Half-hearted attempt at a human-friendly wrapper around the JSON
>   syntax.  I have no use for this myself.

I use this fairly often as its a useful debugging / experimentation
/ trouble shooting tool. There's similar ish functionality in
virsh qemu-monitor-command. I think there's scope of a supported
tool here that can talk to libvirt or a UNIX socket for doing
QMP commands, with a friendlier syntax & pretty printing. 

> > 4. Go and Rust bindings would also be useful.  There is
> > https://github.com/intel/govmm but I think it makes sense to keep it
> > in qemu.git and provide an interface similar to our Python modules.
> 
> Mapping QAPI/QMP commands and events to function signatures isn't hard
> (the QAPI code generator does).  Two problems (at least):
> 
> 1. Leads to some pretty ridiculous functions.  Here's one:
> 
>     void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
>                              const char *device,
>                              const char *target,
>                              bool has_replaces, const char *replaces,
>                              MirrorSyncMode sync,
>                              bool has_speed, int64_t speed,
>                              bool has_granularity, uint32_t granularity,
>                              bool has_buf_size, int64_t buf_size,
>                              bool has_on_source_error,
>                              BlockdevOnError on_source_error,
>                              bool has_on_target_error, BlockdevOnError on_target_error,
>                              bool has_filter_node_name, const char *filter_node_name,
>                              bool has_copy_mode, MirrorCopyMode copy_mode, 
>                              bool has_auto_finalize, bool auto_finalize,
>                              bool has_auto_dismiss, bool auto_dismiss,
>                              Error **errp);
> 
>   We commonly use 'boxed': true for such beasts, which results in
>   functions like this one:
> 
>     void qmp_blockdev_add(BlockdevOptions *arg, Error **errp);
> 
> 2. Many schema changes that are nicely backward compatible in QMP are
>    anything but in such an "obvious" C API.  Adding optional arguments,
>    for instance, or changing integer type width.  The former is less of
>    an issue with 'boxed': true.
> 
> Perhaps less of an issue with dynamic languages.
> 
> I figure a static language would need much more expressive oomph than C
> to be a good target.  No idea how well Go or Rust bindings can work.

In libvirt we've got this kind of problem already and have two different
approaches to tackling it.

For the really complex cases with lots of conceptual nesting, we would
end up using XML.

For the simpler case where its just a set of flat parameters, we have
something we call "virTypedParameters" which is effectively a poor man's
hash table where the keys are parameter names, and the values are a
union accepting different types.

In the Go APIs for libvirt though we end up mapping both those approaches
into Go structs. The key reason why we are able todo this in Go and not
C, is that we don't promise ABI compat in Go bindings. We'll extend the
structs at will, and some times a struct field gets turned into a nested
struct which is an API break too. With Go apps typically bundle all their
3rd party deps & often lock themselves to a specific release tag, so they
don't need ABI compat in the same way apps want at C level.

> > 6. A configuration file format is sorely needed so that guest
> > configuration can be persisted and easily launched.  Today you have to
> > create a shell script that executes a QEMU command-line, but this is
> > suboptimal because sharing executable scripts is dangerous from a
> > security perspective and is hard to parse or modify programmatically.
> 
> No argument.  There is -readconfig, but it falls way short.
> 
> With command line QAPIfication, a real configuration file will be quite
> feasible.
> 
> The main reason for the lack of progress there is our dedication to
> backward compatibility.  A functional replacement of our CLI is a huge
> task already.  Throwing in backward compatibility makes it a daunting
> one.  Not least because nobody fully understands all the quirks.

I think we need to re-think how we're attacking this problem, because
despite heroic efforts in developing QAPI & converting code it use it,
we've still not got the finish line in sight after 10 years.

As I mentioned in my other reply I think we should target the CLI as
something exclusively for humans, with convenient syntax and more
relaxed back compat requirements. ie don't guarantee perfect back
compat for the CLI - just modest effor to not gratuitously break it
without cause. Humans using the CLI can adapt if they hit obscure
quirks.

Our back compat pain with the quirks is because we have mgmt apps using
the CLI which can't tolerate semantic changes without warning. We can't
be more relaxed with CLI back compat without getting mgmt apps away
from using this CLI.  We can get mgmt apps off using the CLI until they
have an alternative they can use.

The hard part is that developing the QAPI alternative is that its impl
might accidentaly cause breakage in existing CLI before mgmt apps can
switch to the QAPI based approach.



I'm wondering if we need to think about a temporary *throwaway* fork of
QEMU. I wasn't involved in the work, but as an outside observer I think
it is interesting to see how the NEMU project fork ultimately led to QEMU
moving forward with a new KConfig approach, and the integration of microvm.
They had the freedom to develop these new approaches with zero regard
to back compat or broader QEMU needs that might otherwise hold back dev.
Once the forked impl was mostly done, "unknown problems" had largely become
"known problems", and the work could be integrated back into QEMU with
much less risk of disruption.


Could such a temporary fork approach help us solve the QAPI-ification
end goal ?  It doesn't have to be a separate project - it could be a
temporary qemu-qapi.git on qemu-project.org or github for experimentation,
but still under umbrealla of QEMU.

It would free us to implement an idealized 100% QAPI-ified config
approach in the shortest possible time, with zero regard to back
compat of the current CLI.

We could develop support for this in libvirt in parallel, letting
us test the combination to gain confidence in the result.

Essentially we'll be aiming to flush out all the "unknown problems"
that hold us back today. As we get confidence we'll be able to merge
it back into QEMU proper with much lower risk of introducing suprising
compat problems.

At some level you can consider each subsys maintainer's branch a fork,
but those branches are really focused on stable code that's ready to
merge. An NEMU like fork is more aggressive than that, as it provided
a focused place for *experimentation* with no regard for immediate
merge to GIT master. The cost of such a fork approach of course is
that of trying to keep up2date with development in QEMU git master.

The benefit of being able to cut the back-compat knot might still
make this a win, allowing completion of QAPI-ification in a faster
timeframe.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-20 21:07 ` Richard W.M. Jones
@ 2020-01-02 11:26   ` Stefan Hajnoczi
  0 siblings, 0 replies; 34+ messages in thread
From: Stefan Hajnoczi @ 2020-01-02 11:26 UTC (permalink / raw)
  To: Richard W.M. Jones
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	qemu-devel, Markus Armbruster, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

[-- Attachment #1: Type: text/plain, Size: 478 bytes --]

On Fri, Dec 20, 2019 at 09:07:50PM +0000, Richard W.M. Jones wrote:
> On Fri, Dec 20, 2019 at 04:13:59PM +0000, Stefan Hajnoczi wrote:
> > 6. A configuration file format is sorely needed so that guest
> > configuration can be persisted and easily launched. 
> 
> Actually qemu already has that, but it's really half-assed, barely
> documented, and doesn't cover major features that qemu provides :-)

Yes, I mean a fully-working configuration file format :).

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-24 13:00 ` Daniel P. Berrangé
@ 2020-01-02 14:22   ` Stefan Hajnoczi
  0 siblings, 0 replies; 34+ messages in thread
From: Stefan Hajnoczi @ 2020-01-02 14:22 UTC (permalink / raw)
  To: Daniel P. Berrangé
  Cc: Kevin Wolf, Peter Maydell, Denis V. Lunev, qemu-devel,
	Markus Armbruster, Paolo Bonzini, Marc-André Lureau,
	John Snow, Dominik Csapak

[-- Attachment #1: Type: text/plain, Size: 11722 bytes --]

On Tue, Dec 24, 2019 at 01:00:35PM +0000, Daniel P. Berrangé wrote:
> On Fri, Dec 20, 2019 at 04:13:59PM +0000, Stefan Hajnoczi wrote:
> > Hi,
> > QEMU presents a command-line interface and QMP monitor for
> > applications to interact with.  Applications actually need API
> > bindings in their programming language.  Bindings avoid reimplementing
> > code to spawn a QEMU process and interact with QMP.  QEMU is kind of
> > lazy and de facto relies on libvirt for API bindings.
> > 
> > Is it time for better QEMU APIs?
> > 
> > 1. We have qapi-schema.json.  Let's render to HTML and publish
> > versioned documentation online.
> 
> You have patches to publish this to the website already which is
> good for git master. We still need to get this published for
> stable releases, ideally we would have up to publish it for our
> pre-existing stable releases back a year or two too.
> 
> At the same time though this is insufficient. It is *really* hard
> to understand how QAPI schema maps to either JSON or CLI args for
> either QMP or the CLI. Most docs examples just talk in terms of
> -drive. I think the qapi schema docs could benefit from more
> work to provide real world example usage inline.
> 
> 
> > 2. scripts/qmp/ contains command-line tools for QMP communication.
> > They could use some polish and then be shipped.
> 
> The qmp-shell in particular is interesting. Libvirt has a similar
> need for a simpler way to deal with QMP and "virsh qemu-monitor-command"
> has notable overlap with functionality of qmp-shell.
> 
> I wonder if there's a way to spin this off so that we have have a
> standard QMP client shell that can either talk direct to a UNIX
> socket, or indirect via libvirt's QMP command passthrough APIs.

A standard (and easier-to-use and powerful) QMP shell would be nice.

> > 3. python/qemu/ contains Python modules for managing a QEMU process
> > and QMP communication.  This should be packaged in distros and
> > available on PyPI.
> >
> > 4. Go and Rust bindings would also be useful.  There is
> > https://github.com/intel/govmm but I think it makes sense to keep it
> > in qemu.git and provide an interface similar to our Python modules.
> 
> Presumably you mean bindings for the QMP commands ?  This will
> quickly expand to mean bindings for every aspect of QEMU configuration
> that's currently QAPI-ified. It will quickly become a very big job
> to design, develop & maintain, especially if you want to support
> multiple languages in a natural way.

No, I mean just launching QEMU and communicating over QMP.

I think creating bindings for QMP commands is too much effort.  It would
duplicate the QAPI schema documentation because we'd need native API
documentation too.

I think clients should work at the QMP JSON level.

> > 5. A jailer is needed to isolate the QEMU process and vhost-user
> > device backends using seccomp, Linux namespaces, and maybe
> > SELinux/AppArmor.  We used to be able to rely on libvirt for QEMU
> > security, but it's becoming a common task for any device backend and
> > IMO should be its own launcher tool.
> 
> Based on experiance in libvirt, this is an even larger job than (4),
> as the feature set here is huge.  Much of it directly ties into the
> config problem, as to deal with SELinux / namespace setup the code
> needs to understand what resources to provide access to. This
> requires a way to express 100% coverage of all QEMU configuration
> in use & analyse it to determine what resources it implies. So this
> ties strongly into QAPI-ification completion.

I don't have a clear design in mind yet but I've learnt a little from
writing the virtiofsd namespace isolation and seccomp code.  The task is
really hard for a general-purpose QEMU binary that supports all
features, because configuring the sandbox with the minimal attack
surface requires knowledge of the feature set being used in this QEMU
invocation :(.

> On the libvirt side, we've a strong long term desire to move our
> codebase from C to either Go or Rust. In doing this refactoring
> and rewriting, we would quite likely end up with self-contained
> packages / modules / APIs that can be used independantly of the
> main libvirt API. Thus there's probably scope here to collaborate
> between libvirt & QEMU communities to develop new APIs.
> 
> Some of this code could be directly QEMU related, but other parts
> of the code are likely to be generic & not tied to QEMU at all.
> For example libvirt's SELinux, namespace, cgroup setup code is used
> by LXC too. Thus from the libvirt POV, some libraries might be
> deliverable by QEMU project, but others might be deliverable by
> libvirt project. Or might be scope for a indepent new project to
> deliver pieces.

I think duplication is in the nature of these isolation mechanisms.
systemd also supports most of the features necessary for isolation but
neither QEMU nor libvirt take advantage of systemd and it's not easy to
do so either.  I think it will be necessary to be pragmatic and look at
it on a case-by-case basis.  There will be some duplication.

> Essentially we can consider a stack
> 
>      +--------------+
>      |  Libvirt API |
>      +--------------+
>            |
> 	   V
>      +--------------+
>      | Helper APIs  |
>      +--------------+
>            |
> 	   V
>      +--------------+
>      |  QEMU        |
>      +--------------+
> 
> Right now, everything in "Helper APIs" is hidden inside libvirt.
> Some of that conceptually ought to be deliverable by QEMU. Some
> of that ought to be spun out as separate deliverables by libvirt
> or another independant project.

I would like QEMU to directly expose APIs without the need for another
level of APIs, where this makes sense.  It's inefficient and slow to
develop stable APIs in QEMU and then repeat that work again in higher
layers of the stack.  To some extent this is unavoidable but I think
QEMU is close to providing usable APIs and tools itself in the areas
I've outlined in this email thread.

By doing this we'll create a better experience for Kata Containers and
other programs that directly use QEMU.

> > 6. A configuration file format is sorely needed so that guest
> > configuration can be persisted and easily launched.  Today you have to
> > create a shell script that executes a QEMU command-line, but this is
> > suboptimal because sharing executable scripts is dangerous from a
> > security perspective and is hard to parse or modify programmatically.
> 
> A configuration file is just one aspect of our broader configuration
> problem.
> 
> Our CLI in particular suffers from trying to be "all things to all people"
> and IMHO is a direct cause of the perception of QEMU as being overly
> complex, hard to understand & burdened by legacy support. Of course we
> didn't set out intentionally on this path, things just grew organically
> we were not succesfully enough at tackling technical debt over time.
> 
> To pick on blockdev, as it has had the most growth, IIUC we have
> 
>   $QEMU FILENAME
>   $QEMU -hda FILENAME
>   $QEMU -drive if=ide,file=FILENAME
>   $QEMU -drive if=none,file=FILENAME,id=disk0 -device virtio-blk,drive=disk0
>   $QEMU -blockdev driver=raw,file.filename=FILENAME,id=disk0 -device virtio-blk,drive=disk0
> 
> IIUC, in (all/most) cases "FILENAME" can be either a plain filename,
> or a "json:{....}" string, so that's giving us possibly as many as
> 10 ways to configure disks, with increasing generality.
> 
> If you were designing QEMU from a clean slate it is unlikely we'd
> plan to have 10 syntaxes. A clean slate would probably provide
> 2 different syntaxes at most - one simple but convenient, and one
> complex but fully expressive.
> 
> For every one there's going to be some set of users who think
> their chosen syntax is ideal, and will thus be unhappy if we
> cull one choice.  For the health of QEMU long term though, I
> think we're going to have to accept the need to make some people
> temporarily unhappy in order to become sustaininable over the
> long term.

Agreed :).

> This ultimately all stems from the fact that the CLI design has
> aimed to provide 100% coverage of all QEMU features for mgmt apps,
> while at the same time as providing convenient syntax for humans.
> Except it has actually failed in the former, because there's some
> stuff you can only configured after launching QEMU via QMP commands.
> 
> We can reduce some of the maint burden by mapping "convenient syntax"
> onto the more expressive (usually QAPI modelled) syntax internally,
> but I think that alone is insufficient.
> 
> We should step back & consider the bigger question of what we want
> QEMU's approach to configuration to be.
> 
> With my libvirt hat on, we have to support JSON for a large portion
> of config because we need to use QMP hotplug. Thus if we could eliminate
> the need to use the CLI syntax entirely, it would remove / simplify
> libvirt's code base to work exclusively in terms of JSON.
> 
> With this in mind, I think we could reasonably declare a high level
> goal that
> 
>  - CLI arg syntax is *exclusively* for human targetted convenient syntax
>  - CLI arg syntax will *not* aim for 100% feature coverage
>  - CLI arg syntax will not guarantee strict back compat but will not
>    gratituitously break human usage
>  - JSON syntax will provide 100% feature coverage for mmgmt apps and humans
>  - JSON syntax will not be exposed via ARGV
>  - JSON syntax will be usable via a config file
>  - JSON syntax will be usable via QMP
>  - JSON syntax will provide strict back compat, but with deprecations
>    policy followed
> 
> This is more or less what our long term goal with QAPI has always
> been. The sticking point has always been the tension with CLI arg
> syntax back-compat. We need to decide how to cut the knot so that
> we can complete the QAPI-ification in a forseeable amount of time.

Sounds good.

> > In many of these areas we already have a partial solution.  It just
> > needs more work.  I think it would be worth the effort and the mental
> > shift to really providing APIs that are easy to use by applications.
> > 
> > What do you think?
> 
> Personally I think QEMU configuration is the single biggest pain point
> for dealing with QEMU that we're not investing enough resources in.
> 
> > Have I missed things that are needed?
> > 
> > Have I included things that are unnecessary?
> 
> Many of the things you list above are conceptually nice, but imply a
> lot of extra work for maintainers, as they are expanding the scope of
> the QEMU project. There's a risk this could divert resources away from
> tackling technical debt in things we're already maintaining. The flip
> side of course is that some of the things might attract new contributors
> to the project, because new concepts are inevitably more interesting to
> some people than tackling technical debt. IOW both are important and
> we need to find a balance between them.
> 
> One of the serious long term issues we've had in QEMU is our inability
> to finish jobs that we start. The configuration problem is the biggest
> example or something we started JSON-ifying 10 years ago, but are still
> an unknowable amount of time away from finishing.

I think now is the right time to do it because there is awareness and
willingness to find solutions both in libvirt and QEMU.  I'm optimistic
that progress can be made here.

The next step is a more detailed plan so that multiple people can work
on these tasks.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-21  9:02 ` Markus Armbruster
  2019-12-23 15:04   ` Michal Prívozník
  2019-12-24 13:41   ` Daniel P. Berrangé
@ 2020-01-02 14:47   ` Stefan Hajnoczi
  2020-01-16 11:03     ` Kashyap Chamarthy
  2020-01-02 15:05   ` Dr. David Alan Gilbert
  3 siblings, 1 reply; 34+ messages in thread
From: Stefan Hajnoczi @ 2020-01-02 14:47 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Cleber Rosa, qemu-devel, Eduardo Habkost, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

[-- Attachment #1: Type: text/plain, Size: 4227 bytes --]

On Sat, Dec 21, 2019 at 10:02:23AM +0100, Markus Armbruster wrote:
> Stefan Hajnoczi <stefanha@gmail.com> writes:
> 
> > Hi,
> > QEMU presents a command-line interface and QMP monitor for
> > applications to interact with.  Applications actually need API
> > bindings in their programming language.  Bindings avoid reimplementing
> > code to spawn a QEMU process and interact with QMP.  QEMU is kind of
> > lazy and de facto relies on libvirt for API bindings.
> >
> > Is it time for better QEMU APIs?
> >
> > 1. We have qapi-schema.json.  Let's render to HTML and publish
> > versioned documentation online.
> 
> Make can build docs/interop/qemu-qmp-ref.{7,html,info,pdf,txt}.  Grab
> the .html and go for it.  There's also qmp-ga-ref.

The missing step here is to integrate doc generation and upload into the
release process so the documentation is published.

Once my recent documentation publishing patches have been merged I'll
see if Mike Roth wants to extend the release scripts.

> > 2. scripts/qmp/ contains command-line tools for QMP communication.
> > They could use some polish and then be shipped.
> 
> MAINTAINERS blames them on me, but they're effectively unmaintained.
> Prerequisite for shipping: having a maintainer who actually gives a
> damn.
...
> * scripts/qmp/qmp-shell
> 
>   Half-hearted attempt at a human-friendly wrapper around the JSON
>   syntax.  I have no use for this myself.

I think this one is used by people.  John Snow comes to mind.

> > 3. python/qemu/ contains Python modules for managing a QEMU process
> > and QMP communication.  This should be packaged in distros and
> > available on PyPI.
> 
> Currently maintained by Eduardo and Cleber (cc'ed) under "Python
> scripts".
> 
> > 4. Go and Rust bindings would also be useful.  There is
> > https://github.com/intel/govmm but I think it makes sense to keep it
> > in qemu.git and provide an interface similar to our Python modules.
> 
> Mapping QAPI/QMP commands and events to function signatures isn't hard
> (the QAPI code generator does).  Two problems (at least):
> 
> 1. Leads to some pretty ridiculous functions.  Here's one:
> 
>     void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
>                              const char *device,
>                              const char *target,
>                              bool has_replaces, const char *replaces,
>                              MirrorSyncMode sync,
>                              bool has_speed, int64_t speed,
>                              bool has_granularity, uint32_t granularity,
>                              bool has_buf_size, int64_t buf_size,
>                              bool has_on_source_error,
>                              BlockdevOnError on_source_error,
>                              bool has_on_target_error, BlockdevOnError on_target_error,
>                              bool has_filter_node_name, const char *filter_node_name,
>                              bool has_copy_mode, MirrorCopyMode copy_mode, 
>                              bool has_auto_finalize, bool auto_finalize,
>                              bool has_auto_dismiss, bool auto_dismiss,
>                              Error **errp);
> 
>   We commonly use 'boxed': true for such beasts, which results in
>   functions like this one:
> 
>     void qmp_blockdev_add(BlockdevOptions *arg, Error **errp);
> 
> 2. Many schema changes that are nicely backward compatible in QMP are
>    anything but in such an "obvious" C API.  Adding optional arguments,
>    for instance, or changing integer type width.  The former is less of
>    an issue with 'boxed': true.
> 
> Perhaps less of an issue with dynamic languages.
> 
> I figure a static language would need much more expressive oomph than C
> to be a good target.  No idea how well Go or Rust bindings can work.

Most of what govmm does is build QEMU command-lines, not wrap QMP APIs.

Native QMP API bindings are harder and we'd need to see who really needs
this.  Languages with a built-in dict type like Python make working with
JSON-level QMP feasible.  In C it's painful though and native API
bindings are desirable.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-21  9:02 ` Markus Armbruster
                     ` (2 preceding siblings ...)
  2020-01-02 14:47   ` Stefan Hajnoczi
@ 2020-01-02 15:05   ` Dr. David Alan Gilbert
  2020-01-13 13:44     ` Markus Armbruster
  3 siblings, 1 reply; 34+ messages in thread
From: Dr. David Alan Gilbert @ 2020-01-02 15:05 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Cleber Rosa, Stefan Hajnoczi, qemu-devel, Marc-André Lureau,
	Paolo Bonzini, Dominik Csapak, John Snow, Eduardo Habkost

* Markus Armbruster (armbru@redhat.com) wrote:
> Stefan Hajnoczi <stefanha@gmail.com> writes:
> > 4. Go and Rust bindings would also be useful.  There is
> > https://github.com/intel/govmm but I think it makes sense to keep it
> > in qemu.git and provide an interface similar to our Python modules.
> 
> Mapping QAPI/QMP commands and events to function signatures isn't hard
> (the QAPI code generator does).  Two problems (at least):
> 
> 1. Leads to some pretty ridiculous functions.  Here's one:
> 
>     void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
>                              const char *device,
>                              const char *target,
>                              bool has_replaces, const char *replaces,
>                              MirrorSyncMode sync,
>                              bool has_speed, int64_t speed,
>                              bool has_granularity, uint32_t granularity,
>                              bool has_buf_size, int64_t buf_size,
>                              bool has_on_source_error,
>                              BlockdevOnError on_source_error,
>                              bool has_on_target_error, BlockdevOnError on_target_error,
>                              bool has_filter_node_name, const char *filter_node_name,
>                              bool has_copy_mode, MirrorCopyMode copy_mode, 
>                              bool has_auto_finalize, bool auto_finalize,
>                              bool has_auto_dismiss, bool auto_dismiss,
>                              Error **errp);

Those might not be as bad when mapped to other languages, all the
bool/value pairs would become Option<...>  so that removes that doubling.
The Error ** mechanism should somehow map onto functions returning a
normal Rust Result<> type.

Dave

>   We commonly use 'boxed': true for such beasts, which results in
>   functions like this one:
> 
>     void qmp_blockdev_add(BlockdevOptions *arg, Error **errp);
> 
> 2. Many schema changes that are nicely backward compatible in QMP are
>    anything but in such an "obvious" C API.  Adding optional arguments,
>    for instance, or changing integer type width.  The former is less of
>    an issue with 'boxed': true.
> 
> Perhaps less of an issue with dynamic languages.
> 
> I figure a static language would need much more expressive oomph than C
> to be a good target.  No idea how well Go or Rust bindings can work.
> 
> > 5. A jailer is needed to isolate the QEMU process and vhost-user
> > device backends using seccomp, Linux namespaces, and maybe
> > SELinux/AppArmor.  We used to be able to rely on libvirt for QEMU
> > security, but it's becoming a common task for any device backend and
> > IMO should be its own launcher tool.
> 
> Perhaps the libvirt refactoring effort can give us one.
> 
> > 6. A configuration file format is sorely needed so that guest
> > configuration can be persisted and easily launched.  Today you have to
> > create a shell script that executes a QEMU command-line, but this is
> > suboptimal because sharing executable scripts is dangerous from a
> > security perspective and is hard to parse or modify programmatically.
> 
> No argument.  There is -readconfig, but it falls way short.
> 
> With command line QAPIfication, a real configuration file will be quite
> feasible.
> 
> The main reason for the lack of progress there is our dedication to
> backward compatibility.  A functional replacement of our CLI is a huge
> task already.  Throwing in backward compatibility makes it a daunting
> one.  Not least because nobody fully understands all the quirks.
> 
> > In many of these areas we already have a partial solution.  It just
> > needs more work.  I think it would be worth the effort and the mental
> > shift to really providing APIs that are easy to use by applications.
> >
> > What do you think?
> >
> > Have I missed things that are needed?
> >
> > Have I included things that are unnecessary?
> 
> I feel we need to make up our minds what kind of interface(s) we want to
> provide.
> 
> We provide a low-level interface for management applications.  I feel we
> need to nail this one.  Its QMP part is okay, I think, the CLI part is
> not.
> 
> We also try to provide friendlier interfaces for human users.  Not only
> do we suck at it, we're also prone to create liabilities for later: much
> of the compatibility woes that have been holding us back are rooted in
> "convenience" features.  And don't get me started on the folly of us
> dabbling in GUI.
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-20 16:13 Making QEMU easier for management tools and applications Stefan Hajnoczi
                   ` (2 preceding siblings ...)
  2019-12-24 13:00 ` Daniel P. Berrangé
@ 2020-01-02 15:10 ` Dr. David Alan Gilbert
  2020-01-07 17:11 ` Christophe de Dinechin
  4 siblings, 0 replies; 34+ messages in thread
From: Dr. David Alan Gilbert @ 2020-01-02 15:10 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	qemu-devel, Markus Armbruster, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

* Stefan Hajnoczi (stefanha@gmail.com) wrote:

> 5. A jailer is needed to isolate the QEMU process and vhost-user
> device backends using seccomp, Linux namespaces, and maybe
> SELinux/AppArmor.  We used to be able to rely on libvirt for QEMU
> security, but it's becoming a common task for any device backend and
> IMO should be its own launcher tool.

It feels like there's two parts to this:
  a) The sandboxing at startup
  b) Then whatever comms is needed to allow things like hotplugging
    afterwards, passing fd's etc

you'd think (a) would be some code that would be shared between logs of
projects.

Dave

> 6. A configuration file format is sorely needed so that guest
> configuration can be persisted and easily launched.  Today you have to
> create a shell script that executes a QEMU command-line, but this is
> suboptimal because sharing executable scripts is dangerous from a
> security perspective and is hard to parse or modify programmatically.
> 
> In many of these areas we already have a partial solution.  It just
> needs more work.  I think it would be worth the effort and the mental
> shift to really providing APIs that are easy to use by applications.
> 
> What do you think?
> 
> Have I missed things that are needed?
> 
> Have I included things that are unnecessary?
> 
> Stefan
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-23 15:04   ` Michal Prívozník
@ 2020-01-07  9:36     ` Kevin Wolf
  2020-01-07 10:55       ` Michal Privoznik
  0 siblings, 1 reply; 34+ messages in thread
From: Kevin Wolf @ 2020-01-07  9:36 UTC (permalink / raw)
  To: Michal Prívozník
  Cc: Peter Maydell, Daniel P. Berrange, Denis V. Lunev, Cleber Rosa,
	Stefan Hajnoczi, Markus Armbruster, qemu-devel,
	Marc-André Lureau, Paolo Bonzini, Dominik Csapak, John Snow,
	Eduardo Habkost

Am 23.12.2019 um 16:04 hat Michal Prívozník geschrieben:
> On 12/21/19 10:02 AM, Markus Armbruster wrote:
> > Stefan Hajnoczi <stefanha@gmail.com> writes:
> > 
> 
> 
> >> 4. Go and Rust bindings would also be useful.  There is
> >> https://github.com/intel/govmm but I think it makes sense to keep it
> >> in qemu.git and provide an interface similar to our Python modules.
> > 
> > Mapping QAPI/QMP commands and events to function signatures isn't hard
> > (the QAPI code generator does).  Two problems (at least):
> > 
> > 1. Leads to some pretty ridiculous functions.  Here's one:
> > 
> >     void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
> >                              const char *device,
> >                              const char *target,
> >                              bool has_replaces, const char *replaces,
> >                              MirrorSyncMode sync,
> >                              bool has_speed, int64_t speed,
> >                              bool has_granularity, uint32_t granularity,
> >                              bool has_buf_size, int64_t buf_size,
> >                              bool has_on_source_error,
> >                              BlockdevOnError on_source_error,
> >                              bool has_on_target_error, BlockdevOnError on_target_error,
> >                              bool has_filter_node_name, const char *filter_node_name,
> >                              bool has_copy_mode, MirrorCopyMode copy_mode, 
> >                              bool has_auto_finalize, bool auto_finalize,
> >                              bool has_auto_dismiss, bool auto_dismiss,
> >                              Error **errp);
> > 
> >   We commonly use 'boxed': true for such beasts, which results in
> >   functions like this one:
> > 
> >     void qmp_blockdev_add(BlockdevOptions *arg, Error **errp);
> > 
> > 2. Many schema changes that are nicely backward compatible in QMP are
> >    anything but in such an "obvious" C API.  Adding optional arguments,
> >    for instance, or changing integer type width.  The former is less of
> >    an issue with 'boxed': true.
> > 
> > Perhaps less of an issue with dynamic languages.
> > 
> > I figure a static language would need much more expressive oomph than C
> > to be a good target.  No idea how well Go or Rust bindings can work.
> 
> This is something that bothered me for a while now. Even though it's not
> as bad as it used to be because we are not adding so much wrappers for
> monitor commands as we used to. I mean, in libvirt the wrapper for a
> monitor command has to be written by hand. Worse, whenever I'm adding a
> wrapper I look at the QMP schema of it and let my muscle memory write
> the wrapper.
> 
> However, it's not only what Markus already mentioned. Even if we
> generated wrappers by a script, we need to be able to generate wrappers
> for every single supported version of qemu.
> 
> For instance, if qemu version X has a command that accepts some set of
> arguments and this set changes in version X+1 then libvirt needs both
> wrappers and decides at runtime (depending on what version it is talking
> to) what wrapper to use.
> 
> Unfortunately, I don't see any easy way out.

The easy way out would be tying libvirt to a specific QEMU version. And
I'm only half joking.

If libvirt didn't exist yet and we needed a management library for QEMU,
what we would build now would probably not look much like libvirt looks
today. We wouldn't try to have basic support for every hypervisor out
there, but integrate it much closer with QEMU and avoid much of the
backwards compatibility requirements that the interface between QEMU and
libvirt has (which requires us to deal with compatibility twice for
everything).

Maybe it would even be part of the QEMU repository, allowing a single
patch series to implement a new feature in the system emulator and
expose it in the API immediately instead of waiting for the next QEMU
release before libvirt can even think about implementing support for it.

So should libvirt move in that direction? Do people actually still make
much use of its multi-hypervisor nature, or would it make sense to split
it into separate libraries for each hypervisor that can be much tighter
integrated with (a specific version of) the respective hypervisor?

Kevin



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-07  9:36     ` Kevin Wolf
@ 2020-01-07 10:55       ` Michal Privoznik
  2020-01-07 12:57         ` Kevin Wolf
  0 siblings, 1 reply; 34+ messages in thread
From: Michal Privoznik @ 2020-01-07 10:55 UTC (permalink / raw)
  To: Kevin Wolf
  Cc: Peter Maydell, Daniel P. Berrange, Denis V. Lunev, Cleber Rosa,
	Stefan Hajnoczi, Markus Armbruster, qemu-devel,
	Marc-André Lureau, Paolo Bonzini, Dominik Csapak, John Snow,
	Eduardo Habkost

On 1/7/20 10:36 AM, Kevin Wolf wrote:
> The easy way out would be tying libvirt to a specific QEMU version. And
> I'm only half joking.
> 
> If libvirt didn't exist yet and we needed a management library for QEMU,
> what we would build now would probably not look much like libvirt looks
> today. We wouldn't try to have basic support for every hypervisor out
> there, but integrate it much closer with QEMU and avoid much of the
> backwards compatibility requirements that the interface between QEMU and
> libvirt has (which requires us to deal with compatibility twice for
> everything).
> 

By doing this, you would force your consumers to implement compatibility 
layer each on their own. Freshly finished blockdev is a beautiful 
example - OpenStack, oVirt and whatnot - they all are/can use blockdev 
without even noticing, because the API provided by libvirt is stable and 
provides abstraction, i.e. you don't need to change anything in your 
domain XML to use blockdev.

Of course, you can apply the argument one more time and have mgmt 
application tied to a specific version of qemu. But even that is not 
good enough, because with backports version is just meaningless number.

> Maybe it would even be part of the QEMU repository, allowing a single
> patch series to implement a new feature in the system emulator and
> expose it in the API immediately instead of waiting for the next QEMU
> release before libvirt can even think about implementing support for it.

Thing is, it's not just qmp that a mgmt application has to master, it's 
also process managing (and with growing number of helper binaries this 
is not as trivial as fork() + exec()). This would need to be the bare 
minimum your API layer has to provide to be consumable by anybody.
But then you have some advanced subsystems to take care of (CGroups, 
SELinux, etc.) which are used heavily by OpenStack. oVirt and friends.

> 
> So should libvirt move in that direction? Do people actually still make
> much use of its multi-hypervisor nature, or would it make sense to split
> it into separate libraries for each hypervisor that can be much tighter
> integrated with (a specific version of) the respective hypervisor?

Truth to be told, I don't think libvirt is held back by its attempts to 
provide hypervisor agnostic APIs. Sure, it leads to some weirdness (e.g. 
different naming in libvirt and qemu), but as a libvirt developer I 
don't remember feeling blocked by this multi-hypervisor nature (not to 
mention that this has saved us couple of times).

Also, it would be not correct to think that a feature is implemented for 
all hypervisors in libvirt. I mean, when implementing a feature I 
usually implement it only for qemu driver and don't even look at other 
drivers (unless I'm doing a change in a core that causes build 
failures). On the other hand, I do sometimes review patches posted by 
developers from other companies which have interest in different 
hypervisors (e.g. there is a SUSE guy working on LXC driver, and another 
SUSE guy working on libxenlight (basically Xen)), so I do spend some 
time not working on qemu driver, but I'd say it's negligible.

Michal



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-07 10:55       ` Michal Privoznik
@ 2020-01-07 12:57         ` Kevin Wolf
  2020-01-07 17:53           ` Christophe de Dinechin
  0 siblings, 1 reply; 34+ messages in thread
From: Kevin Wolf @ 2020-01-07 12:57 UTC (permalink / raw)
  To: Michal Privoznik
  Cc: Peter Maydell, Daniel P. Berrange, Denis V. Lunev, Cleber Rosa,
	Stefan Hajnoczi, Markus Armbruster, qemu-devel,
	Marc-André Lureau, Paolo Bonzini, Dominik Csapak, John Snow,
	Eduardo Habkost

Am 07.01.2020 um 11:55 hat Michal Privoznik geschrieben:
> On 1/7/20 10:36 AM, Kevin Wolf wrote:
> > The easy way out would be tying libvirt to a specific QEMU version. And
> > I'm only half joking.
> > 
> > If libvirt didn't exist yet and we needed a management library for QEMU,
> > what we would build now would probably not look much like libvirt looks
> > today. We wouldn't try to have basic support for every hypervisor out
> > there, but integrate it much closer with QEMU and avoid much of the
> > backwards compatibility requirements that the interface between QEMU and
> > libvirt has (which requires us to deal with compatibility twice for
> > everything).
> 
> By doing this, you would force your consumers to implement compatibility
> layer each on their own. Freshly finished blockdev is a beautiful example -
> OpenStack, oVirt and whatnot - they all are/can use blockdev without even
> noticing, because the API provided by libvirt is stable and provides
> abstraction, i.e. you don't need to change anything in your domain XML to
> use blockdev.

Yes and no.

You could still keep using the same abstraction that libvirt has always
used while doing this. What my imaginary newly written management
library would do differently isn't necessarily the interface between
libvirt and applications, but getting rid of backwards compatibility
requirements in the interface between QEMU and libvirt.

But of course, blockdev isn't even a feature per se. It's getting the
abstraction right so that it's actually abstract enough to represent
everything. As long as libvirt keeps using an abstraction that is based
on simplistic setups, it won't be able to expose the full feature set of
QEMU. This is less than satisfying. In the long run, libvirt will have
to extend its abstraction to make full use of new features either way.

> Of course, you can apply the argument one more time and have mgmt
> application tied to a specific version of qemu. But even that is not
> good enough, because with backports version is just meaningless
> number.

I think this would be too much indeed.

> > Maybe it would even be part of the QEMU repository, allowing a single
> > patch series to implement a new feature in the system emulator and
> > expose it in the API immediately instead of waiting for the next QEMU
> > release before libvirt can even think about implementing support for it.
> 
> Thing is, it's not just qmp that a mgmt application has to master, it's also
> process managing (and with growing number of helper binaries this is not as
> trivial as fork() + exec()). This would need to be the bare minimum your API
> layer has to provide to be consumable by anybody.
> But then you have some advanced subsystems to take care of (CGroups,
> SELinux, etc.) which are used heavily by OpenStack. oVirt and friends.

Someone has to do this anyway. Note that here I'm still talking about
the hypothetical case where no libvirt existed yet.

If we cared only about OpenStack, oVirt and friends, this would still
all be QEMU-based, so not a big problem to have it tied to QEMU.

I'm not sure what this looks like in practice in libvirt: Are these
components shared between multiple hypervisor interfaces or is it only
for QEMU anyway?

If multiple hypervisors make use of it, how crazy would it be to imagine
reversing which project consumes which? Instead of having the libvirt
core consume the hypervisor-specific sublibraries, could a QEMU-specific
part live closer to QEMU and consume the libvirt core as an external
library?

I guess much of what I write in this thread is pure heresy. :-)
Maybe most of it isn't even useful. But maybe there is an idea or two in
it that are worth having a closer look at.

> > So should libvirt move in that direction? Do people actually still make
> > much use of its multi-hypervisor nature, or would it make sense to split
> > it into separate libraries for each hypervisor that can be much tighter
> > integrated with (a specific version of) the respective hypervisor?
> 
> Truth to be told, I don't think libvirt is held back by its attempts to
> provide hypervisor agnostic APIs. Sure, it leads to some weirdness (e.g.
> different naming in libvirt and qemu), but as a libvirt developer I don't
> remember feeling blocked by this multi-hypervisor nature (not to mention
> that this has saved us couple of times).

I would imagine so, because the problem doesn't become visible in the
daily work, but only in the bigger picture: The other hypervisors are
what prevent libvirt from being more tightly intergrated with QEMU.

This means that there is a boundary between QEMU and libvirt that makes
it really slow to get new features to the user. And both QEMU and
libvirt waste a lot of time for maintaining backwards compatibility in
things that wouldn't necessarily have to be stable interfaces if the
management library were developed in lockstep with QEMU.

> Also, it would be not correct to think that a feature is implemented for all
> hypervisors in libvirt. I mean, when implementing a feature I usually
> implement it only for qemu driver and don't even look at other drivers
> (unless I'm doing a change in a core that causes build failures). On the
> other hand, I do sometimes review patches posted by developers from other
> companies which have interest in different hypervisors (e.g. there is a SUSE
> guy working on LXC driver, and another SUSE guy working on libxenlight
> (basically Xen)), so I do spend some time not working on qemu driver, but
> I'd say it's negligible.

Time spent on non-QEMU isn't really my concern. Time spent maintaining
stable interface between QEMU and libvirt, and time spent waiting for
QEMU releases before libvirt development starts are my concern.

Kevin



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2019-12-20 16:13 Making QEMU easier for management tools and applications Stefan Hajnoczi
                   ` (3 preceding siblings ...)
  2020-01-02 15:10 ` Dr. David Alan Gilbert
@ 2020-01-07 17:11 ` Christophe de Dinechin
  2020-01-08 10:43   ` Kevin Wolf
  2020-01-13 16:30   ` Stefan Hajnoczi
  4 siblings, 2 replies; 34+ messages in thread
From: Christophe de Dinechin @ 2020-01-07 17:11 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	qemu-devel, Markus Armbruster, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

Hi Stefan,



> On 20 Dec 2019, at 17:13, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> 
> Hi,
> QEMU presents a command-line interface and QMP monitor for
> applications to interact with.  Applications actually need API
> bindings in their programming language.  Bindings avoid reimplementing
> code to spawn a QEMU process and interact with QMP.  QEMU is kind of
> lazy and de facto relies on libvirt for API bindings.
> 
> Is it time for better QEMU APIs?

I hear that often ;-)

After catching up with this thread, I think that there is a common,
intuitive sense that a number of topics are interconnected and
relate to the QEMU API, from command-line to language bindings
to sandboxing to configuration files, and even to rewriting in Rust…

A good question is: why do we believe that? Thinking about it, it’s
not entirely obvious, is it, that a C API has anything to do with a
configuration file, is it?

(I _believe_ that it is true, but I’m trying to explore why our intuition
tells us so)

My gut feeling at this point is that we are not talking about an API,
we are first talking about a _language_, with relatively complicated
and loosely defined semantics. There are signs of this language
everywhere, from some custom object-model to several distinct
syntaxen (from ever-more-complex command-line options to JSON).
In order to see what I mean with “relatively complicated semantics”,
see the recent discussions about -hda vs. -drive vs. -device vs. -blockdev…

It does not help that a large fraction of this language is used to describe
graphs, e.g. network or bus topology. Textual languages make it
notoriously difficult to describe graphs, forcing us to add intermediate
user-provided names e.g. to interconnect a guest device and the
associated host resource, or to precisely define how guest NICs
connect to various virtual networks or host interface cards.

We then have several dialects of what should be, at least conceptually,
the same language, including, but not limited, to an XML dialect, a virsh
dialect, a command-line dialect, a qmp dialect, a JSON dialect that is
partially incorporated in several of the above, etc…

Of note, several tools, including virsh, git, OpenShift or dnf, have started
moving away from the simple “list of switches” towards having sub-commands.
Some, like virsh, even switch to “shell mode” if called without arguments.
So maybe qemu should enter “shell mode” if given no argument, and
accept a command file as input… If only there was no backward
compatibility issue…

So I think that it might help, in the long run, to start defining the
language in question in some abstract way, and then to have rules
for how to transform that abstract language into concrete bindings.
This definition itself is not obvious (at least not to me). For
example, do we have, anywhere but in the C code, the specification
of how one can add a disk to qemu, and what it means?
Say, looking at qemu-options.def, how do I tell that -hda has
anything to do with -device or -blockdev or -help?

I think that the following piece of code from vl.c is revealing:

            case QEMU_OPTION_hda:
            case QEMU_OPTION_hdb:
            case QEMU_OPTION_hdc:
            case QEMU_OPTION_hdd:
                drive_add(IF_DEFAULT, popt->index - QEMU_OPTION_hda, optarg,
                          HD_OPTS);
                break;
            case QEMU_OPTION_blockdev:
                {
                    Visitor *v;
                    BlockdevOptionsQueueEntry *bdo;

                    v = qobject_input_visitor_new_str(optarg, "driver",
                                                      &error_fatal);

                    bdo = g_new(BlockdevOptionsQueueEntry, 1);
                    visit_type_BlockdevOptions(v, NULL, &bdo->bdo,
                                               &error_fatal);
                    visit_free(v);
                    loc_save(&bdo->loc);
                    QSIMPLEQ_INSERT_TAIL(&bdo_queue, bdo, entry);
                    break;
                }
            case QEMU_OPTION_drive:
                if (drive_def(optarg) == NULL) {
                    exit(1);
                }
                break;

Here, we have three cases related to disks in a way or another,
and three entirely different ways of doing things.

AFAICT, qemu already created several meta-languages to define
several aspects of the API, from qemu-options.def to qapi-schema.json.
But maybe at some point we need to go meta once more, and define
a language defining the API from which we could automatically
derive the various bindings, including FFI-style bindings for Rust and Go,
as well as some internal data structures. Ideally, that meta-definition
is something that could be shared between libvirt and qemu so that they
literally speak the same language. Or that could be used to automatically
build a REST interface.

A big issue, though, is that of compatibility. Doing the above starting
from scratch does not seem that complicated. Doing it in a way that
preserves a minimum of interoperability with earlier-generation
software is another ordeal.

So I think that Daniel is right. We may need at some point to start
a NEMU-style offshoot that does not attempt to be compatible,
but explores describing an increasing surface of the API using a
new meta-language from which we can generate, in a consistent
way, at least:

- C bindings
- Command-line options
- Shell bindings (or “HMP”)
- JSON schema or qom description
- Bindings in other languages (Rust, Go, Python)
- Networked versions of the API (socket, REST)
- Client-side code e.g. for libvirt.
- Serialization / deserialization, e.g. for configuration files
- Documentation, including man page and API docs
- Command-line help

At the most fundamental level, I think we need to describe:

- Values, e.g. how we represent names, sizes, paths, etc, possibly
with some user-friendly aspects, e.g. path shortcuts, memory units,
spelling shortcuts (e.g. being able to consistently say -blo for -blockdev
if that’s the shortest option that matches)
- Relations, e.g. how we represent “contains”, “derives from”, “needs”,
“one of”, “one or several”, “attaches to”…
- States, e.g. how do we represent the machine configuration,
or the desired new disk setting
- Verbs, e.g. how we represent “add”, “connect”, “remove”, “find”,
“start”, “notify”, etc. and how we describe the kind of input they need.
- Possibly more subtle things like support for transactions, commit/rollback,
i.e. “I want to add connect a virtual nic to some host vf, but if anything
along the way fails, I’d like all the cleanup to happen automatically)


> 
> 1. We have qapi-schema.json.  Let's render to HTML and publish
> versioned documentation online.

Added “Documentation” above to make sure this is preserved.
I agree with the assessment that the existing commenting state
is not good enough to make this that valuable at this point.
Notably missing are all the relation aspects.


> 
> 2. scripts/qmp/ contains command-line tools for QMP communication.
> They could use some polish and then be shipped.

Shouldn’t we first identify what they do and how to use them? ;-)

> 
> 3. python/qemu/ contains Python modules for managing a QEMU process
> and QMP communication.  This should be packaged in distros and
> available on PyPI.

That is probably somewhat independent from the rest, although
in my “meta” model, we should certainly be able to also generate
some python bindings.

> 
> 4. Go and Rust bindings would also be useful.  There is
> https://github.com/intel/govmm but I think it makes sense to keep it
> in qemu.git and provide an interface similar to our Python modules.

Same comment as above.

> 
> 5. A jailer is needed to isolate the QEMU process and vhost-user
> device backends using seccomp, Linux namespaces, and maybe
> SELinux/AppArmor.  We used to be able to rely on libvirt for QEMU
> security, but it's becoming a common task for any device backend and
> IMO should be its own launcher tool.

I believe that this “jailer” comment is interesting. It’s not really
related to a refactoring of APIs, it looks more like an API that we
forgot because libvirt did it, and we now realize that this may not
be the right place for some use cases.

> 
> 6. A configuration file format is sorely needed so that guest
> configuration can be persisted and easily launched.  Today you have to
> create a shell script that executes a QEMU command-line, but this is
> suboptimal because sharing executable scripts is dangerous from a
> security perspective and is hard to parse or modify programmatically.

Agreed. That’s one reason why I started talking about a “language” above.


> In many of these areas we already have a partial solution.  It just
> needs more work.  I think it would be worth the effort and the mental
> shift to really providing APIs that are easy to use by applications.
> 
> What do you think?

Overall, I think this is not a simple problem.

> 
> Have I missed things that are needed?

The main one for me is the meta-language I talked about above.

> 
> Have I included things that are unnecessary?

No, but some of the things that you included surprised me in an “API”
discussion, and that’s what launched me in this “meta-language” direction.


> 
> Stefan
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-07 12:57         ` Kevin Wolf
@ 2020-01-07 17:53           ` Christophe de Dinechin
  0 siblings, 0 replies; 34+ messages in thread
From: Christophe de Dinechin @ 2020-01-07 17:53 UTC (permalink / raw)
  To: Kevin Wolf
  Cc: Peter Maydell, Daniel P. Berrange, Denis V. Lunev, Cleber Rosa,
	Stefan Hajnoczi, Markus Armbruster, Eduardo Habkost, qemu-devel,
	Paolo Bonzini, Michal Privoznik, Marc-André Lureau,
	John Snow, Dominik Csapak



> On 7 Jan 2020, at 13:57, Kevin Wolf <kwolf@redhat.com> wrote:
> 
> Am 07.01.2020 um 11:55 hat Michal Privoznik geschrieben:
>> On 1/7/20 10:36 AM, Kevin Wolf wrote:
>>> The easy way out would be tying libvirt to a specific QEMU version. And
>>> I'm only half joking.
>>> 
>>> If libvirt didn't exist yet and we needed a management library for QEMU,
>>> what we would build now would probably not look much like libvirt looks
>>> today. We wouldn't try to have basic support for every hypervisor out
>>> there, but integrate it much closer with QEMU and avoid much of the
>>> backwards compatibility requirements that the interface between QEMU and
>>> libvirt has (which requires us to deal with compatibility twice for
>>> everything).
>> 
>> By doing this, you would force your consumers to implement compatibility
>> layer each on their own. Freshly finished blockdev is a beautiful example -
>> OpenStack, oVirt and whatnot - they all are/can use blockdev without even
>> noticing, because the API provided by libvirt is stable and provides
>> abstraction, i.e. you don't need to change anything in your domain XML to
>> use blockdev.
> 
> Yes and no.
> 
> You could still keep using the same abstraction that libvirt has always
> used while doing this. What my imaginary newly written management
> library would do differently isn't necessarily the interface between
> libvirt and applications, but getting rid of backwards compatibility
> requirements in the interface between QEMU and libvirt.
> 
> But of course, blockdev isn't even a feature per se. It's getting the
> abstraction right so that it's actually abstract enough to represent
> everything. As long as libvirt keeps using an abstraction that is based
> on simplistic setups, it won't be able to expose the full feature set of
> QEMU. This is less than satisfying. In the long run, libvirt will have
> to extend its abstraction to make full use of new features either way.
> 
>> Of course, you can apply the argument one more time and have mgmt
>> application tied to a specific version of qemu. But even that is not
>> good enough, because with backports version is just meaningless
>> number.
> 
> I think this would be too much indeed.
> 
>>> Maybe it would even be part of the QEMU repository, allowing a single
>>> patch series to implement a new feature in the system emulator and
>>> expose it in the API immediately instead of waiting for the next QEMU
>>> release before libvirt can even think about implementing support for it.
>> 
>> Thing is, it's not just qmp that a mgmt application has to master, it's also
>> process managing (and with growing number of helper binaries this is not as
>> trivial as fork() + exec()). This would need to be the bare minimum your API
>> layer has to provide to be consumable by anybody.
>> But then you have some advanced subsystems to take care of (CGroups,
>> SELinux, etc.) which are used heavily by OpenStack. oVirt and friends.
> 
> Someone has to do this anyway. Note that here I'm still talking about
> the hypothetical case where no libvirt existed yet.
> 
> If we cared only about OpenStack, oVirt and friends, this would still
> all be QEMU-based, so not a big problem to have it tied to QEMU.
> 
> I'm not sure what this looks like in practice in libvirt: Are these
> components shared between multiple hypervisor interfaces or is it only
> for QEMU anyway?
> 
> If multiple hypervisors make use of it, how crazy would it be to imagine
> reversing which project consumes which? Instead of having the libvirt
> core consume the hypervisor-specific sublibraries, could a QEMU-specific
> part live closer to QEMU and consume the libvirt core as an external
> library?
> 
> I guess much of what I write in this thread is pure heresy. :-)
> Maybe most of it isn't even useful. But maybe there is an idea or two in
> it that are worth having a closer look at.

Well, I don’t know if it is heresy, but at least as far as sandboxing / jailing
is concerned, I suggested something similar in earlier iterations of
the discussion. My way to say that I am part of your heresy, I guess.

> 
>>> So should libvirt move in that direction? Do people actually still make
>>> much use of its multi-hypervisor nature, or would it make sense to split
>>> it into separate libraries for each hypervisor that can be much tighter
>>> integrated with (a specific version of) the respective hypervisor?
>> 
>> Truth to be told, I don't think libvirt is held back by its attempts to
>> provide hypervisor agnostic APIs. Sure, it leads to some weirdness (e.g.
>> different naming in libvirt and qemu), but as a libvirt developer I don't
>> remember feeling blocked by this multi-hypervisor nature (not to mention
>> that this has saved us couple of times).
> 
> I would imagine so, because the problem doesn't become visible in the
> daily work, but only in the bigger picture: The other hypervisors are
> what prevent libvirt from being more tightly intergrated with QEMU.
> 
> This means that there is a boundary between QEMU and libvirt that makes
> it really slow to get new features to the user. And both QEMU and
> libvirt waste a lot of time for maintaining backwards compatibility in
> things that wouldn't necessarily have to be stable interfaces if the
> management library were developed in lockstep with QEMU.
> 
>> Also, it would be not correct to think that a feature is implemented for all
>> hypervisors in libvirt. I mean, when implementing a feature I usually
>> implement it only for qemu driver and don't even look at other drivers
>> (unless I'm doing a change in a core that causes build failures). On the
>> other hand, I do sometimes review patches posted by developers from other
>> companies which have interest in different hypervisors (e.g. there is a SUSE
>> guy working on LXC driver, and another SUSE guy working on libxenlight
>> (basically Xen)), so I do spend some time not working on qemu driver, but
>> I'd say it's negligible.
> 
> Time spent on non-QEMU isn't really my concern. Time spent maintaining
> stable interface between QEMU and libvirt, and time spent waiting for
> QEMU releases before libvirt development starts are my concern.
> 
> Kevin



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-07 17:11 ` Christophe de Dinechin
@ 2020-01-08 10:43   ` Kevin Wolf
  2020-01-08 11:40     ` Christophe de Dinechin
  2020-01-13 16:30   ` Stefan Hajnoczi
  1 sibling, 1 reply; 34+ messages in thread
From: Kevin Wolf @ 2020-01-08 10:43 UTC (permalink / raw)
  To: Christophe de Dinechin
  Cc: Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, Markus Armbruster, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

Am 07.01.2020 um 18:11 hat Christophe de Dinechin geschrieben:
> So I think that it might help, in the long run, to start defining the
> language in question in some abstract way, and then to have rules
> for how to transform that abstract language into concrete bindings.

I think this abstract language is QAPI. The problem is that we're not
even close to using QAPI for everything. Adding a new language on top of
QAPI instead isn't going to make the conversion process any faster.

> This definition itself is not obvious (at least not to me). For
> example, do we have, anywhere but in the C code, the specification
> of how one can add a disk to qemu, and what it means?
> Say, looking at qemu-options.def, how do I tell that -hda has
> anything to do with -device or -blockdev or -help?

BlockdevOptions in the QAPI schema is what tells you how it _really_
works. The connection to the various command line syntaxes isn't defined
in a declarative way because we don't have a QAPIfied command line yet.
I know that Markus wants to work on this, but I don't know how much time
he actually has to invest in it.

> I think that the following piece of code from vl.c is revealing:
> 
>             case QEMU_OPTION_hda:
>             case QEMU_OPTION_hdb:
>             case QEMU_OPTION_hdc:
>             case QEMU_OPTION_hdd:
>                 drive_add(IF_DEFAULT, popt->index - QEMU_OPTION_hda, optarg,
>                           HD_OPTS);
>                 break;
>             case QEMU_OPTION_blockdev:
>                 {
>                     Visitor *v;
>                     BlockdevOptionsQueueEntry *bdo;
> 
>                     v = qobject_input_visitor_new_str(optarg, "driver",
>                                                       &error_fatal);
> 
>                     bdo = g_new(BlockdevOptionsQueueEntry, 1);
>                     visit_type_BlockdevOptions(v, NULL, &bdo->bdo,
>                                                &error_fatal);
>                     visit_free(v);
>                     loc_save(&bdo->loc);
>                     QSIMPLEQ_INSERT_TAIL(&bdo_queue, bdo, entry);
>                     break;
>                 }
>             case QEMU_OPTION_drive:
>                 if (drive_def(optarg) == NULL) {
>                     exit(1);
>                 }
>                 break;
> 
> Here, we have three cases related to disks in a way or another,
> and three entirely different ways of doing things.

I would say two different ways because drive_add() is just a small
wrapper around drive_def() that overrides a few options.

Describing the semantics of the -drive way is hard. This is one of the
reasons why I would love to get rid of it and replace it with a new
user-friendly option that has a more direct mapping to the -blockdev
way, which in turn just is BlockdevOptions mapped 1:1 to the command
line.

> AFAICT, qemu already created several meta-languages to define
> several aspects of the API, from qemu-options.def to qapi-schema.json.
> But maybe at some point we need to go meta once more, and define
> a language defining the API from which we could automatically
> derive the various bindings, including FFI-style bindings for Rust and Go,
> as well as some internal data structures. Ideally, that meta-definition
> is something that could be shared between libvirt and qemu so that they
> literally speak the same language. Or that could be used to automatically
> build a REST interface.

I think adding an output for additional languages to the QAPI generator
shouldn't be too hard. It already creates multiple things from a single
schema (C data structures and command wrappers, schema introspection
data, documentation, and probably other things that I forgot).

libvirt already speaks QAPI, however without reusing the schema and the
generator from QEMU.

> A big issue, though, is that of compatibility. Doing the above starting
> from scratch does not seem that complicated. Doing it in a way that
> preserves a minimum of interoperability with earlier-generation
> software is another ordeal.

Indeed, this is the major reason why QAPI isn't as pervasive as it
should be.

> So I think that Daniel is right. We may need at some point to start
> a NEMU-style offshoot that does not attempt to be compatible,
> but explores describing an increasing surface of the API using a
> new meta-language from which we can generate, in a consistent
> way, at least:
> 
> - C bindings
> - Command-line options
> - Shell bindings (or “HMP”)
> - JSON schema or qom description
> - Bindings in other languages (Rust, Go, Python)
> - Networked versions of the API (socket, REST)
> - Client-side code e.g. for libvirt.
> - Serialization / deserialization, e.g. for configuration files
> - Documentation, including man page and API docs
> - Command-line help

I think the only thing in this list that can't obviously be covered
easily by QAPI is QOM. Or rather, it's covered by passing through
key=value lists without describing their structure - which, as far as I
understand, is mainly because QOM properties aren't necessarily static,
so we can't provide a statically defined interface for them. Probably
solvable in QEMU, but not without a major effort. In a fork that doesn't
care about compatibility, it should be easier.

> At the most fundamental level, I think we need to describe:
> 
> - Values, e.g. how we represent names, sizes, paths, etc, possibly
> with some user-friendly aspects, e.g. path shortcuts, memory units,
> spelling shortcuts (e.g. being able to consistently say -blo for -blockdev
> if that’s the shortest option that matches)

I don't think user-friendly shortcuts on the command line are "most
fundamental". Whether to accept -blo is an implementation detail of the
command line parser which translates a bunch of strings into QAPI
objects.

> - Relations, e.g. how we represent “contains”, “derives from”, “needs”,
> “one of”, “one or several”, “attaches to”…
> - States, e.g. how do we represent the machine configuration,
> or the desired new disk setting
> - Verbs, e.g. how we represent “add”, “connect”, “remove”, “find”,
> “start”, “notify”, etc. and how we describe the kind of input they need.
> - Possibly more subtle things like support for transactions, commit/rollback,
> i.e. “I want to add connect a virtual nic to some host vf, but if anything
> along the way fails, I’d like all the cleanup to happen automatically)

This sounds like a different approach from our current QAPI command set
(use a smaller set of operations that can work with a greater variety of
objects).

Does it actually provide more functionality, though?

Kevin



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-08 10:43   ` Kevin Wolf
@ 2020-01-08 11:40     ` Christophe de Dinechin
  2020-01-08 13:38       ` Kevin Wolf
  0 siblings, 1 reply; 34+ messages in thread
From: Christophe de Dinechin @ 2020-01-08 11:40 UTC (permalink / raw)
  To: Kevin Wolf
  Cc: Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, Markus Armbruster, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak



> On 8 Jan 2020, at 11:43, Kevin Wolf <kwolf@redhat.com> wrote:
> 
> Am 07.01.2020 um 18:11 hat Christophe de Dinechin geschrieben:
>> So I think that it might help, in the long run, to start defining the
>> language in question in some abstract way, and then to have rules
>> for how to transform that abstract language into concrete bindings.
> 
> I think this abstract language is QAPI.

Currently, yes, that’s the closest we have to a well-defined language,
I pointed it out in my mail, and there is qapi-code-gen.txt to prove it.

So it would certainly make sense for the language I’m describing to
be a forward evolution of QAPI, or something that is trivially transformed
to / from the existing json schema.

> The problem is that we're not even close to using QAPI for everything.

That’s the symptom of the problem. The problem as I see it is that
we could not. at least not with today’s QAPI (maybe I’m wrong).
A good example is shortcut command-line options. I don’t think
you can, today, derive a user-friendly command-line syntax from
qapi-schema.json, and even less so one that is compatible with what
we have today.

> Adding a new language on top of QAPI instead isn't going to make the
> conversion process any faster.

I fully agree that whatever language we select has to have an easy,
systematic forward path from QAPI, but also from the other
existing meta-languages, like the .def files used for options,
or build configuration (there is a hint of the connection between
the two in the option ‘if’ member in the .json files).

In other words, the language I have in mind would be something
that I could auto-generate from, say, the current qapi-schema.json
and the qemu-options.def. Maybe in the end, that would be simply
by having qemu-options.def being used to build qemu-options.json,
and then add the relevant entries in qemu-options.json. Maybe
there is a better way.

> 
>> This definition itself is not obvious (at least not to me). For
>> example, do we have, anywhere but in the C code, the specification
>> of how one can add a disk to qemu, and what it means?
>> Say, looking at qemu-options.def, how do I tell that -hda has
>> anything to do with -device or -blockdev or -help?
> 
> BlockdevOptions in the QAPI schema is what tells you how it _really_
> works.

Not really, IMO, in the sense that it gives me no clue as to how -hda
or -device relate to it.

An important point that you raise through your response, though, is
that the QAPI schema is only an IDL (interface definition language).
In other words, it can tell me what a BlockdevOptions look like on
the wire, but at least in the current state, it cannot tell me what happens
with it.

This is probably a good thing (it’s already complicated enough as is),
but it’s worth pointing out that, even sticking to a declarative language,
one could add constraints, as hinted by HAS_ARG in the .def file
(but we could have a richer constraints language, e.g. describing
-m option so that there can be at most one, the arg is a number,
can’t be negative or floating-point, and so on.

And of course, we could decide to not stick to a declarative language,
but to have a way to have some, possibly limited, processing
described in the language. I’m thinking notably of what is called
“desugaring” in other threads.

> The connection to the various command line syntaxes isn't defined
> in a declarative way because we don't have a QAPIfied command line yet.
> I know that Markus wants to work on this, but I don't know how much time
> he actually has to invest in it.

Agreed. As I said, that’s a symptom that QAPI is not the language
I am talking about, but at best a subset of it.

> 
>> I think that the following piece of code from vl.c is revealing:
>> 
>>            case QEMU_OPTION_hda:
>>            case QEMU_OPTION_hdb:
>>            case QEMU_OPTION_hdc:
>>            case QEMU_OPTION_hdd:
>>                drive_add(IF_DEFAULT, popt->index - QEMU_OPTION_hda, optarg,
>>                          HD_OPTS);
>>                break;
>>            case QEMU_OPTION_blockdev:
>>                {
>>                    Visitor *v;
>>                    BlockdevOptionsQueueEntry *bdo;
>> 
>>                    v = qobject_input_visitor_new_str(optarg, "driver",
>>                                                      &error_fatal);
>> 
>>                    bdo = g_new(BlockdevOptionsQueueEntry, 1);
>>                    visit_type_BlockdevOptions(v, NULL, &bdo->bdo,
>>                                               &error_fatal);
>>                    visit_free(v);
>>                    loc_save(&bdo->loc);
>>                    QSIMPLEQ_INSERT_TAIL(&bdo_queue, bdo, entry);
>>                    break;
>>                }
>>            case QEMU_OPTION_drive:
>>                if (drive_def(optarg) == NULL) {
>>                    exit(1);
>>                }
>>                break;
>> 
>> Here, we have three cases related to disks in a way or another,
>> and three entirely different ways of doing things.
> 
> I would say two different ways because drive_add() is just a small
> wrapper around drive_def() that overrides a few options.

Well, I stick to three, because one option has a test and an exit(1),
and the other does not. Which is the kind of inconsistencies a
well-defined language would get rid of.

> 
> Describing the semantics of the -drive way is hard. This is one of the
> reasons why I would love to get rid of it and replace it with a new
> user-friendly option that has a more direct mapping to the -blockdev
> way, which in turn just is BlockdevOptions mapped 1:1 to the command
> line.

I would love that too :-)

> 
>> AFAICT, qemu already created several meta-languages to define
>> several aspects of the API, from qemu-options.def to qapi-schema.json.
>> But maybe at some point we need to go meta once more, and define
>> a language defining the API from which we could automatically
>> derive the various bindings, including FFI-style bindings for Rust and Go,
>> as well as some internal data structures. Ideally, that meta-definition
>> is something that could be shared between libvirt and qemu so that they
>> literally speak the same language. Or that could be used to automatically
>> build a REST interface.
> 
> I think adding an output for additional languages to the QAPI generator
> shouldn't be too hard. It already creates multiple things from a single
> schema (C data structures and command wrappers, schema introspection
> data, documentation, and probably other things that I forgot).

Agreed.

> libvirt already speaks QAPI, however without reusing the schema and the
> generator from QEMU.

That’s another thread ;-)

> 
>> A big issue, though, is that of compatibility. Doing the above starting
>> from scratch does not seem that complicated. Doing it in a way that
>> preserves a minimum of interoperability with earlier-generation
>> software is another ordeal.
> 
> Indeed, this is the major reason why QAPI isn't as pervasive as it
> should be.

Probably, but also because QAPI does not address the needs of
some of the things a slightly more general language could do.

> 
>> So I think that Daniel is right. We may need at some point to start
>> a NEMU-style offshoot that does not attempt to be compatible,
>> but explores describing an increasing surface of the API using a
>> new meta-language from which we can generate, in a consistent
>> way, at least:
>> 
>> - C bindings
>> - Command-line options
>> - Shell bindings (or “HMP”)
>> - JSON schema or qom description
>> - Bindings in other languages (Rust, Go, Python)
>> - Networked versions of the API (socket, REST)
>> - Client-side code e.g. for libvirt.
>> - Serialization / deserialization, e.g. for configuration files
>> - Documentation, including man page and API docs
>> - Command-line help
> 
> I think the only thing in this list that can't obviously be covered
> easily by QAPI is QOM. Or rather, it's covered by passing through
> key=value lists without describing their structure

That’s close enough to me. (In my mind, that part was already “done”
by QAPI, even if in the convoluted way you describe)

> - which, as far as I
> understand, is mainly because QOM properties aren't necessarily static,
> so we can't provide a statically defined interface for them. Probably
> solvable in QEMU, but not without a major effort.

Or maybe extend the language so that it’s internal semantics
knows about this aspect of QOM?

> In a fork that doesn’t care about compatibility, it should be easier.
> 
>> At the most fundamental level, I think we need to describe:
>> 
>> - Values, e.g. how we represent names, sizes, paths, etc, possibly
>> with some user-friendly aspects, e.g. path shortcuts, memory units,
>> spelling shortcuts (e.g. being able to consistently say -blo for -blockdev
>> if that’s the shortest option that matches)
> 
> I don't think user-friendly shortcuts on the command line are "most
> fundamental".

I agree, my phrasing was poor. What I meant is that the language in
question should include a user-convenience dimension that is
not necessary (and not present) in QAPI. Agree the details are not
fundamental, I meant them as an illustration.

> Whether to accept -blo is an implementation detail of the
> command line parser which translates a bunch of strings into QAPI
> objects.

Yes.

> 
>> - Relations, e.g. how we represent “contains”, “derives from”, “needs”,
>> “one of”, “one or several”, “attaches to”…
>> - States, e.g. how do we represent the machine configuration,
>> or the desired new disk setting
>> - Verbs, e.g. how we represent “add”, “connect”, “remove”, “find”,
>> “start”, “notify”, etc. and how we describe the kind of input they need.
>> - Possibly more subtle things like support for transactions, commit/rollback,
>> i.e. “I want to add connect a virtual nic to some host vf, but if anything
>> along the way fails, I’d like all the cleanup to happen automatically)
> 
> This sounds like a different approach from our current QAPI command set

Well, except for purposefully trying to use a different wording to avoid
the risk of getting your mind stuck in one of the particular existing
meta-languages in QEMU, the approach is not very different.

- Relations already exist in QAPI for example: ’struct’, ‘union’, ‘alternate’

- States exist, today represented using JSON

- Verbs are called “commands” in the QAPI json files, and are what
follows DEF( in the .def file

- Transactions do not exist today that I know of, although we see
signs of them in discussions about the fact that this options destroys
that back end but that option does not.

> (use a smaller set of operations that can work with a greater variety of
> objects).

More like trying to find terminology that lets you reason about it
without being too mind-constrainted by existing stuff.


> Does it actually provide more functionality, though?

It’s not intended to provide more, but to require less to do the same thing.


Thanks,
Christophe



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-08 11:40     ` Christophe de Dinechin
@ 2020-01-08 13:38       ` Kevin Wolf
  2020-01-14 13:04         ` Markus Armbruster
  0 siblings, 1 reply; 34+ messages in thread
From: Kevin Wolf @ 2020-01-08 13:38 UTC (permalink / raw)
  To: Christophe de Dinechin
  Cc: Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, Markus Armbruster, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

Am 08.01.2020 um 12:40 hat Christophe de Dinechin geschrieben:
> > On 8 Jan 2020, at 11:43, Kevin Wolf <kwolf@redhat.com> wrote:
> > Am 07.01.2020 um 18:11 hat Christophe de Dinechin geschrieben:
> >> So I think that it might help, in the long run, to start defining the
> >> language in question in some abstract way, and then to have rules
> >> for how to transform that abstract language into concrete bindings.
> > 
> > I think this abstract language is QAPI.
> 
> Currently, yes, that’s the closest we have to a well-defined language,
> I pointed it out in my mail, and there is qapi-code-gen.txt to prove it.

Oh, did you? I must have missed that point, sorry.

> So it would certainly make sense for the language I’m describing to
> be a forward evolution of QAPI, or something that is trivially transformed
> to / from the existing json schema.

I guess I need to be more specific about what I mean with "QAPI". Of
course, many things you can't currently do in QAPI because they are not
implemented. When I'm talking about that QAPI should be used more, I'm
mostly talking about applying its core concepts and infrastructure with
the obvious extensions in some places, not about the existing
implementation.

So I guess "a forward evoluion of QAPI" is what I really have in mind.

> > The problem is that we're not even close to using QAPI for everything.
> 
> That’s the symptom of the problem. The problem as I see it is that
> we could not. at least not with today’s QAPI (maybe I’m wrong).
> A good example is shortcut command-line options. I don’t think
> you can, today, derive a user-friendly command-line syntax from
> qapi-schema.json, and even less so one that is compatible with what
> we have today.

Well, you can't because it's not implemented. Implementing it from
scratch without compatibility requirements would be relatively easy, and
we're in fact going to try it out with the qemu-storage-daemon.

But there is nothing in defining a command line option that would
inherently be different in structure from defining a QMP command, so I
would consider providing a command line option that is described by a
QAPI type not a problem, even though it's not implemented yet.

Compatibility is much harder, which is not the least because the
existing command line options are bad (primarily: using inconsistent
syntax). The question is if we should make bad options work unchanged in
the new infrastructure or if it wouldn't be better to replace them with
something that behaves the same as the rest.

> > Adding a new language on top of QAPI instead isn't going to make the
> > conversion process any faster.
> 
> I fully agree that whatever language we select has to have an easy,
> systematic forward path from QAPI, but also from the other
> existing meta-languages, like the .def files used for options,
> or build configuration (there is a hint of the connection between
> the two in the option ‘if’ member in the .json files).
> 
> In other words, the language I have in mind would be something
> that I could auto-generate from, say, the current qapi-schema.json
> and the qemu-options.def. Maybe in the end, that would be simply
> by having qemu-options.def being used to build qemu-options.json,
> and then add the relevant entries in qemu-options.json. Maybe
> there is a better way.

I would rather say that .def should go away and its content should
be converted to a JSON schema that becomes the new source format rather
than an intermediate generated file. There is nothing in .def files that
couldn't be represented in the schema.

> >> This definition itself is not obvious (at least not to me). For
> >> example, do we have, anywhere but in the C code, the specification
> >> of how one can add a disk to qemu, and what it means?
> >> Say, looking at qemu-options.def, how do I tell that -hda has
> >> anything to do with -device or -blockdev or -help?
> > 
> > BlockdevOptions in the QAPI schema is what tells you how it _really_
> > works.
> 
> Not really, IMO, in the sense that it gives me no clue as to how -hda
> or -device relate to it.
> 
> An important point that you raise through your response, though, is
> that the QAPI schema is only an IDL (interface definition language).
> In other words, it can tell me what a BlockdevOptions look like on
> the wire, but at least in the current state, it cannot tell me what happens
> with it.
> 
> This is probably a good thing (it’s already complicated enough as is),
> but it’s worth pointing out that, even sticking to a declarative language,
> one could add constraints, as hinted by HAS_ARG in the .def file
> (but we could have a richer constraints language, e.g. describing
> -m option so that there can be at most one, the arg is a number,
> can’t be negative or floating-point, and so on.

To a certain degree, QAPI does just that, by supporting different data
types for options. If necessary, I'm sure the type system could be
extended, but it's not clear to me to which degree we actually need
this.

Just drawing parallels from what we do for QMP commands, I imagine
something like this to describe the -m option as it actually exists in
qemu-options.def today:

##
# @m:
#
# Configure guest RAM.
#
# @size: initial amount of guest memory
# @slots: number of hotplug slots (default: none)
# @maxmem: maximum amount of guest memory (default: none)
##
{ 'cmdline-option': 'm',
  'arguments': {
    'size': 'size',
    'slots': '*int',
    'maxmem': '*size'
  },
  'default-key': 'size',
  'repeat': 'OVERRIDE' }

HAS_ARG is automatically covered simply by 'arguments' being non-empty.

(Markus probably already has a prototype with slightly different syntax,
but the idea should be similar.)

> And of course, we could decide to not stick to a declarative language,
> but to have a way to have some, possibly limited, processing
> described in the language. I’m thinking notably of what is called
> “desugaring” in other threads.

We could. But is it actually worth inventing a new programming language?
I think this is something that should be done in C code even in the
future. I think what we're looking for is a way to describe interfaces,
not implementations.

Even just for adding more sophisticated constraints, like in the example
above that maxmem >= size, it's questionable whether doing this in the
schema provides enough value for actually implementing it there.

> >> A big issue, though, is that of compatibility. Doing the above starting
> >> from scratch does not seem that complicated. Doing it in a way that
> >> preserves a minimum of interoperability with earlier-generation
> >> software is another ordeal.
> > 
> > Indeed, this is the major reason why QAPI isn't as pervasive as it
> > should be.
> 
> Probably, but also because QAPI does not address the needs of
> some of the things a slightly more general language could do.

So what are the things that you would like to describe, but can't see an
easy way to actually describe them with QAPI concepts?

> >> So I think that Daniel is right. We may need at some point to start
> >> a NEMU-style offshoot that does not attempt to be compatible,
> >> but explores describing an increasing surface of the API using a
> >> new meta-language from which we can generate, in a consistent
> >> way, at least:
> >> 
> >> - C bindings
> >> - Command-line options
> >> - Shell bindings (or “HMP”)
> >> - JSON schema or qom description
> >> - Bindings in other languages (Rust, Go, Python)
> >> - Networked versions of the API (socket, REST)
> >> - Client-side code e.g. for libvirt.
> >> - Serialization / deserialization, e.g. for configuration files
> >> - Documentation, including man page and API docs
> >> - Command-line help
> > 
> > I think the only thing in this list that can't obviously be covered
> > easily by QAPI is QOM. Or rather, it's covered by passing through
> > key=value lists without describing their structure
> 
> That’s close enough to me. (In my mind, that part was already “done”
> by QAPI, even if in the convoluted way you describe)

That's not really close enough. If you are happy with this level of
abstraction, you can define any command line option to be either a flag
or take a string as its argument and be done. This is obviously not very
helpful because it says nothing about the structure of that string. In
the same way, QAPI can't say anything about the structure of a QOM
object, and I think that's a problem.

> > - which, as far as I
> > understand, is mainly because QOM properties aren't necessarily static,
> > so we can't provide a statically defined interface for them. Probably
> > solvable in QEMU, but not without a major effort.
> 
> Or maybe extend the language so that it’s internal semantics
> knows about this aspect of QOM?

My point is that for example you can't generate a C struct (which is
statically defined) from something that has a dynamic structure. The
best you could do is fall back to key=value even in the C source, both
key and value being strings. But then you still have to parse these
strings manually, so it doesn't actually help much compared to a state
without C bindings.

Maybe what could be done is covering at least the static properties and
then having key=value for the dynamic part (which should be the
exception).

> >> - Relations, e.g. how we represent “contains”, “derives from”, “needs”,
> >> “one of”, “one or several”, “attaches to”…
> >> - States, e.g. how do we represent the machine configuration,
> >> or the desired new disk setting
> >> - Verbs, e.g. how we represent “add”, “connect”, “remove”, “find”,
> >> “start”, “notify”, etc. and how we describe the kind of input they need.
> >> - Possibly more subtle things like support for transactions, commit/rollback,
> >> i.e. “I want to add connect a virtual nic to some host vf, but if anything
> >> along the way fails, I’d like all the cleanup to happen automatically)
> > 
> > This sounds like a different approach from our current QAPI command set
> 
> Well, except for purposefully trying to use a different wording to avoid
> the risk of getting your mind stuck in one of the particular existing
> meta-languages in QEMU, the approach is not very different.

I didn't necessarily mean relations/state/verbs, which obviously exist,
but the examples that seemed to express things in a deliberately
different way from what we have today.

> - Transactions do not exist today that I know of, although we see
> signs of them in discussions about the fact that this options destroys
> that back end but that option does not.

We have a 'transaction' QMP command, but they are not an integral part
of the language. Considering the trouble to implement transactional
commands, I suppose we don't even want it to be a fundamental part of
the language.

> > Does it actually provide more functionality, though?
> 
> It’s not intended to provide more, but to require less to do the same thing.

Though we always need to keep in mind that if requiring less for future
additions requires a huge effort now, the investment may pay off only in
a very distant future (if the abstraction we build even survives until
then).

Kevin



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-02 15:05   ` Dr. David Alan Gilbert
@ 2020-01-13 13:44     ` Markus Armbruster
  0 siblings, 0 replies; 34+ messages in thread
From: Markus Armbruster @ 2020-01-13 13:44 UTC (permalink / raw)
  To: Dr. David Alan Gilbert
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Cleber Rosa, Stefan Hajnoczi, qemu-devel, Eduardo Habkost,
	Paolo Bonzini, Marc-André Lureau, John Snow, Dominik Csapak

"Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:

> * Markus Armbruster (armbru@redhat.com) wrote:
>> Stefan Hajnoczi <stefanha@gmail.com> writes:
>> > 4. Go and Rust bindings would also be useful.  There is
>> > https://github.com/intel/govmm but I think it makes sense to keep it
>> > in qemu.git and provide an interface similar to our Python modules.
>> 
>> Mapping QAPI/QMP commands and events to function signatures isn't hard
>> (the QAPI code generator does).  Two problems (at least):
>> 
>> 1. Leads to some pretty ridiculous functions.  Here's one:
>> 
>>     void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
>>                              const char *device,
>>                              const char *target,
>>                              bool has_replaces, const char *replaces,
>>                              MirrorSyncMode sync,
>>                              bool has_speed, int64_t speed,
>>                              bool has_granularity, uint32_t granularity,
>>                              bool has_buf_size, int64_t buf_size,
>>                              bool has_on_source_error,
>>                              BlockdevOnError on_source_error,
>>                              bool has_on_target_error, BlockdevOnError on_target_error,
>>                              bool has_filter_node_name, const char *filter_node_name,
>>                              bool has_copy_mode, MirrorCopyMode copy_mode, 
>>                              bool has_auto_finalize, bool auto_finalize,
>>                              bool has_auto_dismiss, bool auto_dismiss,
>>                              Error **errp);
>
> Those might not be as bad when mapped to other languages, all the
> bool/value pairs would become Option<...>  so that removes that doubling.
> The Error ** mechanism should somehow map onto functions returning a
> normal Rust Result<> type.

Fifteen parameters even with the has_FOO and the errp dropped.
Still ridiculous enough for me.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-07 17:11 ` Christophe de Dinechin
  2020-01-08 10:43   ` Kevin Wolf
@ 2020-01-13 16:30   ` Stefan Hajnoczi
  1 sibling, 0 replies; 34+ messages in thread
From: Stefan Hajnoczi @ 2020-01-13 16:30 UTC (permalink / raw)
  To: Christophe de Dinechin
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	qemu-devel, Markus Armbruster, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

[-- Attachment #1: Type: text/plain, Size: 2552 bytes --]

On Tue, Jan 07, 2020 at 06:11:13PM +0100, Christophe de Dinechin wrote:
> > On 20 Dec 2019, at 17:13, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> So I think that Daniel is right. We may need at some point to start
> a NEMU-style offshoot that does not attempt to be compatible,
> but explores describing an increasing surface of the API using a
> new meta-language from which we can generate, in a consistent
> way, at least:
> 
> - C bindings
> - Command-line options
> - Shell bindings (or “HMP”)
> - JSON schema or qom description
> - Bindings in other languages (Rust, Go, Python)
> - Networked versions of the API (socket, REST)
> - Client-side code e.g. for libvirt.
> - Serialization / deserialization, e.g. for configuration files
> - Documentation, including man page and API docs
> - Command-line help
> 
> At the most fundamental level, I think we need to describe:
> 
> - Values, e.g. how we represent names, sizes, paths, etc, possibly
> with some user-friendly aspects, e.g. path shortcuts, memory units,
> spelling shortcuts (e.g. being able to consistently say -blo for -blockdev
> if that’s the shortest option that matches)
> - Relations, e.g. how we represent “contains”, “derives from”, “needs”,
> “one of”, “one or several”, “attaches to”…
> - States, e.g. how do we represent the machine configuration,
> or the desired new disk setting
> - Verbs, e.g. how we represent “add”, “connect”, “remove”, “find”,
> “start”, “notify”, etc. and how we describe the kind of input they need.
> - Possibly more subtle things like support for transactions, commit/rollback,
> i.e. “I want to add connect a virtual nic to some host vf, but if anything
> along the way fails, I’d like all the cleanup to happen automatically)

Extending QAPI to achieve these things is a possibility.

If we afford ourselves the luxury of breaking backwards compatibility
then I would instead use the opportunity to eliminate complexity:
1. Get rid of the CLI
2. Get rid of HMP
3. No per-command C bindings, just a qmp_call() API
4. No configuration file, just a sequence of QMP commands

The new QEMU would be very different and solely focussed on QMP (or a
standards-based RPC system).

It's not very fun working on projects that have a lot of custom
infrastructure.  Making a one-time change requires a lot of learning
weird infrastructure that you won't use often.  We already have too much
of this and it slows down QEMU development.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-08 13:38       ` Kevin Wolf
@ 2020-01-14 13:04         ` Markus Armbruster
  2020-01-14 17:31           ` Christophe de Dinechin
  0 siblings, 1 reply; 34+ messages in thread
From: Markus Armbruster @ 2020-01-14 13:04 UTC (permalink / raw)
  To: Kevin Wolf
  Cc: Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, Marc-André Lureau,
	Christophe de Dinechin, Paolo Bonzini, John Snow, Dominik Csapak

Kevin Wolf <kwolf@redhat.com> writes:

> Am 08.01.2020 um 12:40 hat Christophe de Dinechin geschrieben:
>> > On 8 Jan 2020, at 11:43, Kevin Wolf <kwolf@redhat.com> wrote:
>> > Am 07.01.2020 um 18:11 hat Christophe de Dinechin geschrieben:
>> >> So I think that it might help, in the long run, to start defining the
>> >> language in question in some abstract way, and then to have rules
>> >> for how to transform that abstract language into concrete bindings.
>> > 
>> > I think this abstract language is QAPI.
>> 
>> Currently, yes, that’s the closest we have to a well-defined language,
>> I pointed it out in my mail, and there is qapi-code-gen.txt to prove it.
>
> Oh, did you? I must have missed that point, sorry.
>
>> So it would certainly make sense for the language I’m describing to
>> be a forward evolution of QAPI, or something that is trivially transformed
>> to / from the existing json schema.
>
> I guess I need to be more specific about what I mean with "QAPI". Of
> course, many things you can't currently do in QAPI because they are not
> implemented. When I'm talking about that QAPI should be used more, I'm
> mostly talking about applying its core concepts and infrastructure with
> the obvious extensions in some places, not about the existing
> implementation.
>
> So I guess "a forward evoluion of QAPI" is what I really have in mind.
>
>> > The problem is that we're not even close to using QAPI for everything.
>> 
>> That’s the symptom of the problem. The problem as I see it is that
>> we could not. at least not with today’s QAPI (maybe I’m wrong).
>> A good example is shortcut command-line options. I don’t think
>> you can, today, derive a user-friendly command-line syntax from
>> qapi-schema.json, and even less so one that is compatible with what
>> we have today.
>
> Well, you can't because it's not implemented. Implementing it from
> scratch without compatibility requirements would be relatively easy, and
> we're in fact going to try it out with the qemu-storage-daemon.

Yes.

Prior art:

    Presentation
    KVM Forum 2017: Towards a More Expressive and Introspectable QEMU
    Command Line
    https://www.youtube.com/watch?v=gtpOLQgnwug
    https://www.linux-kvm.org/images/f/f2/Armbru-qapi-cmdline_1.pdf

    RFC patches
    https://lists.nongnu.org/archive/html/qemu-devel/2017-10/msg00209.html
    Message-Id: <20171002152552.27999-1-armbru@redhat.com>
    https://repo.or.cz/qemu/armbru.git/shortlog/refs/heads/qapi-cmdline

> But there is nothing in defining a command line option that would
> inherently be different in structure from defining a QMP command, so I
> would consider providing a command line option that is described by a
> QAPI type not a problem, even though it's not implemented yet.
>
> Compatibility is much harder, which is not the least because the
> existing command line options are bad (primarily: using inconsistent
> syntax).

Semantics are just as problematic.

>          The question is if we should make bad options work unchanged in
> the new infrastructure or if it wouldn't be better to replace them with
> something that behaves the same as the rest.
>
>> > Adding a new language on top of QAPI instead isn't going to make the
>> > conversion process any faster.
>> 
>> I fully agree that whatever language we select has to have an easy,
>> systematic forward path from QAPI, but also from the other
>> existing meta-languages, like the .def files used for options,

You mean the qemu-options.hx.  qemu-options.def is generated from it.

We used to have qmp-commands.hx, until QAPI swallowed it.  Command line
QAPIfication will swallow qemu-options.hx.

>> or build configuration (there is a hint of the connection between
>> the two in the option ‘if’ member in the .json files).

I'm not sure what exactly you have in mind for build configuration.

>> In other words, the language I have in mind would be something
>> that I could auto-generate from, say, the current qapi-schema.json
>> and the qemu-options.def. Maybe in the end, that would be simply
>> by having qemu-options.def being used to build qemu-options.json,
>> and then add the relevant entries in qemu-options.json. Maybe
>> there is a better way.
>
> I would rather say that .def should go away and its content should
> be converted to a JSON schema that becomes the new source format rather
> than an intermediate generated file. There is nothing in .def files that
> couldn't be represented in the schema.

Yes.

>> >> This definition itself is not obvious (at least not to me). For
>> >> example, do we have, anywhere but in the C code, the specification
>> >> of how one can add a disk to qemu, and what it means?
>> >> Say, looking at qemu-options.def, how do I tell that -hda has
>> >> anything to do with -device or -blockdev or -help?
>> > 
>> > BlockdevOptions in the QAPI schema is what tells you how it _really_
>> > works.
>> 
>> Not really, IMO, in the sense that it gives me no clue as to how -hda
>> or -device relate to it.
>> 
>> An important point that you raise through your response, though, is
>> that the QAPI schema is only an IDL (interface definition language).
>> In other words, it can tell me what a BlockdevOptions look like on
>> the wire, but at least in the current state, it cannot tell me what happens
>> with it.

True.

>> This is probably a good thing (it’s already complicated enough as is),
>> but it’s worth pointing out that, even sticking to a declarative language,
>> one could add constraints, as hinted by HAS_ARG in the .def file
>> (but we could have a richer constraints language, e.g. describing
>> -m option so that there can be at most one, the arg is a number,
>> can’t be negative or floating-point, and so on.
>
> To a certain degree, QAPI does just that, by supporting different data
> types for options. If necessary, I'm sure the type system could be
> extended, but it's not clear to me to which degree we actually need
> this.

QAPI is much more expressive than qemu-options.hx.  All the latter can
do is "has option argument" and "option is arch-specific".  QAPI lets
you specify an option argument's structure, and supports compile-time
conditionals.  It can't do things like "must specify either argument A
or B", or "numeric argument C must not exceed argument D", or "multiple
options combine vs. last one wins".  Mostly because "it’s already
complicated enough as is".

> Just drawing parallels from what we do for QMP commands, I imagine
> something like this to describe the -m option as it actually exists in
> qemu-options.def today:
>
> ##
> # @m:
> #
> # Configure guest RAM.
> #
> # @size: initial amount of guest memory
> # @slots: number of hotplug slots (default: none)
> # @maxmem: maximum amount of guest memory (default: none)
> ##
> { 'cmdline-option': 'm',
>   'arguments': {
>     'size': 'size',
>     'slots': '*int',
>     'maxmem': '*size'
>   },
>   'default-key': 'size',
>   'repeat': 'OVERRIDE' }
>
> HAS_ARG is automatically covered simply by 'arguments' being non-empty.
>
> (Markus probably already has a prototype with slightly different syntax,
> but the idea should be similar.)

A similar example from my RFC patches:

##
# @--add-fd:
# !texinfo
# -add-fd fd=@var{fd},set=@var{set}[,opaque=@var{opaque}]@*
#
# Add a file descriptor to an fd set.  Valid options are:
#
# @table @option
# @item fd=@var{fd}
# This option defines the file descriptor of which a duplicate is added to fd set.
# The file descriptor cannot be stdin, stdout, or stderr.
# @item set=@var{set}
# This option defines the ID of the fd set to add the file descriptor to.
# @item opaque=@var{opaque}
# This option defines a free-form string that can be used to describe @var{fd}.
# @end table
#
# You can open an image using pre-opened file descriptors from an fd set:
# @example
# qemu-system-i386
# -add-fd fd=3,set=2,opaque="rdwr:/path/to/file"
# -add-fd fd=4,set=2,opaque="rdonly:/path/to/file"
# -drive file=/dev/fdset/2,index=0,media=disk
# @end example
# !end texinfo
##
{ 'option': '--add-fd',
  'data': { 'fd': 'int', 'set': 'int', '*opaque': 'str' },
  'help': [
"-add-fd fd=fd,set=set[,opaque=opaque]",
"                Add 'fd' to fd 'set'"] }

>> And of course, we could decide to not stick to a declarative language,
>> but to have a way to have some, possibly limited, processing
>> described in the language. I’m thinking notably of what is called
>> “desugaring” in other threads.
>
> We could. But is it actually worth inventing a new programming language?
> I think this is something that should be done in C code even in the
> future. I think what we're looking for is a way to describe interfaces,
> not implementations.

Sane sugar can be described declaratively as macro expansion.

Less than sane sugar takes code.

Desugaring in code promotes (accidental) abandonment of sanity.

That said, we have to keep things simple to succeed.  Declarative
desugaring seems beyond our reach.

> Even just for adding more sophisticated constraints, like in the example
> above that maxmem >= size, it's questionable whether doing this in the
> schema provides enough value for actually implementing it there.

Yes.

>> >> A big issue, though, is that of compatibility. Doing the above starting
>> >> from scratch does not seem that complicated. Doing it in a way that
>> >> preserves a minimum of interoperability with earlier-generation
>> >> software is another ordeal.
>> > 
>> > Indeed, this is the major reason why QAPI isn't as pervasive as it
>> > should be.
>> 
>> Probably, but also because QAPI does not address the needs of
>> some of the things a slightly more general language could do.
>
> So what are the things that you would like to describe, but can't see an
> easy way to actually describe them with QAPI concepts?
>
>> >> So I think that Daniel is right. We may need at some point to start
>> >> a NEMU-style offshoot that does not attempt to be compatible,
>> >> but explores describing an increasing surface of the API using a
>> >> new meta-language from which we can generate, in a consistent
>> >> way, at least:
>> >> 
>> >> - C bindings
>> >> - Command-line options
>> >> - Shell bindings (or “HMP”)
>> >> - JSON schema or qom description
>> >> - Bindings in other languages (Rust, Go, Python)
>> >> - Networked versions of the API (socket, REST)
>> >> - Client-side code e.g. for libvirt.
>> >> - Serialization / deserialization, e.g. for configuration files
>> >> - Documentation, including man page and API docs
>> >> - Command-line help
>> > 
>> > I think the only thing in this list that can't obviously be covered
>> > easily by QAPI is QOM. Or rather, it's covered by passing through
>> > key=value lists without describing their structure
>> 
>> That’s close enough to me. (In my mind, that part was already “done”
>> by QAPI, even if in the convoluted way you describe)
>
> That's not really close enough. If you are happy with this level of
> abstraction, you can define any command line option to be either a flag
> or take a string as its argument and be done. This is obviously not very
> helpful because it says nothing about the structure of that string.

Similar to all QMP commands taking a single 'any' argument.

>                                                                     In
> the same way, QAPI can't say anything about the structure of a QOM
> object, and I think that's a problem.
>
>> > - which, as far as I
>> > understand, is mainly because QOM properties aren't necessarily static,
>> > so we can't provide a statically defined interface for them. Probably
>> > solvable in QEMU, but not without a major effort.
>> 
>> Or maybe extend the language so that it’s internal semantics
>> knows about this aspect of QOM?
>
> My point is that for example you can't generate a C struct (which is
> statically defined) from something that has a dynamic structure. The
> best you could do is fall back to key=value even in the C source, both
> key and value being strings. But then you still have to parse these
> strings manually, so it doesn't actually help much compared to a state
> without C bindings.

QOM and QAPI sabotage each other.  Ironic, considering they were dreamed
up by the same guy :)

QAPI is compile-time static by design.

QOM is run-time dynamic by design.  Some support for static definitions
has been grafted on since.

We use QAPI type system escapes for QOM.  Defeats QAPI introspection and
doc generation.  We provide separate QOM introspection instead, which is
clumsier and less expressive.  QOM documentation doesn't exist.

> Maybe what could be done is covering at least the static properties and
> then having key=value for the dynamic part (which should be the
> exception).

To make this worthwhile, we'd have to replace dynamic QOM properties by
static ones when possible.  Monumental task.

>> >> - Relations, e.g. how we represent “contains”, “derives from”, “needs”,
>> >> “one of”, “one or several”, “attaches to”…
>> >> - States, e.g. how do we represent the machine configuration,
>> >> or the desired new disk setting
>> >> - Verbs, e.g. how we represent “add”, “connect”, “remove”, “find”,
>> >> “start”, “notify”, etc. and how we describe the kind of input they need.
>> >> - Possibly more subtle things like support for transactions, commit/rollback,
>> >> i.e. “I want to add connect a virtual nic to some host vf, but if anything
>> >> along the way fails, I’d like all the cleanup to happen automatically)
>> > 
>> > This sounds like a different approach from our current QAPI command set
>> 
>> Well, except for purposefully trying to use a different wording to avoid
>> the risk of getting your mind stuck in one of the particular existing
>> meta-languages in QEMU, the approach is not very different.
>
> I didn't necessarily mean relations/state/verbs, which obviously exist,
> but the examples that seemed to express things in a deliberately
> different way from what we have today.
>
>> - Transactions do not exist today that I know of, although we see
>> signs of them in discussions about the fact that this options destroys
>> that back end but that option does not.
>
> We have a 'transaction' QMP command, but they are not an integral part
> of the language. Considering the trouble to implement transactional
> commands, I suppose we don't even want it to be a fundamental part of
> the language.
>
>> > Does it actually provide more functionality, though?
>> 
>> It’s not intended to provide more, but to require less to do the same thing.
>
> Though we always need to keep in mind that if requiring less for future
> additions requires a huge effort now, the investment may pay off only in
> a very distant future (if the abstraction we build even survives until
> then).

Worse is better.

http://dreamsongs.com/RiseOfWorseIsBetter.html



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-14 13:04         ` Markus Armbruster
@ 2020-01-14 17:31           ` Christophe de Dinechin
  2020-01-15  9:20             ` Markus Armbruster
  0 siblings, 1 reply; 34+ messages in thread
From: Christophe de Dinechin @ 2020-01-14 17:31 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, Marc-André Lureau,
	Paolo Bonzini, John Snow, Dominik Csapak

I started cutting some stuff out.

> On 14 Jan 2020, at 14:04, Markus Armbruster <armbru@redhat.com> wrote:
> 
> Prior art:
> 
>    Presentation
>    KVM Forum 2017: Towards a More Expressive and Introspectable QEMU
>    Command Line
>    https://www.youtube.com/watch?v=gtpOLQgnwug
>    https://www.linux-kvm.org/images/f/f2/Armbru-qapi-cmdline_1.pdf
> 
>    RFC patches
>    https://lists.nongnu.org/archive/html/qemu-devel/2017-10/msg00209.html
>    Message-Id: <20171002152552.27999-1-armbru@redhat.com>
>    https://repo.or.cz/qemu/armbru.git/shortlog/refs/heads/qapi-cmdline

Very useful, thanks.

> 
>> Compatibility is much harder, which is not the least because the
>> existing command line options are bad (primarily: using inconsistent
>> syntax).
> 
> Semantics are just as problematic.

Yes, but often you can leverage something (e.g. JSON, XML, etc)

We could decide that the new QAPI language is a custom implementation
of Python ;-) Or more seriously, we could have some part of the JSON
schema that are “escapes” to C code that we put in some generated file.
I used {{ }} below for this escape, in which case the semantics would be
that of C plus some specific macros we insert in it.

{ 'option': '--add-fd',
 'data': { 'fd': 'int', 'set': 'int', '*opaque': 'str' },
 'help': [
"-add-fd fd=fd,set=set[,opaque=opaque]",
"                Add 'fd' to fd 'set'”] 
 ‘validate’ : {{
    error_if(QOPT.fd < 0, “fd option cannot be negative”);
    error_if(QOPT.fd <= STDERR_FILENO, “fd cannot be a standard I/O stream”);
 }},
 ‘exec’ : {{
    if (do_add_fd(QOPT.fd, QOPT.fset, QOPT.opaque) < 0) {
        exit(1);
    }
 }},

I’m not necessarily saying this is a good idea, mind you.

> 
> You mean the qemu-options.hx.  qemu-options.def is generated from it.

Oh, I had missed that, thanks.

>>> or build configuration (there is a hint of the connection between
>>> the two in the option ‘if’ member in the .json files).
> 
> I'm not sure what exactly you have in mind for build configuration.

I meant what you pointed out wrt. options, that this “language” needs to
know about the qemu configuration. There is ‘if’ today, you pointed
out arch-specific for options. But the configuration itself is yet
another “language” with its own semantics… which JSON knows
nothing about.

> 
>>> In other words, the language I have in mind would be something
>>> that I could auto-generate from, say, the current qapi-schema.json
>>> and the qemu-options.def. Maybe in the end, that would be simply
>>> by having qemu-options.def being used to build qemu-options.json,
>>> and then add the relevant entries in qemu-options.json. Maybe
>>> there is a better way.
>> 
>> I would rather say that .def should go away and its content should
>> be converted to a JSON schema that becomes the new source format rather
>> than an intermediate generated file. There is nothing in .def files that
>> couldn't be represented in the schema.
> 
> Yes.

Agreed too. The meta-schema would need to be extended.

>> 
>> To a certain degree, QAPI does just that, by supporting different data
>> types for options. If necessary, I'm sure the type system could be
>> extended, but it's not clear to me to which degree we actually need
>> this.
> 
> QAPI is much more expressive than qemu-options.hx.  All the latter can
> do is "has option argument" and "option is arch-specific".  QAPI lets
> you specify an option argument's structure, and supports compile-time
> conditionals.  It can't do things like "must specify either argument A
> or B", or "numeric argument C must not exceed argument D", or "multiple
> options combine vs. last one wins".  Mostly because "it’s already
> complicated enough as is".

Understood.


>> 
>> We could. But is it actually worth inventing a new programming language?
>> I think this is something that should be done in C code even in the
>> future. I think what we're looking for is a way to describe interfaces,
>> not implementations.
> 
> Sane sugar can be described declaratively as macro expansion.
> 
> Less than sane sugar takes code.
> 
> Desugaring in code promotes (accidental) abandonment of sanity.
> 
> That said, we have to keep things simple to succeed.  Declarative
> desugaring seems beyond our reach.

Well, “keeping things simple” is a complicated thing.

> 
>> Even just for adding more sophisticated constraints, like in the example
>> above that maxmem >= size, it's questionable whether doing this in the
>> schema provides enough value for actually implementing it there.
> 
> Yes.

If we do it, yes. If we delegate to an existing language (I showed C above),
maybe it adds clarity without too much of an additional cost.

> 
>>>> I think the only thing in this list that can't obviously be covered
>>>> easily by QAPI is QOM. Or rather, it's covered by passing through
>>>> key=value lists without describing their structure
>>> 
>>> That’s close enough to me. (In my mind, that part was already “done”
>>> by QAPI, even if in the convoluted way you describe)
>> 
>> That's not really close enough. If you are happy with this level of
>> abstraction, you can define any command line option to be either a flag
>> or take a string as its argument and be done. This is obviously not very
>> helpful because it says nothing about the structure of that string.
> 
> Similar to all QMP commands taking a single 'any' argument.

Hmmm, I guess I have not understood what you meant by
"QOM could not be covered easily by QAPI" then.

We don’t want the QAPI to let arbitrary fields of a QOM object
be modified, do we?

As for the “public” aspects of a QOM object, that is static if it
comes from QAPI, so why couldn’t we define it there?


> 
>>                                                                    In
>> the same way, QAPI can't say anything about the structure of a QOM
>> object, and I think that's a problem.
>> 
>>>> - which, as far as I
>>>> understand, is mainly because QOM properties aren't necessarily static,
>>>> so we can't provide a statically defined interface for them. Probably
>>>> solvable in QEMU, but not without a major effort.
>>> 
>>> Or maybe extend the language so that it’s internal semantics
>>> knows about this aspect of QOM?
>> 
>> My point is that for example you can't generate a C struct (which is
>> statically defined) from something that has a dynamic structure. The
>> best you could do is fall back to key=value even in the C source, both
>> key and value being strings. But then you still have to parse these
>> strings manually, so it doesn't actually help much compared to a state
>> without C bindings.

I really don’t understand that point. To me, all these operations
seem relatively simple to generate.

I think that what confuses me is when you write “something that has a
dynamic structure”. I understand that as referring to the structure
defined in the schema. But the schema itself is static. So you can
generate static code from it, and it’s already done today.

Another hypothesis on my side is that we don’t want, ever, to
have the API provide for example the option to create its own
arbitrary QOM classes, or to tag arbitrary properties to an object,
where by “arbitrary” I mean not explicitly mentioned somewhere in
something like the schema.

So I suspect you are talking about something else.


> 
> QOM and QAPI sabotage each other.  Ironic, considering they were dreamed
> up by the same guy :)
> 
> QAPI is compile-time static by design.
> 
> QOM is run-time dynamic by design.  Some support for static definitions
> has been grafted on since.
> 
> We use QAPI type system escapes for QOM.  Defeats QAPI introspection and
> doc generation.  We provide separate QOM introspection instead, which is
> clumsier and less expressive.  QOM documentation doesn't exist.

But back to the original discussion about management tools,
do we let upper layers tag their own arbitrary stuff in QOM objects?


> 
>> Maybe what could be done is covering at least the static properties and
>> then having key=value for the dynamic part (which should be the
>> exception).
> 
> To make this worthwhile, we'd have to replace dynamic QOM properties by
> static ones when possible.  Monumental task.

I’m sure you are right, but it’s hard for me to evaluate, given how
many ways there are to access an object. Naively, grepping for
set_prop and for new_with_prop does not give me that many hits.


> 
>>>>> - Relations, e.g. how we represent “contains”, “derives from”, “needs”,
>>>>> “one of”, “one or several”, “attaches to”…
>>>>> - States, e.g. how do we represent the machine configuration,
>>>>> or the desired new disk setting
>>>>> - Verbs, e.g. how we represent “add”, “connect”, “remove”, “find”,
>>>>> “start”, “notify”, etc. and how we describe the kind of input they need.
>>>>> - Possibly more subtle things like support for transactions, commit/rollback,
>>>>> i.e. “I want to add connect a virtual nic to some host vf, but if anything
>>>>> along the way fails, I’d like all the cleanup to happen automatically)
>>>> 
>>>> This sounds like a different approach from our current QAPI command set
>>> 
>>> Well, except for purposefully trying to use a different wording to avoid
>>> the risk of getting your mind stuck in one of the particular existing
>>> meta-languages in QEMU, the approach is not very different.
>> 
>> I didn't necessarily mean relations/state/verbs, which obviously exist,
>> but the examples that seemed to express things in a deliberately
>> different way from what we have today.
>> 
>>> - Transactions do not exist today that I know of, although we see
>>> signs of them in discussions about the fact that this options destroys
>>> that back end but that option does not.
>> 
>> We have a 'transaction' QMP command, but they are not an integral part
>> of the language. Considering the trouble to implement transactional
>> commands, I suppose we don't even want it to be a fundamental part of
>> the language.
>> 
>>>> Does it actually provide more functionality, though?
>>> 
>>> It’s not intended to provide more, but to require less to do the same thing.
>> 
>> Though we always need to keep in mind that if requiring less for future
>> additions requires a huge effort now, the investment may pay off only in
>> a very distant future (if the abstraction we build even survives until
>> then).
> 
> Worse is better.
> 
> http://dreamsongs.com/RiseOfWorseIsBetter.html

You know that I positively hate this ;-)

Well, I guess we can expand the schema. #ILoveJSON.


Thanks
Christophe




^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-14 17:31           ` Christophe de Dinechin
@ 2020-01-15  9:20             ` Markus Armbruster
  2020-01-15  9:34               ` Christophe de Dinechin
  2020-01-15  9:35               ` Marc-André Lureau
  0 siblings, 2 replies; 34+ messages in thread
From: Markus Armbruster @ 2020-01-15  9:20 UTC (permalink / raw)
  To: Christophe de Dinechin
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Stefan Hajnoczi, Markus Armbruster, qemu-devel, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

Christophe de Dinechin <dinechin@redhat.com> writes:

> I started cutting some stuff out.
>
>> On 14 Jan 2020, at 14:04, Markus Armbruster <armbru@redhat.com> wrote:
>> 
>> Prior art:
>> 
>>    Presentation
>>    KVM Forum 2017: Towards a More Expressive and Introspectable QEMU
>>    Command Line
>>    https://www.youtube.com/watch?v=gtpOLQgnwug
>>    https://www.linux-kvm.org/images/f/f2/Armbru-qapi-cmdline_1.pdf
>> 
>>    RFC patches
>>    https://lists.nongnu.org/archive/html/qemu-devel/2017-10/msg00209.html
>>    Message-Id: <20171002152552.27999-1-armbru@redhat.com>
>>    https://repo.or.cz/qemu/armbru.git/shortlog/refs/heads/qapi-cmdline
>
> Very useful, thanks.
>
>> 
>>> Compatibility is much harder, which is not the least because the
>>> existing command line options are bad (primarily: using inconsistent
>>> syntax).
>> 
>> Semantics are just as problematic.
>
> Yes, but often you can leverage something (e.g. JSON, XML, etc)
>
> We could decide that the new QAPI language is a custom implementation
> of Python ;-) Or more seriously, we could have some part of the JSON
> schema that are “escapes” to C code that we put in some generated file.
> I used {{ }} below for this escape, in which case the semantics would be
> that of C plus some specific macros we insert in it.
>
> { 'option': '--add-fd',
>  'data': { 'fd': 'int', 'set': 'int', '*opaque': 'str' },
>  'help': [
> "-add-fd fd=fd,set=set[,opaque=opaque]",
> "                Add 'fd' to fd 'set'”] 
>  ‘validate’ : {{
>     error_if(QOPT.fd < 0, “fd option cannot be negative”);
>     error_if(QOPT.fd <= STDERR_FILENO, “fd cannot be a standard I/O stream”);
>  }},
>  ‘exec’ : {{
>     if (do_add_fd(QOPT.fd, QOPT.fset, QOPT.opaque) < 0) {
>         exit(1);
>     }
>  }},
>
> I’m not necessarily saying this is a good idea, mind you.

Precedence: the value of 'if' is also a C snippet.

>> You mean the qemu-options.hx.  qemu-options.def is generated from it.
>
> Oh, I had missed that, thanks.
>
>>>> or build configuration (there is a hint of the connection between
>>>> the two in the option ‘if’ member in the .json files).
>> 
>> I'm not sure what exactly you have in mind for build configuration.
>
> I meant what you pointed out wrt. options, that this “language” needs to
> know about the qemu configuration. There is ‘if’ today, you pointed
> out arch-specific for options. But the configuration itself is yet
> another “language” with its own semantics… which JSON knows
> nothing about.

QAPI 'if' is a pragmatic design balancing simplicty with expressiveness.

It's part of the QAPI language except for the actual conditions.  The
QAPI generator blindly wraps whatever it generates for the thing
carrying an 'if': 'COND' in #if COND ... #endif /* COND */.  In other
words, we express "what is conditional" in the QAPI language, but resort
to the C language for expressing the actual conditionals.

PRO:
* Supports arbitrary #if.
* QAPI language impact is light: just another (string-valued) property
  of definitions and members.
* Generated code is the same regardless of QEMU configuration.

CON:
* Generating anything but C is problematic.  Currently that's just
  documentation.  Having C conditionals there isn't exactly great, but
  tolerable.

>>>> In other words, the language I have in mind would be something
>>>> that I could auto-generate from, say, the current qapi-schema.json
>>>> and the qemu-options.def. Maybe in the end, that would be simply
>>>> by having qemu-options.def being used to build qemu-options.json,
>>>> and then add the relevant entries in qemu-options.json. Maybe
>>>> there is a better way.
>>> 
>>> I would rather say that .def should go away and its content should
>>> be converted to a JSON schema that becomes the new source format rather
>>> than an intermediate generated file. There is nothing in .def files that
>>> couldn't be represented in the schema.
>> 
>> Yes.
>
> Agreed too. The meta-schema would need to be extended.
>
>>> 
>>> To a certain degree, QAPI does just that, by supporting different data
>>> types for options. If necessary, I'm sure the type system could be
>>> extended, but it's not clear to me to which degree we actually need
>>> this.
>> 
>> QAPI is much more expressive than qemu-options.hx.  All the latter can
>> do is "has option argument" and "option is arch-specific".  QAPI lets
>> you specify an option argument's structure, and supports compile-time
>> conditionals.  It can't do things like "must specify either argument A
>> or B", or "numeric argument C must not exceed argument D", or "multiple
>> options combine vs. last one wins".  Mostly because "it’s already
>> complicated enough as is".
>
> Understood.
>
>
>>> 
>>> We could. But is it actually worth inventing a new programming language?
>>> I think this is something that should be done in C code even in the
>>> future. I think what we're looking for is a way to describe interfaces,
>>> not implementations.
>> 
>> Sane sugar can be described declaratively as macro expansion.
>> 
>> Less than sane sugar takes code.
>> 
>> Desugaring in code promotes (accidental) abandonment of sanity.
>> 
>> That said, we have to keep things simple to succeed.  Declarative
>> desugaring seems beyond our reach.
>
> Well, “keeping things simple” is a complicated thing.

True!

>>> Even just for adding more sophisticated constraints, like in the example
>>> above that maxmem >= size, it's questionable whether doing this in the
>>> schema provides enough value for actually implementing it there.
>> 
>> Yes.
>
> If we do it, yes. If we delegate to an existing language (I showed C above),
> maybe it adds clarity without too much of an additional cost.

Possible.

>>>>> I think the only thing in this list that can't obviously be covered
>>>>> easily by QAPI is QOM. Or rather, it's covered by passing through
>>>>> key=value lists without describing their structure
>>>> 
>>>> That’s close enough to me. (In my mind, that part was already “done”
>>>> by QAPI, even if in the convoluted way you describe)
>>> 
>>> That's not really close enough. If you are happy with this level of
>>> abstraction, you can define any command line option to be either a flag
>>> or take a string as its argument and be done. This is obviously not very
>>> helpful because it says nothing about the structure of that string.
>> 
>> Similar to all QMP commands taking a single 'any' argument.
>
> Hmmm, I guess I have not understood what you meant by
> "QOM could not be covered easily by QAPI" then.

I think Kevin's point is that QAPI can't fully describe QOM.

Let's talk specifics.  Here's how QAPI defines blockdev-add:

    { 'command': 'blockdev-add', 'data': 'BlockdevOptions', 'boxed': true }
    { 'union': 'BlockdevOptions',
      'base': { 'driver': 'BlockdevDriver',
                [...]
                '*detect-zeroes': 'BlockdevDetectZeroesOptions' },
      'discriminator': 'driver',
      'data': {
          'blkdebug':   'BlockdevOptionsBlkdebug',
          'blklogwrites':'BlockdevOptionsBlklogwrites',
          [...]
          'vxhs':       'BlockdevOptionsVxHS'
      } }
    { 'enum': 'BlockdevDriver',
      'data': [ 'blkdebug', 'blklogwrites', [...], 'vxhs' ] }

QAPI specifies the available blockdev object types and their properties,
with types.

Compare object-add:

    { 'command': 'object-add',
      'data': {'qom-type': 'str', 'id': 'str', '*props': 'any'} }

QAPI specifies neither the available QOM object types, nor their
properties.  Consequently, QAPI/QMP introspection tells you nothing
about them.  You need to use QOM introspection instead.  qom-list-types
shows what QOM object types are available, and qom-list-properties shows
an object type's properties.  Except for dynamic ones created only after
object creation).  With undocumented type name strings instead of types.

> We don’t want the QAPI to let arbitrary fields of a QOM object
> be modified, do we?

We already do: QMP command qom-set.  If it breaks your guest, you get to
keep the pieces.

> As for the “public” aspects of a QOM object, that is static if it
> comes from QAPI, so why couldn’t we define it there?

QAPI specifies a certain kind of data types.

QOM "specifies" data types as an imperative program for creating them.
Maximally flexible, and fundamentally at odds with static analysis.
I've hated this since day one.

There's no hard reason why QOM could not specify static aspects
declaratively.  It just doesn't, and changing it now would be a
monumental task.

The imperative program mostly creates identical data types every time.
In other words, the data types are static.  Two problems.  One,
converting such a program to an equivalent declaration takes manual
labor.  Two, there are exceptions, and identifying them is more manual
labor.

>>> the same way, QAPI can't say anything about the structure of a QOM
>>> object, and I think that's a problem.
>>> 
>>>>> - which, as far as I
>>>>> understand, is mainly because QOM properties aren't necessarily static,
>>>>> so we can't provide a statically defined interface for them. Probably
>>>>> solvable in QEMU, but not without a major effort.
>>>> 
>>>> Or maybe extend the language so that it’s internal semantics
>>>> knows about this aspect of QOM?
>>> 
>>> My point is that for example you can't generate a C struct (which is
>>> statically defined) from something that has a dynamic structure. The
>>> best you could do is fall back to key=value even in the C source, both
>>> key and value being strings. But then you still have to parse these
>>> strings manually, so it doesn't actually help much compared to a state
>>> without C bindings.
>
> I really don’t understand that point. To me, all these operations
> seem relatively simple to generate.

Yes, if the QAPI fairy gives us a declarative specification equivalent
to the existing imperative one, then replacing the hand-written
imperative code by code generated from the declarative specification
would be relatively simple.

> I think that what confuses me is when you write “something that has a
> dynamic structure”. I understand that as referring to the structure
> defined in the schema. But the schema itself is static. So you can
> generate static code from it, and it’s already done today.

QAPI data types are static.

QOM types are not.  They're effectively almost static in practice.

> Another hypothesis on my side is that we don’t want, ever, to
> have the API provide for example the option to create its own
> arbitrary QOM classes, or to tag arbitrary properties to an object,
> where by “arbitrary” I mean not explicitly mentioned somewhere in
> something like the schema.
>
> So I suspect you are talking about something else.

Kevin's talking about the imperative code creating different QOM
properties depending on how and in what context it is run.

QOM is perfectly capable of supporting a QMP command to add arbitrary
QOM properties to an object at run time, or even add arbitrary QOM
types, but as you wrote we don't want that.

>> QOM and QAPI sabotage each other.  Ironic, considering they were dreamed
>> up by the same guy :)
>> 
>> QAPI is compile-time static by design.
>> 
>> QOM is run-time dynamic by design.  Some support for static definitions
>> has been grafted on since.
>> 
>> We use QAPI type system escapes for QOM.  Defeats QAPI introspection and
>> doc generation.  We provide separate QOM introspection instead, which is
>> clumsier and less expressive.  QOM documentation doesn't exist.
>
> But back to the original discussion about management tools,
> do we let upper layers tag their own arbitrary stuff in QOM objects?

We don't want management applications to add QOM properties for their
own purposes.

>>> Maybe what could be done is covering at least the static properties and
>>> then having key=value for the dynamic part (which should be the
>>> exception).
>> 
>> To make this worthwhile, we'd have to replace dynamic QOM properties by
>> static ones when possible.  Monumental task.
>
> I’m sure you are right, but it’s hard for me to evaluate, given how
> many ways there are to access an object. Naively, grepping for
> set_prop and for new_with_prop does not give me that many hits.

Look for object_property_add*().  Some 450 hits.

See also object_class_property_add(), grafted on in

commit 16bf7f522a2ff68993f80631ed86254c71eaf5d4
Author: Daniel P. Berrange <berrange@redhat.com>
Date:   Tue Oct 13 13:37:46 2015 +0100

    qom: Allow properties to be registered against classes
    
    When there are many instances of a given class, registering
    properties against the instance is wasteful of resources. The
    majority of objects have a statically defined list of possible
    properties, so most of the properties are easily registerable
    against the class. Only those properties which are conditionally
    registered at runtime need be recorded against the klass.
    
    Registering properties against classes also makes it possible
    to provide static introspection of QOM - currently introspection
    is only possible after creating an instance of a class, which
    severely limits its usefulness.
    
    This impl only supports simple scalar properties. It does not
    attempt to allow child object / link object properties against
    the class. There are ways to support those too, but it would
    make this patch more complicated, so it is left as an exercise
    for the future.
    
    There is no equivalent to object_property_del() provided, since
    classes must be immutable once they are defined.
    
    Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
    Signed-off-by: Andreas Färber <afaerber@suse.de>

>>>>>> - Relations, e.g. how we represent “contains”, “derives from”, “needs”,
>>>>>> “one of”, “one or several”, “attaches to”…
>>>>>> - States, e.g. how do we represent the machine configuration,
>>>>>> or the desired new disk setting
>>>>>> - Verbs, e.g. how we represent “add”, “connect”, “remove”, “find”,
>>>>>> “start”, “notify”, etc. and how we describe the kind of input they need.
>>>>>> - Possibly more subtle things like support for transactions, commit/rollback,
>>>>>> i.e. “I want to add connect a virtual nic to some host vf, but if anything
>>>>>> along the way fails, I’d like all the cleanup to happen automatically)
>>>>> 
>>>>> This sounds like a different approach from our current QAPI command set
>>>> 
>>>> Well, except for purposefully trying to use a different wording to avoid
>>>> the risk of getting your mind stuck in one of the particular existing
>>>> meta-languages in QEMU, the approach is not very different.
>>> 
>>> I didn't necessarily mean relations/state/verbs, which obviously exist,
>>> but the examples that seemed to express things in a deliberately
>>> different way from what we have today.
>>> 
>>>> - Transactions do not exist today that I know of, although we see
>>>> signs of them in discussions about the fact that this options destroys
>>>> that back end but that option does not.
>>> 
>>> We have a 'transaction' QMP command, but they are not an integral part
>>> of the language. Considering the trouble to implement transactional
>>> commands, I suppose we don't even want it to be a fundamental part of
>>> the language.
>>> 
>>>>> Does it actually provide more functionality, though?
>>>> 
>>>> It’s not intended to provide more, but to require less to do the same thing.
>>> 
>>> Though we always need to keep in mind that if requiring less for future
>>> additions requires a huge effort now, the investment may pay off only in
>>> a very distant future (if the abstraction we build even survives until
>>> then).
>> 
>> Worse is better.
>> 
>> http://dreamsongs.com/RiseOfWorseIsBetter.html
>
> You know that I positively hate this ;-)

It's been a tough lesson for me, too.

> Well, I guess we can expand the schema. #ILoveJSON.

Basing the QAPI language on JSON was a poor choice.  Not sure that's
fixable at a reasonable cost.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-15  9:20             ` Markus Armbruster
@ 2020-01-15  9:34               ` Christophe de Dinechin
  2020-01-15 12:15                 ` Markus Armbruster
  2020-01-15  9:35               ` Marc-André Lureau
  1 sibling, 1 reply; 34+ messages in thread
From: Christophe de Dinechin @ 2020-01-15  9:34 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, Paolo Bonzini,
	Marc-André Lureau, John Snow, Dominik Csapak

Thanks a bunch. This clarifies a number of my misconceptions about
how this is currently used. Most notably this one:

> On 15 Jan 2020, at 10:20, Markus Armbruster <armbru@redhat.com> wrote:
> 
>> We don’t want the QAPI to let arbitrary fields of a QOM object
>> be modified, do we?
> 
> We already do: QMP command qom-set.  If it breaks your guest, you get to
> keep the pieces.

Ouch. I certainly did not expect that.

"It is not what you don’t know that kills you, it is what you know that ain’t so".

> 
>>> http://dreamsongs.com/RiseOfWorseIsBetter.html
>> 
>> You know that I positively hate this ;-)
> 
> It's been a tough lesson for me, too.

Not sure I can call it a “lesson”. Much like a philosophy to fight against, IMO.


>> Well, I guess we can expand the schema. #ILoveJSON.
> 
> Basing the QAPI language on JSON was a poor choice.  Not sure that's
> fixable at a reasonable cost.

Well, at least now I have a slightly better understanding of the related costs
and trade-offs. Thanks a lot for explaining.


Christophe



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-15  9:20             ` Markus Armbruster
  2020-01-15  9:34               ` Christophe de Dinechin
@ 2020-01-15  9:35               ` Marc-André Lureau
  2020-01-15 12:25                 ` Markus Armbruster
  1 sibling, 1 reply; 34+ messages in thread
From: Marc-André Lureau @ 2020-01-15  9:35 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, Christophe de Dinechin,
	Paolo Bonzini, John Snow, Dominik Csapak

Hi

On Wed, Jan 15, 2020 at 1:21 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Christophe de Dinechin <dinechin@redhat.com> writes:
>
> >> To make this worthwhile, we'd have to replace dynamic QOM properties by
> >> static ones when possible.  Monumental task.
> >
> > I’m sure you are right, but it’s hard for me to evaluate, given how
> > many ways there are to access an object. Naively, grepping for
> > set_prop and for new_with_prop does not give me that many hits.
>
> Look for object_property_add*().  Some 450 hits.

fwiw, I have started tackling that.

Easy first step is to move all QDev properties to class properties,
and this is done in :
https://patchew.org/QEMU/20200110153039.1379601-1-marcandre.lureau@redhat.com/

Moving from instance to class properties is straightforward many times
(when the property is unconditonally added in instance init for
example). There are a few complicated cases though.

To me, the most problematic is reviewer-time and willingness to do
such low-benefits changes.

> Basing the QAPI language on JSON was a poor choice.  Not sure that's
> fixable at a reasonable cost.

Translating it to another language should be relatively easy, but to what?

-- 
Marc-André Lureau


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-15  9:34               ` Christophe de Dinechin
@ 2020-01-15 12:15                 ` Markus Armbruster
  2020-01-15 12:19                   ` Daniel P. Berrangé
  2020-01-20 10:08                   ` Stefan Hajnoczi
  0 siblings, 2 replies; 34+ messages in thread
From: Markus Armbruster @ 2020-01-15 12:15 UTC (permalink / raw)
  To: Christophe de Dinechin
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, Marc-André Lureau,
	Paolo Bonzini, John Snow, Dominik Csapak

Christophe de Dinechin <dinechin@redhat.com> writes:

> Thanks a bunch. This clarifies a number of my misconceptions about
> how this is currently used. Most notably this one:
>
>> On 15 Jan 2020, at 10:20, Markus Armbruster <armbru@redhat.com> wrote:
>> 
>>> We don’t want the QAPI to let arbitrary fields of a QOM object
>>> be modified, do we?
>> 
>> We already do: QMP command qom-set.  If it breaks your guest, you get to
>> keep the pieces.
>
> Ouch. I certainly did not expect that.
>
> "It is not what you don’t know that kills you, it is what you know that ain’t so".

Do we have a legitimate use for qom-set right now?  Hmm, let's check
libvirt... aha:

* qemuMonitorJSONSetMemoryStatsPeriod() uses it to control
  virtio-balloon's guest-stats-polling-interval property, in accordance
  with docs/virtio-balloon-stats.txt.

* qemuMonitorJSONSetIOThread() uses it to control iothread's properties
  poll-max-ns, poll-grow, poll-shrink.  Their use with -object is
  documented (in qemu-options.hx), their use with qom-set is not.

Oh well.

>>>> http://dreamsongs.com/RiseOfWorseIsBetter.html
>>> 
>>> You know that I positively hate this ;-)
>> 
>> It's been a tough lesson for me, too.
>
> Not sure I can call it a “lesson”. Much like a philosophy to fight against, IMO.

The tough lesson isn't philosophy, it's an observation: "I believe that
worse-is-better [...] has better survival characteristics than
the-right-thing".

>>> Well, I guess we can expand the schema. #ILoveJSON.
>> 
>> Basing the QAPI language on JSON was a poor choice.  Not sure that's
>> fixable at a reasonable cost.
>
> Well, at least now I have a slightly better understanding of the related costs
> and trade-offs. Thanks a lot for explaining.

You're welcome!



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-15 12:15                 ` Markus Armbruster
@ 2020-01-15 12:19                   ` Daniel P. Berrangé
  2020-01-15 14:02                     ` Markus Armbruster
  2020-01-20 10:08                   ` Stefan Hajnoczi
  1 sibling, 1 reply; 34+ messages in thread
From: Daniel P. Berrangé @ 2020-01-15 12:19 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Kevin Wolf, Peter Maydell, Denis V. Lunev, Stefan Hajnoczi,
	qemu-devel, Marc-André Lureau, Christophe de Dinechin,
	Paolo Bonzini, John Snow, Dominik Csapak

On Wed, Jan 15, 2020 at 01:15:17PM +0100, Markus Armbruster wrote:
> Christophe de Dinechin <dinechin@redhat.com> writes:
> 
> > Thanks a bunch. This clarifies a number of my misconceptions about
> > how this is currently used. Most notably this one:
> >
> >> On 15 Jan 2020, at 10:20, Markus Armbruster <armbru@redhat.com> wrote:
> >> 
> >>> We don’t want the QAPI to let arbitrary fields of a QOM object
> >>> be modified, do we?
> >> 
> >> We already do: QMP command qom-set.  If it breaks your guest, you get to
> >> keep the pieces.
> >
> > Ouch. I certainly did not expect that.
> >
> > "It is not what you don’t know that kills you, it is what you know that ain’t so".
> 
> Do we have a legitimate use for qom-set right now?  Hmm, let's check
> libvirt... aha:
> 
> * qemuMonitorJSONSetMemoryStatsPeriod() uses it to control
>   virtio-balloon's guest-stats-polling-interval property, in accordance
>   with docs/virtio-balloon-stats.txt.
> 
> * qemuMonitorJSONSetIOThread() uses it to control iothread's properties
>   poll-max-ns, poll-grow, poll-shrink.  Their use with -object is
>   documented (in qemu-options.hx), their use with qom-set is not.
> 
> Oh well.

Libvirt is of course happy to switch to something else instead of
qom-set for these features if QEMU wants to provide a safer
alternative.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-15  9:35               ` Marc-André Lureau
@ 2020-01-15 12:25                 ` Markus Armbruster
  0 siblings, 0 replies; 34+ messages in thread
From: Markus Armbruster @ 2020-01-15 12:25 UTC (permalink / raw)
  To: Marc-André Lureau
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, Christophe de Dinechin,
	Paolo Bonzini, John Snow, Dominik Csapak

Marc-André Lureau <marcandre.lureau@gmail.com> writes:

> Hi
>
> On Wed, Jan 15, 2020 at 1:21 PM Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Christophe de Dinechin <dinechin@redhat.com> writes:
>>
>> >> To make this worthwhile, we'd have to replace dynamic QOM properties by
>> >> static ones when possible.  Monumental task.
>> >
>> > I’m sure you are right, but it’s hard for me to evaluate, given how
>> > many ways there are to access an object. Naively, grepping for
>> > set_prop and for new_with_prop does not give me that many hits.
>>
>> Look for object_property_add*().  Some 450 hits.
>
> fwiw, I have started tackling that.

Laudable.

> Easy first step is to move all QDev properties to class properties,
> and this is done in :
> https://patchew.org/QEMU/20200110153039.1379601-1-marcandre.lureau@redhat.com/

Easy because qdev properties are declarative by design.  Imperative came
in when they got rebased onto QOM.

> Moving from instance to class properties is straightforward many times
> (when the property is unconditonally added in instance init for
> example). There are a few complicated cases though.
>
> To me, the most problematic is reviewer-time and willingness to do
> such low-benefits changes.

Understand.

>> Basing the QAPI language on JSON was a poor choice.

Aggravated by immediately extending it in ways that defeat all common
JSON tools.

>>                                                      Not sure that's
>> fixable at a reasonable cost.
>
> Translating it to another language should be relatively easy, but to what?

The QAPI language is layered on top of JSON (see
docs/devel/qapi-code-gen.txt section "Schema syntax").  Swapping out
JSON for another language that is at least as expressive is an
relatively simple change.  Obvious candidates: s-expressions, TOML,
YAML.  The latter is far too complex for my taste.

The expensive part is dealing with the conflicts during the transition,
and having everybody relearn QAPI schema syntax.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-15 12:19                   ` Daniel P. Berrangé
@ 2020-01-15 14:02                     ` Markus Armbruster
  0 siblings, 0 replies; 34+ messages in thread
From: Markus Armbruster @ 2020-01-15 14:02 UTC (permalink / raw)
  To: Daniel P. Berrangé
  Cc: Kevin Wolf, Peter Maydell, Denis V. Lunev, Stefan Hajnoczi,
	qemu-devel, Paolo Bonzini, Christophe de Dinechin,
	Marc-André Lureau, John Snow, Dominik Csapak

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Wed, Jan 15, 2020 at 01:15:17PM +0100, Markus Armbruster wrote:
>> Christophe de Dinechin <dinechin@redhat.com> writes:
>> 
>> > Thanks a bunch. This clarifies a number of my misconceptions about
>> > how this is currently used. Most notably this one:
>> >
>> >> On 15 Jan 2020, at 10:20, Markus Armbruster <armbru@redhat.com> wrote:
>> >> 
>> >>> We don’t want the QAPI to let arbitrary fields of a QOM object
>> >>> be modified, do we?
>> >> 
>> >> We already do: QMP command qom-set.  If it breaks your guest, you get to
>> >> keep the pieces.
>> >
>> > Ouch. I certainly did not expect that.
>> >
>> > "It is not what you don’t know that kills you, it is what you know that ain’t so".
>> 
>> Do we have a legitimate use for qom-set right now?  Hmm, let's check
>> libvirt... aha:
>> 
>> * qemuMonitorJSONSetMemoryStatsPeriod() uses it to control
>>   virtio-balloon's guest-stats-polling-interval property, in accordance
>>   with docs/virtio-balloon-stats.txt.
>> 
>> * qemuMonitorJSONSetIOThread() uses it to control iothread's properties
>>   poll-max-ns, poll-grow, poll-shrink.  Their use with -object is
>>   documented (in qemu-options.hx), their use with qom-set is not.
>> 
>> Oh well.
>
> Libvirt is of course happy to switch to something else instead of
> qom-set for these features if QEMU wants to provide a safer
> alternative.

Noted.

libvirt's use of qom-set is okay.  What's not okay is the near-complete
lack of QOM documentation, its poor QMP interface, in part due to
non-integration with QAPI, and last but not least the lack of QOM
leadership leaving it adrift.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-02 14:47   ` Stefan Hajnoczi
@ 2020-01-16 11:03     ` Kashyap Chamarthy
  2020-01-20  9:55       ` Stefan Hajnoczi
  0 siblings, 1 reply; 34+ messages in thread
From: Kashyap Chamarthy @ 2020-01-16 11:03 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Cleber Rosa, Markus Armbruster, qemu-devel,
	Marc-André Lureau, Paolo Bonzini, Dominik Csapak, John Snow,
	Eduardo Habkost

On Thu, Jan 02, 2020 at 02:47:22PM +0000, Stefan Hajnoczi wrote:
> On Sat, Dec 21, 2019 at 10:02:23AM +0100, Markus Armbruster wrote:
> > Stefan Hajnoczi <stefanha@gmail.com> writes:

[...]

> > > 2. scripts/qmp/ contains command-line tools for QMP communication.
> > > They could use some polish and then be shipped.
> > 
> > MAINTAINERS blames them on me, but they're effectively unmaintained.
> > Prerequisite for shipping: having a maintainer who actually gives a
> > damn.
> ...
> > * scripts/qmp/qmp-shell
> > 
> >   Half-hearted attempt at a human-friendly wrapper around the JSON
> >   syntax.  I have no use for this myself.
> 
> I think this one is used by people.  John Snow comes to mind.

FWIW I too frequently use 'qmp-shell'.  And some of the examples in this
document[1] are demonstrated with it.

I'm reasonably happy with it (particularly the persistent history
captured in ~/.qmp-shell_history), and it has some "known issues" that
can trip up a new user.  The one that immediately jumps to mind:
asynchronous events won't be printed without a prompt from the user --
e.g. after a `blockdev-commit`, you won't see BLOCK_JOB_{READY,
COMPLETED} events printed unless you manually hit enter from the
'qmp-shell'.

(Not complaining; I have a long-standing TODO to make time to
investigate this.)

[1] https://qemu.readthedocs.io/en/latest/interop/live-block-operations.html

> > > 3. python/qemu/ contains Python modules for managing a QEMU process
> > > and QMP communication.  This should be packaged in distros and
> > > available on PyPI.
> > 
> > Currently maintained by Eduardo and Cleber (cc'ed) under "Python
> > scripts".

Yeah, that'd certainly be useful.  I vaguely recall a discussion on this
topic here on the list (or probably some hallway discussion, can't
recall.)


[...]


-- 
/kashyap



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-16 11:03     ` Kashyap Chamarthy
@ 2020-01-20  9:55       ` Stefan Hajnoczi
  2020-01-20 13:57         ` Kashyap Chamarthy
  0 siblings, 1 reply; 34+ messages in thread
From: Stefan Hajnoczi @ 2020-01-20  9:55 UTC (permalink / raw)
  To: Kashyap Chamarthy
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Cleber Rosa, Markus Armbruster, qemu-devel,
	Marc-André Lureau, Paolo Bonzini, Dominik Csapak, John Snow,
	Eduardo Habkost

[-- Attachment #1: Type: text/plain, Size: 2219 bytes --]

On Thu, Jan 16, 2020 at 12:03:14PM +0100, Kashyap Chamarthy wrote:
> On Thu, Jan 02, 2020 at 02:47:22PM +0000, Stefan Hajnoczi wrote:
> > On Sat, Dec 21, 2019 at 10:02:23AM +0100, Markus Armbruster wrote:
> > > Stefan Hajnoczi <stefanha@gmail.com> writes:
> 
> [...]
> 
> > > > 2. scripts/qmp/ contains command-line tools for QMP communication.
> > > > They could use some polish and then be shipped.
> > > 
> > > MAINTAINERS blames them on me, but they're effectively unmaintained.
> > > Prerequisite for shipping: having a maintainer who actually gives a
> > > damn.
> > ...
> > > * scripts/qmp/qmp-shell
> > > 
> > >   Half-hearted attempt at a human-friendly wrapper around the JSON
> > >   syntax.  I have no use for this myself.
> > 
> > I think this one is used by people.  John Snow comes to mind.
> 
> FWIW I too frequently use 'qmp-shell'.  And some of the examples in this
> document[1] are demonstrated with it.
> 
> I'm reasonably happy with it (particularly the persistent history
> captured in ~/.qmp-shell_history), and it has some "known issues" that
> can trip up a new user.  The one that immediately jumps to mind:
> asynchronous events won't be printed without a prompt from the user --
> e.g. after a `blockdev-commit`, you won't see BLOCK_JOB_{READY,
> COMPLETED} events printed unless you manually hit enter from the
> 'qmp-shell'.
> 
> (Not complaining; I have a long-standing TODO to make time to
> investigate this.)
> 
> [1] https://qemu.readthedocs.io/en/latest/interop/live-block-operations.html

John and I discussed async events in the past.  qmp-shell currently uses
the input() built-in function.  If we modify it with a
select(2)/poll(2)-style function that monitors both stdin and the QMP
socket then it could print QMP events as soon as they are received.

There might be a nicer way of doing it, but pseudo-code for the idea is:

  def input_with_events(prompt):
      while True:
          print(prompt, end='', flush=True)
          readable_files = select([sys.stdin, qmp_socket])
          if qmp_socket in readable_files:
              print_qmp_events()

      # stdin is ready, read a line
      return input()

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-15 12:15                 ` Markus Armbruster
  2020-01-15 12:19                   ` Daniel P. Berrangé
@ 2020-01-20 10:08                   ` Stefan Hajnoczi
  1 sibling, 0 replies; 34+ messages in thread
From: Stefan Hajnoczi @ 2020-01-20 10:08 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	qemu-devel, Marc-André Lureau, Christophe de Dinechin,
	Paolo Bonzini, John Snow, Dominik Csapak

[-- Attachment #1: Type: text/plain, Size: 855 bytes --]

On Wed, Jan 15, 2020 at 01:15:17PM +0100, Markus Armbruster wrote:
> Christophe de Dinechin <dinechin@redhat.com> writes:
> >> On 15 Jan 2020, at 10:20, Markus Armbruster <armbru@redhat.com> wrote:
> * qemuMonitorJSONSetIOThread() uses it to control iothread's properties
>   poll-max-ns, poll-grow, poll-shrink.  Their use with -object is
>   documented (in qemu-options.hx), their use with qom-set is not.

I'm happy to use a different interface.

Writing a boilerplate "iothread-set-poll-params" QMP command in C would
be a step backwards.

Maybe the QAPI code generator could map something like this:

  { 'command': 'iothread-set-poll-params',
    'data': {
        'id': 'str',
	'*max-ns': 'uint64',
	'*grow': 'uint64',
	'*shrink': 'uint64'
    },
    'map-to-qom-set': 'IOThread'
  }

And turn it into QOM accessors on the IOThread object.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: Making QEMU easier for management tools and applications
  2020-01-20  9:55       ` Stefan Hajnoczi
@ 2020-01-20 13:57         ` Kashyap Chamarthy
  0 siblings, 0 replies; 34+ messages in thread
From: Kashyap Chamarthy @ 2020-01-20 13:57 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrange, Denis V. Lunev,
	Cleber Rosa, Markus Armbruster, qemu-devel,
	Marc-André Lureau, Paolo Bonzini, Dominik Csapak, John Snow,
	Eduardo Habkost

On Mon, Jan 20, 2020 at 09:55:54AM +0000, Stefan Hajnoczi wrote:
> On Thu, Jan 16, 2020 at 12:03:14PM +0100, Kashyap Chamarthy wrote:

[...]

> > I'm reasonably happy with it (particularly the persistent history
> > captured in ~/.qmp-shell_history), and it has some "known issues" that
> > can trip up a new user.  The one that immediately jumps to mind:
> > asynchronous events won't be printed without a prompt from the user --
> > e.g. after a `blockdev-commit`, you won't see BLOCK_JOB_{READY,
> > COMPLETED} events printed unless you manually hit enter from the
> > 'qmp-shell'.

[...]

> John and I discussed async events in the past.  qmp-shell currently uses
> the input() built-in function.  If we modify it with a
> select(2)/poll(2)-style function that monitors both stdin and the QMP
> socket then it could print QMP events as soon as they are received.
> 
> There might be a nicer way of doing it, but pseudo-code for the idea is:
> 
>   def input_with_events(prompt):
>       while True:
>           print(prompt, end='', flush=True)
>           readable_files = select([sys.stdin, qmp_socket])
>           if qmp_socket in readable_files:
>               print_qmp_events()
> 
>       # stdin is ready, read a line
>       return input()

Thanks for the suggestion.  The Python 'select' module[1] indeed seems
to provide access to select() and poll() Linux system calls.

[1] https://docs.python.org/3/library/select.html

-- 
/kashyap



^ permalink raw reply	[flat|nested] 34+ messages in thread

end of thread, back to index

Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-20 16:13 Making QEMU easier for management tools and applications Stefan Hajnoczi
2019-12-20 21:07 ` Richard W.M. Jones
2020-01-02 11:26   ` Stefan Hajnoczi
2019-12-21  9:02 ` Markus Armbruster
2019-12-23 15:04   ` Michal Prívozník
2020-01-07  9:36     ` Kevin Wolf
2020-01-07 10:55       ` Michal Privoznik
2020-01-07 12:57         ` Kevin Wolf
2020-01-07 17:53           ` Christophe de Dinechin
2019-12-24 13:41   ` Daniel P. Berrangé
2020-01-02 14:47   ` Stefan Hajnoczi
2020-01-16 11:03     ` Kashyap Chamarthy
2020-01-20  9:55       ` Stefan Hajnoczi
2020-01-20 13:57         ` Kashyap Chamarthy
2020-01-02 15:05   ` Dr. David Alan Gilbert
2020-01-13 13:44     ` Markus Armbruster
2019-12-24 13:00 ` Daniel P. Berrangé
2020-01-02 14:22   ` Stefan Hajnoczi
2020-01-02 15:10 ` Dr. David Alan Gilbert
2020-01-07 17:11 ` Christophe de Dinechin
2020-01-08 10:43   ` Kevin Wolf
2020-01-08 11:40     ` Christophe de Dinechin
2020-01-08 13:38       ` Kevin Wolf
2020-01-14 13:04         ` Markus Armbruster
2020-01-14 17:31           ` Christophe de Dinechin
2020-01-15  9:20             ` Markus Armbruster
2020-01-15  9:34               ` Christophe de Dinechin
2020-01-15 12:15                 ` Markus Armbruster
2020-01-15 12:19                   ` Daniel P. Berrangé
2020-01-15 14:02                     ` Markus Armbruster
2020-01-20 10:08                   ` Stefan Hajnoczi
2020-01-15  9:35               ` Marc-André Lureau
2020-01-15 12:25                 ` Markus Armbruster
2020-01-13 16:30   ` Stefan Hajnoczi

QEMU-Devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/qemu-devel/0 qemu-devel/git/0.git
	git clone --mirror https://lore.kernel.org/qemu-devel/1 qemu-devel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 qemu-devel qemu-devel/ https://lore.kernel.org/qemu-devel \
		qemu-devel@nongnu.org
	public-inbox-index qemu-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.nongnu.qemu-devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git