Re: Making QEMU easier for management tools and applications

[Andrea Bolognani <abologna@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 5321 bytes --]

On Fri, 2020-01-31 at 07:50 +0100, Markus Armbruster wrote:
> Kevin Wolf <kwolf@redhat.com> writes:
> > Much of this threads plays with the though that maybe we don't need any
> > compatibility and make the radical conclusion that we don't need any
> > human-friendly interface at all. Keeping full compatibility is the other
> > extreme.
> > 
> > There might be some middle ground where we break compatibility where the
> > old way can't easily be maintained with the new infrastructure, but
> > don't give up on the idea of being used by humans.
> 
> I'm not sure the connection between maintaining compatibility and
> supporting human use is as strong as you seem to imply.
> 
> As far as I can tell, the "maybe we don't need any compatibility"
> discussion is about the CLI.  I'd rephrase it as "maybe we need a
> machine-friendly CLI on par with QMP more than we need compatibility to
> the current CLI".
> 
> "We don't need any human-friendly interface at all" comes in not because
> machine-friendly necessarily precludes human-friendly, but only if we're
> unwilling (unable?) to do the extra work for it.
> 
> Compare the monitor:
> 
> * QMP is primarily for machines.  We promise stability: no incompatible
>   changes without clear communicaton of intent and a grace period.  We
>   provide machine clients tools to deal with the interface evolution,
>   e.g. query-qmp-schema.
> 
> * HMP is exclusively for humans.  It may change at any time.
> 
> For the CLI, we don't have such a separation, and our offerings for
> dealing with interface evolution are wholly inadequate.  We *need* to do
> better for machines.
> 
> Now, the monitor also informs us about the cost of providing a
> completely separate interface for humans.
> 
> Elsewhere in this thread, we discussed layering (a replacement for) HMP
> on top of QMP cleanly, possibly in a separate process, possibly written
> in a high-level language like Python.
> 
> HMP predates QMP.  We reworked it so the HMP commands are implemented on
> top of the QMP commands, or at least on top of common helpers.  But this
> is not quite the same as layering HMP on top of QMP.
> 
> If we decide to radically break the CLI so we can start over, we get to
> decide whether and how to do a human-friendly CLI, in particular how it
> relates to the machine-friendly CLI.

Does a machine-friendly CLI need to exist at all? Once you decide
that throwing away the current one is acceptable, you might as well
reduce the maintainance burden by requiring that software only
communicates with QEMU via QMP.

Does a human-friendly CLI need to be part of QEMU? We have built so
much amazing infrastructure on top of QEMU, and as of today none of
that work is benefiting people who run it directly.


As a proof of concept, I have spent a couple of hours writing the
attached shell script, which I hope will illustrate my point.

Usage is extremely simple: just do something like

  $ ./virt-run debian-10-openstack-amd64.qcow2

and after a few seconds the guest display will appear on your screen.

Behind the scenes, it uses a number of existing high-level tools:

  * virt-inspector, to figure out what guest OS is installed in the
    image;

  * virt-install, to produce a domain XML tailored to that specific
    guest OS and to create the corresponding libvirt domain;

  * virt-viewer, to provide the UI.

All these tools use libvirt under the hood, and additionally
virt-install uses libosinfo to obtain information about the guest
OS, such as whether or not it supports Virtio devices and how much
memory it needs to run smoothly.


The result is that, if you run

  $ qemu-system-x86_64 -hda debian-10-openstack-amd64.qcow2

you will get

  * a single CPU emulated with TCG;

  * 128 MiB of memory;

  * emulated I/O devices;

whereas the script will give you

  * 2 CPUs accelerated with KVM;

  * 1 GiB of memory;

  * Virtio devices for pretty much everything, including a
    virtio-rng device that will for example speed up the first boot
    significantly if SSH keys need to be (re)created.

Unsurprisingly, performance is different: when QEMU is invoked
directly, the login prompt for this specific image shows up after
~40 seconds, whereas when we use the script it only takes ~13 seconds
to get there. And the command line is just as simple, if not more so!

All of the above was obtained by hastily cobbling together existing
tools with <100 lines of shell scripting. Imagine how much better it
could be if we actually put some serious work in!


With my argument hopefully demonstrated: I think an architecture akin
to the one Dan has outlined earlier[1] would be a great direction to
take. QEMU can continue to focus on its core competency, that is,
virtual hardware, and leave most of the user interaction up to the
software interacting with its JSON-based API.


Obviously QEMU developers, for their own use, could still benefit
from having access to a user interface that doesn't require either
rigging up libvirt support or messing with JSON directly, and such
an interface could even look very similarly to the current CLI, but
it could simply not be made user-facing and thus not subject to any
compatibility concerns.


[1] https://lists.nongnu.org/archive/html/qemu-devel/2020-01/msg06034.html
-- 
Andrea Bolognani / Red Hat / Virtualization

[-- Attachment #2: virt-run --]
[-- Type: application/x-shellscript, Size: 1774 bytes --]