Re: Making QEMU easier for management tools and applications

[Kevin Wolf <kwolf@redhat.com>]

Am 24.01.2020 um 11:27 hat Daniel P. Berrangé geschrieben:
> On Fri, Jan 24, 2020 at 08:59:41AM +0100, Markus Armbruster wrote:
> > John Snow <jsnow@redhat.com> writes:
> > 
> > > On 1/23/20 2:01 PM, Daniel P. Berrangé wrote:
> > >> So when configuring objects you'll always provide a JSON/YAML doc.
> > >> They've got some clever stuff for updating objects where you can
> > >> provide a JSON patch for only the bits which need changing.
> > >> 
> > >> When querying/listing objects by default it displays only a small
> > >> subset of their config information in a human friendly-ish format.
> > >> If you want to see everything then you ask for it in JSON/YAML
> > >> format. There's also an expression language that lets you extract
> > >> particular pieces of information based on requested properties,
> > >> and you can filter the list of objects based on attributes and so
> > >> on.
> > >> 
> > >> I think it is fair to say the structure of kubernetes object config
> > >> is on a par with hierarchical complexity of QEMU. The lack of a simple
> > >> human targetted data input format does not appear to have negatively
> > >> impacted the adoption of Kubernetes. It is worth questioning why this
> > >> is the case, while we feel the human CLI syntax for QEMU is so
> > >> critically important to QEMU's future ?
> > 
> > I consider human CLI syntax for QEMU a mostly solved *design* problem:
> > dotted keys.  It's an unsolved *implementation* problem: the CLI is a
> > tangled mess of almost two decades' worth of ideas, and only (some of)
> > the latest strands actually use dotted keys infrastructure.  The
> > proposed solution is CLI QAPIfication.  Gives us configuration file(s)
> > and introspection.
> > 
> > Dotted keys are merely yet another concrete syntax.  They're designed to
> > satisfy the CLI requirements we have, which include a measure of
> > compatibility to what's in the tangled mess.  They're reasonably usable
> > for simple stuff, but complex stuff can be too verbose to be readable.
> > They can't express all of the abstract syntax.  Tolerable, since they
> > provide an escape to JSON.  I recommend programs use the JSON escape
> > always.  Awkward for humans due to shell quoting.
> 
> I agree that the dotted key syntax is our chosen / solved design
> for expressing JSON on the CLI. I would also say that, in retrospect,
> this was a incorrect design decision that is one of the key things
> responsible for QEMU having a bad reputation for complexity.

I doubt this. Whenever I get a bug report with a command line created by
libvirt, the command line is huge, but basically nothing in it uses
dotted syntax. Yes, you may have cache.direct=on in it somewhere, but
that's not actual nesting.

The problem is the amount of options that is specified by management
tools, and then humans are looking at it and feel it's way too complex.

Command lines written by human users are usually much simpler because
they just use QEMU's defaults instead of explicitly specifying
everything.

> We should simply never have tried to invent a way to map the full
> hiearchy of JSON onto the CLI as the result will always be unpleasant.
> The dotted notation is the most verbose way to do this type of
> configuration, because of the string repetition it requires for
> nested structures.

True. I would have liked a different syntax that used some kind of
brackets (at least optionally), but Markus didn't like adding another
character that must be escaped.

I think if we want, we can still evolve our human syntax to be more user
friendly. My impression was that we don't want to.

> Lets consider how libvirt uses blockdev for a LUKS volume stored
> in iSCSI
> 
>   $ qemu-system-x86_64 \
>   -object secret,id=libvirt-5-storage-secret0,\
>     data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\
>     keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
>   -object secret,id=libvirt-5-format-luks-secret0,\
>     data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\
>     keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
>   -blockdev '{"driver":"iscsi","portal":"example.org:6000",\
>     "target":"iqn.1992-01.com.example:storage","lun":1,"transport":"tcp",\
>     "user":"myname","password-secret":"libvirt-5-storage-secret0",\
>     "node-name":"libvirt-5-storage","auto-read-only":true,"discard":"unmap"}' \
>   -blockdev '{"node-name":"libvirt-5-format","read-only":false,"driver":"qcow2",\
>     "encrypt":{"format":"luks","key-secret":"libvirt-5-format-luks-secret0"},\
>     "file":"libvirt-5-storage"}' \
> 
> We all know JSON is horrible on the CLI, no surprise. So
> 
> Lets use human "friendly" dotted syntax instead:
> 
>   $ qemu-system-x86_64 \
>   -object secret,id=libvirt-5-storage-secret0,\
>     data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\
>     keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
>   -object secret,id=libvirt-5-format-luks-secret0,\
>     data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\
>     keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
>   -blockdev driver=qcow2,node-name=libvirt-5-format,read-only=false,\
>     encrypt.format=luks,encrypt.key-secret=libvirt-5-format-luks-secret0,\
>     file.driver=iscsi,file.portal=example.org:6000,\
>     file.target=iqn.1992-01.com.example:storage,file.lun=1,file.transport=tcp,\
>     file.user=myname,file.password-secret=libvirt-6-storage-secret0,\
>     file.node-name=libvirt-5-storage,file.auto-read-only=true,file.dicard=unmap
> 
> I don't think that's much of an improvement, aside from not having
> to worry about matching "}".

I see you merged the two -blockdev arguments into a single one in order
to get at least some repetition with the file.* prefixes. ;-)

> If we move to JSON in a config file
> 
>   $ cat qemu.json
>   {
>     "arguments": [
>       {
>         "arg": "object",
>         "data": {
>           "type": "secret",
>           "id":"libvirt-5-storage-secret0",
>           "data": "9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1",
>           "keyid": "masterKey0",
>           "iv": "AAECAwQFBgcICQoLDA0ODw==",
>           "format": "base64"
>         }
>       },
>       {
>         "arg": "object",
>         "data": {
>           "type": "secret",
>           "id":"libvirt-5-format-luks-secret0",
>           "data": "9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1",
>           "keyid": "masterKey0",
>           "iv": "AAECAwQFBgcICQoLDA0ODw==",
>           "format": "base64"
>         }
>       },
>       {
>         "arg": "blockdev",
>         "data": {
>           "node-name":"libvirt-5-format",
>           "read-only":false,"driver":"qcow2",
>           "encrypt":{
>             "format":"luks","key-secret":
>             "libvirt-5-format-luks-secret0"
>           },
>           "file":{
>             "driver": "iscsi",
>             "portal": "example.org:6000",
>             "target":"iqn.1992-01.com.example:storage",
>             "lun": 1,
>             "transport": "tcp",
>             "user": "myname",
>             "password-secret": "libvirt-5-storage-secret0",
>             "node-name":"libvirt-5-storage",
>             "auto-read-only":"true",
>             "discard":"unmap"
>           }
>         }
>       }
>     ]
>   }
>   $ qemu-system-x86_64 -f qemu.json
> 
> The config file is more volumous than the CLI, but it is also
> massively more intelligible to humans because you can see the
> structure of the data.
> 
> I still screwed up many times with missing quotes, incorrect
> commas, etc. All the fun of JSON

JSON is usually easy enough to read, quite reasonable for making minor
modifications, and a PITA for writing something (like a QMP request)
from scratch.

> So if we allowed YAML instead of JSON, now we get...
> 
>   $ cat qemu.yaml
>   ---
>   arguments:
>   - arg: object
>     data:
>       type: secret
>       id: libvirt-5-storage-secret0
>       data: 9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1
>       keyid: masterKey0
>       iv: AAECAwQFBgcICQoLDA0ODw==
>       format: base64
>   - arg: object
>     data:
>       type: secret
>       id: libvirt-5-format-luks-secret0
>       data: 9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1
>       keyid: masterKey0
>       iv: AAECAwQFBgcICQoLDA0ODw==
>       format: base64
>   - arg: blockdev
>     data:
>       node-name: libvirt-5-format
>       read-only: false
>       driver: qcow2
>       encrypt:
>         format: luks
>         key-secret: libvirt-5-format-luks-secret0
>       file:
>         driver: iscsi
>         portal: example.org:6000
>         target: iqn.1992-01.com.example:storage
>         lun: 1
>         transport: tcp
>         user: myname
>         password-secret: libvirt-5-storage-secret0
>         node-name: libvirt-5-storage
>         auto-read-only: true
>         discard: unmap
>   $ qemu-system-x86_64 -f qemu.yaml
> 
> This is finally something I'd consider to be on a par with the
> original QEMU syntax, before we added hierarchical data. You
> have the minimal possible amount of syntax here. No commas,
> no quotes, no curly brackets, etc.

This seems to have the same problems as the QEMU command line (how do
you distinguish strings from ints, from bools, from null?). It's
basically just a pretty-printed version of it with the consequence that
it needs to be stored in an external file and there is no reasonable way
to keep it in my shell history.

> > >> IOW, the difficulty with configuring QEMU via JSON is not the fault
> > >> of JSON itself, it is the lack of knowledge amongst users and docs,
> > >> compounded by our never ending "improvements" to the human syntax.
> > >> There are other factors too, such as our only partial coverage of
> > >> config using JSON - some is only possible via the CLI still.
> > >
> > > I'm fine with getting rid of HMP entirely, I think. It's a weird
> > > interface with bizarre behavior that's hard to support.
> > >
> > > There's a few commands in there we just don't support at all, but maybe
> > > it's time to start deprecating one-by-one any of the individual commands
> > > that are better served by QMP these days, to send the message that HMP's
> > > days are numbered.
> > >
> > > Bye-bye!
> > 
> > Experience tells that no matter how weird and bizarre a feature is, once
> > you attempt to remove it, it *will* find champions willing to fight for
> > it to the death.

Hi! :-)

Well, not fighting for it to the death and I'm certainly not married to
its exact syntax, but I use HMP all the time because it's convenient to
use manually and QMP isn't.

If you want to remove HMP, get us a decent QMP shell first. And ideally
a way how to use it without manually configuring a QMP socket and
starting up and connecting the shell to it with ten kilometers of
command line options on both sides first. HMP is available by default,
and so should its replacement be.

> > I'm not trying to tell you "don't go there", only "if
> > you go there, go armed and prepared".
> 
> Yeah and it gets worse the older / more mature a project is. This
> is what pushes projects into doing a clean break "2.0" design,
> which is something we've never done for QEMU.  I don't mean just
> breaking backcompat in a handful of CLI args here. I mean a proper
> scorched earth, everything is up for re-evaluation, "2.0".

As a developer I would love this because so many things in the current
design are wrong. As a user I would hate it because this kind of "2.0"
release tends to break lots of features I like. (Anyone remember KDE 4?)

Kevin