Re: Making QEMU easier for management tools and applications

[John Snow <jsnow@redhat.com>]

On 12/24/19 8:41 AM, Daniel P. Berrangé wrote:
>> * scripts/qmp/qmp-shell
>>
>>   Half-hearted attempt at a human-friendly wrapper around the JSON
>>   syntax.  I have no use for this myself.
> I use this fairly often as its a useful debugging / experimentation
> / trouble shooting tool. There's similar ish functionality in
> virsh qemu-monitor-command. I think there's scope of a supported
> tool here that can talk to libvirt or a UNIX socket for doing
> QMP commands, with a friendlier syntax & pretty printing. 
> 

qmp-shell is one of my go-to tools for working through bitmap workflows
where we don't have convenience commands yet, as some of the setups
required for fleecing et al involve quite a number of steps.

I can copy-paste raw JSON into a socket, but personally I like seeing my
commands neatly organized in a format where I can visually reduce them
to their components at a glance.

(What I mean is: It's hard to remember which QMP commands you've barfed
into a terminal because JSON is hard to read and looks very visually
repetitive.)

I tried to rewrite qmp-shell late last year, actually. I wanted to write
a new REPL that was json-aware in some manner such that you could write
multi-line commands like this:

> example-command arg={
  "hello": "world"
}

This requires, sadly, a streamable JSON parser. Most JSON parsers built
into Python as-is simply take a file pointer and consume the entirety of
the rest of the stream -- they don't play very nice with incomplete
input or input that may have trailing data, e.g.:

> example-command arg={
  "hello": "world"
} arg2={
  "oops!": "more json!"
}

Also, due to the nature of JSON as being a single discrete object and
never a stream of objects, no existing JSON parser really supports the
idea of ever seeing more than one object per buffer.

...So I investigated writing a proper grammar for qmp-shell.
Unfortunately, this basically means including the JSON grammar as a
subset of the shell grammar and writing your own parser for it entirely.

I looked into using Python's own lexer; but it's designed to lex
*python*, not *json*. I got a prototype lexer working for this purpose
under a grammar that I think reflects JSON, but I got that sinking
feeling that it was all more trouble than it was worth, and scrapped
working on it any further.

I did not find any other flex/yacc-like tools that seemed properly
idiomatic or otherwise heavily specialized. I gave up on the idea of
writing a new parser.

I'd love to offer a nice robust QMP shell that is available for use by
end users, but the syntax of the shell will need some major considerations.


(Coffee break)


The other thing I wanted to do was offer a TUI-esque backlog of
commands; something reminiscent of mutt or TUI mail clients. Arrow keys
could be used to browse the list of previous TX/RX objects,
spacebar/enter to view them in detail; '?' to pull up the relevant QAPI
documentation in a side-pane. That sort of thing.

I thought it would be nice to offer an "IRC-esque" view of the QMP
protocol where new events stream in from the bottom, and new commands
can be typed into the buffer bar. (I found no convincing way to combine
e.g. readline with a TUI interface. It seemed like you get one or the
other.)

Allowing qmp-shell to show events received asynchronously would also be
a big leap up from the current shell.

It's a lot of fantasy stuff I wasn't *quite* able to tie together.

--js