Re: [PATCH] schemas: Add vim modeline

[Kevin Wolf <kwolf@redhat.com>]

Am 31.07.2020 um 11:01 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
> 
> > Am 30.07.2020 um 17:11 hat Eric Blake geschrieben:
> >> > JSON is a exceptionally poor choice for a DSL, or even a configuration
> >> > language.
> >> > 
> >> > Correcting our mistake involves a flag day and a re-learn.  We need to
> >> > weigh costs against benefits.
> >> > 
> >> > The QAPI schema language has two layers:
> >> > 
> >> > * JSON, with a lexical and a syntactical sub-layer (both in parser.py)
> 
> An incompatible dialect of JSON with a doc comment language, actually.
> 
> The need to keep doc generation working could complicate replacing the
> lower layer.

Good point, we would have to keep the comment parser either way to be
used on top of the YAML (or whatever) parser.

Whatever parser we use would have to actually make comments available
rather than immediately filtering them out. This might exist for most
languages, but it will probably not be the most commonly used parser
either (or at least it will not allow using a simple interface like
json.loads() in Python).

> >> > 
> >> > * QAPI, with a context-free and a context-dependend sub-layer (in
> >> >    expr.py and schema.py, respectively)
> >> > 
> >> > Replacing the JSON layer is possible as long as the replacement is
> >> > sufficiently expressive (not a tall order).
> >> 
> >> I'm open to the idea, if we want to attempt it, and agree with the
> >> assessment that it is not a tall order.
> 
> Careful, "not a tall order" is meant to apply to the "sufficiently
> expressive" requirement for a replacemnt syntax.
> 
> On actually replacing the lower layer, I wrote "we need to weigh costs
> against benefits."
> 
> > I'm not so sure about that. I mean, it certainly sounds doable if need
> > be, but getting better syntax highlighting by default in some editors
> > feels like a pretty weak reason to switch out the complete schema
> > language.
> >
> > At first I was going to say "but if you don't have anything else to do
> > with your time...", but it's actually not only your time, but the time
> > of everyone who has development branches or downstream repositories and
> > will suffer rather nasty merge conflicts. So this will likely end up
> > having a non-negligible cost.
> 
> Yup.
> 
> > So is there more to it or are we really considering doing this just
> > because editors can tell more easily what to do with a different syntax?
> 
> If memory serves, the following arguments have been raised:
> 
> 1. A chance to improve ergonomics for developers
> 
>    Pain points include
> 
>    - Confusion
> 
>      It claims to be JSON, but it's not.
> 
>    - Need to learn another syntax
> 
>      Sunk cost for old hands, but it's a valid point all the same.

We use a similar (same?) form of "almost JSON" for QMP which will still
exist. So we'd be moving from having to learn one (non-standard)
language to two languages (one still non-standard and another one that
is hopefully more standard).

>    - Poor tool support
> 
>      JSON tools don't work.  Python tools do, but you may have to work
>      around the issue of true, false.

This is mostly the editor question this patch was about, right? Or are
people trying to use more sophisticated tools on it?

>    - Excessive quoting
> 
>    - Verbosity
> 
>      When all you have is KEY: VALUE, defining things with multiple
>      properties becomes verbose like
> 
>          'status': { 'type': 'DirtyBitmapStatus',
>                      'features': [ 'deprecated' ] }
> 
>      We need syntactic sugar to keep vebosity in check for the most
>      common cases.  More complexity.

I don't think this is something any of the suggested languages would
address.

>    - No trailing comma in arrays and objects
> 
>    - No way to split long strings for legibility
> 
>    - The doc comment language is poorly specified
> 
>    - Parse error reporting could be better (JSON part) / could hardly be
>      worse (doc comment part)

Has anyone looked into what error messages are like for the suggested
alternatives? "error reporting could be better" is something that is
true for a lot of software.

The doc comment part is not going to change unless we get rid of
comments and actually make documentation part of the objects themselves.
This might not be very readable.

Or I should rather say, making the doc comment part change is possible,
but would require the same changes as with our current lower layer
language and parser.

> 2. Not having to maintain our own code for the lower layer
> 
>    I consider this argument quite weak.  parser.py has some 400 SLOC.
>    Writing and rewriting it is sunk cost.  Keeping it working has been
>    cheap.  Keeping the glue for some off-the-shelf parser working isn't
>    free, either.  No big savings to be had here, sorry.
> 
>    Almost half of parser.c is about doc comments, and it's the hairier
>    part by far.  Peter has patches to drag the doc comment language
>    closer to rST.  I don't remember whether they shrink parser.py.
> 
> 3. Make the schema more easily consumable by other programs
> 
>    Use of a "standard" syntax instead of our funky dialect of JSON means
>    other programs can use an off-the-shelf parser instead of using or
>    reimplementing parser.py.
> 
>    Valid point for programs that parse the lower layer, and no more, say
>    for basic syntax highlighting.
> 
>    Pretty much irrelevant for programs that need to go beyond the lower
>    layer.  Parsing the lower layer is the easy part.  The code dealing
>    with the upper layer is much larger (expr.py and schema.py), and it
>    actually changes as we add features to the schema language.
>    Duplicating it would be a Bad Idea.  Reuse the existing frontend
>    instead.

Do other programs that go beyond syntax highlighting actually get to
parse our QAPI schema definitions? Or would they rather deal with the
return value of query-qmp-schema?

Neither the QAPI schema nor a YAML file with the same structure are a
standard approach to describe JSON documents. So even if we replace JSON
in the lower layer, the whole thing (and as you say, the upper layer is
the more interesting part) still stays non-standard in a way and more
advanced tools can't be used with it.

And of course, even if we did use something more standard like JSON
Schema or whatever exists for YAML, we would still have to massively
extend it because the QAPI schema contains much more information than
just what would be needed to validate some input. We control all aspects
of generated C code with it.

Kevin