Re: [PATCH 19/29] qapi/qapi-schema.json: Put headers in their own doc-comment blocks

[Peter Maydell <peter.maydell@linaro.org>]

On Fri, 7 Feb 2020 at 15:35, Markus Armbruster <armbru@redhat.com> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > Our current QAPI doc-comment markup allows section headers
> > (introduced with a leading '=' or '==') anywhere in any documentation
> > comment.  This works for texinfo because the texi generator simply
> > prints a texinfo heading directive at that point in the output
> > stream.  For rST generation, since we're assembling a tree of
> > docutils nodes, this is awkward because a new section implies
> > starting a new section node at the top level of the tree and
> > generating text into there.
> >
> > New section headings in the middle of the documentation of a command
> > or event would be pretty nonsensical, and in fact we only ever output
> > new headings using 'freeform' doc comment blocks whose only content
> > is the single line of the heading, with two exceptions, which are in
> > the introductory freeform-doc-block at the top of
> > qapi/qapi-schema.json.
> >
> > Split that doc-comment up so that the heading lines are in their own
> > doc-comment.  This will allow us to tighten the specification to
> > insist that heading lines are always standalone, rather than
> > requiring the rST document generator to look at every line in a doc
> > comment block and handle headings in odd places.

> I figure this is a minimally invasive patch to avoid complications in
> your rST generator.  I'm afraid it sweeps the actual problem under the
> rug, namely flaws in our parsing and representation of doc comments.
>
> The doc comment parser doesn't recognize headings.  Instead, that's done
> somewhere in the bowels of the Texinfo generator.  Works as long as the
> input is "sane", happily generates invalid Texinfo otherwise, see
> tests/qapi-schema/doc-bad-section.json.
>
> The proper fix is to make the parser recognize headers in the places
> where headers make sense, and reject them elsewhere.
>
> But maybe we don't have to.  Do you plan to support full rST in doc
> comments?  If yes, why have our own syntax for headings?  Why not leave
> it to rST?  If no, do you plan to support a subset of rST?  If yes,
> define it, please.  How will it be enforced?

Doc comments do support full rST. However, (as the commit message
here notes), if you're generating a tree of docutils nodes and
one of them has a section heading in it then you'll get a result
that looks like this:

[root]
  - [ some section created by the script for a QAPI command ]
  - [ some section ]
      - [text nodes, etc going into this section]
      - [a section resulting from rST parsing the header inside the docstring]
  - [ next section created by the script for a QAPI command ]

(ie you'll have defined a subsection within whatever document
paragraph/section the current command is documenting, not
a new top-level subsection which subsequent commands will
become children of)

What you actually want is that the new header results in
a differently structured tree:
[root]
  - [ some section created by the script for a QAPI command ]
  - [ some section ]
      - [text nodes, etc going into this section]
  - [ a new top level section whose header is whatever this header is ]
     - [ next section created by the script is a child of that section ]
     - [ etc ]

There's no way to get that without actually noticing and handling
headings specially as being something entirely different from
a lump of documentation text. "A heading is a single-line special-case
of a freeform comment" happens to be the way we mark up headings
now in 99% of cases, so that's what I implemented. (The Sphinx
extension will complain if there's trailing junk lines after
a heading line at the beginning of a freeform comment block.
If you use '== something' in a line in the middle of a doc
comment, we'll just interpret that as rST source, which is to
say a couple of literal equals signs at the start of a line.)

thanks
-- PMM