Re: [RFC] doc: fix code snippet build warnings

[Jani Nikula <jani.nikula@linux.intel.com>]

On Wed, 10 Jan 2018, Jonathan Corbet <corbet@lwn.net> wrote:
> On Wed, 10 Jan 2018 15:04:53 +1100
> "Tobin C. Harding" <me@tobin.cc> wrote:
>
>> Posting as RFC in the hope that someone knows how to massage sphinx
>> correctly to fix this patch.
>> 
>> Currently function kernel-doc contains a multi-line code snippet. This
>> is causing sphinx to emit 5 build warnings
>> 
>> 	WARNING: Unexpected indentation.
>> 	WARNING: Unexpected indentation.
>> 	WARNING: Block quote ends without a blank line; unexpected unindent.
>> 	WARNING: Block quote ends without a blank line; unexpected unindent.
>> 	WARNING: Inline literal start-string without end-string.
>> 
>> And the snippet is not rendering correctly in HTML.
>> 
>> We can stop shpinx complaining by using '::' instead of the currently
>> used '``' however this still does not render correctly in HTML. The
>> rendering is [arguably] better but still incorrect. Sphinx renders two
>> function calls thus:
>> 
>> 	:c:func:`rcu_read_lock()`;
>> 
>> The rest of the snippet does however have correct spacing.
>
> The behavior when `` was used is not surprising, that really just does a
> font change.  Once you went with a literal block (with "::") though, the
> situation changes a bit.  That really should work.
>
> I looked a bit.  This isn't a sphinx (or "shpinx" :) problem, the bug is
> in kernel-doc.  Once we go into the literal mode, it shouldn't be
> screwing around with the text anymore.  Of course, kernel-doc doesn't
> understand enough RST to know that.  I'm a little nervous about trying to
> teach it more, but maybe we have to do that; we should certainly be able
> to put code snippets into the docs and have them come through unmolested.

I think eventually the Right Thing would be to move most of kernel-doc's
mucking of the comments (i.e. highlights) into kernel-doc the Sphinx
extension. I've toyed with the idea, but it's not as trivial as I would
like it to be.

Basically it involves flagging all the kernel-doc nodes during the read
phase, and registering handlers to post process the doctrees. At that
stage, it's no longer simple regexp replaces, it's replacing doctree
nodes with ones that have reference nodes within them. It's more
complicated, but at that stage we can ignore stuff that should stay
verbatim. I think it's also possible to check that the reference targets
actually exist. In short, at that phase we have all the knowledge about
the rst structure, parsed by Sphinx, and we don't have to duplicate that
knowledge in kernel-doc the perl script.

Note that this has nothing to do with swithing to Python based
kernel-doc suggested by Markus previously.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center