Re: [Ksummit-discuss] [TECH TOPIC] What kernel documentation could be

From: Jonathan Corbet <corbet@lwn.net>
To: Mauro Carvalho Chehab <mchehab@kernel.org>,
	Jani Nikula <jani.nikula@intel.com>
Cc: ksummit-discuss@lists.linuxfoundation.org, ksummit@lists.linux.dev
Subject: Re: [Ksummit-discuss] [TECH TOPIC] What kernel documentation could be
Date: Thu, 23 Jun 2022 07:40:45 -0600	[thread overview]
Message-ID: <87a6a38plu.fsf@meer.lwn.net> (raw)
In-Reply-To: <20220623105747.079ac92b@sal.lan>

Mauro Carvalho Chehab <mchehab@kernel.org> writes:

>> The solution would be to finally convert the
>> script to a proper python Sphinx extension that can do caching. (This is
>> how it works in Hawkmoth, FWIW.)

I've been pondering on this for a bit, and would like to do it, but I
don't know when I might find the time for it.

> That's one solution, but see: there is already a python extension
> that currently calls kernel-doc everytime. It could, instead,
> cache the rst returned by its first run (or a parsed version of it)
> and use the cached results the other 3 times.
>
> Porting kernel-doc to python could be doable, but not trivial, due to several
> reasons:
>
> - it should keep running standalone, as otherwise debugging parsing issues
>   on kernel-doc would be a lot harder. In particular, kernel-doc --none is
>   really helpful to report kernel-doc tag errors;

Yes, of course.  As Jani noted, that's just how you would do it, not a
problem. 

> - regressions will likely be introduced on a change like that;

The nice thing is ... we already have a really nice regression test in
the form of the current docs build and diff.

> - regular contributors to kernel-doc will need to ramp up with the newer
>   version;

We have those?  That script is a nightmare and nobody goes near it if
they can possibly avoid it.  I would expect to have more contributors
with a decent Python version that doesn't include 25 years of regex
accretion.

> - a port like that could increase the script run time, as the
>   optimizations and regular expressions there could behave different on
>   python.

It could also decrease it by improving caching opportunities, getting
rid of a lot of fork()/exec() pairs and Perl interpreter startups, etc.

I've actually, in a spare moment or two, been doing some profiling of
the kernel docs build and trying to track down the sources of the
slowness.  I am thinking that nearly 700 *million* calls to the iterator
for the C-domain symbol list might have something to do with it...

> True, but independently if the script would be rewritten in python or not,
> one way would be to enrich the 'DOCS:' kernel-doc tag in order to mention 
> there the symbols that belong to each part of the document, e. g. something
> like:
>
> 	/**
> 	 * DOC: foo
> 	 *
> 	 * Some comments...
> 	 *
> 	 * symbols:
> 	 *     foo
> 	 *     bar
> 	 */
>
> One advantage is that all documentation will be on a single place,
> so hopefully it would be easier to maintain.

I'm not quite sure I get this...you want to put the TOC tree in the
source comments?  This looks like the kind of thing that nobody ever
remembers to update, but maybe I'm missing something.

Thanks,

jon