kernel-doc and xen.git

kernel-doc and xen.git

[Stefano Stabellini]

Hi all,

I would like to ask for your feedback on the adoption of the kernel-doc
format for in-code comments.

In the FuSa SIG we have started looking into FuSa documents for Xen. One
of the things we are investigating are ways to link these documents to
in-code comments in xen.git and vice versa.

In this context, Andrew Cooper suggested to have a look at "kernel-doc"
[1] during one of the virtual beer sessions at the last Xen Summit.

I did give a look at kernel-doc and it is very promising. kernel-doc is
a script that can generate nice rst text documents from in-code
comments. (The generated rst files can then be used as input for sphinx
to generate html docs.) The comment syntax [2] is simple and similar to
Doxygen:

    /**
     * function_name() - Brief description of function.
     * @arg1: Describe the first argument.
     * @arg2: Describe the second argument.
     *        One can provide multiple line descriptions
     *        for arguments.
     */

kernel-doc is actually better than Doxygen because it is a much simpler
tool, one we could customize to our needs and with predictable output.
Specifically, we could add the tagging, numbering, and referencing
required by FuSa requirement documents.

I would like your feedback on whether it would be good to start
converting xen.git in-code comments to the kernel-doc format so that
proper documents can be generated out of them. One day we could import
kernel-doc into xen.git/scripts and use it to generate a set of html
documents via sphinx.

At a minimum we'll need to start the in-code comment blocks with two
stars:

    /**

There could be also other small changes required to make sure the output
is appropriate.


Feedback is welcome!

Cheers,

Stefano

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/kernel-doc
[2] https://www.kernel.org/doc/html/latest/doc-guide/kernel-doc.html

Re: kernel-doc and xen.git

[George Dunlap]

> On Jul 30, 2020, at 2:27 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> Hi all,
> 
> I would like to ask for your feedback on the adoption of the kernel-doc
> format for in-code comments.
> 
> In the FuSa SIG we have started looking into FuSa documents for Xen. One
> of the things we are investigating are ways to link these documents to
> in-code comments in xen.git and vice versa.
> 
> In this context, Andrew Cooper suggested to have a look at "kernel-doc"
> [1] during one of the virtual beer sessions at the last Xen Summit.
> 
> I did give a look at kernel-doc and it is very promising. kernel-doc is
> a script that can generate nice rst text documents from in-code
> comments. (The generated rst files can then be used as input for sphinx
> to generate html docs.) The comment syntax [2] is simple and similar to
> Doxygen:
> 
>    /**
>     * function_name() - Brief description of function.
>     * @arg1: Describe the first argument.
>     * @arg2: Describe the second argument.
>     *        One can provide multiple line descriptions
>     *        for arguments.
>     */
> 
> kernel-doc is actually better than Doxygen because it is a much simpler
> tool, one we could customize to our needs and with predictable output.
> Specifically, we could add the tagging, numbering, and referencing
> required by FuSa requirement documents.
> 
> I would like your feedback on whether it would be good to start
> converting xen.git in-code comments to the kernel-doc format so that
> proper documents can be generated out of them. One day we could import
> kernel-doc into xen.git/scripts and use it to generate a set of html
> documents via sphinx.

`git-grep ‘^/\*\*$’ ` turns up loads of instances of kernel-doc-style comments in the tree already.  I think it makes complete sense to:

1. Start using tools to pull the existing ones into sphinx docs
2. Skim through the existing ones to make sure they’re accurate / useful
3. Add such comments for elements of key importance to the FUSA SIG
4. Encourage people include documentation for new features, &c

 -George

Re: kernel-doc and xen.git

[Ian Jackson]

George Dunlap writes ("Re: kernel-doc and xen.git"):
> > On Jul 30, 2020, at 2:27 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:
...
> > I did give a look at kernel-doc and it is very promising. kernel-doc is
> > a script that can generate nice rst text documents from in-code
> > comments. (The generated rst files can then be used as input for sphinx
> > to generate html docs.) The comment syntax [2] is simple and similar to
> > Doxygen:
> > 
> >    /**
> >     * function_name() - Brief description of function.
> >     * @arg1: Describe the first argument.
> >     * @arg2: Describe the second argument.
> >     *        One can provide multiple line descriptions
> >     *        for arguments.
> >     */
> > 
> > kernel-doc is actually better than Doxygen because it is a much simpler
> > tool, one we could customize to our needs and with predictable output.
> > Specifically, we could add the tagging, numbering, and referencing
> > required by FuSa requirement documents.
> > 
> > I would like your feedback on whether it would be good to start
> > converting xen.git in-code comments to the kernel-doc format so that
> > proper documents can be generated out of them. One day we could import
> > kernel-doc into xen.git/scripts and use it to generate a set of html
> > documents via sphinx.
> 
> `git-grep ‘^/\*\*$’ ` turns up loads of instances of kernel-doc-style comments in the tree already.  I think it makes complete sense to:
> 
> 1. Start using tools to pull the existing ones into sphinx docs
> 2. Skim through the existing ones to make sure they’re accurate / useful
> 3. Add such comments for elements of key importance to the FUSA SIG
> 4. Encourage people include documentation for new features, &c

I have no objection to this.  Indeed switching to something the kernel
folks find useable is likely to be a good idea.

We should ideally convert the existing hypercall documentation, which
is parsed from a bespoke magic comment format by a script in xen.git.

Ian.

Re: kernel-doc and xen.git

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 2170 bytes --]

On Thu, 30 Jul 2020, Ian Jackson wrote:
> George Dunlap writes ("Re: kernel-doc and xen.git"):
> > > On Jul 30, 2020, at 2:27 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:
> ...
> > > I did give a look at kernel-doc and it is very promising. kernel-doc is
> > > a script that can generate nice rst text documents from in-code
> > > comments. (The generated rst files can then be used as input for sphinx
> > > to generate html docs.) The comment syntax [2] is simple and similar to
> > > Doxygen:
> > > 
> > >    /**
> > >     * function_name() - Brief description of function.
> > >     * @arg1: Describe the first argument.
> > >     * @arg2: Describe the second argument.
> > >     *        One can provide multiple line descriptions
> > >     *        for arguments.
> > >     */
> > > 
> > > kernel-doc is actually better than Doxygen because it is a much simpler
> > > tool, one we could customize to our needs and with predictable output.
> > > Specifically, we could add the tagging, numbering, and referencing
> > > required by FuSa requirement documents.
> > > 
> > > I would like your feedback on whether it would be good to start
> > > converting xen.git in-code comments to the kernel-doc format so that
> > > proper documents can be generated out of them. One day we could import
> > > kernel-doc into xen.git/scripts and use it to generate a set of html
> > > documents via sphinx.
> > 
> > `git-grep ‘^/\*\*$’ ` turns up loads of instances of kernel-doc-style comments in the tree already.  I think it makes complete sense to:
> > 
> > 1. Start using tools to pull the existing ones into sphinx docs
> > 2. Skim through the existing ones to make sure they’re accurate / useful
> > 3. Add such comments for elements of key importance to the FUSA SIG
> > 4. Encourage people include documentation for new features, &c
> 
> I have no objection to this.  Indeed switching to something the kernel
> folks find useable is likely to be a good idea.
> 
> We should ideally convert the existing hypercall documentation, which
> is parsed from a bespoke magic comment format by a script in xen.git.

I agree.

Great, thank you both for the feedback!

Re: kernel-doc and xen.git

[Jan Beulich]

On 30.07.2020 03:27, Stefano Stabellini wrote:
> Hi all,
> 
> I would like to ask for your feedback on the adoption of the kernel-doc
> format for in-code comments.
> 
> In the FuSa SIG we have started looking into FuSa documents for Xen. One
> of the things we are investigating are ways to link these documents to
> in-code comments in xen.git and vice versa.
> 
> In this context, Andrew Cooper suggested to have a look at "kernel-doc"
> [1] during one of the virtual beer sessions at the last Xen Summit.
> 
> I did give a look at kernel-doc and it is very promising. kernel-doc is
> a script that can generate nice rst text documents from in-code
> comments. (The generated rst files can then be used as input for sphinx
> to generate html docs.) The comment syntax [2] is simple and similar to
> Doxygen:
> 
>     /**
>      * function_name() - Brief description of function.
>      * @arg1: Describe the first argument.
>      * @arg2: Describe the second argument.
>      *        One can provide multiple line descriptions
>      *        for arguments.
>      */
> 
> kernel-doc is actually better than Doxygen because it is a much simpler
> tool, one we could customize to our needs and with predictable output.
> Specifically, we could add the tagging, numbering, and referencing
> required by FuSa requirement documents.
> 
> I would like your feedback on whether it would be good to start
> converting xen.git in-code comments to the kernel-doc format so that
> proper documents can be generated out of them. One day we could import
> kernel-doc into xen.git/scripts and use it to generate a set of html
> documents via sphinx.

How far is this intended to go? The example is description of a
function's parameters, which is definitely fine (albeit I wonder
if there's a hidden implication then that _all_ functions
whatsoever are supposed to gain such comments). But the text just
says much more generally "in-code comments", which could mean all
of them. I'd consider the latter as likely going too far.

Jan

Re: kernel-doc and xen.git

[Bertrand Marquis]

> On 31 Jul 2020, at 13:29, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 30.07.2020 03:27, Stefano Stabellini wrote:
>> Hi all,
>> 
>> I would like to ask for your feedback on the adoption of the kernel-doc
>> format for in-code comments.
>> 
>> In the FuSa SIG we have started looking into FuSa documents for Xen. One
>> of the things we are investigating are ways to link these documents to
>> in-code comments in xen.git and vice versa.
>> 
>> In this context, Andrew Cooper suggested to have a look at "kernel-doc"
>> [1] during one of the virtual beer sessions at the last Xen Summit.
>> 
>> I did give a look at kernel-doc and it is very promising. kernel-doc is
>> a script that can generate nice rst text documents from in-code
>> comments. (The generated rst files can then be used as input for sphinx
>> to generate html docs.) The comment syntax [2] is simple and similar to
>> Doxygen:
>> 
>>    /**
>>     * function_name() - Brief description of function.
>>     * @arg1: Describe the first argument.
>>     * @arg2: Describe the second argument.
>>     *        One can provide multiple line descriptions
>>     *        for arguments.
>>     */
>> 
>> kernel-doc is actually better than Doxygen because it is a much simpler
>> tool, one we could customize to our needs and with predictable output.
>> Specifically, we could add the tagging, numbering, and referencing
>> required by FuSa requirement documents.
>> 
>> I would like your feedback on whether it would be good to start
>> converting xen.git in-code comments to the kernel-doc format so that
>> proper documents can be generated out of them. One day we could import
>> kernel-doc into xen.git/scripts and use it to generate a set of html
>> documents via sphinx.
> 
> How far is this intended to go? The example is description of a
> function's parameters, which is definitely fine (albeit I wonder
> if there's a hidden implication then that _all_ functions
> whatsoever are supposed to gain such comments). But the text just
> says much more generally "in-code comments", which could mean all
> of them. I'd consider the latter as likely going too far.

The idea in the FuSa project is more to start with external interfaces like hypercall definitions and public headers elements.

Bertrand

> 
> Jan

Re: kernel-doc and xen.git

[George Dunlap]

> On Jul 31, 2020, at 12:29 PM, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 30.07.2020 03:27, Stefano Stabellini wrote:
>> Hi all,
>> 
>> I would like to ask for your feedback on the adoption of the kernel-doc
>> format for in-code comments.
>> 
>> In the FuSa SIG we have started looking into FuSa documents for Xen. One
>> of the things we are investigating are ways to link these documents to
>> in-code comments in xen.git and vice versa.
>> 
>> In this context, Andrew Cooper suggested to have a look at "kernel-doc"
>> [1] during one of the virtual beer sessions at the last Xen Summit.
>> 
>> I did give a look at kernel-doc and it is very promising. kernel-doc is
>> a script that can generate nice rst text documents from in-code
>> comments. (The generated rst files can then be used as input for sphinx
>> to generate html docs.) The comment syntax [2] is simple and similar to
>> Doxygen:
>> 
>>    /**
>>     * function_name() - Brief description of function.
>>     * @arg1: Describe the first argument.
>>     * @arg2: Describe the second argument.
>>     *        One can provide multiple line descriptions
>>     *        for arguments.
>>     */
>> 
>> kernel-doc is actually better than Doxygen because it is a much simpler
>> tool, one we could customize to our needs and with predictable output.
>> Specifically, we could add the tagging, numbering, and referencing
>> required by FuSa requirement documents.
>> 
>> I would like your feedback on whether it would be good to start
>> converting xen.git in-code comments to the kernel-doc format so that
>> proper documents can be generated out of them. One day we could import
>> kernel-doc into xen.git/scripts and use it to generate a set of html
>> documents via sphinx.
> 
> How far is this intended to go? The example is description of a
> function's parameters, which is definitely fine (albeit I wonder
> if there's a hidden implication then that _all_ functions
> whatsoever are supposed to gain such comments). But the text just
> says much more generally "in-code comments", which could mean all
> of them. I'd consider the latter as likely going too far.

I took him to mean comments in the code at the moment, which describe some interface, but aren’t in kernel-doc format.  Naturally we wouldn’t want *all* comments to be stuffed into a document somewhere.

 -George

Re: kernel-doc and xen.git

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 2488 bytes --]

On Fri, 31 Jul 2020, George Dunlap wrote:
> > On Jul 31, 2020, at 12:29 PM, Jan Beulich <jbeulich@suse.com> wrote:
> > 
> > On 30.07.2020 03:27, Stefano Stabellini wrote:
> >> Hi all,
> >> 
> >> I would like to ask for your feedback on the adoption of the kernel-doc
> >> format for in-code comments.
> >> 
> >> In the FuSa SIG we have started looking into FuSa documents for Xen. One
> >> of the things we are investigating are ways to link these documents to
> >> in-code comments in xen.git and vice versa.
> >> 
> >> In this context, Andrew Cooper suggested to have a look at "kernel-doc"
> >> [1] during one of the virtual beer sessions at the last Xen Summit.
> >> 
> >> I did give a look at kernel-doc and it is very promising. kernel-doc is
> >> a script that can generate nice rst text documents from in-code
> >> comments. (The generated rst files can then be used as input for sphinx
> >> to generate html docs.) The comment syntax [2] is simple and similar to
> >> Doxygen:
> >> 
> >>    /**
> >>     * function_name() - Brief description of function.
> >>     * @arg1: Describe the first argument.
> >>     * @arg2: Describe the second argument.
> >>     *        One can provide multiple line descriptions
> >>     *        for arguments.
> >>     */
> >> 
> >> kernel-doc is actually better than Doxygen because it is a much simpler
> >> tool, one we could customize to our needs and with predictable output.
> >> Specifically, we could add the tagging, numbering, and referencing
> >> required by FuSa requirement documents.
> >> 
> >> I would like your feedback on whether it would be good to start
> >> converting xen.git in-code comments to the kernel-doc format so that
> >> proper documents can be generated out of them. One day we could import
> >> kernel-doc into xen.git/scripts and use it to generate a set of html
> >> documents via sphinx.
> > 
> > How far is this intended to go? The example is description of a
> > function's parameters, which is definitely fine (albeit I wonder
> > if there's a hidden implication then that _all_ functions
> > whatsoever are supposed to gain such comments). But the text just
> > says much more generally "in-code comments", which could mean all
> > of them. I'd consider the latter as likely going too far.
> 
> I took him to mean comments in the code at the moment, which describe some interface, but aren’t in kernel-doc format.  Naturally we wouldn’t want *all* comments to be stuffed into a document somewhere.

+1

Re: kernel-doc and xen.git

[Andrew Cooper]

On 30/07/2020 02:27, Stefano Stabellini wrote:
> Hi all,
>
> I would like to ask for your feedback on the adoption of the kernel-doc
> format for in-code comments.
>
> In the FuSa SIG we have started looking into FuSa documents for Xen. One
> of the things we are investigating are ways to link these documents to
> in-code comments in xen.git and vice versa.
>
> In this context, Andrew Cooper suggested to have a look at "kernel-doc"
> [1] during one of the virtual beer sessions at the last Xen Summit.
>
> I did give a look at kernel-doc and it is very promising. kernel-doc is
> a script that can generate nice rst text documents from in-code
> comments. (The generated rst files can then be used as input for sphinx
> to generate html docs.) The comment syntax [2] is simple and similar to
> Doxygen:
>
>     /**
>      * function_name() - Brief description of function.
>      * @arg1: Describe the first argument.
>      * @arg2: Describe the second argument.
>      *        One can provide multiple line descriptions
>      *        for arguments.
>      */
>
> kernel-doc is actually better than Doxygen because it is a much simpler
> tool, one we could customize to our needs and with predictable output.
> Specifically, we could add the tagging, numbering, and referencing
> required by FuSa requirement documents.
>
> I would like your feedback on whether it would be good to start
> converting xen.git in-code comments to the kernel-doc format so that
> proper documents can be generated out of them. One day we could import
> kernel-doc into xen.git/scripts and use it to generate a set of html
> documents via sphinx.
>
> At a minimum we'll need to start the in-code comment blocks with two
> stars:
>
>     /**
>
> There could be also other small changes required to make sure the output
> is appropriate.
>
>
> Feedback is welcome!

I think it goes without saying that I'm +1 to this in principle.

We definitely have some /** comments already, but I have no idea if they
are valid kernel-doc or not.  It seems that the kernel-doc has some
ability to report warnings, so it would be interesting to see what that
spits out.

I also think getting rid of our home-grown syntax for the public headers
will be major improvement.  We actually have a reasonable amount of
ancillary documentation

As with everything else in the Sphinx docs, I'd request that we don't
just blindly throw this in and call it done.  We need to curate
additions properly to avoid the docs turning into a mess.  I'm happy to
help out in my copious free time.

~Andrew

Re: kernel-doc and xen.git

[Bertrand Marquis]

> On 31 Jul 2020, at 15:48, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> 
> On 30/07/2020 02:27, Stefano Stabellini wrote:
>> Hi all,
>> 
>> I would like to ask for your feedback on the adoption of the kernel-doc
>> format for in-code comments.
>> 
>> In the FuSa SIG we have started looking into FuSa documents for Xen. One
>> of the things we are investigating are ways to link these documents to
>> in-code comments in xen.git and vice versa.
>> 
>> In this context, Andrew Cooper suggested to have a look at "kernel-doc"
>> [1] during one of the virtual beer sessions at the last Xen Summit.
>> 
>> I did give a look at kernel-doc and it is very promising. kernel-doc is
>> a script that can generate nice rst text documents from in-code
>> comments. (The generated rst files can then be used as input for sphinx
>> to generate html docs.) The comment syntax [2] is simple and similar to
>> Doxygen:
>> 
>>    /**
>>     * function_name() - Brief description of function.
>>     * @arg1: Describe the first argument.
>>     * @arg2: Describe the second argument.
>>     *        One can provide multiple line descriptions
>>     *        for arguments.
>>     */
>> 
>> kernel-doc is actually better than Doxygen because it is a much simpler
>> tool, one we could customize to our needs and with predictable output.
>> Specifically, we could add the tagging, numbering, and referencing
>> required by FuSa requirement documents.
>> 
>> I would like your feedback on whether it would be good to start
>> converting xen.git in-code comments to the kernel-doc format so that
>> proper documents can be generated out of them. One day we could import
>> kernel-doc into xen.git/scripts and use it to generate a set of html
>> documents via sphinx.
>> 
>> At a minimum we'll need to start the in-code comment blocks with two
>> stars:
>> 
>>    /**
>> 
>> There could be also other small changes required to make sure the output
>> is appropriate.
>> 
>> 
>> Feedback is welcome!
> 
> I think it goes without saying that I'm +1 to this in principle.
> 
> We definitely have some /** comments already, but I have no idea if they
> are valid kernel-doc or not.  It seems that the kernel-doc has some
> ability to report warnings, so it would be interesting to see what that
> spits out.

From my first crash test, not much is “kernel-doc” friendly but the content
is there, it is only a matter of doing some formatting.

> 
> I also think getting rid of our home-grown syntax for the public headers
> will be major improvement.  We actually have a reasonable amount of
> ancillary documentation
> 
> As with everything else in the Sphinx docs, I'd request that we don't
> just blindly throw this in and call it done.  We need to curate
> additions properly to avoid the docs turning into a mess.  I'm happy to
> help out in my copious free time.

Thanks :-)

Bertrand