[Ksummit-discuss] [TECH TOPIC] Documentation

[Ksummit-discuss] [TECH TOPIC] Documentation

[Jonathan Corbet]

The long process of converting the kernel's documentation into RST is
finally coming to an end...what has that bought us?  We have gone from a
chaotic pile of incomplete, crufty, and un-integrated docs to a slightly
better organized pile of incomplete, crufty, slightly better integrated
docs.  Plus we have the infrastructure to make something better from here.

What are the next steps for kernel documentation?  What would we really
like our docs to look like, and how might we find the resources to get
them to that point?  What sorts of improvements to the build
infrastructure would be useful?  I'll come with some ideas (some of which
you've certainly heard before) but will be more interested in listening.

jon
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Dan Carpenter]

On Tue, Jun 09, 2020 at 02:53:53PM -0600, Jonathan Corbet wrote:
> The long process of converting the kernel's documentation into RST is
> finally coming to an end...what has that bought us?  We have gone from a
> chaotic pile of incomplete, crufty, and un-integrated docs to a slightly
> better organized pile of incomplete, crufty, slightly better integrated
> docs.  Plus we have the infrastructure to make something better from here.
> 
> What are the next steps for kernel documentation?  What would we really
> like our docs to look like, and how might we find the resources to get
> them to that point?  What sorts of improvements to the build
> infrastructure would be useful?  I'll come with some ideas (some of which
> you've certainly heard before) but will be more interested in listening.

The truth is that I only ever read the documentation in front of the
function implementations.  And LDD.  ;)

There are some subsystems which document their functions in the
Documentation directory.  For example, pm_request_resume() is documented
in Documentation/power/runtime_pm.rst where it describes why it might
return 1 on success.  It would be awesome if we had an automated way to
find this.

I wish that there were a replacement for cscope which showed results in
a sorted order:

Global Implementation
Documentation
Local Implemenatations
(For me the struct device is my 995th result...  Cscope is terrible.
But it's what I use as my default search in the kernel source.  I just
would like to have one search for everything that would find
documentation as well).

regards,
dan carpenter



_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Daniel Vetter]

On Wed, Jun 10, 2020 at 10:51 AM Dan Carpenter <dan.carpenter@oracle.com> wrote:
>
> On Tue, Jun 09, 2020 at 02:53:53PM -0600, Jonathan Corbet wrote:
> > The long process of converting the kernel's documentation into RST is
> > finally coming to an end...what has that bought us?  We have gone from a
> > chaotic pile of incomplete, crufty, and un-integrated docs to a slightly
> > better organized pile of incomplete, crufty, slightly better integrated
> > docs.  Plus we have the infrastructure to make something better from here.
> >
> > What are the next steps for kernel documentation?  What would we really
> > like our docs to look like, and how might we find the resources to get
> > them to that point?  What sorts of improvements to the build
> > infrastructure would be useful?  I'll come with some ideas (some of which
> > you've certainly heard before) but will be more interested in listening.
>
> The truth is that I only ever read the documentation in front of the
> function implementations.  And LDD.  ;)
>
> There are some subsystems which document their functions in the
> Documentation directory.  For example, pm_request_resume() is documented
> in Documentation/power/runtime_pm.rst where it describes why it might
> return 1 on success.  It would be awesome if we had an automated way to
> find this.

My recommendation is to move these into the code, as kerneldoc. And
then pull it into the .rst using one of the kerneldoc include
directives.

Large chunks of Documentation/gpu/*.rst files are just scaffolding to
pull the comments out of source files. The only thing we have in .rst
files is the big picture overview stuff - even local overview stuff is
simply done as DOC: kerneldoc comments in source code.
-Daniel


>
> I wish that there were a replacement for cscope which showed results in
> a sorted order:
>
> Global Implementation
> Documentation
> Local Implemenatations
> (For me the struct device is my 995th result...  Cscope is terrible.
> But it's what I use as my default search in the kernel source.  I just
> would like to have one search for everything that would find
> documentation as well).
>
> regards,
> dan carpenter
>
>
>
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss



--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Linus Walleij]

On Tue, Jun 9, 2020 at 10:54 PM Jonathan Corbet <corbet@lwn.net> wrote:

> What are the next steps for kernel documentation?  What would we really
> like our docs to look like, and how might we find the resources to get
> them to that point?

We have a whole slew of quite basic introductory materials to the very
fundamental kernel data structures that are scattered all over the planet
and in the LWN kernel archive. All in often slightly dated variants.
Examples:
https://www.kernel.org/doc/gorman/html/understand/understand006.html
http://vger.kernel.org/~davem/skb_data.html

I suppose technical writers could benefit from a global TODO list with
this kind of subjects such as "document the basic arch interface",
"document what an skb is", "document how the page directory works".

Right now it is a mixture of read the code, "everybody knows how that
works" and random sources on the Internet, which isn't very helpful
to newcomers.

I second Vetters point to tie this documentation in tightly with the code
implementing it even pretty substantial text chunks IMO, the idea can be
attributed to Donald Knuth's book on literate programming from 1984,
and while he was a bit enthusiastic the idea isn't bad at all.

Yours,
Linus Walleij
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Shuah Khan]

On 6/11/20 8:48 AM, Linus Walleij wrote:
> On Tue, Jun 9, 2020 at 10:54 PM Jonathan Corbet <corbet@lwn.net> wrote:
> 
>> What are the next steps for kernel documentation?  What would we really
>> like our docs to look like, and how might we find the resources to get
>> them to that point?
> 
> We have a whole slew of quite basic introductory materials to the very
> fundamental kernel data structures that are scattered all over the planet
> and in the LWN kernel archive. All in often slightly dated variants.
> Examples:
> https://www.kernel.org/doc/gorman/html/understand/understand006.html
> http://vger.kernel.org/~davem/skb_data.html
> 
> I suppose technical writers could benefit from a global TODO list with
> this kind of subjects such as "document the basic arch interface",
> "document what an skb is", "document how the page directory works".
> 

I totally agree. A solid TODO list is necessary to channel efforts.

> Right now it is a mixture of read the code, "everybody knows how that
> works" and random sources on the Internet, which isn't very helpful
> to newcomers.
> 

Also true. I New comers are intimidated to ask questions in the first
place and they might not get responses. It boils down to bandwidth
more often than not. People that know the sub-systems well might not
have time to document and more importantly keeping the documents
updated.

> I second Vetters point to tie this documentation in tightly with the code
> implementing it even pretty substantial text chunks IMO, the idea can be
> attributed to Donald Knuth's book on literate programming from 1984,
> and while he was a bit enthusiastic the idea isn't bad at all.
> 

It definitely helps to that. I do see lots of calls without no comments
and usage information. For this to work, we have to give feedback during
the review process. This is necessary for commit logs as well.

I am interested in this topic as I help new developers and I often hear
the same concern that there aren't resources for them to learn and
understand kernel subsystems.

I have been reaching out to new contributors to kernel for the last
6 releases starting with Linux 5.1 asking

"What suggestions do you have for improving the materials and guidance
available to new contributors?"

The feedback always is request for updated information in kernel
documentation and online resources. I will be glad to share detailed
feedback if there is interest.

Addressing the need in providing kernel documentation could address
the need.

thanks,
-- Shuah
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Joe Perches]

On Thu, 2020-06-11 at 12:03 -0600, Shuah Khan wrote:
> People that know the sub-systems well might not
> have time to document and more importantly keeping the documents
> updated.

So you somehow expect people that do _not_ know the
sub-system well to produce good and useful documentation?


_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Shuah Khan]

On 6/11/20 12:28 PM, Joe Perches wrote:
> On Thu, 2020-06-11 at 12:03 -0600, Shuah Khan wrote:
>> People that know the sub-systems well might not
>> have time to document and more importantly keeping the documents
>> updated.
> 
> So you somehow expect people that do _not_ know the
> sub-system well to produce good and useful documentation?
> 
> 

I didn't suggest anything like that. I merely mentioned that
lack of bandwidth is a reason for outdated documentation.

As for solutions, I am hoping one or two viable options will
bubble up as we talk about the issue on this thread and at
the kernel summit.

thanks,
-- Shuah
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Laurent Pinchart]

H Shuah,

On Thu, Jun 11, 2020 at 01:44:45PM -0600, Shuah Khan wrote:
> On 6/11/20 12:28 PM, Joe Perches wrote:
> > On Thu, 2020-06-11 at 12:03 -0600, Shuah Khan wrote:
> >> People that know the sub-systems well might not
> >> have time to document and more importantly keeping the documents
> >> updated.
> > 
> > So you somehow expect people that do _not_ know the
> > sub-system well to produce good and useful documentation?
> 
> I didn't suggest anything like that. I merely mentioned that
> lack of bandwidth is a reason for outdated documentation.

I think it may be more of a priority issue than a bandwidth issue. Most
subsystems have traditionally not put lots of effort in documenting
designs and APIs, and very few of them, even today, enforce
documentation rules. I don't disagree that some area of the kernel have
little available bandwidth, we have some very small subsystems, or niche
areas where only a handful of people would be able to understand what's
going on (and if you ask me, that's a reason to have more documentation,
not less :-)). We also have large subsystems where policies have been
put in place over time to get developers to write documentation. I'm
thinking about V4L2 or DRM for instance, where the former has extensive
documentation of the userspace API, and the latter extensive
documentation of the in-kernel API. I believe other subsystems could
follow if maintainers considered lack of documentation to be a blocker
on patch submission.

This would however leave some developers in an uncomfortable situation.
While writing documentation is no fun for most of us, it gets much worse
for developers who are less fluent in English. Learning how to write
good documentation is hard enough without having to also learn a new
language. I think this is a problem that needs to be addressed on its
own.

Another issue that I also believe is important is where to put the bar
when it comes to documentation quality. I've reviewed many documentation
patches where I wasn't completely happy with the proposal (as I'm prone
to nit-picking during review I'll take blames here). Something that is
clear to the author of the code won't be to someone who has less
knowledge in that area, and more often that not patch authors don't
write documentation from the point of view of the reader. This is
something that can be learnt though, and I believe the kernel would
benefit from involving more technical writers who could help reviewing
documentation submissions and point out inconsistencies. There's a
matter of budget there though, it may be difficult to attract technical
writers who would help just for the fun of it.

> As for solutions, I am hoping one or two viable options will
> bubble up as we talk about the issue on this thread and at
> the kernel summit.

-- 
Regards,

Laurent Pinchart
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mike Rapoport]

On Thu, Jun 11, 2020 at 11:28:09AM -0700, Joe Perches wrote:
> On Thu, 2020-06-11 at 12:03 -0600, Shuah Khan wrote:
> > People that know the sub-systems well might not
> > have time to document and more importantly keeping the documents
> > updated.
> 
> So you somehow expect people that do _not_ know the
> sub-system well to produce good and useful documentation?

Writing documentation is a way to learn.

Besides, to produce good and usefull documentation it is not necessary
to have very deep understanding of all the small details. The writer
must capture the large picture and be able to explain it.
 
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

-- 
Sincerely yours,
Mike.
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mike Rapoport]

On Fri, Jun 12, 2020 at 11:18:52AM +0300, Laurent Pinchart wrote:
> H Shuah,
> 
> On Thu, Jun 11, 2020 at 01:44:45PM -0600, Shuah Khan wrote:
> > On 6/11/20 12:28 PM, Joe Perches wrote:
> > > On Thu, 2020-06-11 at 12:03 -0600, Shuah Khan wrote:
> > >> People that know the sub-systems well might not
> > >> have time to document and more importantly keeping the documents
> > >> updated.
> > > 
> > > So you somehow expect people that do _not_ know the
> > > sub-system well to produce good and useful documentation?
> > 
> > I didn't suggest anything like that. I merely mentioned that
> > lack of bandwidth is a reason for outdated documentation.
> 
> I think it may be more of a priority issue than a bandwidth issue.

Second that. Writing code usually seems more rewarding :)

> Most subsystems have traditionally not put lots of effort in documenting
> designs and APIs, and very few of them, even today, enforce
> documentation rules. I don't disagree that some area of the kernel have
> little available bandwidth, we have some very small subsystems, or niche
> areas where only a handful of people would be able to understand what's
> going on (and if you ask me, that's a reason to have more documentation,
> not less :-)). We also have large subsystems where policies have been
> put in place over time to get developers to write documentation. I'm
> thinking about V4L2 or DRM for instance, where the former has extensive
> documentation of the userspace API, and the latter extensive
> documentation of the in-kernel API. I believe other subsystems could
> follow if maintainers considered lack of documentation to be a blocker
> on patch submission.
>
> This would however leave some developers in an uncomfortable situation.
> While writing documentation is no fun for most of us, it gets much worse
> for developers who are less fluent in English. Learning how to write
> good documentation is hard enough without having to also learn a new
> language. I think this is a problem that needs to be addressed on its
> own.

I think we should make a distinction between adding new documentation
and updating the existing one. I agree that writing documentation is not
easy, especially when you need to write it from scratch. But making
small amends to the existing documentation along with the code changes
so that docs and code would still much is not that difficult. 
For such cases a request to have documentation update as a part of the
changes wouldn't be too much to ask.

> Another issue that I also believe is important is where to put the bar
> when it comes to documentation quality. I've reviewed many documentation
> patches where I wasn't completely happy with the proposal (as I'm prone
> to nit-picking during review I'll take blames here). Something that is
> clear to the author of the code won't be to someone who has less
> knowledge in that area, and more often that not patch authors don't
> write documentation from the point of view of the reader. This is
> something that can be learnt though, and I believe the kernel would
> benefit from involving more technical writers who could help reviewing
> documentation submissions and point out inconsistencies. There's a
> matter of budget there though, it may be difficult to attract technical
> writers who would help just for the fun of it.
> 
> > As for solutions, I am hoping one or two viable options will
> > bubble up as we talk about the issue on this thread and at
> > the kernel summit.
> 
> -- 
> Regards,
> 
> Laurent Pinchart

-- 
Sincerely yours,
Mike.
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1301 bytes --]

On Fri, Jun 12, 2020 at 12:19:39PM +0300, Mike Rapoport wrote:
> On Fri, Jun 12, 2020 at 11:18:52AM +0300, Laurent Pinchart wrote:

> > This would however leave some developers in an uncomfortable situation.
> > While writing documentation is no fun for most of us, it gets much worse
> > for developers who are less fluent in English. Learning how to write
> > good documentation is hard enough without having to also learn a new
> > language. I think this is a problem that needs to be addressed on its
> > own.

> I think we should make a distinction between adding new documentation
> and updating the existing one. I agree that writing documentation is not
> easy, especially when you need to write it from scratch. But making
> small amends to the existing documentation along with the code changes
> so that docs and code would still much is not that difficult. 
> For such cases a request to have documentation update as a part of the
> changes wouldn't be too much to ask.

I wouldn't underestimate the barriers this can create for developers who
are less fluent in English, take a browse through changelogs sometime.
There's a big jump from being able to write understandable code and
communicate what it's about to being able to write clear and useful
documentation.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 186 bytes --]

_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Shuah Khan]

On 6/12/20 2:18 AM, Laurent Pinchart wrote:
> H Shuah,
> 
> On Thu, Jun 11, 2020 at 01:44:45PM -0600, Shuah Khan wrote:
>> On 6/11/20 12:28 PM, Joe Perches wrote:
>>> On Thu, 2020-06-11 at 12:03 -0600, Shuah Khan wrote:
>>>> People that know the sub-systems well might not
>>>> have time to document and more importantly keeping the documents
>>>> updated.
>>>
>>> So you somehow expect people that do _not_ know the
>>> sub-system well to produce good and useful documentation?
>>
>> I didn't suggest anything like that. I merely mentioned that
>> lack of bandwidth is a reason for outdated documentation.
> 
> I think it may be more of a priority issue than a bandwidth issue. Most
> subsystems have traditionally not put lots of effort in documenting
> designs and APIs, and very few of them, even today, enforce
> documentation rules.

First of all - thanks for a very detailed email.

+1. Yes priority is probably a bigger reason. We can make time for the
things we care about. In general documentation isn't fun for engineers
and also not as glamorous.

I don't disagree that some area of the kernel have
> little available bandwidth, we have some very small subsystems, or niche
> areas where only a handful of people would be able to understand what's
> going on (and if you ask me, that's a reason to have more documentation,
> not less :-)). We also have large subsystems where policies have been
> put in place over time to get developers to write documentation.

> I'm
> thinking about V4L2 or DRM for instance, where the former has extensive
> documentation of the userspace API, and the latter extensive
> documentation of the in-kernel API. I believe other subsystems could
> follow if maintainers considered lack of documentation to be a blocker
> on patch submission.
> 

Correct. Media subsystem does this very well and can be held up as an
example for good documentation mainly because media maintainers
emphasize and make it a priority. I have been well trained by to start
with documentation when I am doing media work. :)

I follow this for the driver I maintain and kselftest. The latter not
as successfully. It is harder to have a cohesive strategy for subsystem
or an area that is generic and have more people involved, unless
everybody is on board and agree that documentation is important.

> This would however leave some developers in an uncomfortable situation.
> While writing documentation is no fun for most of us, it gets much worse
> for developers who are less fluent in English. Learning how to write
> good documentation is hard enough without having to also learn a new
> language. I think this is a problem that needs to be addressed on its
> own.
> 

Yes. That is also correct. The way I address it depending on bandwidth
is working during the reviews to get get documentation right, when I
consider it very important to understand the patch and be able to
maintain it later.

> Another issue that I also believe is important is where to put the bar
> when it comes to documentation quality. I've reviewed many documentation
> patches where I wasn't completely happy with the proposal (as I'm prone
> to nit-picking during review I'll take blames here). Something that is
> clear to the author of the code won't be to someone who has less
> knowledge in that area, and more often that not patch authors don't
> write documentation from the point of view of the reader. This is
> something that can be learnt though, and I believe the kernel would
> benefit from involving more technical writers who could help reviewing
> documentation submissions and point out inconsistencies. There's a
> matter of budget there though, it may be difficult to attract technical
> writers who would help just for the fun of it.
> 

Right. I find myself in the same place as you. I end up asking for info.
which annoys people at times. I find documenting work is an important
step in the development process. I might be in the minority, however, at
times, as I am writing my commit log explaining what I am doing in a
patch in, I find a better way to fix a problem.

So I find it useful and I also use it as learning tool. When I attempt
document or explain a concept or an area, I get a deeper understanding
of that area. I would counter the argument that new developers can't
document an area, they can and the document can be reviewed by experts.
I have had some level of success adding documentation tasks in the
mentoring program. It does requires experts spending time reviewing to
make sure it is correct.

The larger problem goes back to, documentation isn't glamorous. It
isn't valued at the same level of code. If we collectively agree that
it is, then we can make it happen. We might sending a wrong message
by saying "Code walks ...". I understand what we mean by that we don't
want a long proposal and we would rather see code.

Maybe we can amend it to say "Code with good documentation walks ...".

The documentation in my mind includes:

commit logs
doc area at the top of functions declarations
API documentation

thanks,
-- Shuah
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Shuah Khan]

On 6/12/20 3:07 AM, Mike Rapoport wrote:
> On Thu, Jun 11, 2020 at 11:28:09AM -0700, Joe Perches wrote:
>> On Thu, 2020-06-11 at 12:03 -0600, Shuah Khan wrote:
>>> People that know the sub-systems well might not
>>> have time to document and more importantly keeping the documents
>>> updated.
>>
>> So you somehow expect people that do _not_ know the
>> sub-system well to produce good and useful documentation?
> 
> Writing documentation is a way to learn.
> 

+1

I find it useful and I also use it as learning tool. When I attempt
document or explain a concept or an area, I get a deeper understanding
of that area.

> Besides, to produce good and usefull documentation it is not necessary
> to have very deep understanding of all the small details. The writer
> must capture the large picture and be able to explain it.

+1.

I would counter the argument that new developers can't document an area,
they can and the document can be reviewed by experts.

A fresh set of eyes bring a new perspective and could help improve the
code, just like the benefits of having others test your code.

I have had some level of success adding documentation tasks in the
mentoring program. It does require experts spending time reviewing to
make sure it is correct.

One thing we could do is creating a list documentation todo and review
which is part of our development workflow anyway.

thanks,
-- Shuah
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Julia Lawall]

I tried collecting information about missing or incorrectly formatted
function documentation using Coccinelle.  Here is an example of the
output:

drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: extra names mod, f_isr, h_src_arg
drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name isr_cb
drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name module
drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name src_arg
drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: return comment but no return value
drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: extra names mod
drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: missing name module
drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: return comment but no return value
drivers/net/ethernet/freescale/fman/fman.c:2355 fman_get_revision: return comment but no return value
drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: extra names dev
drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: missing name fm_dev
drivers/net/ethernet/freescale/fman/fman.c:2527 fman_has_errata_a050385: no comment for fman_has_errata_a050385

It's not illustrated here, but if it can find documentation for the same
variable with the same type somewhere else in the same file, it suggests
that, and it finds a mention of the function in the Documentation
directory, it give the name of that file.

I assumed that all EXPORT_SYMBOLs and EXPORT_SYMBOL_GPLs should be
documented.  Perhaps this was an overapproximation, because it seems like
a lot of them are not.

julia
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Joe Perches]

On Sat, 2020-06-13 at 18:42 +0200, Julia Lawall wrote:
> I tried collecting information about missing or incorrectly formatted
> function documentation using Coccinelle.  Here is an example of the
> output:
> 
> drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: extra names mod, f_isr, h_src_arg
> drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name isr_cb
> drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name module
> drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name src_arg
> drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: return comment but no return value
> drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: extra names mod
> drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: missing name module
> drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: return comment but no return value
> drivers/net/ethernet/freescale/fman/fman.c:2355 fman_get_revision: return comment but no return value
> drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: extra names dev
> drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: missing name fm_dev
> drivers/net/ethernet/freescale/fman/fman.c:2527 fman_has_errata_a050385: no comment for fman_has_errata_a050385

just fyi: scripts/kernel-doc already does:

$ ./scripts/kernel-doc drivers/net/ethernet/freescale/fman/fman.c > /dev/null
drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function parameter or member 'module' not described in 'fman_register_intr'
drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function parameter or member 'isr_cb' not described in 'fman_register_intr'
drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function parameter or member 'src_arg' not described in 'fman_register_intr'
drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function parameter 'mod' description in 'fman_register_intr'
drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function parameter 'f_isr' description in 'fman_register_intr'
drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function parameter 'h_src_arg' description in 'fman_register_intr'
drivers/net/ethernet/freescale/fman/fman.c:2105: warning: Function parameter or member 'module' not described in 'fman_unregister_intr'
drivers/net/ethernet/freescale/fman/fman.c:2105: warning: Excess function parameter 'mod' description in 'fman_unregister_intr'
drivers/net/ethernet/freescale/fman/fman.c:2356: warning: Function parameter or member 'fman' not described in 'fman_get_revision'
drivers/net/ethernet/freescale/fman/fman.c:2356: warning: Function parameter or member 'rev_info' not described in 'fman_get_revision'
drivers/net/ethernet/freescale/fman/fman.c:2521: warning: Function parameter or member 'fm_dev' not described in 'fman_bind'
drivers/net/ethernet/freescale/fman/fman.c:2521: warning: Excess function parameter 'dev' description in 'fman_bind'



_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Julia Lawall]

On Sat, 13 Jun 2020, Joe Perches wrote:

> On Sat, 2020-06-13 at 18:42 +0200, Julia Lawall wrote:
> > I tried collecting information about missing or incorrectly formatted
> > function documentation using Coccinelle.  Here is an example of the
> > output:
> >
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: extra names mod, f_isr, h_src_arg
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name isr_cb
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name module
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name src_arg
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: return comment but no return value
> > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: extra names mod
> > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: missing name module
> > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: return comment but no return value
> > drivers/net/ethernet/freescale/fman/fman.c:2355 fman_get_revision: return comment but no return value
> > drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: extra names dev
> > drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: missing name fm_dev
> > drivers/net/ethernet/freescale/fman/fman.c:2527 fman_has_errata_a050385: no comment for fman_has_errata_a050385
>
> just fyi: scripts/kernel-doc already does:
>
> $ ./scripts/kernel-doc drivers/net/ethernet/freescale/fman/fman.c > /dev/null
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function parameter or member 'module' not described in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function parameter or member 'isr_cb' not described in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function parameter or member 'src_arg' not described in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function parameter 'mod' description in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function parameter 'f_isr' description in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function parameter 'h_src_arg' description in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2105: warning: Function parameter or member 'module' not described in 'fman_unregister_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2105: warning: Excess function parameter 'mod' description in 'fman_unregister_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2356: warning: Function parameter or member 'fman' not described in 'fman_get_revision'
> drivers/net/ethernet/freescale/fman/fman.c:2356: warning: Function parameter or member 'rev_info' not described in 'fman_get_revision'
> drivers/net/ethernet/freescale/fman/fman.c:2521: warning: Function parameter or member 'fm_dev' not described in 'fman_bind'
> drivers/net/ethernet/freescale/fman/fman.c:2521: warning: Excess function parameter 'dev' description in 'fman_bind'

OK, that's about the same then.  I manage to make a few suggestions on
some examples, but not that many.

julia
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Laurent Pinchart]

Hi Shuah,

On Fri, Jun 12, 2020 at 10:08:36AM -0600, Shuah Khan wrote:
> On 6/12/20 3:07 AM, Mike Rapoport wrote:
> > On Thu, Jun 11, 2020 at 11:28:09AM -0700, Joe Perches wrote:
> >> On Thu, 2020-06-11 at 12:03 -0600, Shuah Khan wrote:
> >>> People that know the sub-systems well might not
> >>> have time to document and more importantly keeping the documents
> >>> updated.
> >>
> >> So you somehow expect people that do _not_ know the
> >> sub-system well to produce good and useful documentation?
> > 
> > Writing documentation is a way to learn.
> 
> +1
> 
> I find it useful and I also use it as learning tool. When I attempt
> document or explain a concept or an area, I get a deeper understanding
> of that area.

I only partly agree with that. I started the DRM documentation effort
because I had to write a driver, and there was no documentation. Writing
doc was indeed a way to learn, but it's a really high barrier to entry,
more difficult than "just" writing a new driver. You need a very high
level of motivation to do so, I don't think it's a suitable for someone
who has little technical knowledge of the Linux kernel for instance. It
also requires lots of time.

> > Besides, to produce good and usefull documentation it is not necessary
> > to have very deep understanding of all the small details. The writer
> > must capture the large picture and be able to explain it.
> 
> +1.
> 
> I would counter the argument that new developers can't document an area,
> they can and the document can be reviewed by experts.
> 
> A fresh set of eyes bring a new perspective and could help improve the
> code, just like the benefits of having others test your code.

A fresh set of eyes can point to inconsistencies in the code while
writing documentation, but I don't share the view that we can consider
documentation writing as a good learning curve task in general, it
really requires lots of time and a strong incentive in addition to
documentation writing itself.

> I have had some level of success adding documentation tasks in the
> mentoring program. It does require experts spending time reviewing to
> make sure it is correct.

Are those proposed, ongoing or completed tasks ? If some are completed,
could you share information about what has been produced ?

> One thing we could do is creating a list documentation todo and review
> which is part of our development workflow anyway.

-- 
Regards,

Laurent Pinchart
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Matthew Wilcox]

[-- Attachment #1.1: Type: text/plain, Size: 3804 bytes --]

Also W=1 will enable documentation warnings. We can move that to default if
we want people to take documentation seriously. Last time I tried it added
about 700 warnings to the build and I didn't have the spoons to make a
serious dent in fixing those warnings.

Maybe _that_ would be a good beginner task because it doesn't take much in
depth knowledge of any particular subsystem.

On Sat., Jun. 13, 2020, 12:57 Joe Perches, <joe@perches.com> wrote:

> On Sat, 2020-06-13 at 18:42 +0200, Julia Lawall wrote:
> > I tried collecting information about missing or incorrectly formatted
> > function documentation using Coccinelle.  Here is an example of the
> > output:
> >
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr:
> extra names mod, f_isr, h_src_arg
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr:
> missing name isr_cb
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr:
> missing name module
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr:
> missing name src_arg
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr:
> return comment but no return value
> > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr:
> extra names mod
> > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr:
> missing name module
> > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr:
> return comment but no return value
> > drivers/net/ethernet/freescale/fman/fman.c:2355 fman_get_revision:
> return comment but no return value
> > drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: extra names
> dev
> > drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: missing name
> fm_dev
> > drivers/net/ethernet/freescale/fman/fman.c:2527 fman_has_errata_a050385:
> no comment for fman_has_errata_a050385
>
> just fyi: scripts/kernel-doc already does:
>
> $ ./scripts/kernel-doc drivers/net/ethernet/freescale/fman/fman.c >
> /dev/null
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function
> parameter or member 'module' not described in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function
> parameter or member 'isr_cb' not described in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function
> parameter or member 'src_arg' not described in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function
> parameter 'mod' description in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function
> parameter 'f_isr' description in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function
> parameter 'h_src_arg' description in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2105: warning: Function
> parameter or member 'module' not described in 'fman_unregister_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2105: warning: Excess function
> parameter 'mod' description in 'fman_unregister_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2356: warning: Function
> parameter or member 'fman' not described in 'fman_get_revision'
> drivers/net/ethernet/freescale/fman/fman.c:2356: warning: Function
> parameter or member 'rev_info' not described in 'fman_get_revision'
> drivers/net/ethernet/freescale/fman/fman.c:2521: warning: Function
> parameter or member 'fm_dev' not described in 'fman_bind'
> drivers/net/ethernet/freescale/fman/fman.c:2521: warning: Excess function
> parameter 'dev' description in 'fman_bind'
>
>
>
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
>

[-- Attachment #1.2: Type: text/html, Size: 4654 bytes --]

[-- Attachment #2: Type: text/plain, Size: 186 bytes --]

_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mike Rapoport]

On Sun, Jun 14, 2020 at 09:23:41AM -0400, Matthew Wilcox wrote:
> Also W=1 will enable documentation warnings. We can move that to default if we
> want people to take documentation seriously. Last time I tried it added about
> 700 warnings to the build and I didn't have the spoons to make a serious dent
> in fixing those warnings.
> 
> Maybe _that_ would be a good beginner task because it doesn't take much in
> depth knowledge of any particular subsystem.

That's true, but unless we pay more attention to include kerneldoc
changes with patches that change APIs we would train many beginners this
way ;-)

> On Sat., Jun. 13, 2020, 12:57 Joe Perches, <joe@perches.com> wrote:
> 
>     On Sat, 2020-06-13 at 18:42 +0200, Julia Lawall wrote:
>     > I tried collecting information about missing or incorrectly formatted
>     > function documentation using Coccinelle.  Here is an example of the
>     > output:
>     >
>     > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: extra
>     names mod, f_isr, h_src_arg
>     > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr:
>     missing name isr_cb
>     > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr:
>     missing name module
>     > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr:
>     missing name src_arg
>     > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr:
>     return comment but no return value
>     > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr:
>     extra names mod
>     > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr:
>     missing name module
>     > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr:
>     return comment but no return value
>     > drivers/net/ethernet/freescale/fman/fman.c:2355 fman_get_revision: return
>     comment but no return value
>     > drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: extra names
>     dev
>     > drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: missing name
>     fm_dev
>     > drivers/net/ethernet/freescale/fman/fman.c:2527 fman_has_errata_a050385:
>     no comment for fman_has_errata_a050385
> 
>     just fyi: scripts/kernel-doc already does:
> 
>     $ ./scripts/kernel-doc drivers/net/ethernet/freescale/fman/fman.c > /dev/
>     null
>     drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function
>     parameter or member 'module' not described in 'fman_register_intr'
>     drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function
>     parameter or member 'isr_cb' not described in 'fman_register_intr'
>     drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function
>     parameter or member 'src_arg' not described in 'fman_register_intr'
>     drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function
>     parameter 'mod' description in 'fman_register_intr'
>     drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function
>     parameter 'f_isr' description in 'fman_register_intr'
>     drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function
>     parameter 'h_src_arg' description in 'fman_register_intr'
>     drivers/net/ethernet/freescale/fman/fman.c:2105: warning: Function
>     parameter or member 'module' not described in 'fman_unregister_intr'
>     drivers/net/ethernet/freescale/fman/fman.c:2105: warning: Excess function
>     parameter 'mod' description in 'fman_unregister_intr'
>     drivers/net/ethernet/freescale/fman/fman.c:2356: warning: Function
>     parameter or member 'fman' not described in 'fman_get_revision'
>     drivers/net/ethernet/freescale/fman/fman.c:2356: warning: Function
>     parameter or member 'rev_info' not described in 'fman_get_revision'
>     drivers/net/ethernet/freescale/fman/fman.c:2521: warning: Function
>     parameter or member 'fm_dev' not described in 'fman_bind'
>     drivers/net/ethernet/freescale/fman/fman.c:2521: warning: Excess function
>     parameter 'dev' description in 'fman_bind'
> 
> 
> 
>     _______________________________________________
>     Ksummit-discuss mailing list
>     Ksummit-discuss@lists.linuxfoundation.org
>     https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
> 

> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss


-- 
Sincerely yours,
Mike.
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Jani Nikula]

On Sat, 13 Jun 2020, Joe Perches <joe@perches.com> wrote:
> just fyi: scripts/kernel-doc already does:
>
> $ ./scripts/kernel-doc drivers/net/ethernet/freescale/fman/fman.c > /dev/null

FYI, kernel-doc has the -none option to not output any documentation,
only the warnings. Perhaps not the most intuitive thing you ever saw,
but might be slightly more efficient than redirecting stdout to
/dev/null.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mike Rapoport]

Hi Joe,

On Sat, Jun 13, 2020 at 09:51:44AM -0700, Joe Perches wrote:
> On Sat, 2020-06-13 at 18:42 +0200, Julia Lawall wrote:
> > I tried collecting information about missing or incorrectly formatted
> > function documentation using Coccinelle.  Here is an example of the
> > output:
> > 
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: extra names mod, f_isr, h_src_arg
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name isr_cb
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name module
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: missing name src_arg
> > drivers/net/ethernet/freescale/fman/fman.c:2077 fman_register_intr: return comment but no return value
> > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: extra names mod
> > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: missing name module
> > drivers/net/ethernet/freescale/fman/fman.c:2103 fman_unregister_intr: return comment but no return value
> > drivers/net/ethernet/freescale/fman/fman.c:2355 fman_get_revision: return comment but no return value
> > drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: extra names dev
> > drivers/net/ethernet/freescale/fman/fman.c:2520 fman_bind: missing name fm_dev
> > drivers/net/ethernet/freescale/fman/fman.c:2527 fman_has_errata_a050385: no comment for fman_has_errata_a050385
> 
> just fyi: scripts/kernel-doc already does:
> 
> $ ./scripts/kernel-doc drivers/net/ethernet/freescale/fman/fman.c > /dev/null
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function parameter or member 'module' not described in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function parameter or member 'isr_cb' not described in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Function parameter or member 'src_arg' not described in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function parameter 'mod' description in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function parameter 'f_isr' description in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2080: warning: Excess function parameter 'h_src_arg' description in 'fman_register_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2105: warning: Function parameter or member 'module' not described in 'fman_unregister_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2105: warning: Excess function parameter 'mod' description in 'fman_unregister_intr'
> drivers/net/ethernet/freescale/fman/fman.c:2356: warning: Function parameter or member 'fman' not described in 'fman_get_revision'
> drivers/net/ethernet/freescale/fman/fman.c:2356: warning: Function parameter or member 'rev_info' not described in 'fman_get_revision'
> drivers/net/ethernet/freescale/fman/fman.c:2521: warning: Function parameter or member 'fm_dev' not described in 'fman_bind'
> drivers/net/ethernet/freescale/fman/fman.c:2521: warning: Excess function parameter 'dev' description in 'fman_bind'
 
Can this be somehow reported by checkpatch.pl?

What I mean is that if a patch breaks the sync between the code and the
kerneldoc can it be detected by checkpatch.pl?

-- 
Sincerely yours,
Mike.
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mike Rapoport]

On Tue, Jun 09, 2020 at 02:53:53PM -0600, Jonathan Corbet wrote:
> The long process of converting the kernel's documentation into RST is
> finally coming to an end...what has that bought us?  We have gone from a
> chaotic pile of incomplete, crufty, and un-integrated docs to a slightly
> better organized pile of incomplete, crufty, slightly better integrated
> docs.  Plus we have the infrastructure to make something better from here.
> 
> What are the next steps for kernel documentation?  What would we really
> like our docs to look like, and how might we find the resources to get
> them to that point?  What sorts of improvements to the build
> infrastructure would be useful?  I'll come with some ideas (some of which
> you've certainly heard before) but will be more interested in listening.

Antoher question I've been asking myself is what to do with good but old
guides. For instance, Documentation/virt/uml/user_mode_linux.rst
describes how to debug the kernel with gdb, but most of the code
references there are stale by now.

Another example is "Unreliable Guide To Hacking The Linux Kernel"
(Documentation/kernel-hacking/hacking.rst). It's really nicely written
by it is way too out of date.

> jon

-- 
Sincerely yours,
Mike.
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Joe Perches]

On Thu, 2020-06-18 at 12:04 +0300, Mike Rapoport wrote:
> if a patch breaks the sync between the code and the
> kerneldoc can it be detected by checkpatch.pl?

No, it can not.  Not directly.

Please remember that checkpatch is a trivial set
of regexes and that checkpatch looks only at
patch diff blocks line-by-line.

A patch block that changes a function argument
may not also contain the kernel-doc.

As there already is an existing tool, that tool
should be used when possible.

_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Joe Perches]

On Thu, 2020-07-09 at 12:21 +0200, Mauro Carvalho Chehab wrote:
> Em Thu, 18 Jun 2020 07:40:22 -0700
> Joe Perches <joe@perches.com> escreveu:
> 
> > On Thu, 2020-06-18 at 12:04 +0300, Mike Rapoport wrote:
> > > if a patch breaks the sync between the code and the
> > > kerneldoc can it be detected by checkpatch.pl?  
> > 
> > No, it can not.  Not directly.
> 
> Well, perhaps checkpatch.pl could run:
> 
> 	./scripts/kernel-doc -none
> 
> if it finds "/**" on a patch, showing the warnings produced by it, if any.

checkpatch runs on patches.
Use the tools that are designed for this instead.

> > As there already is an existing tool, that tool
> > should be used when possible.


_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mike Rapoport]

On Thu, Jul 09, 2020 at 04:42:58AM -0700, Joe Perches wrote:
> On Thu, 2020-07-09 at 12:21 +0200, Mauro Carvalho Chehab wrote:
> > Em Thu, 18 Jun 2020 07:40:22 -0700
> > Joe Perches <joe@perches.com> escreveu:
> > 
> > > On Thu, 2020-06-18 at 12:04 +0300, Mike Rapoport wrote:
> > > > if a patch breaks the sync between the code and the
> > > > kerneldoc can it be detected by checkpatch.pl?  
> > > 
> > > No, it can not.  Not directly.
> > 
> > Well, perhaps checkpatch.pl could run:
> > 
> > 	./scripts/kernel-doc -none
> > 
> > if it finds "/**" on a patch, showing the warnings produced by it, if any.
> 
> checkpatch runs on patches.
> Use the tools that are designed for this instead.

The problem is that people usually do run checkpatch and do not run
kernel-doc. That's where the idea to somehow put documentation checks
into checkpatch came from.

> > > As there already is an existing tool, that tool
> > > should be used when possible.
> 
> 

-- 
Sincerely yours,
Mike.
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Joe Perches]

On Thu, 2020-07-09 at 15:11 +0300, Mike Rapoport wrote:
> On Thu, Jul 09, 2020 at 04:42:58AM -0700, Joe Perches wrote:
> > On Thu, 2020-07-09 at 12:21 +0200, Mauro Carvalho Chehab wrote:
> > > Em Thu, 18 Jun 2020 07:40:22 -0700
> > > Joe Perches <joe@perches.com> escreveu:
> > > 
> > > > On Thu, 2020-06-18 at 12:04 +0300, Mike Rapoport wrote:
> > > > > if a patch breaks the sync between the code and the
> > > > > kerneldoc can it be detected by checkpatch.pl?  
> > > > 
> > > > No, it can not.  Not directly.
> > > 
> > > Well, perhaps checkpatch.pl could run:
> > > 
> > > 	./scripts/kernel-doc -none
> > > 
> > > if it finds "/**" on a patch, showing the warnings produced by it, if any.
> > 
> > checkpatch runs on patches.
> > Use the tools that are designed for this instead.
> 
> The problem is that people usually do run checkpatch and do not run
> kernel-doc. That's where the idea to somehow put documentation checks
> into checkpatch came from.

I much prefer this patch:
https://lore.kernel.org/patchwork/patch/1260408/


_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mike Rapoport]

On Thu, Jul 09, 2020 at 09:59:46AM -0700, Joe Perches wrote:
> On Thu, 2020-07-09 at 15:11 +0300, Mike Rapoport wrote:
> > On Thu, Jul 09, 2020 at 04:42:58AM -0700, Joe Perches wrote:
> > > On Thu, 2020-07-09 at 12:21 +0200, Mauro Carvalho Chehab wrote:
> > > > Em Thu, 18 Jun 2020 07:40:22 -0700
> > > > Joe Perches <joe@perches.com> escreveu:
> > > > 
> > > > > On Thu, 2020-06-18 at 12:04 +0300, Mike Rapoport wrote:
> > > > > > if a patch breaks the sync between the code and the
> > > > > > kerneldoc can it be detected by checkpatch.pl?  
> > > > > 
> > > > > No, it can not.  Not directly.
> > > > 
> > > > Well, perhaps checkpatch.pl could run:
> > > > 
> > > > 	./scripts/kernel-doc -none
> > > > 
> > > > if it finds "/**" on a patch, showing the warnings produced by it, if any.
> > > 
> > > checkpatch runs on patches.
> > > Use the tools that are designed for this instead.
> > 
> > The problem is that people usually do run checkpatch and do not run
> > kernel-doc. That's where the idea to somehow put documentation checks
> > into checkpatch came from.
> 
> I much prefer this patch:
> https://lore.kernel.org/patchwork/patch/1260408/

I don't see how it will help reducing amount of the kernel-doc warnings
and how it'll make people to check if their patch introduces new ones...


-- 
Sincerely yours,
Mike.
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Andrew Lunn]

On Thu, Jul 09, 2020 at 08:29:14PM +0300, Mike Rapoport wrote:
> On Thu, Jul 09, 2020 at 09:59:46AM -0700, Joe Perches wrote:
> > On Thu, 2020-07-09 at 15:11 +0300, Mike Rapoport wrote:
> > > On Thu, Jul 09, 2020 at 04:42:58AM -0700, Joe Perches wrote:
> > > > On Thu, 2020-07-09 at 12:21 +0200, Mauro Carvalho Chehab wrote:
> > > > > Em Thu, 18 Jun 2020 07:40:22 -0700
> > > > > Joe Perches <joe@perches.com> escreveu:
> > > > > 
> > > > > > On Thu, 2020-06-18 at 12:04 +0300, Mike Rapoport wrote:
> > > > > > > if a patch breaks the sync between the code and the
> > > > > > > kerneldoc can it be detected by checkpatch.pl?  
> > > > > > 
> > > > > > No, it can not.  Not directly.
> > > > > 
> > > > > Well, perhaps checkpatch.pl could run:
> > > > > 
> > > > > 	./scripts/kernel-doc -none
> > > > > 
> > > > > if it finds "/**" on a patch, showing the warnings produced by it, if any.
> > > > 
> > > > checkpatch runs on patches.
> > > > Use the tools that are designed for this instead.
> > > 
> > > The problem is that people usually do run checkpatch and do not run
> > > kernel-doc. That's where the idea to somehow put documentation checks
> > > into checkpatch came from.
> > 
> > I much prefer this patch:
> > https://lore.kernel.org/patchwork/patch/1260408/
> 
> I don't see how it will help reducing amount of the kernel-doc warnings
> and how it'll make people to check if their patch introduces new ones...

It seems like 0-day and the like would be a better solution, since
they know how to apply the patch, run tools on the sources as a whole,
and look for new warnings mixed in with older warnings.

    Andrew
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Joe Perches]

On Thu, 2020-07-09 at 17:00 -0400, Valdis Klētnieks wrote:
> On Thu, 09 Jul 2020 19:57:45 +0200, Andrew Lunn said:
> 
> > It seems like 0-day and the like would be a better solution, since
> > they know how to apply the patch, run tools on the sources as a whole,
> > and look for new warnings mixed in with older warnings.
[]
> (For the record, I'm OK with Masahiro Yamada's suggestion to add
> a W=d option to the current W=123 - that's probably a better idea than
> my original patch...)

I'm fine with that too.


_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mauro Carvalho Chehab]

Em Wed, 12 Jun 2019 08:54:05 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> What could be more fun than talking about kernel documentation?  Things we
> could get into:
> 
>  - The state of the RST transition

Assuming that all patches on my development branch gets merged, we should
have:

	- about 300 files missing conversion;
	- about 40 files under Documentation/*.txt that needs to be
	  moved to a subdir and be renamed to *.rst.

>  - Overall organization of Documentation/ and moving docs when the need
>    arises.

Looking at Documentation/index.rst, I guess we're aiming to have something 
like:

	Documentation/
	├── ABI
	├── admin-guide
	├── kbuild
	├── arch
	│   ├── arm
	│   ├── arm64
	│   ...
	│   ├── x86
	│   │   ├── i386
	│   │   └── x86_64
	│   └── xtensa
	├── core-api
	├── devicetree
	├── dev-tools
	├── doc-guide
	├── driver-api
	├── features
	├── fault-injection
	├── filesystems
	├── firmware-guide
	├── kernel-hacking
	├── livepatch
	├── maintainer
	├── process
	├── trace
	└── translations

Btw, right now, ext4 fs docs is on two separate parts of 
Documentation/index.rst:

   filesystems/index
   filesystems/ext4/index

We should probably get rid of filesystems/ext4/index entry (and the 
corresponding PDF entry at conf.py).

-

IMO, the main work to be done in order to achieve that is related to
Driver's subsystem documentation.

What I've been doing so far - at least for most (if not all) driver docs 
that carry more than one documentation type at the same subdir (kABI, 
uABI and/or admin-guide) is to keep the directory as-is, adding them under
this section at Documentation/index.rst:

	Kernel API documentation
	------------------------

There are a couple of reasons why I opted for this strategy when
I did such conversions:

1) There are *lots* of docs that contain all 3 types of information
   on it on a single file.

2) On media, we use SPHINXDIRS to produce the media book from our
   devel tree:

	https://linuxtv.org/downloads/v4l-dvb-apis-new/index.html

   When documents are built with either PDF or SPHINXDIRS, each subdir
   will be on a different book and all inter-book cross-references
   will break.
   
   For this to be fixed, intersphinx extension would be required,
   but this would probably require a per-subsystem mapping 
   (for example, saying that the site used to resolve media
   broken cross references is linuxtv.org). 

   Maintaining it can be painful, as we would have a big table
   at conf.py with subsystem-specific stuff.

3) So far, I was unable to split even the media docs. Shame on
   me! The reason is that this is not an easy task.

One interesting example is the open() documentation at the
media media uAPI book:

	Documentation/media/uapi/v4l/open.rst

This file actually contains a lot of sysadmin relevant data (so, it
is a good candidate for the admin-guide).

Yet, it was written focused on what a media uAPI developer needs
to know. So, it also mentions Kernel userspace API syscalls:
open(), read(), close() - with has cross-references to other parts
of the uAPI book.

Splitting this file on two separate books won't be that easy.

Ideally, we should split what's there at media/uapi into admin-guide
and userspace-api, but this would mean *a lot* of effort. Not sure
if it is worth the time.

Also, while a sysadmin might want to know what a /dev/video0 device
means, the intended audience is really uAPI developers, as an user
will just click on its GUI to call a media application.

>  - The ultimate vision for kernel docs (for now).  RST conversion and
>    imposing some organization are important, but they will not,
>    themselves, give us a coherent set of documentation.  What can we do to
>    have documentation that is useful, current, and maintainable, rather
>    than the dusty attic we have now?

The more I think the more I'm convinced that the best way to proceed would
be to use some Kernel books as an example and organize the main index.rst
files on a way that it will cover what a newbie Kernel developer would need
to know.

For example, looking at LDD3 organization (https://lwn.net/Kernel/LDD3/),
we could use its index as an starting point for what a driver-api book
should contain:

	Chapter 1: An Introduction to Device Drivers
	Chapter 3: Char Drivers
	Chapter 6: Advanced Char Driver Operations
	Chapter 9: Communicating with Hardware
	Chapter 12: PCI Drivers
	Chapter 13: USB Drivers
	Chapter 14: The Linux Device Model
	Chapter 16: Block Drivers
	Chapter 17: Network Drivers
	Chapter 18: TTY Drivers

Just looking on these, our driver-api book seem to be missing the
texts that would glue its contents, e. g. an introduction to device
drivers, to char/block devices and about how to communicate with the
hardware.

It can also help to identify the contents that a driver developer
would need from a core-api and a Kernel development bookset:

	Chapter 2: Building and Running Modules
	Chapter 4: Debugging Techniques
	Chapter 5: Concurrency and Race Conditions
	Chapter 7: Time, Delays, and Deferred Work
	Chapter 8: Allocating Memory
	Chapter 10: Interrupt Handling
	Chapter 11: Data Types in the Kernel

I would do the same with other Linux and Linux Kernel related books.

-

Btw, if the authors of some existing old books release their stuff under
GPLv2 and allow us to import their contents, we could try to import some
parts of it that aren't too obsolete. The new automarkup extension
can help a lot to identify outdated documents, as it won't be able
to solve the func() calls.

Thanks,
Mauro

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Jonathan Corbet]

On Thu, 20 Jun 2019 11:36:38 -0700
Kees Cook <keescook@chromium.org> wrote:

> Just to put it into this thread... I've seen a recurring request for
> markup improvement to have Sphinx be smart enough to linkify:
> 
> 	function()
> 
> instead of needing to do:
> 
> 	:c:func:`iounmap()`

I already have a patch to do that:

	https://lwn.net/ml/linux-doc/20190425200125.12302-1-corbet@lwn.net/

It needs another bit of attention before it's ready, and I've not yet
managed to do that.  Soon, I promise :)

Thanks,

jon

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Kees Cook]

On Wed, Jun 12, 2019 at 08:54:05AM -0600, Jonathan Corbet wrote:
>  - Things we'd like to improve in the documentation toolchain.

Just to put it into this thread... I've seen a recurring request for
markup improvement to have Sphinx be smart enough to linkify:

	function()

instead of needing to do:

	:c:func:`iounmap()`

And similarly for: struct namegoeshere

-- 
Kees Cook

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mauro Carvalho Chehab]

Em Mon, 17 Jun 2019 07:12:06 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu:

Answering myself here...

> > 	Documentation/features  
> 
> This is actually a special case. I don't think that a plain txt->rst
> conversion is the right thing to be done here.
> 
> IMO, it requires some further discussions.

> The script should be able to produce valid ReST output on all the
> above formats, e. g.:
> 
> 1) output per feature;
> 2) output per architecture;
> 3) output as a matrix feature x architecture.

I'm actually thinking on something like:

1) A page that could be part of a feature documentation
   (On this example, it documents scheduler ARCH_HAS_MEMBARRIER_SYNC_CORE)

	https://www.infradead.org/~mchehab/rst_features/feature_membarrier-sync-core.html

2) A page that could be part of the per-archtecture books
   (here, arm64 one):

	https://www.infradead.org/~mchehab/rst_features/feature_arm64.html

3) A page with a matrix with all architectures and all features:

	https://www.infradead.org/~mchehab/rst_features/feature_matrix.html

I just wrote a patch with such script.

I should be submitting it soon to the ML for review, after some final
cleanups.

Thanks,
Mauro

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mauro Carvalho Chehab]

Em Fri, 14 Jun 2019 21:05:36 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu:

> Em Fri, 14 Jun 2019 15:40:26 -0600
> Shuah Khan <skhan@linuxfoundation.org> escreveu:
> 
> > On 6/13/19 8:25 AM, Mauro Carvalho Chehab wrote:  
> > > Em Wed, 12 Jun 2019 12:22:55 -0600
> > > Shuah Khan <skhan@linuxfoundation.org> escreveu:
> > >     
> 
> > > So, my advice here is to really invert things:
> > > 
> > > - do the conversion;    
> > 
> > Even this can be made into tasks. If you would like to experiment
> > with and see how it works, send me a list of documents that you
> > would like to see converted first.  
> 
> After the patchsets I'm working it:
> 
> 	https://git.linuxtv.org/mchehab/experimental.git/log/?h=convert_rst_renames_v5
> 
> There will be very few directories that doesn't any any .rst file:
> 
> $ for i in $(find Documentation/ -name '*.txt' | sed -E "s,(Documentation/[^\/+]+).*,\1,"|uniq|grep -v output |grep -v binding|grep -v devicetree); do if [ "$(find $i -name '*.rst')" == "" ]; then if [ -d $i ]; then echo $i; fi; fi; done
> 
> Documentation/platform
> Documentation/scsi
> Documentation/RCU
> Documentation/cpu-freq
> Documentation/ABI
> Documentation/sphinx
> Documentation/features
> Documentation/acpi
> 
> Among those:
> 
> - There's nothing to be done at the sphinx directory.
> - The cpu-freq maintainer won't want changes there, as the stuff inside
>   that specific dir is obsolete.
> - If I'm not mistaken, there's already a pending patchset for the
>   acpi directory too.
> - Platform has just a single file, easily convertible to ReST.
>   I suspect that it should be moved to the laptops dir too.
>   I guess I wrote a patch for it, but it seems it got lost on
>   some rebase. Anyway, I can take care of this one.	
> - We are handling the ABI directory on a different way. 
> 
> So, I guess that what it was left behind is:
> 
> 	Documentation/scsi
> 	Documentation/RCU


> 	Documentation/features

This is actually a special case. I don't think that a plain txt->rst
conversion is the right thing to be done here.

IMO, it requires some further discussions.

The files there have a standard format, with is (poorly) documented
at Documentation/features/arch-support.txt.

Getting a random file there, the format has a description part and a
table (Documentation/features/debug/kprobes/arch-support.txt):

	# Feature name:          kprobes
	#         Kconfig:       HAVE_KPROBES
	#         description:   arch supports live patched kernel probe
	#
	    -----------------------
	    |         arch |status|
	    -----------------------
	    |       alpha: | TODO |
	    |         arc: |  ok  |
	    |         arm: |  ok  |
	...
	    |      xtensa: | TODO |
	    -----------------------

E. g. each file contains the name of a feature, its Kconfig
symbol, a description and a table with shows what architectures
support and what architectures don't.

I actually started converting it, but the results don't look nice.

The thing is: it is a lot easier for a "feature" maintainer to have
a single file with a per-architecture status, as he's free to update
the file without needing to be concerned about merge conflicts.

So, the current way makes a perfect sense from a developer's PoV.

-

However, for a Kernel user's perspective, he may be interested on
looking at this as a completely different way.

1) he may want to buy a hardware whose support a certain feature
   subset;

2) He has already some hardware, any may want to identify if a
   certain feature is there.


For (1), the best would be a table like:

=======  =====  ===== =====   =====       ======
Feature  alpha  arc   arm     arm64 ....  xtensa
=======  =====  ===== =====   =====       ======
KASAN    TODO   TODO  TODO    ok          ok
kdb      TODO   ok    ok      ok          TODO
...
=======  =====  ===== =====   =====       ======

(eventually with Kconfig and description, but we need to double check
if it won't bee to big at the pdf output).


And for (2), one file per architecture, with something like:


Feature status at x86 Architecture
==================================

=======   =================  ======  ==============================================
Feature   Kconfig            Status  Description
=======   =================  ======  ==============================================
KASAN	  HAVE_ARCH_KASAN    ok      arch supports the KASAN runtime memory checker
kgdb      HAVE_ARCH_KGDB     ok      arch supports the kGDB kernel debugger
...
=======   =================  ======  ==============================================


So, IMHO, the best thing to do with the feature files is an approach similar
to the one we're doing with ABI, e. g.:

1) improve the contents of "arch-support.txt" file for it to precisely
describe the format of the file, including the header. While not required,
eventually we could change the headers to something less prune to human
errors, e. g. to something like:

	name:	       kprobes
	Kconfig:       HAVE_KPROBES
	description:   arch supports live patched kernel probe

2) have a script similar to get_abi.pl that would parse, validate
and produce a per-architecture output file;

3) add a per-arch features file that will be automatically
generated by the script.

The script should be able to produce valid ReST output on all the
above formats, e. g.:

1) output per feature;

2) output per architecture;

3) output as a matrix feature x architecture.

This way, we could add the feature lists on multiple books:

	- admin-doc: feature x architecture view
	- arch-specific view: feature status for such specific arch
	- kAPI book: per feature output

Comments?

Thanks,
Mauro

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mauro Carvalho Chehab]

Em Fri, 14 Jun 2019 15:40:26 -0600
Shuah Khan <skhan@linuxfoundation.org> escreveu:

> On 6/13/19 8:25 AM, Mauro Carvalho Chehab wrote:
> > Em Wed, 12 Jun 2019 12:22:55 -0600
> > Shuah Khan <skhan@linuxfoundation.org> escreveu:
> >   

> > So, my advice here is to really invert things:
> > 
> > - do the conversion;  
> 
> Even this can be made into tasks. If you would like to experiment
> with and see how it works, send me a list of documents that you
> would like to see converted first.

After the patchsets I'm working it:

	https://git.linuxtv.org/mchehab/experimental.git/log/?h=convert_rst_renames_v5

There will be very few directories that doesn't any any .rst file:

$ for i in $(find Documentation/ -name '*.txt' | sed -E "s,(Documentation/[^\/+]+).*,\1,"|uniq|grep -v output |grep -v binding|grep -v devicetree); do if [ "$(find $i -name '*.rst')" == "" ]; then if [ -d $i ]; then echo $i; fi; fi; done

Documentation/platform
Documentation/scsi
Documentation/RCU
Documentation/cpu-freq
Documentation/ABI
Documentation/sphinx
Documentation/features
Documentation/acpi

Among those:

- There's nothing to be done at the sphinx directory.
- The cpu-freq maintainer won't want changes there, as the stuff inside
  that specific dir is obsolete.
- If I'm not mistaken, there's already a pending patchset for the
  acpi directory too.
- Platform has just a single file, easily convertible to ReST.
  I suspect that it should be moved to the laptops dir too.
  I guess I wrote a patch for it, but it seems it got lost on
  some rebase. Anyway, I can take care of this one.	
- We are handling the ABI directory on a different way. 

So, I guess that what it was left behind is:

	Documentation/scsi
	Documentation/RCU
	Documentation/features

You should notice, however, that there are some other directories
that have a mix of rst and txt files, with also have some files 
that could be needing conversion:

$ for i in $(find Documentation/ -name '*.txt' | sed -E "s,(Documentation/[^\/+]+).*,\1,"|uniq|grep -v output |grep -v binding|grep -v devicetree); do if [ -d $i ]; then echo $i; fi; done
Documentation/device-mapper
Documentation/sh
Documentation/trace
Documentation/misc-devices
Documentation/arm64
Documentation/platform
Documentation/scsi
Documentation/RCU
Documentation/virtual
Documentation/cpu-freq
Documentation/admin-guide
Documentation/block
Documentation/ABI
Documentation/translations
Documentation/PCI
Documentation/filesystems
Documentation/networking
Documentation/sphinx
Documentation/features
Documentation/netlabel
Documentation/acpi
Documentation/crypto
Documentation/sparc

I suspect that those dirs with contain botn .rst and .txt files are
in the process of being updated and converted by the subsystem
maintainers, but double check is needed.

Plus other places where there are files whose filename doesn't have any
extension at all and could be a text file to be converted:

$ for i in $(find Documentation/  -type f ! -name "*.*" | sed -E "s,(Documentation/[^\/+]+).*,\1,"|uniq|grep -v output |grep -v binding|grep -v devicetree); do if [ -d $i ]; then echo $i; fi; done
Documentation/misc-devices
Documentation/target
Documentation/firmware_class
Documentation/isdn
Documentation/media
Documentation/parisc
Documentation/spi
Documentation/scsi
Documentation/hwmon
Documentation/EDID
Documentation/nios2
Documentation/virtual
Documentation/i2c
Documentation/usb
Documentation/ABI
Documentation/translations
Documentation/openrisc
Documentation/w1
Documentation/filesystems
Documentation/networking
Documentation/auxdisplay

There are a few exceptions that should be kept as plain text
files. I remember of two such cases:

- a configuration file whose extension is .txt;
- a dump of chapter extracted from an old arch-specific manual
  with seems to be provided by its manufacturer.

While the last one could eventually be converted, it sounded
too much work for too little gain - and a format change may
eventually require some negotiation with the copyright holder.

Thanks,
Mauro

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Shuah Khan]

On 6/13/19 8:25 AM, Mauro Carvalho Chehab wrote:
> Em Wed, 12 Jun 2019 12:22:55 -0600
> Shuah Khan <skhan@linuxfoundation.org> escreveu:
> 
>> On 6/12/19 8:54 AM, Jonathan Corbet wrote:
>>> What could be more fun than talking about kernel documentation?  Things we
>>> could get into:
>>>
>>>    - The state of the RST transition, what remains to be done, whether it's
>>>      all just useless churn that makes the documentation worse, etc.
>>>
>>>    - Things we'd like to improve in the documentation toolchain.
>>>
>>>    - Overall organization of Documentation/ and moving docs when the need
>>>      arises.  It seems I end up fighting about this more than just about
>>>      anything else, but I think it's important to organize our docs for the
>>>      convenience of the people using them.
>>>
>>>    - The ultimate vision for kernel docs (for now).  RST conversion and
>>>      imposing some organization are important, but they will not,
>>>      themselves, give us a coherent set of documentation.  What can we do to
>>>      have documentation that is useful, current, and maintainable, rather
>>>      than the dusty attic we have now?
>>>    
>>
>> The first step is identifying the ones that are out of date and/or
>> incorrect. I think this is probably the most time consuming part
>> and the second is start updating.
> 
> I would do just the opposite: convert the documentation, then
> update it.
> 

Yes. That is fine. I am referring to the outdated part. Converting
and updating them would work just fine.

> The reason is that, when someone sends a patch to the documentation,
> people tend to look more closely on it, and we start receiving
> updates for it.
> 


> Ok, this doesn't happen every time, but it is still worth.
> 
> There's another thing to consider here: there are lots of docs
> written for "stable" subsystems, in the sense that the documented
> code doesn't change for a lot of time (except for trivial patches
> not directly related to it.
> 
> While doing large chunks of code conversion, I noticed that there
> are lots of still valid documents that fit on this.
> 
> There's a big issue with those "stable" subsystems: they tend
> to not have any active maintainer anymore. So, there are not
> much people that are willing to actively review patches to it.
> 
> So, my advice here is to really invert things:
> 
> - do the conversion;

Even this can be made into tasks. If you would like to experiment
with and see how it works, send me a list of documents that you
would like to see converted first.

> - find a good place to put the converted the book;
> - add kernel-doc markups to be sure that the symbols will always
>    match upstream;
> - review the remaining content of the docs.
> 
>> I am currently putting together a task list for mentorship program
>> with a goal to create tasks that would more meaningful and helpful
>> instead of whitespace patches.
>>
>> I will gladly add documentation tasks to the list to help improve
>> the documentation.
> 
> I can surely mentor people interested on doing such tasks.
> 

That will be great.

thanks,
-- Shuah

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mauro Carvalho Chehab]

Em Thu, 13 Jun 2019 20:48:08 +0200
Greg KH <greg@kroah.com> escreveu:

> On Thu, Jun 13, 2019 at 11:57:12AM -0300, Mauro Carvalho Chehab wrote:
> > >  - Things we'd like to improve in the documentation toolchain.  
> > 
> > I remember once I submitted a patch with a script capable of
> > parsing the files at ABI and produce a parsed content, with the
> > goal of validating the ABI files against its syntax and to
> > generate an ABI book.
> > 
> > IMO, it makes sense to have one book containing the Linux ABI, but, 
> > as the patch didn't have much attention, and I got sidetracked with
> > something else, I ended giving up on trying to push it. 
> > 
> > Perhaps by KS it would be time to rescue it from some old git tree
> > and see if it would be worth the time to work on it.  
> 
> I liked that patcheset.  If you can dig it up, I can work on reviving it
> if you don't want to.

The patches are here:

	https://git.linuxtv.org/mchehab/experimental-old.git/commit/?h=abi_docs_v3&id=da4b94205aced79df7389ca6f2ec4328fb8604a7

I'll rebase them on the top of linux-next and re-submit.

Thanks,
Mauro

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Greg KH]

On Thu, Jun 13, 2019 at 11:57:12AM -0300, Mauro Carvalho Chehab wrote:
> >  - Things we'd like to improve in the documentation toolchain.
> 
> I remember once I submitted a patch with a script capable of
> parsing the files at ABI and produce a parsed content, with the
> goal of validating the ABI files against its syntax and to
> generate an ABI book.
> 
> IMO, it makes sense to have one book containing the Linux ABI, but, 
> as the patch didn't have much attention, and I got sidetracked with
> something else, I ended giving up on trying to push it. 
> 
> Perhaps by KS it would be time to rescue it from some old git tree
> and see if it would be worth the time to work on it.

I liked that patcheset.  If you can dig it up, I can work on reviving it
if you don't want to.

thanks,

greg k-h

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mauro Carvalho Chehab]

Em Wed, 12 Jun 2019 08:54:05 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> What could be more fun than talking about kernel documentation?

That sounds a topic that we should have some discussion. IMO, the best
would be to put people on some round tables and exchange some ideas
about how we can improve the process. I would try to do it as a
half-day topic, if the agenda would allow, as a standard 50' min
slot doesn't seem to be enough to address the needed things there.

> Things we
> could get into:
> 
>  - The state of the RST transition, what remains to be done, whether it's
>    all just useless churn that makes the documentation worse, etc.

I have a large changeset (with about one patch per Documentation/
dir) with manually written patches with addresses a large amount of
the remaining stuff. I'm hoping to have most of it applied before
KS :-)

There will still be 4 or 5 directories with lots of documentation
files on it, plus ABI and DT. 

>  - Things we'd like to improve in the documentation toolchain.

I remember once I submitted a patch with a script capable of
parsing the files at ABI and produce a parsed content, with the
goal of validating the ABI files against its syntax and to
generate an ABI book.

IMO, it makes sense to have one book containing the Linux ABI, but, 
as the patch didn't have much attention, and I got sidetracked with
something else, I ended giving up on trying to push it. 

Perhaps by KS it would be time to rescue it from some old git tree
and see if it would be worth the time to work on it.

> 
>  - Overall organization of Documentation/ and moving docs when the need
>    arises.  It seems I end up fighting about this more than just about
>    anything else, but I think it's important to organize our docs for the
>    convenience of the people using them.

Fully agreed. It is not always clear where to place some converted
docs.

Also, in the case of driver-specific information, it is not unusual to
have both kernel and user's information at the same file. Ideally, the
best would be to split, but I'm not sure if it would worth the time
for doing such changes.

>  - The ultimate vision for kernel docs (for now).  RST conversion and
>    imposing some organization are important, but they will not,
>    themselves, give us a coherent set of documentation.  What can we do to
>    have documentation that is useful, current, and maintainable, rather
>    than the dusty attic we have now?

Yeah, RST conversion and enforcing an organization of them is important,
as it will enforce a single "coding" style for the documents, but this
is only a small part of the issue.

IMO, we should work to attract people focused on maintaining docs.

I've been doing some userspace maintainership on a few media-related
projects. Among them, I'm maintaining a KDE media player (Kaffeine).

The KDE project has a group of people that it is focused only on
writing documentation, and one group of people per translation language.

IMO, if we want to have a good set of documents, we should have a
group of people working mainly with a "documentation hat", reviewing,
improving and rewriting large chunks of documents.

If we ever have such kind of people working around the Kernel,
I guess the first task would be to imagine what kind of things a new 
Kernel developer needs to know and try to structure a coherent
documentation from that. Such group could use the existing Kernel
printed books as a starting point (perhaps using existing books
documentation that their authors could release them under GPLv2),
and modifying the texts to use what we have nowadays inside the
Kernel's tree.

I guess the main problem is that, for this to work, someone
(possibly a non-profit org) would likely need to sponsor such work,
as this would probably require more time than we could do on our
spare time.

Thanks,
Mauro

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mauro Carvalho Chehab]

Em Wed, 12 Jun 2019 12:22:55 -0600
Shuah Khan <skhan@linuxfoundation.org> escreveu:

> On 6/12/19 8:54 AM, Jonathan Corbet wrote:
> > What could be more fun than talking about kernel documentation?  Things we
> > could get into:
> > 
> >   - The state of the RST transition, what remains to be done, whether it's
> >     all just useless churn that makes the documentation worse, etc.
> > 
> >   - Things we'd like to improve in the documentation toolchain.
> > 
> >   - Overall organization of Documentation/ and moving docs when the need
> >     arises.  It seems I end up fighting about this more than just about
> >     anything else, but I think it's important to organize our docs for the
> >     convenience of the people using them.
> > 
> >   - The ultimate vision for kernel docs (for now).  RST conversion and
> >     imposing some organization are important, but they will not,
> >     themselves, give us a coherent set of documentation.  What can we do to
> >     have documentation that is useful, current, and maintainable, rather
> >     than the dusty attic we have now?
> >   
> 
> The first step is identifying the ones that are out of date and/or
> incorrect. I think this is probably the most time consuming part
> and the second is start updating.

I would do just the opposite: convert the documentation, then
update it.

The reason is that, when someone sends a patch to the documentation,
people tend to look more closely on it, and we start receiving
updates for it.

Ok, this doesn't happen every time, but it is still worth.

There's another thing to consider here: there are lots of docs
written for "stable" subsystems, in the sense that the documented 
code doesn't change for a lot of time (except for trivial patches
not directly related to it.

While doing large chunks of code conversion, I noticed that there
are lots of still valid documents that fit on this.

There's a big issue with those "stable" subsystems: they tend
to not have any active maintainer anymore. So, there are not
much people that are willing to actively review patches to it.

So, my advice here is to really invert things:

- do the conversion;
- find a good place to put the converted the book;
- add kernel-doc markups to be sure that the symbols will always
  match upstream;
- review the remaining content of the docs.

> I am currently putting together a task list for mentorship program
> with a goal to create tasks that would more meaningful and helpful
> instead of whitespace patches.
> 
> I will gladly add documentation tasks to the list to help improve
> the documentation.

I can surely mentor people interested on doing such tasks.

> 
> thanks,
> -- Shuah
> 
> 
> 
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss



Thanks,
Mauro

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Mauro Carvalho Chehab]

Em Wed, 12 Jun 2019 15:33:36 -0500
Kate Stewart <kstewart@linuxfoundation.org> escreveu:

> On Wed, Jun 12, 2019 at 9:56 AM Jonathan Corbet <corbet@lwn.net> wrote:
> >
> > What could be more fun than talking about kernel documentation?  Things we
> > could get into:
> >
> >  - The state of the RST transition, what remains to be done, whether it's
> >    all just useless churn that makes the documentation worse, etc.
> >
> >  - Things we'd like to improve in the documentation toolchain.
> >
> >  - Overall organization of Documentation/ and moving docs when the need
> >    arises.  It seems I end up fighting about this more than just about
> >    anything else, but I think it's important to organize our docs for the
> >    convenience of the people using them.
> >
> >  - The ultimate vision for kernel docs (for now).  RST conversion and
> >    imposing some organization are important, but they will not,
> >    themselves, give us a coherent set of documentation.  What can we do to
> >    have documentation that is useful, current, and maintainable, rather
> >    than the dusty attic we have now?  
> 
> Also,  it would be great if we could talk about cleaning up the
> documentation licensing, so it too can have its licenses be
> automatically detected.   :-)

With that regards, I'm still waiting for a solution with for the
GFDL license:

	https://github.com/spdx/license-list-XML/issues/686

All documents under Documentation/media now have a SPDX header,
except for the ones that were originally licensed under
the free version of GNU Free Document License (e. g. with
no invariant sections, no Front-Cover texts and no Back-Cover
texts) or that are dual-licensed GFDL/GPL.

While we don't have a SPDX tag for those, we can't finish adding
SPDX headers there though :-(


Thanks,
Mauro

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Kate Stewart]

On Wed, Jun 12, 2019 at 9:56 AM Jonathan Corbet <corbet@lwn.net> wrote:
>
> What could be more fun than talking about kernel documentation?  Things we
> could get into:
>
>  - The state of the RST transition, what remains to be done, whether it's
>    all just useless churn that makes the documentation worse, etc.
>
>  - Things we'd like to improve in the documentation toolchain.
>
>  - Overall organization of Documentation/ and moving docs when the need
>    arises.  It seems I end up fighting about this more than just about
>    anything else, but I think it's important to organize our docs for the
>    convenience of the people using them.
>
>  - The ultimate vision for kernel docs (for now).  RST conversion and
>    imposing some organization are important, but they will not,
>    themselves, give us a coherent set of documentation.  What can we do to
>    have documentation that is useful, current, and maintainable, rather
>    than the dusty attic we have now?

Also,  it would be great if we could talk about cleaning up the
documentation licensing, so it too can have its licenses be
automatically detected.   :-)

Kate

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Shuah Khan]

On 6/12/19 1:12 PM, Martin K. Petersen wrote:
> 
> Shuah,
> 
>> I am currently putting together a task list for mentorship program
>> with a goal to create tasks that would more meaningful and helpful
>> instead of whitespace patches.
> 
> If somebody is looking for meaningful documentation work, I have a
> substantial project in need of volunteers: The target stack wiki is
> stale, unmaintained, and often inaccessible. I would love for somebody
> to convert all that information to RST so we can stick it in the kernel
> and keep it in sync with the code changes.
> 

Awesome. Please send the project description to me. Feel free to break
it up as needed. We can chat offline about it.

thanks,
-- Shuah

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Martin K. Petersen]

Shuah,

> I am currently putting together a task list for mentorship program
> with a goal to create tasks that would more meaningful and helpful
> instead of whitespace patches.

If somebody is looking for meaningful documentation work, I have a
substantial project in need of volunteers: The target stack wiki is
stale, unmaintained, and often inaccessible. I would love for somebody
to convert all that information to RST so we can stick it in the kernel
and keep it in sync with the code changes.

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [Ksummit-discuss] [TECH TOPIC] Documentation

[Shuah Khan]

On 6/12/19 8:54 AM, Jonathan Corbet wrote:
> What could be more fun than talking about kernel documentation?  Things we
> could get into:
> 
>   - The state of the RST transition, what remains to be done, whether it's
>     all just useless churn that makes the documentation worse, etc.
> 
>   - Things we'd like to improve in the documentation toolchain.
> 
>   - Overall organization of Documentation/ and moving docs when the need
>     arises.  It seems I end up fighting about this more than just about
>     anything else, but I think it's important to organize our docs for the
>     convenience of the people using them.
> 
>   - The ultimate vision for kernel docs (for now).  RST conversion and
>     imposing some organization are important, but they will not,
>     themselves, give us a coherent set of documentation.  What can we do to
>     have documentation that is useful, current, and maintainable, rather
>     than the dusty attic we have now?
> 

The first step is identifying the ones that are out of date and/or
incorrect. I think this is probably the most time consuming part
and the second is start updating.

I am currently putting together a task list for mentorship program
with a goal to create tasks that would more meaningful and helpful
instead of whitespace patches.

I will gladly add documentation tasks to the list to help improve
the documentation.

thanks,
-- Shuah

[Ksummit-discuss] [TECH TOPIC] Documentation

[Jonathan Corbet]

What could be more fun than talking about kernel documentation?  Things we
could get into:

 - The state of the RST transition, what remains to be done, whether it's
   all just useless churn that makes the documentation worse, etc.

 - Things we'd like to improve in the documentation toolchain.

 - Overall organization of Documentation/ and moving docs when the need
   arises.  It seems I end up fighting about this more than just about
   anything else, but I think it's important to organize our docs for the
   convenience of the people using them.

 - The ultimate vision for kernel docs (for now).  RST conversion and
   imposing some organization are important, but they will not,
   themselves, give us a coherent set of documentation.  What can we do to
   have documentation that is useful, current, and maintainable, rather
   than the dusty attic we have now?

jon