Re: [PATCH bpf-next 00/17] Improve BPF syscall command documentation

[Jonathan Corbet <corbet@lwn.net>]

Joe Stringer <joe@cilium.io> writes:

> Hey Jon, thanks for the feedback. Absolutely, what you say makes
> sense. The intent here wasn't to come up with something new. Based on
> your prompt from this email (and a quick look at your KR '19
> presentation), I'm hearing a few observations:
> * Storing the documentation in the code next to the things that
> contributors edit is a reasonable approach to documentation of this
> kind.

Yes.  At least, it's what we do for a lot of our other documentation in
the kernel.  The assumption is that it will encourage developers to keep
the docs current; in my experience that's somewhat optimistic, but
optimism is good...:)

> * This series currently proposes adding some new Makefile
> infrastructure. However, good use of the "kernel-doc" sphinx directive
> + "DOC: " incantations in the header should be able to achieve the
> same without adding such dedicated build system logic to the tree.

If it can, I would certainly prefer to see it used - or extended, if
need be, to meet your needs.

> * The changes in patch 16 here extended Documentation/bpf/index.rst,
> but to assist in improving the overall kernel documentation
> organisation / hierarchy, you would prefer to instead introduce a
> dedicated Documentation/userspace-api/bpf/ directory where the bpf
> uAPI portions can be documented.

An objective I've been working on for some years is reorienting the
documentation with a focus on who the readers are.  We've tended to
organize it by subsystem, requiring people to wade through a lot of
stuff that isn't useful to them.  So yes, my preference would be to
document the kernel's user-space API in the relevant manual.

That said, I do tend to get pushback here at times, and the BPF API is
arguably a bit different that much of the rest.  So while the above
preference exists and is reasonably strong, the higher priority is to
get good, current documentation in *somewhere* so that it's available to
users.  I don't want to make life too difficult for people working
toward that goal, even if I would paint it a different color.

> In addition to this, today the bpf helpers documentation is built
> through the bpftool build process as well as the runtime bpf
> selftests, mostly as a way to ensure that the API documentation
> conforms to a particular style, which then assists with the generation
> of ReStructured Text and troff output. I can probably simplify the
> make infrastructure involved in triggering the bpf docs build for bpf
> subsystem developers and maintainers. I think there's likely still
> interest from bpf folks to keep that particular dependency in the
> selftests like today and even extend it to include this new
> Documentation, so that we don't either introduce text that fails
> against the parser or in some other way break the parser. Whether that
> validation is done by scripts/kernel-doc or scripts/bpf_helpers_doc.py
> doesn't make a big difference to me, other than I have zero experience
> with Perl. My first impressions are that the bpf_helpers_doc.py is
> providing stricter formatting requirements than what "DOC: " +
> kernel-doc would provide, so my baseline inclination would be to keep
> those patches to enhance that script and use that for the validation
> side (help developers with stronger linting feedback), then use
> kernel-doc for the actual html docs generation side, which would help
> to satisfy your concern around duplication of the documentation build
> systems.

This doesn't sound entirely unreasonable.  I wonder if the BPF helper
could be built into an sphinx extension to make it easy to pull that
information into the docs build.  The advantage there is that it can be
done in Python :)

Looking forward to the next set.

Thanks,

jon