Re: [PATCH bpf-next 0/3] Autogenerating API documentation

[Grant Seltzer Richman <grantseltzer@gmail.com>]

On Mon, Jun 7, 2021 at 6:45 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:

> So I assume you looked at the DOC: block that Jonathan suggested
> above? Can you walk us a bit on how that would look like and why you
> think it's not going to work?

If I understand correctly, the using the  DOC: comment block and
extracting it would require you to manually update a comment in code
that labels the libbpf version number and then cut a release. The
problems are:

1) Having to do that manually
2) The docs on kernel.org would not show previous libbpf releases
(just the ones that were released with different kernel versions)

> > If you check out libbpf.readthedocs.io you can see what that would
> > look like. I made a test release (v21.21.21) to show how easy this is.
> > That is being pulled from my PR at github.com/libbpf/libbpf/pull/260.
>
> It looks pretty nice. Where does v21.21.21 come from, though?

libbpf.readthedocs.org currently tracks my libbpf fork at
github.com/grantseltzer/libbpf, I cut the 21.21.21 release there.

What this demonstrates is that readthedocs pulls in any release or
branch I enable in the admin panel and builds a seperate site for each
one. You can see the release/branches by toggling them in the bottom
left.

> also weird that docs are under src/docs, not just under docs/, but I
> assume that's a quick hack to demonstrate this?

I can move that no problem. All the config files have relative paths
that point at each other.

> To be entirely honest, I'm already a bit lost on all the
> possibilities. It would be great if you can summarize what's possible
> and how it would look like.

That's my fault, I'm not making this easy to review (2 different patch
series, plus github PR). Here is a summary of the 3 different
approaches I've presented. At the bottom I have my recommendation.

1) Integrate libbpf docs into the kernel documentation system:

- Add libbpf documentation files (e.g. 'naming-convention',
'building-libbpf') under the Linux 'Documentation' directory.
- Include a file which has sphinx directives to pull in API
documentation from the code under tools/lib/bpf.
- This documentation would appear on kernel.org/doc alongside all
other kernel documentation

Pro: Make use of existing kernel documentation infrastructure
Con: In order to have the libbpf documentation be versioned based on
github releases it would require manually updating comments in code.
To get it how we really want it (i.e. select a libbpf release and see
the documentation specific for it), it would likely require
rearranging/duplicating code in libbpf under subdirectories.
Contributing simple documentation has to go through the mailing list
which is daunting for new contributors.

2) Have all libbpf documentation in the kernel repo under
tools/lib/bpf/docs, including libbpf's own sphinx configuration files

- The github mirror would pull in these documentation files and sphinx
configuration files.
- readthedocs would be configured to build off of the github libbpf mirror repo.
- The documentation would be accessible at libbpf.readthedocs.org.
- Users would be able to navigate documentation of different releases
of libbpf easily. Maintainers would not need to do any manual work to
make that happen.

Pro: Documentation all in one place, in the kernel repo, integrates
easily with the github mirror, and we easily get to version
documentation by libbpf releases.
Con: We don't make use of existing kernel documentation
infrastructure. Contributing simple documentation has to go through
the mailing list which is daunting for new contributors.

3) Host all the documentation on the libbpf github mirror

- Documents (e.g. 'naming-convention', 'building-libbpf') would be
available at libbpf.readthedocs.org
- Sphinx configuration files would live in the libbpf github mirror
- Code comment documentation would still go through the mailing list

Pro: Documentation is explorable by different versions. Some
documentation can be contributed via github which is easier than the
mailing list.
Con: Documentation would be scattered across different places.

My recommendation:

I would go with option 2, it feels like the best compromise. However,
I'd like to contribute changes to the kernel documentation such that
it can support versioning particular documentation files based on
specified git hashes. That way we can have libbpf docs on kernel.org
but still get to version docs based on libbpf releases, not kernel
ones.

I'm happy to explore other options, but if one of these sounds like
something that we can agree on for now, I can submit a fresh patch for
it.


>
> As a general guidance, I think we should try to keep all the
> documentation in one place (which means kernel sources, because that's
> where API documentation will have to leave). As for config files,
> unless they will "stick out" too much, I'd keep them close to the docs
> themselves. If not, putting them in Github mirror is fine by me as
> well.
>
> Pretty much the only important aspect, from my point of view, is that
> docs are versioned according to the libbpf version, not kernel
> version. Otherwise huge confusion will ensue for all the users of
> libbpf, most of not all of which are using Github mirror.
>
> Does this make sense and is doable?
>
> > > >
> > > > If you're wanting to replace the version code that appears at the top of
> > > > the left column in the HTML output, though, it's going to be a bit
> > > > harder.  I don't doubt we can do it, but it may require messing around
> > > > with template files and such.
> > > >
> > > > Thanks,
> > > >
> > > > jon