From: Joe Stringer <joe@cilium.io>
To: bpf@vger.kernel.org
Cc: linux-man@vger.kernel.org, linux-doc@vger.kernel.org,
mtk.manpages@gmail.com, ast@kernel.org, brianvv@google.com,
daniel@iogearbox.net, daniel@zonque.org,
john.fastabend@gmail.com, ppenkov@google.com,
quentin@isovalent.com, sean@mess.org, yhs@fb.com
Subject: [PATCHv2 bpf-next 00/15] Improve BPF syscall command documentation
Date: Tue, 2 Mar 2021 09:19:32 -0800 [thread overview]
Message-ID: <20210302171947.2268128-1-joe@cilium.io> (raw)
The state of bpf(2) manual pages today is not exactly ideal. For the
most part, it was written several years ago and has not kept up with the
pace of development in the kernel tree. For instance, out of a total of
~35 commands to the BPF syscall available today, when I pull the
kernel-man-pages tree today I find just 6 documented commands: The very
basics of map interaction and program load.
In contrast, looking at bpf-helpers(7), I am able today to run one
command[0] to fetch API documentation of the very latest eBPF helpers
that have been added to the kernel. This documentation is up to date
because kernel maintainers enforce documenting the APIs as part of
the feature submission process. As far as I can tell, we rely on manual
synchronization from the kernel tree to the kernel-man-pages tree to
distribute these more widely, so all locations may not be completely up
to date. That said, the documentation does in fact exist in the first
place which is a major initial hurdle to overcome.
Given the relative success of the process around bpf-helpers(7) to
encourage developers to document their user-facing changes, in this
patch series I explore applying this technique to bpf(2) as well.
Unfortunately, even with bpf(2) being so out-of-date, there is still a
lot of content to convert over. In particular, the following aspects of
the bpf syscall could also be individually be generated from separate
documentation in the header:
* BPF syscall commands
* BPF map types
* BPF program types
* BPF program subtypes (aka expected_attach_type)
* BPF attachment points
Rather than tackle everything at once, I have focused in this series on
the syscall commands, "enum bpf_cmd". This series is structured to first
import what useful descriptions there are from the kernel-man-pages
tree, then piece-by-piece document a few of the syscalls in more detail
in cases where I could find useful documentation from the git tree or
from a casual read of the code. Not all documentation is comprehensive
at this point, but a basis is provided with examples that can be further
enhanced with subsequent follow-up patches. Note, the series in its
current state only includes documentation around the syscall commands
themselves, so in the short term it doesn't allow us to automate bpf(2)
generation; Only one section of the man page could be replaced. Though
if there is appetite for this approach, this should be trivial to
improve on, even if just by importing the remaining static text from the
kernel-man-pages tree.
Following that, the series enhances the python scripting around parsing
the descriptions from the header files and generating dedicated
ReStructured Text and troff output. Finally, to expose the new text and
reduce the likelihood of having it get out of date or break the docs
parser, it is added to the selftests and exposed through the kernel
documentation web pages.
The eventual goal of this effort would be to extend the kernel UAPI
headers such that each of the categories I had listed above (commands,
maps, progs, hooks) have dedicated documentation in the kernel tree, and
that developers must update the comments in the headers to document the
APIs prior to patch acceptance, and that we could auto-generate the
latest version of the bpf(2) manual pages based on a few static
description sections combined with the dynamically-generated output from
the header.
This patch series can also be found at the following location on GitHub:
https://github.com/joestringer/linux/tree/submit/bpf-command-docs_v2
Thanks also to Quentin Monnet for initial review.
[0]: make -C tools/testing/selftests/bpf docs
---
v2:
* Remove build infrastructure in favor of kernel-doc directives
* Shift userspace-api docs under Documentation/userspace-api/ebpf
* Fix scripts/bpf_doc.py syscall --header (throw unsupported error)
* Improve .gitignore handling of newly autogenerated files
---
Joe Stringer (15):
bpf: Import syscall arg documentation
bpf: Add minimal bpf() command documentation
bpf: Document BPF_F_LOCK in syscall commands
bpf: Document BPF_PROG_PIN syscall command
bpf: Document BPF_PROG_ATTACH syscall command
bpf: Document BPF_PROG_TEST_RUN syscall command
bpf: Document BPF_PROG_QUERY syscall command
bpf: Document BPF_MAP_*_BATCH syscall commands
scripts/bpf: Abstract eBPF API target parameter
scripts/bpf: Add syscall commands printer
tools/bpf: Remove bpf-helpers from bpftool docs
selftests/bpf: Templatize man page generation
selftests/bpf: Test syscall command parsing
docs/bpf: Add bpf() syscall command reference
tools: Sync uapi bpf.h header with latest changes
Documentation/bpf/index.rst | 9 +-
Documentation/userspace-api/ebpf/index.rst | 17 +
Documentation/userspace-api/ebpf/syscall.rst | 24 +
Documentation/userspace-api/index.rst | 1 +
MAINTAINERS | 2 +
include/uapi/linux/bpf.h | 714 +++++++++++++++++-
scripts/{bpf_helpers_doc.py => bpf_doc.py} | 191 ++++-
tools/bpf/Makefile.helpers | 60 --
tools/bpf/bpftool/.gitignore | 1 -
tools/bpf/bpftool/Documentation/Makefile | 11 +-
tools/include/uapi/linux/bpf.h | 714 +++++++++++++++++-
tools/lib/bpf/Makefile | 2 +-
tools/perf/MANIFEST | 2 +-
tools/testing/selftests/bpf/.gitignore | 2 +
tools/testing/selftests/bpf/Makefile | 20 +-
tools/testing/selftests/bpf/Makefile.docs | 82 ++
.../selftests/bpf/test_bpftool_build.sh | 21 -
tools/testing/selftests/bpf/test_doc_build.sh | 13 +
18 files changed, 1746 insertions(+), 140 deletions(-)
create mode 100644 Documentation/userspace-api/ebpf/index.rst
create mode 100644 Documentation/userspace-api/ebpf/syscall.rst
rename scripts/{bpf_helpers_doc.py => bpf_doc.py} (82%)
delete mode 100644 tools/bpf/Makefile.helpers
create mode 100644 tools/testing/selftests/bpf/Makefile.docs
create mode 100755 tools/testing/selftests/bpf/test_doc_build.sh
--
2.27.0
next reply other threads:[~2021-03-03 3:55 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-02 17:19 Joe Stringer [this message]
2021-03-02 17:19 ` [PATCHv2 bpf-next 01/15] bpf: Import syscall arg documentation Joe Stringer
2021-03-03 19:38 ` Yonghong Song
2021-03-03 20:20 ` Yonghong Song
2021-03-02 17:19 ` [PATCHv2 bpf-next 02/15] bpf: Add minimal bpf() command documentation Joe Stringer
2021-03-03 19:44 ` Yonghong Song
2021-03-02 17:19 ` [PATCHv2 bpf-next 03/15] bpf: Document BPF_F_LOCK in syscall commands Joe Stringer
2021-03-03 20:16 ` Yonghong Song
2021-03-02 17:19 ` [PATCHv2 bpf-next 04/15] bpf: Document BPF_PROG_PIN syscall command Joe Stringer
2021-03-03 20:21 ` Yonghong Song
2021-03-02 17:19 ` [PATCHv2 bpf-next 05/15] bpf: Document BPF_PROG_ATTACH " Joe Stringer
2021-03-03 20:23 ` Yonghong Song
2021-03-02 17:19 ` [PATCHv2 bpf-next 06/15] bpf: Document BPF_PROG_TEST_RUN " Joe Stringer
2021-03-03 20:29 ` Yonghong Song
2021-03-03 23:53 ` Joe Stringer
2021-04-10 18:12 ` Joe Stringer
2021-03-02 17:19 ` [PATCHv2 bpf-next 07/15] bpf: Document BPF_PROG_QUERY " Joe Stringer
2021-03-03 20:31 ` Yonghong Song
2021-03-02 17:19 ` [PATCHv2 bpf-next 08/15] bpf: Document BPF_MAP_*_BATCH syscall commands Joe Stringer
2021-03-03 17:38 ` Brian Vazquez
2021-03-03 20:44 ` Yonghong Song
2021-03-02 17:19 ` [PATCHv2 bpf-next 09/15] scripts/bpf: Abstract eBPF API target parameter Joe Stringer
2021-03-02 17:19 ` [PATCHv2 bpf-next 10/15] scripts/bpf: Add syscall commands printer Joe Stringer
2021-03-02 17:19 ` [PATCHv2 bpf-next 11/15] tools/bpf: Remove bpf-helpers from bpftool docs Joe Stringer
2021-03-02 17:19 ` [PATCHv2 bpf-next 12/15] selftests/bpf: Templatize man page generation Joe Stringer
2021-03-02 17:19 ` [PATCHv2 bpf-next 13/15] selftests/bpf: Test syscall command parsing Joe Stringer
2021-03-03 22:23 ` Yonghong Song
2021-03-03 23:50 ` Joe Stringer
2021-03-02 17:19 ` [PATCHv2 bpf-next 14/15] docs/bpf: Add bpf() syscall command reference Joe Stringer
2021-03-02 17:19 ` [PATCHv2 bpf-next 15/15] tools: Sync uapi bpf.h header with latest changes Joe Stringer
2021-03-03 17:25 ` [PATCHv2 bpf-next 00/15] Improve BPF syscall command documentation Jonathan Corbet
2021-03-03 18:50 ` Joe Stringer
2021-03-05 2:51 ` Alexei Starovoitov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210302171947.2268128-1-joe@cilium.io \
--to=joe@cilium.io \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=brianvv@google.com \
--cc=daniel@iogearbox.net \
--cc=daniel@zonque.org \
--cc=john.fastabend@gmail.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-man@vger.kernel.org \
--cc=mtk.manpages@gmail.com \
--cc=ppenkov@google.com \
--cc=quentin@isovalent.com \
--cc=sean@mess.org \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).