[PATCH bpf-next v3 0/2] Add table of BPF program types to docs

[PATCH bpf-next v3 0/2] Add table of BPF program types to docs

[Donald Hunter]

Extend the libbpf documentation with a table of program types, attach
points and ELF section names. The table uses data from program_types.csv
which is generated from tools/lib/bpf/libbpf.c during the documentation
build.

Patch 1 adds subdir support to Documentation/Makefile and changes
userspace-api/media to use this instead of being a special case.

Patch 2 adds the program_types documentation with a new makefile in
the libbpf doc directory to generate program_types.csv

I plan to look at adding info about the format of section "extras" for
each program type as a follow-on.

v2 -> v3:
Put program_types after API docs in TOC as suggested by Andrii Nakryiko
Fix formatting as reported by Andrii Nakryiko
Include USDT extras example as suggested by Andrii Nakryiko
Include sample of program_types.csv as suggested by Andrii Nakryiko

v1 -> v2:
Automate the generation of program_types.csv as suggested by
Andrii Nakryiko.

Donald Hunter (2):
  Add subdir support to Documentation makefile
  Add table of BPF program types to libbpf docs

 Documentation/Makefile                     | 16 ++++++-
 Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
 Documentation/bpf/libbpf/index.rst         |  3 ++
 Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
 Documentation/bpf/programs.rst             |  3 ++
 Documentation/userspace-api/media/Makefile |  2 +
 6 files changed, 103 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/bpf/libbpf/Makefile
 create mode 100644 Documentation/bpf/libbpf/program_types.rst

-- 
2.35.1

[PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

[Donald Hunter]

Run make in list of subdirs to build generated sources and migrate
userspace-api/media to use this instead of being a special case.

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 Documentation/Makefile                     | 15 +++++++++++++--
 Documentation/userspace-api/media/Makefile |  2 ++
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 64d44c1ecad3..8a63ef2dcd1c 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -65,6 +65,12 @@ I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 # commands; the 'cmd' from scripts/Kbuild.include is not *loopable*
 loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 
+BUILD_SUBDIRS = \
+	Documentation/userspace-api/media
+
+quiet_cmd_build_subdir = SUBDIR  $2
+      cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
+
 # $2 sphinx builder e.g. "html"
 # $3 name of the build subfolder / e.g. "userspace-api/media", used as:
 #    * dest folder relative to $(BUILDDIR) and
@@ -74,7 +80,7 @@ loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 #    e.g. "userspace-api/media" for the linux-tv book-set at ./Documentation/userspace-api/media
 
 quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
-      cmd_sphinx = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/userspace-api/media $2 && \
+      cmd_sphinx =  \
 	PYTHONDONTWRITEBYTECODE=1 \
 	BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \
 	$(PYTHON3) $(srctree)/scripts/jobserver-exec \
@@ -93,6 +99,7 @@ quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
 
 htmldocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),html))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
 
 linkcheckdocs:
@@ -100,6 +107,7 @@ linkcheckdocs:
 
 latexdocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),latex))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
 
 ifeq ($(HAVE_PDFLATEX),0)
@@ -112,6 +120,7 @@ else # HAVE_PDFLATEX
 
 pdfdocs: latexdocs
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),latex))
 	$(foreach var,$(SPHINXDIRS), \
 	   $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit; \
 	   mkdir -p $(BUILDDIR)/$(var)/pdf; \
@@ -122,10 +131,12 @@ endif # HAVE_PDFLATEX
 
 epubdocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),epub))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
 
 xmldocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),xml))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
 
 endif # HAVE_SPHINX
@@ -138,7 +149,7 @@ refcheckdocs:
 
 cleandocs:
 	$(Q)rm -rf $(BUILDDIR)
-	$(Q)$(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/userspace-api/media clean
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),clean))
 
 dochelp:
 	@echo  ' Linux kernel internal documentation in different formats from ReST:'
diff --git a/Documentation/userspace-api/media/Makefile b/Documentation/userspace-api/media/Makefile
index 00922aa7efde..783c6f880b72 100644
--- a/Documentation/userspace-api/media/Makefile
+++ b/Documentation/userspace-api/media/Makefile
@@ -50,6 +50,8 @@ $(BUILDDIR)/lirc.h.rst: ${UAPI}/lirc.h ${PARSER} $(SRC_DIR)/lirc.h.rst.exception
 .PHONY: all html epub xml latex
 
 all: $(IMGDOT) $(BUILDDIR) ${TARGETS}
+	@:
+
 html: all
 epub: all
 xml: all
-- 
2.35.1

[PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

[Donald Hunter]

Extend the libbpf documentation with a table of program types,
attach points and ELF section names. This table uses data from
program_types.csv which is generated from tools/lib/bpf/libbpf.c
during the documentation build.

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 Documentation/Makefile                     |  3 +-
 Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
 Documentation/bpf/libbpf/index.rst         |  3 ++
 Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
 Documentation/bpf/programs.rst             |  3 ++
 5 files changed, 89 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/bpf/libbpf/Makefile
 create mode 100644 Documentation/bpf/libbpf/program_types.rst

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 8a63ef2dcd1c..f007314770e1 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -66,7 +66,8 @@ I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 
 BUILD_SUBDIRS = \
-	Documentation/userspace-api/media
+	Documentation/userspace-api/media \
+	Documentation/bpf/libbpf
 
 quiet_cmd_build_subdir = SUBDIR  $2
       cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
diff --git a/Documentation/bpf/libbpf/Makefile b/Documentation/bpf/libbpf/Makefile
new file mode 100644
index 000000000000..b3dc096c4a96
--- /dev/null
+++ b/Documentation/bpf/libbpf/Makefile
@@ -0,0 +1,49 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Rules to convert BPF program types in tools/lib/bpf/libbpf.c
+# into a .csv file
+
+FILES = program_types.csv
+
+TARGETS := $(addprefix $(BUILDDIR)/, $(FILES))
+
+# Extract program types and properties from the section definitions in libbpf.c such as
+# SEC_DEF("socket", SOCKET_FILTER, 0, SEC_NONE) to generate program_types.csv
+#
+# Here is a sample of the generated output that includes .rst formatting:
+#
+#  Program Type,Attach Type,ELF Section Name,Sleepable
+#  ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
+#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
+#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+
+$(BUILDDIR)/program_types.csv:	$(srctree)/tools/lib/bpf/libbpf.c
+	$(Q)awk -F'[",[:space:]]+' \
+	'BEGIN { print "Program Type,Attach Type,ELF Section Name,Sleepable" } \
+	/SEC_DEF\(\"/ && !/SEC_DEPRECATED/ { \
+	type = "``BPF_PROG_TYPE_" $$4 "``"; \
+	attach = index($$5, "0") ? "" : "``" $$5 "``"; \
+	section = "``" $$3 "``"; \
+	sleepable = index($$0, "SEC_SLEEPABLE") ? "Yes" : ""; \
+	print type "," attach "," section "," sleepable }' \
+	$< > $@
+
+.PHONY: all html epub xml latex linkcheck clean
+
+all: $(BUILDDIR) ${TARGETS}
+	@:
+
+html: all
+epub: all
+xml: all
+latex: all
+linkcheck:
+
+clean:
+	-$(Q)rm -f ${TARGETS} 2>/dev/null
+
+$(BUILDDIR):
+	$(Q)mkdir -p $@
diff --git a/Documentation/bpf/libbpf/index.rst b/Documentation/bpf/libbpf/index.rst
index 3722537d1384..f9b3b252e28f 100644
--- a/Documentation/bpf/libbpf/index.rst
+++ b/Documentation/bpf/libbpf/index.rst
@@ -1,5 +1,7 @@
 .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 
+.. _libbpf:
+
 libbpf
 ======
 
@@ -7,6 +9,7 @@ libbpf
    :maxdepth: 1
 
    API Documentation <https://libbpf.readthedocs.io/en/latest/api.html>
+   program_types
    libbpf_naming_convention
    libbpf_build
 
diff --git a/Documentation/bpf/libbpf/program_types.rst b/Documentation/bpf/libbpf/program_types.rst
new file mode 100644
index 000000000000..04fbb48b8a6a
--- /dev/null
+++ b/Documentation/bpf/libbpf/program_types.rst
@@ -0,0 +1,32 @@
+.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+
+.. _program_types_and_elf:
+
+Program Types and ELF Sections
+==============================
+
+The table below lists the program types, their attach types where relevant and the ELF section
+names supported by libbpf for them. The ELF section names follow these rules:
+
+- ``type`` is an exact match, e.g. ``SEC("socket")``
+- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")``
+  with a ‘``/``’ separator between ``type`` and ``extras``.
+
+When ``extras`` are specified, they provide details of how to auto-attach the BPF program.
+The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
+for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
+
+..
+  program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this:
+    Program Type,Attach Type,ELF Section Name,Sleepable
+    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
+    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+
+.. csv-table:: Program Types and Their ELF Section Names
+   :file: ../../output/program_types.csv
+   :widths: 40 30 20 10
+   :header-rows: 1
diff --git a/Documentation/bpf/programs.rst b/Documentation/bpf/programs.rst
index 620eb667ac7a..c99000ab6d9b 100644
--- a/Documentation/bpf/programs.rst
+++ b/Documentation/bpf/programs.rst
@@ -7,3 +7,6 @@ Program Types
    :glob:
 
    prog_*
+
+For a list of all program types, see :ref:`program_types_and_elf` in
+the :ref:`libbpf` documentation.
-- 
2.35.1

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

[Daniel Borkmann]

On 8/29/22 11:14 AM, Donald Hunter wrote:
> Run make in list of subdirs to build generated sources and migrate
> userspace-api/media to use this instead of being a special case.
> 
> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>

Jonathan, given this touches Documentation/Makefile, could you ACK if
it looks good to you? Noticed both patches don't have doc: $subj prefix,
but that's something we could fix up.

Maybe one small request, would be nice to build Documentation/bpf/libbpf/
also with every BPF CI run to avoid breakage of program_types.csv. Donald
could you check if feasible? Follow-up might be ok too, but up to Andrii.

Thanks,
Daniel

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

[Jani Nikula]

On Mon, 29 Aug 2022, Donald Hunter <donald.hunter@gmail.com> wrote:
> Extend the libbpf documentation with a table of program types,
> attach points and ELF section names. This table uses data from
> program_types.csv which is generated from tools/lib/bpf/libbpf.c
> during the documentation build.

Oh, would be nice to turn all of these into proper but simple Sphinx
extensions that take the deps into account in the Sphinx build
process. And, of course, would be pure python instead of a combo of
Make, shell, and awk.

That's one of the reasons we chose Sphinx originally, to be able to
write Sphinx extensions and avoid complicated pipelines.

BR,
Jani.


>
> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
> ---
>  Documentation/Makefile                     |  3 +-
>  Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
>  Documentation/bpf/libbpf/index.rst         |  3 ++
>  Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
>  Documentation/bpf/programs.rst             |  3 ++
>  5 files changed, 89 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/bpf/libbpf/Makefile
>  create mode 100644 Documentation/bpf/libbpf/program_types.rst
>
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index 8a63ef2dcd1c..f007314770e1 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -66,7 +66,8 @@ I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
>  loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
>  
>  BUILD_SUBDIRS = \
> -	Documentation/userspace-api/media
> +	Documentation/userspace-api/media \
> +	Documentation/bpf/libbpf
>  
>  quiet_cmd_build_subdir = SUBDIR  $2
>        cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
> diff --git a/Documentation/bpf/libbpf/Makefile b/Documentation/bpf/libbpf/Makefile
> new file mode 100644
> index 000000000000..b3dc096c4a96
> --- /dev/null
> +++ b/Documentation/bpf/libbpf/Makefile
> @@ -0,0 +1,49 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Rules to convert BPF program types in tools/lib/bpf/libbpf.c
> +# into a .csv file
> +
> +FILES = program_types.csv
> +
> +TARGETS := $(addprefix $(BUILDDIR)/, $(FILES))
> +
> +# Extract program types and properties from the section definitions in libbpf.c such as
> +# SEC_DEF("socket", SOCKET_FILTER, 0, SEC_NONE) to generate program_types.csv
> +#
> +# Here is a sample of the generated output that includes .rst formatting:
> +#
> +#  Program Type,Attach Type,ELF Section Name,Sleepable
> +#  ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
> +#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
> +#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
> +#  ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
> +#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
> +#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
> +
> +$(BUILDDIR)/program_types.csv:	$(srctree)/tools/lib/bpf/libbpf.c
> +	$(Q)awk -F'[",[:space:]]+' \
> +	'BEGIN { print "Program Type,Attach Type,ELF Section Name,Sleepable" } \
> +	/SEC_DEF\(\"/ && !/SEC_DEPRECATED/ { \
> +	type = "``BPF_PROG_TYPE_" $$4 "``"; \
> +	attach = index($$5, "0") ? "" : "``" $$5 "``"; \
> +	section = "``" $$3 "``"; \
> +	sleepable = index($$0, "SEC_SLEEPABLE") ? "Yes" : ""; \
> +	print type "," attach "," section "," sleepable }' \
> +	$< > $@
> +
> +.PHONY: all html epub xml latex linkcheck clean
> +
> +all: $(BUILDDIR) ${TARGETS}
> +	@:
> +
> +html: all
> +epub: all
> +xml: all
> +latex: all
> +linkcheck:
> +
> +clean:
> +	-$(Q)rm -f ${TARGETS} 2>/dev/null
> +
> +$(BUILDDIR):
> +	$(Q)mkdir -p $@
> diff --git a/Documentation/bpf/libbpf/index.rst b/Documentation/bpf/libbpf/index.rst
> index 3722537d1384..f9b3b252e28f 100644
> --- a/Documentation/bpf/libbpf/index.rst
> +++ b/Documentation/bpf/libbpf/index.rst
> @@ -1,5 +1,7 @@
>  .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
>  
> +.. _libbpf:
> +
>  libbpf
>  ======
>  
> @@ -7,6 +9,7 @@ libbpf
>     :maxdepth: 1
>  
>     API Documentation <https://libbpf.readthedocs.io/en/latest/api.html>
> +   program_types
>     libbpf_naming_convention
>     libbpf_build
>  
> diff --git a/Documentation/bpf/libbpf/program_types.rst b/Documentation/bpf/libbpf/program_types.rst
> new file mode 100644
> index 000000000000..04fbb48b8a6a
> --- /dev/null
> +++ b/Documentation/bpf/libbpf/program_types.rst
> @@ -0,0 +1,32 @@
> +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
> +
> +.. _program_types_and_elf:
> +
> +Program Types and ELF Sections
> +==============================
> +
> +The table below lists the program types, their attach types where relevant and the ELF section
> +names supported by libbpf for them. The ELF section names follow these rules:
> +
> +- ``type`` is an exact match, e.g. ``SEC("socket")``
> +- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")``
> +  with a ‘``/``’ separator between ``type`` and ``extras``.
> +
> +When ``extras`` are specified, they provide details of how to auto-attach the BPF program.
> +The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
> +for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
> +
> +..
> +  program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this:
> +    Program Type,Attach Type,ELF Section Name,Sleepable
> +    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
> +    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
> +    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
> +
> +.. csv-table:: Program Types and Their ELF Section Names
> +   :file: ../../output/program_types.csv
> +   :widths: 40 30 20 10
> +   :header-rows: 1
> diff --git a/Documentation/bpf/programs.rst b/Documentation/bpf/programs.rst
> index 620eb667ac7a..c99000ab6d9b 100644
> --- a/Documentation/bpf/programs.rst
> +++ b/Documentation/bpf/programs.rst
> @@ -7,3 +7,6 @@ Program Types
>     :glob:
>  
>     prog_*
> +
> +For a list of all program types, see :ref:`program_types_and_elf` in
> +the :ref:`libbpf` documentation.

-- 
Jani Nikula, Intel Open Source Graphics Center

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

[Jesper D. Brouer]

On 29/08/2022 11.15, Donald Hunter wrote:
> Extend the libbpf documentation with a table of program types,
> attach points and ELF section names. This table uses data from
> program_types.csv which is generated from tools/lib/bpf/libbpf.c
> during the documentation build.
> 
> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
> ---
[...]
> diff --git a/Documentation/bpf/libbpf/program_types.rst b/Documentation/bpf/libbpf/program_types.rst
> new file mode 100644
> index 000000000000..04fbb48b8a6a
> --- /dev/null
> +++ b/Documentation/bpf/libbpf/program_types.rst
> @@ -0,0 +1,32 @@
> +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
> +
> +.. _program_types_and_elf:
> +
> +Program Types and ELF Sections
> +==============================
> +
> +The table below lists the program types, their attach types where relevant and the ELF section
> +names supported by libbpf for them. The ELF section names follow these rules:
> +
> +- ``type`` is an exact match, e.g. ``SEC("socket")``
> +- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")``
> +  with a ‘``/``’ separator between ``type`` and ``extras``.
> +
> +When ``extras`` are specified, they provide details of how to auto-attach the BPF program.
> +The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
> +for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
> +
> +..
> +  program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this:
                                                                        ^
Typo:  s/fomatted/formatted

> +    Program Type,Attach Type,ELF Section Name,Sleepable
> +    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
> +    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
> +    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
> +
> +.. csv-table:: Program Types and Their ELF Section Names
> +   :file: ../../output/program_types.csv
> +   :widths: 40 30 20 10
> +   :header-rows: 1

Nice, I didn't know this formatting trick :-)

Appreciate you are working on this :-)
--Jesper

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

[Donald Hunter]

Daniel Borkmann <daniel@iogearbox.net> writes:

> On 8/29/22 11:14 AM, Donald Hunter wrote:
>> Run make in list of subdirs to build generated sources and migrate
>> userspace-api/media to use this instead of being a special case.
>> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
>
> Jonathan, given this touches Documentation/Makefile, could you ACK if
> it looks good to you? Noticed both patches don't have doc: $subj prefix,
> but that's something we could fix up.
>
> Maybe one small request, would be nice to build Documentation/bpf/libbpf/
> also with every BPF CI run to avoid breakage of program_types.csv. Donald
> could you check if feasible? Follow-up might be ok too, but up to Andrii.

Sure, I can look at what is needed for the BPF CI run.

> Thanks,
> Daniel

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

[Donald Hunter]

Jani Nikula <jani.nikula@linux.intel.com> writes:

> On Mon, 29 Aug 2022, Donald Hunter <donald.hunter@gmail.com> wrote:
>> Extend the libbpf documentation with a table of program types,
>> attach points and ELF section names. This table uses data from
>> program_types.csv which is generated from tools/lib/bpf/libbpf.c
>> during the documentation build.
>
> Oh, would be nice to turn all of these into proper but simple Sphinx
> extensions that take the deps into account in the Sphinx build
> process. And, of course, would be pure python instead of a combo of
> Make, shell, and awk.
>
> That's one of the reasons we chose Sphinx originally, to be able to
> write Sphinx extensions and avoid complicated pipelines.

I could look at this as a followup patch since I would need to learn how
to write Sphinx extensions first. It seems that it will require a new
reST directive, is that right?

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

[Jani Nikula]

On Tue, 06 Sep 2022, Donald Hunter <donald.hunter@gmail.com> wrote:
> Jani Nikula <jani.nikula@linux.intel.com> writes:
>
>> On Mon, 29 Aug 2022, Donald Hunter <donald.hunter@gmail.com> wrote:
>>> Extend the libbpf documentation with a table of program types,
>>> attach points and ELF section names. This table uses data from
>>> program_types.csv which is generated from tools/lib/bpf/libbpf.c
>>> during the documentation build.
>>
>> Oh, would be nice to turn all of these into proper but simple Sphinx
>> extensions that take the deps into account in the Sphinx build
>> process. And, of course, would be pure python instead of a combo of
>> Make, shell, and awk.
>>
>> That's one of the reasons we chose Sphinx originally, to be able to
>> write Sphinx extensions and avoid complicated pipelines.
>
> I could look at this as a followup patch since I would need to learn how
> to write Sphinx extensions first. It seems that it will require a new
> reST directive, is that right?

Correct. It's not very complicated. The extension that handles
kernel-doc is under 200 lines, Documentation/sphinx/kerneldoc.py. And a
lot of that is just translating directive options to kernel-doc the perl
script options.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

[Jani Nikula]

On Mon, 29 Aug 2022, Donald Hunter <donald.hunter@gmail.com> wrote:
> Extend the libbpf documentation with a table of program types,
> attach points and ELF section names. This table uses data from
> program_types.csv which is generated from tools/lib/bpf/libbpf.c
> during the documentation build.
>
> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
> ---
>  Documentation/Makefile                     |  3 +-
>  Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
>  Documentation/bpf/libbpf/index.rst         |  3 ++
>  Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
>  Documentation/bpf/programs.rst             |  3 ++
>  5 files changed, 89 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/bpf/libbpf/Makefile
>  create mode 100644 Documentation/bpf/libbpf/program_types.rst
>
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index 8a63ef2dcd1c..f007314770e1 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -66,7 +66,8 @@ I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
>  loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
>  
>  BUILD_SUBDIRS = \
> -	Documentation/userspace-api/media
> +	Documentation/userspace-api/media \
> +	Documentation/bpf/libbpf
>  
>  quiet_cmd_build_subdir = SUBDIR  $2
>        cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
> diff --git a/Documentation/bpf/libbpf/Makefile b/Documentation/bpf/libbpf/Makefile
> new file mode 100644
> index 000000000000..b3dc096c4a96
> --- /dev/null
> +++ b/Documentation/bpf/libbpf/Makefile
> @@ -0,0 +1,49 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Rules to convert BPF program types in tools/lib/bpf/libbpf.c
> +# into a .csv file
> +
> +FILES = program_types.csv
> +
> +TARGETS := $(addprefix $(BUILDDIR)/, $(FILES))
> +
> +# Extract program types and properties from the section definitions in libbpf.c such as
> +# SEC_DEF("socket", SOCKET_FILTER, 0, SEC_NONE) to generate program_types.csv
> +#
> +# Here is a sample of the generated output that includes .rst formatting:
> +#
> +#  Program Type,Attach Type,ELF Section Name,Sleepable
> +#  ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
> +#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
> +#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
> +#  ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
> +#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
> +#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
> +
> +$(BUILDDIR)/program_types.csv:	$(srctree)/tools/lib/bpf/libbpf.c
> +	$(Q)awk -F'[",[:space:]]+' \
> +	'BEGIN { print "Program Type,Attach Type,ELF Section Name,Sleepable" } \
> +	/SEC_DEF\(\"/ && !/SEC_DEPRECATED/ { \
> +	type = "``BPF_PROG_TYPE_" $$4 "``"; \
> +	attach = index($$5, "0") ? "" : "``" $$5 "``"; \
> +	section = "``" $$3 "``"; \
> +	sleepable = index($$0, "SEC_SLEEPABLE") ? "Yes" : ""; \
> +	print type "," attach "," section "," sleepable }' \
> +	$< > $@
> +
> +.PHONY: all html epub xml latex linkcheck clean
> +
> +all: $(BUILDDIR) ${TARGETS}
> +	@:
> +
> +html: all
> +epub: all
> +xml: all
> +latex: all
> +linkcheck:
> +
> +clean:
> +	-$(Q)rm -f ${TARGETS} 2>/dev/null
> +
> +$(BUILDDIR):
> +	$(Q)mkdir -p $@
> diff --git a/Documentation/bpf/libbpf/index.rst b/Documentation/bpf/libbpf/index.rst
> index 3722537d1384..f9b3b252e28f 100644
> --- a/Documentation/bpf/libbpf/index.rst
> +++ b/Documentation/bpf/libbpf/index.rst
> @@ -1,5 +1,7 @@
>  .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
>  
> +.. _libbpf:
> +
>  libbpf
>  ======
>  
> @@ -7,6 +9,7 @@ libbpf
>     :maxdepth: 1
>  
>     API Documentation <https://libbpf.readthedocs.io/en/latest/api.html>
> +   program_types
>     libbpf_naming_convention
>     libbpf_build
>  
> diff --git a/Documentation/bpf/libbpf/program_types.rst b/Documentation/bpf/libbpf/program_types.rst
> new file mode 100644
> index 000000000000..04fbb48b8a6a
> --- /dev/null
> +++ b/Documentation/bpf/libbpf/program_types.rst
> @@ -0,0 +1,32 @@
> +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
> +
> +.. _program_types_and_elf:
> +
> +Program Types and ELF Sections
> +==============================
> +
> +The table below lists the program types, their attach types where relevant and the ELF section
> +names supported by libbpf for them. The ELF section names follow these rules:
> +
> +- ``type`` is an exact match, e.g. ``SEC("socket")``
> +- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")``
> +  with a ‘``/``’ separator between ``type`` and ``extras``.
> +
> +When ``extras`` are specified, they provide details of how to auto-attach the BPF program.
> +The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
> +for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
> +
> +..
> +  program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this:
> +    Program Type,Attach Type,ELF Section Name,Sleepable
> +    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
> +    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
> +    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
> +
> +.. csv-table:: Program Types and Their ELF Section Names
> +   :file: ../../output/program_types.csv

Oh. You should probably test this with out-of-tree builds.

BR,
Jani.

> +   :widths: 40 30 20 10
> +   :header-rows: 1
> diff --git a/Documentation/bpf/programs.rst b/Documentation/bpf/programs.rst
> index 620eb667ac7a..c99000ab6d9b 100644
> --- a/Documentation/bpf/programs.rst
> +++ b/Documentation/bpf/programs.rst
> @@ -7,3 +7,6 @@ Program Types
>     :glob:
>  
>     prog_*
> +
> +For a list of all program types, see :ref:`program_types_and_elf` in
> +the :ref:`libbpf` documentation.

-- 
Jani Nikula, Intel Open Source Graphics Center

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

[Donald Hunter]

Jani Nikula <jani.nikula@linux.intel.com> writes:

>> +
>> +.. csv-table:: Program Types and Their ELF Section Names
>> +   :file: ../../output/program_types.csv
>
> Oh. You should probably test this with out-of-tree builds.

By out of tree, do you mean make htmldocs O=... or something else?

I have tested with O=... whih works for the kernel docs, but I think I
will need to rework the location to support the libbpf doc build which
has different directory layout.

Thanks,
Donald.

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

[Jani Nikula]

On Tue, 06 Sep 2022, Donald Hunter <donald.hunter@gmail.com> wrote:
> Jani Nikula <jani.nikula@linux.intel.com> writes:
>
>>> +
>>> +.. csv-table:: Program Types and Their ELF Section Names
>>> +   :file: ../../output/program_types.csv
>>
>> Oh. You should probably test this with out-of-tree builds.
>
> By out of tree, do you mean make htmldocs O=... or something else?

Yeah, O= is what I mean.

BR,
Jani.

>
> I have tested with O=... whih works for the kernel docs, but I think I
> will need to rework the location to support the libbpf doc build which
> has different directory layout.
>
> Thanks,
> Donald.

-- 
Jani Nikula, Intel Open Source Graphics Center

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

[Andrii Nakryiko]

On Tue, Sep 6, 2022 at 3:50 AM Donald Hunter <donald.hunter@gmail.com> wrote:
>
> Daniel Borkmann <daniel@iogearbox.net> writes:
>
> > On 8/29/22 11:14 AM, Donald Hunter wrote:
> >> Run make in list of subdirs to build generated sources and migrate
> >> userspace-api/media to use this instead of being a special case.
> >> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
> >
> > Jonathan, given this touches Documentation/Makefile, could you ACK if
> > it looks good to you? Noticed both patches don't have doc: $subj prefix,
> > but that's something we could fix up.
> >
> > Maybe one small request, would be nice to build Documentation/bpf/libbpf/
> > also with every BPF CI run to avoid breakage of program_types.csv. Donald
> > could you check if feasible? Follow-up might be ok too, but up to Andrii.
>
> Sure, I can look at what is needed for the BPF CI run.
>

Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?

> > Thanks,
> > Daniel

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

[Donald Hunter]

Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:

> On Tue, Sep 6, 2022 at 3:50 AM Donald Hunter <donald.hunter@gmail.com> wrote:
>>
>> Daniel Borkmann <daniel@iogearbox.net> writes:
>>
>> > On 8/29/22 11:14 AM, Donald Hunter wrote:
>> >> Run make in list of subdirs to build generated sources and migrate
>> >> userspace-api/media to use this instead of being a special case.
>> >> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
>> >
>> > Jonathan, given this touches Documentation/Makefile, could you ACK if
>> > it looks good to you? Noticed both patches don't have doc: $subj prefix,
>> > but that's something we could fix up.
>> >
>> > Maybe one small request, would be nice to build Documentation/bpf/libbpf/
>> > also with every BPF CI run to avoid breakage of program_types.csv. Donald
>> > could you check if feasible? Follow-up might be ok too, but up to Andrii.
>>
>> Sure, I can look at what is needed for the BPF CI run.
>>
>
> Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?

It looks to me like it can be added to BPF CI if we change docs/conf.py
to call a new make target in docs/sphinx/Makefile. Hopefully Daniel can
confirm whether this is the case.

Given that the tree layouts of the kernel and libbpf are very different
I think we need to do one of:

1. Use a separate a .csv generator script with dedicated makefiles per
project.

2. Move this entirely to libbpf and include a readthedocs URL in
index.rst in the same way as currently done for the API docs.

I think my preference would be for 2.

Thanks
Donald.

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

[Daniel Müller]

On Fri, Sep 09, 2022 at 11:12:22AM +0100, Donald Hunter wrote:
> Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
> 
> > On Tue, Sep 6, 2022 at 3:50 AM Donald Hunter <donald.hunter@gmail.com> wrote:
> >>
> >> Daniel Borkmann <daniel@iogearbox.net> writes:
> >>
> >> > On 8/29/22 11:14 AM, Donald Hunter wrote:
> >> >> Run make in list of subdirs to build generated sources and migrate
> >> >> userspace-api/media to use this instead of being a special case.
> >> >> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
> >> >
> >> > Jonathan, given this touches Documentation/Makefile, could you ACK if
> >> > it looks good to you? Noticed both patches don't have doc: $subj prefix,
> >> > but that's something we could fix up.
> >> >
> >> > Maybe one small request, would be nice to build Documentation/bpf/libbpf/
> >> > also with every BPF CI run to avoid breakage of program_types.csv. Donald
> >> > could you check if feasible? Follow-up might be ok too, but up to Andrii.
> >>
> >> Sure, I can look at what is needed for the BPF CI run.
> >>
> >
> > Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?

I think as long as all required packages are available in the CI distribution
(which I believe is currently a Ubuntu image, but may in the future become Arch
Linux) it should not be a problem to perform checking in CI. It seems as if
generating the documentation may take a while, so we should likely try to have
it run in a separate job. I can't tell what hidden dependencies there may be,
though.

> It looks to me like it can be added to BPF CI if we change docs/conf.py
> to call a new make target in docs/sphinx/Makefile. Hopefully Daniel can
> confirm whether this is the case.

I am not familiar with the documentation generation, but my quick search seems
to suggest that this is done by a 3rd party service and is decoupled from BPF
CI. Specifically, what you suggest may be reflected in the generated docs at
https://libbpf.readthedocs.io/, but I believe they are created from the libbpf
GitHub repository, which is only infrequently synced from bpf tree sources. I
didn't find any indication that CI triggers documentation creation, but it's
possible I missed something.

[...]

Thanks,
Daniel

Re: [PATCH bpf-next v3 0/2] Add table of BPF program types to docs

[Andrii Nakryiko]

On Mon, Aug 29, 2022 at 2:15 AM Donald Hunter <donald.hunter@gmail.com> wrote:
>
> Extend the libbpf documentation with a table of program types, attach
> points and ELF section names. The table uses data from program_types.csv
> which is generated from tools/lib/bpf/libbpf.c during the documentation
> build.
>
> Patch 1 adds subdir support to Documentation/Makefile and changes
> userspace-api/media to use this instead of being a special case.
>
> Patch 2 adds the program_types documentation with a new makefile in
> the libbpf doc directory to generate program_types.csv
>
> I plan to look at adding info about the format of section "extras" for
> each program type as a follow-on.
>
> v2 -> v3:
> Put program_types after API docs in TOC as suggested by Andrii Nakryiko
> Fix formatting as reported by Andrii Nakryiko
> Include USDT extras example as suggested by Andrii Nakryiko
> Include sample of program_types.csv as suggested by Andrii Nakryiko
>
> v1 -> v2:
> Automate the generation of program_types.csv as suggested by
> Andrii Nakryiko.
>
> Donald Hunter (2):
>   Add subdir support to Documentation makefile
>   Add table of BPF program types to libbpf docs
>
>  Documentation/Makefile                     | 16 ++++++-
>  Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
>  Documentation/bpf/libbpf/index.rst         |  3 ++
>  Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
>  Documentation/bpf/programs.rst             |  3 ++
>  Documentation/userspace-api/media/Makefile |  2 +
>  6 files changed, 103 insertions(+), 2 deletions(-)
>  create mode 100644 Documentation/bpf/libbpf/Makefile
>  create mode 100644 Documentation/bpf/libbpf/program_types.rst
>
> --
> 2.35.1
>

This is marked as Changes Requested, so I presume there are some
fixes/updates pending on top of v3? BPF CI should probably be done as
a follow up, though.

But otherwise looks good to me and I appreciate the effort to improve
libbpf's documentations.

Acked-by: Andrii Nakryiko <andrii@kernel.org>

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

[Donald Hunter]

Daniel Müller <deso@posteo.net> writes:

> On Fri, Sep 09, 2022 at 11:12:22AM +0100, Donald Hunter wrote:
>> Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
>> 
>> > On Tue, Sep 6, 2022 at 3:50 AM Donald Hunter <donald.hunter@gmail.com> wrote:
>> >>
>> >> Daniel Borkmann <daniel@iogearbox.net> writes:
>> >>
>> >> > On 8/29/22 11:14 AM, Donald Hunter wrote:
>> >> >> Run make in list of subdirs to build generated sources and migrate
>> >> >> userspace-api/media to use this instead of being a special case.
>> >> >> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
>> >> >
>> >> > Jonathan, given this touches Documentation/Makefile, could you ACK if
>> >> > it looks good to you? Noticed both patches don't have doc: $subj prefix,
>> >> > but that's something we could fix up.
>> >> >
>> >> > Maybe one small request, would be nice to build Documentation/bpf/libbpf/
>> >> > also with every BPF CI run to avoid breakage of program_types.csv. Donald
>> >> > could you check if feasible? Follow-up might be ok too, but up to Andrii.
>> >>
>> >> Sure, I can look at what is needed for the BPF CI run.
>> >>
>> >
>> > Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?
>
> I think as long as all required packages are available in the CI distribution
> (which I believe is currently a Ubuntu image, but may in the future become Arch
> Linux) it should not be a problem to perform checking in CI. It seems as if
> generating the documentation may take a while, so we should likely try to have
> it run in a separate job. I can't tell what hidden dependencies there may be,
> though.
>
>> It looks to me like it can be added to BPF CI if we change docs/conf.py
>> to call a new make target in docs/sphinx/Makefile. Hopefully Daniel can
>> confirm whether this is the case.
>
> I am not familiar with the documentation generation, but my quick search seems
> to suggest that this is done by a 3rd party service and is decoupled from BPF
> CI. Specifically, what you suggest may be reflected in the generated docs at
> https://libbpf.readthedocs.io/, but I believe they are created from the libbpf
> GitHub repository, which is only infrequently synced from bpf tree sources. I
> didn't find any indication that CI triggers documentation creation, but it's
> possible I missed something.

Apologies, I was referring to the libbpf GitHub repository. Anything
committed to the kernel Documentation/bpf/libbpf risks breaking the
documentation generation in the libbpf GitHub repository which, as you
say, uses readthedocs.

Re: [PATCH bpf-next v3 0/2] Add table of BPF program types to docs

[Donald Hunter]

Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:

> On Mon, Aug 29, 2022 at 2:15 AM Donald Hunter <donald.hunter@gmail.com> wrote:
>>
>> Extend the libbpf documentation with a table of program types, attach
>> points and ELF section names. The table uses data from program_types.csv
>> which is generated from tools/lib/bpf/libbpf.c during the documentation
>> build.
>>
>> Patch 1 adds subdir support to Documentation/Makefile and changes
>> userspace-api/media to use this instead of being a special case.
>>
>> Patch 2 adds the program_types documentation with a new makefile in
>> the libbpf doc directory to generate program_types.csv
>>
>> I plan to look at adding info about the format of section "extras" for
>> each program type as a follow-on.
>>
>> v2 -> v3:
>> Put program_types after API docs in TOC as suggested by Andrii Nakryiko
>> Fix formatting as reported by Andrii Nakryiko
>> Include USDT extras example as suggested by Andrii Nakryiko
>> Include sample of program_types.csv as suggested by Andrii Nakryiko
>>
>> v1 -> v2:
>> Automate the generation of program_types.csv as suggested by
>> Andrii Nakryiko.
>>
>> Donald Hunter (2):
>>   Add subdir support to Documentation makefile
>>   Add table of BPF program types to libbpf docs
>>
>>  Documentation/Makefile                     | 16 ++++++-
>>  Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
>>  Documentation/bpf/libbpf/index.rst         |  3 ++
>>  Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
>>  Documentation/bpf/programs.rst             |  3 ++
>>  Documentation/userspace-api/media/Makefile |  2 +
>>  6 files changed, 103 insertions(+), 2 deletions(-)
>>  create mode 100644 Documentation/bpf/libbpf/Makefile
>>  create mode 100644 Documentation/bpf/libbpf/program_types.rst
>>
>> --
>> 2.35.1
>>
>
> This is marked as Changes Requested, so I presume there are some
> fixes/updates pending on top of v3? BPF CI should probably be done as
> a follow up, though.

Yes, there is a typo reported by Jesper Brouer that will require a v4.

> But otherwise looks good to me and I appreciate the effort to improve
> libbpf's documentations.
>
> Acked-by: Andrii Nakryiko <andrii@kernel.org>

This patchset will break the documentation build in the libbpf GitHub
repository when Documentation/bpf/libbpf gets synced to there.

At a minimum, I think I would need to refactor this patchset so that it
is easier to integrate the .csv generation into the libbpf GitHub
documentation build. But a simpler way forward might be to move this
entirely to the libbpf Github repository and then just add a readthedocs
URL to Documentation/bpf/libbpf/index.rst, the same as currently done
for the API docs.