[PATCH bpf-next v4 0/2] Add table of BPF program types to docs

[PATCH bpf-next v4 0/2] Add table of BPF program types to docs

[Donald Hunter]

Extend the libbpf documentation with a table of program types, attach
points and ELF section names. The table uses data from program_types.csv
which is generated from tools/lib/bpf/libbpf.c during the documentation
build.

Patch 1 adds subdir support to Documentation/Makefile and changes
userspace-api/media to use this instead of being a special case.

Patch 2 adds the program_types documentation with a new makefile in
the libbpf doc directory to generate program_types.csv, using
scripts/gen-bpf-progtypes.sh

The generator is now in scripts/gen-bpf-progtypes.sh so that it can be
exported to the libbpf git repository along with the libbpf docs. The
libbpf repository will need to be updated to sync and use
scripts/gen-btf-progtypes.sh after this patch lands.

I plan to look at adding info about the format of section "extras" for
each program type as a follow-on.

v3 -> v4:
Fix typo reported by Jesper Brouer
Move the generator from the Makefile to a separate script

v2 -> v3:
Put program_types after API docs in TOC as suggested by Andrii Nakryiko
Fix formatting as reported by Andrii Nakryiko
Include USDT extras example as suggested by Andrii Nakryiko
Include sample of program_types.csv as suggested by Andrii Nakryiko

v1 -> v2:
Automate the generation of program_types.csv as suggested by
Andrii Nakryiko.

Donald Hunter (2):
  Add subdir support to Documentation makefile
  Add table of BPF program types to libbpf docs

 Documentation/Makefile                     | 16 ++++++++--
 Documentation/bpf/libbpf/.gitignore        |  1 +
 Documentation/bpf/libbpf/Makefile          | 29 ++++++++++++++++++
 Documentation/bpf/libbpf/index.rst         |  3 ++
 Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++++++++
 Documentation/bpf/programs.rst             |  3 ++
 Documentation/userspace-api/media/Makefile |  2 ++
 scripts/gen-bpf-progtypes.sh               | 34 ++++++++++++++++++++++
 8 files changed, 118 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/bpf/libbpf/.gitignore
 create mode 100644 Documentation/bpf/libbpf/Makefile
 create mode 100644 Documentation/bpf/libbpf/program_types.rst
 create mode 100755 scripts/gen-bpf-progtypes.sh

-- 
2.35.1

[PATCH bpf-next v4 1/2] Add subdir support to Documentation makefile

[Donald Hunter]

Run make in list of subdirs to build generated sources and migrate
userspace-api/media to use this instead of being a special case.

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 Documentation/Makefile                     | 15 +++++++++++++--
 Documentation/userspace-api/media/Makefile |  2 ++
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 64d44c1ecad3..8a63ef2dcd1c 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -65,6 +65,12 @@ I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 # commands; the 'cmd' from scripts/Kbuild.include is not *loopable*
 loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 
+BUILD_SUBDIRS = \
+	Documentation/userspace-api/media
+
+quiet_cmd_build_subdir = SUBDIR  $2
+      cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
+
 # $2 sphinx builder e.g. "html"
 # $3 name of the build subfolder / e.g. "userspace-api/media", used as:
 #    * dest folder relative to $(BUILDDIR) and
@@ -74,7 +80,7 @@ loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 #    e.g. "userspace-api/media" for the linux-tv book-set at ./Documentation/userspace-api/media
 
 quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
-      cmd_sphinx = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/userspace-api/media $2 && \
+      cmd_sphinx =  \
 	PYTHONDONTWRITEBYTECODE=1 \
 	BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \
 	$(PYTHON3) $(srctree)/scripts/jobserver-exec \
@@ -93,6 +99,7 @@ quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
 
 htmldocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),html))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
 
 linkcheckdocs:
@@ -100,6 +107,7 @@ linkcheckdocs:
 
 latexdocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),latex))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
 
 ifeq ($(HAVE_PDFLATEX),0)
@@ -112,6 +120,7 @@ else # HAVE_PDFLATEX
 
 pdfdocs: latexdocs
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),latex))
 	$(foreach var,$(SPHINXDIRS), \
 	   $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit; \
 	   mkdir -p $(BUILDDIR)/$(var)/pdf; \
@@ -122,10 +131,12 @@ endif # HAVE_PDFLATEX
 
 epubdocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),epub))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
 
 xmldocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),xml))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
 
 endif # HAVE_SPHINX
@@ -138,7 +149,7 @@ refcheckdocs:
 
 cleandocs:
 	$(Q)rm -rf $(BUILDDIR)
-	$(Q)$(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/userspace-api/media clean
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),clean))
 
 dochelp:
 	@echo  ' Linux kernel internal documentation in different formats from ReST:'
diff --git a/Documentation/userspace-api/media/Makefile b/Documentation/userspace-api/media/Makefile
index 00922aa7efde..783c6f880b72 100644
--- a/Documentation/userspace-api/media/Makefile
+++ b/Documentation/userspace-api/media/Makefile
@@ -50,6 +50,8 @@ $(BUILDDIR)/lirc.h.rst: ${UAPI}/lirc.h ${PARSER} $(SRC_DIR)/lirc.h.rst.exception
 .PHONY: all html epub xml latex
 
 all: $(IMGDOT) $(BUILDDIR) ${TARGETS}
+	@:
+
 html: all
 epub: all
 xml: all
-- 
2.35.1

[PATCH bpf-next v4 2/2] Add table of BPF program types to libbpf docs

[Donald Hunter]

Extend the libbpf documentation with a table of program types,
attach points and ELF section names. This table uses data from
program_types.csv which is generated from tools/lib/bpf/libbpf.c
during the documentation build.

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 Documentation/Makefile                     |  3 +-
 Documentation/bpf/libbpf/.gitignore        |  1 +
 Documentation/bpf/libbpf/Makefile          | 29 ++++++++++++++++++
 Documentation/bpf/libbpf/index.rst         |  3 ++
 Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++++++++
 Documentation/bpf/programs.rst             |  3 ++
 scripts/gen-bpf-progtypes.sh               | 34 ++++++++++++++++++++++
 7 files changed, 104 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/bpf/libbpf/.gitignore
 create mode 100644 Documentation/bpf/libbpf/Makefile
 create mode 100644 Documentation/bpf/libbpf/program_types.rst
 create mode 100755 scripts/gen-bpf-progtypes.sh

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 8a63ef2dcd1c..f007314770e1 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -66,7 +66,8 @@ I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 
 BUILD_SUBDIRS = \
-	Documentation/userspace-api/media
+	Documentation/userspace-api/media \
+	Documentation/bpf/libbpf
 
 quiet_cmd_build_subdir = SUBDIR  $2
       cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
diff --git a/Documentation/bpf/libbpf/.gitignore b/Documentation/bpf/libbpf/.gitignore
new file mode 100644
index 000000000000..c9013b8cae08
--- /dev/null
+++ b/Documentation/bpf/libbpf/.gitignore
@@ -0,0 +1 @@
+/program_types.csv
diff --git a/Documentation/bpf/libbpf/Makefile b/Documentation/bpf/libbpf/Makefile
new file mode 100644
index 000000000000..9cc3f67b5602
--- /dev/null
+++ b/Documentation/bpf/libbpf/Makefile
@@ -0,0 +1,29 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Rules to convert BPF program types in tools/lib/bpf/libbpf.c
+# into a .csv file
+
+PROGRAM_TYPES = $(srctree)/Documentation/bpf/libbpf/program_types.csv
+
+TARGETS := $(PROGRAM_TYPES)
+
+
+$(PROGRAM_TYPES):	$(srctree)/tools/lib/bpf/libbpf.c
+	$(Q)$(srctree)/scripts/gen-bpf-progtypes.sh $< $@
+
+.PHONY: all html epub xml latex linkcheck clean
+
+all: $(BUILDDIR) ${TARGETS}
+	@:
+
+html: all
+epub: all
+xml: all
+latex: all
+linkcheck:
+
+clean:
+	-$(Q)rm -f ${TARGETS} 2>/dev/null
+
+$(BUILDDIR):
+	$(Q)mkdir -p $@
diff --git a/Documentation/bpf/libbpf/index.rst b/Documentation/bpf/libbpf/index.rst
index 3722537d1384..f9b3b252e28f 100644
--- a/Documentation/bpf/libbpf/index.rst
+++ b/Documentation/bpf/libbpf/index.rst
@@ -1,5 +1,7 @@
 .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 
+.. _libbpf:
+
 libbpf
 ======
 
@@ -7,6 +9,7 @@ libbpf
    :maxdepth: 1
 
    API Documentation <https://libbpf.readthedocs.io/en/latest/api.html>
+   program_types
    libbpf_naming_convention
    libbpf_build
 
diff --git a/Documentation/bpf/libbpf/program_types.rst b/Documentation/bpf/libbpf/program_types.rst
new file mode 100644
index 000000000000..b74fbf3363dd
--- /dev/null
+++ b/Documentation/bpf/libbpf/program_types.rst
@@ -0,0 +1,32 @@
+.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+
+.. _program_types_and_elf:
+
+Program Types and ELF Sections
+==============================
+
+The table below lists the program types, their attach types where relevant and the ELF section
+names supported by libbpf for them. The ELF section names follow these rules:
+
+- ``type`` is an exact match, e.g. ``SEC("socket")``
+- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")``
+  with a ‘``/``’ separator between ``type`` and ``extras``.
+
+When ``extras`` are specified, they provide details of how to auto-attach the BPF program.
+The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
+for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
+
+..
+  program_types.csv is generated from tools/lib/bpf/libbpf.c and is formatted like this:
+    Program Type,Attach Type,ELF Section Name,Sleepable
+    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
+    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+
+.. csv-table:: Program Types and Their ELF Section Names
+   :file: program_types.csv
+   :widths: 40 30 20 10
+   :header-rows: 1
diff --git a/Documentation/bpf/programs.rst b/Documentation/bpf/programs.rst
index 620eb667ac7a..c99000ab6d9b 100644
--- a/Documentation/bpf/programs.rst
+++ b/Documentation/bpf/programs.rst
@@ -7,3 +7,6 @@ Program Types
    :glob:
 
    prog_*
+
+For a list of all program types, see :ref:`program_types_and_elf` in
+the :ref:`libbpf` documentation.
diff --git a/scripts/gen-bpf-progtypes.sh b/scripts/gen-bpf-progtypes.sh
new file mode 100755
index 000000000000..3bf36f26a2f7
--- /dev/null
+++ b/scripts/gen-bpf-progtypes.sh
@@ -0,0 +1,34 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Red Hat.
+#
+# Generate a .csv table of BPF program types
+
+if [ "$#" -lt 2 ]; then
+    echo "Usage: gen-bpf-progtypes.sh </path/to/libbpf.c> </path/to/generated.csv>"
+    exit 1
+fi
+
+# Extract program types and properties from the section definitions in libbpf.c such as
+# SEC_DEF("socket", SOCKET_FILTER, 0, SEC_NONE) to generate a table of program types in
+# .csv format.
+#
+# Here is a sample of the generated output that includes .rst formatting:
+#
+#  Program Type,Attach Type,ELF Section Name,Sleepable
+#  ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
+#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
+#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+
+awk -F'[",[:space:]]+' \
+    'BEGIN { print "Program Type,Attach Type,ELF Section Name,Sleepable" }
+    /SEC_DEF\(\"/ && !/SEC_DEPRECATED/ {
+    type = "``BPF_PROG_TYPE_" $4 "``"
+    attach = index($5, "0") ? "" : "``" $5 "``";
+    section = "``" $3 "``"
+    sleepable = index($0, "SEC_SLEEPABLE") ? "Yes" : "";
+    print type "," attach "," section "," sleepable }' $1 | sort > $2
-- 
2.35.1

Re: [PATCH bpf-next v4 1/2] Add subdir support to Documentation makefile

[Jonathan Corbet]

Donald Hunter <donald.hunter@gmail.com> writes:

> Run make in list of subdirs to build generated sources and migrate
> userspace-api/media to use this instead of being a special case.
>
> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>

This could really use a bit more information on exactly what you're
doing and why you want to do it.

Beyond that, I would *really* like to see more use of Sphinx extensions
for this kind of special-case build rather than hacking in more
special-purpose scripts.  Is there a reason why it couldn't be done that
way?

Thanks,

jon

Re: [PATCH bpf-next v4 2/2] Add table of BPF program types to libbpf docs

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 2483 bytes --]

On Thu, Sep 22, 2022 at 12:52:57PM +0100, Donald Hunter wrote:
> +..
> +  program_types.csv is generated from tools/lib/bpf/libbpf.c and is formatted like this:
> +    Program Type,Attach Type,ELF Section Name,Sleepable
> +    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
> +    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
> +    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
> +    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes

The note above doesn't get rendered on htmldocs output, so I have applied
the fixup:

---- >8 ----

diff --git a/Documentation/bpf/libbpf/program_types.rst b/Documentation/bpf/libbpf/program_types.rst
index b74fbf3363dd6c..3ce0ec94b399b4 100644
--- a/Documentation/bpf/libbpf/program_types.rst
+++ b/Documentation/bpf/libbpf/program_types.rst
@@ -16,15 +16,17 @@ When ``extras`` are specified, they provide details of how to auto-attach the BP
 The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
 for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
 
-..
-  program_types.csv is generated from tools/lib/bpf/libbpf.c and is formatted like this:
-    Program Type,Attach Type,ELF Section Name,Sleepable
-    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
-    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
-    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
-    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
-    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
-    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+.. note::
+   The table below is generated from ``tools/lib/bpf/libbpf.c`` and is
+   formatted like this (in CSV format)::
+
+     Program Type,Attach Type,ELF Section Name,Sleepable
+     BPF_PROG_TYPE_SOCKET_FILTER,,socket,
+     BPF_PROG_TYPE_SK_REUSEPORT,BPF_SK_REUSEPORT_SELECT_OR_MIGRATE,sk_reuseport/migrate,
+     BPF_PROG_TYPE_SK_REUSEPORT,BPF_SK_REUSEPORT_SELECT,sk_reuseport,
+     BPF_PROG_TYPE_KPROBE,,kprobe+,
+     BPF_PROG_TYPE_KPROBE,,uprobe+,
+     BPF_PROG_TYPE_KPROBE,,uprobe.s+,Yes
 
 .. csv-table:: Program Types and Their ELF Section Names
    :file: program_types.csv
 
Thanks.

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH bpf-next v4 1/2] Add subdir support to Documentation makefile

[Donald Hunter]

Jonathan Corbet <corbet@lwn.net> writes:

> Donald Hunter <donald.hunter@gmail.com> writes:
>
>> Run make in list of subdirs to build generated sources and migrate
>> userspace-api/media to use this instead of being a special case.
>>
>> Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
>
> This could really use a bit more information on exactly what you're
> doing and why you want to do it.

What would you like me to add? Something like:

"... in preparation for running a generator script to produce .csv data
used by bpf documentation"

> Beyond that, I would *really* like to see more use of Sphinx extensions
> for this kind of special-case build rather than hacking in more
> special-purpose scripts.  Is there a reason why it couldn't be done that
> way?

I looked at writing the BPF program types as a Sphinx extension but
found that approach problematic for the following reasons:

- This needs to run both in the kernel tree and the libbpf Github
  project. The tree layouts are different so the relative paths to
  source files are different. I don't see an elegant way to handle this
  inline in a .rst file. This can easily be handled in Makefiles
  that are specific to each project.

- It makes use of csv-table which does all the heavy lifting to produce
  the desired output.

- I have zero experience of extending Sphinx.

I thought about submitting this directly to the libbpf Github project
and then just linking from the kernel docs to the page about program
types in the libbpf project docs. But I think it is preferable to master
the gen-bpf-progtypes.sh script in the kernel tree where it can be
maintained in the same repo as the libbpf.c source file it parses.

Re: [PATCH bpf-next v4 2/2] Add table of BPF program types to libbpf docs

[Donald Hunter]

Bagas Sanjaya <bagasdotme@gmail.com> writes:

> On Thu, Sep 22, 2022 at 12:52:57PM +0100, Donald Hunter wrote:
>> +..
>> +  program_types.csv is generated from tools/lib/bpf/libbpf.c and is formatted like this:
>> +    Program Type,Attach Type,ELF Section Name,Sleepable
>> +    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
>> +    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
>> +    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
>> +    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
>> +    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
>> +    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
>
> The note above doesn't get rendered on htmldocs output, so I have applied
> the fixup:

It was intended to be a comment to the reader of program_types.rst that
this is the format of the .csv file that will be rendered. It was not
meant to be a note on the rendered page.

The rendered page will show the table that is produced by the csv-table
directive which is self explanatory.

>
> ---- >8 ----
>
> diff --git a/Documentation/bpf/libbpf/program_types.rst b/Documentation/bpf/libbpf/program_types.rst
> index b74fbf3363dd6c..3ce0ec94b399b4 100644
> --- a/Documentation/bpf/libbpf/program_types.rst
> +++ b/Documentation/bpf/libbpf/program_types.rst
> @@ -16,15 +16,17 @@ When ``extras`` are specified, they provide details of how to auto-attach the BP
>  The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
>  for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
>  
> -..
> -  program_types.csv is generated from tools/lib/bpf/libbpf.c and is formatted like this:
> -    Program Type,Attach Type,ELF Section Name,Sleepable
> -    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
> -    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
> -    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
> -    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
> -    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
> -    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
> +.. note::
> +   The table below is generated from ``tools/lib/bpf/libbpf.c`` and is
> +   formatted like this (in CSV format)::
> +
> +     Program Type,Attach Type,ELF Section Name,Sleepable
> +     BPF_PROG_TYPE_SOCKET_FILTER,,socket,
> +     BPF_PROG_TYPE_SK_REUSEPORT,BPF_SK_REUSEPORT_SELECT_OR_MIGRATE,sk_reuseport/migrate,
> +     BPF_PROG_TYPE_SK_REUSEPORT,BPF_SK_REUSEPORT_SELECT,sk_reuseport,
> +     BPF_PROG_TYPE_KPROBE,,kprobe+,
> +     BPF_PROG_TYPE_KPROBE,,uprobe+,
> +     BPF_PROG_TYPE_KPROBE,,uprobe.s+,Yes
>  
>  .. csv-table:: Program Types and Their ELF Section Names
>     :file: program_types.csv
>  
> Thanks.

Re: [PATCH bpf-next v4 2/2] Add table of BPF program types to libbpf docs

[Bagas Sanjaya]

On 9/23/22 22:05, Donald Hunter wrote:
>> The note above doesn't get rendered on htmldocs output, so I have applied
>> the fixup:
> 
> It was intended to be a comment to the reader of program_types.rst that
> this is the format of the .csv file that will be rendered. It was not
> meant to be a note on the rendered page.
> 
> The rendered page will show the table that is produced by the csv-table
> directive which is self explanatory.
> 

AFAIK, I don't see any disclaimer for that table...

-- 
An old man doll... just what I always wanted! - Clara

Re: [PATCH bpf-next v4 1/2] Add subdir support to Documentation makefile

[Donald Hunter]

Donald Hunter <donald.hunter@gmail.com> writes:

> Jonathan Corbet <corbet@lwn.net> writes:
>
>> Beyond that, I would *really* like to see more use of Sphinx extensions
>> for this kind of special-case build rather than hacking in more
>> special-purpose scripts.  Is there a reason why it couldn't be done that
>> way?
>
> I looked at writing the BPF program types as a Sphinx extension but
> found that approach problematic for the following reasons:
>
> - This needs to run both in the kernel tree and the libbpf Github
>   project. The tree layouts are different so the relative paths to
>   source files are different. I don't see an elegant way to handle this
>   inline in a .rst file. This can easily be handled in Makefiles
>   that are specific to each project.
>
> - It makes use of csv-table which does all the heavy lifting to produce
>   the desired output.
>
> - I have zero experience of extending Sphinx.
>
> I thought about submitting this directly to the libbpf Github project
> and then just linking from the kernel docs to the page about program
> types in the libbpf project docs. But I think it is preferable to master
> the gen-bpf-progtypes.sh script in the kernel tree where it can be
> maintained in the same repo as the libbpf.c source file it parses.

Given the pushback on Makefile changes and the need for this patch to be
compatible with both the kernel tree and the libbpf repo, can I suggest
a pragmatic way forward.

I suggest that I drop the gen-bpf-progtypes.sh script and Makefile
changes from the patchset and just submit static documentation contents
for the table of BPF program types. This would avoid any downstream
breakage when syncing from the kernel tree to the libbpf github
repo. The table of BPF program types can be maintained manually which
should not be a burden going forward. Another benefit would be that the
resulting documentation can be curated more easily than if it were
auto-generated.

Re: [PATCH bpf-next v4 1/2] Add subdir support to Documentation makefile

[Andrii Nakryiko]

On Fri, Sep 30, 2022 at 2:58 AM Donald Hunter <donald.hunter@gmail.com> wrote:
>
> Donald Hunter <donald.hunter@gmail.com> writes:
>
> > Jonathan Corbet <corbet@lwn.net> writes:
> >
> >> Beyond that, I would *really* like to see more use of Sphinx extensions
> >> for this kind of special-case build rather than hacking in more
> >> special-purpose scripts.  Is there a reason why it couldn't be done that
> >> way?
> >
> > I looked at writing the BPF program types as a Sphinx extension but
> > found that approach problematic for the following reasons:
> >
> > - This needs to run both in the kernel tree and the libbpf Github
> >   project. The tree layouts are different so the relative paths to
> >   source files are different. I don't see an elegant way to handle this
> >   inline in a .rst file. This can easily be handled in Makefiles
> >   that are specific to each project.
> >
> > - It makes use of csv-table which does all the heavy lifting to produce
> >   the desired output.
> >
> > - I have zero experience of extending Sphinx.
> >
> > I thought about submitting this directly to the libbpf Github project
> > and then just linking from the kernel docs to the page about program
> > types in the libbpf project docs. But I think it is preferable to master
> > the gen-bpf-progtypes.sh script in the kernel tree where it can be
> > maintained in the same repo as the libbpf.c source file it parses.
>
> Given the pushback on Makefile changes and the need for this patch to be
> compatible with both the kernel tree and the libbpf repo, can I suggest
> a pragmatic way forward.
>
> I suggest that I drop the gen-bpf-progtypes.sh script and Makefile

It's way too easy to forget to update this table when adding new
program type support in libbpf. So if possible I think script is the
way to go.

Jonathan, given the script is really minimal and allows to keep
documentation in sync with libbpf source code, do you have a strong
objection? Writing a custom plugin seems like a too high bar for
something pretty straightforward like this?

Also, should this be routed through your tree or you'd like us to take
it through bpf-next tree?


Donald, if Jonathan is feeling really strongly, then I guess manually
generated table is ok approach as well, but let's hear from Jonathan
first.

Also cc Grant about logistics of kernel vs libbpf docs.


> changes from the patchset and just submit static documentation contents
> for the table of BPF program types. This would avoid any downstream
> breakage when syncing from the kernel tree to the libbpf github
> repo. The table of BPF program types can be maintained manually which
> should not be a burden going forward. Another benefit would be that the
> resulting documentation can be curated more easily than if it were
> auto-generated.