[PATCH bpf-next] bpf: Document bpf_inspect drgn tool

[PATCH bpf-next] bpf: Document bpf_inspect drgn tool

[Andrey Ignatov]

It's a follow-up for discussion in [1].

drgn tool bpf_inspect.py was merged to drgn repo in [2]. Document it in
kernel tree to make BPF developers aware that the tool exists and can
help with getting BPF state unavailable via UAPI.

For now it's just one tool but the doc is written in a way that allows
to cover more tools in the future if needed.

Please refer to the doc itself for more details.

The patch was tested by `make htmldocs` and sanity-checking that
resulting html looks good.

[1]
https://lore.kernel.org/bpf/20200228201514.GB51456@rdna-mbp/T/#mefed65e8a98116bd5d07d09a570a3eac46724951
[2] https://github.com/osandov/drgn/pull/49

Signed-off-by: Andrey Ignatov <rdna@fb.com>
---
 Documentation/bpf/drgn.rst  | 42 +++++++++++++++++++++++++++++++++++++
 Documentation/bpf/index.rst |  5 +++--
 2 files changed, 45 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/bpf/drgn.rst

diff --git a/Documentation/bpf/drgn.rst b/Documentation/bpf/drgn.rst
new file mode 100644
index 000000000000..9a9ad75ab066
--- /dev/null
+++ b/Documentation/bpf/drgn.rst
@@ -0,0 +1,42 @@
+.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+
+==============
+BPF drgn tools
+==============
+
+drgn scripts are great to debug kernel internals including BPF and get
+information unavailable via conventional kernel UAPI.
+
+If there is a piece of kernel state useful for a small number of users, e.g.
+only for BPF developers, or too expensive to expose to user space, drgn script
+can be a good option to still have access to that state but without extending
+UAPI.
+
+This document describes BPF related drgn tools.
+
+See `drgn/tools`_ for all tools available at the moment and `drgn/doc`_ for
+more details on drgn itself.
+
+bpf_inspect.py
+**************
+
+`bpf_inspect.py`_ is a tool intended to inspect BPF programs and maps. It can
+iterate over all programs and maps in the system and print basic information
+about these objects, including id, type and name.
+
+The main use-case `bpf_inspect.py`_ covers is to show BPF programs of types
+``BPF_PROG_TYPE_EXT`` and ``BPF_PROG_TYPE_TRACING`` attached to other BPF
+programs via ``freplace``/``fentry``/``fexit`` mechanisms, since there is no
+user-space API to get this information.
+
+But developer can edit the tool and get any piece of ``struct bpf_prog`` or
+``struct bpf_map`` they're interested in, e.g. the whole ``struct
+bpf_prog_aux``.
+
+See ``--help`` for more details.
+
+.. Links
+.. _drgn/doc: https://drgn.readthedocs.io/en/latest/
+.. _drgn/tools: https://github.com/osandov/drgn/tree/master/tools
+.. _bpf_inspect.py:
+   https://github.com/osandov/drgn/blob/master/tools/bpf_inspect.py
diff --git a/Documentation/bpf/index.rst b/Documentation/bpf/index.rst
index 4f5410b61441..7be43c5f2dcf 100644
--- a/Documentation/bpf/index.rst
+++ b/Documentation/bpf/index.rst
@@ -47,12 +47,13 @@ Program types
    prog_flow_dissector
 
 
-Testing BPF
-===========
+Testing and debugging BPF
+=========================
 
 .. toctree::
    :maxdepth: 1
 
+   drgn
    s390
 
 
-- 
2.17.1

Re: [PATCH bpf-next] bpf: Document bpf_inspect drgn tool

[Alexei Starovoitov]

On Wed, Mar 11, 2020 at 12:14:40PM -0700, Andrey Ignatov wrote:
> It's a follow-up for discussion in [1].
> 
> drgn tool bpf_inspect.py was merged to drgn repo in [2]. Document it in
> kernel tree to make BPF developers aware that the tool exists and can
> help with getting BPF state unavailable via UAPI.
> 
> For now it's just one tool but the doc is written in a way that allows
> to cover more tools in the future if needed.
> 
> Please refer to the doc itself for more details.
> 
> The patch was tested by `make htmldocs` and sanity-checking that
> resulting html looks good.
> 
> [1]
> https://lore.kernel.org/bpf/20200228201514.GB51456@rdna-mbp/T/#mefed65e8a98116bd5d07d09a570a3eac46724951
> [2] https://github.com/osandov/drgn/pull/49
> 
> Signed-off-by: Andrey Ignatov <rdna@fb.com>
> ---
>  Documentation/bpf/drgn.rst  | 42 +++++++++++++++++++++++++++++++++++++
>  Documentation/bpf/index.rst |  5 +++--
>  2 files changed, 45 insertions(+), 2 deletions(-)
>  create mode 100644 Documentation/bpf/drgn.rst

Location looks good, but I gotta nit pick on wording...

> diff --git a/Documentation/bpf/drgn.rst b/Documentation/bpf/drgn.rst
> new file mode 100644
> index 000000000000..9a9ad75ab066
> --- /dev/null
> +++ b/Documentation/bpf/drgn.rst
> @@ -0,0 +1,42 @@
> +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
> +
> +==============
> +BPF drgn tools
> +==============
> +
> +drgn scripts are great to debug kernel internals including BPF and get
> +information unavailable via conventional kernel UAPI.
> +
> +If there is a piece of kernel state useful for a small number of users, e.g.
> +only for BPF developers, or too expensive to expose to user space, drgn script
> +can be a good option to still have access to that state but without extending
> +UAPI.

Above two paragraphs are true for any piece of kernel data.
I think they're unnecessary focusing attention on bpf.
May be rephrase the whole thing like:
"
drgn scripts is a convenient and easy to use mechanism to retrieve arbitrary
kernel data structures. drgn is not relying on kernel UAPI to read the data.
Instead it's reading directly from /proc/kcore or vmcore and pretty prints the
data based on dwarf debug information from vmlinux.
"

> +
> +This document describes BPF related drgn tools.
> +
> +See `drgn/tools`_ for all tools available at the moment and `drgn/doc`_ for
> +more details on drgn itself.
> +
> +bpf_inspect.py
> +**************
> +
> +`bpf_inspect.py`_ is a tool intended to inspect BPF programs and maps. It can
> +iterate over all programs and maps in the system and print basic information
> +about these objects, including id, type and name.
> +
> +The main use-case `bpf_inspect.py`_ covers is to show BPF programs of types
> +``BPF_PROG_TYPE_EXT`` and ``BPF_PROG_TYPE_TRACING`` attached to other BPF
> +programs via ``freplace``/``fentry``/``fexit`` mechanisms, since there is no
> +user-space API to get this information.
> +
> +But developer can edit the tool and get any piece of ``struct bpf_prog`` or

Just drop 'but' and say 'Any developer can edit ...'

> +``struct bpf_map`` they're interested in, e.g. the whole ``struct
> +bpf_prog_aux``.
> +

Re: [PATCH bpf-next] bpf: Document bpf_inspect drgn tool

[Andrey Ignatov]

Alexei Starovoitov <alexei.starovoitov@gmail.com> [Thu, 2020-03-12 18:27 -0700]:
> On Wed, Mar 11, 2020 at 12:14:40PM -0700, Andrey Ignatov wrote:
> > It's a follow-up for discussion in [1].
> > 
> > drgn tool bpf_inspect.py was merged to drgn repo in [2]. Document it in
> > kernel tree to make BPF developers aware that the tool exists and can
> > help with getting BPF state unavailable via UAPI.
> > 
> > For now it's just one tool but the doc is written in a way that allows
> > to cover more tools in the future if needed.
> > 
> > Please refer to the doc itself for more details.
> > 
> > The patch was tested by `make htmldocs` and sanity-checking that
> > resulting html looks good.
> > 
> > [1]
> > https://lore.kernel.org/bpf/20200228201514.GB51456@rdna-mbp/T/#mefed65e8a98116bd5d07d09a570a3eac46724951
> > [2] https://github.com/osandov/drgn/pull/49
> > 
> > Signed-off-by: Andrey Ignatov <rdna@fb.com>
> > ---
> >  Documentation/bpf/drgn.rst  | 42 +++++++++++++++++++++++++++++++++++++
> >  Documentation/bpf/index.rst |  5 +++--
> >  2 files changed, 45 insertions(+), 2 deletions(-)
> >  create mode 100644 Documentation/bpf/drgn.rst
> 
> Location looks good, but I gotta nit pick on wording...

Thanks for review. Both comments below make sense. I'll send v2.


> > diff --git a/Documentation/bpf/drgn.rst b/Documentation/bpf/drgn.rst
> > new file mode 100644
> > index 000000000000..9a9ad75ab066
> > --- /dev/null
> > +++ b/Documentation/bpf/drgn.rst
> > @@ -0,0 +1,42 @@
> > +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
> > +
> > +==============
> > +BPF drgn tools
> > +==============
> > +
> > +drgn scripts are great to debug kernel internals including BPF and get
> > +information unavailable via conventional kernel UAPI.
> > +
> > +If there is a piece of kernel state useful for a small number of users, e.g.
> > +only for BPF developers, or too expensive to expose to user space, drgn script
> > +can be a good option to still have access to that state but without extending
> > +UAPI.
> 
> Above two paragraphs are true for any piece of kernel data.
> I think they're unnecessary focusing attention on bpf.
> May be rephrase the whole thing like:
> "
> drgn scripts is a convenient and easy to use mechanism to retrieve arbitrary
> kernel data structures. drgn is not relying on kernel UAPI to read the data.
> Instead it's reading directly from /proc/kcore or vmcore and pretty prints the
> data based on dwarf debug information from vmlinux.
> "
> 
> > +
> > +This document describes BPF related drgn tools.
> > +
> > +See `drgn/tools`_ for all tools available at the moment and `drgn/doc`_ for
> > +more details on drgn itself.
> > +
> > +bpf_inspect.py
> > +**************
> > +
> > +`bpf_inspect.py`_ is a tool intended to inspect BPF programs and maps. It can
> > +iterate over all programs and maps in the system and print basic information
> > +about these objects, including id, type and name.
> > +
> > +The main use-case `bpf_inspect.py`_ covers is to show BPF programs of types
> > +``BPF_PROG_TYPE_EXT`` and ``BPF_PROG_TYPE_TRACING`` attached to other BPF
> > +programs via ``freplace``/``fentry``/``fexit`` mechanisms, since there is no
> > +user-space API to get this information.
> > +
> > +But developer can edit the tool and get any piece of ``struct bpf_prog`` or
> 
> Just drop 'but' and say 'Any developer can edit ...'
> 
> > +``struct bpf_map`` they're interested in, e.g. the whole ``struct
> > +bpf_prog_aux``.
> > +

-- 
Andrey Ignatov