Re: [PATCH v6 bpf-next 1/3] libbpf: BTF dumper support for typed data

Re: [PATCH v6 bpf-next 1/3] libbpf: BTF dumper support for typed data

[Naresh Kamboju]

On Thu, 15 Jul 2021 at 20:46, Alan Maguire <alan.maguire@oracle.com> wrote:
>
> Add a BTF dumper for typed data, so that the user can dump a typed
> version of the data provided.
>
> The API is
>
> int btf_dump__dump_type_data(struct btf_dump *d, __u32 id,
>                              void *data, size_t data_sz,
>                              const struct btf_dump_type_data_opts *opts);
>
> ...where the id is the BTF id of the data pointed to by the "void *"
> argument; for example the BTF id of "struct sk_buff" for a
> "struct skb *" data pointer.  Options supported are
>
>  - a starting indent level (indent_lvl)
>  - a user-specified indent string which will be printed once per
>    indent level; if NULL, tab is chosen but any string <= 32 chars
>    can be provided.
>  - a set of boolean options to control dump display, similar to those
>    used for BPF helper bpf_snprintf_btf().  Options are
>         - compact : omit newlines and other indentation
>         - skip_names: omit member names
>         - emit_zeroes: show zero-value members
>
> Default output format is identical to that dumped by bpf_snprintf_btf(),
> for example a "struct sk_buff" representation would look like this:
>
> struct sk_buff){
>         (union){
>                 (struct){
>                         .next = (struct sk_buff *)0xffffffffffffffff,
>                         .prev = (struct sk_buff *)0xffffffffffffffff,
>                 (union){
>                         .dev = (struct net_device *)0xffffffffffffffff,
>                         .dev_scratch = (long unsigned int)18446744073709551615,
>                 },
>         },
> ...
>
> If the data structure is larger than the *data_sz*
> number of bytes that are available in *data*, as much
> of the data as possible will be dumped and -E2BIG will
> be returned.  This is useful as tracers will sometimes
> not be able to capture all of the data associated with
> a type; for example a "struct task_struct" is ~16k.
> Being able to specify that only a subset is available is
> important for such cases.  On success, the amount of data
> dumped is returned.
>
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>  tools/lib/bpf/btf.h      |  19 ++
>  tools/lib/bpf/btf_dump.c | 819 ++++++++++++++++++++++++++++++++++++++++++++++-
>  tools/lib/bpf/libbpf.map |   1 +
>  3 files changed, 834 insertions(+), 5 deletions(-)

<trim>

> diff --git a/tools/lib/bpf/btf_dump.c b/tools/lib/bpf/btf_dump.c
> index 5dc6b517..929cf93 100644
> --- a/tools/lib/bpf/btf_dump.c
> +++ b/tools/lib/bpf/btf_dump.c


Following perf build errors noticed on i386 and arm 32-bit architectures on
linux next 20210719 tag with gcc-11.

metadata:
--------------
   git_repo: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next
   git_short_log: 08076eab6fef ( Add linux-next specific files for 20210719 )
   toolchain: gcc-11
   target_arch: arm and i386


> +static void btf_dump_int128(struct btf_dump *d,
> +                           const struct btf_type *t,
> +                           const void *data)
> +{
> +       __int128 num = *(__int128 *)data;


btf_dump.c: In function 'btf_dump_int128':
btf_dump.c:1559:9: error: expected expression before '__int128'
 1559 |         __int128 num = *(__int128 *)data;
      |         ^~~~~~~~
btf_dump.c:1561:14: error: 'num' undeclared (first use in this function)
 1561 |         if ((num >> 64) == 0)
      |              ^~~
btf_dump.c:1561:14: note: each undeclared identifier is reported only
once for each function it appears in
btf_dump.c: At top level:
btf_dump.c:1568:17: error: '__int128' is not supported on this target
 1568 | static unsigned __int128 btf_dump_bitfield_get_data(struct btf_dump *d,
      |                 ^~~~~~~~
btf_dump.c: In function 'btf_dump_bitfield_get_data':
btf_dump.c:1576:18: error: '__int128' is not supported on this target
 1576 |         unsigned __int128 num = 0, ret;
      |                  ^~~~~~~~
btf_dump.c: In function 'btf_dump_bitfield_check_zero':
btf_dump.c:1608:9: error: expected expression before '__int128'
 1608 |         __int128 check_num;
      |         ^~~~~~~~
btf_dump.c:1610:9: error: 'check_num' undeclared (first use in this function)
 1610 |         check_num = btf_dump_bitfield_get_data(d, t, data,
bits_offset, bit_sz);
      |         ^~~~~~~~~
btf_dump.c: In function 'btf_dump_bitfield_data':
btf_dump.c:1622:18: error: '__int128' is not supported on this target
 1622 |         unsigned __int128 print_num;
      |                  ^~~~~~~~
btf_dump.c: In function 'btf_dump_dump_type_data':
btf_dump.c:2212:34: error: '__int128' is not supported on this target
 2212 |                         unsigned __int128 print_num;
      |                                  ^~~~~~~~


Reported-by: Linux Kernel Functional Testing <lkft@linaro.org>

reference build link,
build: https://builds.tuxbuild.com/1vWeCpIox9EoV35c80bwOvU9nbb/
config: https://builds.tuxbuild.com/1vWeCpIox9EoV35c80bwOvU9nbb/config


steps to reproduce:
---------------------
# TuxMake is a command line tool and Python library that provides
# portable and repeatable Linux kernel builds across a variety of
# architectures, toolchains, kernel configurations, and make targets.
#
# TuxMake supports the concept of runtimes.
# See https://docs.tuxmake.org/runtimes/, for that to work it requires
# that you install podman or docker on your system.
#
# To install tuxmake on your system globally:
# sudo pip3 install -U tuxmake
#
# See https://docs.tuxmake.org/ for complete documentation.


tuxmake --runtime podman --target-arch arm --toolchain gcc-11
--kconfig defconfig --kconfig-add
https://builds.tuxbuild.com/1vWeCpIox9EoV35c80bwOvU9nbb/config


--
Linaro LKFT
https://lkft.linaro.org

Re: [PATCH v6 bpf-next 1/3] libbpf: BTF dumper support for typed data

[Alan Maguire]

On Mon, 19 Jul 2021, Naresh Kamboju wrote:

> On Thu, 15 Jul 2021 at 20:46, Alan Maguire <alan.maguire@oracle.com> wrote:
> >
> > Add a BTF dumper for typed data, so that the user can dump a typed
> > version of the data provided.
> 
> <trim>
> 
> > diff --git a/tools/lib/bpf/btf_dump.c b/tools/lib/bpf/btf_dump.c
> > index 5dc6b517..929cf93 100644
> > --- a/tools/lib/bpf/btf_dump.c
> > +++ b/tools/lib/bpf/btf_dump.c
> 
> 
> Following perf build errors noticed on i386 and arm 32-bit architectures on
> linux next 20210719 tag with gcc-11.
> 
> metadata:
> --------------
>    git_repo: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next
>    git_short_log: 08076eab6fef ( Add linux-next specific files for 20210719 )
>    toolchain: gcc-11
>    target_arch: arm and i386
> 
> 
> > +static void btf_dump_int128(struct btf_dump *d,
> > +                           const struct btf_type *t,
> > +                           const void *data)
> > +{
> > +       __int128 num = *(__int128 *)data;
> 
> 
> btf_dump.c: In function 'btf_dump_int128':
> btf_dump.c:1559:9: error: expected expression before '__int128'
>  1559 |         __int128 num = *(__int128 *)data;
>       |         ^~~~~~~~
> btf_dump.c:1561:14: error: 'num' undeclared (first use in this function)
>  1561 |         if ((num >> 64) == 0)
>       |              ^~~
> btf_dump.c:1561:14: note: each undeclared identifier is reported only
> once for each function it appears in
> btf_dump.c: At top level:
> btf_dump.c:1568:17: error: '__int128' is not supported on this target
>  1568 | static unsigned __int128 btf_dump_bitfield_get_data(struct btf_dump *d,
>       |                 ^~~~~~~~
> btf_dump.c: In function 'btf_dump_bitfield_get_data':
> btf_dump.c:1576:18: error: '__int128' is not supported on this target
>  1576 |         unsigned __int128 num = 0, ret;
>       |                  ^~~~~~~~
> btf_dump.c: In function 'btf_dump_bitfield_check_zero':
> btf_dump.c:1608:9: error: expected expression before '__int128'
>  1608 |         __int128 check_num;
>       |         ^~~~~~~~
> btf_dump.c:1610:9: error: 'check_num' undeclared (first use in this function)
>  1610 |         check_num = btf_dump_bitfield_get_data(d, t, data,
> bits_offset, bit_sz);
>       |         ^~~~~~~~~
> btf_dump.c: In function 'btf_dump_bitfield_data':
> btf_dump.c:1622:18: error: '__int128' is not supported on this target
>  1622 |         unsigned __int128 print_num;
>       |                  ^~~~~~~~
> btf_dump.c: In function 'btf_dump_dump_type_data':
> btf_dump.c:2212:34: error: '__int128' is not supported on this target
>  2212 |                         unsigned __int128 print_num;
>       |                                  ^~~~~~~~
> 
>

Thanks for the report Naresh! Andrii, I'm thinking the best
approach might be to remove use of int128 and have the bitfield
computations operate on a __u64 representation instead.  With
that change, we would only lose the ability to handle int128
bitfields; what do you think? I hope to have something ready
shortly covering that, the non-propogation of return values
and the endianness issues with enum handling - in fact the
latter goes away if the bitfield computations are done for
64-bit values.

Thanks!

Alan