Re: Is BTF info sufficient enough for BPFTrace and other debug tools to run ?

[Johnny young <johnny96.young@gmail.com>]

Hey Stephen,

Thank you for the detailed explanation. I hope that after your work is
done for drgn as mentioned and the necessary work is done from the
debug tool side,  then the debug tools will rely on the kernel
internal symbol table and BTF only in the future.  That's wonderful.

Best,

On Thu, Sep 29, 2022 at 2:35 PM Stephen Brennan
<stephen.s.brennan@oracle.com> wrote:
>
> Johnny young <johnny96.young@gmail.com> writes:
> > Hello BPF
> >
> > I understand that CONFIG_DEBUG_INFO_BTF=y will generate .BTF and
> > .BTF_xx  sections in the kernel image which are much smaller than
> > those DWARF sections.  But I also try to understand how BTF can impact
> > bpftrace and the existing debug tools:
> >
> > 1) If the kernel is built with CONFIG_DEBUG_INFO_BTF=y, can
> > bpftrace relies on BTF only without kernel_devel ?
> >
> > 2) Can the existing kernel debugging tools like crash(1) or
> > kgdb(1) take advantage of BTF ?
>
> Hi Johnny,
>
> I can't answer all your questions but I can chime in regarding
> debuggers. BTF could provide information to both of these debuggers, but
> currently neither of them know how to use it. There's a few reasons and
> challenges.
>
> 1. Kernel and module BTF today only includes the type information
>    necessary to describe all functions, and percpu variables. Most
>    debuggers would also like to know the types of global variables. I've
>    got a patch series [1] which allows pahole (which generates the
>    kernel & module BTF) to output information for global variable
>    declarations as well.
>
> 2. Assuming you had the BTF, you'd also need a symbol table.  The kernel
>    has kallsyms, which is an internal symbol table, and most debuggers
>    today don't know how to read that - instead, they rely on the symbol
>    table from the ELF debuginfo file. I frequently work with a debugger
>    library called drgn [2], and I've got a branch out for review [3]
>    which enables drgn to read the kallsyms. In order to do that,
>    debuggers need to be able to *find* the kallsyms table, so I added
>    information to the vmcoreinfo note in the commits 5fd8fea935a1
>    ("vmcoreinfo: include kallsyms symbols") and f09bddbd8661
>    ("vmcoreinfo: add kallsyms_num_syms symbol").
>
> 3. Assuming you have symbol table access and the BTF is complete enough
>    for you to use it for real debugging, then your debugger still needs
>    to have logic to *find* the BTF and parse it (probably with libbpf).
>    I've got a branch for drgn [4] which implements BTF parsing on top of
>    the kallsyms parsing logic. With those things together, you get a
>    debugger which relies on nothing except data it finds inside the
>    kernel, and it works well enough.
>
> So the short answer is - no, currently there is no debugger (that I know
> of) which can leverage BTF. But it's in the works for drgn, and once we
> get there with drgn, I'd hope to see that developers for other debuggers
> might see the power of it and consider implementing it.
>
> [1]: https://lore.kernel.org/bpf/20220826184911.168442-1-stephen.s.brennan@oracle.com/
> [2]: https://drgn.readthedocs.io/
> [3]: https://github.com/osandov/drgn/pull/177
> [4]: https://github.com/brenns10/drgn/tree/kallsyms_plus_btf
>
> >
> > 3) If the kernel is built with CONFIG_DEBUG_INFO_BTF=y, are the
> > symbolic info and types info in the debug-info section replaced with
> > BTF formatted info?
>
> No, BTF is generated _in addition to_ the existing type information. BTF
> doesn't store "symbolic info" i.e. symbol table data, so it couldn't
> replace that data. It only stores type information, so hypothetically it
> could be used instead of DWARF .debug_types data, but in practice that
> doesn't happen.
>
> Stephen
>
> >
> > 4) Given the current upstream development effort for BTF, can we run
> > bpftrace without LLVM now ? and can we run bpftrace without the help
> > of kernel header files (kernel-devel) ?
> >
> > 5) Has bpf CO-RE become reality now?
> >
> > Thank you!
> > Johnny