Re: Help using libbpf with kernel 4.14

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Mon, Sep 28, 2020 at 1:08 PM Yaniv Agman <yanivagman@gmail.com> wrote:
>
> ‫בתאריך יום ב׳, 28 בספט׳ 2020 ב-8:50 מאת ‪Andrii Nakryiko‬‏
> <‪andrii.nakryiko@gmail.com‬‏>:‬
> >
> > On Fri, Sep 25, 2020 at 4:58 PM Yaniv Agman <yanivagman@gmail.com> wrote:
> > >
> > > Hello,
> > >
> > > I'm developing a tool which is now based on BCC, and would like to
> > > make the move to libbpf.
> > > I need the tool to support a minimal kernel version 4.14, which
> > > doesn't have CO-RE.
> >
> > You don't need kernel itself to support CO-RE, you just need that
> > kernel to have BTF in it. If the kernel is too old to have
> > CONFIG_DEBUG_INFO_BTF config, you can still add BTF by running `pahole
> > -J <path-to-vmlinux-image>`, if that's at all an option for your
> > setup.
> >
>
> Thanks, I didn't know that
>
> > >
> > > I have read bcc-to-libbpf-howto-guide, and looked at the libbpf-tools of bcc,
> > > but both only deal with newer kernels, and I failed to change them to
> > > run with a 4.14 kernel.
> > >
> > > Although some of the bpf samples in the kernel source don't use CO-RE,
> > > they all use bpf_load.h,
> > > and have dependencies on the tools dir, which I would like to avoid.
> >
> > Depending on what exactly you are trying to achieve with your BPF
> > application, you might not need BPF CO-RE, and using libbpf without
> > CO-RE would be enough for your needs. This would be the case if you
> > don't need to access any of the kernel data structures (e.g., all sort
> > of networking BPF apps: TC programs, cgroup sock progs, XDP). But if
> > you need to do anything tracing related (e.g., looking at kernel's
> > task_struct or any other internal structure), then you have no choice
> > and you either have to do on-the-target-host runtime compilation (BCC
> > way) or relocations (libbpf + BPF CO-RE). This is because of changing
> > memory layout of kernel structures.
> >
> > So, unless you can compile one specific version of your BPF code for a
> > one specific version of the kernel, you need either BCC or BPF CO-RE.
> >
>
> I'm working on a tracing application
> (https://github.com/aquasecurity/tracee) which now uses bcc. We now
> require a minimal kernel version 4.14, and bcc, but eventually we
> would like to support CO-RE. I thought that we could do the move in
> two steps. First moving to libbpf and keeping the 4.14 minimal
> requirement, then adding CO-RE support in the future.
> In order to do that, I thought of changing bcc requirement to clang
> requirement, and compile the program once during installation on the
> target host. This way we get the added value of fast start time
> without the need to compile every time the program starts (like bcc
> does), plus having an easier move to CO-RE in the future.

Right, pre-compiling on the target machine with host kernel headers
should work. So just don't use any of CO-RE features (no CO-RE
relocations, no vmlinux.h), and it should just work.

>
> A problem that I encountered with kernel 4.14 and libbpf was that when
> using bpf_prog_load (If I remember correctly), it returned an error of
> invalid argument (-22). Doing a small investigation I saw that it
> happened when trying to create bpf maps with names. Indeed I saw that
> libbpf API changed between kernel 4.14 and 4.15 and the function
> bpf_create_map_node now takes map name as an argument. Is there a way
> to workaround this with kernel 4.14 and still use map names in
> userspace to refer to bpf maps with libbpf?

So we do run a few simple tests loading BPF programs (using libbpf) on
4.9 kernel, so map name should definitely not be a problem at all
(libbpf is smart about detecting what's not supported in kernel and
omitting non-essential things). It might be because of bpf_prog_load
itself, which was long deprecated and you shouldn't use it for
real-world applications. Please either use BPF skeleton or bpf_object
APIs. It should just work, but if it doesn't please report back.

>
> > >
> > > I would appreciate it if someone can help with a simple working
> > > example of using libbpf on 4.14 kernel, without having any
> > > dependencies. Specifically, I'm looking for an example makefile, and
> > > to know how to load my bpf code with libbpf.
> >
> > libbpf-tools's Makefile would still work. Just drop dependency on
> > vmlinux.h and include system headers directly, if necessary (and if
> > you considered implications of kernel memory layout changes).
> >
>
> Thanks, I'll try that
>
> > >
> > > Thanks,
> > > Yaniv