Re: [BPF CO-RE clarification] Use CO-RE on older kernel versions.

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Wed, Mar 17, 2021 at 7:31 AM Rafael David Tinoco
<rafaeldtinoco@ubuntu.com> wrote:
>
>
> >>
> >>>> From what I see all the CO-RE relocations applied successfully (even
> >>>> though all the offsets stayed the same, so presumably you compiled
> >>>> your BPF program with vmlinux.h from the exact same kernel you are
> >>>> running it on?). Are you sure that vmlinux image you are providing
> >>>> corresponds to the actual kernel you are running on?
> >
> > FOUND the cause of the issue…
> >
>
> ...
>
> >
> > bpf_check():
> >
> > if (log->len_total < 128 || log->len_total > UINT_MAX >> 8 || !log->level
> > || !log->ubuf)
> >
> > and a simple change in libbpf (mitigation of course):
> >
> >   attr.log_buf = 0;
> >   attr.log_level = 0;
> >   attr.log_size = 0;
> >
> > before
> >
> > fd = sys_bpf_prog_load(&attr, sizeof(attr));
>
> With instrumented kernel… no changes to libbpf or code:
>
> attr.log_buf = (nil)
> attr.log_level = 0
> attr.log_size = 0
> load_attr.log_buf = 0x7fd1c0a92010
> load_attr.log_level = 0
> load_attr.log_size = 16777215
> libbpf: load bpf program failed: Invalid argument
> libbpf: failed to load program 'ip_set_create'
> libbpf: failed to load object 'mine_bpf'
> libbpf: failed to load BPF skeleton 'mine_bpf': -22
> failed to load BPF object: -22
>
> [   27.857858] MINE: BPF_PROG_TYPE_KPROBE KERNEL VERSION ISSUE
> [   27.857993] MINE: LINUX_VERSION_CODE: 266002
> [   27.858131] MINE: YOUR VERSION: 265984
>
> 2 problems here:
>
> 0) attr.kern_version
>
> - looks like for some reason attr.kern_version is different from
> the one running
>
> - bypassing kernel BPF_KPROBE version check, I get:
>
> 1) load_attr has log_buf and log_size but not log_level for some reason.
>
> - this triggers an issue in bpf_check() within kernel IF I bypass
> the BPF_KPROBE version check (kerne 4.15).
>
> NOW, If I hard-code attr.kern_version in bpf.c to:
>
> attr.kern_version = 266002;
> fd = sys_bpf_prog_load(&attr, sizeof(attr));
>
> then
>
> attr.log_buf = (nil)
> attr.log_level = 0
> attr.log_size = 0
> load_attr.log_buf = (nil)
> load_attr.log_level = 0
> load_attr.log_size = 0
> Tracing... Hit Ctrl-C to end.
>
> I don’t have the 2 problems, as log_level is zeroed, together with
> log_buf and log_size.
>
> Any clue on why this happens ?

So seems like you figured out kernel_version check, right? And it
seems like log_buf is not really a problem as well? Or it still
causing issues?

>
> Thank you!
>
> -rafaeldtinoco
>