Re: one prog multi fentry. Was: [PATCH bpf-next] libbpf: support module BTF for BPF_TYPE_ID_TARGET CO-RE relocation

[Alan Maguire <alan.maguire@oracle.com>]

On Wed, 16 Dec 2020, Alexei Starovoitov wrote:

> > > $ ksnoop "ip_send_skb(skb->sk)"
> > >
> > > ...will trace the skb->sk value.  The user-space side of the program
> > > matches the function/arg name and looks up the referenced type, setting it
> > > in the function's map.  For field references such as skb->sk, it also
> > > records offset and whether that offset is a pointer (as is the case for
> > > skb->sk) - in such cases we need to read the offset value via bpf_probe_read()
> > > and use it in bpf_snprintf_btf() along with the referenced type.  Only a
> > > single simple reference like the above is supported currently, but
> > > multiple levels of reference could be made to work too.
> 
> Alan,
> 
> I'm not sure why the last example is so different form the first two.
> I think ksnoop tool will generate the program on the fly, right?

Nope, the BPF program is hard-coded; it adapts to different functions
through use of the map entries describing function signatures and their
BTF ids, and other associated tracing info.  The aim is to provide a
generic tracing tool which displays kernel function arguments but
doesn't require LLVM/clang on the target, just a kernel built with BTF 
and libbpf.  Sorry this wasn't clearer in my explanation; I'm working
on rewriting the code and will send it out ASAP.

> So it can generate normal LDX insn with CO-RE relocation (instead of bpf_probe_read)
> to access skb->sk. It can also add relo for that LDX to point to
> struct sk_buff's btf_id defined inside prog's BTF.
> The 'sk' offset inside bpf program and inside BTF can be anything: 0, 4, ...
> libbpf relocation logic will find the right offset in kernel's sk_buff.
> If ksnoop doesn't have an ability to parse vmlinux.h file or kernel's BTF
> it can 'cheat'.
> If the cmdline looks like:
> $ ksnoop "ip_send_skb(skb->sk)"
> It can generate BTF:
> struct sk_buff {
>    struct sock *sk;
> };
> 
> If cmdline looks like:
> $ ksnoop "ip_send_skb(skb->sock)"
> It can generate BTF:
> struct sk_buff {
>    struct sock *sock;
> };
> Obviously there is no 'sock' field inside kernel's struct sk_buff, but tool
> doesn't need to care. It can let libbpf do the checking and match
> fields properly.
> 
> > > into that a bit more if you don't mind because I think some form of
> > > user-space-specified BTF ids may be the easiest approach for more flexible
> > > generic tracing that covers more than function arguments.
> 
> I think you're trying to figure out kernel's btf_ids in ksnoop tool.

Yep.

> I suggest to leave that job to libbpf. Generate local BTFs in ksnoop
> with CO-RE relocs and let libbpf handle insn patching.
> No FDs to worry about from ksnoop side either.
> 

The current approach doesn't rely on instruction patching outside
of limited CORE use around struct pt_regs fields (args, IP, etc)
which shouldn't require LLVM/clang availability on the target system. 
I'll try and get it ready for RFC submission by the weekend so you
can see more details of the approach.

Thanks!

Alan