Re: [PATCH bpf-next 1/7] libbpf: add BPF-side of USDT support

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Wed, Mar 30, 2022 at 8:22 AM Hengqi Chen <hengqi.chen@gmail.com> wrote:
>
>
>
> On 2022/3/30 11:10 AM, Hengqi Chen wrote:
> > On 2022/3/25 1:29 PM, Andrii Nakryiko wrote:
> >> Add BPF-side implementation of libbpf-provided USDT support. This
> >> consists of single header library, usdt.bpf.h, which is meant to be used
> >> from user's BPF-side source code. This header is added to the list of
> >> installed libbpf header, along bpf_helpers.h and others.
> >>
> >> BPF-side implementation consists of two BPF maps:
> >>   - spec map, which contains "a USDT spec" which encodes information
>
> ...
>
> >> +}
> >> +
> >> +/* Fetch USDT argument *arg* (zero-indexed) and put its value into *res.
> >> + * Returns 0 on success; negative error, otherwise.
> >> + * On error *res is guaranteed to be set to zero.
> >> + */
> >> +__hidden __weak
> >> +int bpf_usdt_arg(struct pt_regs *ctx, int arg, long *res)
> >> +{
> >> +    struct __bpf_usdt_spec *spec;
> >> +    struct __bpf_usdt_arg_spec *arg_spec;
> >> +    unsigned long val;
> >> +    int err, spec_id;
> >> +
> >> +    *res = 0;
> >> +
> >> +    spec_id = __bpf_usdt_spec_id(ctx);
> >> +    if (spec_id < 0)
> >> +            return -ESRCH;
> >> +
> >> +    spec = bpf_map_lookup_elem(&__bpf_usdt_specs, &spec_id);
> >> +    if (!spec)
> >> +            return -ESRCH;
> >> +
> >> +    if (arg >= spec->arg_cnt)
> >> +            return -ENOENT;
> >> +
> >> +    arg_spec = &spec->args[arg];
> >> +    switch (arg_spec->arg_type) {
> >> +    case BPF_USDT_ARG_CONST:
> >> +            val = arg_spec->val_off;
> >> +            break;
> >> +    case BPF_USDT_ARG_REG:
> >> +            err = bpf_probe_read_kernel(&val, sizeof(val), (void *)ctx + arg_spec->reg_off);
> >> +            if (err)
> >> +                    return err;
> >> +            break;
> >> +    case BPF_USDT_ARG_REG_DEREF:
> >> +            err = bpf_probe_read_kernel(&val, sizeof(val), (void *)ctx + arg_spec->reg_off);
> >> +            if (err)
> >> +                    return err;
> >> +            err = bpf_probe_read_user(&val, sizeof(val), (void *)val + arg_spec->val_off);
> >> +            if (err)
> >> +                    return err;
>
> Can you elaborate more on these two probe read call ?
>

I can add some comments here for each BPF_USDT_xxx case.

> I replace bpf_probe_read_kernel with bpf_probe_read_user, it also works.
>

You must be running some pretty old kernel on which there is no
bpf_probe_read_{user,kernel} and libbpf "downgrades" them to
bpf_probe_read() which works for both. It needs to be kernel read
because we are reading a field from struct pt_regs, which is in kernel
address space.

> Thanks.
>
> >> +            break;
> >> +    default:
> >> +            return -EINVAL;
> >> +    }
> >> +
> >> +    val <<= arg_spec->arg_bitshift;
> >> +    if (arg_spec->arg_signed)
> >> +            val = ((long)val) >> arg_spec->arg_bitshift;
>
> >> + * BPF_USDT serves the same purpose for USDT handlers as BPF_PROG for
> >> + * tp_btf/fentry/fexit BPF programs and BPF_KPROBE for kprobes.
> >> + * Original struct pt_regs * context is preserved as 'ctx' argument.
> >> + */
> >> +#define BPF_USDT(name, args...)                                                 \
> >> +name(struct pt_regs *ctx);                                              \
> >> +static __attribute__((always_inline)) typeof(name(0))                           \
> >> +____##name(struct pt_regs *ctx, ##args);                                \
> >> +typeof(name(0)) name(struct pt_regs *ctx)                               \
> >> +{                                                                       \
> >> +        _Pragma("GCC diagnostic push")                                          \
> >> +        _Pragma("GCC diagnostic ignored \"-Wint-conversion\"")                  \
> >> +        return ____##name(___bpf_usdt_args(args));                      \
> >> +        _Pragma("GCC diagnostic pop")                                           \
> >> +}                                                                       \
> >> +static __attribute__((always_inline)) typeof(name(0))                           \
> >> +____##name(struct pt_regs *ctx, ##args)
> >> +
> >> +#endif /* __USDT_BPF_H__ */