Re: [PATCH v2 bpf-next] libbpf: usdt arm arg parsing support

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Sat, Mar 4, 2023 at 11:18 AM Puranjay Mohan <puranjay12@gmail.com> wrote:
>
> Hi Andrii,
>
> On Sat, Mar 4, 2023 at 9:52 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Fri, Mar 3, 2023 at 12:37 AM Puranjay Mohan <puranjay12@gmail.com> wrote:
> > >
> > > Parsing of USDT arguments is architecture-specific; on arm it is
> > > relatively easy since registers used are r[0-10], fp, ip, sp, lr,
> > > pc. Format is slightly different compared to aarch64; forms are
> > >
> > > - "size @ [ reg, #offset ]" for dereferences, for example
> > >   "-8 @ [ sp, #76 ]" ; " -4 @ [ sp ]"
> > > - "size @ reg" for register values; for example
> > >   "-4@r0"
> > > - "size @ #value" for raw values; for example
> > >   "-8@#1"
> > >
> > > Add support for parsing USDT arguments for ARM architecture.
> > >
> > > Signed-off-by: Puranjay Mohan <puranjay12@gmail.com>
> > > ---
> >
> > You don't mention that in the commit message, but how did you test
> > these changes?
>
> I use the  QEMU's virt[1] board with cortex-a15 CPU. I take the
> libbpf-bootstrap's usdt example[2] and
> modify it to attach it to my custom program with
> DTRACE_PROBE1/2/3/4... probes to test different combinations.
>

Nice, please mention that in the commit message. We don't have 32-bit
arm tests in CI, so explicitly mentioning manual testing is good to
have.

> >
> > > Changes in V1[1] to V2
> > > - Resending as V1 shows up as Superseded in patchwork.
> > >
> > > [1] https://patchwork.kernel.org/project/netdevbpf/patch/20230220212741.13515-1-puranjay12@gmail.com/
> > > ---
> > >  tools/lib/bpf/usdt.c | 82 ++++++++++++++++++++++++++++++++++++++++++++
> > >  1 file changed, 82 insertions(+)
> > >
> > > diff --git a/tools/lib/bpf/usdt.c b/tools/lib/bpf/usdt.c
> > > index 75b411fc2c77..ef097b882a4d 100644
> > > --- a/tools/lib/bpf/usdt.c
> > > +++ b/tools/lib/bpf/usdt.c
> > > @@ -1505,6 +1505,88 @@ static int parse_usdt_arg(const char *arg_str, int arg_num, struct usdt_arg_spec
> > >         return len;
> > >  }
> > >
> > > +#elif defined(__arm__)
> > > +
> > > +static int calc_pt_regs_off(const char *reg_name)
> > > +{
> > > +       int reg_num;
> > > +
> > > +       if (sscanf(reg_name, "r%d", &reg_num) == 1) {
> > > +               if (reg_num >= 0 && reg_num <= 10)
> > > +                       return offsetof(struct pt_regs, uregs[reg_num]);
> > > +       } else if (strcmp(reg_name, "fp") == 0) {
> > > +               return offsetof(struct pt_regs, ARM_fp);
> > > +       } else if (strcmp(reg_name, "ip") == 0) {
> > > +               return offsetof(struct pt_regs, ARM_ip);
> > > +       } else if (strcmp(reg_name, "sp") == 0) {
> > > +               return offsetof(struct pt_regs, ARM_sp);
> > > +       } else if (strcmp(reg_name, "lr") == 0) {
> > > +               return offsetof(struct pt_regs, ARM_lr);
> > > +       } else if (strcmp(reg_name, "pc") == 0) {
> > > +               return offsetof(struct pt_regs, ARM_pc);
> > > +       }
> > > +       pr_warn("usdt: unrecognized register '%s'\n", reg_name);
> > > +       return -ENOENT;
> > > +}
> > > +
> >
> > let's use a more tabular approach, just like, say, riscv does?
>
> As R0-R10 directly map to uregs[0->10], I used sscanf for that, and as
> there are only five named registers
> (FP, IP, SP, LR, PC), I thought that using if-else would be good
> enough. But I can change it if it is necessary.
>

let's go with a table approach, it's consistent with riscv, and I find
it easier to follow (even if it's a bit repetitive)

> >
> > > +static int parse_usdt_arg(const char *arg_str, int arg_num, struct usdt_arg_spec *arg)
> > > +{
> > > +       char reg_name[16];
> > > +       int arg_sz, len, reg_off;
> > > +       long off;
> > > +
> > > +       if (sscanf(arg_str, " %d @ \[ %15[a-z0-9], #%ld ] %n", &arg_sz, reg_name,
> > > +                                                               &off, &len) == 3) {
> >
> > if long function call is wrapped, argument on new line should be
> > aligned with the first argument on previous line. I'd suggest wrapping
> > right after format string, and start with &arg_sz aligned with arg_str
>
> Will change it in the next version.
>
> >
> > > +               /* Memory dereference case, e.g., -4@[fp, #96] */
> > > +               arg->arg_type = USDT_ARG_REG_DEREF;
> > > +               arg->val_off = off;
> > > +               reg_off = calc_pt_regs_off(reg_name);
> > > +               if (reg_off < 0)
> > > +                       return reg_off;
> > > +               arg->reg_off = reg_off;
> > > +       } else if (sscanf(arg_str, " %d @ \[ %15[a-z0-9] ] %n", &arg_sz, reg_name, &len) == 2) {
> > > +               /* Memory dereference case, e.g., -4@[sp] */
> > > +               arg->arg_type = USDT_ARG_REG_DEREF;
> > > +               arg->val_off = 0;
> > > +               reg_off = calc_pt_regs_off(reg_name);
> > > +               if (reg_off < 0)
> > > +                       return reg_off;
> > > +               arg->reg_off = reg_off;
> > > +       } else if (sscanf(arg_str, " %d @ #%ld %n", &arg_sz, &off, &len) == 2) {
> >
> > is the '#<num>' value always in decimal or it could be hex sometimes?
>
> I have found all these combinations using trying out different things
> in my test program as I couldn't
> find documentation about this. I could not generate a combination
> where a hex value is returned here.

ok, that's fine, let's stick to decimal for now

>
> >
> > > +               /* Constant value case, e.g., 4@#5 */
> > > +               arg->arg_type = USDT_ARG_CONST;
> > > +               arg->val_off = off;
> > > +               arg->reg_off = 0;
> > > +       } else if (sscanf(arg_str, " %d @ %15[a-z0-9] %n", &arg_sz, reg_name, &len) == 2) {
> > > +               /* Register read case, e.g., -8@r4 */
> > > +               arg->arg_type = USDT_ARG_REG;
> > > +               arg->val_off = 0;
> > > +               reg_off = calc_pt_regs_off(reg_name);
> > > +               if (reg_off < 0)
> > > +                       return reg_off;
> > > +               arg->reg_off = reg_off;
> > > +       } else {
> > > +               pr_warn("usdt: unrecognized arg #%d spec '%s'\n", arg_num, arg_str);
> > > +               return -EINVAL;
> > > +       }
> > > +
> > > +       arg->arg_signed = arg_sz < 0;
> > > +       if (arg_sz < 0)
> > > +               arg_sz = -arg_sz;
> > > +
> > > +       switch (arg_sz) {
> > > +       case 1: case 2: case 4: case 8:
> > > +               arg->arg_bitshift = 64 - arg_sz * 8;
> > > +               break;
> > > +       default:
> > > +               pr_warn("usdt: unsupported arg #%d (spec '%s') size: %d\n",
> > > +                       arg_num, arg_str, arg_sz);
> > > +               return -EINVAL;
> > > +       }
> >
> > This part is repeated verbatim for each architecture, perhaps it's
> > better to do this post-processing and checking in parse_usdt_spec().
> > Would you mind adding another patch to your series that refactors
> > parse_usdt_arg() implementation to fill out struct usdt_arg_spec and
> > return arg_sz as out parameter. And then parse_usdt_spec() will check
> > arg_sz, set arg_signed and arg_bitshift parts?
>
> Sure, I will refactor this in the first patch and then add ARM support
> in the second patch.
>

sounds good, thanks!

> >
> > > +
> > > +       return len;
> > > +}
> > > +
> > >  #else
> > >
> > >  static int parse_usdt_arg(const char *arg_str, int arg_num, struct usdt_arg_spec *arg)
> > > --
> > > 2.39.1
> > >
>
> Thanks,
> Puranjay
>
> [1] https://www.qemu.org/docs/master/system/arm/virt.html
> [2] https://github.com/libbpf/libbpf-bootstrap/blob/master/examples/c/usdt.bpf.c