Re: [PATCH bpf-next 7/7] bpf: unify PTR_TO_MAP_{KEY,VALUE} with default case in regsafe()

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Tue, Dec 27, 2022 at 6:00 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Thu, Dec 22, 2022 at 09:49:21PM -0800, Andrii Nakryiko wrote:
> > Make default case in regsafe() safer. Instead of doing byte-by-byte
>
> I love the patches 1-6, but this one is not making it safer.
> It looks to be going less safe route.
> More below.
>
> > comparison, take into account ID remapping and also range and var_off
>
> ID remapping is handled by the patch 6 in regs_exact().
> This patch adds range and var_off check as default.
> Which might not be correct in the future.
>
> > checks. For most of registers range and var_off will be zero (not set),
> > which doesn't matter. For some, like PTR_TO_MAP_{KEY,VALUE}, this
> > generalized logic is exactly matching what regsafe() was doing as
> > a special case. But in any case, if register has id and/or ref_obj_id
> > set, check it using check_ids() logic, taking into account idmap.
>
> That was already done in patch 6. So the commit log is misleading.
> It's arguing that it's a benefit of this change while it was in the previous patch.

True, I think I had regs_exact() and regs_equals() change in one
commit and split it at the last minute before submitting (I felt like
patch #7 will be controversial ;) ), should have proofread messages
more carefully. Sorry about that.

>
> > With these changes, default case should be handling most registers more
> > correctly, and even for future register would be a good default. For some
> > special cases, like PTR_TO_PACKET, one would still need to implement extra
> > checks (like special handling of reg->range) to get better state
> > equivalence, but default logic shouldn't be wrong.
>
> PTR_TO_BTF_ID with var_off would be a counter example where
> such default of comparing ranges and var_off would lead to issues.
> Currently PTR_TO_BTF_ID doesn't allow var_off, but folks requested this support.
> The range_within() logic is safe only for types like PTR_TO_MAP_KEY/VALUE
> that start from zero and have uniform typeless blob of bytes.
> PTR_TO_BTF_ID with var_off would be wrong to do with just range_within().

I'm trying to understand this future problem. I think this is the same
issue that Kumar was trying to fix before, but when I asked for more
specifics I didn't really get good answer of when this combined
var_off and range_within() would be incorrect.

Do you mind showing (even if hypothetically) an example when
var_off+range_within() won't work? I'm trying to understand this. We
should either document why this is not safe, in general, or come to
conclusion that it is safe. It's second time this comes up, so let's
spend a bit of time getting to the bottom of this?

>
> SCALARS and PTR_TO_BTF_ID will likely dominate future bpf progs.
> Keeping default as regs_exact (that does ID match) is safer default.

It's fine, though the point of this patch set was patch #7, enabling
logic similar to PTR_TO_MAP_VALUE for PTR_TO_MEM and PTR_TO_BUF. I can
send specific fixes for that, no problem. But as I said above, I'm
really curious to understand what kind of situations will lead to
unsafety if we do var_off+range_within checks.

>
> Having said all that the focus on safety should be balanced with focus on performance
> of the verifier itself.
> The regsafe is the hottest function.
> That first memcmp used to be the hottest part of the whole verifier.

yeah, and it was done unconditionally even if not needed, which was
kind of weird when I started looking at this. Probably some
refactoring leftover.

> I suspect this refactoring won't change the perf profile, but we can optimize it.
> Assuming that SCALAR, PTR_TO_BTF_ID and PTR_TO_MAP will be the majority of types
> we can special case them and refactor comparison to only things
> that matter to these types. var_off and min/max_value are the biggest part
> of bpf_reg_state. They should be zero for PTR_TO_BTF_ID, but we already check
> that in other parts of the verifier. There is no need to compare zeros again
> in the hottest regsafe() function.
> Same thing for SCALAR. Doing regs_exact() with big memcmp and then finer range_within()
> on the same bytes is probably wasteful and can be optimized.
> We might consider reshuffling bpf_reg_state fields again depending on cache line usage.
> I suspect doing "smart" reg comparison we will be able to significantly
> improve verification speed. Please consider for a follow up.

I agree. Perf wasn't the point for me (this is a preliminary for
iterator stuff to improve state equivalence checks), so I didn't want
to spend extra time on this (especially that benchmarking this
properly is time consuming, as benchmarking under QEMU isn't
representative (from me experiences with BPF ringbuf benchmarking).
But I'll keep it on TODO list, either for me or anyone interested in
contributing.

>
> I've applied the first 6 patches.

Cool, thanks, less patches to carry around. If you don't mind, let's
look at this var_off concern in details. I can send
PTR_TO_MEM-specific follow up fix, but if we can convince ourselves
that generic logic is safe and future-proof, I'd rather do a generic
change.