Re: [PATCH v2 bpf-next 0/6] BPF verifier rotating log

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Tue, Mar 28, 2023 at 4:56 PM Andrii Nakryiko <andrii@kernel.org> wrote:
>
> This patch set changes BPF verifier log behavior to behave as a rotating log,
> by default. If user-supplied log buffer is big enough to contain entire
> verifier log output, there is no effective difference. But where previously
> user supplied too small log buffer and would get -ENOSPC error result and the
> beginning part of the verifier log, now there will be no error and user will
> get ending part of verifier log filling up user-supplied log buffer.  Which
> is, in absolute majority of cases, is exactly what's useful, relevant, and
> what users want and need, as the ending of the verifier log is containing
> details of verifier failure and relevant state that got us to that failure. So
> this rotating mode is made default, but for some niche advanced debugging
> scenarios it's possible to request old behavior by specifying additional
> BPF_LOG_FIXED (8) flag.
>
> This patch set adjusts libbpf to allow specifying flags beyond 1 | 2 | 4. We
> also add --log-size and --log-fixed options to veristat to be able to both
> test this functionality manually, but also to be used in various debugging
> scenarios. We also add selftests that tries many variants of log buffer size
> to stress-test correctness of internal verifier log bookkeeping code.
>
> v1->v2:
>   - return -ENOSPC even in rotating log mode for preserving backwards
>     compatibility (Lorenz);

I haven't implemented the feature we discussed, where the
BPF_PROG_LOAD (and BPF_BTF_LOAD) command will return back the full
size of the buffer that's necessary to contain the complete log
buffer. I'm building it on top of this patch set and would like to
send it separately as a follow up, as it touches UAPI some more, and I
feel like we'll have few revisions just for this. So I didn't want to
delay these changes. Plus, I think to add this even for BPF_LOG_FIXED
mode, so it's provided consistently. So I need a bit more time to
implement this. Hopefully this version will work for everyone and can
go in sooner.

>
> Andrii Nakryiko (6):
>   bpf: split off basic BPF verifier log into separate file
>   bpf: remove minimum size restrictions on verifier log buffer
>   bpf: switch BPF verifier log to be a rotating log by default
>   libbpf: don't enforce verifier log levels on libbpf side
>   selftests/bpf: add more veristat control over verifier log options
>   selftests/bpf: add fixed vs rotating verifier log tests
>
>  include/linux/bpf_verifier.h                  |  48 ++--
>  kernel/bpf/Makefile                           |   3 +-
>  kernel/bpf/btf.c                              |   3 +-
>  kernel/bpf/log.c                              | 262 ++++++++++++++++++
>  kernel/bpf/verifier.c                         |  88 +-----
>  tools/lib/bpf/bpf.c                           |   2 -
>  .../selftests/bpf/prog_tests/log_fixup.c      |   1 +
>  .../selftests/bpf/prog_tests/verifier_log.c   | 164 +++++++++++
>  tools/testing/selftests/bpf/veristat.c        |  42 ++-
>  9 files changed, 506 insertions(+), 107 deletions(-)
>  create mode 100644 kernel/bpf/log.c
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/verifier_log.c
>
> --
> 2.34.1
>