pull-request: bpf-next 2023-09-16

pull-request: bpf-next 2023-09-16

[Alexei Starovoitov]

Hi David, hi Jakub, hi Paolo, hi Eric,

The following pull-request contains BPF updates for your *net-next* tree.

We've added 73 non-merge commits during the last 9 day(s) which contain
a total of 79 files changed, 5275 insertions(+), 600 deletions(-).

The main changes are:

1) Basic BTF validation in libbpf, from Andrii Nakryiko.

2) bpf_assert(), bpf_throw(), exceptions in bpf progs, from Kumar Kartikeya Dwivedi.

3) next_thread cleanups, from Oleg Nesterov.

4) Add mcpu=v4 support to arm32, from Puranjay Mohan.

5) Add support for __percpu pointers in bpf progs, from Yonghong Song.

6) Fix bpf tailcall interaction with bpf trampoline, from Leon Hwang.

7) Raise irq_work in bpf_mem_alloc while irqs are disabled to improve refill probabablity, from Hou Tao.

Please consider pulling these changes from:

  git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git 

Thanks a lot!

Also thanks to reporters, reviewers and testers of commits in this pull-request:

Alan Maguire, Andrey Konovalov, Dave Marchevsky, "Eric W. Biederman", 
Jiri Olsa, Maciej Fijalkowski, Quentin Monnet, Russell King (Oracle), 
Song Liu, Stanislav Fomichev, Yonghong Song

----------------------------------------------------------------

The following changes since commit 73be7fb14e83d24383f840a22f24d3ed222ca319:

  Merge tag 'net-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net (2023-09-07 18:33:07 -0700)

are available in the Git repository at:

  https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git 

for you to fetch changes up to ec6f1b4db95b7eedb3fe85f4f14e08fa0e9281c3:

  Merge branch 'exceptions-1-2' (2023-09-16 09:36:44 -0700)

----------------------------------------------------------------
Alexei Starovoitov (7):
      Merge branch 'bpf-add-support-for-local-percpu-kptr'
      Merge branch 'bpf-enable-irq-after-irq_work_raise-completes'
      Merge branch 'bpf-task_group_seq_get_next-misc-cleanups'
      Merge branch 'bpf-x64-fix-tailcall-infinite-loop'
      Merge branch 'seltests-xsk-various-improvements-to-xskxceiver'
      Merge branch 'arm32-bpf-add-support-for-cpuv4-insns'
      Merge branch 'exceptions-1-2'

Andrii Nakryiko (2):
      libbpf: Add basic BTF sanity validation
      Merge branch 'selftests/bpf: Optimize kallsyms cache'

Artem Savkov (1):
      selftests/bpf: Skip module_fentry_shadow test when bpf_testmod is not available

Denys Zagorui (1):
      bpftool: Fix -Wcast-qual warning

Hou Tao (3):
      bpf: Enable IRQ after irq_work_raise() completes in unit_alloc()
      bpf: Enable IRQ after irq_work_raise() completes in unit_free{_rcu}()
      selftests/bpf: Test preemption between bpf_obj_new() and bpf_obj_drop()

Kumar Kartikeya Dwivedi (17):
      bpf: Use bpf_is_subprog to check for subprogs
      arch/x86: Implement arch_bpf_stack_walk
      bpf: Implement support for adding hidden subprogs
      bpf: Implement BPF exceptions
      bpf: Refactor check_btf_func and split into two phases
      bpf: Add support for custom exception callbacks
      bpf: Perform CFG walk for exception callback
      bpf: Treat first argument as return value for bpf_throw
      mm: kasan: Declare kasan_unpoison_task_stack_below in kasan.h
      bpf: Prevent KASAN false positive with bpf_throw
      bpf: Detect IP == ksym.end as part of BPF program
      bpf: Disallow fentry/fexit/freplace for exception callbacks
      bpf: Fix kfunc callback register type handling
      libbpf: Refactor bpf_object__reloc_code
      libbpf: Add support for custom exception callbacks
      selftests/bpf: Add BPF assertion macros
      selftests/bpf: Add tests for BPF exceptions

Larysa Zaremba (1):
      bpf: Allow to use kfunc XDP hints and frags together

Leon Hwang (4):
      selftests/bpf: Correct map_fd to data_fd in tailcalls
      bpf, x64: Comment tail_call_cnt initialisation
      bpf, x64: Fix tailcall infinite loop
      selftests/bpf: Add testcases for tailcall infinite loop fixing

Magnus Karlsson (10):
      selftests/xsk: print per packet info in verbose mode
      selftests/xsk: add timeout for Tx thread
      selftests/xsk: add option to only run tests in a single mode
      selftests/xsk: move all tests to separate functions
      selftests/xsk: declare test names in struct
      selftests/xsk: add option that lists all tests
      selftests/xsk: add option to run single test
      selftests/xsk: use ksft_print_msg uniformly
      selftests/xsk: fail single test instead of all tests
      selftests/xsk: display command line options with -h

Martin KaFai Lau (1):
      Merge branch 'bpf: expose information about netdev xdp-metadata kfunc support'

Oleg Nesterov (5):
      bpf: task_group_seq_get_next: cleanup the usage of next_thread()
      bpf: task_group_seq_get_next: cleanup the usage of get/put_task_struct
      bpf: task_group_seq_get_next: fix the skip_if_dup_files check
      bpf: task_group_seq_get_next: kill next_task
      bpf: task_group_seq_get_next: simplify the "next tid" logic

Puranjay Mohan (9):
      arm32, bpf: add support for 32-bit offset jmp instruction
      arm32, bpf: add support for sign-extension load instruction
      arm32, bpf: add support for sign-extension mov instruction
      arm32, bpf: add support for unconditional bswap instruction
      arm32, bpf: add support for 32-bit signed division
      arm32, bpf: add support for 64 bit division instruction
      selftest, bpf: enable cpu v4 tests for arm32
      bpf/tests: add tests for cpuv4 instructions
      MAINTAINERS: Add myself for ARM32 BPF JIT maintainer.

Quan Tian (1):
      docs/bpf: update out-of-date doc in BPF flow dissector

Rong Tao (2):
      selftests/bpf: trace_helpers.c: Optimize kallsyms cache
      selftests/bpf: trace_helpers.c: Add a global ksyms initialization mutex

Song Liu (1):
      bpf: Charge modmem for struct_ops trampoline

Stanislav Fomichev (3):
      bpf: make it easier to add new metadata kfunc
      bpf: expose information about supported xdp metadata kfunc
      tools: ynl: extend netdev sample to dump xdp-rx-metadata-features

Tirthendu Sarkar (1):
      xsk: add multi-buffer support for sockets sharing umem

Yonghong Song (13):
      bpf: Add support for non-fix-size percpu mem allocation
      bpf: Add BPF_KPTR_PERCPU as a field type
      bpf: Add alloc/xchg/direct_access support for local percpu kptr
      bpf: Add bpf_this_cpu_ptr/bpf_per_cpu_ptr support for allocated percpu obj
      selftests/bpf: Update error message in negative linked_list test
      libbpf: Add __percpu_kptr macro definition
      selftests/bpf: Add bpf_percpu_obj_{new,drop}() macro in bpf_experimental.h
      selftests/bpf: Add tests for array map with local percpu kptr
      bpf: Mark OBJ_RELEASE argument as MEM_RCU when possible
      selftests/bpf: Remove unnecessary direct read of local percpu kptr
      selftests/bpf: Add tests for cgrp_local_storage with local percpu kptr
      selftests/bpf: Add some negative tests
      bpf: Mark BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE deprecated

 Documentation/bpf/prog_flow_dissector.rst          |   2 +-
 Documentation/netlink/specs/netdev.yaml            |  21 +
 Documentation/networking/xdp-rx-metadata.rst       |   7 +
 MAINTAINERS                                        |   5 +-
 arch/arm/net/bpf_jit_32.c                          | 280 ++++++++-
 arch/arm/net/bpf_jit_32.h                          |   4 +
 arch/arm64/net/bpf_jit_comp.c                      |   2 +-
 arch/s390/net/bpf_jit_comp.c                       |   2 +-
 arch/x86/net/bpf_jit_comp.c                        | 149 ++++-
 include/linux/bpf.h                                |  40 +-
 include/linux/bpf_verifier.h                       |   9 +-
 include/linux/filter.h                             |  58 +-
 include/linux/kasan.h                              |   2 +
 include/net/xdp.h                                  |  19 +-
 include/net/xdp_sock.h                             |   2 +
 include/uapi/linux/bpf.h                           |   9 +-
 include/uapi/linux/netdev.h                        |  16 +
 kernel/bpf/bpf_struct_ops.c                        |  26 +-
 kernel/bpf/btf.c                                   |  34 +-
 kernel/bpf/core.c                                  |  37 +-
 kernel/bpf/helpers.c                               |  61 ++
 kernel/bpf/memalloc.c                              |  30 +-
 kernel/bpf/offload.c                               |  18 +-
 kernel/bpf/syscall.c                               |   6 +-
 kernel/bpf/task_iter.c                             |  40 +-
 kernel/bpf/trampoline.c                            |   4 +-
 kernel/bpf/verifier.c                              | 645 ++++++++++++++++++---
 lib/test_bpf.c                                     | 371 ++++++++++++
 mm/kasan/kasan.h                                   |   1 -
 net/core/netdev-genl.c                             |  12 +-
 net/core/xdp.c                                     |   4 +-
 net/xdp/xsk.c                                      |   2 +-
 net/xdp/xsk_buff_pool.c                            |   3 +
 samples/bpf/Makefile                               |   4 +
 tools/bpf/bpftool/gen.c                            |   2 +-
 tools/include/uapi/linux/bpf.h                     |   9 +-
 tools/include/uapi/linux/netdev.h                  |  16 +
 tools/lib/bpf/bpf_helpers.h                        |   1 +
 tools/lib/bpf/btf.c                                | 160 +++++
 tools/lib/bpf/libbpf.c                             | 166 +++++-
 tools/net/ynl/generated/netdev-user.c              |  19 +
 tools/net/ynl/generated/netdev-user.h              |   3 +
 tools/net/ynl/samples/Makefile                     |   2 +-
 tools/net/ynl/samples/netdev.c                     |   8 +-
 tools/testing/selftests/bpf/DENYLIST.aarch64       |   1 +
 tools/testing/selftests/bpf/DENYLIST.s390x         |   1 +
 tools/testing/selftests/bpf/bpf_experimental.h     | 319 ++++++++++
 tools/testing/selftests/bpf/prog_tests/btf.c       |   4 +-
 .../testing/selftests/bpf/prog_tests/exceptions.c  | 408 +++++++++++++
 .../selftests/bpf/prog_tests/fill_link_info.c      |   2 +-
 .../bpf/prog_tests/kprobe_multi_testmod_test.c     |  20 +-
 .../testing/selftests/bpf/prog_tests/libbpf_str.c  |   6 +-
 .../testing/selftests/bpf/prog_tests/linked_list.c |   4 +-
 .../bpf/prog_tests/module_fentry_shadow.c          |   5 +
 .../selftests/bpf/prog_tests/percpu_alloc.c        | 125 ++++
 .../selftests/bpf/prog_tests/preempted_bpf_ma_op.c |  89 +++
 tools/testing/selftests/bpf/prog_tests/tailcalls.c | 269 ++++++++-
 tools/testing/selftests/bpf/progs/exceptions.c     | 368 ++++++++++++
 .../selftests/bpf/progs/exceptions_assert.c        | 135 +++++
 tools/testing/selftests/bpf/progs/exceptions_ext.c |  72 +++
 .../testing/selftests/bpf/progs/exceptions_fail.c  | 347 +++++++++++
 .../selftests/bpf/progs/percpu_alloc_array.c       | 183 ++++++
 .../bpf/progs/percpu_alloc_cgrp_local_storage.c    | 105 ++++
 .../selftests/bpf/progs/percpu_alloc_fail.c        | 164 ++++++
 .../selftests/bpf/progs/preempted_bpf_ma_op.c      | 106 ++++
 .../selftests/bpf/progs/tailcall_bpf2bpf_fentry.c  |  18 +
 .../selftests/bpf/progs/tailcall_bpf2bpf_fexit.c   |  18 +
 tools/testing/selftests/bpf/progs/verifier_bswap.c |   3 +-
 tools/testing/selftests/bpf/progs/verifier_gotol.c |   3 +-
 tools/testing/selftests/bpf/progs/verifier_ldsx.c  |   3 +-
 tools/testing/selftests/bpf/progs/verifier_movsx.c |   3 +-
 tools/testing/selftests/bpf/progs/verifier_sdiv.c  |   3 +-
 .../selftests/bpf/test_bpftool_synctypes.py        |   9 +
 tools/testing/selftests/bpf/test_xsk.sh            |  40 +-
 tools/testing/selftests/bpf/trace_helpers.c        | 134 +++--
 tools/testing/selftests/bpf/trace_helpers.h        |   8 +-
 tools/testing/selftests/bpf/xsk_prereqs.sh         |  10 +-
 tools/testing/selftests/bpf/xskxceiver.c           | 535 ++++++++++-------
 tools/testing/selftests/bpf/xskxceiver.h           |  44 +-
 79 files changed, 5275 insertions(+), 600 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/exceptions.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/percpu_alloc.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/preempted_bpf_ma_op.c
 create mode 100644 tools/testing/selftests/bpf/progs/exceptions.c
 create mode 100644 tools/testing/selftests/bpf/progs/exceptions_assert.c
 create mode 100644 tools/testing/selftests/bpf/progs/exceptions_ext.c
 create mode 100644 tools/testing/selftests/bpf/progs/exceptions_fail.c
 create mode 100644 tools/testing/selftests/bpf/progs/percpu_alloc_array.c
 create mode 100644 tools/testing/selftests/bpf/progs/percpu_alloc_cgrp_local_storage.c
 create mode 100644 tools/testing/selftests/bpf/progs/percpu_alloc_fail.c
 create mode 100644 tools/testing/selftests/bpf/progs/preempted_bpf_ma_op.c
 create mode 100644 tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_fentry.c
 create mode 100644 tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_fexit.c

Re: pull-request: bpf-next 2023-09-16

[Eric Dumazet]

On Sat, Sep 16, 2023 at 6:59 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> Hi David, hi Jakub, hi Paolo, hi Eric,
>
> The following pull-request contains BPF updates for your *net-next* tree.
>
> We've added 73 non-merge commits during the last 9 day(s) which contain
> a total of 79 files changed, 5275 insertions(+), 600 deletions(-).
>
> The main changes are:
>
> 1) Basic BTF validation in libbpf, from Andrii Nakryiko.
>
> 2) bpf_assert(), bpf_throw(), exceptions in bpf progs, from Kumar Kartikeya Dwivedi.
>
> 3) next_thread cleanups, from Oleg Nesterov.
>
> 4) Add mcpu=v4 support to arm32, from Puranjay Mohan.
>
> 5) Add support for __percpu pointers in bpf progs, from Yonghong Song.
>
> 6) Fix bpf tailcall interaction with bpf trampoline, from Leon Hwang.
>
> 7) Raise irq_work in bpf_mem_alloc while irqs are disabled to improve refill probabablity, from Hou Tao.
>
> Please consider pulling these changes from:
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
>

This might have been raised already, but bpf on x86 now depends on
CONFIG_UNWINDER_ORC ?

$ grep CONFIG_UNWINDER_ORC .config
# CONFIG_UNWINDER_ORC is not set

$ make ...
arch/x86/net/bpf_jit_comp.c:3022:58: error: no member named 'sp' in
'struct unwind_state'
                if (!addr || !consume_fn(cookie, (u64)addr,
(u64)state.sp, (u64)state.bp))
                                                                 ~~~~~ ^
1 error generated.

Re: pull-request: bpf-next 2023-09-16

[Alexei Starovoitov]

On Mon, Sep 18, 2023 at 6:25 AM Eric Dumazet <edumazet@google.com> wrote:
>
> On Sat, Sep 16, 2023 at 6:59 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > Hi David, hi Jakub, hi Paolo, hi Eric,
> >
> > The following pull-request contains BPF updates for your *net-next* tree.
> >
> > We've added 73 non-merge commits during the last 9 day(s) which contain
> > a total of 79 files changed, 5275 insertions(+), 600 deletions(-).
> >
> > The main changes are:
> >
> > 1) Basic BTF validation in libbpf, from Andrii Nakryiko.
> >
> > 2) bpf_assert(), bpf_throw(), exceptions in bpf progs, from Kumar Kartikeya Dwivedi.
> >
> > 3) next_thread cleanups, from Oleg Nesterov.
> >
> > 4) Add mcpu=v4 support to arm32, from Puranjay Mohan.
> >
> > 5) Add support for __percpu pointers in bpf progs, from Yonghong Song.
> >
> > 6) Fix bpf tailcall interaction with bpf trampoline, from Leon Hwang.
> >
> > 7) Raise irq_work in bpf_mem_alloc while irqs are disabled to improve refill probabablity, from Hou Tao.
> >
> > Please consider pulling these changes from:
> >
> >   git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
> >
>
> This might have been raised already, but bpf on x86 now depends on
> CONFIG_UNWINDER_ORC ?
>
> $ grep CONFIG_UNWINDER_ORC .config
> # CONFIG_UNWINDER_ORC is not set
>
> $ make ...
> arch/x86/net/bpf_jit_comp.c:3022:58: error: no member named 'sp' in
> 'struct unwind_state'
>                 if (!addr || !consume_fn(cookie, (u64)addr,
> (u64)state.sp, (u64)state.bp))
>                                                                  ~~~~~ ^
> 1 error generated.

Kumar,
can probably explain better,
but no the bpf as whole doesn't depend.
One feature needs either ORC or frame unwinder.
It won't work with unwinder_guess.
The build error is a separate issue.
It hasn't been reported before.

Re: pull-request: bpf-next 2023-09-16

[Eric Dumazet]

On Mon, Sep 18, 2023 at 3:41 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Mon, Sep 18, 2023 at 6:25 AM Eric Dumazet <edumazet@google.com> wrote:
> >
> > On Sat, Sep 16, 2023 at 6:59 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > Hi David, hi Jakub, hi Paolo, hi Eric,
> > >
> > > The following pull-request contains BPF updates for your *net-next* tree.
> > >
> > > We've added 73 non-merge commits during the last 9 day(s) which contain
> > > a total of 79 files changed, 5275 insertions(+), 600 deletions(-).
> > >
> > > The main changes are:
> > >
> > > 1) Basic BTF validation in libbpf, from Andrii Nakryiko.
> > >
> > > 2) bpf_assert(), bpf_throw(), exceptions in bpf progs, from Kumar Kartikeya Dwivedi.
> > >
> > > 3) next_thread cleanups, from Oleg Nesterov.
> > >
> > > 4) Add mcpu=v4 support to arm32, from Puranjay Mohan.
> > >
> > > 5) Add support for __percpu pointers in bpf progs, from Yonghong Song.
> > >
> > > 6) Fix bpf tailcall interaction with bpf trampoline, from Leon Hwang.
> > >
> > > 7) Raise irq_work in bpf_mem_alloc while irqs are disabled to improve refill probabablity, from Hou Tao.
> > >
> > > Please consider pulling these changes from:
> > >
> > >   git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
> > >
> >
> > This might have been raised already, but bpf on x86 now depends on
> > CONFIG_UNWINDER_ORC ?
> >
> > $ grep CONFIG_UNWINDER_ORC .config
> > # CONFIG_UNWINDER_ORC is not set
> >
> > $ make ...
> > arch/x86/net/bpf_jit_comp.c:3022:58: error: no member named 'sp' in
> > 'struct unwind_state'
> >                 if (!addr || !consume_fn(cookie, (u64)addr,
> > (u64)state.sp, (u64)state.bp))
> >                                                                  ~~~~~ ^
> > 1 error generated.
>
> Kumar,
> can probably explain better,
> but no the bpf as whole doesn't depend.
> One feature needs either ORC or frame unwinder.
> It won't work with unwinder_guess.
> The build error is a separate issue.
> It hasn't been reported before.

In my builds, I do have CONFIG_UNWINDER_FRAME_POINTER=y

$ grep UNWIND .config
# CONFIG_UNWINDER_ORC is not set
CONFIG_UNWINDER_FRAME_POINTER=y


I note state.sp is only available to CONFIG_UNWINDER_ORC

arch/x86/include/asm/unwind.h

#if defined(CONFIG_UNWINDER_ORC)
    bool signal, full_regs;
    unsigned long sp, bp, ip;
    struct pt_regs *regs, *prev_regs;
#elif defined(CONFIG_UNWINDER_FRAME_POINTER)
   bool got_irq;
   unsigned long *bp, *orig_sp, ip;   // this is orig_sp , not sp.
   ...

Re: pull-request: bpf-next 2023-09-16

[Alexei Starovoitov]

On Mon, Sep 18, 2023 at 6:40 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Mon, Sep 18, 2023 at 6:25 AM Eric Dumazet <edumazet@google.com> wrote:
> >
> > On Sat, Sep 16, 2023 at 6:59 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > Hi David, hi Jakub, hi Paolo, hi Eric,
> > >
> > > The following pull-request contains BPF updates for your *net-next* tree.
> > >
> > > We've added 73 non-merge commits during the last 9 day(s) which contain
> > > a total of 79 files changed, 5275 insertions(+), 600 deletions(-).
> > >
> > > The main changes are:
> > >
> > > 1) Basic BTF validation in libbpf, from Andrii Nakryiko.
> > >
> > > 2) bpf_assert(), bpf_throw(), exceptions in bpf progs, from Kumar Kartikeya Dwivedi.
> > >
> > > 3) next_thread cleanups, from Oleg Nesterov.
> > >
> > > 4) Add mcpu=v4 support to arm32, from Puranjay Mohan.
> > >
> > > 5) Add support for __percpu pointers in bpf progs, from Yonghong Song.
> > >
> > > 6) Fix bpf tailcall interaction with bpf trampoline, from Leon Hwang.
> > >
> > > 7) Raise irq_work in bpf_mem_alloc while irqs are disabled to improve refill probabablity, from Hou Tao.
> > >
> > > Please consider pulling these changes from:
> > >
> > >   git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
> > >
> >
> > This might have been raised already, but bpf on x86 now depends on
> > CONFIG_UNWINDER_ORC ?
> >
> > $ grep CONFIG_UNWINDER_ORC .config
> > # CONFIG_UNWINDER_ORC is not set
> >
> > $ make ...
> > arch/x86/net/bpf_jit_comp.c:3022:58: error: no member named 'sp' in
> > 'struct unwind_state'
> >                 if (!addr || !consume_fn(cookie, (u64)addr,
> > (u64)state.sp, (u64)state.bp))
> >                                                                  ~~~~~ ^
> > 1 error generated.
>
> Kumar,
> can probably explain better,
> but no the bpf as whole doesn't depend.
> One feature needs either ORC or frame unwinder.
> It won't work with unwinder_guess.
> The build error is a separate issue.
> It hasn't been reported before.

I see the error with CONFIG_UNWINDER_FRAME_POINTER.
That's unexpected.
Kumar,
looks like this config path wasn't tested.

Eric, Paolo, Dave, Kuba,
please ignore this PR.
We need to fix this first.

Re: pull-request: bpf-next 2023-09-16

[Alexei Starovoitov]

On Mon, Sep 18, 2023 at 6:54 AM Eric Dumazet <edumazet@google.com> wrote:
>
> On Mon, Sep 18, 2023 at 3:41 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Mon, Sep 18, 2023 at 6:25 AM Eric Dumazet <edumazet@google.com> wrote:
> > >
> > > On Sat, Sep 16, 2023 at 6:59 PM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > Hi David, hi Jakub, hi Paolo, hi Eric,
> > > >
> > > > The following pull-request contains BPF updates for your *net-next* tree.
> > > >
> > > > We've added 73 non-merge commits during the last 9 day(s) which contain
> > > > a total of 79 files changed, 5275 insertions(+), 600 deletions(-).
> > > >
> > > > The main changes are:
> > > >
> > > > 1) Basic BTF validation in libbpf, from Andrii Nakryiko.
> > > >
> > > > 2) bpf_assert(), bpf_throw(), exceptions in bpf progs, from Kumar Kartikeya Dwivedi.
> > > >
> > > > 3) next_thread cleanups, from Oleg Nesterov.
> > > >
> > > > 4) Add mcpu=v4 support to arm32, from Puranjay Mohan.
> > > >
> > > > 5) Add support for __percpu pointers in bpf progs, from Yonghong Song.
> > > >
> > > > 6) Fix bpf tailcall interaction with bpf trampoline, from Leon Hwang.
> > > >
> > > > 7) Raise irq_work in bpf_mem_alloc while irqs are disabled to improve refill probabablity, from Hou Tao.
> > > >
> > > > Please consider pulling these changes from:
> > > >
> > > >   git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
> > > >
> > >
> > > This might have been raised already, but bpf on x86 now depends on
> > > CONFIG_UNWINDER_ORC ?
> > >
> > > $ grep CONFIG_UNWINDER_ORC .config
> > > # CONFIG_UNWINDER_ORC is not set
> > >
> > > $ make ...
> > > arch/x86/net/bpf_jit_comp.c:3022:58: error: no member named 'sp' in
> > > 'struct unwind_state'
> > >                 if (!addr || !consume_fn(cookie, (u64)addr,
> > > (u64)state.sp, (u64)state.bp))
> > >                                                                  ~~~~~ ^
> > > 1 error generated.
> >
> > Kumar,
> > can probably explain better,
> > but no the bpf as whole doesn't depend.
> > One feature needs either ORC or frame unwinder.
> > It won't work with unwinder_guess.
> > The build error is a separate issue.
> > It hasn't been reported before.
>
> In my builds, I do have CONFIG_UNWINDER_FRAME_POINTER=y
>
> $ grep UNWIND .config
> # CONFIG_UNWINDER_ORC is not set
> CONFIG_UNWINDER_FRAME_POINTER=y
>
>
> I note state.sp is only available to CONFIG_UNWINDER_ORC
>
> arch/x86/include/asm/unwind.h
>
> #if defined(CONFIG_UNWINDER_ORC)
>     bool signal, full_regs;
>     unsigned long sp, bp, ip;
>     struct pt_regs *regs, *prev_regs;
> #elif defined(CONFIG_UNWINDER_FRAME_POINTER)
>    bool got_irq;
>    unsigned long *bp, *orig_sp, ip;   // this is orig_sp , not sp.

Right. Our replies crossed.
Please ignore this PR. We need to fix this first.

Re: pull-request: bpf-next 2023-09-16

[Kumar Kartikeya Dwivedi]

On Mon, 18 Sept 2023 at 15:56, Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Mon, Sep 18, 2023 at 6:54 AM Eric Dumazet <edumazet@google.com> wrote:
> >
> > On Mon, Sep 18, 2023 at 3:41 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Mon, Sep 18, 2023 at 6:25 AM Eric Dumazet <edumazet@google.com> wrote:
> > > >
> > > > On Sat, Sep 16, 2023 at 6:59 PM Alexei Starovoitov
> > > > <alexei.starovoitov@gmail.com> wrote:
> > > > >
> > > > > Hi David, hi Jakub, hi Paolo, hi Eric,
> > > > >
> > > > > The following pull-request contains BPF updates for your *net-next* tree.
> > > > >
> > > > > We've added 73 non-merge commits during the last 9 day(s) which contain
> > > > > a total of 79 files changed, 5275 insertions(+), 600 deletions(-).
> > > > >
> > > > > The main changes are:
> > > > >
> > > > > 1) Basic BTF validation in libbpf, from Andrii Nakryiko.
> > > > >
> > > > > 2) bpf_assert(), bpf_throw(), exceptions in bpf progs, from Kumar Kartikeya Dwivedi.
> > > > >
> > > > > 3) next_thread cleanups, from Oleg Nesterov.
> > > > >
> > > > > 4) Add mcpu=v4 support to arm32, from Puranjay Mohan.
> > > > >
> > > > > 5) Add support for __percpu pointers in bpf progs, from Yonghong Song.
> > > > >
> > > > > 6) Fix bpf tailcall interaction with bpf trampoline, from Leon Hwang.
> > > > >
> > > > > 7) Raise irq_work in bpf_mem_alloc while irqs are disabled to improve refill probabablity, from Hou Tao.
> > > > >
> > > > > Please consider pulling these changes from:
> > > > >
> > > > >   git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
> > > > >
> > > >
> > > > This might have been raised already, but bpf on x86 now depends on
> > > > CONFIG_UNWINDER_ORC ?
> > > >
> > > > $ grep CONFIG_UNWINDER_ORC .config
> > > > # CONFIG_UNWINDER_ORC is not set
> > > >
> > > > $ make ...
> > > > arch/x86/net/bpf_jit_comp.c:3022:58: error: no member named 'sp' in
> > > > 'struct unwind_state'
> > > >                 if (!addr || !consume_fn(cookie, (u64)addr,
> > > > (u64)state.sp, (u64)state.bp))
> > > >                                                                  ~~~~~ ^
> > > > 1 error generated.
> > >
> > > Kumar,
> > > can probably explain better,
> > > but no the bpf as whole doesn't depend.
> > > One feature needs either ORC or frame unwinder.
> > > It won't work with unwinder_guess.
> > > The build error is a separate issue.
> > > It hasn't been reported before.
> >
> > In my builds, I do have CONFIG_UNWINDER_FRAME_POINTER=y
> >
> > $ grep UNWIND .config
> > # CONFIG_UNWINDER_ORC is not set
> > CONFIG_UNWINDER_FRAME_POINTER=y
> >
> >
> > I note state.sp is only available to CONFIG_UNWINDER_ORC
> >
> > arch/x86/include/asm/unwind.h
> >
> > #if defined(CONFIG_UNWINDER_ORC)
> >     bool signal, full_regs;
> >     unsigned long sp, bp, ip;
> >     struct pt_regs *regs, *prev_regs;
> > #elif defined(CONFIG_UNWINDER_FRAME_POINTER)
> >    bool got_irq;
> >    unsigned long *bp, *orig_sp, ip;   // this is orig_sp , not sp.
>
> Right. Our replies crossed.
> Please ignore this PR. We need to fix this first.

Hello,
This is my bad. I totally missed it since I initially wrote this patch
and never looked at it again.
I suggest that I send a fix to disable this feature with
CONFIG_UNWINDER_FRAME_POINTER=y, while I work on reenabling it again
for it with a follow up.

Re: pull-request: bpf-next 2023-09-16

[Kumar Kartikeya Dwivedi]

[-- Attachment #1: Type: text/plain, Size: 3911 bytes --]

On Mon, 18 Sept 2023 at 16:15, Kumar Kartikeya Dwivedi <memxor@gmail.com> wrote:
>
> On Mon, 18 Sept 2023 at 15:56, Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Mon, Sep 18, 2023 at 6:54 AM Eric Dumazet <edumazet@google.com> wrote:
> > >
> > > On Mon, Sep 18, 2023 at 3:41 PM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > On Mon, Sep 18, 2023 at 6:25 AM Eric Dumazet <edumazet@google.com> wrote:
> > > > >
> > > > > On Sat, Sep 16, 2023 at 6:59 PM Alexei Starovoitov
> > > > > <alexei.starovoitov@gmail.com> wrote:
> > > > > >
> > > > > > Hi David, hi Jakub, hi Paolo, hi Eric,
> > > > > >
> > > > > > The following pull-request contains BPF updates for your *net-next* tree.
> > > > > >
> > > > > > We've added 73 non-merge commits during the last 9 day(s) which contain
> > > > > > a total of 79 files changed, 5275 insertions(+), 600 deletions(-).
> > > > > >
> > > > > > The main changes are:
> > > > > >
> > > > > > 1) Basic BTF validation in libbpf, from Andrii Nakryiko.
> > > > > >
> > > > > > 2) bpf_assert(), bpf_throw(), exceptions in bpf progs, from Kumar Kartikeya Dwivedi.
> > > > > >
> > > > > > 3) next_thread cleanups, from Oleg Nesterov.
> > > > > >
> > > > > > 4) Add mcpu=v4 support to arm32, from Puranjay Mohan.
> > > > > >
> > > > > > 5) Add support for __percpu pointers in bpf progs, from Yonghong Song.
> > > > > >
> > > > > > 6) Fix bpf tailcall interaction with bpf trampoline, from Leon Hwang.
> > > > > >
> > > > > > 7) Raise irq_work in bpf_mem_alloc while irqs are disabled to improve refill probabablity, from Hou Tao.
> > > > > >
> > > > > > Please consider pulling these changes from:
> > > > > >
> > > > > >   git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
> > > > > >
> > > > >
> > > > > This might have been raised already, but bpf on x86 now depends on
> > > > > CONFIG_UNWINDER_ORC ?
> > > > >
> > > > > $ grep CONFIG_UNWINDER_ORC .config
> > > > > # CONFIG_UNWINDER_ORC is not set
> > > > >
> > > > > $ make ...
> > > > > arch/x86/net/bpf_jit_comp.c:3022:58: error: no member named 'sp' in
> > > > > 'struct unwind_state'
> > > > >                 if (!addr || !consume_fn(cookie, (u64)addr,
> > > > > (u64)state.sp, (u64)state.bp))
> > > > >                                                                  ~~~~~ ^
> > > > > 1 error generated.
> > > >
> > > > Kumar,
> > > > can probably explain better,
> > > > but no the bpf as whole doesn't depend.
> > > > One feature needs either ORC or frame unwinder.
> > > > It won't work with unwinder_guess.
> > > > The build error is a separate issue.
> > > > It hasn't been reported before.
> > >
> > > In my builds, I do have CONFIG_UNWINDER_FRAME_POINTER=y
> > >
> > > $ grep UNWIND .config
> > > # CONFIG_UNWINDER_ORC is not set
> > > CONFIG_UNWINDER_FRAME_POINTER=y
> > >
> > >
> > > I note state.sp is only available to CONFIG_UNWINDER_ORC
> > >
> > > arch/x86/include/asm/unwind.h
> > >
> > > #if defined(CONFIG_UNWINDER_ORC)
> > >     bool signal, full_regs;
> > >     unsigned long sp, bp, ip;
> > >     struct pt_regs *regs, *prev_regs;
> > > #elif defined(CONFIG_UNWINDER_FRAME_POINTER)
> > >    bool got_irq;
> > >    unsigned long *bp, *orig_sp, ip;   // this is orig_sp , not sp.
> >
> > Right. Our replies crossed.
> > Please ignore this PR. We need to fix this first.
>
> Hello,
> This is my bad. I totally missed it since I initially wrote this patch
> and never looked at it again.
> I suggest that I send a fix to disable this feature with
> CONFIG_UNWINDER_FRAME_POINTER=y, while I work on reenabling it again
> for it with a follow up.

Hi, I've attached a fix that should disable it for now. I'll work on a
follow up to reenable it for this config option.
Really sorry about this, I'll try to be more careful going forward.

[-- Attachment #2: 0001-bpf-Disable-exceptions-when-CONFIG_UNWINDER_FRAME_PO.patch --]
[-- Type: text/x-patch, Size: 1805 bytes --]

From e95240ede0cbe3aa35bbb12bf7eb39e93c7c6e61 Mon Sep 17 00:00:00 2001
From: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Date: Mon, 18 Sep 2023 16:07:27 +0200
Subject: [PATCH] bpf: Disable exceptions when CONFIG_UNWINDER_FRAME_POINTER=y

The build with CONFIG_UNWINDER_FRAME_POINTER=y is broken for
current exceptions feature as it assumes ORC unwinder specific fields in
the unwind_state. Disable exceptions when frame_pointer unwinder is
enabled for now.

Fixes: fd5d27b70188 ("arch/x86: Implement arch_bpf_stack_walk")
Reported-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 arch/x86/net/bpf_jit_comp.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 84005f2114e0..8c10d9abc239 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3003,16 +3003,15 @@ void bpf_jit_free(struct bpf_prog *prog)
 bool bpf_jit_supports_exceptions(void)
 {
 	/* We unwind through both kernel frames (starting from within bpf_throw
-	 * call) and BPF frames. Therefore we require one of ORC or FP unwinder
-	 * to be enabled to walk kernel frames and reach BPF frames in the stack
-	 * trace.
+	 * call) and BPF frames. Therefore we require ORC unwinder to be enabled
+	 * to walk kernel frames and reach BPF frames in the stack trace.
 	 */
-	return IS_ENABLED(CONFIG_UNWINDER_ORC) || IS_ENABLED(CONFIG_UNWINDER_FRAME_POINTER);
+	return IS_ENABLED(CONFIG_UNWINDER_ORC);
 }
 
 void arch_bpf_stack_walk(bool (*consume_fn)(void *cookie, u64 ip, u64 sp, u64 bp), void *cookie)
 {
-#if defined(CONFIG_UNWINDER_ORC) || defined(CONFIG_UNWINDER_FRAME_POINTER)
+#if defined(CONFIG_UNWINDER_ORC)
 	struct unwind_state state;
 	unsigned long addr;
 
-- 
2.41.0

Re: pull-request: bpf-next 2023-09-16

[Alexei Starovoitov]

On Mon, Sep 18, 2023 at 7:24 AM Kumar Kartikeya Dwivedi
<memxor@gmail.com> wrote:
>
> On Mon, 18 Sept 2023 at 16:15, Kumar Kartikeya Dwivedi <memxor@gmail.com> wrote:
> >
> > On Mon, 18 Sept 2023 at 15:56, Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Mon, Sep 18, 2023 at 6:54 AM Eric Dumazet <edumazet@google.com> wrote:
> > > >
> > > > On Mon, Sep 18, 2023 at 3:41 PM Alexei Starovoitov
> > > > <alexei.starovoitov@gmail.com> wrote:
> > > > >
> > > > > On Mon, Sep 18, 2023 at 6:25 AM Eric Dumazet <edumazet@google.com> wrote:
> > > > > >
> > > > > > On Sat, Sep 16, 2023 at 6:59 PM Alexei Starovoitov
> > > > > > <alexei.starovoitov@gmail.com> wrote:
> > > > > > >
> > > > > > > Hi David, hi Jakub, hi Paolo, hi Eric,
> > > > > > >
> > > > > > > The following pull-request contains BPF updates for your *net-next* tree.
> > > > > > >
> > > > > > > We've added 73 non-merge commits during the last 9 day(s) which contain
> > > > > > > a total of 79 files changed, 5275 insertions(+), 600 deletions(-).
> > > > > > >
> > > > > > > The main changes are:
> > > > > > >
> > > > > > > 1) Basic BTF validation in libbpf, from Andrii Nakryiko.
> > > > > > >
> > > > > > > 2) bpf_assert(), bpf_throw(), exceptions in bpf progs, from Kumar Kartikeya Dwivedi.
> > > > > > >
> > > > > > > 3) next_thread cleanups, from Oleg Nesterov.
> > > > > > >
> > > > > > > 4) Add mcpu=v4 support to arm32, from Puranjay Mohan.
> > > > > > >
> > > > > > > 5) Add support for __percpu pointers in bpf progs, from Yonghong Song.
> > > > > > >
> > > > > > > 6) Fix bpf tailcall interaction with bpf trampoline, from Leon Hwang.
> > > > > > >
> > > > > > > 7) Raise irq_work in bpf_mem_alloc while irqs are disabled to improve refill probabablity, from Hou Tao.
> > > > > > >
> > > > > > > Please consider pulling these changes from:
> > > > > > >
> > > > > > >   git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
> > > > > > >
> > > > > >
> > > > > > This might have been raised already, but bpf on x86 now depends on
> > > > > > CONFIG_UNWINDER_ORC ?
> > > > > >
> > > > > > $ grep CONFIG_UNWINDER_ORC .config
> > > > > > # CONFIG_UNWINDER_ORC is not set
> > > > > >
> > > > > > $ make ...
> > > > > > arch/x86/net/bpf_jit_comp.c:3022:58: error: no member named 'sp' in
> > > > > > 'struct unwind_state'
> > > > > >                 if (!addr || !consume_fn(cookie, (u64)addr,
> > > > > > (u64)state.sp, (u64)state.bp))
> > > > > >                                                                  ~~~~~ ^
> > > > > > 1 error generated.
> > > > >
> > > > > Kumar,
> > > > > can probably explain better,
> > > > > but no the bpf as whole doesn't depend.
> > > > > One feature needs either ORC or frame unwinder.
> > > > > It won't work with unwinder_guess.
> > > > > The build error is a separate issue.
> > > > > It hasn't been reported before.
> > > >
> > > > In my builds, I do have CONFIG_UNWINDER_FRAME_POINTER=y
> > > >
> > > > $ grep UNWIND .config
> > > > # CONFIG_UNWINDER_ORC is not set
> > > > CONFIG_UNWINDER_FRAME_POINTER=y
> > > >
> > > >
> > > > I note state.sp is only available to CONFIG_UNWINDER_ORC
> > > >
> > > > arch/x86/include/asm/unwind.h
> > > >
> > > > #if defined(CONFIG_UNWINDER_ORC)
> > > >     bool signal, full_regs;
> > > >     unsigned long sp, bp, ip;
> > > >     struct pt_regs *regs, *prev_regs;
> > > > #elif defined(CONFIG_UNWINDER_FRAME_POINTER)
> > > >    bool got_irq;
> > > >    unsigned long *bp, *orig_sp, ip;   // this is orig_sp , not sp.
> > >
> > > Right. Our replies crossed.
> > > Please ignore this PR. We need to fix this first.
> >
> > Hello,
> > This is my bad. I totally missed it since I initially wrote this patch
> > and never looked at it again.
> > I suggest that I send a fix to disable this feature with
> > CONFIG_UNWINDER_FRAME_POINTER=y, while I work on reenabling it again
> > for it with a follow up.
>
> Hi, I've attached a fix that should disable it for now. I'll work on a
> follow up to reenable it for this config option.
> Really sorry about this, I'll try to be more careful going forward.

Patchwork doesn't recognize patches this way.
Pls submit it properly.

Re: pull-request: bpf-next 2023-09-16

[Alexei Starovoitov]

On Mon, Sep 18, 2023 at 6:54 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
> > > >
> > >
> > > This might have been raised already, but bpf on x86 now depends on
> > > CONFIG_UNWINDER_ORC ?
> > >
> > > $ grep CONFIG_UNWINDER_ORC .config
> > > # CONFIG_UNWINDER_ORC is not set
> > >
> > > $ make ...
> > > arch/x86/net/bpf_jit_comp.c:3022:58: error: no member named 'sp' in
> > > 'struct unwind_state'
> > >                 if (!addr || !consume_fn(cookie, (u64)addr,
> > > (u64)state.sp, (u64)state.bp))
> > >                                                                  ~~~~~ ^
> > > 1 error generated.
> >
> > Kumar,
> > can probably explain better,
> > but no the bpf as whole doesn't depend.
> > One feature needs either ORC or frame unwinder.
> > It won't work with unwinder_guess.
> > The build error is a separate issue.
> > It hasn't been reported before.
>
> I see the error with CONFIG_UNWINDER_FRAME_POINTER.
> That's unexpected.
> Kumar,
> looks like this config path wasn't tested.
>
> Eric, Paolo, Dave, Kuba,
> please ignore this PR.
> We need to fix this first.

Sorry, wifi is slow here. I didn't notice that it got merged
and pw-bot didn't notice it either (no emails).
We'll send another bpf-next PR with fixes right away.
Sorry about the build breakage.