[PATCH bpf-next 00/24] Second set of verifier/*.c migrated to inline assembly

[Eduard Zingerman <eddyz87@gmail.com>]

This is a follow up for RFC [1]. It migrates a second batch of 23
verifier/*.c tests to inline assembly and use of ./test_progs for
actual execution. Link to the first batch is [2].

The migration is done by a python script (see [3]) with minimal manual
adjustments.

Each migrated verifier/xxx.c file is mapped to progs/verifier_xxx.c
plus an entry in the prog_tests/verifier.c. One patch per each file.

The first patch in a series adds a notion of auxiliary programs to
test_loader. This is necessary to support tests that use
BPF_MAP_TYPE_PROG_ARRAY maps. Programs marked as auxiliary are always
loaded but are not treated as a separate tests.

The main differences compared to the previous batch:
- Tests that use pseudo call instruction are migrated,
  called functions are translated as below:

    static __naked __noinline __attribute__((used))
    void bounded_recursion__1(void)
    {
    	asm volatile (...);
    }
    
  Pseudo call in the inline assembly looks as follows:

    __naked void bounded_recursion(void)
    {
        asm volatile ("                                 \
        r1 = 0;                                         \
        call bounded_recursion__1;                      \
        exit;                                           \
    "   ::: __clobber_all);
    }
    
  Interestingly enough, callee declaration does not have to precede
  caller definition when used from inline assembly.

- Tests that specify .expected_attach_type are migrated using
  corresponding SEC annotations. For example, the following test
  specification:

    {
            "reference tracking: ...",
            .insns = { ... },
            .prog_type = BPF_PROG_TYPE_LSM,
            .kfunc = "bpf",
            .expected_attach_type = BPF_LSM_MAC,
            .flags = BPF_F_SLEEPABLE,
            ...
    },
    
  Becomes:

    SEC("lsm.s/bpf")
    __description("reference tracking: ...")
    __success
    __naked void acquire_release_user_key_reference(void) { ... }
    
- Tests that use BPF_MAP_TYPE_PROG_ARRAY are migrated, the definitions
  of map and dummy programs that populate it are repeated in each test.

There `__imm_insn` macro had to be used in a few tests because of the
limitations of clang BPF inline assembler:
- For BPF_ST_MEM instruction in verifier_precise.c and verifier_unpriv.c;
- For BPF_LD_IND with three arguments in verifier_ref_tracking.c.

Migrated tests could be selected for execution using the following filter:

  ./test_progs -a verifier_*

While reviewing the changes I noticed the following irregularities in
the original tests:
- verifier_sock.c:
  Tests "bpf_sk_select_reuseport(ctx, sockhash, &key, flags)"
    and "bpf_sk_select_reuseport(ctx, sockmap, &key, flags)"
  have identical bodies.
- unpriv.c:
  Despite the name of the file, 12 tests defined in it have program
  types that are not considered for unprivileged execution by
  test_verifier (e.g. BPF_PROG_TYPE_TRACEPOINT).

Modifications were applied for the following tests:
- loop1.c:

  Some of the tests have .retval field specified, but program type is
  BPF_PROG_TYPE_TRACEPOINT, which does not support BPF_PROG_TEST_RUN
  command, for these tests
  - either program type is changed to "xdp", which supports
    BPF_PROG_TEST_RUN;
  - or retval tag is removed, if test run result is not actually
    predictable (e.g. depends on bpf_get_prandom_u32()).
- unpriv.c:
  This file is split in two parts:
  - progs/verifier_unpriv.c
  - progs/verifier_unpriv_perf.c
  First part requires inclusion of filter.h,
  second part requires inclusion of vmlinux.h.
- value_ptr_arith.c:
  "sanitation: alu with different scalars 2" and
  "sanitation: alu with different scalars 3"
  are modified to avoid retval "-EINVAL * 2" which cannot be encoded
  as a type tag.
  
Additional details are in the relevant commit messages.

[1] RFC
    https://lore.kernel.org/bpf/20230123145148.2791939-1-eddyz87@gmail.com/
[2] First batch of migrated tests
    https://lore.kernel.org/bpf/20230325025524.144043-1-eddyz87@gmail.com/
[3] Migration tool
    https://github.com/eddyz87/verifier-tests-migrator

Eduard Zingerman (24):
  selftests/bpf: Add notion of auxiliary programs for test_loader
  selftests/bpf: verifier/bounds converted to inline assembly
  selftests/bpf: verifier/bpf_get_stack converted to inline assembly
  selftests/bpf: verifier/btf_ctx_access converted to inline assembly
  selftests/bpf: verifier/ctx converted to inline assembly
  selftests/bpf: verifier/d_path converted to inline assembly
  selftests/bpf: verifier/direct_packet_access converted to inline
    assembly
  selftests/bpf: verifier/jeq_infer_not_null converted to inline
    assembly
  selftests/bpf: verifier/loops1 converted to inline assembly
  selftests/bpf: verifier/lwt converted to inline assembly
  selftests/bpf: verifier/map_in_map converted to inline assembly
  selftests/bpf: verifier/map_ptr_mixing converted to inline assembly
  selftests/bpf: verifier/precise converted to inline assembly
  selftests/bpf: verifier/prevent_map_lookup converted to inline
    assembly
  selftests/bpf: verifier/ref_tracking converted to inline assembly
  selftests/bpf: verifier/regalloc converted to inline assembly
  selftests/bpf: verifier/runtime_jit converted to inline assembly
  selftests/bpf: verifier/search_pruning converted to inline assembly
  selftests/bpf: verifier/sock converted to inline assembly
  selftests/bpf: verifier/spin_lock converted to inline assembly
  selftests/bpf: verifier/subreg converted to inline assembly
  selftests/bpf: verifier/unpriv converted to inline assembly
  selftests/bpf: verifier/value_illegal_alu converted to inline assembly
  selftests/bpf: verifier/value_ptr_arith converted to inline assembly

 .../selftests/bpf/prog_tests/verifier.c       |   80 +-
 tools/testing/selftests/bpf/progs/bpf_misc.h  |    6 +
 .../selftests/bpf/progs/verifier_bounds.c     | 1076 ++++++++++++
 .../bpf/progs/verifier_bpf_get_stack.c        |  124 ++
 .../bpf/progs/verifier_btf_ctx_access.c       |   32 +
 .../selftests/bpf/progs/verifier_ctx.c        |  221 +++
 .../selftests/bpf/progs/verifier_d_path.c     |   48 +
 .../bpf/progs/verifier_direct_packet_access.c |  803 +++++++++
 .../bpf/progs/verifier_jeq_infer_not_null.c   |  213 +++
 .../selftests/bpf/progs/verifier_loops1.c     |  259 +++
 .../selftests/bpf/progs/verifier_lwt.c        |  234 +++
 .../selftests/bpf/progs/verifier_map_in_map.c |  142 ++
 .../bpf/progs/verifier_map_ptr_mixing.c       |  265 +++
 .../selftests/bpf/progs/verifier_precise.c    |  269 +++
 .../bpf/progs/verifier_prevent_map_lookup.c   |   65 +
 .../bpf/progs/verifier_ref_tracking.c         | 1495 +++++++++++++++++
 .../selftests/bpf/progs/verifier_regalloc.c   |  364 ++++
 .../bpf/progs/verifier_runtime_jit.c          |  360 ++++
 .../bpf/progs/verifier_search_pruning.c       |  339 ++++
 .../selftests/bpf/progs/verifier_sock.c       |  980 +++++++++++
 .../selftests/bpf/progs/verifier_spin_lock.c  |  533 ++++++
 .../selftests/bpf/progs/verifier_subreg.c     |  673 ++++++++
 .../selftests/bpf/progs/verifier_unpriv.c     |  726 ++++++++
 .../bpf/progs/verifier_unpriv_perf.c          |   34 +
 .../bpf/progs/verifier_value_illegal_alu.c    |  149 ++
 .../bpf/progs/verifier_value_ptr_arith.c      | 1423 ++++++++++++++++
 tools/testing/selftests/bpf/test_loader.c     |   89 +-
 tools/testing/selftests/bpf/verifier/bounds.c |  884 ----------
 .../selftests/bpf/verifier/bpf_get_stack.c    |   87 -
 .../selftests/bpf/verifier/btf_ctx_access.c   |   25 -
 tools/testing/selftests/bpf/verifier/ctx.c    |  186 --
 tools/testing/selftests/bpf/verifier/d_path.c |   37 -
 .../bpf/verifier/direct_packet_access.c       |  710 --------
 .../bpf/verifier/jeq_infer_not_null.c         |  174 --
 tools/testing/selftests/bpf/verifier/loops1.c |  206 ---
 tools/testing/selftests/bpf/verifier/lwt.c    |  189 ---
 .../selftests/bpf/verifier/map_in_map.c       |   96 --
 .../selftests/bpf/verifier/map_ptr_mixing.c   |  100 --
 .../testing/selftests/bpf/verifier/precise.c  |  219 ---
 .../bpf/verifier/prevent_map_lookup.c         |   29 -
 .../selftests/bpf/verifier/ref_tracking.c     | 1082 ------------
 .../testing/selftests/bpf/verifier/regalloc.c |  277 ---
 .../selftests/bpf/verifier/runtime_jit.c      |  231 ---
 .../selftests/bpf/verifier/search_pruning.c   |  266 ---
 tools/testing/selftests/bpf/verifier/sock.c   |  706 --------
 .../selftests/bpf/verifier/spin_lock.c        |  447 -----
 tools/testing/selftests/bpf/verifier/subreg.c |  533 ------
 tools/testing/selftests/bpf/verifier/unpriv.c |  562 -------
 .../bpf/verifier/value_illegal_alu.c          |   95 --
 .../selftests/bpf/verifier/value_ptr_arith.c  | 1140 -------------
 50 files changed, 10974 insertions(+), 8309 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_bounds.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_bpf_get_stack.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_btf_ctx_access.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_ctx.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_d_path.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_direct_packet_access.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_jeq_infer_not_null.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_loops1.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_lwt.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_map_in_map.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_map_ptr_mixing.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_precise.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_prevent_map_lookup.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_ref_tracking.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_regalloc.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_runtime_jit.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_search_pruning.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_sock.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_spin_lock.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_subreg.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_unpriv.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_unpriv_perf.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_value_illegal_alu.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_value_ptr_arith.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/bounds.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/bpf_get_stack.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/btf_ctx_access.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/ctx.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/d_path.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/direct_packet_access.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/jeq_infer_not_null.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/loops1.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/lwt.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/map_in_map.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/map_ptr_mixing.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/precise.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/prevent_map_lookup.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/ref_tracking.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/regalloc.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/runtime_jit.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/search_pruning.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/sock.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/spin_lock.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/subreg.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/unpriv.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/value_illegal_alu.c
 delete mode 100644 tools/testing/selftests/bpf/verifier/value_ptr_arith.c

-- 
2.40.0