Re: [PATCH bpf-next 2/2] selftests/bpf: Test BPF map creation using BTF-defined key/value

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Thu, Sep 30, 2021 at 9:05 AM Hengqi Chen <hengqi.chen@gmail.com> wrote:
>
>
>
> On 9/9/21 12:29 PM, Andrii Nakryiko wrote:
> > On Sun, Sep 5, 2021 at 3:09 AM Hengqi Chen <hengqi.chen@gmail.com> wrote:
> >>
> >> Test BPF map creation using BTF-defined key/value. The test defines
> >> some specialized maps by specifying BTF types for key/value and
> >> checks those maps are correctly initialized and loaded.
> >>
> >> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
> >> ---
> >>  .../selftests/bpf/prog_tests/map_create.c     |  87 ++++++++++++++
> >>  .../selftests/bpf/progs/test_map_create.c     | 110 ++++++++++++++++++
> >>  2 files changed, 197 insertions(+)
> >>  create mode 100644 tools/testing/selftests/bpf/prog_tests/map_create.c
> >>  create mode 100644 tools/testing/selftests/bpf/progs/test_map_create.c
> >>
> >> diff --git a/tools/testing/selftests/bpf/prog_tests/map_create.c b/tools/testing/selftests/bpf/prog_tests/map_create.c
> >> new file mode 100644
> >> index 000000000000..6ca32d0dffd2
> >> --- /dev/null
> >> +++ b/tools/testing/selftests/bpf/prog_tests/map_create.c
> >> @@ -0,0 +1,87 @@
> >> +/* SPDX-License-Identifier: GPL-2.0 */
> >> +/* Copyright (c) 2021 Hengqi Chen */
> >> +
> >> +#include <test_progs.h>
> >> +#include "test_map_create.skel.h"
> >> +
> >> +void test_map_create(void)
> >> +{
> >> +       struct test_map_create *skel;
> >> +       int err, fd;
> >> +
> >> +       skel = test_map_create__open();
> >> +       if (!ASSERT_OK_PTR(skel, "test_map_create__open failed"))
> >> +               return;
> >> +
> >> +       err = test_map_create__load(skel);
> >
> > If load() succeeds, all the maps will definitely be created, so all
> > the below tests are meaningless.
> >
> > I think it's better to just change all the existing map definitions
> > used throughout selftests to use key/value types, instead of
> > key_size/value_size. That will automatically test this feature without
> > adding an extra test. Unfortunately to really test that the logic is
> > working, we'd need to check that libbpf doesn't emit the warning about
> > retrying map creation w/o BTF, but I think one-time manual check
> > (please use ./test_progs -v to see libbpf warnings during tests)
> > should be sufficient for this.
> >
>
> Hello, Andrii
>
> I updated these existing tests as you suggested,
> but I was unable to run the whole bpf selftests locally.
>
> Running ./test_progs -v made my system hang up,
> didn't find the root cause yet.

This is strange. Do you know at which test this happens? Do you get
kernel warning/oops when this happens in dmesg?

But overall, the development and testing workflow for people working
on bpf/bpf-next involves building latest kernel and running selftests
inside the QEMU VM for testing. We also have vmtest.sh script in
selftests/bpf that automates a lot of building steps. It will build
kernel, test_progs and other selftest binaries, and will spin up QEMU
VM with the same image that our BPF CIs are using. You just need to
have very recent/latest Clang available and similarly pahole (from
dwarves package) should be up to date and available through $PATH.
After that, running ./vmtest.sh will just run all ./test_progs
automatically.

Either way, our CI will also run your changes through the tests
(except there are some intermittent issues right now, so we'll have to
wait a bit for that to kick in). You can monitor [0], or the link will
actually appear on each of your patches (e.g., [1]) in "Checks"
section, once everything is up and running properly.

  [0] https://github.com/kernel-patches/bpf/pulls
  [1] https://patchwork.kernel.org/project/netdevbpf/patch/20210930161456.3444544-2-hengqi.chen@gmail.com/

>
> Instead I just run the modified test with the following commands:
>
> sudo ./test_progs -v --name=kfree_skb,perf_event_stackmap,btf_map_in_map,pe_preserve_elems,stacktrace_map,stacktrace_build_id,xdp_bpf2bpf,select_reuseport,tcpbpf_user
>
> >> +       if (!ASSERT_OK(err, "test_map_create__load failed"))
> >> +               goto cleanup;
> >> +
> >> +       fd = bpf_map__fd(skel->maps.map1);
> >> +       if (!ASSERT_GT(fd, 0, "bpf_map__fd failed"))
> >> +               goto cleanup;
> >> +       close(fd);
> >> +
> >> +       fd = bpf_map__fd(skel->maps.map2);
> >> +       if (!ASSERT_GT(fd, 0, "bpf_map__fd failed"))
> >> +               goto cleanup;
> >> +       close(fd);
> >
> > [...]
> >