Re: [RESEND PATCH bpf-next v9 5/5] selftests/bpf: add a selftest for cgroup hierarchical stats collection

[Yosry Ahmed <yosryahmed@google.com>]

On Wed, Aug 24, 2022 at 7:09 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Wed, Aug 24, 2022 at 4:31 PM Hao Luo <haoluo@google.com> wrote:
> > +
> > +       for (i = 0; i < N_CGROUPS; i++) {
> > +               fd = create_and_get_cgroup(cgroups[i].path);
> > +               if (!ASSERT_GE(fd, 0, "create_and_get_cgroup"))
> > +                       return fd;
> > +
> > +               cgroups[i].fd = fd;
> > +               cgroups[i].id = get_cgroup_id(cgroups[i].path);
> > +
> > +               /*
> > +                * Enable memcg controller for the entire hierarchy.
> > +                * Note that stats are collected for all cgroups in a hierarchy
> > +                * with memcg enabled anyway, but are only exposed for cgroups
> > +                * that have memcg enabled.
> > +                */
> > +               if (i < N_NON_LEAF_CGROUPS) {
> > +                       err = enable_controllers(cgroups[i].path, "memory");
> > +                       if (!ASSERT_OK(err, "enable_controllers"))
> > +                               return err;
> > +               }
> > +       }
>
> It passes BPF CI, but fails in my setup with:
>
> # ./test_progs -t cgroup_hier -vv
> bpf_testmod.ko is already unloaded.
> Loading bpf_testmod.ko...
> Successfully loaded bpf_testmod.ko.
> setup_bpffs:PASS:mount 0 nsec
> setup_cgroups:PASS:setup_cgroup_environment 0 nsec
> setup_cgroups:PASS:get_root_cgroup 0 nsec
> setup_cgroups:PASS:create_and_get_cgroup 0 nsec
> (cgroup_helpers.c:92: errno: No such file or directory) Enabling
> controller memory:
> /mnt/cgroup-test-work-dir6526//test/cgroup.subtree_control
> setup_cgroups:FAIL:enable_controllers unexpected error: 1 (errno 2)
> cleanup_bpffs:FAIL:rmdir /sys/fs/bpf/vmscan/ unexpected error: -1 (errno 2)
> #36      cgroup_hierarchical_stats:FAIL
> Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED
>
> How do I debug it?

The failure with ENOENT happens when we try to write "+memory" to
/mnt/cgroup-test-work-dir6526//test/cgroup.subtree_control, not when
we try to open it. So the file is there. AFAICT, ENOENT can be
returned from this write if the memory controller is not enabled on
this cgroup.

In setup_cgroup_environment(), we should be enabling all available
controllers on /mnt and /mnt/cgroup-test-work-dir6526 by this line:

        if (__enable_controllers(CGROUP_MOUNT_PATH, NULL) ||
              __enable_controllers(cgroup_workdir, NULL))
                  return 1;

The first thing that comes to mind is that maybe the memory controller
is not enabled on your setup at all? Can you check
/sys/fs/cgroup/cgroup.controllers (or wherever your global cgroup
mount is)?

I don't know much about namespaces, so I am not sure if the privately
mounted /mnt directory here would be the same as the cgroup root or
not. Maybe we can add a pause() somewhere and check
/mnt/cgroup.controllers as well?