All of lore.kernel.org
 help / color / mirror / Atom feed
From: Kumar Kartikeya Dwivedi <memxor@gmail.com>
To: Hao Luo <haoluo@google.com>
Cc: linux-kernel@vger.kernel.org, bpf@vger.kernel.org,
	cgroups@vger.kernel.org, netdev@vger.kernel.org,
	Alexei Starovoitov <ast@kernel.org>,
	Andrii Nakryiko <andrii@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Martin KaFai Lau <martin.lau@linux.dev>,
	Song Liu <song@kernel.org>, Yonghong Song <yhs@fb.com>,
	Tejun Heo <tj@kernel.org>, Zefan Li <lizefan.x@bytedance.com>,
	KP Singh <kpsingh@kernel.org>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Michal Hocko <mhocko@kernel.org>,
	John Fastabend <john.fastabend@gmail.com>,
	Jiri Olsa <jolsa@kernel.org>, Michal Koutny <mkoutny@suse.com>,
	Roman Gushchin <roman.gushchin@linux.dev>,
	David Rientjes <rientjes@google.com>,
	Stanislav Fomichev <sdf@google.com>,
	Shakeel Butt <shakeelb@google.com>,
	Yosry Ahmed <yosryahmed@google.com>
Subject: Re: [PATCH bpf-next v9 5/5] selftests/bpf: add a selftest for cgroup hierarchical stats collection
Date: Thu, 25 Aug 2022 01:09:08 +0200	[thread overview]
Message-ID: <CAP01T77oxbfQnHSX5irq0d=srArq=ZTf_VAMuw0QNhfcjJVdKQ@mail.gmail.com> (raw)
In-Reply-To: <CA+khW7go3_KNjju=auaX0A0Ff4-DcmGr9=+TW1tpuqxFv8uwag@mail.gmail.com>

On Thu, 25 Aug 2022 at 01:07, Hao Luo <haoluo@google.com> wrote:
>
> On Tue, Aug 23, 2022 at 8:01 PM Hao Luo <haoluo@google.com> wrote:
> >
> > From: Yosry Ahmed <yosryahmed@google.com>
> >
> > Add a selftest that tests the whole workflow for collecting,
> > aggregating (flushing), and displaying cgroup hierarchical stats.
> >
> > TL;DR:
> > - Userspace program creates a cgroup hierarchy and induces memcg reclaim
> >   in parts of it.
> > - Whenever reclaim happens, vmscan_start and vmscan_end update
> >   per-cgroup percpu readings, and tell rstat which (cgroup, cpu) pairs
> >   have updates.
> > - When userspace tries to read the stats, vmscan_dump calls rstat to flush
> >   the stats, and outputs the stats in text format to userspace (similar
> >   to cgroupfs stats).
> > - rstat calls vmscan_flush once for every (cgroup, cpu) pair that has
> >   updates, vmscan_flush aggregates cpu readings and propagates updates
> >   to parents.
> > - Userspace program makes sure the stats are aggregated and read
> >   correctly.
> >
> > Detailed explanation:
> > - The test loads tracing bpf programs, vmscan_start and vmscan_end, to
> >   measure the latency of cgroup reclaim. Per-cgroup readings are stored in
> >   percpu maps for efficiency. When a cgroup reading is updated on a cpu,
> >   cgroup_rstat_updated(cgroup, cpu) is called to add the cgroup to the
> >   rstat updated tree on that cpu.
> >
> > - A cgroup_iter program, vmscan_dump, is loaded and pinned to a file, for
> >   each cgroup. Reading this file invokes the program, which calls
> >   cgroup_rstat_flush(cgroup) to ask rstat to propagate the updates for all
> >   cpus and cgroups that have updates in this cgroup's subtree. Afterwards,
> >   the stats are exposed to the user. vmscan_dump returns 1 to terminate
> >   iteration early, so that we only expose stats for one cgroup per read.
> >
> > - An ftrace program, vmscan_flush, is also loaded and attached to
> >   bpf_rstat_flush. When rstat flushing is ongoing, vmscan_flush is invoked
> >   once for each (cgroup, cpu) pair that has updates. cgroups are popped
> >   from the rstat tree in a bottom-up fashion, so calls will always be
> >   made for cgroups that have updates before their parents. The program
> >   aggregates percpu readings to a total per-cgroup reading, and also
> >   propagates them to the parent cgroup. After rstat flushing is over, all
> >   cgroups will have correct updated hierarchical readings (including all
> >   cpus and all their descendants).
> >
> > - Finally, the test creates a cgroup hierarchy and induces memcg reclaim
> >   in parts of it, and makes sure that the stats collection, aggregation,
> >   and reading workflow works as expected.
> >
> > Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
> > Signed-off-by: Hao Luo <haoluo@google.com>
> > ---
>
> I saw this test failed on CI on s390x [0], because of using kfunc, and
> on s390x, "JIT does not support calling kernel function". Is there
> anything I can do about it
>

You can add it to the deny list, like this patch:
https://lore.kernel.org/bpf/20220824163906.1186832-1-deso@posteo.net

WARNING: multiple messages have this Message-ID (diff)
From: Kumar Kartikeya Dwivedi <memxor@gmail.com>
To: Hao Luo <haoluo@google.com>
Cc: linux-kernel@vger.kernel.org, bpf@vger.kernel.org,
	cgroups@vger.kernel.org, netdev@vger.kernel.org,
	Alexei Starovoitov <ast@kernel.org>,
	Andrii Nakryiko <andrii@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Martin KaFai Lau <martin.lau@linux.dev>,
	Song Liu <song@kernel.org>, Yonghong Song <yhs@fb.com>,
	Tejun Heo <tj@kernel.org>, Zefan Li <lizefan.x@bytedance.com>,
	KP Singh <kpsingh@kernel.org>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Michal Hocko <mhocko@kernel.org>,
	John Fastabend <john.fastabend@gmail.com>,
	Jiri Olsa <jolsa@kernel.org>, Michal Koutny <mkoutny@suse.com>,
	Roman Gushchin <roman.gushchin@linux.dev>,
	David Rientjes <rientjes@google.com>,
	Stanislav Fomichev <sdf@google.com>,
	Shakeel Butt <shakeelb@google.com>, Yosry Ahmed <yosryahme>
Subject: Re: [PATCH bpf-next v9 5/5] selftests/bpf: add a selftest for cgroup hierarchical stats collection
Date: Thu, 25 Aug 2022 01:09:08 +0200	[thread overview]
Message-ID: <CAP01T77oxbfQnHSX5irq0d=srArq=ZTf_VAMuw0QNhfcjJVdKQ@mail.gmail.com> (raw)
In-Reply-To: <CA+khW7go3_KNjju=auaX0A0Ff4-DcmGr9=+TW1tpuqxFv8uwag@mail.gmail.com>

On Thu, 25 Aug 2022 at 01:07, Hao Luo <haoluo@google.com> wrote:
>
> On Tue, Aug 23, 2022 at 8:01 PM Hao Luo <haoluo@google.com> wrote:
> >
> > From: Yosry Ahmed <yosryahmed@google.com>
> >
> > Add a selftest that tests the whole workflow for collecting,
> > aggregating (flushing), and displaying cgroup hierarchical stats.
> >
> > TL;DR:
> > - Userspace program creates a cgroup hierarchy and induces memcg reclaim
> >   in parts of it.
> > - Whenever reclaim happens, vmscan_start and vmscan_end update
> >   per-cgroup percpu readings, and tell rstat which (cgroup, cpu) pairs
> >   have updates.
> > - When userspace tries to read the stats, vmscan_dump calls rstat to flush
> >   the stats, and outputs the stats in text format to userspace (similar
> >   to cgroupfs stats).
> > - rstat calls vmscan_flush once for every (cgroup, cpu) pair that has
> >   updates, vmscan_flush aggregates cpu readings and propagates updates
> >   to parents.
> > - Userspace program makes sure the stats are aggregated and read
> >   correctly.
> >
> > Detailed explanation:
> > - The test loads tracing bpf programs, vmscan_start and vmscan_end, to
> >   measure the latency of cgroup reclaim. Per-cgroup readings are stored in
> >   percpu maps for efficiency. When a cgroup reading is updated on a cpu,
> >   cgroup_rstat_updated(cgroup, cpu) is called to add the cgroup to the
> >   rstat updated tree on that cpu.
> >
> > - A cgroup_iter program, vmscan_dump, is loaded and pinned to a file, for
> >   each cgroup. Reading this file invokes the program, which calls
> >   cgroup_rstat_flush(cgroup) to ask rstat to propagate the updates for all
> >   cpus and cgroups that have updates in this cgroup's subtree. Afterwards,
> >   the stats are exposed to the user. vmscan_dump returns 1 to terminate
> >   iteration early, so that we only expose stats for one cgroup per read.
> >
> > - An ftrace program, vmscan_flush, is also loaded and attached to
> >   bpf_rstat_flush. When rstat flushing is ongoing, vmscan_flush is invoked
> >   once for each (cgroup, cpu) pair that has updates. cgroups are popped
> >   from the rstat tree in a bottom-up fashion, so calls will always be
> >   made for cgroups that have updates before their parents. The program
> >   aggregates percpu readings to a total per-cgroup reading, and also
> >   propagates them to the parent cgroup. After rstat flushing is over, all
> >   cgroups will have correct updated hierarchical readings (including all
> >   cpus and all their descendants).
> >
> > - Finally, the test creates a cgroup hierarchy and induces memcg reclaim
> >   in parts of it, and makes sure that the stats collection, aggregation,
> >   and reading workflow works as expected.
> >
> > Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
> > Signed-off-by: Hao Luo <haoluo@google.com>
> > ---
>
> I saw this test failed on CI on s390x [0], because of using kfunc, and
> on s390x, "JIT does not support calling kernel function". Is there
> anything I can do about it
>

You can add it to the deny list, like this patch:
https://lore.kernel.org/bpf/20220824163906.1186832-1-deso@posteo.net

  reply	other threads:[~2022-08-24 23:09 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-24  3:00 [PATCH bpf-next v9 0/5] bpf: rstat: cgroup hierarchical stats Hao Luo
2022-08-24  3:00 ` Hao Luo
2022-08-24  3:00 ` [PATCH bpf-next v9 1/5] bpf: Introduce cgroup iter Hao Luo
2022-08-24  3:00   ` Hao Luo
2022-08-25 15:24   ` Michal Koutný
2022-08-25 17:58     ` Hao Luo
2022-08-25 17:58       ` Hao Luo
2022-08-26 17:15       ` Michal Koutný
2022-08-25 18:56     ` Hao Luo
2022-08-25 18:56       ` Hao Luo
2022-08-26 16:57       ` Michal Koutný
2022-08-26 16:57         ` Michal Koutný
2022-08-25 20:18   ` Andrii Nakryiko
2022-08-25 20:18     ` Andrii Nakryiko
2022-08-25 20:35     ` Hao Luo
2022-08-25 20:35       ` Hao Luo
2022-08-24  3:00 ` [PATCH bpf-next v9 2/5] selftests/bpf: Test cgroup_iter Hao Luo
2022-08-24  3:00   ` Hao Luo
2022-08-24  3:00 ` [PATCH bpf-next v9 3/5] cgroup: bpf: enable bpf programs to integrate with rstat Hao Luo
2022-08-24  3:00 ` [PATCH bpf-next v9 4/5] selftests/bpf: extend cgroup helpers Hao Luo
2022-08-24  3:00   ` Hao Luo
2022-08-24  3:00 ` [PATCH bpf-next v9 5/5] selftests/bpf: add a selftest for cgroup hierarchical stats collection Hao Luo
2022-08-24  3:00   ` Hao Luo
2022-08-24 23:02   ` Hao Luo
2022-08-24 23:09     ` Kumar Kartikeya Dwivedi [this message]
2022-08-24 23:09       ` Kumar Kartikeya Dwivedi
2022-08-24 23:14       ` Hao Luo
2022-08-24 23:14         ` Hao Luo
2022-08-25 18:40 ` [PATCH bpf-next v9 0/5] bpf: rstat: cgroup hierarchical stats patchwork-bot+netdevbpf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAP01T77oxbfQnHSX5irq0d=srArq=ZTf_VAMuw0QNhfcjJVdKQ@mail.gmail.com' \
    --to=memxor@gmail.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=cgroups@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=hannes@cmpxchg.org \
    --cc=haoluo@google.com \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kpsingh@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizefan.x@bytedance.com \
    --cc=martin.lau@linux.dev \
    --cc=mhocko@kernel.org \
    --cc=mkoutny@suse.com \
    --cc=netdev@vger.kernel.org \
    --cc=rientjes@google.com \
    --cc=roman.gushchin@linux.dev \
    --cc=sdf@google.com \
    --cc=shakeelb@google.com \
    --cc=song@kernel.org \
    --cc=tj@kernel.org \
    --cc=yhs@fb.com \
    --cc=yosryahmed@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.