linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: Hao Luo <haoluo@google.com>
Cc: LKML <linux-kernel@vger.kernel.org>, bpf <bpf@vger.kernel.org>,
	"open list:CONTROL GROUP (CGROUP)" <cgroups@vger.kernel.org>,
	Network Development <netdev@vger.kernel.org>,
	Alexei Starovoitov <ast@kernel.org>,
	Andrii Nakryiko <andrii@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Martin KaFai Lau <martin.lau@linux.dev>,
	Song Liu <song@kernel.org>, Yonghong Song <yhs@fb.com>,
	Tejun Heo <tj@kernel.org>, Zefan Li <lizefan.x@bytedance.com>,
	KP Singh <kpsingh@kernel.org>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Michal Hocko <mhocko@kernel.org>,
	John Fastabend <john.fastabend@gmail.com>,
	Jiri Olsa <jolsa@kernel.org>, Michal Koutny <mkoutny@suse.com>,
	Roman Gushchin <roman.gushchin@linux.dev>,
	David Rientjes <rientjes@google.com>,
	Stanislav Fomichev <sdf@google.com>,
	Shakeel Butt <shakeelb@google.com>,
	Yosry Ahmed <yosryahmed@google.com>
Subject: Re: [RESEND PATCH bpf-next v9 0/5] bpf: rstat: cgroup hierarchical
Date: Wed, 24 Aug 2022 17:29:43 -0700	[thread overview]
Message-ID: <CAADnVQLT3JE8LtOYrs30mL88PNs+NaSeXgQqAPEAup5LUC+BPQ@mail.gmail.com> (raw)
In-Reply-To: <20220824233117.1312810-1-haoluo@google.com>

On Wed, Aug 24, 2022 at 4:31 PM Hao Luo <haoluo@google.com> wrote:
>
> This patch series allows for using bpf to collect hierarchical cgroup
> stats efficiently by integrating with the rstat framework. The rstat
> framework provides an efficient way to collect cgroup stats percpu and
> propagate them through the cgroup hierarchy.
>
> The stats are exposed to userspace in textual form by reading files in
> bpffs, similar to cgroupfs stats by using a cgroup_iter program.
> cgroup_iter is a type of bpf_iter. It walks over cgroups in four modes:
> - walking a cgroup's descendants in pre-order.
> - walking a cgroup's descendants in post-order.
> - walking a cgroup's ancestors.
> - process only a single object.
>
> When attaching cgroup_iter, one needs to set a cgroup to the iter_link
> created from attaching. This cgroup can be passed either as a file
> descriptor or a cgroup id. That cgroup serves as the starting point of
> the walk.
>
> One can also terminate the walk early by returning 1 from the iter
> program.
>
> Note that because walking cgroup hierarchy holds cgroup_mutex, the iter
> program is called with cgroup_mutex held.
>
> ** Background on rstat for stats collection **
> (I am using a subscriber analogy that is not commonly used)
>
> The rstat framework maintains a tree of cgroups that have updates and
> which cpus have updates. A subscriber to the rstat framework maintains
> their own stats. The framework is used to tell the subscriber when
> and what to flush, for the most efficient stats propagation. The
> workflow is as follows:
>
> - When a subscriber updates a cgroup on a cpu, it informs the rstat
>   framework by calling cgroup_rstat_updated(cgrp, cpu).
>
> - When a subscriber wants to read some stats for a cgroup, it asks
>   the rstat framework to initiate a stats flush (propagation) by calling
>   cgroup_rstat_flush(cgrp).
>
> - When the rstat framework initiates a flush, it makes callbacks to
>   subscribers to aggregate stats on cpus that have updates, and
>   propagate updates to their parent.
>
> Currently, the main subscribers to the rstat framework are cgroup
> subsystems (e.g. memory, block). This patch series allow bpf programs to
> become subscribers as well.
>
> Patches in this series are organized as follows:
> * Patches 1-2 introduce cgroup_iter prog, and a selftest.
> * Patches 3-5 allow bpf programs to integrate with rstat by adding the
>   necessary hook points and kfunc. A comprehensive selftest that
>   demonstrates the entire workflow for using bpf and rstat to
>   efficiently collect and output cgroup stats is added.
>
> ---
> Changelog:
> v8 -> v9:
> - Make UNSPEC (an invalid option) as the default order for cgroup_iter.
> - Use enum for specifying cgroup_iter order, instead of u32.
> - Add BPF_ITER_RESHCED to cgroup_iter.
> - Add cgroup_hierarchical_stats to s390x denylist.

What 'RESEND' is for?
It seems to confuse patchwork and BPF CI.

The v9 series made it to patchwork...

Please just bump the version to v10 next time.
Don't add things to subject, since automation cannot recognize
that yet.

  parent reply	other threads:[~2022-08-25  0:30 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-24 23:31 [RESEND PATCH bpf-next v9 0/5] bpf: rstat: cgroup hierarchical Hao Luo
2022-08-24 23:31 ` [RESEND PATCH bpf-next v9 1/5] bpf: Introduce cgroup iter Hao Luo
2022-08-25 23:00   ` Yonghong Song
2022-08-29 18:07     ` Hao Luo
2022-08-24 23:31 ` [RESEND PATCH bpf-next v9 2/5] selftests/bpf: Test cgroup_iter Hao Luo
2022-08-24 23:31 ` [RESEND PATCH bpf-next v9 3/5] cgroup: bpf: enable bpf programs to integrate with rstat Hao Luo
2022-08-24 23:31 ` [RESEND PATCH bpf-next v9 4/5] selftests/bpf: extend cgroup helpers Hao Luo
2022-08-24 23:31 ` [RESEND PATCH bpf-next v9 5/5] selftests/bpf: add a selftest for cgroup hierarchical stats collection Hao Luo
2022-08-25  2:09   ` Alexei Starovoitov
2022-08-25  2:41     ` Yosry Ahmed
2022-08-25 18:43       ` Alexei Starovoitov
2022-08-25 18:45         ` Yosry Ahmed
2022-08-25 18:52         ` Hao Luo
2022-08-25  0:29 ` Alexei Starovoitov [this message]
2022-08-25  0:42   ` [RESEND PATCH bpf-next v9 0/5] bpf: rstat: cgroup hierarchical Hao Luo
2022-08-25  0:47     ` Alexei Starovoitov
2022-08-25  0:48       ` Hao Luo
2022-08-25 18:40 ` patchwork-bot+netdevbpf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAADnVQLT3JE8LtOYrs30mL88PNs+NaSeXgQqAPEAup5LUC+BPQ@mail.gmail.com \
    --to=alexei.starovoitov@gmail.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=cgroups@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=hannes@cmpxchg.org \
    --cc=haoluo@google.com \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kpsingh@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizefan.x@bytedance.com \
    --cc=martin.lau@linux.dev \
    --cc=mhocko@kernel.org \
    --cc=mkoutny@suse.com \
    --cc=netdev@vger.kernel.org \
    --cc=rientjes@google.com \
    --cc=roman.gushchin@linux.dev \
    --cc=sdf@google.com \
    --cc=shakeelb@google.com \
    --cc=song@kernel.org \
    --cc=tj@kernel.org \
    --cc=yhs@fb.com \
    --cc=yosryahmed@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).