From: Yosry Ahmed <yosryahmed@google.com>
To: Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>,
Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
Yonghong Song <yhs@fb.com>,
John Fastabend <john.fastabend@gmail.com>,
KP Singh <kpsingh@kernel.org>, Hao Luo <haoluo@google.com>,
Tejun Heo <tj@kernel.org>, Zefan Li <lizefan.x@bytedance.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Shuah Khan <shuah@kernel.org>, Michal Hocko <mhocko@kernel.org>
Cc: Roman Gushchin <roman.gushchin@linux.dev>,
David Rientjes <rientjes@google.com>,
Stanislav Fomichev <sdf@google.com>,
Greg Thelen <gthelen@google.com>,
Shakeel Butt <shakeelb@google.com>,
linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
bpf@vger.kernel.org, cgroups@vger.kernel.org,
Yosry Ahmed <yosryahmed@google.com>
Subject: [PATCH bpf-next v2 6/8] cgroup: bpf: enable bpf programs to integrate with rstat
Date: Fri, 10 Jun 2022 19:44:33 +0000 [thread overview]
Message-ID: <20220610194435.2268290-7-yosryahmed@google.com> (raw)
In-Reply-To: <20220610194435.2268290-1-yosryahmed@google.com>
Enable bpf programs to make use of rstat to collect cgroup hierarchical
stats efficiently:
- Add cgroup_rstat_updated() kfunc, for bpf progs that collect stats.
- Add cgroup_rstat_flush() kfunc, for bpf progs that read stats.
- Add an empty bpf_rstat_flush() hook that is called during rstat
flushing, for bpf progs that flush stats to attach to. Attaching a bpf
prog to this hook effectively registers it as a flush callback.
Signed-off-by: Yosry Ahmed <yosryahmed@google.com>
---
kernel/cgroup/rstat.c | 46 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 46 insertions(+)
diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c
index 24b5c2ab55983..94140bd0d02a4 100644
--- a/kernel/cgroup/rstat.c
+++ b/kernel/cgroup/rstat.c
@@ -3,6 +3,11 @@
#include <linux/sched/cputime.h>
+#include <linux/bpf.h>
+#include <linux/btf.h>
+#include <linux/btf_ids.h>
+
+
static DEFINE_SPINLOCK(cgroup_rstat_lock);
static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_cpu_lock);
@@ -141,6 +146,23 @@ static struct cgroup *cgroup_rstat_cpu_pop_updated(struct cgroup *pos,
return pos;
}
+/*
+ * A hook for bpf stat collectors to attach to and flush their stats.
+ * Together with providing bpf kfuncs for cgroup_rstat_updated() and
+ * cgroup_rstat_flush(), this enables a complete workflow where bpf progs that
+ * collect cgroup stats can integrate with rstat for efficient flushing.
+ *
+ * A static noinline declaration here could cause the compiler to optimize away
+ * the function. A global noinline declaration will keep the definition, but may
+ * optimize away the callsite. Therefore, __weak is needed to ensure that the
+ * call is still emitted, by telling the compiler that we don't know what the
+ * function might eventually be.
+ */
+__weak noinline void bpf_rstat_flush(struct cgroup *cgrp,
+ struct cgroup *parent, int cpu)
+{
+}
+
/* see cgroup_rstat_flush() */
static void cgroup_rstat_flush_locked(struct cgroup *cgrp, bool may_sleep)
__releases(&cgroup_rstat_lock) __acquires(&cgroup_rstat_lock)
@@ -168,6 +190,7 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp, bool may_sleep)
struct cgroup_subsys_state *css;
cgroup_base_stat_flush(pos, cpu);
+ bpf_rstat_flush(pos, cgroup_parent(pos), cpu);
rcu_read_lock();
list_for_each_entry_rcu(css, &pos->rstat_css_list,
@@ -469,3 +492,26 @@ void cgroup_base_stat_cputime_show(struct seq_file *seq)
"system_usec %llu\n",
usage, utime, stime);
}
+
+/* Add bpf kfuncs for cgroup_rstat_updated() and cgroup_rstat_flush() */
+BTF_SET_START(bpf_rstat_check_kfunc_ids)
+BTF_ID(func, cgroup_rstat_updated)
+BTF_ID(func, cgroup_rstat_flush)
+BTF_SET_END(bpf_rstat_check_kfunc_ids)
+
+BTF_SET_START(bpf_rstat_sleepable_kfunc_ids)
+BTF_ID(func, cgroup_rstat_flush)
+BTF_SET_END(bpf_rstat_sleepable_kfunc_ids)
+
+static const struct btf_kfunc_id_set bpf_rstat_kfunc_set = {
+ .owner = THIS_MODULE,
+ .check_set = &bpf_rstat_check_kfunc_ids,
+ .sleepable_set = &bpf_rstat_sleepable_kfunc_ids,
+};
+
+static int __init bpf_rstat_kfunc_init(void)
+{
+ return register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING,
+ &bpf_rstat_kfunc_set);
+}
+late_initcall(bpf_rstat_kfunc_init);
--
2.36.1.476.g0c4daa206d-goog
next prev parent reply other threads:[~2022-06-10 19:45 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-10 19:44 [PATCH bpf-next v2 0/8] bpf: rstat: cgroup hierarchical stats Yosry Ahmed
2022-06-10 19:44 ` [PATCH bpf-next v2 1/8] cgroup: enable cgroup_get_from_file() on cgroup1 Yosry Ahmed
2022-06-10 19:44 ` [PATCH bpf-next v2 2/8] cgroup: Add cgroup_put() in !CONFIG_CGROUPS case Yosry Ahmed
2022-06-10 19:44 ` [PATCH bpf-next v2 3/8] bpf, iter: Fix the condition on p when calling stop Yosry Ahmed
2022-06-20 18:48 ` Yonghong Song
2022-06-21 7:25 ` Hao Luo
2022-06-24 17:46 ` Yonghong Song
2022-06-24 18:23 ` Yosry Ahmed
2022-06-10 19:44 ` [PATCH bpf-next v2 4/8] bpf: Introduce cgroup iter Yosry Ahmed
2022-06-11 6:23 ` kernel test robot
2022-06-11 7:34 ` kernel test robot
2022-06-11 12:44 ` kernel test robot
2022-06-11 12:55 ` kernel test robot
2022-06-28 4:09 ` Yonghong Song
2022-06-28 6:06 ` Yosry Ahmed
2022-07-07 23:33 ` Hao Luo
2022-06-28 4:14 ` Yonghong Song
2022-06-28 6:03 ` Yosry Ahmed
2022-06-28 17:03 ` Yonghong Song
2022-07-07 23:36 ` Hao Luo
2022-06-10 19:44 ` [PATCH bpf-next v2 5/8] selftests/bpf: Test cgroup_iter Yosry Ahmed
2022-06-28 6:11 ` Yonghong Song
2022-06-10 19:44 ` Yosry Ahmed [this message]
2022-06-10 20:52 ` [PATCH bpf-next v2 6/8] cgroup: bpf: enable bpf programs to integrate with rstat kernel test robot
2022-06-10 21:22 ` kernel test robot
2022-06-10 21:30 ` Yosry Ahmed
2022-06-11 19:57 ` Alexei Starovoitov
2022-06-13 17:05 ` Yosry Ahmed
2022-06-11 10:22 ` kernel test robot
2022-06-28 6:12 ` Yonghong Song
2022-06-10 19:44 ` [PATCH bpf-next v2 7/8] selftests/bpf: extend cgroup helpers Yosry Ahmed
2022-06-10 19:44 ` [PATCH bpf-next v2 8/8] bpf: add a selftest for cgroup hierarchical stats collection Yosry Ahmed
2022-06-28 6:14 ` Yonghong Song
2022-06-28 6:47 ` Yosry Ahmed
2022-06-28 7:14 ` Yosry Ahmed
2022-06-29 0:09 ` Yosry Ahmed
2022-06-29 6:48 ` Yonghong Song
2022-06-29 8:04 ` Yosry Ahmed
2022-07-02 0:55 ` Yonghong Song
2022-07-06 21:29 ` Yosry Ahmed
2022-06-29 6:17 ` Yonghong Song
2022-06-28 7:43 ` Yosry Ahmed
2022-06-29 6:26 ` Yonghong Song
2022-06-29 8:03 ` Yosry Ahmed
2022-07-01 23:28 ` Hao Luo
2022-06-10 19:48 ` [PATCH bpf-next v2 0/8] bpf: rstat: cgroup hierarchical stats Yosry Ahmed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220610194435.2268290-7-yosryahmed@google.com \
--to=yosryahmed@google.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=cgroups@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=gthelen@google.com \
--cc=hannes@cmpxchg.org \
--cc=haoluo@google.com \
--cc=john.fastabend@gmail.com \
--cc=kafai@fb.com \
--cc=kpsingh@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan.x@bytedance.com \
--cc=mhocko@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=rientjes@google.com \
--cc=roman.gushchin@linux.dev \
--cc=sdf@google.com \
--cc=shakeelb@google.com \
--cc=shuah@kernel.org \
--cc=songliubraving@fb.com \
--cc=tj@kernel.org \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).