From: Roman Gushchin <guro@fb.com>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>
Cc: Mel Gorman <mgorman@techsingularity.net>, <bpf@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, Roman Gushchin <guro@fb.com>
Subject: [PATCH rfc 3/6] bpf: sched: introduce bpf_sched_enable()
Date: Thu, 16 Sep 2021 09:24:48 -0700 [thread overview]
Message-ID: <20210916162451.709260-4-guro@fb.com> (raw)
In-Reply-To: <20210916162451.709260-1-guro@fb.com>
Introduce a dedicated static key and the bpf_sched_enabled() wrapper
to guard all invocations of bpf programs in the scheduler code.
It will help to avoid any potential performance regression in a case
when no scheduler bpf programs are attached.
Signed-off-by: Roman Gushchin <guro@fb.com>
---
include/linux/bpf_sched.h | 24 ++++++++++++++++++++++++
kernel/bpf/syscall.c | 7 +++++++
kernel/sched/bpf_sched.c | 2 ++
3 files changed, 33 insertions(+)
diff --git a/include/linux/bpf_sched.h b/include/linux/bpf_sched.h
index 0f8d3dae53df..6e773aecdff7 100644
--- a/include/linux/bpf_sched.h
+++ b/include/linux/bpf_sched.h
@@ -6,6 +6,8 @@
#ifdef CONFIG_BPF_SYSCALL
+#include <linux/jump_label.h>
+
#define BPF_SCHED_HOOK(RET, DEFAULT, NAME, ...) \
RET bpf_sched_##NAME(__VA_ARGS__);
#include <linux/sched_hook_defs.h>
@@ -14,6 +16,23 @@
int bpf_sched_verify_prog(struct bpf_verifier_log *vlog,
const struct bpf_prog *prog);
+DECLARE_STATIC_KEY_FALSE(bpf_sched_enabled_key);
+
+static inline bool bpf_sched_enabled(void)
+{
+ return static_branch_unlikely(&bpf_sched_enabled_key);
+}
+
+static inline void bpf_sched_inc(void)
+{
+ static_branch_inc(&bpf_sched_enabled_key);
+}
+
+static inline void bpf_sched_dec(void)
+{
+ static_branch_dec(&bpf_sched_enabled_key);
+}
+
#else /* CONFIG_BPF_SYSCALL */
#define BPF_SCHED_HOOK(RET, DEFAULT, NAME, ...) \
@@ -23,6 +42,11 @@ static inline RET bpf_sched_##NAME(__VA_ARGS__) \
}
#undef BPF_SCHED_HOOK
+static inline bool bpf_sched_enabled(void)
+{
+ return false;
+}
+
#endif /* CONFIG_BPF_SYSCALL */
#endif /* _BPF_CGROUP_H */
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 67e062376f22..aa5565110498 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -31,6 +31,7 @@
#include <linux/bpf-netns.h>
#include <linux/rcupdate_trace.h>
#include <linux/memcontrol.h>
+#include <linux/bpf_sched.h>
#define IS_FD_ARRAY(map) ((map)->map_type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || \
(map)->map_type == BPF_MAP_TYPE_CGROUP_ARRAY || \
@@ -2602,6 +2603,9 @@ static void bpf_tracing_link_release(struct bpf_link *link)
struct bpf_tracing_link *tr_link =
container_of(link, struct bpf_tracing_link, link);
+ if (link->prog->type == BPF_PROG_TYPE_SCHED)
+ bpf_sched_dec();
+
WARN_ON_ONCE(bpf_trampoline_unlink_prog(link->prog,
tr_link->trampoline));
@@ -2804,6 +2808,9 @@ static int bpf_tracing_prog_attach(struct bpf_prog *prog,
goto out_unlock;
}
+ if (prog->type == BPF_PROG_TYPE_SCHED)
+ bpf_sched_inc();
+
link->tgt_prog = tgt_prog;
link->trampoline = tr;
diff --git a/kernel/sched/bpf_sched.c b/kernel/sched/bpf_sched.c
index ead691dc6e85..bf92cfb5ecf4 100644
--- a/kernel/sched/bpf_sched.c
+++ b/kernel/sched/bpf_sched.c
@@ -6,6 +6,8 @@
#include <linux/btf_ids.h>
#include "sched.h"
+DEFINE_STATIC_KEY_FALSE(bpf_sched_enabled_key);
+
/*
* For every hook declare a nop function where a BPF program can be attached.
*/
--
2.31.1
next prev parent reply other threads:[~2021-09-16 16:27 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20210915213550.3696532-1-guro@fb.com>
2021-09-16 0:19 ` [PATCH rfc 0/6] Scheduler BPF Hao Luo
2021-09-16 1:42 ` Roman Gushchin
2021-09-16 16:24 ` Roman Gushchin
2021-09-16 16:24 ` [PATCH rfc 1/6] bpf: sched: basic infrastructure for scheduler bpf Roman Gushchin
2021-09-16 16:24 ` [PATCH rfc 2/6] bpf: sched: add convenient helpers to identify sched entities Roman Gushchin
2021-11-25 6:09 ` Yafang Shao
2021-11-26 19:50 ` Roman Gushchin
2021-09-16 16:24 ` Roman Gushchin [this message]
2021-09-16 16:24 ` [PATCH rfc 4/6] sched: cfs: add bpf hooks to control wakeup and tick preemption Roman Gushchin
2021-10-01 3:35 ` Barry Song
2021-10-02 0:13 ` Roman Gushchin
2021-09-16 16:24 ` [PATCH rfc 5/6] libbpf: add support for scheduler bpf programs Roman Gushchin
2021-09-16 16:24 ` [PATCH rfc 6/6] bpftool: recognize scheduler programs Roman Gushchin
2021-09-16 16:36 ` [PATCH rfc 0/6] Scheduler BPF Roman Gushchin
2021-10-06 16:39 ` Qais Yousef
2021-10-06 18:50 ` Roman Gushchin
2021-10-11 16:38 ` Qais Yousef
2021-10-11 18:09 ` Roman Gushchin
2021-10-12 10:16 ` Qais Yousef
[not found] ` <52EC1E80-4C89-43AD-8A59-8ACA184EAE53@gmail.com>
2021-11-25 6:00 ` Yafang Shao
2021-11-26 19:46 ` Roman Gushchin
2022-01-15 8:29 ` Huichun Feng
2022-01-18 22:54 ` Roman Gushchin
2022-07-19 13:05 ` Ren Zhijie
2022-07-19 13:17 ` Ren Zhijie
2022-07-19 23:21 ` Roman Gushchin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210916162451.709260-4-guro@fb.com \
--to=guro@fb.com \
--cc=bpf@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).