From: Frederic Weisbecker <frederic@kernel.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: LKML <linux-kernel@vger.kernel.org>, Mel Gorman <mgorman@suse.de>,
Michal Hocko <mhocko@kernel.org>,
Frederic Weisbecker <frederic@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
"Paul E . McKenney" <paulmck@kernel.org>,
Ingo Molnar <mingo@redhat.com>, Michal Hocko <mhocko@suse.com>
Subject: [RFC PATCH 6/8] preempt/dynamic: Provide preempt_schedule[_notrace]() static calls
Date: Mon, 18 Jan 2021 15:12:21 +0100 [thread overview]
Message-ID: <20210118141223.123667-7-frederic@kernel.org> (raw)
In-Reply-To: <20210118141223.123667-1-frederic@kernel.org>
From: "Peter Zijlstra (Intel)" <peterz@infradead.org>
Provide static calls to control preempt_schedule[_notrace]()
(called in CONFIG_PREEMPT) so that we can override their behaviour when
preempt= is overriden.
Since the default behaviour is full preemption, both their calls are
initialized to the arch provided wrapper, if any.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Paul E. McKenney <paulmck@kernel.org>
[only define static calls when PREEMPT_DYNAMIC, make it less dependent
on x86 with __preempt_schedule_func()]
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
---
arch/x86/include/asm/preempt.h | 34 ++++++++++++++++++++++++++--------
kernel/sched/core.c | 12 ++++++++++++
2 files changed, 38 insertions(+), 8 deletions(-)
diff --git a/arch/x86/include/asm/preempt.h b/arch/x86/include/asm/preempt.h
index 69485ca13665..3db9cb8b1a25 100644
--- a/arch/x86/include/asm/preempt.h
+++ b/arch/x86/include/asm/preempt.h
@@ -5,6 +5,7 @@
#include <asm/rmwcc.h>
#include <asm/percpu.h>
#include <linux/thread_info.h>
+#include <linux/static_call_types.h>
DECLARE_PER_CPU(int, __preempt_count);
@@ -103,16 +104,33 @@ static __always_inline bool should_resched(int preempt_offset)
}
#ifdef CONFIG_PREEMPTION
- extern asmlinkage void preempt_schedule_thunk(void);
-# define __preempt_schedule() \
- asm volatile ("call preempt_schedule_thunk" : ASM_CALL_CONSTRAINT)
- extern asmlinkage void preempt_schedule(void);
- extern asmlinkage void preempt_schedule_notrace_thunk(void);
-# define __preempt_schedule_notrace() \
- asm volatile ("call preempt_schedule_notrace_thunk" : ASM_CALL_CONSTRAINT)
+extern asmlinkage void preempt_schedule(void);
+extern asmlinkage void preempt_schedule_thunk(void);
+
+#define __preempt_schedule_func() preempt_schedule_thunk
+
+DECLARE_STATIC_CALL(preempt_schedule, __preempt_schedule_func());
+
+#define __preempt_schedule() \
+do { \
+ __ADDRESSABLE(STATIC_CALL_KEY(preempt_schedule)); \
+ asm volatile ("call " STATIC_CALL_TRAMP_STR(preempt_schedule) : ASM_CALL_CONSTRAINT); \
+} while (0)
+
+extern asmlinkage void preempt_schedule_notrace(void);
+extern asmlinkage void preempt_schedule_notrace_thunk(void);
+
+#define __preempt_schedule_notrace_func() preempt_schedule_notrace_thunk
+
+DECLARE_STATIC_CALL(preempt_schedule_notrace, __preempt_schedule_notrace_func());
+
+#define __preempt_schedule_notrace() \
+do { \
+ __ADDRESSABLE(STATIC_CALL_KEY(preempt_schedule_notrace)); \
+ asm volatile ("call " STATIC_CALL_TRAMP_STR(preempt_schedule_notrace) : ASM_CALL_CONSTRAINT); \
+} while (0)
- extern asmlinkage void preempt_schedule_notrace(void);
#endif
#endif /* __ASM_PREEMPT_H */
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index d6de12b4eef2..faff4b546c5f 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5251,6 +5251,12 @@ asmlinkage __visible void __sched notrace preempt_schedule(void)
NOKPROBE_SYMBOL(preempt_schedule);
EXPORT_SYMBOL(preempt_schedule);
+#ifdef CONFIG_PREEMPT_DYNAMIC
+DEFINE_STATIC_CALL(preempt_schedule, __preempt_schedule_func());
+EXPORT_STATIC_CALL(preempt_schedule);
+#endif
+
+
/**
* preempt_schedule_notrace - preempt_schedule called by tracing
*
@@ -5303,6 +5309,12 @@ asmlinkage __visible void __sched notrace preempt_schedule_notrace(void)
}
EXPORT_SYMBOL_GPL(preempt_schedule_notrace);
+#ifdef CONFIG_PREEMPT_DYNAMIC
+DEFINE_STATIC_CALL(preempt_schedule_notrace, __preempt_schedule_notrace_func());
+EXPORT_STATIC_CALL(preempt_schedule_notrace);
+#endif
+
+
#endif /* CONFIG_PREEMPTION */
/*
--
2.25.1
next prev parent reply other threads:[~2021-01-18 14:15 UTC|newest]
Thread overview: 59+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-18 14:12 [RFC PATCH 0/7] preempt: Tune preemption flavour on boot v4 Frederic Weisbecker
2021-01-18 14:12 ` [RFC PATCH 1/8] static_call/x86: Add __static_call_return0() Frederic Weisbecker
2021-02-08 12:00 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-02-17 13:17 ` tip-bot2 for Peter Zijlstra
2021-01-18 14:12 ` [RFC PATCH 2/8] static_call: Provide DEFINE_STATIC_CALL_RET0() Frederic Weisbecker
2021-02-08 12:00 ` [tip: sched/core] " tip-bot2 for Frederic Weisbecker
2021-02-17 13:17 ` tip-bot2 for Frederic Weisbecker
2021-01-18 14:12 ` [RFC PATCH 3/8] static_call: Pull some static_call declarations to the type headers Frederic Weisbecker
2021-01-19 10:46 ` Jürgen Groß
2021-02-08 12:00 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-02-17 13:17 ` tip-bot2 for Peter Zijlstra
2021-01-18 14:12 ` [RFC PATCH 4/8] preempt: Introduce CONFIG_PREEMPT_DYNAMIC Frederic Weisbecker
2021-01-22 16:53 ` Peter Zijlstra
2021-01-28 12:17 ` Frederic Weisbecker
2021-02-08 12:00 ` [tip: sched/core] " tip-bot2 for Michal Hocko
2021-02-17 13:17 ` tip-bot2 for Michal Hocko
2021-01-18 14:12 ` [RFC PATCH 5/8] preempt/dynamic: Provide cond_resched() and might_resched() static calls Frederic Weisbecker
2021-02-08 12:00 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra (Intel)
2021-02-17 13:17 ` tip-bot2 for Peter Zijlstra (Intel)
2021-01-18 14:12 ` Frederic Weisbecker [this message]
2021-01-21 21:58 ` [RFC PATCH 6/8] preempt/dynamic: Provide preempt_schedule[_notrace]() " Peter Zijlstra
2021-01-21 22:25 ` Peter Zijlstra
2021-01-22 16:52 ` Peter Zijlstra
2021-01-22 16:57 ` Ard Biesheuvel
2021-01-22 17:08 ` Peter Zijlstra
2021-02-08 12:00 ` [tip: sched/core] sched: Add /debug/sched_preempt tip-bot2 for Peter Zijlstra
2021-02-09 15:45 ` tip-bot2 for Peter Zijlstra
2021-02-17 13:17 ` tip-bot2 for Peter Zijlstra
2021-01-25 23:40 ` [RFC PATCH 6/8] preempt/dynamic: Provide preempt_schedule[_notrace]() static calls Josh Poimboeuf
2021-01-26 9:24 ` Peter Zijlstra
2021-01-26 23:57 ` Josh Poimboeuf
2021-01-27 9:13 ` Peter Zijlstra
2021-01-27 11:27 ` Peter Zijlstra
2021-01-27 15:59 ` Josh Poimboeuf
2021-01-27 16:19 ` Peter Zijlstra
2021-01-27 16:33 ` Josh Poimboeuf
2021-01-27 18:44 ` Peter Zijlstra
2021-01-27 19:00 ` Josh Poimboeuf
2021-01-27 19:02 ` Josh Poimboeuf
2021-01-27 23:18 ` Josh Poimboeuf
2021-02-03 14:04 ` Peter Zijlstra
2021-02-05 15:30 ` Peter Zijlstra
2021-02-06 2:31 ` Josh Poimboeuf
2021-02-06 9:03 ` Peter Zijlstra
2021-02-05 15:22 ` Peter Zijlstra
2021-02-08 12:00 ` [tip: sched/core] static_call: Allow module use without exposing static_call_key tip-bot2 for Josh Poimboeuf
2021-02-09 15:45 ` tip-bot2 for Josh Poimboeuf
2021-02-17 13:17 ` tip-bot2 for Josh Poimboeuf
2021-02-08 12:00 ` [tip: sched/core] preempt/dynamic: Provide preempt_schedule[_notrace]() static calls tip-bot2 for Peter Zijlstra (Intel)
2021-02-17 13:17 ` tip-bot2 for Peter Zijlstra (Intel)
2021-01-18 14:12 ` [RFC PATCH 7/8] preempt/dynamic: Provide irqentry_exit_cond_resched() static call Frederic Weisbecker
2021-02-08 12:00 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra (Intel)
2021-02-17 13:17 ` tip-bot2 for Peter Zijlstra (Intel)
2021-01-18 14:12 ` [RFC PATCH 8/8] preempt/dynamic: Support dynamic preempt with preempt= boot option Frederic Weisbecker
2021-02-08 12:00 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra (Intel)
2021-02-09 15:45 ` tip-bot2 for Peter Zijlstra (Intel)
2021-02-17 13:17 ` tip-bot2 for Peter Zijlstra (Intel)
2021-01-21 21:22 ` [RFC PATCH 0/7] preempt: Tune preemption flavour on boot v4 Peter Zijlstra
2021-01-22 15:02 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210118141223.123667-7-frederic@kernel.org \
--to=frederic@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mhocko@kernel.org \
--cc=mhocko@suse.com \
--cc=mingo@redhat.com \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).