From: Peter Zijlstra <peterz@infradead.org>
To: joel@joelfernandes.org, chris.hyser@oracle.com,
joshdon@google.com, mingo@kernel.org, vincent.guittot@linaro.org,
valentin.schneider@arm.com, mgorman@suse.de
Cc: linux-kernel@vger.kernel.org, peterz@infradead.org,
tj@kernel.org, tglx@linutronix.de
Subject: [PATCH 1/9] sched: Allow sched_core_put() from atomic context
Date: Thu, 01 Apr 2021 15:10:13 +0200 [thread overview]
Message-ID: <20210401133917.111845286@infradead.org> (raw)
In-Reply-To: 20210401131012.395311786@infradead.org
Stuff the meat of sched_core_put() into a work such that we can use
sched_core_put() from atomic context.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
kernel/sched/core.c | 33 +++++++++++++++++++++++++++------
1 file changed, 27 insertions(+), 6 deletions(-)
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -223,7 +223,7 @@ static struct task_struct *sched_core_ne
*/
static DEFINE_MUTEX(sched_core_mutex);
-static int sched_core_count;
+static atomic_t sched_core_count;
static struct cpumask sched_core_mask;
static void __sched_core_flip(bool enabled)
@@ -286,18 +286,39 @@ static void __sched_core_disable(void)
void sched_core_get(void)
{
+ if (atomic_inc_not_zero(&sched_core_count))
+ return;
+
mutex_lock(&sched_core_mutex);
- if (!sched_core_count++)
+ if (!atomic_read(&sched_core_count))
__sched_core_enable();
+
+ smp_mb__before_atomic();
+ atomic_inc(&sched_core_count);
mutex_unlock(&sched_core_mutex);
}
-void sched_core_put(void)
+static void __sched_core_put(struct work_struct *work)
{
- mutex_lock(&sched_core_mutex);
- if (!--sched_core_count)
+ if (atomic_dec_and_mutex_lock(&sched_core_count, &sched_core_mutex)) {
__sched_core_disable();
- mutex_unlock(&sched_core_mutex);
+ mutex_unlock(&sched_core_mutex);
+ }
+}
+
+void sched_core_put(void)
+{
+ static DECLARE_WORK(_work, __sched_core_put);
+
+ /*
+ * "There can only be one"
+ *
+ * Either this is the last one, or we don't actually need to do any
+ * 'work'. If it is the last *again*, we rely on
+ * WORK_STRUCT_PENDING_BIT.
+ */
+ if (!atomic_add_unless(&sched_core_count, -1, 1))
+ schedule_work(&_work);
}
#else /* !CONFIG_SCHED_CORE */
next prev parent reply other threads:[~2021-04-01 17:51 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-01 13:10 [PATCH 0/9] sched: Core scheduling interfaces Peter Zijlstra
2021-04-01 13:10 ` Peter Zijlstra [this message]
2021-04-01 13:10 ` [PATCH 2/9] sched: Implement core-sched assertions Peter Zijlstra
2021-04-01 13:10 ` [PATCH 3/9] sched: Trivial core scheduling cookie management Peter Zijlstra
2021-04-01 20:04 ` Josh Don
2021-04-02 7:13 ` Peter Zijlstra
2021-04-01 13:10 ` [PATCH 4/9] sched: Default core-sched policy Peter Zijlstra
2021-04-21 13:33 ` Peter Zijlstra
2021-04-21 14:31 ` Chris Hyser
2021-04-01 13:10 ` [PATCH 5/9] sched: prctl() core-scheduling interface Peter Zijlstra
2021-04-07 17:00 ` Peter Zijlstra
2021-04-18 3:52 ` Joel Fernandes
2021-04-01 13:10 ` [PATCH 6/9] kselftest: Add test for core sched prctl interface Peter Zijlstra
2021-04-01 13:10 ` [PATCH 7/9] sched: Cgroup core-scheduling interface Peter Zijlstra
2021-04-02 0:34 ` Josh Don
2021-04-01 13:10 ` [PATCH 8/9] rbtree: Remove const from the rb_find_add() comparator Peter Zijlstra
2021-04-01 13:10 ` [PATCH 9/9] sched: prctl() and cgroup interaction Peter Zijlstra
2021-04-03 1:30 ` Josh Don
2021-04-06 15:12 ` Peter Zijlstra
2021-04-04 23:39 ` [PATCH 0/9] sched: Core scheduling interfaces Tejun Heo
2021-04-05 18:46 ` Joel Fernandes
2021-04-06 14:16 ` Tejun Heo
2021-04-18 1:35 ` Joel Fernandes
2021-04-19 9:00 ` Peter Zijlstra
2021-04-21 13:35 ` Peter Zijlstra
2021-04-21 14:45 ` Chris Hyser
2021-04-06 15:32 ` Peter Zijlstra
2021-04-06 16:08 ` Tejun Heo
2021-04-07 18:39 ` Peter Zijlstra
2021-04-07 16:50 ` Michal Koutný
2021-04-07 18:34 ` Peter Zijlstra
2021-04-08 13:25 ` Michal Koutný
2021-04-08 15:02 ` Peter Zijlstra
2021-04-09 0:16 ` Josh Don
2021-04-19 11:30 ` Tejun Heo
2021-04-20 1:17 ` Josh Don
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210401133917.111845286@infradead.org \
--to=peterz@infradead.org \
--cc=chris.hyser@oracle.com \
--cc=joel@joelfernandes.org \
--cc=joshdon@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@kernel.org \
--cc=tglx@linutronix.de \
--cc=tj@kernel.org \
--cc=valentin.schneider@arm.com \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).