From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
To: peterz@infradead.org
Cc: linux-kernel@vger.kernel.org, io-uring@vger.kernel.org,
Ingo Molnar <mingo@redhat.com>,
Juri Lelli <juri.lelli@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
Jens Axboe <axboe@kernel.dk>,
Thomas Gleixner <tglx@linutronix.de>
Subject: [PATCH 2/2] sched: Cache task_struct::flags in sched_submit_work()
Date: Wed, 19 Aug 2020 22:00:25 +0200 [thread overview]
Message-ID: <20200819200025.lqvmyefqnbok5i4f@linutronix.de> (raw)
In-Reply-To: <20200819195505.y3fxk72sotnrkczi@linutronix.de>
sched_submit_work() is considered to be a hot path. The preempt_disable()
instruction is a compiler barrier and forces the compiler to load
task_struct::flags for the second comparison.
By using a local variable, the compiler can load the value once and keep it in
a register for the second comparison.
Verified on x86-64 with gcc-10.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
Optimisation at molecule level, part two. Drop this in case this branch
isn't consider *that* hot and the cache hot value can be loaded again.
But then the value is around and be speculated early on :)
kernel/sched/core.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 8471a0f7eb322..c36dc1ae58beb 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4551,9 +4551,12 @@ void __noreturn do_task_dead(void)
static inline void sched_submit_work(struct task_struct *tsk)
{
+ unsigned int task_flags;
+
if (!tsk->state)
return;
+ task_flags = tsk->flags;
/*
* If a worker went to sleep, notify and ask workqueue whether
* it wants to wake up a task to maintain concurrency.
@@ -4562,9 +4565,9 @@ static inline void sched_submit_work(struct task_struct *tsk)
* in the possible wakeup of a kworker and because wq_worker_sleeping()
* requires it.
*/
- if (tsk->flags & (PF_WQ_WORKER | PF_IO_WORKER)) {
+ if (task_flags & (PF_WQ_WORKER | PF_IO_WORKER)) {
preempt_disable();
- if (tsk->flags & PF_WQ_WORKER)
+ if (task_flags & PF_WQ_WORKER)
wq_worker_sleeping(tsk);
else
io_wq_worker_sleeping(tsk);
--
2.28.0
next prev parent reply other threads:[~2020-08-19 20:00 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-19 12:37 [RFC PATCH] sched: Invoke io_wq_worker_sleeping() with enabled preemption Sebastian Andrzej Siewior
2020-08-19 13:15 ` peterz
2020-08-19 13:18 ` Jens Axboe
2020-08-19 19:44 ` [PATCH] io_wq: Make io_wqe::lock a raw_spinlock_t Sebastian Andrzej Siewior
2020-09-01 8:41 ` [PATCH v2] " Sebastian Andrzej Siewior
2020-09-01 14:17 ` Jens Axboe
2020-08-19 13:33 ` [RFC PATCH] sched: Invoke io_wq_worker_sleeping() with enabled preemption Sebastian Andrzej Siewior
2020-08-19 14:21 ` peterz
2020-08-19 19:55 ` [PATCH 1/2] sched: Bring the PF_IO_WORKER and PF_WQ_WORKER bits closer together Sebastian Andrzej Siewior
2020-08-19 20:00 ` Sebastian Andrzej Siewior [this message]
2020-08-19 20:11 ` [PATCH 2/2] sched: Cache task_struct::flags in sched_submit_work() Peter Zijlstra
2020-08-27 7:54 ` [tip: sched/core] " tip-bot2 for Sebastian Andrzej Siewior
2020-08-27 7:54 ` [tip: sched/core] sched: Bring the PF_IO_WORKER and PF_WQ_WORKER bits closer together tip-bot2 for Sebastian Andrzej Siewior
2020-09-07 12:58 ` [PATCH 1/2] " Will Deacon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200819200025.lqvmyefqnbok5i4f@linutronix.de \
--to=bigeasy@linutronix.de \
--cc=axboe@kernel.dk \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=io-uring@vger.kernel.org \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.