From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
To: linux-kernel@vger.kernel.org, io-uring@vger.kernel.org
Cc: Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Juri Lelli <juri.lelli@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
Jens Axboe <axboe@kernel.dk>,
Thomas Gleixner <tglx@linutronix.de>
Subject: [RFC PATCH] sched: Invoke io_wq_worker_sleeping() with enabled preemption
Date: Wed, 19 Aug 2020 14:37:58 +0200 [thread overview]
Message-ID: <20200819123758.6v45rj2gvojddsnn@linutronix.de> (raw)
During a context switch the scheduler invokes wq_worker_sleeping() with
disabled preemption. Disabling preemption is needed because it protects
access to `worker->sleeping'. As an optimisation it avoids invoking
schedule() within the schedule path as part of possible wake up (thus
preempt_enable_no_resched() afterwards).
The io-wq has been added to the mix in the same section with disabled
preemption. This breaks on PREEMPT_RT because io_wq_worker_sleeping()
acquires a spinlock_t. Also within the schedule() the spinlock_t must be
acquired after tsk_is_pi_blocked() otherwise it will block on the sleeping lock
again while scheduling out.
While playing with `io_uring-bench' I didn't notice a significant
latency spike after converting io_wqe::lock to a raw_spinlock_t. The
latency was more or less the same.
I don't see a significant reason why this lock should become a
raw_spinlock_t therefore I suggest to move it after the
tsk_is_pi_blocked() check.
The io_worker::flags are usually modified under the lock except in the
scheduler path. Ideally the lock is always acquired since the
IO_WORKER_F_UP flag is set early in the startup and IO_WORKER_F_RUNNING
should be set unless the task loops within schedule(). I *think* ::flags
requires the same protection like workqueue's ::sleeping and therefore I
move the check within the locked section.
Any feedback on this vs raw_spinlock_t?
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
fs/io-wq.c | 8 ++++----
kernel/sched/core.c | 10 +++++-----
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/fs/io-wq.c b/fs/io-wq.c
index e92c4724480ca..a7e07b3ac5b95 100644
--- a/fs/io-wq.c
+++ b/fs/io-wq.c
@@ -623,15 +623,15 @@ void io_wq_worker_sleeping(struct task_struct *tsk)
struct io_worker *worker = kthread_data(tsk);
struct io_wqe *wqe = worker->wqe;
+ spin_lock_irq(&wqe->lock);
if (!(worker->flags & IO_WORKER_F_UP))
- return;
+ goto out;
if (!(worker->flags & IO_WORKER_F_RUNNING))
- return;
+ goto out;
worker->flags &= ~IO_WORKER_F_RUNNING;
-
- spin_lock_irq(&wqe->lock);
io_wqe_dec_running(wqe, worker);
+out:
spin_unlock_irq(&wqe->lock);
}
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 3bbb60b97c73c..b76c0f27bd95e 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4694,18 +4694,18 @@ static inline void sched_submit_work(struct task_struct *tsk)
* in the possible wakeup of a kworker and because wq_worker_sleeping()
* requires it.
*/
- if (tsk->flags & (PF_WQ_WORKER | PF_IO_WORKER)) {
+ if (tsk->flags & PF_WQ_WORKER) {
preempt_disable();
- if (tsk->flags & PF_WQ_WORKER)
- wq_worker_sleeping(tsk);
- else
- io_wq_worker_sleeping(tsk);
+ wq_worker_sleeping(tsk);
preempt_enable_no_resched();
}
if (tsk_is_pi_blocked(tsk))
return;
+ if (tsk->flags & PF_IO_WORKER)
+ io_wq_worker_sleeping(tsk);
+
/*
* If we are going to sleep and we have plugged IO queued,
* make sure to submit it to avoid deadlocks.
--
2.28.0
next reply other threads:[~2020-08-19 12:38 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-19 12:37 Sebastian Andrzej Siewior [this message]
2020-08-19 13:15 ` [RFC PATCH] sched: Invoke io_wq_worker_sleeping() with enabled preemption peterz
2020-08-19 13:18 ` Jens Axboe
2020-08-19 19:44 ` [PATCH] io_wq: Make io_wqe::lock a raw_spinlock_t Sebastian Andrzej Siewior
2020-09-01 8:41 ` [PATCH v2] " Sebastian Andrzej Siewior
2020-09-01 14:17 ` Jens Axboe
2020-08-19 13:33 ` [RFC PATCH] sched: Invoke io_wq_worker_sleeping() with enabled preemption Sebastian Andrzej Siewior
2020-08-19 14:21 ` peterz
2020-08-19 19:55 ` [PATCH 1/2] sched: Bring the PF_IO_WORKER and PF_WQ_WORKER bits closer together Sebastian Andrzej Siewior
2020-08-19 20:00 ` [PATCH 2/2] sched: Cache task_struct::flags in sched_submit_work() Sebastian Andrzej Siewior
2020-08-19 20:11 ` Peter Zijlstra
2020-09-07 12:58 ` [PATCH 1/2] sched: Bring the PF_IO_WORKER and PF_WQ_WORKER bits closer together Will Deacon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200819123758.6v45rj2gvojddsnn@linutronix.de \
--to=bigeasy@linutronix.de \
--cc=axboe@kernel.dk \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=io-uring@vger.kernel.org \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).