From: Pavel Begunkov <asml.silence@gmail.com>
To: Jens Axboe <axboe@kernel.dk>, io-uring@vger.kernel.org
Subject: [PATCH 07/23] io_uring: extract a helper for ctx quiesce
Date: Thu, 29 Jul 2021 16:05:34 +0100 [thread overview]
Message-ID: <f466e8502993fcfafefeea0895660e197c1a8ba0.1627570633.git.asml.silence@gmail.com> (raw)
In-Reply-To: <cover.1627570633.git.asml.silence@gmail.com>
Refactor __io_uring_register() by extracting a helper responsible for
ctx queisce. Looks better and will make it easier to add more
optimisations.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
fs/io_uring.c | 53 ++++++++++++++++++++++++++++-----------------------
1 file changed, 29 insertions(+), 24 deletions(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c
index 85cd8c3a33e1..db43aedcfc42 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -10066,6 +10066,33 @@ static bool io_register_op_must_quiesce(int op)
}
}
+static int io_ctx_quiesce(struct io_ring_ctx *ctx)
+{
+ long ret;
+
+ percpu_ref_kill(&ctx->refs);
+
+ /*
+ * Drop uring mutex before waiting for references to exit. If another
+ * thread is currently inside io_uring_enter() it might need to grab the
+ * uring_lock to make progress. If we hold it here across the drain
+ * wait, then we can deadlock. It's safe to drop the mutex here, since
+ * no new references will come in after we've killed the percpu ref.
+ */
+ mutex_unlock(&ctx->uring_lock);
+ do {
+ ret = wait_for_completion_interruptible(&ctx->ref_comp);
+ if (!ret)
+ break;
+ ret = io_run_task_work_sig();
+ } while (ret >= 0);
+ mutex_lock(&ctx->uring_lock);
+
+ if (ret)
+ io_refs_resurrect(&ctx->refs, &ctx->ref_comp);
+ return ret;
+}
+
static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode,
void __user *arg, unsigned nr_args)
__releases(ctx->uring_lock)
@@ -10090,31 +10117,9 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode,
}
if (io_register_op_must_quiesce(opcode)) {
- percpu_ref_kill(&ctx->refs);
-
- /*
- * Drop uring mutex before waiting for references to exit. If
- * another thread is currently inside io_uring_enter() it might
- * need to grab the uring_lock to make progress. If we hold it
- * here across the drain wait, then we can deadlock. It's safe
- * to drop the mutex here, since no new references will come in
- * after we've killed the percpu ref.
- */
- mutex_unlock(&ctx->uring_lock);
- do {
- ret = wait_for_completion_interruptible(&ctx->ref_comp);
- if (!ret)
- break;
- ret = io_run_task_work_sig();
- if (ret < 0)
- break;
- } while (1);
- mutex_lock(&ctx->uring_lock);
-
- if (ret) {
- io_refs_resurrect(&ctx->refs, &ctx->ref_comp);
+ ret = io_ctx_quiesce(ctx);
+ if (ret)
return ret;
- }
}
switch (opcode) {
--
2.32.0
next prev parent reply other threads:[~2021-07-29 15:06 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-29 15:05 [PATCH for-next 00/23] first 5.15 batch Pavel Begunkov
2021-07-29 15:05 ` [PATCH 01/23] io_uring: remove unnecessary PF_EXITING check Pavel Begunkov
2021-07-29 15:05 ` [PATCH 02/23] io-wq: improve wq_list_add_tail() Pavel Begunkov
2021-07-29 15:05 ` [PATCH 03/23] io_uring: refactor io_alloc_req Pavel Begunkov
2021-07-29 15:05 ` [PATCH 04/23] io_uring: don't halt iopoll too early Pavel Begunkov
2021-07-29 15:05 ` [PATCH 05/23] io_uring: add more locking annotations for submit Pavel Begunkov
2021-07-29 15:05 ` [PATCH 06/23] io_uring: optimise io_cqring_wait() hot path Pavel Begunkov
2021-07-29 15:05 ` Pavel Begunkov [this message]
2021-07-29 15:05 ` [PATCH 08/23] io_uring: move io_put_task() definition Pavel Begunkov
2021-07-29 15:05 ` [PATCH 09/23] io_uring: move io_rsrc_node_alloc() definition Pavel Begunkov
2021-07-29 15:05 ` [PATCH 10/23] io_uring: inline io_free_req_deferred Pavel Begunkov
2021-07-29 15:05 ` [PATCH 11/23] io_uring: deduplicate open iopoll check Pavel Begunkov
2021-07-29 15:05 ` [PATCH 12/23] io_uring: improve ctx hang handling Pavel Begunkov
2021-07-29 15:05 ` [PATCH 13/23] io_uring: kill unused IO_IOPOLL_BATCH Pavel Begunkov
2021-07-29 15:05 ` [PATCH 14/23] io_uring: drop exec checks from io_req_task_submit Pavel Begunkov
2021-07-29 15:05 ` [PATCH 15/23] io_uring: optimise putting task struct Pavel Begunkov
2021-07-29 15:05 ` [PATCH 16/23] io_uring: hide async dadta behind flags Pavel Begunkov
2021-07-29 15:05 ` [PATCH 17/23] io_uring: move io_fallback_req_func() Pavel Begunkov
2021-07-29 15:05 ` [PATCH 18/23] io_uring: cache __io_free_req()'d requests Pavel Begunkov
2021-07-29 15:05 ` [PATCH 19/23] io_uring: use inflight_entry instead of compl.list Pavel Begunkov
2021-07-29 15:05 ` [PATCH 20/23] io_uring: remove redundant args from cache_free Pavel Begunkov
2021-07-29 15:05 ` [PATCH 21/23] io_uring: inline struct io_comp_state Pavel Begunkov
2021-07-29 15:05 ` [PATCH 22/23] io_uring: remove extra argument for overflow flush Pavel Begunkov
2021-07-29 15:05 ` [PATCH 23/23] io_uring: inline io_poll_remove_waitqs Pavel Begunkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f466e8502993fcfafefeea0895660e197c1a8ba0.1627570633.git.asml.silence@gmail.com \
--to=asml.silence@gmail.com \
--cc=axboe@kernel.dk \
--cc=io-uring@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).