io-uring.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/3] commit/posted fixes
@ 2021-01-07  3:15 Pavel Begunkov
  2021-01-07  3:15 ` [PATCH 1/3] io_uring: trigger eventfd for IOPOLL Pavel Begunkov
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Pavel Begunkov @ 2021-01-07  3:15 UTC (permalink / raw)
  To: Jens Axboe, io-uring

Looks like I accidentially didn't send this series.

Regarding mb in 3/3, I have a for-next patch removing
it (one mb less in total), just prefer it to settle a
bit so there is more time to be reported if anything
goes wrong.

Pavel Begunkov (3):
  io_uring: trigger eventfd for IOPOLL
  io_uring: dont kill fasync under completion_lock
  io_uring: synchronise ev_posted() with waitqueues

 fs/io_uring.c | 32 +++++++++++++++++++++++++-------
 1 file changed, 25 insertions(+), 7 deletions(-)

-- 
2.24.0


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 1/3] io_uring: trigger eventfd for IOPOLL
  2021-01-07  3:15 [PATCH 0/3] commit/posted fixes Pavel Begunkov
@ 2021-01-07  3:15 ` Pavel Begunkov
  2021-01-07  3:15 ` [PATCH 2/3] io_uring: dont kill fasync under completion_lock Pavel Begunkov
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Pavel Begunkov @ 2021-01-07  3:15 UTC (permalink / raw)
  To: Jens Axboe, io-uring

Make sure io_iopoll_complete() tries to wake up eventfd, which currently
is skipped together with io_cqring_ev_posted() for non-SQPOLL IOPOLL.

Add an iopoll version of io_cqring_ev_posted(), duplicates a bit of
code, but they actually use different sets of wait queues may be for
better.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 fs/io_uring.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 27a8c226abf8..91e517ad1421 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -1713,6 +1713,16 @@ static void io_cqring_ev_posted(struct io_ring_ctx *ctx)
 		eventfd_signal(ctx->cq_ev_fd, 1);
 }
 
+static void io_cqring_ev_posted_iopoll(struct io_ring_ctx *ctx)
+{
+	if (ctx->flags & IORING_SETUP_SQPOLL) {
+		if (waitqueue_active(&ctx->wait))
+			wake_up(&ctx->wait);
+	}
+	if (io_should_trigger_evfd(ctx))
+		eventfd_signal(ctx->cq_ev_fd, 1);
+}
+
 /* Returns true if there are no backlogged entries after the flush */
 static bool __io_cqring_overflow_flush(struct io_ring_ctx *ctx, bool force,
 				       struct task_struct *tsk,
@@ -2428,8 +2438,7 @@ static void io_iopoll_complete(struct io_ring_ctx *ctx, unsigned int *nr_events,
 	}
 
 	io_commit_cqring(ctx);
-	if (ctx->flags & IORING_SETUP_SQPOLL)
-		io_cqring_ev_posted(ctx);
+	io_cqring_ev_posted_iopoll(ctx);
 	io_req_free_batch_finish(ctx, &rb);
 
 	if (!list_empty(&again))
-- 
2.24.0


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 2/3] io_uring: dont kill fasync under completion_lock
  2021-01-07  3:15 [PATCH 0/3] commit/posted fixes Pavel Begunkov
  2021-01-07  3:15 ` [PATCH 1/3] io_uring: trigger eventfd for IOPOLL Pavel Begunkov
@ 2021-01-07  3:15 ` Pavel Begunkov
  2021-01-07  3:15 ` [PATCH 3/3] io_uring: synchronise ev_posted() with waitqueues Pavel Begunkov
  2021-01-07 14:48 ` [PATCH 0/3] commit/posted fixes Jens Axboe
  3 siblings, 0 replies; 5+ messages in thread
From: Pavel Begunkov @ 2021-01-07  3:15 UTC (permalink / raw)
  To: Jens Axboe, io-uring; +Cc: syzbot+91ca3f25bd7f795f019c

      CPU0                    CPU1
       ----                    ----
  lock(&new->fa_lock);
                               local_irq_disable();
                               lock(&ctx->completion_lock);
                               lock(&new->fa_lock);
  <Interrupt>
    lock(&ctx->completion_lock);

 *** DEADLOCK ***

Move kill_fasync() out of io_commit_cqring() to io_cqring_ev_posted(),
so it doesn't hold completion_lock while doing it. That saves from the
reported deadlock, and it's just nice to shorten the locking time and
untangle nested locks (compl_lock -> wq_head::lock).

Reported-by: syzbot+91ca3f25bd7f795f019c@syzkaller.appspotmail.com
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 fs/io_uring.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 91e517ad1421..401316fe2ae2 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -1345,11 +1345,6 @@ static void __io_commit_cqring(struct io_ring_ctx *ctx)
 
 	/* order cqe stores with ring update */
 	smp_store_release(&rings->cq.tail, ctx->cached_cq_tail);
-
-	if (wq_has_sleeper(&ctx->cq_wait)) {
-		wake_up_interruptible(&ctx->cq_wait);
-		kill_fasync(&ctx->cq_fasync, SIGIO, POLL_IN);
-	}
 }
 
 static void io_put_identity(struct io_uring_task *tctx, struct io_kiocb *req)
@@ -1711,6 +1706,10 @@ static void io_cqring_ev_posted(struct io_ring_ctx *ctx)
 		wake_up(&ctx->sq_data->wait);
 	if (io_should_trigger_evfd(ctx))
 		eventfd_signal(ctx->cq_ev_fd, 1);
+	if (wq_has_sleeper(&ctx->cq_wait)) {
+		wake_up_interruptible(&ctx->cq_wait);
+		kill_fasync(&ctx->cq_fasync, SIGIO, POLL_IN);
+	}
 }
 
 static void io_cqring_ev_posted_iopoll(struct io_ring_ctx *ctx)
@@ -1721,6 +1720,10 @@ static void io_cqring_ev_posted_iopoll(struct io_ring_ctx *ctx)
 	}
 	if (io_should_trigger_evfd(ctx))
 		eventfd_signal(ctx->cq_ev_fd, 1);
+	if (wq_has_sleeper(&ctx->cq_wait)) {
+		wake_up_interruptible(&ctx->cq_wait);
+		kill_fasync(&ctx->cq_fasync, SIGIO, POLL_IN);
+	}
 }
 
 /* Returns true if there are no backlogged entries after the flush */
-- 
2.24.0


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 3/3] io_uring: synchronise ev_posted() with waitqueues
  2021-01-07  3:15 [PATCH 0/3] commit/posted fixes Pavel Begunkov
  2021-01-07  3:15 ` [PATCH 1/3] io_uring: trigger eventfd for IOPOLL Pavel Begunkov
  2021-01-07  3:15 ` [PATCH 2/3] io_uring: dont kill fasync under completion_lock Pavel Begunkov
@ 2021-01-07  3:15 ` Pavel Begunkov
  2021-01-07 14:48 ` [PATCH 0/3] commit/posted fixes Jens Axboe
  3 siblings, 0 replies; 5+ messages in thread
From: Pavel Begunkov @ 2021-01-07  3:15 UTC (permalink / raw)
  To: Jens Axboe, io-uring

waitqueue_active() needs smp_mb() to be in sync with waitqueues
modification, but we miss it in io_cqring_ev_posted*() apart from
cq_wait() case.

Take an smb_mb() out of wq_has_sleeper() making it waitqueue_active(),
and place it a few lines before, so it can synchronise other
waitqueue_active() as well.

The patch doesn't add any additional overhead, so even if there are
no problems currently, it's just safer to have it this way.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 fs/io_uring.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 401316fe2ae2..cb57e0360fcb 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -1700,13 +1700,16 @@ static inline unsigned __io_cqring_events(struct io_ring_ctx *ctx)
 
 static void io_cqring_ev_posted(struct io_ring_ctx *ctx)
 {
+	/* see waitqueue_active() comment */
+	smp_mb();
+
 	if (waitqueue_active(&ctx->wait))
 		wake_up(&ctx->wait);
 	if (ctx->sq_data && waitqueue_active(&ctx->sq_data->wait))
 		wake_up(&ctx->sq_data->wait);
 	if (io_should_trigger_evfd(ctx))
 		eventfd_signal(ctx->cq_ev_fd, 1);
-	if (wq_has_sleeper(&ctx->cq_wait)) {
+	if (waitqueue_active(&ctx->cq_wait)) {
 		wake_up_interruptible(&ctx->cq_wait);
 		kill_fasync(&ctx->cq_fasync, SIGIO, POLL_IN);
 	}
@@ -1714,13 +1717,16 @@ static void io_cqring_ev_posted(struct io_ring_ctx *ctx)
 
 static void io_cqring_ev_posted_iopoll(struct io_ring_ctx *ctx)
 {
+	/* see waitqueue_active() comment */
+	smp_mb();
+
 	if (ctx->flags & IORING_SETUP_SQPOLL) {
 		if (waitqueue_active(&ctx->wait))
 			wake_up(&ctx->wait);
 	}
 	if (io_should_trigger_evfd(ctx))
 		eventfd_signal(ctx->cq_ev_fd, 1);
-	if (wq_has_sleeper(&ctx->cq_wait)) {
+	if (waitqueue_active(&ctx->cq_wait)) {
 		wake_up_interruptible(&ctx->cq_wait);
 		kill_fasync(&ctx->cq_fasync, SIGIO, POLL_IN);
 	}
-- 
2.24.0


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [PATCH 0/3] commit/posted fixes
  2021-01-07  3:15 [PATCH 0/3] commit/posted fixes Pavel Begunkov
                   ` (2 preceding siblings ...)
  2021-01-07  3:15 ` [PATCH 3/3] io_uring: synchronise ev_posted() with waitqueues Pavel Begunkov
@ 2021-01-07 14:48 ` Jens Axboe
  3 siblings, 0 replies; 5+ messages in thread
From: Jens Axboe @ 2021-01-07 14:48 UTC (permalink / raw)
  To: Pavel Begunkov, io-uring

On 1/6/21 8:15 PM, Pavel Begunkov wrote:
> Looks like I accidentially didn't send this series.
> 
> Regarding mb in 3/3, I have a for-next patch removing
> it (one mb less in total), just prefer it to settle a
> bit so there is more time to be reported if anything
> goes wrong.

Applied, thanks.

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2021-01-07 14:49 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-07  3:15 [PATCH 0/3] commit/posted fixes Pavel Begunkov
2021-01-07  3:15 ` [PATCH 1/3] io_uring: trigger eventfd for IOPOLL Pavel Begunkov
2021-01-07  3:15 ` [PATCH 2/3] io_uring: dont kill fasync under completion_lock Pavel Begunkov
2021-01-07  3:15 ` [PATCH 3/3] io_uring: synchronise ev_posted() with waitqueues Pavel Begunkov
2021-01-07 14:48 ` [PATCH 0/3] commit/posted fixes Jens Axboe

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).