[PATCH 6/7] io_uring: single shot poll removal optimisation

From: Pavel Begunkov <asml.silence@gmail.com>
To: io-uring@vger.kernel.org
Cc: asml.silence@gmail.com
Subject: [PATCH 6/7] io_uring: single shot poll removal optimisation
Date: Wed, 15 Dec 2021 22:08:49 +0000	[thread overview]
Message-ID: <ee170a344a18c9ef36b554d806c64caadfd61c31.1639605189.git.asml.silence@gmail.com> (raw)
In-Reply-To: <cover.1639605189.git.asml.silence@gmail.com>

We don't need to poll oneshot request if we've got a desired mask in
io_poll_wake(), task_work will clean it up correctly, but as we already
hold a wq spinlock, we can remove ourselves and save on additional
spinlocking in io_poll_remove_entries().

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 fs/io_uring.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index d59d3fa93c9c..20feca3d86ae 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -5568,8 +5568,14 @@ static int io_poll_wake(struct wait_queue_entry *wait, unsigned mode, int sync,
 	if (mask && !(mask & poll->events))
 		return 0;
 
-	if (io_poll_get_ownership(req))
+	if (io_poll_get_ownership(req)) {
+		/* optional, saves extra locking for removal in tw handler */
+		if (mask && poll->events & EPOLLONESHOT) {
+			list_del_init(&poll->wait.entry);
+			poll->head = NULL;
+		}
 		__io_poll_execute(req, mask);
+	}
 	return 1;
 }
 
-- 
2.34.0

