All of lore.kernel.org
 help / color / mirror / Atom feed
From: Pavel Begunkov <asml.silence@gmail.com>
To: Jens Axboe <axboe@kernel.dk>, io-uring@vger.kernel.org
Subject: [PATCH 03/12] io_uring: shuffle more fields into SQ ctx section
Date: Mon, 14 Jun 2021 23:37:22 +0100	[thread overview]
Message-ID: <8a5899a50afc6ccca63249e716f580b246f3dec6.1623709150.git.asml.silence@gmail.com> (raw)
In-Reply-To: <cover.1623709150.git.asml.silence@gmail.com>

Since moving locked_free_* out of struct io_submit_state
ctx->submit_state is accessed on submission side only, so move it into
the submission section. Same goes for rsrc table pointers/nodes/etc.,
they must be taken and checked during submission because sync'ed by
uring_lock, so move them there as well.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 fs/io_uring.c | 35 +++++++++++++++++------------------
 1 file changed, 17 insertions(+), 18 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index a4460383bd25..5cc0c4dd2709 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -353,6 +353,7 @@ struct io_ring_ctx {
 		unsigned int		restricted: 1;
 	} ____cacheline_aligned_in_smp;
 
+	/* submission data */
 	struct {
 		/*
 		 * Ring buffer of indices into array of io_uring_sqe, which is
@@ -369,13 +370,27 @@ struct io_ring_ctx {
 		struct io_uring_sqe	*sq_sqes;
 		unsigned		cached_sq_head;
 		unsigned		sq_entries;
-		unsigned		sq_thread_idle;
 		unsigned		cached_sq_dropped;
 		unsigned long		sq_check_overflow;
-
 		struct list_head	defer_list;
+
+		/*
+		 * Fixed resources fast path, should be accessed only under
+		 * uring_lock, and updated through io_uring_register(2)
+		 */
+		struct io_rsrc_node	*rsrc_node;
+		struct io_file_table	file_table;
+		unsigned		nr_user_files;
+		unsigned		nr_user_bufs;
+		struct io_mapped_ubuf	**user_bufs;
+
+		struct io_submit_state	submit_state;
 		struct list_head	timeout_list;
 		struct list_head	cq_overflow_list;
+		struct xarray		io_buffers;
+		struct xarray		personalities;
+		u32			pers_next;
+		unsigned		sq_thread_idle;
 	} ____cacheline_aligned_in_smp;
 
 	struct {
@@ -383,7 +398,6 @@ struct io_ring_ctx {
 		wait_queue_head_t	wait;
 	} ____cacheline_aligned_in_smp;
 
-	struct io_submit_state		submit_state;
 	/* IRQ completion list, under ->completion_lock */
 	struct list_head	locked_free_list;
 	unsigned int		locked_free_nr;
@@ -394,21 +408,6 @@ struct io_ring_ctx {
 	struct wait_queue_head	sqo_sq_wait;
 	struct list_head	sqd_list;
 
-	/*
-	 * Fixed resources fast path, should be accessed only under uring_lock,
-	 * and updated through io_uring_register(2)
-	 */
-	struct io_rsrc_node	*rsrc_node;
-
-	struct io_file_table	file_table;
-	unsigned		nr_user_files;
-	unsigned		nr_user_bufs;
-	struct io_mapped_ubuf	**user_bufs;
-
-	struct xarray		io_buffers;
-	struct xarray		personalities;
-	u32			pers_next;
-
 	struct {
 		unsigned		cached_cq_tail;
 		unsigned		cq_entries;
-- 
2.31.1


  parent reply	other threads:[~2021-06-14 22:39 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-14 22:37 [PATCH 5.14 00/12] for-next optimisations Pavel Begunkov
2021-06-14 22:37 ` [PATCH 01/12] io_uring: keep SQ pointers in a single cacheline Pavel Begunkov
2021-06-14 22:37 ` [PATCH 02/12] io_uring: move ctx->flags from SQ cacheline Pavel Begunkov
2021-06-14 22:37 ` Pavel Begunkov [this message]
2021-06-14 22:37 ` [PATCH 04/12] io_uring: refactor io_get_sqe() Pavel Begunkov
2021-06-14 22:37 ` [PATCH 05/12] io_uring: don't cache number of dropped SQEs Pavel Begunkov
2021-06-14 22:37 ` [PATCH 06/12] io_uring: optimise completion timeout flushing Pavel Begunkov
2021-06-14 22:37 ` [PATCH 07/12] io_uring: small io_submit_sqe() optimisation Pavel Begunkov
2021-06-14 22:37 ` [PATCH 08/12] io_uring: clean up check_overflow flag Pavel Begunkov
2021-06-14 22:37 ` [PATCH 09/12] io_uring: wait heads renaming Pavel Begunkov
2021-06-14 22:37 ` [PATCH 10/12] io_uring: move uring_lock location Pavel Begunkov
2021-06-14 22:37 ` [PATCH 11/12] io_uring: refactor io_req_defer() Pavel Begunkov
2021-06-14 22:37 ` [PATCH 12/12] io_uring: optimise non-drain path Pavel Begunkov
2021-06-15 12:27 ` [PATCH 5.14 00/12] for-next optimisations Pavel Begunkov
2021-06-15 21:38 ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8a5899a50afc6ccca63249e716f580b246f3dec6.1623709150.git.asml.silence@gmail.com \
    --to=asml.silence@gmail.com \
    --cc=axboe@kernel.dk \
    --cc=io-uring@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.