IO-Uring Archive on lore.kernel.org
 help / color / Atom feed
From: Pavel Begunkov <asml.silence@gmail.com>
To: Jens Axboe <axboe@kernel.dk>, io-uring@vger.kernel.org
Subject: Re: [RFC 0/2] 3 cacheline io_kiocb
Date: Sat, 25 Jul 2020 21:24:33 +0300
Message-ID: <faf48a78-4327-50e6-083a-f5c762f66e8a@gmail.com> (raw)
In-Reply-To: <467e93fb-876d-e2a5-7596-4b9e21317d67@kernel.dk>

On 25/07/2020 18:45, Jens Axboe wrote:
> On 7/25/20 2:31 AM, Pavel Begunkov wrote:
>> That's not final for a several reasons, but good enough for discussion.
>> That brings io_kiocb down to 192B. I didn't try to benchmark it
>> properly, but quick nop test gave +5% throughput increase.
>> 7531 vs 7910 KIOPS with fio/t/io_uring
>>
>> The whole situation is obviously a bunch of tradeoffs. For instance,
>> instead of shrinking it, we can inline apoll to speed apoll path.
>>
>> [2/2] just for a reference, I'm thinking about other ways to shrink it.
>> e.g. ->link_list can be a single-linked list with linked tiemouts
>> storing a back-reference. This can turn out to be better, because
>> that would move ->fixed_file_refs to the 2nd cacheline, so we won't
>> ever touch 3rd cacheline in the submission path.
>> Any other ideas?
> 
> Nothing noticeable for me, still about the same performance. But
> generally speaking, I don't necessarily think we need to go all in on
> making this as tiny as possible. It's much more important to chase the
> items where we only use 2 cachelines for the hot path, and then we have
> the extra space in there already for the semi hot paths like poll driven
> retry. Yes, we're still allocating from a pool that has slightly larger
> objects, but that doesn't really matter _that_ much. Avoiding an extra
> kmalloc+kfree for the semi hot paths are a bigger deal than making
> io_kiocb smaller and smaller.
> 
> That said, for no-brainer changes, we absolutely should make it smaller.
> I just don't want to jump through convoluted hoops to get there.

Agree, but that's not the end goal. The first point is to kill the union,
but it already has enough space for that.

The second is to see, whether we can use the space in a better way. From
the high level perspective ->apoll and ->work are alike and both serve to
provide asynchronous paths, hence the idea to swap them naturally comes to
mind. TBH, I don't think it'd do much, because init of ->io would probably
hide any benefit.

-- 
Pavel Begunkov

  reply index

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-25  8:31 Pavel Begunkov
2020-07-25  8:31 ` [PATCH 1/2] io_uring: allocate req->work dynamically Pavel Begunkov
2020-07-25  8:31 ` [PATCH 2/2] io_uring: unionise ->apoll and ->work Pavel Begunkov
2020-07-25 15:45 ` [RFC 0/2] 3 cacheline io_kiocb Jens Axboe
2020-07-25 18:24   ` Pavel Begunkov [this message]
2020-07-25 19:40     ` Jens Axboe
2020-07-25 20:14       ` Pavel Begunkov
2020-07-25 20:25         ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=faf48a78-4327-50e6-083a-f5c762f66e8a@gmail.com \
    --to=asml.silence@gmail.com \
    --cc=axboe@kernel.dk \
    --cc=io-uring@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

IO-Uring Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/io-uring/0 io-uring/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 io-uring io-uring/ https://lore.kernel.org/io-uring \
		io-uring@vger.kernel.org
	public-inbox-index io-uring

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.io-uring


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git