IO-Uring Archive on lore.kernel.org
 help / color / Atom feed
* [RFC] do_hashed and wq enqueue
@ 2020-02-11 20:20 Pavel Begunkov
  2020-02-11 20:30 ` Jens Axboe
  0 siblings, 1 reply; 3+ messages in thread
From: Pavel Begunkov @ 2020-02-11 20:20 UTC (permalink / raw)
  To: Jens Axboe, io-uring

[-- Attachment #1.1: Type: text/plain, Size: 476 bytes --]

Hi,

I've been looking for hashed io-wq enqueuing, but not fully understand the
issue. As I remember, it was something about getting around blocking for
buffered I/O. Is that so? Is there any write-up of a thread for this issue?

My case is 2-fd request, and I'm not sure how it should look. For splice() it's
probably Ok to hash the non-pipe end (if any), but I wonder how it should work,
if, for example, the restriction will be removed.


-- 
Pavel Begunkov


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [RFC] do_hashed and wq enqueue
  2020-02-11 20:20 [RFC] do_hashed and wq enqueue Pavel Begunkov
@ 2020-02-11 20:30 ` Jens Axboe
  2020-02-11 20:36   ` Pavel Begunkov
  0 siblings, 1 reply; 3+ messages in thread
From: Jens Axboe @ 2020-02-11 20:30 UTC (permalink / raw)
  To: Pavel Begunkov, io-uring

On 2/11/20 1:20 PM, Pavel Begunkov wrote:
> Hi,
> 
> I've been looking for hashed io-wq enqueuing, but not fully understand the
> issue. As I remember, it was something about getting around blocking for
> buffered I/O. Is that so? Is there any write-up of a thread for this issue?

Not sure if there's a writeup, but the issue is that buffered writes all
need to grab the per-inode mutex. Hence if you don't serialize these writes,
you end up having potentially quite a few threads banging on the same mutex.
This causes a high level of lock contention (and scheduling). By serializing
by inode hash we avoid that, and yield the same performance.

Dave Chinner is working on lifting this restriction, at which point we'll
have to gate the hashing based on whether or not the fs is smart or dumb
when it comes to buffered writes and locking.

> My case is 2-fd request, and I'm not sure how it should look. For splice() it's
> probably Ok to hash the non-pipe end (if any), but I wonder how it should work,
> if, for example, the restriction will be removed.

Probably just do the same, but for the output fd only (and following the
same restrictions in terms of file type).

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [RFC] do_hashed and wq enqueue
  2020-02-11 20:30 ` Jens Axboe
@ 2020-02-11 20:36   ` Pavel Begunkov
  0 siblings, 0 replies; 3+ messages in thread
From: Pavel Begunkov @ 2020-02-11 20:36 UTC (permalink / raw)
  To: Jens Axboe, io-uring

[-- Attachment #1.1: Type: text/plain, Size: 1320 bytes --]

On 11/02/2020 23:30, Jens Axboe wrote:
> On 2/11/20 1:20 PM, Pavel Begunkov wrote:
>> Hi,
>>
>> I've been looking for hashed io-wq enqueuing, but not fully understand the
>> issue. As I remember, it was something about getting around blocking for
>> buffered I/O. Is that so? Is there any write-up of a thread for this issue?
> 
> Not sure if there's a writeup, but the issue is that buffered writes all
> need to grab the per-inode mutex. Hence if you don't serialize these writes,
> you end up having potentially quite a few threads banging on the same mutex.
> This causes a high level of lock contention (and scheduling). By serializing
> by inode hash we avoid that, and yield the same performance.
> 
> Dave Chinner is working on lifting this restriction, at which point we'll
> have to gate the hashing based on whether or not the fs is smart or dumb
> when it comes to buffered writes and locking.

Got it, thanks!

>> My case is 2-fd request, and I'm not sure how it should look. For splice() it's
>> probably Ok to hash the non-pipe end (if any), but I wonder how it should work,
>> if, for example, the restriction will be removed.
> 
> Probably just do the same, but for the output fd only (and following the
> same restrictions in terms of file type).
> 

-- 
Pavel Begunkov


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-11 20:20 [RFC] do_hashed and wq enqueue Pavel Begunkov
2020-02-11 20:30 ` Jens Axboe
2020-02-11 20:36   ` Pavel Begunkov

IO-Uring Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/io-uring/0 io-uring/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 io-uring io-uring/ https://lore.kernel.org/io-uring \
		io-uring@vger.kernel.org
	public-inbox-index io-uring

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.io-uring


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git