IO-Uring Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events
@ 2020-01-16 13:49 Stefano Garzarella
  2020-01-16 15:29 ` Jens Axboe
  0 siblings, 1 reply; 10+ messages in thread
From: Stefano Garzarella @ 2020-01-16 13:49 UTC (permalink / raw)
  To: Jens Axboe; +Cc: Alexander Viro, io-uring, linux-kernel, linux-fsdevel

io_uring_poll() sets EPOLLOUT flag if there is space in the
SQ ring, then we should wakeup threads waiting for EPOLLOUT
events when we expose the new SQ head to the userspace.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
---

Do you think is better to change the name of 'cq_wait' and 'cq_fasync'?

Thanks,
Stefano
---
 fs/io_uring.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 38b54051facd..5c6ff5f9e741 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -3687,6 +3687,11 @@ static void io_commit_sqring(struct io_ring_ctx *ctx)
 		 * write new data to them.
 		 */
 		smp_store_release(&rings->sq.head, ctx->cached_sq_head);
+
+		if (wq_has_sleeper(&ctx->cq_wait)) {
+			wake_up_interruptible(&ctx->cq_wait);
+			kill_fasync(&ctx->cq_fasync, SIGIO, POLL_OUT);
+		}
 	}
 }
 
-- 
2.24.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events
  2020-01-16 13:49 [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events Stefano Garzarella
@ 2020-01-16 15:29 ` Jens Axboe
  2020-01-16 15:55   ` Stefano Garzarella
  0 siblings, 1 reply; 10+ messages in thread
From: Jens Axboe @ 2020-01-16 15:29 UTC (permalink / raw)
  To: Stefano Garzarella; +Cc: Alexander Viro, io-uring, linux-kernel, linux-fsdevel

On 1/16/20 6:49 AM, Stefano Garzarella wrote:
> io_uring_poll() sets EPOLLOUT flag if there is space in the
> SQ ring, then we should wakeup threads waiting for EPOLLOUT
> events when we expose the new SQ head to the userspace.
> 
> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
> ---
> 
> Do you think is better to change the name of 'cq_wait' and 'cq_fasync'?

I honestly think it'd be better to have separate waits for in/out poll,
the below patch will introduce some unfortunate cacheline traffic
between the submitter and completer side.

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events
  2020-01-16 15:29 ` Jens Axboe
@ 2020-01-16 15:55   ` Stefano Garzarella
  2020-01-16 16:00     ` Jens Axboe
  0 siblings, 1 reply; 10+ messages in thread
From: Stefano Garzarella @ 2020-01-16 15:55 UTC (permalink / raw)
  To: Jens Axboe; +Cc: Alexander Viro, io-uring, linux-kernel, linux-fsdevel

On Thu, Jan 16, 2020 at 08:29:07AM -0700, Jens Axboe wrote:
> On 1/16/20 6:49 AM, Stefano Garzarella wrote:
> > io_uring_poll() sets EPOLLOUT flag if there is space in the
> > SQ ring, then we should wakeup threads waiting for EPOLLOUT
> > events when we expose the new SQ head to the userspace.
> > 
> > Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
> > ---
> > 
> > Do you think is better to change the name of 'cq_wait' and 'cq_fasync'?
> 
> I honestly think it'd be better to have separate waits for in/out poll,
> the below patch will introduce some unfortunate cacheline traffic
> between the submitter and completer side.

Agree, make sense. I'll send a v2 with a new 'sq_wait'.

About fasync, do you think could be useful the POLL_OUT support?
In this case, maybe is not simple to have two separate fasync_struct,
do you have any advice?

Thanks,
Stefano


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events
  2020-01-16 15:55   ` Stefano Garzarella
@ 2020-01-16 16:00     ` Jens Axboe
  2020-01-16 16:26       ` Stefano Garzarella
  0 siblings, 1 reply; 10+ messages in thread
From: Jens Axboe @ 2020-01-16 16:00 UTC (permalink / raw)
  To: Stefano Garzarella; +Cc: Alexander Viro, io-uring, linux-kernel, linux-fsdevel

On 1/16/20 8:55 AM, Stefano Garzarella wrote:
> On Thu, Jan 16, 2020 at 08:29:07AM -0700, Jens Axboe wrote:
>> On 1/16/20 6:49 AM, Stefano Garzarella wrote:
>>> io_uring_poll() sets EPOLLOUT flag if there is space in the
>>> SQ ring, then we should wakeup threads waiting for EPOLLOUT
>>> events when we expose the new SQ head to the userspace.
>>>
>>> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
>>> ---
>>>
>>> Do you think is better to change the name of 'cq_wait' and 'cq_fasync'?
>>
>> I honestly think it'd be better to have separate waits for in/out poll,
>> the below patch will introduce some unfortunate cacheline traffic
>> between the submitter and completer side.
> 
> Agree, make sense. I'll send a v2 with a new 'sq_wait'.
> 
> About fasync, do you think could be useful the POLL_OUT support?
> In this case, maybe is not simple to have two separate fasync_struct,
> do you have any advice?

The fasync should not matter, it's all in the checking of whether the sq
side has any sleepers. This is rarely going to be the case, so as long
as we can keep the check cheap, then I think we're fine.

Since the use case is mostly single submitter, unless you're doing
something funky or unusual, you're not going to be needing POLLOUT ever.
Hence I don't want to add any cost for it, I'd even advocate just doing
waitqueue_active() perhaps, if we can safely pull it off.

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events
  2020-01-16 16:00     ` Jens Axboe
@ 2020-01-16 16:26       ` Stefano Garzarella
  2020-01-16 16:30         ` Jens Axboe
  0 siblings, 1 reply; 10+ messages in thread
From: Stefano Garzarella @ 2020-01-16 16:26 UTC (permalink / raw)
  To: Jens Axboe; +Cc: Alexander Viro, io-uring, linux-kernel, linux-fsdevel

On Thu, Jan 16, 2020 at 09:00:24AM -0700, Jens Axboe wrote:
> On 1/16/20 8:55 AM, Stefano Garzarella wrote:
> > On Thu, Jan 16, 2020 at 08:29:07AM -0700, Jens Axboe wrote:
> >> On 1/16/20 6:49 AM, Stefano Garzarella wrote:
> >>> io_uring_poll() sets EPOLLOUT flag if there is space in the
> >>> SQ ring, then we should wakeup threads waiting for EPOLLOUT
> >>> events when we expose the new SQ head to the userspace.
> >>>
> >>> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
> >>> ---
> >>>
> >>> Do you think is better to change the name of 'cq_wait' and 'cq_fasync'?
> >>
> >> I honestly think it'd be better to have separate waits for in/out poll,
> >> the below patch will introduce some unfortunate cacheline traffic
> >> between the submitter and completer side.
> > 
> > Agree, make sense. I'll send a v2 with a new 'sq_wait'.
> > 
> > About fasync, do you think could be useful the POLL_OUT support?
> > In this case, maybe is not simple to have two separate fasync_struct,
> > do you have any advice?
> 
> The fasync should not matter, it's all in the checking of whether the sq
> side has any sleepers. This is rarely going to be the case, so as long
> as we can keep the check cheap, then I think we're fine.

Right.

> 
> Since the use case is mostly single submitter, unless you're doing
> something funky or unusual, you're not going to be needing POLLOUT ever.

The case that I had in mind was with kernel side polling enabled and
a single submitter that can use epoll() to wait free slots in the SQ
ring. (I don't have a test, maybe I can write one...)

> Hence I don't want to add any cost for it, I'd even advocate just doing
> waitqueue_active() perhaps, if we can safely pull it off.

I'll try!

Thanks,
Stefano


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events
  2020-01-16 16:26       ` Stefano Garzarella
@ 2020-01-16 16:30         ` Jens Axboe
  2020-01-16 17:03           ` Stefano Garzarella
  0 siblings, 1 reply; 10+ messages in thread
From: Jens Axboe @ 2020-01-16 16:30 UTC (permalink / raw)
  To: Stefano Garzarella; +Cc: Alexander Viro, io-uring, linux-kernel, linux-fsdevel

On 1/16/20 9:26 AM, Stefano Garzarella wrote:
>> Since the use case is mostly single submitter, unless you're doing
>> something funky or unusual, you're not going to be needing POLLOUT ever.
> 
> The case that I had in mind was with kernel side polling enabled and
> a single submitter that can use epoll() to wait free slots in the SQ
> ring. (I don't have a test, maybe I can write one...)

Right, I think that's the only use case where it makes sense, because
you have someone else draining the sq side for you. A test case would
indeed be nice, liburing has a good arsenal of test cases and this would
be a good addition!

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events
  2020-01-16 16:30         ` Jens Axboe
@ 2020-01-16 17:03           ` Stefano Garzarella
  2020-01-23 19:13             ` Jens Axboe
  0 siblings, 1 reply; 10+ messages in thread
From: Stefano Garzarella @ 2020-01-16 17:03 UTC (permalink / raw)
  To: Jens Axboe; +Cc: Alexander Viro, io-uring, linux-kernel, linux-fsdevel

On Thu, Jan 16, 2020 at 09:30:12AM -0700, Jens Axboe wrote:
> On 1/16/20 9:26 AM, Stefano Garzarella wrote:
> >> Since the use case is mostly single submitter, unless you're doing
> >> something funky or unusual, you're not going to be needing POLLOUT ever.
> > 
> > The case that I had in mind was with kernel side polling enabled and
> > a single submitter that can use epoll() to wait free slots in the SQ
> > ring. (I don't have a test, maybe I can write one...)
> 
> Right, I think that's the only use case where it makes sense, because
> you have someone else draining the sq side for you. A test case would
> indeed be nice, liburing has a good arsenal of test cases and this would
> be a good addition!

Sure, I'll send a test to liburing for this case!

Thanks,
Stefano


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events
  2020-01-16 17:03           ` Stefano Garzarella
@ 2020-01-23 19:13             ` Jens Axboe
  2020-01-23 21:45               ` Stefano Garzarella
  0 siblings, 1 reply; 10+ messages in thread
From: Jens Axboe @ 2020-01-23 19:13 UTC (permalink / raw)
  To: Stefano Garzarella; +Cc: Alexander Viro, io-uring, linux-kernel, linux-fsdevel

On 1/16/20 10:03 AM, Stefano Garzarella wrote:
> On Thu, Jan 16, 2020 at 09:30:12AM -0700, Jens Axboe wrote:
>> On 1/16/20 9:26 AM, Stefano Garzarella wrote:
>>>> Since the use case is mostly single submitter, unless you're doing
>>>> something funky or unusual, you're not going to be needing POLLOUT ever.
>>>
>>> The case that I had in mind was with kernel side polling enabled and
>>> a single submitter that can use epoll() to wait free slots in the SQ
>>> ring. (I don't have a test, maybe I can write one...)
>>
>> Right, I think that's the only use case where it makes sense, because
>> you have someone else draining the sq side for you. A test case would
>> indeed be nice, liburing has a good arsenal of test cases and this would
>> be a good addition!
> 
> Sure, I'll send a test to liburing for this case!

Gentle ping on the test case :-)

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events
  2020-01-23 19:13             ` Jens Axboe
@ 2020-01-23 21:45               ` Stefano Garzarella
  2020-01-24  1:28                 ` Jens Axboe
  0 siblings, 1 reply; 10+ messages in thread
From: Stefano Garzarella @ 2020-01-23 21:45 UTC (permalink / raw)
  To: Jens Axboe; +Cc: Alexander Viro, io-uring, linux-kernel, linux-fsdevel

On Thu, Jan 23, 2020 at 12:13:57PM -0700, Jens Axboe wrote:
> On 1/16/20 10:03 AM, Stefano Garzarella wrote:
> > On Thu, Jan 16, 2020 at 09:30:12AM -0700, Jens Axboe wrote:
> >> On 1/16/20 9:26 AM, Stefano Garzarella wrote:
> >>>> Since the use case is mostly single submitter, unless you're doing
> >>>> something funky or unusual, you're not going to be needing POLLOUT ever.
> >>>
> >>> The case that I had in mind was with kernel side polling enabled and
> >>> a single submitter that can use epoll() to wait free slots in the SQ
> >>> ring. (I don't have a test, maybe I can write one...)
> >>
> >> Right, I think that's the only use case where it makes sense, because
> >> you have someone else draining the sq side for you. A test case would
> >> indeed be nice, liburing has a good arsenal of test cases and this would
> >> be a good addition!
> > 
> > Sure, I'll send a test to liburing for this case!
> 
> Gentle ping on the test case :-)
> 

Yes, you are right :-)

I was a little busy this week to finish some works before DevConf.
I hope to work on the test case these days, so by Monday I hope I have it ;-)

Cheers,
Stefano


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events
  2020-01-23 21:45               ` Stefano Garzarella
@ 2020-01-24  1:28                 ` Jens Axboe
  0 siblings, 0 replies; 10+ messages in thread
From: Jens Axboe @ 2020-01-24  1:28 UTC (permalink / raw)
  To: Stefano Garzarella; +Cc: Alexander Viro, io-uring, linux-kernel, linux-fsdevel

On 1/23/20 2:45 PM, Stefano Garzarella wrote:
> On Thu, Jan 23, 2020 at 12:13:57PM -0700, Jens Axboe wrote:
>> On 1/16/20 10:03 AM, Stefano Garzarella wrote:
>>> On Thu, Jan 16, 2020 at 09:30:12AM -0700, Jens Axboe wrote:
>>>> On 1/16/20 9:26 AM, Stefano Garzarella wrote:
>>>>>> Since the use case is mostly single submitter, unless you're doing
>>>>>> something funky or unusual, you're not going to be needing POLLOUT ever.
>>>>>
>>>>> The case that I had in mind was with kernel side polling enabled and
>>>>> a single submitter that can use epoll() to wait free slots in the SQ
>>>>> ring. (I don't have a test, maybe I can write one...)
>>>>
>>>> Right, I think that's the only use case where it makes sense, because
>>>> you have someone else draining the sq side for you. A test case would
>>>> indeed be nice, liburing has a good arsenal of test cases and this would
>>>> be a good addition!
>>>
>>> Sure, I'll send a test to liburing for this case!
>>
>> Gentle ping on the test case :-)
>>
> 
> Yes, you are right :-)
> 
> I was a little busy this week to finish some works before DevConf.  I
> hope to work on the test case these days, so by Monday I hope I have
> it ;-)

Thanks, all good, just a gentle nudge ;-)

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, back to index

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-16 13:49 [PATCH] io_uring: wakeup threads waiting for EPOLLOUT events Stefano Garzarella
2020-01-16 15:29 ` Jens Axboe
2020-01-16 15:55   ` Stefano Garzarella
2020-01-16 16:00     ` Jens Axboe
2020-01-16 16:26       ` Stefano Garzarella
2020-01-16 16:30         ` Jens Axboe
2020-01-16 17:03           ` Stefano Garzarella
2020-01-23 19:13             ` Jens Axboe
2020-01-23 21:45               ` Stefano Garzarella
2020-01-24  1:28                 ` Jens Axboe

IO-Uring Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/io-uring/0 io-uring/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 io-uring io-uring/ https://lore.kernel.org/io-uring \
		io-uring@vger.kernel.org
	public-inbox-index io-uring

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.io-uring


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git