* [PATCH v2] block/io_uring: resubmit when result is -EAGAIN
@ 2021-07-29 9:10 Fabian Ebner
2021-07-29 9:54 ` Stefano Garzarella
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Fabian Ebner @ 2021-07-29 9:10 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, open list:Linux io_uring, Stefan Hajnoczi,
Julia Suvorova, Max Reitz, Stefan Hajnoczi, Fabian Ebner,
Aarushi Mehta, Stefano Garzarella
Linux SCSI can throw spurious -EAGAIN in some corner cases in its
completion path, which will end up being the result in the completed
io_uring request.
Resubmitting such requests should allow block jobs to complete, even
if such spurious errors are encountered.
Co-authored-by: Stefan Hajnoczi <stefanha@gmail.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
Changes from v1:
* Focus on what's relevant for the patch itself in the commit
message.
* Add Stefan's comment.
* Add Stefano's R-b tag (I hope that's fine, since there was no
change code-wise).
block/io_uring.c | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/block/io_uring.c b/block/io_uring.c
index 00a3ee9fb8..dfa475cc87 100644
--- a/block/io_uring.c
+++ b/block/io_uring.c
@@ -165,7 +165,21 @@ static void luring_process_completions(LuringState *s)
total_bytes = ret + luringcb->total_read;
if (ret < 0) {
- if (ret == -EINTR) {
+ /*
+ * Only writev/readv/fsync requests on regular files or host block
+ * devices are submitted. Therefore -EAGAIN is not expected but it's
+ * known to happen sometimes with Linux SCSI. Submit again and hope
+ * the request completes successfully.
+ *
+ * For more information, see:
+ * https://lore.kernel.org/io-uring/20210727165811.284510-3-axboe@kernel.dk/T/#u
+ *
+ * If the code is changed to submit other types of requests in the
+ * future, then this workaround may need to be extended to deal with
+ * genuine -EAGAIN results that should not be resubmitted
+ * immediately.
+ */
+ if (ret == -EINTR || ret == -EAGAIN) {
luring_resubmit(s, luringcb);
continue;
}
--
2.30.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v2] block/io_uring: resubmit when result is -EAGAIN
2021-07-29 9:10 [PATCH v2] block/io_uring: resubmit when result is -EAGAIN Fabian Ebner
@ 2021-07-29 9:54 ` Stefano Garzarella
2021-07-29 16:15 ` Stefan Hajnoczi
2021-08-02 12:40 ` Kevin Wolf
2 siblings, 0 replies; 7+ messages in thread
From: Stefano Garzarella @ 2021-07-29 9:54 UTC (permalink / raw)
To: Fabian Ebner
Cc: Kevin Wolf, open list:Linux io_uring, Stefan Hajnoczi,
Julia Suvorova, qemu-devel, Max Reitz, Stefan Hajnoczi,
Aarushi Mehta
On Thu, Jul 29, 2021 at 11:10:29AM +0200, Fabian Ebner wrote:
>Linux SCSI can throw spurious -EAGAIN in some corner cases in its
>completion path, which will end up being the result in the completed
>io_uring request.
>
>Resubmitting such requests should allow block jobs to complete, even
>if such spurious errors are encountered.
>
>Co-authored-by: Stefan Hajnoczi <stefanha@gmail.com>
>Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
>Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
>---
>
>Changes from v1:
> * Focus on what's relevant for the patch itself in the commit
> message.
> * Add Stefan's comment.
> * Add Stefano's R-b tag (I hope that's fine, since there was no
> change code-wise).
Yep, it's fine :-)
Thanks,
Stefano
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] block/io_uring: resubmit when result is -EAGAIN
2021-07-29 9:10 [PATCH v2] block/io_uring: resubmit when result is -EAGAIN Fabian Ebner
2021-07-29 9:54 ` Stefano Garzarella
@ 2021-07-29 16:15 ` Stefan Hajnoczi
2021-08-02 12:40 ` Kevin Wolf
2 siblings, 0 replies; 7+ messages in thread
From: Stefan Hajnoczi @ 2021-07-29 16:15 UTC (permalink / raw)
To: Fabian Ebner
Cc: Kevin Wolf, open list:Linux io_uring, Stefan Hajnoczi,
Julia Suvorova, qemu-devel, Max Reitz, Aarushi Mehta,
Stefano Garzarella
[-- Attachment #1: Type: text/plain, Size: 957 bytes --]
On Thu, Jul 29, 2021 at 11:10:29AM +0200, Fabian Ebner wrote:
> Linux SCSI can throw spurious -EAGAIN in some corner cases in its
> completion path, which will end up being the result in the completed
> io_uring request.
>
> Resubmitting such requests should allow block jobs to complete, even
> if such spurious errors are encountered.
>
> Co-authored-by: Stefan Hajnoczi <stefanha@gmail.com>
> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
> Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
> ---
>
> Changes from v1:
> * Focus on what's relevant for the patch itself in the commit
> message.
> * Add Stefan's comment.
> * Add Stefano's R-b tag (I hope that's fine, since there was no
> change code-wise).
>
> block/io_uring.c | 16 +++++++++++++++-
> 1 file changed, 15 insertions(+), 1 deletion(-)
Thanks, applied to my block tree:
https://gitlab.com/stefanha/qemu/commits/block
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] block/io_uring: resubmit when result is -EAGAIN
2021-07-29 9:10 [PATCH v2] block/io_uring: resubmit when result is -EAGAIN Fabian Ebner
2021-07-29 9:54 ` Stefano Garzarella
2021-07-29 16:15 ` Stefan Hajnoczi
@ 2021-08-02 12:40 ` Kevin Wolf
2021-08-04 14:50 ` Stefano Garzarella
2 siblings, 1 reply; 7+ messages in thread
From: Kevin Wolf @ 2021-08-02 12:40 UTC (permalink / raw)
To: Fabian Ebner
Cc: open list:Linux io_uring, Stefan Hajnoczi, Julia Suvorova,
qemu-devel, Max Reitz, Stefan Hajnoczi, Aarushi Mehta,
Stefano Garzarella
Am 29.07.2021 um 11:10 hat Fabian Ebner geschrieben:
> Linux SCSI can throw spurious -EAGAIN in some corner cases in its
> completion path, which will end up being the result in the completed
> io_uring request.
>
> Resubmitting such requests should allow block jobs to complete, even
> if such spurious errors are encountered.
>
> Co-authored-by: Stefan Hajnoczi <stefanha@gmail.com>
> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
> Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
> ---
>
> Changes from v1:
> * Focus on what's relevant for the patch itself in the commit
> message.
> * Add Stefan's comment.
> * Add Stefano's R-b tag (I hope that's fine, since there was no
> change code-wise).
>
> block/io_uring.c | 16 +++++++++++++++-
> 1 file changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/block/io_uring.c b/block/io_uring.c
> index 00a3ee9fb8..dfa475cc87 100644
> --- a/block/io_uring.c
> +++ b/block/io_uring.c
> @@ -165,7 +165,21 @@ static void luring_process_completions(LuringState *s)
> total_bytes = ret + luringcb->total_read;
>
> if (ret < 0) {
> - if (ret == -EINTR) {
> + /*
> + * Only writev/readv/fsync requests on regular files or host block
> + * devices are submitted. Therefore -EAGAIN is not expected but it's
> + * known to happen sometimes with Linux SCSI. Submit again and hope
> + * the request completes successfully.
> + *
> + * For more information, see:
> + * https://lore.kernel.org/io-uring/20210727165811.284510-3-axboe@kernel.dk/T/#u
> + *
> + * If the code is changed to submit other types of requests in the
> + * future, then this workaround may need to be extended to deal with
> + * genuine -EAGAIN results that should not be resubmitted
> + * immediately.
> + */
> + if (ret == -EINTR || ret == -EAGAIN) {
> luring_resubmit(s, luringcb);
> continue;
> }
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Question about the preexisting code, though: luring_resubmit() requires
that the caller calls ioq_submit() later so that the request doesn't
just sit in a queue without getting any attention, but actually gets
submitted to the kernel.
In the call chain ioq_submit() -> luring_process_completions() ->
luring_resubmit(), who takes care of that?
Kevin
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] block/io_uring: resubmit when result is -EAGAIN
2021-08-02 12:40 ` Kevin Wolf
@ 2021-08-04 14:50 ` Stefano Garzarella
2021-08-04 16:52 ` Kevin Wolf
0 siblings, 1 reply; 7+ messages in thread
From: Stefano Garzarella @ 2021-08-04 14:50 UTC (permalink / raw)
To: Kevin Wolf
Cc: open list:Linux io_uring, Stefan Hajnoczi, Julia Suvorova,
qemu-devel, Max Reitz, Stefan Hajnoczi, Fabian Ebner,
Aarushi Mehta
On Mon, Aug 02, 2021 at 02:40:36PM +0200, Kevin Wolf wrote:
>Am 29.07.2021 um 11:10 hat Fabian Ebner geschrieben:
>> Linux SCSI can throw spurious -EAGAIN in some corner cases in its
>> completion path, which will end up being the result in the completed
>> io_uring request.
>>
>> Resubmitting such requests should allow block jobs to complete, even
>> if such spurious errors are encountered.
>>
>> Co-authored-by: Stefan Hajnoczi <stefanha@gmail.com>
>> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
>> Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
>> ---
>>
>> Changes from v1:
>> * Focus on what's relevant for the patch itself in the commit
>> message.
>> * Add Stefan's comment.
>> * Add Stefano's R-b tag (I hope that's fine, since there was no
>> change code-wise).
>>
>> block/io_uring.c | 16 +++++++++++++++-
>> 1 file changed, 15 insertions(+), 1 deletion(-)
>>
>> diff --git a/block/io_uring.c b/block/io_uring.c
>> index 00a3ee9fb8..dfa475cc87 100644
>> --- a/block/io_uring.c
>> +++ b/block/io_uring.c
>> @@ -165,7 +165,21 @@ static void luring_process_completions(LuringState *s)
>> total_bytes = ret + luringcb->total_read;
>>
>> if (ret < 0) {
>> - if (ret == -EINTR) {
>> + /*
>> + * Only writev/readv/fsync requests on regular files or host block
>> + * devices are submitted. Therefore -EAGAIN is not expected but it's
>> + * known to happen sometimes with Linux SCSI. Submit again and hope
>> + * the request completes successfully.
>> + *
>> + * For more information, see:
>> + * https://lore.kernel.org/io-uring/20210727165811.284510-3-axboe@kernel.dk/T/#u
>> + *
>> + * If the code is changed to submit other types of requests in the
>> + * future, then this workaround may need to be extended to deal with
>> + * genuine -EAGAIN results that should not be resubmitted
>> + * immediately.
>> + */
>> + if (ret == -EINTR || ret == -EAGAIN) {
>> luring_resubmit(s, luringcb);
>> continue;
>> }
>
>Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>
>Question about the preexisting code, though: luring_resubmit() requires
>that the caller calls ioq_submit() later so that the request doesn't
>just sit in a queue without getting any attention, but actually gets
>submitted to the kernel.
>
>In the call chain ioq_submit() -> luring_process_completions() ->
>luring_resubmit(), who takes care of that?
Mmm, good point.
There should be the same problem with ioq_submit() ->
luring_process_completions() -> luring_resubmit_short_read() ->
luring_resubmit().
Should we schedule a BH for example in luring_resubmit() to make sure
that ioq_submit() is invoked after a resubmission?
Thanks,
Stefano
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] block/io_uring: resubmit when result is -EAGAIN
2021-08-04 14:50 ` Stefano Garzarella
@ 2021-08-04 16:52 ` Kevin Wolf
2021-08-05 8:31 ` Stefano Garzarella
0 siblings, 1 reply; 7+ messages in thread
From: Kevin Wolf @ 2021-08-04 16:52 UTC (permalink / raw)
To: Stefano Garzarella
Cc: open list:Linux io_uring, Stefan Hajnoczi, Julia Suvorova,
qemu-devel, Max Reitz, Stefan Hajnoczi, Fabian Ebner,
Aarushi Mehta
Am 04.08.2021 um 16:50 hat Stefano Garzarella geschrieben:
> On Mon, Aug 02, 2021 at 02:40:36PM +0200, Kevin Wolf wrote:
> > Am 29.07.2021 um 11:10 hat Fabian Ebner geschrieben:
> > > Linux SCSI can throw spurious -EAGAIN in some corner cases in its
> > > completion path, which will end up being the result in the completed
> > > io_uring request.
> > >
> > > Resubmitting such requests should allow block jobs to complete, even
> > > if such spurious errors are encountered.
> > >
> > > Co-authored-by: Stefan Hajnoczi <stefanha@gmail.com>
> > > Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
> > > Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
> > > ---
> > >
> > > Changes from v1:
> > > * Focus on what's relevant for the patch itself in the commit
> > > message.
> > > * Add Stefan's comment.
> > > * Add Stefano's R-b tag (I hope that's fine, since there was no
> > > change code-wise).
> > >
> > > block/io_uring.c | 16 +++++++++++++++-
> > > 1 file changed, 15 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/block/io_uring.c b/block/io_uring.c
> > > index 00a3ee9fb8..dfa475cc87 100644
> > > --- a/block/io_uring.c
> > > +++ b/block/io_uring.c
> > > @@ -165,7 +165,21 @@ static void luring_process_completions(LuringState *s)
> > > total_bytes = ret + luringcb->total_read;
> > >
> > > if (ret < 0) {
> > > - if (ret == -EINTR) {
> > > + /*
> > > + * Only writev/readv/fsync requests on regular files or host block
> > > + * devices are submitted. Therefore -EAGAIN is not expected but it's
> > > + * known to happen sometimes with Linux SCSI. Submit again and hope
> > > + * the request completes successfully.
> > > + *
> > > + * For more information, see:
> > > + * https://lore.kernel.org/io-uring/20210727165811.284510-3-axboe@kernel.dk/T/#u
> > > + *
> > > + * If the code is changed to submit other types of requests in the
> > > + * future, then this workaround may need to be extended to deal with
> > > + * genuine -EAGAIN results that should not be resubmitted
> > > + * immediately.
> > > + */
> > > + if (ret == -EINTR || ret == -EAGAIN) {
> > > luring_resubmit(s, luringcb);
> > > continue;
> > > }
> >
> > Reviewed-by: Kevin Wolf <kwolf@redhat.com>
> >
> > Question about the preexisting code, though: luring_resubmit() requires
> > that the caller calls ioq_submit() later so that the request doesn't
> > just sit in a queue without getting any attention, but actually gets
> > submitted to the kernel.
> >
> > In the call chain ioq_submit() -> luring_process_completions() ->
> > luring_resubmit(), who takes care of that?
>
> Mmm, good point.
> There should be the same problem with ioq_submit() ->
> luring_process_completions() -> luring_resubmit_short_read() ->
> luring_resubmit().
>
> Should we schedule a BH for example in luring_resubmit() to make sure that
> ioq_submit() is invoked after a resubmission?
Or just loop in ioq_submit() after calling luring_process_completions()
if new requests were added to the queue?
Kevin
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] block/io_uring: resubmit when result is -EAGAIN
2021-08-04 16:52 ` Kevin Wolf
@ 2021-08-05 8:31 ` Stefano Garzarella
0 siblings, 0 replies; 7+ messages in thread
From: Stefano Garzarella @ 2021-08-05 8:31 UTC (permalink / raw)
To: Kevin Wolf
Cc: open list:Linux io_uring, Stefan Hajnoczi, Julia Suvorova,
qemu-devel, Max Reitz, Stefan Hajnoczi, Fabian Ebner,
Aarushi Mehta
On Wed, Aug 04, 2021 at 06:52:15PM +0200, Kevin Wolf wrote:
>Am 04.08.2021 um 16:50 hat Stefano Garzarella geschrieben:
>> On Mon, Aug 02, 2021 at 02:40:36PM +0200, Kevin Wolf wrote:
>> > Am 29.07.2021 um 11:10 hat Fabian Ebner geschrieben:
>> > > Linux SCSI can throw spurious -EAGAIN in some corner cases in its
>> > > completion path, which will end up being the result in the completed
>> > > io_uring request.
>> > >
>> > > Resubmitting such requests should allow block jobs to complete, even
>> > > if such spurious errors are encountered.
>> > >
>> > > Co-authored-by: Stefan Hajnoczi <stefanha@gmail.com>
>> > > Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
>> > > Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
>> > > ---
>> > >
>> > > Changes from v1:
>> > > * Focus on what's relevant for the patch itself in the commit
>> > > message.
>> > > * Add Stefan's comment.
>> > > * Add Stefano's R-b tag (I hope that's fine, since there was no
>> > > change code-wise).
>> > >
>> > > block/io_uring.c | 16 +++++++++++++++-
>> > > 1 file changed, 15 insertions(+), 1 deletion(-)
>> > >
>> > > diff --git a/block/io_uring.c b/block/io_uring.c
>> > > index 00a3ee9fb8..dfa475cc87 100644
>> > > --- a/block/io_uring.c
>> > > +++ b/block/io_uring.c
>> > > @@ -165,7 +165,21 @@ static void luring_process_completions(LuringState *s)
>> > > total_bytes = ret + luringcb->total_read;
>> > >
>> > > if (ret < 0) {
>> > > - if (ret == -EINTR) {
>> > > + /*
>> > > + * Only writev/readv/fsync requests on regular files or host block
>> > > + * devices are submitted. Therefore -EAGAIN is not expected but it's
>> > > + * known to happen sometimes with Linux SCSI. Submit again and hope
>> > > + * the request completes successfully.
>> > > + *
>> > > + * For more information, see:
>> > > + * https://lore.kernel.org/io-uring/20210727165811.284510-3-axboe@kernel.dk/T/#u
>> > > + *
>> > > + * If the code is changed to submit other types of requests in the
>> > > + * future, then this workaround may need to be extended to deal with
>> > > + * genuine -EAGAIN results that should not be resubmitted
>> > > + * immediately.
>> > > + */
>> > > + if (ret == -EINTR || ret == -EAGAIN) {
>> > > luring_resubmit(s, luringcb);
>> > > continue;
>> > > }
>> >
>> > Reviewed-by: Kevin Wolf <kwolf@redhat.com>
>> >
>> > Question about the preexisting code, though: luring_resubmit() requires
>> > that the caller calls ioq_submit() later so that the request doesn't
>> > just sit in a queue without getting any attention, but actually gets
>> > submitted to the kernel.
>> >
>> > In the call chain ioq_submit() -> luring_process_completions() ->
>> > luring_resubmit(), who takes care of that?
>>
>> Mmm, good point.
>> There should be the same problem with ioq_submit() ->
>> luring_process_completions() -> luring_resubmit_short_read() ->
>> luring_resubmit().
>>
>> Should we schedule a BH for example in luring_resubmit() to make sure that
>> ioq_submit() is invoked after a resubmission?
>
>Or just loop in ioq_submit() after calling luring_process_completions()
>if new requests were added to the queue?
>
I was just concerned that we might cycle a bit if a request always
returns -EAGAIN, while scheduling a task might give room for other
devices to queue other requests.
But maybe this happens so occasionally that we might not worry about
it...
Stefano
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-08-05 8:32 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-29 9:10 [PATCH v2] block/io_uring: resubmit when result is -EAGAIN Fabian Ebner
2021-07-29 9:54 ` Stefano Garzarella
2021-07-29 16:15 ` Stefan Hajnoczi
2021-08-02 12:40 ` Kevin Wolf
2021-08-04 14:50 ` Stefano Garzarella
2021-08-04 16:52 ` Kevin Wolf
2021-08-05 8:31 ` Stefano Garzarella
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).