* [PATCH for-5.10] io_uring: remove req cancel in ->flush() @ 2020-10-19 15:45 Pavel Begunkov 2020-10-19 20:08 ` Jens Axboe 2020-10-22 6:42 ` Xiaoguang Wang 0 siblings, 2 replies; 9+ messages in thread From: Pavel Begunkov @ 2020-10-19 15:45 UTC (permalink / raw) To: Jens Axboe, io-uring Every close(io_uring) causes cancellation of all inflight requests carrying ->files. That's not nice but was neccessary up until recently. Now task->files removal is handled in the core code, so that part of flush can be removed. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- fs/io_uring.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 95d2bb7069c6..6536e24eb44e 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8748,16 +8748,12 @@ void __io_uring_task_cancel(void) static int io_uring_flush(struct file *file, void *data) { - struct io_ring_ctx *ctx = file->private_data; + bool exiting = !data; - /* - * If the task is going away, cancel work it may have pending - */ if (fatal_signal_pending(current) || (current->flags & PF_EXITING)) - data = NULL; + exiting = true; - io_uring_cancel_task_requests(ctx, data); - io_uring_attempt_task_drop(file, !data); + io_uring_attempt_task_drop(file, exiting); return 0; } -- 2.24.0 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH for-5.10] io_uring: remove req cancel in ->flush() 2020-10-19 15:45 [PATCH for-5.10] io_uring: remove req cancel in ->flush() Pavel Begunkov @ 2020-10-19 20:08 ` Jens Axboe 2020-10-19 23:40 ` Pavel Begunkov 2020-10-22 6:42 ` Xiaoguang Wang 1 sibling, 1 reply; 9+ messages in thread From: Jens Axboe @ 2020-10-19 20:08 UTC (permalink / raw) To: Pavel Begunkov, io-uring On 10/19/20 9:45 AM, Pavel Begunkov wrote: > Every close(io_uring) causes cancellation of all inflight requests > carrying ->files. That's not nice but was neccessary up until recently. > Now task->files removal is handled in the core code, so that part of > flush can be removed. It does change the behavior, but I'd wager that's safe. One minor comment: > diff --git a/fs/io_uring.c b/fs/io_uring.c > index 95d2bb7069c6..6536e24eb44e 100644 > --- a/fs/io_uring.c > +++ b/fs/io_uring.c > @@ -8748,16 +8748,12 @@ void __io_uring_task_cancel(void) > > static int io_uring_flush(struct file *file, void *data) > { > - struct io_ring_ctx *ctx = file->private_data; > + bool exiting = !data; > > - /* > - * If the task is going away, cancel work it may have pending > - */ > if (fatal_signal_pending(current) || (current->flags & PF_EXITING)) > - data = NULL; > + exiting = true; > > - io_uring_cancel_task_requests(ctx, data); > - io_uring_attempt_task_drop(file, !data); > + io_uring_attempt_task_drop(file, exiting); > return 0; > } Why not just keep the !data for task_drop? Would make the diff take away just the hunk we're interested in. Even adding a comment would be better, imho. -- Jens Axboe ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH for-5.10] io_uring: remove req cancel in ->flush() 2020-10-19 20:08 ` Jens Axboe @ 2020-10-19 23:40 ` Pavel Begunkov 2020-10-20 14:09 ` Jens Axboe 0 siblings, 1 reply; 9+ messages in thread From: Pavel Begunkov @ 2020-10-19 23:40 UTC (permalink / raw) To: Jens Axboe, io-uring On 19/10/2020 21:08, Jens Axboe wrote: > On 10/19/20 9:45 AM, Pavel Begunkov wrote: >> Every close(io_uring) causes cancellation of all inflight requests >> carrying ->files. That's not nice but was neccessary up until recently. >> Now task->files removal is handled in the core code, so that part of >> flush can be removed. > > It does change the behavior, but I'd wager that's safe. One minor > comment: Right, but I would think that users are not happy that every close kills requests without apparent reasons. > >> diff --git a/fs/io_uring.c b/fs/io_uring.c >> index 95d2bb7069c6..6536e24eb44e 100644 >> --- a/fs/io_uring.c >> +++ b/fs/io_uring.c >> @@ -8748,16 +8748,12 @@ void __io_uring_task_cancel(void) >> >> static int io_uring_flush(struct file *file, void *data) >> { >> - struct io_ring_ctx *ctx = file->private_data; >> + bool exiting = !data; >> >> - /* >> - * If the task is going away, cancel work it may have pending >> - */ >> if (fatal_signal_pending(current) || (current->flags & PF_EXITING)) >> - data = NULL; >> + exiting = true; >> >> - io_uring_cancel_task_requests(ctx, data); >> - io_uring_attempt_task_drop(file, !data); >> + io_uring_attempt_task_drop(file, exiting); >> return 0; >> } > > Why not just keep the !data for task_drop? Would make the diff take > away just the hunk we're interested in. Even adding a comment would be > better, imho. That would look cleaner, but I just left what already was there. TBH, I don't even entirely understand why exiting=!data. Looking up how exit_files() works, it passes down non-NULL files to put_files_struct() -> ... filp_close() -> f_op->flush(). I'm curious how does this filp_close(file, files=NULL) happens? Moreover, if that's exit_files() which is interesting, then first it calls io_uring_cancel_task_requests(), which should remove all struct file from tctx->xa. I haven't tested it though. -- Pavel Begunkov ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH for-5.10] io_uring: remove req cancel in ->flush() 2020-10-19 23:40 ` Pavel Begunkov @ 2020-10-20 14:09 ` Jens Axboe 2020-10-20 16:29 ` Pavel Begunkov 0 siblings, 1 reply; 9+ messages in thread From: Jens Axboe @ 2020-10-20 14:09 UTC (permalink / raw) To: Pavel Begunkov, io-uring On 10/19/20 5:40 PM, Pavel Begunkov wrote: > On 19/10/2020 21:08, Jens Axboe wrote: >> On 10/19/20 9:45 AM, Pavel Begunkov wrote: >>> Every close(io_uring) causes cancellation of all inflight requests >>> carrying ->files. That's not nice but was neccessary up until recently. >>> Now task->files removal is handled in the core code, so that part of >>> flush can be removed. >> >> It does change the behavior, but I'd wager that's safe. One minor >> comment: > > Right, but I would think that users are not happy that every close > kills requests without apparent reasons. To be fair, closing the ring fd is kind of unexpected and I would not expect any real applications to do that. If they did, I would have expected queries on why file table requests get canceled on close. Hence I'm not too worried about anyone hitting any issues related to this change, as the change is certainly one for the better. >>> diff --git a/fs/io_uring.c b/fs/io_uring.c >>> index 95d2bb7069c6..6536e24eb44e 100644 >>> --- a/fs/io_uring.c >>> +++ b/fs/io_uring.c >>> @@ -8748,16 +8748,12 @@ void __io_uring_task_cancel(void) >>> >>> static int io_uring_flush(struct file *file, void *data) >>> { >>> - struct io_ring_ctx *ctx = file->private_data; >>> + bool exiting = !data; >>> >>> - /* >>> - * If the task is going away, cancel work it may have pending >>> - */ >>> if (fatal_signal_pending(current) || (current->flags & PF_EXITING)) >>> - data = NULL; >>> + exiting = true; >>> >>> - io_uring_cancel_task_requests(ctx, data); >>> - io_uring_attempt_task_drop(file, !data); >>> + io_uring_attempt_task_drop(file, exiting); >>> return 0; >>> } >> >> Why not just keep the !data for task_drop? Would make the diff take >> away just the hunk we're interested in. Even adding a comment would be >> better, imho. > > That would look cleaner, but I just left what already was there. TBH, > I don't even entirely understand why exiting=!data. Looking up how > exit_files() works, it passes down non-NULL files to > put_files_struct() -> ... filp_close() -> f_op->flush(). > > I'm curious how does this filp_close(file, files=NULL) happens? It doesn't, we just clear it internall to match all requests, not just files backed ones. > Moreover, if that's exit_files() which is interesting, then first > it calls io_uring_cancel_task_requests(), which should remove all > struct file from tctx->xa. I haven't tested it though. Yep, further cleanups are certainly possible there. I've queued this up, thanks. -- Jens Axboe ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH for-5.10] io_uring: remove req cancel in ->flush() 2020-10-20 14:09 ` Jens Axboe @ 2020-10-20 16:29 ` Pavel Begunkov 2020-10-20 16:59 ` Jens Axboe 0 siblings, 1 reply; 9+ messages in thread From: Pavel Begunkov @ 2020-10-20 16:29 UTC (permalink / raw) To: Jens Axboe, io-uring On 20/10/2020 15:09, Jens Axboe wrote: > On 10/19/20 5:40 PM, Pavel Begunkov wrote: >> On 19/10/2020 21:08, Jens Axboe wrote: >>> On 10/19/20 9:45 AM, Pavel Begunkov wrote: >>>> Every close(io_uring) causes cancellation of all inflight requests >>>> carrying ->files. That's not nice but was neccessary up until recently. >>>> Now task->files removal is handled in the core code, so that part of >>>> flush can be removed. >>> >>> Why not just keep the !data for task_drop? Would make the diff take >>> away just the hunk we're interested in. Even adding a comment would be >>> better, imho. >> >> That would look cleaner, but I just left what already was there. TBH, >> I don't even entirely understand why exiting=!data. Looking up how >> exit_files() works, it passes down non-NULL files to >> put_files_struct() -> ... filp_close() -> f_op->flush(). >> >> I'm curious how does this filp_close(file, files=NULL) happens? > > It doesn't, we just clear it internall to match all requests, not just > files backed ones. Then my "bool exiting = !data;" at the start doesn't make sense since passed in @data is always non-NULL. > >> Moreover, if that's exit_files() which is interesting, then first >> it calls io_uring_cancel_task_requests(), which should remove all >> struct file from tctx->xa. I haven't tested it though. > > Yep, further cleanups are certainly possible there. > > I've queued this up, thanks. > -- Pavel Begunkov ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH for-5.10] io_uring: remove req cancel in ->flush() 2020-10-20 16:29 ` Pavel Begunkov @ 2020-10-20 16:59 ` Jens Axboe 0 siblings, 0 replies; 9+ messages in thread From: Jens Axboe @ 2020-10-20 16:59 UTC (permalink / raw) To: Pavel Begunkov, io-uring On 10/20/20 10:29 AM, Pavel Begunkov wrote: > On 20/10/2020 15:09, Jens Axboe wrote: >> On 10/19/20 5:40 PM, Pavel Begunkov wrote: >>> On 19/10/2020 21:08, Jens Axboe wrote: >>>> On 10/19/20 9:45 AM, Pavel Begunkov wrote: >>>>> Every close(io_uring) causes cancellation of all inflight requests >>>>> carrying ->files. That's not nice but was neccessary up until recently. >>>>> Now task->files removal is handled in the core code, so that part of >>>>> flush can be removed. >>>> >>>> Why not just keep the !data for task_drop? Would make the diff take >>>> away just the hunk we're interested in. Even adding a comment would be >>>> better, imho. >>> >>> That would look cleaner, but I just left what already was there. TBH, >>> I don't even entirely understand why exiting=!data. Looking up how >>> exit_files() works, it passes down non-NULL files to >>> put_files_struct() -> ... filp_close() -> f_op->flush(). >>> >>> I'm curious how does this filp_close(file, files=NULL) happens? >> >> It doesn't, we just clear it internall to match all requests, not just >> files backed ones. > > Then my "bool exiting = !data;" at the start doesn't make sense since > passed in @data is always non-NULL. Right, only if we retain this part: if (fatal_signal_pending(current) || (current->flags & PF_EXITING)) data = NULL; in there. I suspect we'll want something ala the below instead. diff --git a/fs/io_uring.c b/fs/io_uring.c index 09e7a5f20060..4870db000f04 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8695,14 +8695,18 @@ static void __io_uring_attempt_task_drop(struct file *file) * Drop task note for this file if we're the only ones that hold it after * pending fput() */ -static void io_uring_attempt_task_drop(struct file *file, bool exiting) +static void io_uring_attempt_task_drop(struct file *file) { + bool exiting; + if (!current->io_uring) return; + /* * fput() is pending, will be 2 if the only other ref is our potential * task file note. If the task is exiting, drop regardless of count. */ + exiting = fatal_signal_pending(current) || (current->flags & PF_EXITING); if (!exiting && atomic_long_read(&file->f_count) != 2) return; @@ -8764,16 +8768,7 @@ void __io_uring_task_cancel(void) static int io_uring_flush(struct file *file, void *data) { - struct io_ring_ctx *ctx = file->private_data; - - /* - * If the task is going away, cancel work it may have pending - */ - if (fatal_signal_pending(current) || (current->flags & PF_EXITING)) - data = NULL; - - io_uring_cancel_task_requests(ctx, data); - io_uring_attempt_task_drop(file, !data); + io_uring_attempt_task_drop(file); return 0; } -- Jens Axboe ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH for-5.10] io_uring: remove req cancel in ->flush() 2020-10-19 15:45 [PATCH for-5.10] io_uring: remove req cancel in ->flush() Pavel Begunkov 2020-10-19 20:08 ` Jens Axboe @ 2020-10-22 6:42 ` Xiaoguang Wang 2020-10-22 11:44 ` Pavel Begunkov 1 sibling, 1 reply; 9+ messages in thread From: Xiaoguang Wang @ 2020-10-22 6:42 UTC (permalink / raw) To: Pavel Begunkov, Jens Axboe, io-uring hi, > Every close(io_uring) causes cancellation of all inflight requests > carrying ->files. That's not nice but was neccessary up until recently. > Now task->files removal is handled in the core code, so that part of > flush can be removed. I don't catch up with newest io_uring codes yet, but have one question about the initial implementation "io_uring: io_uring: add support for async work inheriting files": https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fcb323cc53e29d9cc696d606bb42736b32dd9825 There was such comments: +static int io_grab_files(struct io_ring_ctx *ctx, struct io_kiocb *req) +{ + int ret = -EBADF; + + rcu_read_lock(); + spin_lock_irq(&ctx->inflight_lock); + /* + * We use the f_ops->flush() handler to ensure that we can flush + * out work accessing these files if the fd is closed. Check if ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ I wonder why we only need to flush reqs specifically when they access current->files, are there any special reasons? Regards, Xiaoguang Wang + * the fd has changed since we started down this path, and disallow + * this operation if it has. + */ + if (fcheck(req->submit.ring_fd) == req->submit.ring_file) { + list_add(&req->inflight_entry, &ctx->inflight_list); + req->flags |= REQ_F_INFLIGHT; + req->work.files = current->files; + ret = 0; + } > > Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> > --- > fs/io_uring.c | 10 +++------- > 1 file changed, 3 insertions(+), 7 deletions(-) > > diff --git a/fs/io_uring.c b/fs/io_uring.c > index 95d2bb7069c6..6536e24eb44e 100644 > --- a/fs/io_uring.c > +++ b/fs/io_uring.c > @@ -8748,16 +8748,12 @@ void __io_uring_task_cancel(void) > > static int io_uring_flush(struct file *file, void *data) > { > - struct io_ring_ctx *ctx = file->private_data; > + bool exiting = !data; > > - /* > - * If the task is going away, cancel work it may have pending > - */ > if (fatal_signal_pending(current) || (current->flags & PF_EXITING)) > - data = NULL; > + exiting = true; > > - io_uring_cancel_task_requests(ctx, data); > - io_uring_attempt_task_drop(file, !data); > + io_uring_attempt_task_drop(file, exiting); > return 0; > } > > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH for-5.10] io_uring: remove req cancel in ->flush() 2020-10-22 6:42 ` Xiaoguang Wang @ 2020-10-22 11:44 ` Pavel Begunkov 2020-10-23 3:33 ` Xiaoguang Wang 0 siblings, 1 reply; 9+ messages in thread From: Pavel Begunkov @ 2020-10-22 11:44 UTC (permalink / raw) To: Xiaoguang Wang, Jens Axboe, io-uring On 22/10/2020 07:42, Xiaoguang Wang wrote: >> Every close(io_uring) causes cancellation of all inflight requests >> carrying ->files. That's not nice but was neccessary up until recently. >> Now task->files removal is handled in the core code, so that part of >> flush can be removed. > I don't catch up with newest io_uring codes yet, but have one question about > the initial implementation "io_uring: io_uring: add support for async work > inheriting files": https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fcb323cc53e29d9cc696d606bb42736b32dd9825 > There was such comments: > +static int io_grab_files(struct io_ring_ctx *ctx, struct io_kiocb *req) > +{ > + int ret = -EBADF; > + > + rcu_read_lock(); > + spin_lock_irq(&ctx->inflight_lock); > + /* > + * We use the f_ops->flush() handler to ensure that we can flush > + * out work accessing these files if the fd is closed. Check if > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > I wonder why we only need to flush reqs specifically when they access current->files, are there > any special reasons? We could have taken a reference to ->files, so it doesn't get freed while a request is using it, but that creates a circular dependency. IIUC, because if there are ->files refs io_uring won't be closed on exit, but io_uring would be holding ->files refs. So, it was working without taking ->files references (i.e. weak refs) on the basis that the files won't be destroyed until the task itself is gone, and we can *kind of* intercept when task is exiting by ->flush() and cancel all requests with ->files there. -- Pavel Begunkov ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH for-5.10] io_uring: remove req cancel in ->flush() 2020-10-22 11:44 ` Pavel Begunkov @ 2020-10-23 3:33 ` Xiaoguang Wang 0 siblings, 0 replies; 9+ messages in thread From: Xiaoguang Wang @ 2020-10-23 3:33 UTC (permalink / raw) To: Pavel Begunkov, Jens Axboe, io-uring hi, > On 22/10/2020 07:42, Xiaoguang Wang wrote: >>> Every close(io_uring) causes cancellation of all inflight requests >>> carrying ->files. That's not nice but was neccessary up until recently. >>> Now task->files removal is handled in the core code, so that part of >>> flush can be removed. >> I don't catch up with newest io_uring codes yet, but have one question about >> the initial implementation "io_uring: io_uring: add support for async work >> inheriting files": https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fcb323cc53e29d9cc696d606bb42736b32dd9825 >> There was such comments: >> +static int io_grab_files(struct io_ring_ctx *ctx, struct io_kiocb *req) >> +{ >> + int ret = -EBADF; >> + >> + rcu_read_lock(); >> + spin_lock_irq(&ctx->inflight_lock); >> + /* >> + * We use the f_ops->flush() handler to ensure that we can flush >> + * out work accessing these files if the fd is closed. Check if >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> I wonder why we only need to flush reqs specifically when they access current->files, are there >> any special reasons? > We could have taken a reference to ->files, so it doesn't get freed > while a request is using it, but that creates a circular dependency. > > IIUC, because if there are ->files refs io_uring won't be closed on exit, > but io_uring would be holding ->files refs. > > So, it was working without taking ->files references (i.e. weak refs) > on the basis that the files won't be destroyed until the task itself is > gone, and we can *kind of* intercept when task is exiting by ->flush() > and cancel all requests with ->files there. Now I see :) will help me to understand current codes better. Really thanks for your explanations. Regards, Xiaoguang Wang > ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-10-23 3:34 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-10-19 15:45 [PATCH for-5.10] io_uring: remove req cancel in ->flush() Pavel Begunkov 2020-10-19 20:08 ` Jens Axboe 2020-10-19 23:40 ` Pavel Begunkov 2020-10-20 14:09 ` Jens Axboe 2020-10-20 16:29 ` Pavel Begunkov 2020-10-20 16:59 ` Jens Axboe 2020-10-22 6:42 ` Xiaoguang Wang 2020-10-22 11:44 ` Pavel Begunkov 2020-10-23 3:33 ` Xiaoguang Wang
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.