Question about fileset unregister and update codes

Question about fileset unregister and update codes

[Xiaoguang Wang]

hi,

Look at below function:
static bool io_register_op_must_quiesce(int op)
{
	switch (op) {
	case IORING_UNREGISTER_FILES:
	case IORING_REGISTER_FILES_UPDATE:
	case IORING_REGISTER_PROBE:
	case IORING_REGISTER_PERSONALITY:
	case IORING_UNREGISTER_PERSONALITY:
		return false;
	default:
		return true;
	}
}

IORING_REGISTER_FILES will quiesces the ctx, but IORING_UNREGISTER_FILES
and IORING_REGISTER_FILES_UPDATE won't, so I wonder how userspace applications
can sure when they can unregister or update the registered fileset.
Imagine below application behaviour:
      ThreadA                     |      ThreadB
                                  |
     1, register a file           |     while (1) {
     2, prepare a batch of sqes   |         wait a cqe and handle this cqe.
     3, submit prepared sqes      |     }
     4, unregister or update file |
                                  |

If IORING_SETUP_SQPOLL is not enabled, I think step4 is safe, because step3
will ensure that all sqes will be prepared by io_req_set_file(), then corresponding
struct file will be resolved and ctx->file_data->refs will be increased, we know
unless requst is completed, struct file will not be put, finally we know step4
is safe.

But if IORING_SETUP_SQPOLL is enabled, step3 will complete quickly, if kernel
thread io_sq_thread submit sqes before step4, everything is ok, but if step4
starts to run, holding uring_lock, later io_sq_thread starts to handle sqes and
all previously submitted sqes will be returned with EBADF.

I'm not sure whether should make application ensure that all sqes against registered
file to complete, then app can unregister or update fileset, if so, I think it'll
introduce extra programming overhead to programmer.

Or if IORING_SETUP_SQPOLL is enabled, we call io_submit_sqes in unregister/update codes,
then applications will not need to worry about above race.

Regards,
Xiaoguang Wang

Re: Question about fileset unregister and update codes

[Xiaoguang Wang]

hi,

Could you please have a look at this mail? Thanks.

Regards,
Xiaoguang Wang

> hi,
> 
> Look at below function:
> static bool io_register_op_must_quiesce(int op)
> {
>      switch (op) {
>      case IORING_UNREGISTER_FILES:
>      case IORING_REGISTER_FILES_UPDATE:
>      case IORING_REGISTER_PROBE:
>      case IORING_REGISTER_PERSONALITY:
>      case IORING_UNREGISTER_PERSONALITY:
>          return false;
>      default:
>          return true;
>      }
> }
> 
> IORING_REGISTER_FILES will quiesces the ctx, but IORING_UNREGISTER_FILES
> and IORING_REGISTER_FILES_UPDATE won't, so I wonder how userspace applications
> can sure when they can unregister or update the registered fileset.
> Imagine below application behaviour:
>       ThreadA                     |      ThreadB
>                                   |
>      1, register a file           |     while (1) {
>      2, prepare a batch of sqes   |         wait a cqe and handle this cqe.
>      3, submit prepared sqes      |     }
>      4, unregister or update file |
>                                   |
> 
> If IORING_SETUP_SQPOLL is not enabled, I think step4 is safe, because step3
> will ensure that all sqes will be prepared by io_req_set_file(), then corresponding
> struct file will be resolved and ctx->file_data->refs will be increased, we know
> unless requst is completed, struct file will not be put, finally we know step4
> is safe.
> 
> But if IORING_SETUP_SQPOLL is enabled, step3 will complete quickly, if kernel
> thread io_sq_thread submit sqes before step4, everything is ok, but if step4
> starts to run, holding uring_lock, later io_sq_thread starts to handle sqes and
> all previously submitted sqes will be returned with EBADF.
> 
> I'm not sure whether should make application ensure that all sqes against registered
> file to complete, then app can unregister or update fileset, if so, I think it'll
> introduce extra programming overhead to programmer.
> 
> Or if IORING_SETUP_SQPOLL is enabled, we call io_submit_sqes in unregister/update codes,
> then applications will not need to worry about above race.
> 
> Regards,
> Xiaoguang Wang