Re: [PATCH] io_uring: add io_uring_enter(2) fixed file support

[Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>]

hi,

>>> I'll take a look at liburing and see what we need to do there. I think
>>> the sanest thing to do here is say that using a registered ring fd means
>>> you cannot share the ring, ever. And then just have a
>>> ring->enter_ring_fd which is normally just set to ring_fd when the ring
>>> is setup, and if you register the ring fd, then we set it to whatever
>>> the registered value is. Everything calling io_uring_enter() then just
>>> needs to be modified to use ->enter_ring_fd instead of ->ring_fd.
>> ok, look forward to use this api.
> Can you take a look at the registered-ring branch for liburing:
>
> https://git.kernel.dk/cgit/liburing/log/?h=registered-ring
>
> which has the basic plumbing for it. Comments (or patches) welcome!
Sorry for late reply, spend time to read your patch today. Basically it 
looks ok,
there is one minor issue in "Add preliminary support for using a 
registered ring fd":
@@ -417,6 +425,10 @@ struct io_uring_sqe *io_uring_get_sqe(struct 
io_uring *ring)

  int __io_uring_sqring_wait(struct io_uring *ring)
  {
-    return  ____sys_io_uring_enter(ring->ring_fd, 0, 0,
-                       IORING_ENTER_SQ_WAIT, NULL);
+    int flags = IORING_ENTER_SQ_WAIT;
+
+    if (ring->int_flags & INT_FLAG_REG_RING)
+        flags |= IORING_ENTER_REGISTERED_RING;
+
+    return  ____sys_io_uring_enter(ring->ring_fd, 0, 0, flags, NULL);
  }

Here it should be enter_ring_fd.

>
> Few things I don't really love:
>
> 1) You need to call io_uring_register_ring_fd() after setting up the
>     ring. We could provide init helpers for that, which just do queue
>     init and then register ring. Maybe that'd make it more likely to get
>     picked up by applications.
Agree, that'd be better in some cases, but consider that currently the 
capacity of ring
fd cache is just 16, I'd suggest to let users make their own decisions, 
in case some
ring fds could not allocate one empty slot, but some ring fds don't need 
them at all,
for example, ring fd which enable sqpoll may no need this feature.

>
> 2) For the setup where you do share the ring between a submitter and
>     reaper, we need to ensure that the registered ring fd is the same
>     between both of them. We need a helper for that. It's basically the
>     same as io_uring_register_ring_fd(), but we need the specific offset.
>     And if that fails with -EBUSY, we should just turn off
>     INT_FLAG_RING_REG for the ring and you don't get the registered fd
>     for either of them. At least it can be handled transparantly.
Storing enter_ring_fd in struct io_uring seems not good, struct io_uring 
is a shared struct,
as what you say, different threads that share one ring fd may have 
differed offset in ring fd
cache. I have two suggestions:
1) Threads keep their offset in ring fd cache alone, and pass it to 
io_uring_submit, which may
look ugly :)
2) define enter_ring_fd in struct io_ring to be a thread_local type, 
then your patches don't
need to do any modifications.

Regards,
Xiaoguang Wang
>
>>>>> Anyway, current version below. Only real change here is allowing either
>>>>> specific offset or generated offset, depending on what the
>>>>> io_uring_rsrc_update->offset is set to. If set to -1U, then io_uring
>>>>> will find a free offset. If set to anything else, io_uring will use that
>>>>> index (as long as it's >=0 && < MAX).
>>>> Seems you forgot to attach the newest version, and also don't see a
>>>> patch attachment. Finally, thanks for your quick response and many
>>>> code improvements, really appreciate it.
>>> Oops, below now. How do you want to handle this patch? It's now a bit of
>>> a mix of both of our stuff...
>> Since you have almost rewritten most of my original patch and now it
>> looks much better, so I would suggest just adds my Reported-by :)
> OK I'll post it, but Co-developed-by is probably a better one.
>