io_uring: buf_index and registered buffers

io_uring: buf_index and registered buffers

[Stephen Bates]

Hi Jens and Team

While experimenting with io_uring and liburing I seem to have hit an issue with registered buffers. I can't see a way that either io_uring_register() or liburing can return the buffer indicies associated with registered buffers. In addition, there is no way to set sqe.buf_index via liburing right now when submitting an IO (it is memset to 0 in liburing.h). The end result is that io_uring IO start to fail when we do them from a set of registered buffers.

I'd be happy to work on a patch for this but could do with confirmation that this a problem and some guidance on a good approach to fixing this in both the system call and the user-space library.

BTW while liburing tests the creation of multiple registered buffers it never issues IO against them so that would be a hole I'd like to fill too.

Cheers
 
Stephen
 

Re: io_uring: buf_index and registered buffers

[Jens Axboe]

On 6/11/19 9:11 AM, Stephen  Bates wrote:
> Hi Jens and Team
> 
> While experimenting with io_uring and liburing I seem to have hit an
> issue with registered buffers. I can't see a way that either
> io_uring_register() or liburing can return the buffer indicies
> associated with registered buffers. In addition, there is no way to

Why would it need to return them? The indices are the index into the
array you passed in for registration.

> set sqe.buf_index via liburing right now when submitting an IO (it is
> memset to 0 in liburing.h). The end result is that io_uring IO start
> to fail when we do them from a set of registered buffers.

liburing is missing a lot of helpers right now, I'll happily take
patches to bring it closer to full feature parity.

> I'd be happy to work on a patch for this but could do with confirmation that this a problem and some guidance on a good approach to fixing this in both the system call and the user-space library.
> 
> BTW while liburing tests the creation of multiple registered buffers
> it never issues IO against them so that would be a hole I'd like to
> fill too.

Fio uses them, as does t/io_uring from fio. I've done lots of testing
with registered buffers, all the performance numbers done were with that
enabled (unless otherwise noted).

But yes, a test case is always a great idea!

-- 
Jens Axboe

Re: io_uring: buf_index and registered buffers

[Stephen Bates]

>> While experimenting with io_uring and liburing I seem to have hit an
>> issue with registered buffers. I can't see a way that either
>> io_uring_register() or liburing can return the buffer indicies
>> associated with registered buffers. In addition, there is no way to
>    
>    Why would it need to return them? The indices are the index into the
>    array you passed in for registration.

Ah this is the part I was missing. I did not know we could be assured the indices passed back were identical to those passed in.

> Fio uses them, as does t/io_uring from fio. I've done lots of testing
>    with registered buffers, all the performance numbers done were with that
>    enabled (unless otherwise noted).
>    
>    But yes, a test case is always a great idea!

Cool. I will go look at what the test in fio does and see if a testcase in liburing makes sense.

Cheers

Stephen