Re: [LSF/MM/BPF TOPIC] block drivers in user space

[Hannes Reinecke <hare@suse.de>]

On 3/3/22 00:04, Gabriel Krisman Bertazi wrote:
> Hannes Reinecke <hare@suse.de> writes:
> 
>> Actually, I'd rather have something like an 'inverse io_uring', where an
>> application creates a memory region separated into several 'ring' for
>> submission and completion.
>> Then the kernel could write/map the incoming data onto the rings, and
>> application can read from there.
>> Maybe it'll be worthwhile to look at virtio here.
> 
>>
>> But in either case, using fds or pipes for commands doesn't really
>> scale, as the number of fds is inherently limited. And using fds
>> restricts you to serial processing (as you can read only sequentially
>> from a fd); with mmap() you'll get a greater flexibility and the option
>> of parallel processing.
> 
> Hannes,
> 
> I'm not trying to push an fd implementation, and seems clear that
> io_uring is the right way to go.  But isn't fd virtually unlimited,
> as they can be extended up to procfs's file-max for a specific user?
> 
In principle, yes. But in practice there will _always_ be a limit (even 
if you raise the limit you _still_ have a limit), as essentially the 
number of files is the size of the fd array table.
So you can only have a very large number of fds, but not an infinite 
number of fds.

And experience with multipath-tools have taught us that you hit the fd 
limit far more often than you thought; we've seen installations where we 
had to increase the fd limit beyond 4096
(Mind you, multipath-tools is using only two fds per device).

So on those installations we'll be running out of fds pretty fast if we 
start using one fd per I/O.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare@suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer