Re: [LSF/MM/BPF TOPIC] block drivers in user space

From: Gabriel Krisman Bertazi <krisman@collabora.com>
To: Damien Le Moal <damien.lemoal@opensource.wdc.com>
Cc: lsf-pc@lists.linux-foundation.org, linux-block@vger.kernel.org
Subject: Re: [LSF/MM/BPF TOPIC] block drivers in user space
Date: Mon, 21 Feb 2022 18:30:25 -0500	[thread overview]
Message-ID: <87fsobg5m6.fsf@collabora.com> (raw)
In-Reply-To: <18da367b-bbe1-c591-358e-6f8111a90eaf@opensource.wdc.com> (Damien Le Moal's message of "Tue, 22 Feb 2022 08:16:21 +0900")

Damien Le Moal <damien.lemoal@opensource.wdc.com> writes:

> On 2/22/22 04:59, Gabriel Krisman Bertazi wrote:
>> I'd like to discuss an interface to implement user space block devices,
>> while avoiding local network NBD solutions.  There has been reiterated
>> interest in the topic, both from researchers [1] and from the community,
>> including a proposed session in LSFMM2018 [2] (though I don't think it
>> happened).
>> 
>> I've been working on top of the Google iblock implementation to find
>> something upstreamable and would like to present my design and gather
>> feedback on some points, in particular zero-copy and overall user space
>> interface.
>> 
>> The design I'm pending towards uses special fds opened by the driver to
>> transfer data to/from the block driver, preferably through direct
>> splicing as much as possible, to keep data only in kernel space.  This
>> is because, in my use case, the driver usually only manipulates
>> metadata, while data is forwarded directly through the network, or
>> similar. It would be neat if we can leverage the existing
>> splice/copy_file_range syscalls such that we don't ever need to bring
>> disk data to user space, if we can avoid it.  I've also experimented
>> with regular pipes, But I found no way around keeping a lot of pipes
>> opened, one for each possible command 'slot'.
>> 
>> [1] https://dl.acm.org/doi/10.1145/3456727.3463768
>
> This is $15 for non ACM members... Any public download available ?

Hi Damien,

Yes. Sorry for the ACM link.  yuck.  One of the authors published the
paper here:

https://rgmacedo.github.io/files/2021/systor21-bdus/bdus-paper.pdf

>
>> [2] https://www.spinics.net/lists/linux-fsdevel/msg120674.html
>> 

-- 
Gabriel Krisman Bertazi