Re: [LSF/MM/BPF TOPIC] block drivers in user space

From: Mike Christie <michael.christie@oracle.com>
To: Hannes Reinecke <hare@suse.de>,
	Gabriel Krisman Bertazi <krisman@collabora.com>,
	lsf-pc@lists.linux-foundation.org
Cc: linux-block@vger.kernel.org
Subject: Re: [LSF/MM/BPF TOPIC] block drivers in user space
Date: Mon, 14 Mar 2022 12:04:06 -0500	[thread overview]
Message-ID: <49964bde-da7b-24be-19ae-9d14c72ce631@oracle.com> (raw)
In-Reply-To: <57af4fe0-9042-729a-18e9-839a5f0a84ac@suse.de>

On 3/3/22 1:09 AM, Hannes Reinecke wrote:
> On 3/2/22 17:52, Mike Christie wrote:
>> On 2/21/22 1:59 PM, Gabriel Krisman Bertazi wrote:
>>> I'd like to discuss an interface to implement user space block devices,
>>> while avoiding local network NBD solutions.  There has been reiterated
>>
>> Besides the tcmu approach, I've also worked on the local nbd based
>> solution like here:
>>
>> https://urldefense.com/v3/__https://github.com/gluster/nbd-runner__;!!ACWV5N9M2RV99hQ!YY39rbV9MpaNUtr7ElzgcG1TyPznVEt1yppLwAGkq32-Fw9rQkqB6FzcaHiwIdgXp00K$
>> Have you looked into a modern take that uses io_uring's socket features
>> with the zero copy work that's being worked on for it? If so, what are
>> the issues you have hit with that? Was it mostly issues with the zero
>> copy part of it?
>>
>>
> Problem is that we'd need an _inverse_ io_uring interface.
> The current io_uring interface writes submission queue elements,
> and waits for completion queue elements.

I'm not sure what you meant here.

io_uring can do recvs right? So userspace nbd would do
IORING_OP_RECVMSG to wait for drivers/block/nbd.c to send userspace
cmds via the local socket. Userspace nbd would do IORING_OP_SENDMSG
to send drivers/block/nbd.c the cmd response.

drivers/block/nbd doesn't know/care what userspace did. It's just
reading/writing from/to the socket.