Re: [PATCH 0/3] Add support for Block Passthrough Endpoint function driver

From: Damien Le Moal <dlemoal@kernel.org>
To: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>,
	Wadim Mueller <wafgo01@gmail.com>
Cc: "Bjorn Helgaas" <bhelgaas@google.com>,
	"Jonathan Corbet" <corbet@lwn.net>,
	"Krzysztof Wilczyński" <kw@linux.com>,
	"Kishon Vijay Abraham I" <kishon@kernel.org>,
	"Jens Axboe" <axboe@kernel.dk>,
	"Lorenzo Pieralisi" <lpieralisi@kernel.org>,
	"Shunsuke Mie" <mie@igel.co.jp>,
	linux-pci@vger.kernel.org, linux-doc@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-block@vger.kernel.org
Subject: Re: [PATCH 0/3] Add support for Block Passthrough Endpoint function driver
Date: Mon, 26 Feb 2024 04:58:24 -0800	[thread overview]
Message-ID: <2f0f67d3-6a69-400b-b653-9353a3ddff02@kernel.org> (raw)
In-Reply-To: <20240226094530.GA2778@thinkpad>

On 2024/02/26 1:45, Manivannan Sadhasivam wrote:

[...]

>> As virtio is made mainly for Hypervisor <->
>> Guest communication I was afraid that a Hypersisor is able to Trap every
>> Register access from the Guest and act accordingly, which I would not be
>> able to do. I hope this make sense to you.
>>
> 
> I'm not worrying about the hypervisor right now. Here the endpoint is exposing
> the virtio devices and host is consuming it. There is no virtualization play
> here. I talked about this in the last plumbers [2].

FYI, we are still working on our NVMe PCI EPF function driver. It is working OK
using either a rockpro64 (PCI Gen2) board and a Radxa Rock 5B board (PCI Gen3,
rk3588 SoC/DWC EPF driver). Just been super busy recently with the block layer &
ATA stuff so I have not been able to rebase/cleanup and send stuff. This driver
also depends on many cleanup/improvement patches (see below).

> 
>> But to make a long story short, yes I agree with you that virtio-blk
>> would satisfy my usecase, and I generally think it would be a better
>> solution, I just did not know that you are working on some
>> infrastructure for that. And yes I would like to implement the endpoint
>> function driver for virtio-blk. Is there already an development tree you
>> use to work on the infrastructre I could have a look at?
>>
> 
> Shunsuke has a WIP branch [3], that I plan to co-work in the coming days.
> You can use it as a reference in the meantime.

This one is very similar to what I did in my series:

https://github.com/torvalds/linux/commit/05e21d458b1eaa8c22697f12a1ae42dcb04ff377

My series is here:

https://github.com/damien-lemoal/linux/tree/rock5b_ep_v8

It is a bit of a mess but what's there is:
1) Add the "map_info" EPF method to get mapping that are not dependent on the
host address alignment. That is similar to the align_mem method Shunsuke
introduced, but with more info to make it generic and allow EPF to deal with any
host DMA address.
2) Fixes for the rockpro64 DMA mapping as it is broken
3) Adds rk2588 EPF driver
4) Adds the NVMe EPF function driver. That is implemented as a PCI EPF frontend
to an NVMe-of controller so that any NMVe-Of supported device can be exposed
over PCI (block device, file, real NVMe controller).

There are also a bunch of API changes and cleanups to make the EPF code (core
and driver) more compact/easier to read.

Once I am done with my current work on the block layer side, I intend to come
back to this for the next cycle. I still need to complete the IRQ lehacy -> intx
renaming as well...

Cheers.

-- 
Damien Le Moal
Western Digital Research