Re: [PATCH 0/4] Allow MMIO regions to be exported through dma-buf

From: "Christian König" <christian.koenig@amd.com>
To: Jason Gunthorpe <jgg@nvidia.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	Cornelia Huck <cohuck@redhat.com>,
	dri-devel@lists.freedesktop.org, kvm@vger.kernel.org,
	linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org,
	Sumit Semwal <sumit.semwal@linaro.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>,
	Leon Romanovsky <leon@kernel.org>,
	linux-rdma@vger.kernel.org, Maor Gottlieb <maorg@nvidia.com>,
	Oded Gabbay <ogabbay@kernel.org>
Subject: Re: [PATCH 0/4] Allow MMIO regions to be exported through dma-buf
Date: Thu, 18 Aug 2022 13:07:16 +0200	[thread overview]
Message-ID: <921de79a-9cb3-4217-f079-4b23958a16aa@amd.com> (raw)
In-Reply-To: <0-v1-9e6e1739ed95+5fa-vfio_dma_buf_jgg@nvidia.com>

Am 17.08.22 um 18:11 schrieb Jason Gunthorpe:
> dma-buf has become a way to safely acquire a handle to non-struct page
> memory that can still have lifetime controlled by the exporter. Notably
> RDMA can now import dma-buf FDs and build them into MRs which allows for
> PCI P2P operations. Extend this to allow vfio-pci to export MMIO memory
> from PCI device BARs.
>
> This series supports a use case for SPDK where a NVMe device will be owned
> by SPDK through VFIO but interacting with a RDMA device. The RDMA device
> may directly access the NVMe CMB or directly manipulate the NVMe device's
> doorbell using PCI P2P.
>
> However, as a general mechanism, it can support many other scenarios with
> VFIO. I imagine this dmabuf approach to be usable by iommufd as well for
> generic and safe P2P mappings.

In general looks good to me, but we really need to get away from using 
sg_tables for this here.

The only thing I'm not 100% convinced of is dma_buf_try_get(), I've seen 
this incorrectly used so many times that I can't count them any more.

Would that be somehow avoidable? Or could you at least explain the use 
case a bit better.

Thanks,
Christian.

>
> This series goes after the "Break up ioctl dispatch functions to one
> function per ioctl" series.
>
> This is on github: https://github.com/jgunthorpe/linux/commits/vfio_dma_buf
>
> Jason Gunthorpe (4):
>    dma-buf: Add dma_buf_try_get()
>    vfio: Add vfio_device_get()
>    vfio_pci: Do not open code pci_try_reset_function()
>    vfio/pci: Allow MMIO regions to be exported through dma-buf
>
>   drivers/vfio/pci/Makefile           |   1 +
>   drivers/vfio/pci/vfio_pci_config.c  |  22 ++-
>   drivers/vfio/pci/vfio_pci_core.c    |  33 +++-
>   drivers/vfio/pci/vfio_pci_dma_buf.c | 265 ++++++++++++++++++++++++++++
>   drivers/vfio/pci/vfio_pci_priv.h    |  24 +++
>   drivers/vfio/vfio_main.c            |   3 +-
>   include/linux/dma-buf.h             |  13 ++
>   include/linux/vfio.h                |   6 +
>   include/linux/vfio_pci_core.h       |   1 +
>   include/uapi/linux/vfio.h           |  18 ++
>   10 files changed, 364 insertions(+), 22 deletions(-)
>   create mode 100644 drivers/vfio/pci/vfio_pci_dma_buf.c
>
>
> base-commit: 385f0411fcd2780b5273992832cdc8edcd5b8ea9