Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf

From: "Christian König" <christian.koenig@amd.com>
To: Christoph Hellwig <hch@infradead.org>
Cc: Leon Romanovsky <leon@kernel.org>,
	kvm@vger.kernel.org, linux-rdma@vger.kernel.org,
	Daniel Vetter <daniel.vetter@ffwll.ch>,
	Oded Gabbay <ogabbay@kernel.org>,
	Cornelia Huck <cohuck@redhat.com>,
	dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org,
	Alex Williamson <alex.williamson@redhat.com>,
	Jason Gunthorpe <jgg@nvidia.com>,
	Maor Gottlieb <maorg@nvidia.com>,
	Sumit Semwal <sumit.semwal@linaro.org>,
	linux-media@vger.kernel.org
Subject: Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf
Date: Wed, 7 Sep 2022 17:08:37 +0200	[thread overview]
Message-ID: <58d6e892-82df-7aa7-4798-9e5da7c634ad@amd.com> (raw)
In-Reply-To: <YxiIkh/yeWQkZ54x@infradead.org>

Am 07.09.22 um 14:03 schrieb Christoph Hellwig:
> On Tue, Sep 06, 2022 at 12:38:44PM +0200, Christian König wrote:
>> The problem is once more that this is MMIO space, in other words register
>> BARs which needs to be exported/imported.
> Everything used for P2P is bar space.
>
>> Adding struct pages for it generally sounds like the wrong approach here.
>> You can't even access this with the CPU or would trigger potentially
>> unwanted hardware actions.
> How would an access from the CPU vs anther device make any difference?

The key point is that you can't do any CPU fallback with this as long as 
the CPU wouldn't do exactly the same thing as the original hardware 
device. E.g. not write combine nor do any fully page copies etc...

See what happens here is not really P2P DMA transfer, but rather P2P 
signaling of events.

For a simple example think of a camera and a video codec. The camera is 
pumping video data into system memory the video codec should encode into 
an H264 stream.

So after every frame the camera hardware issues a P2P write into the BAR 
of the video codec to signal that the frame is completed and it can 
start decoding.

This is not even a memory write, but rather just some trigger event. 
That's essentially the background why I think having struct pages and 
sg_tables doesn't make any sense at all for this use case.

>> Would you mind if I start to tackle this problem?
> Yes, I've been waiting forever for someone to tacke how the scatterlist
> is abused in dma-buf..

How about we separate the scatterlist into page and DMA address container?

Regards,
Christian.