Re: [PATCH 4/6] drm/amdgpu: add checks if DMA-buf P2P is supported

From: Jason Gunthorpe <jgg@ziepe.ca>
To: "Christian König" <ckoenig.leichtzumerken@gmail.com>
Cc: David1.Zhou@amd.com, hch@infradead.org, daniel@ffwll.ch,
	dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org,
	linux-media@vger.kernel.org, intel-gfx@lists.freedesktop.org,
	Logan Gunthorpe <logang@deltatee.com>
Subject: Re: [PATCH 4/6] drm/amdgpu: add checks if DMA-buf P2P is supported
Date: Wed, 11 Mar 2020 11:04:15 -0300	[thread overview]
Message-ID: <20200311140415.GB31668@ziepe.ca> (raw)
In-Reply-To: <20200311135158.3310-5-christian.koenig@amd.com>

On Wed, Mar 11, 2020 at 02:51:56PM +0100, Christian König wrote:
> Check if we can do peer2peer on the PCIe bus.
> 
> Signed-off-by: Christian König <christian.koenig@amd.com>
>  drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> index aef12ee2f1e3..bbf67800c8a6 100644
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> @@ -38,6 +38,7 @@
>  #include <drm/amdgpu_drm.h>
>  #include <linux/dma-buf.h>
>  #include <linux/dma-fence-array.h>
> +#include <linux/pci-p2pdma.h>
>  
>  /**
>   * amdgpu_gem_prime_vmap - &dma_buf_ops.vmap implementation
> @@ -179,6 +180,9 @@ static int amdgpu_dma_buf_attach(struct dma_buf *dmabuf,
>  	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
>  	int r;
>  
> +	if (pci_p2pdma_distance_many(adev->pdev, &attach->dev, 1, true) < 0)
> +		attach->peer2peer = false;
> +

Are there other related patches than this series?

p2p dma mapping needs to be done in common code, in p2pdma.c - ie this
open coding is missing the bus_offset stuff, at least. 

I really do not want to see drivers open code this stuff.

We already have a p2pdma API for handling the struct page case, so I
suggest adding some new p2pdma API to handle this for non-struct page
cases.

ie some thing like:

int 'p2pdma map bar'(
   struct pci_device *source,
   unsigned int source_bar_number, 
   struct pci_device *dest, 
   physaddr&len *array_of_offsets & length pairs into source bar,
   struct scatterlist *output_sgl)

Jason