Re: [PATCH kernel 2/3] vfio_pci: Allow regions to add own capabilities

From: David Gibson <david@gibson.dropbear.id.au>
To: Alexey Kardashevskiy <aik@ozlabs.ru>
Cc: kvm@vger.kernel.org, Alistair Popple <alistair@popple.id.au>,
	linuxppc-dev@lists.ozlabs.org, kvm-ppc@vger.kernel.org,
	Alex Williamson <alex.williamson@redhat.com>,
	Reza Arbab <arbab@linux.ibm.com>
Subject: Re: [PATCH kernel 2/3] vfio_pci: Allow regions to add own capabilities
Date: Thu, 8 Nov 2018 17:21:39 +1100	[thread overview]
Message-ID: <20181108062139.GQ5575@umbus.fritz.box> (raw)
In-Reply-To: <20181015094233.1324-3-aik@ozlabs.ru>

[-- Attachment #1: Type: text/plain, Size: 2301 bytes --]

On Mon, Oct 15, 2018 at 08:42:32PM +1100, Alexey Kardashevskiy wrote:
> VFIO regions already support region capabilities with a limited set of
> fields. However the subdriver might have to report to the userspace
> additional bits.
> 
> This adds an add_capability() hook to vfio_pci_regops.
> 
> This is aiming Witherspoon POWER9 machines which have multiple
> interconnected NVIDIA V100 GPUs with coherent RAM; each GPU's RAM
> is mapped to a system bus and to each of GPU internal system bus and
> the GPUs use this for DMA routing as DMA trafic can go via any
> of many NVLink2 (GPU-GPU or GPU-CPU) or even stay local within a
> GPU.

This description doesn't really make clear how per-region capabilities
are relevant to these devices.

> 
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
> 
> This is based on top of "vfio_pci: Allow mapping extra regions"
> ---
>  drivers/vfio/pci/vfio_pci_private.h | 3 +++
>  drivers/vfio/pci/vfio_pci.c         | 6 ++++++
>  2 files changed, 9 insertions(+)
> 
> diff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pci_private.h
> index 86aab05..93c1738 100644
> --- a/drivers/vfio/pci/vfio_pci_private.h
> +++ b/drivers/vfio/pci/vfio_pci_private.h
> @@ -62,6 +62,9 @@ struct vfio_pci_regops {
>  	int	(*mmap)(struct vfio_pci_device *vdev,
>  			struct vfio_pci_region *region,
>  			struct vm_area_struct *vma);
> +	int	(*add_capability)(struct vfio_pci_device *vdev,
> +				  struct vfio_pci_region *region,
> +				  struct vfio_info_cap *caps);
>  };
>  
>  struct vfio_pci_region {
> diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
> index 7923314..4a3b93e 100644
> --- a/drivers/vfio/pci/vfio_pci.c
> +++ b/drivers/vfio/pci/vfio_pci.c
> @@ -759,6 +759,12 @@ static long vfio_pci_ioctl(void *device_data,
>  			if (ret)
>  				return ret;
>  
> +			if (vdev->region[i].ops->add_capability) {
> +				ret = vdev->region[i].ops->add_capability(vdev,
> +						&vdev->region[i], &caps);
> +				if (ret)
> +					return ret;
> +			}
>  		}
>  		}
>  

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]