Re: [PATCH 0/2] vfio pci: Add support for OpenCAPI devices

[Alexey Kardashevskiy <aik@ozlabs.ru>]

On 25/10/2019 00:28, christophe lombard wrote:
> This series adds support for the OpenCAPI devices for vfio pci.


You can pass any PCI device via vfio already, what is missing today to
make it fully working?

For example, for nvlink gpus it was coherent memory and ATSD which we
needed to expose to the userspace, with opencapi I thought it is going
to be some sort of SRIOV-alike or a mediated device but all you do here
is to pass through the entire device with no new resources shared by the
vfio-pci driver.


> 
> It builds on top of the existing ocxl driver +
> http://patchwork.ozlabs.org/patch/1177999/


Next time please provide a link to the series such as:

http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=136572


> 
> VFIO is a Linux kernel driver framework used by QEMU to make devices
> directly assignable to virtual machines.
> 
> All OpenCAPI devices on the same PCI slot will all be grouped and
> assigned to the same guest.
> 
> - Assume these are the devices you want to assign
>  0007:00:00.0 Processing accelerators: IBM Device 062b
>  0007:00:00.1 Processing accelerators: IBM Device 062b
> 
> - Two Devices in the group
> $ ls /sys/bus/pci/devices/0007\:00\:00.0/iommu_group/devices/
>  0007:00:00.0  0007:00:00.1
> 
> - Find vendor & device ID
> $ lspci -n -s 0007:00:00
>  0007:00:00.0 1200: 1014:062b
>  0007:00:00.1 1200: 1014:062b
> 
> - Unbind from the current ocxl device driver if already loaded
> $ rmmod ocxl
> 
> - Load vfio-pci if it's not already done.
> $ modprobe vfio-pci
> 
> - Bind to vfio-pci
> $ echo 1014 062b > /sys/bus/pci/drivers/vfio-pci/new_id
> 
>   This will result in a new device node "/dev/vfio/7", which will be
>   use by QEMU to setup the devices for passthrough.
> 
> - Pass to qemu using -device vfio-pci
>   -device vfio-pci,multifunction=on,host=0007:00:00.0,addr=2.0 -device
>   vfio-pci,multifunction=on,host=0007:00:00.1,addr=2.1
> 
> It has been tested in a bare-metal and QEMU environment using the memcpy
> and the AFP AFUs.


Is there the corresponding qemu tree somewhere?


> 
> christophe lombard (2):
>   powerpc/powernv: Register IOMMU group for OpenCAPI devices
>   vfio/pci: Introduce OpenCAPI devices support.
> 
>  arch/powerpc/platforms/powernv/ocxl.c     | 164 ++++++++++---
>  arch/powerpc/platforms/powernv/pci-ioda.c |  19 +-
>  arch/powerpc/platforms/powernv/pci.h      |  13 +
>  drivers/vfio/pci/Kconfig                  |   7 +
>  drivers/vfio/pci/Makefile                 |   1 +
>  drivers/vfio/pci/vfio_pci.c               |  19 ++
>  drivers/vfio/pci/vfio_pci_ocxl.c          | 287 ++++++++++++++++++++++
>  drivers/vfio/vfio.c                       |  25 ++
>  include/linux/vfio.h                      |  13 +
>  include/uapi/linux/vfio.h                 |  22 ++
>  10 files changed, 530 insertions(+), 40 deletions(-)
>  create mode 100644 drivers/vfio/pci/vfio_pci_ocxl.c
> 

-- 
Alexey