LinuxPPC-Dev Archive on lore.kernel.org
 help / color / Atom feed
From: Alexey Kardashevskiy <aik@ozlabs.ru>
To: christophe lombard <clombard@linux.vnet.ibm.com>,
	linuxppc-dev@lists.ozlabs.org, fbarrat@linux.vnet.ibm.com,
	ajd@linux.ibm.com, groug@kaod.org, clg@kaod.org
Subject: Re: [PATCH 0/2] vfio pci: Add support for OpenCAPI devices
Date: Mon, 11 Nov 2019 14:55:47 +1100
Message-ID: <eb9306e7-9b57-5e1e-e05f-f9ce505e21b0@ozlabs.ru> (raw)
In-Reply-To: <20191024132805.30701-1-clombard@linux.vnet.ibm.com>



On 25/10/2019 00:28, christophe lombard wrote:
> This series adds support for the OpenCAPI devices for vfio pci.


You can pass any PCI device via vfio already, what is missing today to
make it fully working?

For example, for nvlink gpus it was coherent memory and ATSD which we
needed to expose to the userspace, with opencapi I thought it is going
to be some sort of SRIOV-alike or a mediated device but all you do here
is to pass through the entire device with no new resources shared by the
vfio-pci driver.


> 
> It builds on top of the existing ocxl driver +
> http://patchwork.ozlabs.org/patch/1177999/


Next time please provide a link to the series such as:

http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=136572


> 
> VFIO is a Linux kernel driver framework used by QEMU to make devices
> directly assignable to virtual machines.
> 
> All OpenCAPI devices on the same PCI slot will all be grouped and
> assigned to the same guest.
> 
> - Assume these are the devices you want to assign
>  0007:00:00.0 Processing accelerators: IBM Device 062b
>  0007:00:00.1 Processing accelerators: IBM Device 062b
> 
> - Two Devices in the group
> $ ls /sys/bus/pci/devices/0007\:00\:00.0/iommu_group/devices/
>  0007:00:00.0  0007:00:00.1
> 
> - Find vendor & device ID
> $ lspci -n -s 0007:00:00
>  0007:00:00.0 1200: 1014:062b
>  0007:00:00.1 1200: 1014:062b
> 
> - Unbind from the current ocxl device driver if already loaded
> $ rmmod ocxl
> 
> - Load vfio-pci if it's not already done.
> $ modprobe vfio-pci
> 
> - Bind to vfio-pci
> $ echo 1014 062b > /sys/bus/pci/drivers/vfio-pci/new_id
> 
>   This will result in a new device node "/dev/vfio/7", which will be
>   use by QEMU to setup the devices for passthrough.
> 
> - Pass to qemu using -device vfio-pci
>   -device vfio-pci,multifunction=on,host=0007:00:00.0,addr=2.0 -device
>   vfio-pci,multifunction=on,host=0007:00:00.1,addr=2.1
> 
> It has been tested in a bare-metal and QEMU environment using the memcpy
> and the AFP AFUs.


Is there the corresponding qemu tree somewhere?


> 
> christophe lombard (2):
>   powerpc/powernv: Register IOMMU group for OpenCAPI devices
>   vfio/pci: Introduce OpenCAPI devices support.
> 
>  arch/powerpc/platforms/powernv/ocxl.c     | 164 ++++++++++---
>  arch/powerpc/platforms/powernv/pci-ioda.c |  19 +-
>  arch/powerpc/platforms/powernv/pci.h      |  13 +
>  drivers/vfio/pci/Kconfig                  |   7 +
>  drivers/vfio/pci/Makefile                 |   1 +
>  drivers/vfio/pci/vfio_pci.c               |  19 ++
>  drivers/vfio/pci/vfio_pci_ocxl.c          | 287 ++++++++++++++++++++++
>  drivers/vfio/vfio.c                       |  25 ++
>  include/linux/vfio.h                      |  13 +
>  include/uapi/linux/vfio.h                 |  22 ++
>  10 files changed, 530 insertions(+), 40 deletions(-)
>  create mode 100644 drivers/vfio/pci/vfio_pci_ocxl.c
> 

-- 
Alexey

      parent reply index

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-24 13:28 christophe lombard
2019-10-24 13:28 ` [PATCH 1/2] powerpc/powernv: Register IOMMU group " christophe lombard
2019-10-24 13:28 ` [PATCH 2/2] vfio/pci: Introduce OpenCAPI devices support christophe lombard
2019-11-11  4:17   ` Alexey Kardashevskiy
2019-11-11  4:28     ` Andrew Donnellan
2019-11-12  2:15       ` Alexey Kardashevskiy
2019-10-24 14:32 ` [PATCH 0/2] vfio pci: Add support for OpenCAPI devices Greg Kurz
2019-11-11  3:55 ` Alexey Kardashevskiy [this message]

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=eb9306e7-9b57-5e1e-e05f-f9ce505e21b0@ozlabs.ru \
    --to=aik@ozlabs.ru \
    --cc=ajd@linux.ibm.com \
    --cc=clg@kaod.org \
    --cc=clombard@linux.vnet.ibm.com \
    --cc=fbarrat@linux.vnet.ibm.com \
    --cc=groug@kaod.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LinuxPPC-Dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linuxppc-dev/0 linuxppc-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linuxppc-dev linuxppc-dev/ https://lore.kernel.org/linuxppc-dev \
		linuxppc-dev@lists.ozlabs.org linuxppc-dev@ozlabs.org
	public-inbox-index linuxppc-dev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.ozlabs.lists.linuxppc-dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git