From: Max Gurtovoy <mgurtovoy@nvidia.com>
To: <alex.williamson@redhat.com>, <cohuck@redhat.com>,
<kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<jgg@nvidia.com>
Cc: <aviadye@nvidia.com>, <oren@nvidia.com>, <shahafs@nvidia.com>,
<parav@nvidia.com>, <artemp@nvidia.com>, <kwankhede@nvidia.com>,
<ACurrid@nvidia.com>, <cjia@nvidia.com>, <yishaih@nvidia.com>,
<kevin.tian@intel.com>, <hch@infradead.org>,
<targupta@nvidia.com>, <shameerali.kolothum.thodi@huawei.com>,
<liulongfang@huawei.com>, <yan.y.zhao@intel.com>,
Max Gurtovoy <mgurtovoy@nvidia.com>
Subject: [RFC PATCH v4 00/11] Introduce vfio-pci-core subsystem
Date: Thu, 3 Jun 2021 19:07:58 +0300 [thread overview]
Message-ID: <20210603160809.15845-1-mgurtovoy@nvidia.com> (raw)
Hi Alex, Cornelia, Jason and Co,
This series split the vfio_pci driver into 2 parts: pci drivers and a
subsystem driver that will also be library of code. The main pci driver,
vfio_pci.ko will be used as before and it will bind to the subsystem
driver vfio_pci_core.ko to register to the VFIO subsystem.
This series is coming to solve some of the issues that were raised in
the previous attempts for extending vfio-pci for vendor specific
functionality:
1. https://lkml.org/lkml/2020/5/17/376 by Yan Zhao.
2. https://www.spinics.net/lists/kernel/msg3903996.html by Longfang Liu
This subsystem framework will also ease on adding new vendor specific
functionality to VFIO devices in the future by allowing another module
to provide the pci_driver that can setup number of details before
registering to VFIO subsystem (such as inject its own operations).
This series also extends the "driver_override" mechanism. We added a flag
for pci drivers that will declare themselves as "driver_override" capable
and only declared drivers can use this mechanism in the PCI subsystem.
Other drivers will not be able to bind to devices that use "driver_override".
Also, the PCI driver matching will always look for ID table and will never
generate dummy "match_all" ID table in the PCI subsystem layer. In this
way, we ensure deterministic behaviour with no races with the original
pci drivers. In order to get the best match for "driver_override" drivers,
one can create a userspace program (example can be found at
https://github.com/maxgurtovoy/linux_tools/blob/main/vfio/bind_vfio_pci_driver.py)
that find the 'best match' according to simple algorithm: "the driver
with the fewest '*' matches wins."
For example, the vfio-pci driver will match to any pci device. So it
will have the maximal '*' matches (for all matching IDs: vendor, device,
subvendor, ...).
In case we are looking for a match to mlx5 based device, we'll have a
match to vfio-pci.ko and mlx5-vfio-pci.ko. We'll prefer mlx5-vfio-pci.ko
since it will have less '*' matches (probably vendor and device IDs will
match). This will work in the future for NVMe/Virtio devices that can
match according to a class code or other criteria.
The main goal of this series is to agree on the vfio_pci module split and the
"driver_override" extensions. The follow-up version will include an extended
mlx5_vfio_pci driver that will support VF suspend/resume as well.
This series applied cleanly on top of vfio reflck re-design (still haven't sent
for review) and can be found at:
https://github.com/Mellanox/NVMEoF-P2P/tree/vfio-v4-external.
Max Gurtovoy (11):
vfio-pci: rename vfio_pci.c to vfio_pci_core.c
vfio-pci: rename vfio_pci_private.h to vfio_pci_core.h
vfio-pci: rename vfio_pci_device to vfio_pci_core_device
vfio-pci: rename ops functions to fit core namings
vfio-pci: include vfio header in vfio_pci_core.h
vfio-pci: introduce vfio_pci.c
vfio-pci: move igd initialization to vfio_pci.c
PCI: add flags field to pci_device_id structure
PCI: add matching checks for driver_override binding
vfio-pci: introduce vfio_pci_core subsystem driver
mlx5-vfio-pci: add new vfio_pci driver for mlx5 devices
Documentation/ABI/testing/sysfs-bus-pci | 6 +-
Documentation/PCI/pci.rst | 1 +
drivers/pci/pci-driver.c | 22 +-
drivers/vfio/pci/Kconfig | 27 +-
drivers/vfio/pci/Makefile | 12 +-
drivers/vfio/pci/mlx5_vfio_pci.c | 130 +
drivers/vfio/pci/vfio_pci.c | 2329 +----------------
drivers/vfio/pci/vfio_pci_config.c | 70 +-
drivers/vfio/pci/vfio_pci_core.c | 2239 ++++++++++++++++
drivers/vfio/pci/vfio_pci_igd.c | 16 +-
drivers/vfio/pci/vfio_pci_intrs.c | 42 +-
drivers/vfio/pci/vfio_pci_rdwr.c | 18 +-
drivers/vfio/pci/vfio_pci_zdev.c | 4 +-
include/linux/mod_devicetable.h | 9 +
include/linux/pci.h | 27 +
.../linux/vfio_pci_core.h | 93 +-
scripts/mod/devicetable-offsets.c | 1 +
scripts/mod/file2alias.c | 8 +-
18 files changed, 2695 insertions(+), 2359 deletions(-)
create mode 100644 drivers/vfio/pci/mlx5_vfio_pci.c
create mode 100644 drivers/vfio/pci/vfio_pci_core.c
rename drivers/vfio/pci/vfio_pci_private.h => include/linux/vfio_pci_core.h (56%)
--
2.21.0
next reply other threads:[~2021-06-03 16:08 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-03 16:07 Max Gurtovoy [this message]
2021-06-03 16:07 ` [PATCH 01/11] vfio-pci: rename vfio_pci.c to vfio_pci_core.c Max Gurtovoy
2021-06-03 16:08 ` [PATCH 02/11] vfio-pci: rename vfio_pci_private.h to vfio_pci_core.h Max Gurtovoy
2021-06-03 16:08 ` [PATCH 03/11] vfio-pci: rename vfio_pci_device to vfio_pci_core_device Max Gurtovoy
2021-06-03 16:08 ` [PATCH 04/11] vfio-pci: rename ops functions to fit core namings Max Gurtovoy
2021-06-03 16:08 ` [PATCH 05/11] vfio-pci: include vfio header in vfio_pci_core.h Max Gurtovoy
2021-06-03 16:08 ` [PATCH 06/11] vfio-pci: introduce vfio_pci.c Max Gurtovoy
2021-06-03 16:08 ` [PATCH 07/11] vfio-pci: move igd initialization to vfio_pci.c Max Gurtovoy
2021-06-03 16:08 ` [PATCH 08/11] PCI: add flags field to pci_device_id structure Max Gurtovoy
2021-06-03 16:08 ` [PATCH 09/11] PCI: add matching checks for driver_override binding Max Gurtovoy
2021-06-08 21:26 ` Alex Williamson
2021-06-08 22:45 ` Jason Gunthorpe
2021-06-09 1:27 ` Alex Williamson
2021-06-09 9:26 ` Max Gurtovoy
2021-06-13 8:19 ` Max Gurtovoy
2021-06-14 5:40 ` Christoph Hellwig
2021-06-14 8:18 ` Max Gurtovoy
2021-06-14 15:27 ` Christoph Hellwig
2021-06-14 16:01 ` Jason Gunthorpe
2021-06-14 16:15 ` Christoph Hellwig
2021-06-14 16:33 ` Jason Gunthorpe
2021-06-14 18:42 ` Alex Williamson
2021-06-14 23:12 ` Max Gurtovoy
2021-06-15 15:00 ` Alex Williamson
2021-06-15 15:04 ` Jason Gunthorpe
2021-06-15 16:20 ` Alex Williamson
2021-06-15 20:42 ` Jason Gunthorpe
2021-06-15 21:59 ` Alex Williamson
2021-06-15 23:00 ` Jason Gunthorpe
2021-06-15 23:22 ` Alex Williamson
2021-06-15 23:32 ` Jason Gunthorpe
2021-06-16 0:22 ` Alex Williamson
2021-06-16 0:34 ` Jason Gunthorpe
2021-06-16 23:28 ` Max Gurtovoy
2021-06-16 23:33 ` Jason Gunthorpe
2021-06-16 23:42 ` Max Gurtovoy
2021-06-16 23:44 ` Jason Gunthorpe
2021-06-16 23:51 ` Max Gurtovoy
2021-06-16 23:56 ` Jason Gunthorpe
2021-06-20 14:46 ` Max Gurtovoy
2021-06-03 16:08 ` [PATCH 10/11] vfio-pci: introduce vfio_pci_core subsystem driver Max Gurtovoy
2021-06-08 21:26 ` Alex Williamson
2021-06-09 9:29 ` Max Gurtovoy
2021-06-03 16:08 ` [PATCH 11/11] mlx5-vfio-pci: add new vfio_pci driver for mlx5 devices Max Gurtovoy
2021-07-30 7:53 ` [RFC PATCH v4 00/11] Introduce vfio-pci-core subsystem Shameerali Kolothum Thodi
2021-07-30 11:55 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210603160809.15845-1-mgurtovoy@nvidia.com \
--to=mgurtovoy@nvidia.com \
--cc=ACurrid@nvidia.com \
--cc=alex.williamson@redhat.com \
--cc=artemp@nvidia.com \
--cc=aviadye@nvidia.com \
--cc=cjia@nvidia.com \
--cc=cohuck@redhat.com \
--cc=hch@infradead.org \
--cc=jgg@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=kwankhede@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=liulongfang@huawei.com \
--cc=oren@nvidia.com \
--cc=parav@nvidia.com \
--cc=shahafs@nvidia.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=targupta@nvidia.com \
--cc=yan.y.zhao@intel.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).