From: Brett Creeley <brett.creeley@amd.com>
To: <kvm@vger.kernel.org>, <netdev@vger.kernel.org>,
<alex.williamson@redhat.com>, <cohuck@redhat.com>,
<jgg@nvidia.com>, <yishaih@nvidia.com>,
<shameerali.kolothum.thodi@huawei.com>, <kevin.tian@intel.com>
Cc: <shannon.nelson@amd.com>, <drivers@pensando.io>, <brett.creeley@amd.com>
Subject: [PATCH v3 vfio 0/7] pds vfio driver
Date: Sun, 19 Feb 2023 00:39:01 -0800 [thread overview]
Message-ID: <20230219083908.40013-1-brett.creeley@amd.com> (raw)
This is a draft patchset for a new vendor specific VFIO driver
(pds_vfio) for use with the AMD/Pensando Distributed Services Card
(DSC). This driver is device type agnostic and live migration is
supported as long as the underlying SR-IOV VF supports live migration
on the DSC. This driver is a client of the newly introduced pds_core
driver, which the latest version can be referenced at:
https://lore.kernel.org/netdev/20230217225558.19837-1-shannon.nelson@amd.com/
This driver will use the pds_core device and auxiliary_bus as the VFIO
control path to the DSC. The pds_core device creates auxiliary_bus
devices for each live migratable VF. The devices are named by their
feature plus the VF PCI BDF so the auxiliary_bus driver implemented by
pds_vfio can find its related VF PCI driver instance. Once this
auxiliary bus connection is configured, the pds_vfio driver can send
admin queue commands to the device and receive events from pds_core.
An ASCII diagram of a VFIO instance looks something like this and can
be used with the VFIO subsystem to provide devices VFIO and live
migration support.
.------. .--------------------------.
| QEMU |--| VM .-------------. |
'......' | | PCI driver | |
| | .-------------. |
| | | SR-IOV VF | |
| | '-------------' |
| '---------------||---------'
.--------------. ||
|/dev/<vfio_fd>| ||
'--------------' ||
Host Userspace | ||
=================================================== ||
Host Kernel | ||
| ||
pds_core.LM.2305 <--+ .--------. ||
| | |vfio-pci| ||
| | '--------' ||
| | | ||
.------------. .-------------. ||
| pds_core | | pds_vfio | ||
'------------' '-------------' ||
|| || ||
09:00.0 09:00.1 ||
== PCI ==================================================||=====
|| || ||
.----------. .----------. ||
,-----| PF |---------| VF |-------------------,
| '----------' '----------' | VF |
| DSC | data/control |
| | path |
-----------------------------------------------------------
The pds_vfio driver is targeted to reside in drivers/vfio/pci/pds.
It makes use of and introduces new files in the common include/linux/pds
include directory.
Changes:
v3:
- Update copyright year to 2023 and use "Advanced Micro Devices, Inc."
for the company name
- Clarify the fact that AMD/Pensando's VFIO solution is device type
agnostic, which aligns with other current VFIO solutions
- Add line in drivers/vfio/pci/Makefile to build pds_vfio
- Move documentation to amd sub-directory
- Remove some dead code due to the pds_core implementation of
listening to BIND/UNBIND events
- Move a dev_dbg() to a previous patch in the series
- Add implementation for vfio_migration_ops.migration_get_data_size to
return the maximum possible device state size
RFC to v2:
https://lore.kernel.org/all/20221214232136.64220-1-brett.creeley@amd.com/
- Implement state transitions for VFIO_MIGRATION_P2P flag
- Improve auxiliary driver probe by returning EPROBE_DEFER
when the PCI driver is not set up correctly
- Add pointer to docs in
Documentation/networking/device_drivers/ethernet/index.rst
RFC:
https://lore.kernel.org/all/20221207010705.35128-1-brett.creeley@amd.com/
Brett Creeley (7):
vfio/pds: Initial support for pds_vfio VFIO driver
vfio/pds: Add support to register as PDS client
vfio/pds: Add VFIO live migration support
vfio: Commonize combine_ranges for use in other VFIO drivers
vfio/pds: Add support for dirty page tracking
vfio/pds: Add support for firmware recovery
vfio/pds: Add Kconfig and documentation
.../device_drivers/ethernet/amd/pds_vfio.rst | 88 +++
.../device_drivers/ethernet/index.rst | 1 +
MAINTAINERS | 7 +
drivers/vfio/pci/Kconfig | 2 +
drivers/vfio/pci/Makefile | 2 +
drivers/vfio/pci/mlx5/cmd.c | 48 +-
drivers/vfio/pci/pds/Kconfig | 19 +
drivers/vfio/pci/pds/Makefile | 12 +
drivers/vfio/pci/pds/aux_drv.c | 210 +++++++
drivers/vfio/pci/pds/aux_drv.h | 28 +
drivers/vfio/pci/pds/cmds.c | 485 ++++++++++++++++
drivers/vfio/pci/pds/cmds.h | 44 ++
drivers/vfio/pci/pds/dirty.c | 541 ++++++++++++++++++
drivers/vfio/pci/pds/dirty.h | 48 ++
drivers/vfio/pci/pds/lm.c | 491 ++++++++++++++++
drivers/vfio/pci/pds/lm.h | 53 ++
drivers/vfio/pci/pds/pci_drv.c | 126 ++++
drivers/vfio/pci/pds/pci_drv.h | 14 +
drivers/vfio/pci/pds/vfio_dev.c | 239 ++++++++
drivers/vfio/pci/pds/vfio_dev.h | 42 ++
drivers/vfio/vfio_main.c | 48 ++
include/linux/pds/pds_lm.h | 391 +++++++++++++
include/linux/vfio.h | 3 +
23 files changed, 2895 insertions(+), 47 deletions(-)
create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst
create mode 100644 drivers/vfio/pci/pds/Kconfig
create mode 100644 drivers/vfio/pci/pds/Makefile
create mode 100644 drivers/vfio/pci/pds/aux_drv.c
create mode 100644 drivers/vfio/pci/pds/aux_drv.h
create mode 100644 drivers/vfio/pci/pds/cmds.c
create mode 100644 drivers/vfio/pci/pds/cmds.h
create mode 100644 drivers/vfio/pci/pds/dirty.c
create mode 100644 drivers/vfio/pci/pds/dirty.h
create mode 100644 drivers/vfio/pci/pds/lm.c
create mode 100644 drivers/vfio/pci/pds/lm.h
create mode 100644 drivers/vfio/pci/pds/pci_drv.c
create mode 100644 drivers/vfio/pci/pds/pci_drv.h
create mode 100644 drivers/vfio/pci/pds/vfio_dev.c
create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
create mode 100644 include/linux/pds/pds_lm.h
--
2.17.1
next reply other threads:[~2023-02-19 8:39 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-19 8:39 Brett Creeley [this message]
2023-02-19 8:39 ` [PATCH v3 1/7] vfio/pds: Initial support for pds_vfio VFIO driver Brett Creeley
2023-02-19 8:39 ` [PATCH v3 2/7] vfio/pds: Add support to register as PDS client Brett Creeley
2023-02-19 8:39 ` [PATCH v3 3/7] vfio/pds: Add VFIO live migration support Brett Creeley
2023-02-19 8:39 ` [PATCH v3 4/7] vfio: Commonize combine_ranges for use in other VFIO drivers Brett Creeley
2023-02-19 8:39 ` [PATCH v3 5/7] vfio/pds: Add support for dirty page tracking Brett Creeley
2023-02-19 8:39 ` [PATCH v3 6/7] vfio/pds: Add support for firmware recovery Brett Creeley
2023-02-19 8:39 ` [PATCH v3 7/7] vfio/pds: Add Kconfig and documentation Brett Creeley
2023-02-20 6:29 ` [PATCH v3 vfio 0/7] pds vfio driver Christoph Hellwig
2023-02-21 0:45 ` Brett Creeley
2023-02-21 1:11 ` Jason Gunthorpe
2023-02-23 7:01 ` Brett Creeley
2023-02-23 13:26 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230219083908.40013-1-brett.creeley@amd.com \
--to=brett.creeley@amd.com \
--cc=alex.williamson@redhat.com \
--cc=cohuck@redhat.com \
--cc=drivers@pensando.io \
--cc=jgg@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=shannon.nelson@amd.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).