From: Xin Zeng <xin.zeng@intel.com>
To: herbert@gondor.apana.org.au, alex.williamson@redhat.com,
jgg@nvidia.com, yishaih@nvidia.com,
shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com
Cc: linux-crypto@vger.kernel.org, kvm@vger.kernel.org,
qat-linux@intel.com, Xin Zeng <xin.zeng@intel.com>
Subject: [PATCH v5 00/10] crypto: qat - enable QAT GEN4 SRIOV VF live migration for QAT GEN4
Date: Wed, 6 Mar 2024 21:58:45 +0800 [thread overview]
Message-ID: <20240306135855.4123535-1-xin.zeng@intel.com> (raw)
This set enables live migration for Intel QAT GEN4 SRIOV Virtual
Functions (VFs).
It is composed of 10 patches. Patch 1~6 refactor the original QAT PF
driver implementation which will be reused by the following patches.
Patch 7 introduces the logic to the QAT PF driver that allows to save
and restore the state of a bank (a QAT VF is a wrapper around banks) and
drain a ring pair. Patch 8 adds the QAT PF driver a set of interfaces to
allow to save and restore the state of a VF that will be called by the
module qat_vfio_pci which will be introduced in the last patch. Patch 9
implements the defined device interfaces. The last one adds a vfio pci
extension specific for QAT which intercepts the vfio device operations
for a QAT VF to allow live migration.
Here are the steps required to test the live migration of a QAT GEN4 VF:
1. Bind one or more QAT GEN4 VF devices to the module qat_vfio_pci.ko
2. Assign the VFs to the virtual machine and enable device live
migration
3. Run a workload using a QAT VF inside the VM, for example using qatlib
(https://github.com/intel/qatlib)
4. Migrate the VM from the source node to a destination node
Changes in v5 since v4: https://lore.kernel.org/kvm/20240228143402.89219-9-xin.zeng@intel.com
- Remove device ID recheck as no consensus has been reached yet (Kevin)
- Add missing state PRE_COPY_P2P in precopy_iotcl (Kevin)
- Rearrange the state transition flow for better readability (Kevin)
- Remove unnecessary Reviewed-by in commit message (Kevin)
Changes in v4 since v3: https://lore.kernel.org/kvm/20240221155008.960369-11-xin.zeng@intel.com
- Change the order of maintainer entry for QAT vfio pci driver in
MAINTAINERS to make it alphabetical (Alex)
- Put QAT VFIO PCI driver under vfio/pci directly instead of
vfio/pci/intel (Alex)
- Add id_table recheck during device probe (Alex)
Changes in v3 since v2: https://lore.kernel.org/kvm/20240220032052.66834-1-xin.zeng@intel.com
- Use state_mutex directly instead of unnecessary deferred_reset mode
(Jason)
Changes in v2 since v1: https://lore.kernel.org/all/20240201153337.4033490-1-xin.zeng@intel.com
- Add VFIO_MIGRATION_PRE_COPY support (Alex)
- Remove unnecessary module dependancy in Kconfig (Alex)
- Use direct function calls instead of function pointers in qat vfio
variant driver (Jason)
- Address the comments including uncessary pointer check and kfree,
missing lock and direct use of pci_iov_vf_id (Shameer)
- Change CHECK_STAT macro to avoid repeat comparison (Kamlesh)
Changes in v1 since RFC: https://lore.kernel.org/all/20230630131304.64243-1-xin.zeng@intel.com
- Address comments including the right module dependancy in Kconfig,
source file name and module description (Alex)
- Added PCI error handler and P2P state handler (Suggested by Kevin)
- Refactor the state check duing loading ring state (Kevin)
- Fix missed call to vfio_put_device in the error case (Breet)
- Migrate the shadow states in PF driver
- Rebase on top of 6.8-rc1
Giovanni Cabiddu (2):
crypto: qat - adf_get_etr_base() helper
crypto: qat - relocate CSR access code
Siming Wan (3):
crypto: qat - rename get_sla_arr_of_type()
crypto: qat - expand CSR operations for QAT GEN4 devices
crypto: qat - add bank save and restore flows
Xin Zeng (5):
crypto: qat - relocate and rename 4xxx PF2VM definitions
crypto: qat - move PFVF compat checker to a function
crypto: qat - add interface for live migration
crypto: qat - implement interface for live migration
vfio/qat: Add vfio_pci driver for Intel QAT VF devices
MAINTAINERS | 8 +
.../intel/qat/qat_420xx/adf_420xx_hw_data.c | 3 +
.../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 5 +
.../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c | 1 +
.../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c | 1 +
.../intel/qat/qat_c62x/adf_c62x_hw_data.c | 1 +
.../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c | 1 +
drivers/crypto/intel/qat/qat_common/Makefile | 6 +-
.../intel/qat/qat_common/adf_accel_devices.h | 88 ++
.../intel/qat/qat_common/adf_common_drv.h | 10 +
.../qat/qat_common/adf_gen2_hw_csr_data.c | 101 ++
.../qat/qat_common/adf_gen2_hw_csr_data.h | 86 ++
.../intel/qat/qat_common/adf_gen2_hw_data.c | 97 --
.../intel/qat/qat_common/adf_gen2_hw_data.h | 76 --
.../qat/qat_common/adf_gen4_hw_csr_data.c | 231 ++++
.../qat/qat_common/adf_gen4_hw_csr_data.h | 188 +++
.../intel/qat/qat_common/adf_gen4_hw_data.c | 380 +++++--
.../intel/qat/qat_common/adf_gen4_hw_data.h | 127 +--
.../intel/qat/qat_common/adf_gen4_pfvf.c | 8 +-
.../intel/qat/qat_common/adf_gen4_vf_mig.c | 1010 +++++++++++++++++
.../intel/qat/qat_common/adf_gen4_vf_mig.h | 10 +
.../intel/qat/qat_common/adf_mstate_mgr.c | 318 ++++++
.../intel/qat/qat_common/adf_mstate_mgr.h | 89 ++
.../intel/qat/qat_common/adf_pfvf_pf_proto.c | 8 +-
.../intel/qat/qat_common/adf_pfvf_utils.h | 11 +
drivers/crypto/intel/qat/qat_common/adf_rl.c | 10 +-
drivers/crypto/intel/qat/qat_common/adf_rl.h | 2 +
.../crypto/intel/qat/qat_common/adf_sriov.c | 7 +-
.../intel/qat/qat_common/adf_transport.c | 4 +-
.../crypto/intel/qat/qat_common/qat_mig_dev.c | 130 +++
.../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c | 1 +
.../qat_dh895xccvf/adf_dh895xccvf_hw_data.c | 1 +
drivers/vfio/pci/Kconfig | 2 +
drivers/vfio/pci/Makefile | 2 +
drivers/vfio/pci/qat/Kconfig | 12 +
drivers/vfio/pci/qat/Makefile | 3 +
drivers/vfio/pci/qat/main.c | 662 +++++++++++
include/linux/qat/qat_mig_dev.h | 31 +
38 files changed, 3344 insertions(+), 387 deletions(-)
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.c
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.h
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.c
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.h
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.h
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h
create mode 100644 drivers/crypto/intel/qat/qat_common/qat_mig_dev.c
create mode 100644 drivers/vfio/pci/qat/Kconfig
create mode 100644 drivers/vfio/pci/qat/Makefile
create mode 100644 drivers/vfio/pci/qat/main.c
create mode 100644 include/linux/qat/qat_mig_dev.h
base-commit: 318407ed77e4140d02e43a001b1f4753e3ce6b5f
--
2.18.2
next reply other threads:[~2024-03-06 14:07 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-06 13:58 Xin Zeng [this message]
2024-03-06 13:58 ` [PATCH v5 01/10] crypto: qat - adf_get_etr_base() helper Xin Zeng
2024-03-06 13:58 ` [PATCH v5 02/10] crypto: qat - relocate and rename 4xxx PF2VM definitions Xin Zeng
2024-03-06 13:58 ` [PATCH v5 03/10] crypto: qat - move PFVF compat checker to a function Xin Zeng
2024-03-06 13:58 ` [PATCH v5 04/10] crypto: qat - relocate CSR access code Xin Zeng
2024-03-06 13:58 ` [PATCH v5 05/10] crypto: qat - rename get_sla_arr_of_type() Xin Zeng
2024-03-06 13:58 ` [PATCH v5 06/10] crypto: qat - expand CSR operations for QAT GEN4 devices Xin Zeng
2024-03-06 13:58 ` [PATCH v5 07/10] crypto: qat - add bank save and restore flows Xin Zeng
2024-03-06 13:58 ` [PATCH v5 08/10] crypto: qat - add interface for live migration Xin Zeng
2024-03-08 16:52 ` Jason Gunthorpe
2024-03-11 17:57 ` Cabiddu, Giovanni
2024-03-06 13:58 ` [PATCH v5 09/10] crypto: qat - implement " Xin Zeng
2024-04-30 3:10 ` liulongfang
2024-04-30 9:49 ` Zeng, Xin
2024-03-06 13:58 ` [PATCH v5 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices Xin Zeng
2024-03-08 7:56 ` Tian, Kevin
2024-03-28 10:51 ` [PATCH v5 00/10] crypto: qat - enable QAT GEN4 SRIOV VF live migration for QAT GEN4 Herbert Xu
2024-03-28 15:03 ` Alex Williamson
2024-04-02 2:52 ` Herbert Xu
2024-04-12 14:19 ` Cabiddu, Giovanni
2024-04-12 22:59 ` Alex Williamson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240306135855.4123535-1-xin.zeng@intel.com \
--to=xin.zeng@intel.com \
--cc=alex.williamson@redhat.com \
--cc=herbert@gondor.apana.org.au \
--cc=jgg@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-crypto@vger.kernel.org \
--cc=qat-linux@intel.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.