From: Xin Zeng <xin.zeng@intel.com>
To: herbert@gondor.apana.org.au, alex.williamson@redhat.com,
jgg@nvidia.com, yishaih@nvidia.com,
shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com
Cc: linux-crypto@vger.kernel.org, kvm@vger.kernel.org,
qat-linux@intel.com, Xin Zeng <xin.zeng@intel.com>
Subject: [PATCH v2 00/10] crypto: qat - enable QAT GEN4 SRIOV VF live migration for QAT GEN4
Date: Tue, 20 Feb 2024 11:20:42 +0800 [thread overview]
Message-ID: <20240220032052.66834-1-xin.zeng@intel.com> (raw)
This set enables live migration for Intel QAT GEN4 SRIOV Virtual
Functions (VFs).
It is composed of 10 patches. Patch 1~6 refactor the original QAT PF
driver implementation which will be reused by the following patches.
Patch 7 introduces the logic to the QAT PF driver that allows to save
and restore the state of a bank (a QAT VF is a wrapper around banks) and
drain a ring pair. Patch 8 adds the QAT PF driver a set of interfaces to
allow to save and restore the state of a VF that will be called by the
module qat_vfio_pci which will be introduced in the last patch. Patch 9
implements the defined device interfaces. The last one adds a vfio pci
extension specific for QAT which intercepts the vfio device operations
for a QAT VF to allow live migration.
Here are the steps required to test the live migration of a QAT GEN4 VF:
1. Bind one or more QAT GEN4 VF devices to the module qat_vfio_pci.ko
2. Assign the VFs to the virtual machine and enable device live
migration
3. Run a workload using a QAT VF inside the VM, for example using qatlib
(https://github.com/intel/qatlib)
4. Migrate the VM from the source node to a destination node
Changes in v2 from v1:
https://lore.kernel.org/all/20240201153337.4033490-1-xin.zeng@intel.com
- Add VFIO_MIGRATION_PRE_COPY support (Alex)
- Remove unnecessary module dependancy in Kconfig (Alex)
- Use direct function calls instead of function pointers in qat vfio
variant driver (Jason)
- Address the comments including uncessary pointer check and kfree,
missing lock and direct use of pci_iov_vf_id (Shameer)
- Change CHECK_STAT macro to avoid repeat comparison (Kamlesh)
Changes in v1 from RFC:
https://lore.kernel.org/all/20230630131304.64243-1-xin.zeng@intel.com
- Address comments including the right module dependancy in Kconfig,
source file name and module description (Alex)
- Added PCI error handler and P2P state handler (Suggested by Kevin)
- Refactor the state check duing loading ring state (Kevin)
- Fix missed call to vfio_put_device in the error case (Breet)
- Migrate the shadow states in PF driver
- Rebase on top of 6.8-rc1
Giovanni Cabiddu (2):
crypto: qat - adf_get_etr_base() helper
crypto: qat - relocate CSR access code
Siming Wan (3):
crypto: qat - rename get_sla_arr_of_type()
crypto: qat - expand CSR operations for QAT GEN4 devices
crypto: qat - add bank save and restore flows
Xin Zeng (5):
crypto: qat - relocate and rename 4xxx PF2VM definitions
crypto: qat - move PFVF compat checker to a function
crypto: qat - add interface for live migration
crypto: qat - implement interface for live migration
vfio/qat: Add vfio_pci driver for Intel QAT VF devices
MAINTAINERS | 8 +
.../intel/qat/qat_420xx/adf_420xx_hw_data.c | 3 +
.../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 5 +
.../intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c | 1 +
.../qat/qat_c3xxxvf/adf_c3xxxvf_hw_data.c | 1 +
.../intel/qat/qat_c62x/adf_c62x_hw_data.c | 1 +
.../intel/qat/qat_c62xvf/adf_c62xvf_hw_data.c | 1 +
drivers/crypto/intel/qat/qat_common/Makefile | 6 +-
.../intel/qat/qat_common/adf_accel_devices.h | 88 ++
.../intel/qat/qat_common/adf_common_drv.h | 10 +
.../qat/qat_common/adf_gen2_hw_csr_data.c | 101 ++
.../qat/qat_common/adf_gen2_hw_csr_data.h | 86 ++
.../intel/qat/qat_common/adf_gen2_hw_data.c | 97 --
.../intel/qat/qat_common/adf_gen2_hw_data.h | 76 --
.../qat/qat_common/adf_gen4_hw_csr_data.c | 231 ++++
.../qat/qat_common/adf_gen4_hw_csr_data.h | 188 +++
.../intel/qat/qat_common/adf_gen4_hw_data.c | 380 +++++--
.../intel/qat/qat_common/adf_gen4_hw_data.h | 127 +--
.../intel/qat/qat_common/adf_gen4_pfvf.c | 8 +-
.../intel/qat/qat_common/adf_gen4_vf_mig.c | 1003 +++++++++++++++++
.../intel/qat/qat_common/adf_gen4_vf_mig.h | 10 +
.../intel/qat/qat_common/adf_mstate_mgr.c | 318 ++++++
.../intel/qat/qat_common/adf_mstate_mgr.h | 89 ++
.../intel/qat/qat_common/adf_pfvf_pf_proto.c | 8 +-
.../intel/qat/qat_common/adf_pfvf_utils.h | 11 +
drivers/crypto/intel/qat/qat_common/adf_rl.c | 10 +-
drivers/crypto/intel/qat/qat_common/adf_rl.h | 2 +
.../crypto/intel/qat/qat_common/adf_sriov.c | 7 +-
.../intel/qat/qat_common/adf_transport.c | 4 +-
.../crypto/intel/qat/qat_common/qat_mig_dev.c | 130 +++
.../qat/qat_dh895xcc/adf_dh895xcc_hw_data.c | 1 +
.../qat_dh895xccvf/adf_dh895xccvf_hw_data.c | 1 +
drivers/vfio/pci/Kconfig | 2 +
drivers/vfio/pci/Makefile | 2 +
drivers/vfio/pci/intel/qat/Kconfig | 12 +
drivers/vfio/pci/intel/qat/Makefile | 3 +
drivers/vfio/pci/intel/qat/main.c | 688 +++++++++++
include/linux/qat/qat_mig_dev.h | 31 +
38 files changed, 3363 insertions(+), 387 deletions(-)
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.c
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen2_hw_csr_data.h
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.c
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_hw_csr_data.h
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.h
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c
create mode 100644 drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h
create mode 100644 drivers/crypto/intel/qat/qat_common/qat_mig_dev.c
create mode 100644 drivers/vfio/pci/intel/qat/Kconfig
create mode 100644 drivers/vfio/pci/intel/qat/Makefile
create mode 100644 drivers/vfio/pci/intel/qat/main.c
create mode 100644 include/linux/qat/qat_mig_dev.h
base-commit: a821f4cadc99272701016641bc821182fdfca289
--
2.18.2
next reply other threads:[~2024-02-20 3:29 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-20 3:20 Xin Zeng [this message]
2024-02-20 3:20 ` [PATCH v2 01/10] crypto: qat - adf_get_etr_base() helper Xin Zeng
2024-02-20 3:20 ` [PATCH v2 02/10] crypto: qat - relocate and rename 4xxx PF2VM definitions Xin Zeng
2024-02-20 3:20 ` [PATCH v2 03/10] crypto: qat - move PFVF compat checker to a function Xin Zeng
2024-02-20 3:20 ` [PATCH v2 04/10] crypto: qat - relocate CSR access code Xin Zeng
2024-02-20 3:20 ` [PATCH v2 05/10] crypto: qat - rename get_sla_arr_of_type() Xin Zeng
2024-02-20 3:20 ` [PATCH v2 06/10] crypto: qat - expand CSR operations for QAT GEN4 devices Xin Zeng
2024-02-20 3:20 ` [PATCH v2 07/10] crypto: qat - add bank save and restore flows Xin Zeng
2024-02-20 3:20 ` [PATCH v2 08/10] crypto: qat - add interface for live migration Xin Zeng
2024-02-20 3:20 ` [PATCH v2 09/10] crypto: qat - implement " Xin Zeng
2024-02-20 3:20 ` [PATCH v2 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices Xin Zeng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240220032052.66834-1-xin.zeng@intel.com \
--to=xin.zeng@intel.com \
--cc=alex.williamson@redhat.com \
--cc=herbert@gondor.apana.org.au \
--cc=jgg@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-crypto@vger.kernel.org \
--cc=qat-linux@intel.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).