From: Alex Williamson <alex.williamson@redhat.com>
To: Xin Zeng <xin.zeng@intel.com>
Cc: herbert@gondor.apana.org.au, jgg@nvidia.com, yishaih@nvidia.com,
shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com,
linux-crypto@vger.kernel.org, kvm@vger.kernel.org,
qat-linux@intel.com, Yahui Cao <yahui.cao@intel.com>
Subject: Re: [PATCH 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices
Date: Tue, 6 Feb 2024 14:14:46 -0700 [thread overview]
Message-ID: <20240206141446.3547e4a9.alex.williamson@redhat.com> (raw)
In-Reply-To: <20240201153337.4033490-11-xin.zeng@intel.com>
On Thu, 1 Feb 2024 23:33:37 +0800
Xin Zeng <xin.zeng@intel.com> wrote:
> Add vfio pci driver for Intel QAT VF devices.
>
> This driver uses vfio_pci_core to register to the VFIO subsystem. It
> acts as a vfio agent and interacts with the QAT PF driver to implement
> VF live migration.
>
> Co-developed-by: Yahui Cao <yahui.cao@intel.com>
> Signed-off-by: Yahui Cao <yahui.cao@intel.com>
> Signed-off-by: Xin Zeng <xin.zeng@intel.com>
> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
> ---
> MAINTAINERS | 8 +
> drivers/vfio/pci/Kconfig | 2 +
> drivers/vfio/pci/Makefile | 2 +
> drivers/vfio/pci/intel/qat/Kconfig | 13 +
> drivers/vfio/pci/intel/qat/Makefile | 4 +
> drivers/vfio/pci/intel/qat/main.c | 572 ++++++++++++++++++++++++++++
> 6 files changed, 601 insertions(+)
> create mode 100644 drivers/vfio/pci/intel/qat/Kconfig
> create mode 100644 drivers/vfio/pci/intel/qat/Makefile
> create mode 100644 drivers/vfio/pci/intel/qat/main.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 8d1052fa6a69..c1d3e4cb3892 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -23095,6 +23095,14 @@ S: Maintained
> F: Documentation/networking/device_drivers/ethernet/amd/pds_vfio_pci.rst
> F: drivers/vfio/pci/pds/
>
> +VFIO QAT PCI DRIVER
> +M: Xin Zeng <xin.zeng@intel.com>
> +M: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
> +L: kvm@vger.kernel.org
> +L: qat-linux@intel.com
> +S: Supported
> +F: drivers/vfio/pci/intel/qat/
> +
> VFIO PLATFORM DRIVER
> M: Eric Auger <eric.auger@redhat.com>
> L: kvm@vger.kernel.org
> diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig
> index 18c397df566d..329d25c53274 100644
> --- a/drivers/vfio/pci/Kconfig
> +++ b/drivers/vfio/pci/Kconfig
> @@ -67,4 +67,6 @@ source "drivers/vfio/pci/pds/Kconfig"
>
> source "drivers/vfio/pci/virtio/Kconfig"
>
> +source "drivers/vfio/pci/intel/qat/Kconfig"
> +
> endmenu
> diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile
> index 046139a4eca5..a87b6b43ce1c 100644
> --- a/drivers/vfio/pci/Makefile
> +++ b/drivers/vfio/pci/Makefile
> @@ -15,3 +15,5 @@ obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/
> obj-$(CONFIG_PDS_VFIO_PCI) += pds/
>
> obj-$(CONFIG_VIRTIO_VFIO_PCI) += virtio/
> +
> +obj-$(CONFIG_QAT_VFIO_PCI) += intel/qat/
> diff --git a/drivers/vfio/pci/intel/qat/Kconfig b/drivers/vfio/pci/intel/qat/Kconfig
> new file mode 100644
> index 000000000000..71b28ac0bf6a
> --- /dev/null
> +++ b/drivers/vfio/pci/intel/qat/Kconfig
> @@ -0,0 +1,13 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config QAT_VFIO_PCI
> + tristate "VFIO support for QAT VF PCI devices"
> + select VFIO_PCI_CORE
> + depends on CRYPTO_DEV_QAT
> + depends on CRYPTO_DEV_QAT_4XXX
CRYPTO_DEV_QAT_4XXX selects CRYPTO_DEV_QAT, is it necessary to depend
on CRYPTO_DEV_QAT here?
> + help
> + This provides migration support for Intel(R) QAT Virtual Function
> + using the VFIO framework.
> +
> + To compile this as a module, choose M here: the module
> + will be called qat_vfio_pci. If you don't know what to do here,
> + say N.
> diff --git a/drivers/vfio/pci/intel/qat/Makefile b/drivers/vfio/pci/intel/qat/Makefile
> new file mode 100644
> index 000000000000..9289ae4c51bf
> --- /dev/null
> +++ b/drivers/vfio/pci/intel/qat/Makefile
> @@ -0,0 +1,4 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +obj-$(CONFIG_QAT_VFIO_PCI) += qat_vfio_pci.o
> +qat_vfio_pci-y := main.o
> +
Blank line at end of file.
> diff --git a/drivers/vfio/pci/intel/qat/main.c b/drivers/vfio/pci/intel/qat/main.c
> new file mode 100644
> index 000000000000..85d0ed701397
> --- /dev/null
> +++ b/drivers/vfio/pci/intel/qat/main.c
> @@ -0,0 +1,572 @@
...
> +static int qat_vf_pci_init_dev(struct vfio_device *core_vdev)
> +{
> + struct qat_vf_core_device *qat_vdev = container_of(core_vdev,
> + struct qat_vf_core_device, core_device.vdev);
> + struct qat_migdev_ops *ops;
> + struct qat_mig_dev *mdev;
> + struct pci_dev *parent;
> + int ret, vf_id;
> +
> + core_vdev->migration_flags = VFIO_MIGRATION_STOP_COPY | VFIO_MIGRATION_P2P;
AFAICT it's always good practice to support VFIO_MIGRATION_PRE_COPY,
even if only to leave yourself an option to mark an incompatible ABI
change in the data stream that can be checked before the stop-copy
phase of migration. See for example the mtty sample driver. Thanks,
Alex
> + core_vdev->mig_ops = &qat_vf_pci_mig_ops;
> +
> + ret = vfio_pci_core_init_dev(core_vdev);
> + if (ret)
> + return ret;
> +
> + mutex_init(&qat_vdev->state_mutex);
> + spin_lock_init(&qat_vdev->reset_lock);
> +
> + parent = qat_vdev->core_device.pdev->physfn;
> + vf_id = pci_iov_vf_id(qat_vdev->core_device.pdev);
> + if (!parent || vf_id < 0) {
> + ret = -ENODEV;
> + goto err_rel;
> + }
> +
> + mdev = qat_vfmig_create(parent, vf_id);
> + if (IS_ERR(mdev)) {
> + ret = PTR_ERR(mdev);
> + goto err_rel;
> + }
> +
> + ops = mdev->ops;
> + if (!ops || !ops->init || !ops->cleanup ||
> + !ops->open || !ops->close ||
> + !ops->save_state || !ops->load_state ||
> + !ops->suspend || !ops->resume) {
> + ret = -EIO;
> + dev_err(&parent->dev, "Incomplete device migration ops structure!");
> + goto err_destroy;
> + }
> + ret = ops->init(mdev);
> + if (ret)
> + goto err_destroy;
> +
> + qat_vdev->mdev = mdev;
> +
> + return 0;
> +
> +err_destroy:
> + qat_vfmig_destroy(mdev);
> +err_rel:
> + vfio_pci_core_release_dev(core_vdev);
> + return ret;
> +}
next prev parent reply other threads:[~2024-02-06 21:14 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-01 15:33 [PATCH 00/10] crypto: qat - enable SRIOV VF live migration for Xin Zeng
2024-02-01 15:33 ` [PATCH 01/10] crypto: qat - adf_get_etr_base() helper Xin Zeng
2024-02-01 15:33 ` [PATCH 02/10] crypto: qat - relocate and rename 4xxx PF2VM definitions Xin Zeng
2024-02-01 15:33 ` [PATCH 03/10] crypto: qat - move PFVF compat checker to a function Xin Zeng
2024-02-01 15:33 ` [PATCH 04/10] crypto: qat - relocate CSR access code Xin Zeng
2024-02-01 15:33 ` [PATCH 05/10] crypto: qat - rename get_sla_arr_of_type() Xin Zeng
2024-02-01 15:33 ` [PATCH 06/10] crypto: qat - expand CSR operations for QAT GEN4 devices Xin Zeng
2024-02-01 15:33 ` [PATCH 07/10] crypto: qat - add bank save and restore flows Xin Zeng
2024-02-04 7:49 ` [EXTERNAL] " Kamlesh Gurudasani
2024-02-19 3:41 ` Wan, Siming
2024-02-01 15:33 ` [PATCH 08/10] crypto: qat - add interface for live migration Xin Zeng
2024-02-01 15:33 ` [PATCH 09/10] crypto: qat - implement " Xin Zeng
2024-02-01 15:33 ` [PATCH 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices Xin Zeng
2024-02-06 12:55 ` Jason Gunthorpe
2024-02-09 8:23 ` Zeng, Xin
2024-02-09 12:10 ` Jason Gunthorpe
2024-02-11 8:17 ` Yishai Hadas
2024-02-17 16:20 ` Zeng, Xin
2024-02-20 13:24 ` Jason Gunthorpe
2024-02-20 15:53 ` Zeng, Xin
2024-02-20 17:03 ` Jason Gunthorpe
2024-02-21 8:44 ` Zeng, Xin
2024-02-21 13:18 ` Jason Gunthorpe
2024-02-21 15:11 ` Yishai Hadas
2024-02-22 9:39 ` Yishai Hadas
2024-02-06 21:14 ` Alex Williamson [this message]
2024-02-09 16:16 ` Zeng, Xin
2024-02-08 12:17 ` Shameerali Kolothum Thodi
2024-02-13 13:08 ` Zeng, Xin
2024-02-13 14:55 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240206141446.3547e4a9.alex.williamson@redhat.com \
--to=alex.williamson@redhat.com \
--cc=herbert@gondor.apana.org.au \
--cc=jgg@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-crypto@vger.kernel.org \
--cc=qat-linux@intel.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=xin.zeng@intel.com \
--cc=yahui.cao@intel.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).