From: Alex Williamson <alex.williamson@redhat.com>
To: Yishai Hadas <yishaih@nvidia.com>
Cc: <bhelgaas@google.com>, <jgg@nvidia.com>, <saeedm@nvidia.com>,
<linux-pci@vger.kernel.org>, <kvm@vger.kernel.org>,
<netdev@vger.kernel.org>, <kuba@kernel.org>, <leonro@nvidia.com>,
<kwankhede@nvidia.com>, <mgurtovoy@nvidia.com>,
<maorg@nvidia.com>
Subject: Re: [PATCH V2 mlx5-next 12/14] vfio/mlx5: Implement vfio_pci driver for mlx5 devices
Date: Tue, 19 Oct 2021 12:43:52 -0600 [thread overview]
Message-ID: <20211019124352.74c3b6ba.alex.williamson@redhat.com> (raw)
In-Reply-To: <20211019105838.227569-13-yishaih@nvidia.com>
On Tue, 19 Oct 2021 13:58:36 +0300
Yishai Hadas <yishaih@nvidia.com> wrote:
> This patch adds support for vfio_pci driver for mlx5 devices.
>
> It uses vfio_pci_core to register to the VFIO subsystem and then
> implements the mlx5 specific logic in the migration area.
>
> The migration implementation follows the definition from uapi/vfio.h and
> uses the mlx5 VF->PF command channel to achieve it.
>
> This patch implements the suspend/resume flows.
>
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> ---
> MAINTAINERS | 6 +
> drivers/vfio/pci/Kconfig | 3 +
> drivers/vfio/pci/Makefile | 2 +
> drivers/vfio/pci/mlx5/Kconfig | 10 +
> drivers/vfio/pci/mlx5/Makefile | 4 +
> drivers/vfio/pci/mlx5/main.c | 696 +++++++++++++++++++++++++++++++++
> 6 files changed, 721 insertions(+)
> create mode 100644 drivers/vfio/pci/mlx5/Kconfig
> create mode 100644 drivers/vfio/pci/mlx5/Makefile
> create mode 100644 drivers/vfio/pci/mlx5/main.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index abdcbcfef73d..e824bfab4a01 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -19699,6 +19699,12 @@ L: kvm@vger.kernel.org
> S: Maintained
> F: drivers/vfio/platform/
>
> +VFIO MLX5 PCI DRIVER
> +M: Yishai Hadas <yishaih@nvidia.com>
> +L: kvm@vger.kernel.org
> +S: Maintained
> +F: drivers/vfio/pci/mlx5/
> +
> VGA_SWITCHEROO
> R: Lukas Wunner <lukas@wunner.de>
> S: Maintained
> diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig
> index 860424ccda1b..187b9c259944 100644
> --- a/drivers/vfio/pci/Kconfig
> +++ b/drivers/vfio/pci/Kconfig
> @@ -43,4 +43,7 @@ config VFIO_PCI_IGD
>
> To enable Intel IGD assignment through vfio-pci, say Y.
> endif
> +
> +source "drivers/vfio/pci/mlx5/Kconfig"
> +
> endif
> diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile
> index 349d68d242b4..ed9d6f2e0555 100644
> --- a/drivers/vfio/pci/Makefile
> +++ b/drivers/vfio/pci/Makefile
> @@ -7,3 +7,5 @@ obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o
> vfio-pci-y := vfio_pci.o
> vfio-pci-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o
> obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
> +
> +obj-$(CONFIG_MLX5_VFIO_PCI) += mlx5/
> diff --git a/drivers/vfio/pci/mlx5/Kconfig b/drivers/vfio/pci/mlx5/Kconfig
> new file mode 100644
> index 000000000000..119712656400
> --- /dev/null
> +++ b/drivers/vfio/pci/mlx5/Kconfig
> @@ -0,0 +1,10 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config MLX5_VFIO_PCI
> + tristate "VFIO support for MLX5 PCI devices"
> + depends on MLX5_CORE
> + select VFIO_PCI_CORE
> + help
> + This provides a migration support for MLX5 devices using the VFIO
s/ a//
> + framework.
> +
> + If you don't know what to do here, say N.
> diff --git a/drivers/vfio/pci/mlx5/Makefile b/drivers/vfio/pci/mlx5/Makefile
> new file mode 100644
> index 000000000000..689627da7ff5
> --- /dev/null
> +++ b/drivers/vfio/pci/mlx5/Makefile
> @@ -0,0 +1,4 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +obj-$(CONFIG_MLX5_VFIO_PCI) += mlx5-vfio-pci.o
> +mlx5-vfio-pci-y := main.o cmd.o
> +
> diff --git a/drivers/vfio/pci/mlx5/main.c b/drivers/vfio/pci/mlx5/main.c
> new file mode 100644
> index 000000000000..621b7fc60544
> --- /dev/null
> +++ b/drivers/vfio/pci/mlx5/main.c
> @@ -0,0 +1,696 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + */
> +
> +#include <linux/device.h>
> +#include <linux/eventfd.h>
> +#include <linux/file.h>
> +#include <linux/interrupt.h>
> +#include <linux/iommu.h>
> +#include <linux/module.h>
> +#include <linux/mutex.h>
> +#include <linux/notifier.h>
> +#include <linux/pci.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/types.h>
> +#include <linux/uaccess.h>
> +#include <linux/vfio.h>
> +#include <linux/sched/mm.h>
> +#include <linux/vfio_pci_core.h>
> +
> +#include "cmd.h"
> +
> +enum {
> + MLX5VF_PCI_FREEZED = 1 << 0,
> +};
> +
> +enum {
> + MLX5VF_REGION_PENDING_BYTES = 1 << 0,
> + MLX5VF_REGION_DATA_SIZE = 1 << 1,
> +};
> +
> +enum {
> + MLX5VF_SUPPORTED_DEVICE_STATES = VFIO_DEVICE_STATE_RUNNING |
> + VFIO_DEVICE_STATE_SAVING |
> + VFIO_DEVICE_STATE_RESUMING,
> +};
> +
> +#define MLX5VF_MIG_REGION_DATA_SIZE SZ_128K
> +/* Data section offset from migration region */
> +#define MLX5VF_MIG_REGION_DATA_OFFSET \
> + (sizeof(struct vfio_device_migration_info))
> +
> +#define VFIO_DEVICE_MIGRATION_OFFSET(x) \
> + (offsetof(struct vfio_device_migration_info, x))
> +
> +struct mlx5vf_pci_migration_info {
> + u32 vfio_dev_state; /* VFIO_DEVICE_STATE_XXX */
> + u32 dev_state; /* device migration state */
> + u32 region_state; /* Use MLX5VF_REGION_XXX */
> + u16 vhca_id;
> + struct mlx5_vhca_state_data vhca_state_data;
> +};
> +
> +struct mlx5vf_pci_core_device {
> + struct vfio_pci_core_device core_device;
> + u8 migrate_cap:1;
> + /* protect migartion state */
s/migartion/migration/
> + struct mutex state_mutex;
> + struct mlx5vf_pci_migration_info vmig;
> +};
> +
> +static int mlx5vf_pci_unquiesce_device(struct mlx5vf_pci_core_device *mvdev)
> +{
> + return mlx5vf_cmd_resume_vhca(mvdev->core_device.pdev,
> + mvdev->vmig.vhca_id,
> + MLX5_RESUME_VHCA_IN_OP_MOD_RESUME_MASTER);
> +}
> +
> +static int mlx5vf_pci_quiesce_device(struct mlx5vf_pci_core_device *mvdev)
> +{
> + return mlx5vf_cmd_suspend_vhca(
> + mvdev->core_device.pdev, mvdev->vmig.vhca_id,
> + MLX5_SUSPEND_VHCA_IN_OP_MOD_SUSPEND_MASTER);
> +}
> +
> +static int mlx5vf_pci_unfreeze_device(struct mlx5vf_pci_core_device *mvdev)
> +{
> + int ret;
> +
> + ret = mlx5vf_cmd_resume_vhca(mvdev->core_device.pdev,
> + mvdev->vmig.vhca_id,
> + MLX5_RESUME_VHCA_IN_OP_MOD_RESUME_SLAVE);
> + if (ret)
> + return ret;
> +
> + mvdev->vmig.dev_state &= ~MLX5VF_PCI_FREEZED;
> + return 0;
> +}
> +
> +static int mlx5vf_pci_freeze_device(struct mlx5vf_pci_core_device *mvdev)
> +{
> + int ret;
> +
> + ret = mlx5vf_cmd_suspend_vhca(
> + mvdev->core_device.pdev, mvdev->vmig.vhca_id,
> + MLX5_SUSPEND_VHCA_IN_OP_MOD_SUSPEND_SLAVE);
> + if (ret)
> + return ret;
> +
> + mvdev->vmig.dev_state |= MLX5VF_PCI_FREEZED;
> + return 0;
> +}
> +
> +static int mlx5vf_pci_save_device_data(struct mlx5vf_pci_core_device *mvdev)
> +{
> + u32 state_size = 0;
> + int ret;
> +
> + if (!(mvdev->vmig.dev_state & MLX5VF_PCI_FREEZED))
> + return -EFAULT;
> +
> + /* If we already read state no reason to re-read */
> + if (mvdev->vmig.vhca_state_data.state_size)
> + return 0;
> +
> + ret = mlx5vf_cmd_query_vhca_migration_state(
> + mvdev->core_device.pdev, mvdev->vmig.vhca_id, &state_size);
> + if (ret)
> + return ret;
> +
> + return mlx5vf_cmd_save_vhca_state(mvdev->core_device.pdev,
> + mvdev->vmig.vhca_id, state_size,
> + &mvdev->vmig.vhca_state_data);
> +}
> +
> +static int mlx5vf_pci_new_write_window(struct mlx5vf_pci_core_device *mvdev)
> +{
> + struct mlx5_vhca_state_data *state_data = &mvdev->vmig.vhca_state_data;
> + u32 num_pages_needed;
> + u64 allocated_ready;
> + u32 bytes_needed;
> +
> + /* Check how many bytes are available from previous flows */
> + WARN_ON(state_data->num_pages * PAGE_SIZE <
> + state_data->win_start_offset);
> + allocated_ready = (state_data->num_pages * PAGE_SIZE) -
> + state_data->win_start_offset;
> + WARN_ON(allocated_ready > MLX5VF_MIG_REGION_DATA_SIZE);
> +
> + bytes_needed = MLX5VF_MIG_REGION_DATA_SIZE - allocated_ready;
> + if (!bytes_needed)
> + return 0;
> +
> + num_pages_needed = DIV_ROUND_UP_ULL(bytes_needed, PAGE_SIZE);
> + return mlx5vf_add_migration_pages(state_data, num_pages_needed);
> +}
> +
> +static ssize_t
> +mlx5vf_pci_handle_migration_data_size(struct mlx5vf_pci_core_device *mvdev,
> + char __user *buf, bool iswrite)
> +{
> + struct mlx5vf_pci_migration_info *vmig = &mvdev->vmig;
> + u64 data_size;
> + int ret;
> +
> + if (iswrite) {
> + /* data_size is writable only during resuming state */
> + if (vmig->vfio_dev_state != VFIO_DEVICE_STATE_RESUMING)
> + return -EINVAL;
> +
> + ret = copy_from_user(&data_size, buf, sizeof(data_size));
> + if (ret)
> + return -EFAULT;
> +
> + vmig->vhca_state_data.state_size += data_size;
> + vmig->vhca_state_data.win_start_offset += data_size;
> + ret = mlx5vf_pci_new_write_window(mvdev);
> + if (ret)
> + return ret;
> +
> + } else {
> + if (vmig->vfio_dev_state != VFIO_DEVICE_STATE_SAVING)
> + return -EINVAL;
> +
> + data_size = min_t(u64, MLX5VF_MIG_REGION_DATA_SIZE,
> + vmig->vhca_state_data.state_size -
> + vmig->vhca_state_data.win_start_offset);
> + ret = copy_to_user(buf, &data_size, sizeof(data_size));
> + if (ret)
> + return -EFAULT;
> + }
> +
> + vmig->region_state |= MLX5VF_REGION_DATA_SIZE;
> + return sizeof(data_size);
> +}
> +
> +static ssize_t
> +mlx5vf_pci_handle_migration_data_offset(struct mlx5vf_pci_core_device *mvdev,
> + char __user *buf, bool iswrite)
> +{
> + static const u64 data_offset = MLX5VF_MIG_REGION_DATA_OFFSET;
> + int ret;
> +
> + /* RO field */
> + if (iswrite)
> + return -EFAULT;
> +
> + ret = copy_to_user(buf, &data_offset, sizeof(data_offset));
> + if (ret)
> + return -EFAULT;
> +
> + return sizeof(data_offset);
> +}
> +
> +static ssize_t
> +mlx5vf_pci_handle_migration_pending_bytes(struct mlx5vf_pci_core_device *mvdev,
> + char __user *buf, bool iswrite)
> +{
> + struct mlx5vf_pci_migration_info *vmig = &mvdev->vmig;
> + u64 pending_bytes;
> + int ret;
> +
> + /* RO field */
> + if (iswrite)
> + return -EFAULT;
> +
> + if (vmig->vfio_dev_state == (VFIO_DEVICE_STATE_SAVING |
> + VFIO_DEVICE_STATE_RUNNING)) {
> + /* In pre-copy state we have no data to return for now,
> + * return 0 pending bytes
> + */
> + pending_bytes = 0;
> + } else {
> + if (!vmig->vhca_state_data.state_size)
> + return 0;
> + pending_bytes = vmig->vhca_state_data.state_size -
> + vmig->vhca_state_data.win_start_offset;
> + }
> +
> + ret = copy_to_user(buf, &pending_bytes, sizeof(pending_bytes));
> + if (ret)
> + return -EFAULT;
> +
> + /* Window moves forward once data from previous iteration was read */
> + if (vmig->region_state & MLX5VF_REGION_DATA_SIZE)
> + vmig->vhca_state_data.win_start_offset +=
> + min_t(u64, MLX5VF_MIG_REGION_DATA_SIZE, pending_bytes);
> +
> + WARN_ON(vmig->vhca_state_data.win_start_offset >
> + vmig->vhca_state_data.state_size);
> +
> + /* New iteration started */
> + vmig->region_state = MLX5VF_REGION_PENDING_BYTES;
> + return sizeof(pending_bytes);
> +}
> +
> +static int mlx5vf_load_state(struct mlx5vf_pci_core_device *mvdev)
> +{
> + if (!mvdev->vmig.vhca_state_data.state_size)
> + return 0;
> +
> + return mlx5vf_cmd_load_vhca_state(mvdev->core_device.pdev,
> + mvdev->vmig.vhca_id,
> + &mvdev->vmig.vhca_state_data);
> +}
> +
> +static void mlx5vf_reset_mig_state(struct mlx5vf_pci_core_device *mvdev)
> +{
> + struct mlx5vf_pci_migration_info *vmig = &mvdev->vmig;
> +
> + vmig->region_state = 0;
> + mlx5vf_reset_vhca_state(&vmig->vhca_state_data);
> +}
> +
> +static int mlx5vf_pci_set_device_state(struct mlx5vf_pci_core_device *mvdev,
> + u32 state)
> +{
> + struct mlx5vf_pci_migration_info *vmig = &mvdev->vmig;
> + u32 old_state = vmig->vfio_dev_state;
> + int ret = 0;
> +
> + if (old_state == VFIO_DEVICE_STATE_ERROR ||
> + !VFIO_DEVICE_STATE_VALID(state) ||
> + (state & ~MLX5VF_SUPPORTED_DEVICE_STATES))
> + return -EINVAL;
> +
> + /* Running switches off */
> + if (((old_state ^ state) & VFIO_DEVICE_STATE_RUNNING) &&
> + (old_state & VFIO_DEVICE_STATE_RUNNING)) {
> + ret = mlx5vf_pci_quiesce_device(mvdev);
> + if (ret)
> + return ret;
> + ret = mlx5vf_pci_freeze_device(mvdev);
> + if (ret) {
> + vmig->vfio_dev_state = VFIO_DEVICE_STATE_ERROR;
> + return ret;
> + }
> + }
> +
> + /* Resuming switches off */
> + if (((old_state ^ state) & VFIO_DEVICE_STATE_RESUMING) &&
> + (old_state & VFIO_DEVICE_STATE_RESUMING)) {
> + /* deserialize state into the device */
> + ret = mlx5vf_load_state(mvdev);
> + if (ret) {
> + vmig->vfio_dev_state = VFIO_DEVICE_STATE_ERROR;
> + return ret;
> + }
> + }
> +
> + /* Resuming switches on */
> + if (((old_state ^ state) & VFIO_DEVICE_STATE_RESUMING) &&
> + (state & VFIO_DEVICE_STATE_RESUMING)) {
> + mlx5vf_reset_mig_state(mvdev);
> + ret = mlx5vf_pci_new_write_window(mvdev);
> + if (ret)
> + return ret;
> + }
A couple nits here...
Perhaps:
if ((old_state ^ state) & VFIO_DEVICE_STATE_RESUMING)) {
/* Resuming bit cleared */
if (old_state & VFIO_DEVICE_STATE_RESUMING) {
...
} else { /* Resuming bit set */
...
}
}
Also
u32 flipped_bits = old_state ^ state;
or similar would simplify all these cases slightly.
> +
> + /* Saving switches on */
> + if (((old_state ^ state) & VFIO_DEVICE_STATE_SAVING) &&
> + (state & VFIO_DEVICE_STATE_SAVING)) {
> + if (!(state & VFIO_DEVICE_STATE_RUNNING)) {
> + /* serialize post copy */
> + ret = mlx5vf_pci_save_device_data(mvdev);
> + if (ret)
> + return ret;
> + }
> + }
This doesn't catch all the cases, and in fact misses the most expected
case where userspace clears the _RUNNING bit while _SAVING is already
enabled. Does that mean this hasn't actually been tested with QEMU?
It seems like there also needs to be a clause in the case where
_RUNNING switches off to test if _SAVING is already set and has not
toggled.
> +
> + /* Running switches on */
> + if (((old_state ^ state) & VFIO_DEVICE_STATE_RUNNING) &&
> + (state & VFIO_DEVICE_STATE_RUNNING)) {
> + ret = mlx5vf_pci_unfreeze_device(mvdev);
> + if (ret)
> + return ret;
> + ret = mlx5vf_pci_unquiesce_device(mvdev);
> + if (ret) {
> + vmig->vfio_dev_state = VFIO_DEVICE_STATE_ERROR;
> + return ret;
> + }
> + }
Per previous discussion, I understand that freeze and quiesce are
loosely stop-responding-to-dma and stop-sending-dma, respectively.
Once we're quiesced and frozen, device state doesn't change. What are
the implications to userspace that we don't expose a quiesce state
(yet)? I'm wondering if this needs to be resolved before we introduce
our first in-tree user of the uAPI (and before QEMU support becomes
non-experimental). Thanks,
Alex
next prev parent reply other threads:[~2021-10-19 18:44 UTC|newest]
Thread overview: 100+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-19 10:58 [PATCH V2 mlx5-next 00/14] Add mlx5 live migration driver Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 01/14] PCI/IOV: Add pci_iov_vf_id() to get VF index Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 02/14] net/mlx5: Reuse exported virtfn index function call Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 03/14] net/mlx5: Disable SRIOV before PF removal Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 04/14] PCI/IOV: Add pci_iov_get_pf_drvdata() to allow VF reaching the drvdata of a PF Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 05/14] net/mlx5: Expose APIs to get/put the mlx5 core device Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 06/14] vdpa/mlx5: Use mlx5_vf_get_core_dev() to get PF device Yishai Hadas
2021-10-19 11:16 ` Max Gurtovoy
2021-10-20 8:58 ` Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 07/14] vfio: Fix VFIO_DEVICE_STATE_SET_ERROR macro Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 08/14] vfio: Add a macro for VFIO_DEVICE_STATE_ERROR Yishai Hadas
2021-10-19 15:48 ` Alex Williamson
2021-10-19 15:50 ` Alex Williamson
2021-10-20 7:35 ` Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 09/14] vfio/pci_core: Make the region->release() function optional Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 10/14] net/mlx5: Introduce migration bits and structures Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 11/14] vfio/mlx5: Expose migration commands over mlx5 device Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 12/14] vfio/mlx5: Implement vfio_pci driver for mlx5 devices Yishai Hadas
2021-10-19 18:43 ` Alex Williamson [this message]
2021-10-19 19:23 ` Jason Gunthorpe
2021-10-19 20:58 ` Alex Williamson
2021-10-19 23:04 ` Jason Gunthorpe
2021-10-20 8:28 ` Yishai Hadas
2021-10-20 16:52 ` Alex Williamson
2021-10-20 18:59 ` Jason Gunthorpe
2021-10-20 21:07 ` Alex Williamson
2021-10-21 9:34 ` Cornelia Huck
2021-10-21 21:47 ` Alex Williamson
2021-10-25 12:29 ` Jason Gunthorpe
2021-10-25 14:28 ` Alex Williamson
2021-10-25 14:56 ` Jason Gunthorpe
2021-10-26 14:42 ` Alex Williamson
2021-10-26 15:18 ` Jason Gunthorpe
2021-10-26 19:50 ` Alex Williamson
2021-10-26 23:43 ` Jason Gunthorpe
2021-10-27 19:05 ` Alex Williamson
2021-10-27 19:23 ` Jason Gunthorpe
2021-10-28 15:08 ` Cornelia Huck
2021-10-29 0:26 ` Jason Gunthorpe
2021-10-29 7:35 ` Yishai Hadas
2021-10-28 15:30 ` Alex Williamson
2021-10-28 23:47 ` Jason Gunthorpe
2021-10-29 6:57 ` Cornelia Huck
2021-10-29 7:48 ` Yishai Hadas
2021-10-29 10:32 ` Shameerali Kolothum Thodi
2021-10-29 12:15 ` Jason Gunthorpe
2021-10-29 22:06 ` Alex Williamson
2021-11-01 17:25 ` Jason Gunthorpe
2021-11-02 11:19 ` Shameerali Kolothum Thodi
2021-11-02 14:56 ` Alex Williamson
2021-11-02 15:54 ` Jason Gunthorpe
2021-11-02 16:22 ` Alex Williamson
2021-11-02 16:36 ` Jason Gunthorpe
2021-11-02 20:15 ` Alex Williamson
2021-11-03 12:09 ` Jason Gunthorpe
2021-11-03 15:44 ` Alex Williamson
2021-11-03 16:10 ` Jason Gunthorpe
2021-11-03 18:04 ` Alex Williamson
2021-11-04 11:19 ` Cornelia Huck
2021-11-05 16:53 ` Cornelia Huck
2021-11-16 16:59 ` Cornelia Huck
2021-11-05 13:24 ` Jason Gunthorpe
2021-11-05 15:31 ` Alex Williamson
2021-11-15 23:29 ` Jason Gunthorpe
2021-11-16 17:57 ` Alex Williamson
2021-11-16 19:25 ` Jason Gunthorpe
2021-11-16 21:10 ` Alex Williamson
2021-11-17 1:48 ` Jason Gunthorpe
2021-11-18 18:15 ` Alex Williamson
2021-11-22 19:18 ` Jason Gunthorpe
2021-11-08 8:53 ` Tian, Kevin
2021-11-08 12:35 ` Jason Gunthorpe
2021-11-09 0:58 ` Tian, Kevin
2021-11-09 12:45 ` Jason Gunthorpe
2021-10-25 16:34 ` Dr. David Alan Gilbert
2021-10-25 17:55 ` Alex Williamson
2021-10-25 18:47 ` Dr. David Alan Gilbert
2021-10-25 19:15 ` Jason Gunthorpe
2021-10-26 8:40 ` Dr. David Alan Gilbert
2021-10-26 12:13 ` Jason Gunthorpe
2021-10-26 14:52 ` Alex Williamson
2021-10-26 15:56 ` Jason Gunthorpe
2021-10-26 14:29 ` Alex Williamson
2021-10-26 14:51 ` Dr. David Alan Gilbert
2021-10-26 15:25 ` Jason Gunthorpe
2021-10-20 8:01 ` Yishai Hadas
2021-10-20 16:25 ` Jason Gunthorpe
2021-10-21 10:46 ` Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 13/14] vfio/pci: Expose vfio_pci_aer_err_detected() Yishai Hadas
2021-10-19 10:58 ` [PATCH V2 mlx5-next 14/14] vfio/mlx5: Use its own PCI reset_done error handler Yishai Hadas
2021-10-19 18:55 ` Alex Williamson
2021-10-19 19:10 ` Jason Gunthorpe
2021-10-20 8:46 ` Yishai Hadas
2021-10-20 16:46 ` Jason Gunthorpe
2021-10-20 17:45 ` Alex Williamson
2021-10-20 18:57 ` Jason Gunthorpe
2021-10-20 21:38 ` Alex Williamson
2021-10-21 10:39 ` Yishai Hadas
2021-11-17 16:42 ` vfio migration discussions (was: [PATCH V2 mlx5-next 00/14] Add mlx5 live migration driver) Cornelia Huck
2021-11-17 17:47 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211019124352.74c3b6ba.alex.williamson@redhat.com \
--to=alex.williamson@redhat.com \
--cc=bhelgaas@google.com \
--cc=jgg@nvidia.com \
--cc=kuba@kernel.org \
--cc=kvm@vger.kernel.org \
--cc=kwankhede@nvidia.com \
--cc=leonro@nvidia.com \
--cc=linux-pci@vger.kernel.org \
--cc=maorg@nvidia.com \
--cc=mgurtovoy@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=saeedm@nvidia.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.