* [PATCH v2 0/2] Remove the VFIO_IOMMU_NOTIFY_DMA_UNMAP notifier
@ 2022-06-07 23:02 ` Jason Gunthorpe
0 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-07 23:02 UTC (permalink / raw)
To: Alexander Gordeev, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Christoph Hellwig
This is the last notifier toward the drivers, replace it with a simple op
callback in the vfio_device_ops.
v2:
- Declare and initialize variables in intel_vgpu_dma_unmap()
- Remove 'vendor' when touching comments
- Remove kdoc for vfio dma_unmap notifier
- Add WARN_ON to vfio_register_emulated_iommu_dev() since dma_unmap is
mandatory
- Move dma_unmap call loop to vfio_notify_dma_unmap()
- Document why the double mutex is being used and why the mutex lock is
dropped when calling dma_unmap
v1: https://lore.kernel.org/r/0-v1-896844109f36+a-vfio_unmap_notif_jgg@nvidia.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Jason Gunthorpe (2):
vfio: Replace the DMA unmapping notifier with a callback
vfio: Replace the iommu notifier with a device list
drivers/gpu/drm/i915/gvt/gvt.h | 1 -
drivers/gpu/drm/i915/gvt/kvmgt.c | 75 +++++-------------
drivers/s390/cio/vfio_ccw_ops.c | 41 +++-------
drivers/s390/cio/vfio_ccw_private.h | 1 -
drivers/s390/crypto/vfio_ap_ops.c | 53 ++-----------
drivers/s390/crypto/vfio_ap_private.h | 3 -
drivers/vfio/vfio.c | 108 ++++++--------------------
drivers/vfio/vfio.h | 9 +--
drivers/vfio/vfio_iommu_type1.c | 103 +++++++++++++++---------
include/linux/vfio.h | 21 +----
10 files changed, 134 insertions(+), 281 deletions(-)
base-commit: f2906aa863381afb0015a9eb7fefad885d4e5a56
--
2.36.1
^ permalink raw reply [flat|nested] 34+ messages in thread
* [Intel-gfx] [PATCH v2 0/2] Remove the VFIO_IOMMU_NOTIFY_DMA_UNMAP notifier
@ 2022-06-07 23:02 ` Jason Gunthorpe
0 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-07 23:02 UTC (permalink / raw)
To: Alexander Gordeev, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Christoph Hellwig
This is the last notifier toward the drivers, replace it with a simple op
callback in the vfio_device_ops.
v2:
- Declare and initialize variables in intel_vgpu_dma_unmap()
- Remove 'vendor' when touching comments
- Remove kdoc for vfio dma_unmap notifier
- Add WARN_ON to vfio_register_emulated_iommu_dev() since dma_unmap is
mandatory
- Move dma_unmap call loop to vfio_notify_dma_unmap()
- Document why the double mutex is being used and why the mutex lock is
dropped when calling dma_unmap
v1: https://lore.kernel.org/r/0-v1-896844109f36+a-vfio_unmap_notif_jgg@nvidia.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Jason Gunthorpe (2):
vfio: Replace the DMA unmapping notifier with a callback
vfio: Replace the iommu notifier with a device list
drivers/gpu/drm/i915/gvt/gvt.h | 1 -
drivers/gpu/drm/i915/gvt/kvmgt.c | 75 +++++-------------
drivers/s390/cio/vfio_ccw_ops.c | 41 +++-------
drivers/s390/cio/vfio_ccw_private.h | 1 -
drivers/s390/crypto/vfio_ap_ops.c | 53 ++-----------
drivers/s390/crypto/vfio_ap_private.h | 3 -
drivers/vfio/vfio.c | 108 ++++++--------------------
drivers/vfio/vfio.h | 9 +--
drivers/vfio/vfio_iommu_type1.c | 103 +++++++++++++++---------
include/linux/vfio.h | 21 +----
10 files changed, 134 insertions(+), 281 deletions(-)
base-commit: f2906aa863381afb0015a9eb7fefad885d4e5a56
--
2.36.1
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
@ 2022-06-07 23:02 ` Jason Gunthorpe
-1 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-07 23:02 UTC (permalink / raw)
To: Alexander Gordeev, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Christoph Hellwig
Instead of having drivers register the notifier with explicit code just
have them provide a dma_unmap callback op in their driver ops and rely on
the core code to wire it up.
Suggested-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/gpu/drm/i915/gvt/gvt.h | 1 -
drivers/gpu/drm/i915/gvt/kvmgt.c | 75 ++++-----------
drivers/s390/cio/vfio_ccw_ops.c | 41 ++-------
drivers/s390/cio/vfio_ccw_private.h | 1 -
drivers/s390/crypto/vfio_ap_ops.c | 53 ++---------
drivers/s390/crypto/vfio_ap_private.h | 3 -
drivers/vfio/vfio.c | 126 +++++++++-----------------
drivers/vfio/vfio.h | 5 +
include/linux/vfio.h | 21 +----
9 files changed, 87 insertions(+), 239 deletions(-)
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index aee1a45da74bcb..705689e6401197 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -226,7 +226,6 @@ struct intel_vgpu {
unsigned long nr_cache_entries;
struct mutex cache_lock;
- struct notifier_block iommu_notifier;
atomic_t released;
struct kvm_page_track_notifier_node track_node;
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index e2f6c56ab3420c..ecd5bb37b63a2a 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -729,34 +729,25 @@ int intel_gvt_set_edid(struct intel_vgpu *vgpu, int port_num)
return ret;
}
-static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
- unsigned long action, void *data)
+static void intel_vgpu_dma_unmap(struct vfio_device *vfio_dev, u64 iova,
+ u64 length)
{
- struct intel_vgpu *vgpu =
- container_of(nb, struct intel_vgpu, iommu_notifier);
-
- if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
- struct vfio_iommu_type1_dma_unmap *unmap = data;
- struct gvt_dma *entry;
- unsigned long iov_pfn, end_iov_pfn;
+ struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
+ struct gvt_dma *entry;
+ u64 iov_pfn = iova >> PAGE_SHIFT;
+ u64 end_iov_pfn = iov_pfn + length / PAGE_SIZE;
- iov_pfn = unmap->iova >> PAGE_SHIFT;
- end_iov_pfn = iov_pfn + unmap->size / PAGE_SIZE;
+ mutex_lock(&vgpu->cache_lock);
+ for (; iov_pfn < end_iov_pfn; iov_pfn++) {
+ entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
+ if (!entry)
+ continue;
- mutex_lock(&vgpu->cache_lock);
- for (; iov_pfn < end_iov_pfn; iov_pfn++) {
- entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
- if (!entry)
- continue;
-
- gvt_dma_unmap_page(vgpu, entry->gfn, entry->dma_addr,
- entry->size);
- __gvt_cache_remove_entry(vgpu, entry);
- }
- mutex_unlock(&vgpu->cache_lock);
+ gvt_dma_unmap_page(vgpu, entry->gfn, entry->dma_addr,
+ entry->size);
+ __gvt_cache_remove_entry(vgpu, entry);
}
-
- return NOTIFY_OK;
+ mutex_unlock(&vgpu->cache_lock);
}
static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu)
@@ -783,36 +774,20 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu)
static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
{
struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
- unsigned long events;
- int ret;
-
- vgpu->iommu_notifier.notifier_call = intel_vgpu_iommu_notifier;
- events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
- ret = vfio_register_notifier(vfio_dev, VFIO_IOMMU_NOTIFY, &events,
- &vgpu->iommu_notifier);
- if (ret != 0) {
- gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n",
- ret);
- goto out;
- }
-
- ret = -EEXIST;
if (vgpu->attached)
- goto undo_iommu;
+ return -EEXIST;
- ret = -ESRCH;
if (!vgpu->vfio_device.kvm ||
vgpu->vfio_device.kvm->mm != current->mm) {
gvt_vgpu_err("KVM is required to use Intel vGPU\n");
- goto undo_iommu;
+ return -ESRCH;
}
kvm_get_kvm(vgpu->vfio_device.kvm);
- ret = -EEXIST;
if (__kvmgt_vgpu_exist(vgpu))
- goto undo_iommu;
+ return -EEXIST;
vgpu->attached = true;
@@ -831,12 +806,6 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
atomic_set(&vgpu->released, 0);
return 0;
-
-undo_iommu:
- vfio_unregister_notifier(vfio_dev, VFIO_IOMMU_NOTIFY,
- &vgpu->iommu_notifier);
-out:
- return ret;
}
static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
@@ -853,8 +822,6 @@ static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
{
struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
- struct drm_i915_private *i915 = vgpu->gvt->gt->i915;
- int ret;
if (!vgpu->attached)
return;
@@ -864,11 +831,6 @@ static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
intel_gvt_release_vgpu(vgpu);
- ret = vfio_unregister_notifier(&vgpu->vfio_device, VFIO_IOMMU_NOTIFY,
- &vgpu->iommu_notifier);
- drm_WARN(&i915->drm, ret,
- "vfio_unregister_notifier for iommu failed: %d\n", ret);
-
debugfs_remove(debugfs_lookup(KVMGT_DEBUGFS_FILENAME, vgpu->debugfs));
kvm_page_track_unregister_notifier(vgpu->vfio_device.kvm,
@@ -1610,6 +1572,7 @@ static const struct vfio_device_ops intel_vgpu_dev_ops = {
.write = intel_vgpu_write,
.mmap = intel_vgpu_mmap,
.ioctl = intel_vgpu_ioctl,
+ .dma_unmap = intel_vgpu_dma_unmap,
};
static int intel_vgpu_probe(struct mdev_device *mdev)
diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c
index b49e2e9db2dc6f..09e0ce7b72324c 100644
--- a/drivers/s390/cio/vfio_ccw_ops.c
+++ b/drivers/s390/cio/vfio_ccw_ops.c
@@ -44,31 +44,19 @@ static int vfio_ccw_mdev_reset(struct vfio_ccw_private *private)
return ret;
}
-static int vfio_ccw_mdev_notifier(struct notifier_block *nb,
- unsigned long action,
- void *data)
+static void vfio_ccw_dma_unmap(struct vfio_device *vdev, u64 iova, u64 length)
{
struct vfio_ccw_private *private =
- container_of(nb, struct vfio_ccw_private, nb);
-
- /*
- * Vendor drivers MUST unpin pages in response to an
- * invalidation.
- */
- if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
- struct vfio_iommu_type1_dma_unmap *unmap = data;
-
- if (!cp_iova_pinned(&private->cp, unmap->iova))
- return NOTIFY_OK;
+ container_of(vdev, struct vfio_ccw_private, vdev);
- if (vfio_ccw_mdev_reset(private))
- return NOTIFY_BAD;
+ /* Drivers MUST unpin pages in response to an invalidation. */
+ if (!cp_iova_pinned(&private->cp, iova))
+ return;
- cp_free(&private->cp);
- return NOTIFY_OK;
- }
+ if (vfio_ccw_mdev_reset(private))
+ return;
- return NOTIFY_DONE;
+ cp_free(&private->cp);
}
static ssize_t name_show(struct mdev_type *mtype,
@@ -178,19 +166,11 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev)
{
struct vfio_ccw_private *private =
container_of(vdev, struct vfio_ccw_private, vdev);
- unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
int ret;
- private->nb.notifier_call = vfio_ccw_mdev_notifier;
-
- ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY,
- &events, &private->nb);
- if (ret)
- return ret;
-
ret = vfio_ccw_register_async_dev_regions(private);
if (ret)
- goto out_unregister;
+ return ret;
ret = vfio_ccw_register_schib_dev_regions(private);
if (ret)
@@ -204,7 +184,6 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev)
out_unregister:
vfio_ccw_unregister_dev_regions(private);
- vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
return ret;
}
@@ -222,7 +201,6 @@ static void vfio_ccw_mdev_close_device(struct vfio_device *vdev)
cp_free(&private->cp);
vfio_ccw_unregister_dev_regions(private);
- vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
}
static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private *private,
@@ -645,6 +623,7 @@ static const struct vfio_device_ops vfio_ccw_dev_ops = {
.write = vfio_ccw_mdev_write,
.ioctl = vfio_ccw_mdev_ioctl,
.request = vfio_ccw_mdev_request,
+ .dma_unmap = vfio_ccw_dma_unmap,
};
struct mdev_driver vfio_ccw_mdev_driver = {
diff --git a/drivers/s390/cio/vfio_ccw_private.h b/drivers/s390/cio/vfio_ccw_private.h
index 7272eb78861244..2627791c9006d4 100644
--- a/drivers/s390/cio/vfio_ccw_private.h
+++ b/drivers/s390/cio/vfio_ccw_private.h
@@ -98,7 +98,6 @@ struct vfio_ccw_private {
struct completion *completion;
atomic_t avail;
struct mdev_device *mdev;
- struct notifier_block nb;
struct ccw_io_region *io_region;
struct mutex io_mutex;
struct vfio_ccw_region *region;
diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
index a7d2a95796d360..bb1a1677c5c230 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -1226,34 +1226,14 @@ static int vfio_ap_mdev_set_kvm(struct ap_matrix_mdev *matrix_mdev,
return 0;
}
-/**
- * vfio_ap_mdev_iommu_notifier - IOMMU notifier callback
- *
- * @nb: The notifier block
- * @action: Action to be taken
- * @data: data associated with the request
- *
- * For an UNMAP request, unpin the guest IOVA (the NIB guest address we
- * pinned before). Other requests are ignored.
- *
- * Return: for an UNMAP request, NOFITY_OK; otherwise NOTIFY_DONE.
- */
-static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb,
- unsigned long action, void *data)
+static void vfio_ap_mdev_dma_unmap(struct vfio_device *vdev, u64 iova,
+ u64 length)
{
- struct ap_matrix_mdev *matrix_mdev;
-
- matrix_mdev = container_of(nb, struct ap_matrix_mdev, iommu_notifier);
-
- if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
- struct vfio_iommu_type1_dma_unmap *unmap = data;
- unsigned long g_pfn = unmap->iova >> PAGE_SHIFT;
-
- vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
- return NOTIFY_OK;
- }
+ struct ap_matrix_mdev *matrix_mdev =
+ container_of(vdev, struct ap_matrix_mdev, vdev);
+ unsigned long g_pfn = iova >> PAGE_SHIFT;
- return NOTIFY_DONE;
+ vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
}
/**
@@ -1380,27 +1360,11 @@ static int vfio_ap_mdev_open_device(struct vfio_device *vdev)
{
struct ap_matrix_mdev *matrix_mdev =
container_of(vdev, struct ap_matrix_mdev, vdev);
- unsigned long events;
- int ret;
if (!vdev->kvm)
return -EINVAL;
- ret = vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
- if (ret)
- return ret;
-
- matrix_mdev->iommu_notifier.notifier_call = vfio_ap_mdev_iommu_notifier;
- events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
- ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY, &events,
- &matrix_mdev->iommu_notifier);
- if (ret)
- goto err_kvm;
- return 0;
-
-err_kvm:
- vfio_ap_mdev_unset_kvm(matrix_mdev);
- return ret;
+ return vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
}
static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
@@ -1408,8 +1372,6 @@ static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
struct ap_matrix_mdev *matrix_mdev =
container_of(vdev, struct ap_matrix_mdev, vdev);
- vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY,
- &matrix_mdev->iommu_notifier);
vfio_ap_mdev_unset_kvm(matrix_mdev);
}
@@ -1461,6 +1423,7 @@ static const struct vfio_device_ops vfio_ap_matrix_dev_ops = {
.open_device = vfio_ap_mdev_open_device,
.close_device = vfio_ap_mdev_close_device,
.ioctl = vfio_ap_mdev_ioctl,
+ .dma_unmap = vfio_ap_mdev_dma_unmap,
};
static struct mdev_driver vfio_ap_matrix_driver = {
diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h
index a26efd804d0df3..abb59d59f81b20 100644
--- a/drivers/s390/crypto/vfio_ap_private.h
+++ b/drivers/s390/crypto/vfio_ap_private.h
@@ -81,8 +81,6 @@ struct ap_matrix {
* @node: allows the ap_matrix_mdev struct to be added to a list
* @matrix: the adapters, usage domains and control domains assigned to the
* mediated matrix device.
- * @iommu_notifier: notifier block used for specifying callback function for
- * handling the VFIO_IOMMU_NOTIFY_DMA_UNMAP even
* @kvm: the struct holding guest's state
* @pqap_hook: the function pointer to the interception handler for the
* PQAP(AQIC) instruction.
@@ -92,7 +90,6 @@ struct ap_matrix_mdev {
struct vfio_device vdev;
struct list_head node;
struct ap_matrix matrix;
- struct notifier_block iommu_notifier;
struct kvm *kvm;
crypto_hook pqap_hook;
struct mdev_device *mdev;
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 61e71c1154be67..f005b644ab9e69 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct vfio_device *device)
up_write(&device->group->group_rwsem);
}
+static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
+ void *data)
+{
+ struct vfio_device *vfio_device =
+ container_of(nb, struct vfio_device, iommu_nb);
+ struct vfio_iommu_type1_dma_unmap *unmap = data;
+
+ vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
+ return NOTIFY_OK;
+}
+
static struct file *vfio_device_open(struct vfio_device *device)
{
+ struct vfio_iommu_driver *iommu_driver;
struct file *filep;
int ret;
@@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct vfio_device *device)
if (ret)
goto err_undo_count;
}
+
+ iommu_driver = device->group->container->iommu_driver;
+ if (device->ops->dma_unmap && iommu_driver &&
+ iommu_driver->ops->register_notifier) {
+ unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
+
+ device->iommu_nb.notifier_call = vfio_iommu_notifier;
+ iommu_driver->ops->register_notifier(
+ device->group->container->iommu_data, &events,
+ &device->iommu_nb);
+ }
+
up_read(&device->group->group_rwsem);
}
mutex_unlock(&device->dev_set->lock);
@@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct vfio_device *device)
err_close_device:
mutex_lock(&device->dev_set->lock);
down_read(&device->group->group_rwsem);
- if (device->open_count == 1 && device->ops->close_device)
+ if (device->open_count == 1 && device->ops->close_device) {
device->ops->close_device(device);
+
+ iommu_driver = device->group->container->iommu_driver;
+ if (device->ops->dma_unmap && iommu_driver &&
+ iommu_driver->ops->register_notifier)
+ iommu_driver->ops->unregister_notifier(
+ device->group->container->iommu_data,
+ &device->iommu_nb);
+ }
err_undo_count:
device->open_count--;
if (device->open_count == 0 && device->kvm)
@@ -1339,12 +1371,20 @@ static const struct file_operations vfio_group_fops = {
static int vfio_device_fops_release(struct inode *inode, struct file *filep)
{
struct vfio_device *device = filep->private_data;
+ struct vfio_iommu_driver *iommu_driver;
mutex_lock(&device->dev_set->lock);
vfio_assert_device_open(device);
down_read(&device->group->group_rwsem);
if (device->open_count == 1 && device->ops->close_device)
device->ops->close_device(device);
+
+ iommu_driver = device->group->container->iommu_driver;
+ if (device->ops->dma_unmap && iommu_driver &&
+ iommu_driver->ops->register_notifier)
+ iommu_driver->ops->unregister_notifier(
+ device->group->container->iommu_data,
+ &device->iommu_nb);
up_read(&device->group->group_rwsem);
device->open_count--;
if (device->open_count == 0)
@@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data,
}
EXPORT_SYMBOL(vfio_dma_rw);
-static int vfio_register_iommu_notifier(struct vfio_group *group,
- unsigned long *events,
- struct notifier_block *nb)
-{
- struct vfio_container *container;
- struct vfio_iommu_driver *driver;
- int ret;
-
- lockdep_assert_held_read(&group->group_rwsem);
-
- container = group->container;
- driver = container->iommu_driver;
- if (likely(driver && driver->ops->register_notifier))
- ret = driver->ops->register_notifier(container->iommu_data,
- events, nb);
- else
- ret = -ENOTTY;
-
- return ret;
-}
-
-static int vfio_unregister_iommu_notifier(struct vfio_group *group,
- struct notifier_block *nb)
-{
- struct vfio_container *container;
- struct vfio_iommu_driver *driver;
- int ret;
-
- lockdep_assert_held_read(&group->group_rwsem);
-
- container = group->container;
- driver = container->iommu_driver;
- if (likely(driver && driver->ops->unregister_notifier))
- ret = driver->ops->unregister_notifier(container->iommu_data,
- nb);
- else
- ret = -ENOTTY;
-
- return ret;
-}
-
-int vfio_register_notifier(struct vfio_device *device,
- enum vfio_notify_type type, unsigned long *events,
- struct notifier_block *nb)
-{
- struct vfio_group *group = device->group;
- int ret;
-
- if (!nb || !events || (*events == 0) ||
- !vfio_assert_device_open(device))
- return -EINVAL;
-
- switch (type) {
- case VFIO_IOMMU_NOTIFY:
- ret = vfio_register_iommu_notifier(group, events, nb);
- break;
- default:
- ret = -EINVAL;
- }
- return ret;
-}
-EXPORT_SYMBOL(vfio_register_notifier);
-
-int vfio_unregister_notifier(struct vfio_device *device,
- enum vfio_notify_type type,
- struct notifier_block *nb)
-{
- struct vfio_group *group = device->group;
- int ret;
-
- if (!nb || !vfio_assert_device_open(device))
- return -EINVAL;
-
- switch (type) {
- case VFIO_IOMMU_NOTIFY:
- ret = vfio_unregister_iommu_notifier(group, nb);
- break;
- default:
- ret = -EINVAL;
- }
- return ret;
-}
-EXPORT_SYMBOL(vfio_unregister_notifier);
-
/*
* Module/class support
*/
diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
index a6713022115155..cb2e4e9baa8fe8 100644
--- a/drivers/vfio/vfio.h
+++ b/drivers/vfio/vfio.h
@@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
VFIO_IOMMU_CONTAINER_CLOSE = 0,
};
+/* events for register_notifier() */
+enum {
+ VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
+};
+
/**
* struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
*/
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index aa888cc517578e..b76623e3b92fca 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -44,6 +44,7 @@ struct vfio_device {
unsigned int open_count;
struct completion comp;
struct list_head group_next;
+ struct notifier_block iommu_nb;
};
/**
@@ -60,6 +61,8 @@ struct vfio_device {
* @match: Optional device name match callback (return: 0 for no-match, >0 for
* match, -errno for abort (ex. match with insufficient or incorrect
* additional args)
+ * @dma_unmap: Called when userspace unmaps IOVA from the container
+ * this device is attached to.
* @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
* @migration_set_state: Optional callback to change the migration state for
* devices that support migration. It's mandatory for
@@ -85,6 +88,7 @@ struct vfio_device_ops {
int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
void (*request)(struct vfio_device *vdev, unsigned int count);
int (*match)(struct vfio_device *vdev, char *buf);
+ void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
int (*device_feature)(struct vfio_device *device, u32 flags,
void __user *arg, size_t argsz);
struct file *(*migration_set_state)(
@@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn,
extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
void *data, size_t len, bool write);
-/* each type has independent events */
-enum vfio_notify_type {
- VFIO_IOMMU_NOTIFY = 0,
-};
-
-/* events for VFIO_IOMMU_NOTIFY */
-#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
-
-extern int vfio_register_notifier(struct vfio_device *device,
- enum vfio_notify_type type,
- unsigned long *required_events,
- struct notifier_block *nb);
-extern int vfio_unregister_notifier(struct vfio_device *device,
- enum vfio_notify_type type,
- struct notifier_block *nb);
-
-
/*
* Sub-module helpers
*/
--
2.36.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Intel-gfx] [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
@ 2022-06-07 23:02 ` Jason Gunthorpe
0 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-07 23:02 UTC (permalink / raw)
To: Alexander Gordeev, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Christoph Hellwig
Instead of having drivers register the notifier with explicit code just
have them provide a dma_unmap callback op in their driver ops and rely on
the core code to wire it up.
Suggested-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/gpu/drm/i915/gvt/gvt.h | 1 -
drivers/gpu/drm/i915/gvt/kvmgt.c | 75 ++++-----------
drivers/s390/cio/vfio_ccw_ops.c | 41 ++-------
drivers/s390/cio/vfio_ccw_private.h | 1 -
drivers/s390/crypto/vfio_ap_ops.c | 53 ++---------
drivers/s390/crypto/vfio_ap_private.h | 3 -
drivers/vfio/vfio.c | 126 +++++++++-----------------
drivers/vfio/vfio.h | 5 +
include/linux/vfio.h | 21 +----
9 files changed, 87 insertions(+), 239 deletions(-)
diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
index aee1a45da74bcb..705689e6401197 100644
--- a/drivers/gpu/drm/i915/gvt/gvt.h
+++ b/drivers/gpu/drm/i915/gvt/gvt.h
@@ -226,7 +226,6 @@ struct intel_vgpu {
unsigned long nr_cache_entries;
struct mutex cache_lock;
- struct notifier_block iommu_notifier;
atomic_t released;
struct kvm_page_track_notifier_node track_node;
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index e2f6c56ab3420c..ecd5bb37b63a2a 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -729,34 +729,25 @@ int intel_gvt_set_edid(struct intel_vgpu *vgpu, int port_num)
return ret;
}
-static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
- unsigned long action, void *data)
+static void intel_vgpu_dma_unmap(struct vfio_device *vfio_dev, u64 iova,
+ u64 length)
{
- struct intel_vgpu *vgpu =
- container_of(nb, struct intel_vgpu, iommu_notifier);
-
- if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
- struct vfio_iommu_type1_dma_unmap *unmap = data;
- struct gvt_dma *entry;
- unsigned long iov_pfn, end_iov_pfn;
+ struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
+ struct gvt_dma *entry;
+ u64 iov_pfn = iova >> PAGE_SHIFT;
+ u64 end_iov_pfn = iov_pfn + length / PAGE_SIZE;
- iov_pfn = unmap->iova >> PAGE_SHIFT;
- end_iov_pfn = iov_pfn + unmap->size / PAGE_SIZE;
+ mutex_lock(&vgpu->cache_lock);
+ for (; iov_pfn < end_iov_pfn; iov_pfn++) {
+ entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
+ if (!entry)
+ continue;
- mutex_lock(&vgpu->cache_lock);
- for (; iov_pfn < end_iov_pfn; iov_pfn++) {
- entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
- if (!entry)
- continue;
-
- gvt_dma_unmap_page(vgpu, entry->gfn, entry->dma_addr,
- entry->size);
- __gvt_cache_remove_entry(vgpu, entry);
- }
- mutex_unlock(&vgpu->cache_lock);
+ gvt_dma_unmap_page(vgpu, entry->gfn, entry->dma_addr,
+ entry->size);
+ __gvt_cache_remove_entry(vgpu, entry);
}
-
- return NOTIFY_OK;
+ mutex_unlock(&vgpu->cache_lock);
}
static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu)
@@ -783,36 +774,20 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu)
static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
{
struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
- unsigned long events;
- int ret;
-
- vgpu->iommu_notifier.notifier_call = intel_vgpu_iommu_notifier;
- events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
- ret = vfio_register_notifier(vfio_dev, VFIO_IOMMU_NOTIFY, &events,
- &vgpu->iommu_notifier);
- if (ret != 0) {
- gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n",
- ret);
- goto out;
- }
-
- ret = -EEXIST;
if (vgpu->attached)
- goto undo_iommu;
+ return -EEXIST;
- ret = -ESRCH;
if (!vgpu->vfio_device.kvm ||
vgpu->vfio_device.kvm->mm != current->mm) {
gvt_vgpu_err("KVM is required to use Intel vGPU\n");
- goto undo_iommu;
+ return -ESRCH;
}
kvm_get_kvm(vgpu->vfio_device.kvm);
- ret = -EEXIST;
if (__kvmgt_vgpu_exist(vgpu))
- goto undo_iommu;
+ return -EEXIST;
vgpu->attached = true;
@@ -831,12 +806,6 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
atomic_set(&vgpu->released, 0);
return 0;
-
-undo_iommu:
- vfio_unregister_notifier(vfio_dev, VFIO_IOMMU_NOTIFY,
- &vgpu->iommu_notifier);
-out:
- return ret;
}
static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
@@ -853,8 +822,6 @@ static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
{
struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
- struct drm_i915_private *i915 = vgpu->gvt->gt->i915;
- int ret;
if (!vgpu->attached)
return;
@@ -864,11 +831,6 @@ static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
intel_gvt_release_vgpu(vgpu);
- ret = vfio_unregister_notifier(&vgpu->vfio_device, VFIO_IOMMU_NOTIFY,
- &vgpu->iommu_notifier);
- drm_WARN(&i915->drm, ret,
- "vfio_unregister_notifier for iommu failed: %d\n", ret);
-
debugfs_remove(debugfs_lookup(KVMGT_DEBUGFS_FILENAME, vgpu->debugfs));
kvm_page_track_unregister_notifier(vgpu->vfio_device.kvm,
@@ -1610,6 +1572,7 @@ static const struct vfio_device_ops intel_vgpu_dev_ops = {
.write = intel_vgpu_write,
.mmap = intel_vgpu_mmap,
.ioctl = intel_vgpu_ioctl,
+ .dma_unmap = intel_vgpu_dma_unmap,
};
static int intel_vgpu_probe(struct mdev_device *mdev)
diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c
index b49e2e9db2dc6f..09e0ce7b72324c 100644
--- a/drivers/s390/cio/vfio_ccw_ops.c
+++ b/drivers/s390/cio/vfio_ccw_ops.c
@@ -44,31 +44,19 @@ static int vfio_ccw_mdev_reset(struct vfio_ccw_private *private)
return ret;
}
-static int vfio_ccw_mdev_notifier(struct notifier_block *nb,
- unsigned long action,
- void *data)
+static void vfio_ccw_dma_unmap(struct vfio_device *vdev, u64 iova, u64 length)
{
struct vfio_ccw_private *private =
- container_of(nb, struct vfio_ccw_private, nb);
-
- /*
- * Vendor drivers MUST unpin pages in response to an
- * invalidation.
- */
- if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
- struct vfio_iommu_type1_dma_unmap *unmap = data;
-
- if (!cp_iova_pinned(&private->cp, unmap->iova))
- return NOTIFY_OK;
+ container_of(vdev, struct vfio_ccw_private, vdev);
- if (vfio_ccw_mdev_reset(private))
- return NOTIFY_BAD;
+ /* Drivers MUST unpin pages in response to an invalidation. */
+ if (!cp_iova_pinned(&private->cp, iova))
+ return;
- cp_free(&private->cp);
- return NOTIFY_OK;
- }
+ if (vfio_ccw_mdev_reset(private))
+ return;
- return NOTIFY_DONE;
+ cp_free(&private->cp);
}
static ssize_t name_show(struct mdev_type *mtype,
@@ -178,19 +166,11 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev)
{
struct vfio_ccw_private *private =
container_of(vdev, struct vfio_ccw_private, vdev);
- unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
int ret;
- private->nb.notifier_call = vfio_ccw_mdev_notifier;
-
- ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY,
- &events, &private->nb);
- if (ret)
- return ret;
-
ret = vfio_ccw_register_async_dev_regions(private);
if (ret)
- goto out_unregister;
+ return ret;
ret = vfio_ccw_register_schib_dev_regions(private);
if (ret)
@@ -204,7 +184,6 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev)
out_unregister:
vfio_ccw_unregister_dev_regions(private);
- vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
return ret;
}
@@ -222,7 +201,6 @@ static void vfio_ccw_mdev_close_device(struct vfio_device *vdev)
cp_free(&private->cp);
vfio_ccw_unregister_dev_regions(private);
- vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
}
static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private *private,
@@ -645,6 +623,7 @@ static const struct vfio_device_ops vfio_ccw_dev_ops = {
.write = vfio_ccw_mdev_write,
.ioctl = vfio_ccw_mdev_ioctl,
.request = vfio_ccw_mdev_request,
+ .dma_unmap = vfio_ccw_dma_unmap,
};
struct mdev_driver vfio_ccw_mdev_driver = {
diff --git a/drivers/s390/cio/vfio_ccw_private.h b/drivers/s390/cio/vfio_ccw_private.h
index 7272eb78861244..2627791c9006d4 100644
--- a/drivers/s390/cio/vfio_ccw_private.h
+++ b/drivers/s390/cio/vfio_ccw_private.h
@@ -98,7 +98,6 @@ struct vfio_ccw_private {
struct completion *completion;
atomic_t avail;
struct mdev_device *mdev;
- struct notifier_block nb;
struct ccw_io_region *io_region;
struct mutex io_mutex;
struct vfio_ccw_region *region;
diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
index a7d2a95796d360..bb1a1677c5c230 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -1226,34 +1226,14 @@ static int vfio_ap_mdev_set_kvm(struct ap_matrix_mdev *matrix_mdev,
return 0;
}
-/**
- * vfio_ap_mdev_iommu_notifier - IOMMU notifier callback
- *
- * @nb: The notifier block
- * @action: Action to be taken
- * @data: data associated with the request
- *
- * For an UNMAP request, unpin the guest IOVA (the NIB guest address we
- * pinned before). Other requests are ignored.
- *
- * Return: for an UNMAP request, NOFITY_OK; otherwise NOTIFY_DONE.
- */
-static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb,
- unsigned long action, void *data)
+static void vfio_ap_mdev_dma_unmap(struct vfio_device *vdev, u64 iova,
+ u64 length)
{
- struct ap_matrix_mdev *matrix_mdev;
-
- matrix_mdev = container_of(nb, struct ap_matrix_mdev, iommu_notifier);
-
- if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
- struct vfio_iommu_type1_dma_unmap *unmap = data;
- unsigned long g_pfn = unmap->iova >> PAGE_SHIFT;
-
- vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
- return NOTIFY_OK;
- }
+ struct ap_matrix_mdev *matrix_mdev =
+ container_of(vdev, struct ap_matrix_mdev, vdev);
+ unsigned long g_pfn = iova >> PAGE_SHIFT;
- return NOTIFY_DONE;
+ vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
}
/**
@@ -1380,27 +1360,11 @@ static int vfio_ap_mdev_open_device(struct vfio_device *vdev)
{
struct ap_matrix_mdev *matrix_mdev =
container_of(vdev, struct ap_matrix_mdev, vdev);
- unsigned long events;
- int ret;
if (!vdev->kvm)
return -EINVAL;
- ret = vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
- if (ret)
- return ret;
-
- matrix_mdev->iommu_notifier.notifier_call = vfio_ap_mdev_iommu_notifier;
- events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
- ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY, &events,
- &matrix_mdev->iommu_notifier);
- if (ret)
- goto err_kvm;
- return 0;
-
-err_kvm:
- vfio_ap_mdev_unset_kvm(matrix_mdev);
- return ret;
+ return vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
}
static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
@@ -1408,8 +1372,6 @@ static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
struct ap_matrix_mdev *matrix_mdev =
container_of(vdev, struct ap_matrix_mdev, vdev);
- vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY,
- &matrix_mdev->iommu_notifier);
vfio_ap_mdev_unset_kvm(matrix_mdev);
}
@@ -1461,6 +1423,7 @@ static const struct vfio_device_ops vfio_ap_matrix_dev_ops = {
.open_device = vfio_ap_mdev_open_device,
.close_device = vfio_ap_mdev_close_device,
.ioctl = vfio_ap_mdev_ioctl,
+ .dma_unmap = vfio_ap_mdev_dma_unmap,
};
static struct mdev_driver vfio_ap_matrix_driver = {
diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h
index a26efd804d0df3..abb59d59f81b20 100644
--- a/drivers/s390/crypto/vfio_ap_private.h
+++ b/drivers/s390/crypto/vfio_ap_private.h
@@ -81,8 +81,6 @@ struct ap_matrix {
* @node: allows the ap_matrix_mdev struct to be added to a list
* @matrix: the adapters, usage domains and control domains assigned to the
* mediated matrix device.
- * @iommu_notifier: notifier block used for specifying callback function for
- * handling the VFIO_IOMMU_NOTIFY_DMA_UNMAP even
* @kvm: the struct holding guest's state
* @pqap_hook: the function pointer to the interception handler for the
* PQAP(AQIC) instruction.
@@ -92,7 +90,6 @@ struct ap_matrix_mdev {
struct vfio_device vdev;
struct list_head node;
struct ap_matrix matrix;
- struct notifier_block iommu_notifier;
struct kvm *kvm;
crypto_hook pqap_hook;
struct mdev_device *mdev;
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 61e71c1154be67..f005b644ab9e69 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct vfio_device *device)
up_write(&device->group->group_rwsem);
}
+static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
+ void *data)
+{
+ struct vfio_device *vfio_device =
+ container_of(nb, struct vfio_device, iommu_nb);
+ struct vfio_iommu_type1_dma_unmap *unmap = data;
+
+ vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
+ return NOTIFY_OK;
+}
+
static struct file *vfio_device_open(struct vfio_device *device)
{
+ struct vfio_iommu_driver *iommu_driver;
struct file *filep;
int ret;
@@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct vfio_device *device)
if (ret)
goto err_undo_count;
}
+
+ iommu_driver = device->group->container->iommu_driver;
+ if (device->ops->dma_unmap && iommu_driver &&
+ iommu_driver->ops->register_notifier) {
+ unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
+
+ device->iommu_nb.notifier_call = vfio_iommu_notifier;
+ iommu_driver->ops->register_notifier(
+ device->group->container->iommu_data, &events,
+ &device->iommu_nb);
+ }
+
up_read(&device->group->group_rwsem);
}
mutex_unlock(&device->dev_set->lock);
@@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct vfio_device *device)
err_close_device:
mutex_lock(&device->dev_set->lock);
down_read(&device->group->group_rwsem);
- if (device->open_count == 1 && device->ops->close_device)
+ if (device->open_count == 1 && device->ops->close_device) {
device->ops->close_device(device);
+
+ iommu_driver = device->group->container->iommu_driver;
+ if (device->ops->dma_unmap && iommu_driver &&
+ iommu_driver->ops->register_notifier)
+ iommu_driver->ops->unregister_notifier(
+ device->group->container->iommu_data,
+ &device->iommu_nb);
+ }
err_undo_count:
device->open_count--;
if (device->open_count == 0 && device->kvm)
@@ -1339,12 +1371,20 @@ static const struct file_operations vfio_group_fops = {
static int vfio_device_fops_release(struct inode *inode, struct file *filep)
{
struct vfio_device *device = filep->private_data;
+ struct vfio_iommu_driver *iommu_driver;
mutex_lock(&device->dev_set->lock);
vfio_assert_device_open(device);
down_read(&device->group->group_rwsem);
if (device->open_count == 1 && device->ops->close_device)
device->ops->close_device(device);
+
+ iommu_driver = device->group->container->iommu_driver;
+ if (device->ops->dma_unmap && iommu_driver &&
+ iommu_driver->ops->register_notifier)
+ iommu_driver->ops->unregister_notifier(
+ device->group->container->iommu_data,
+ &device->iommu_nb);
up_read(&device->group->group_rwsem);
device->open_count--;
if (device->open_count == 0)
@@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data,
}
EXPORT_SYMBOL(vfio_dma_rw);
-static int vfio_register_iommu_notifier(struct vfio_group *group,
- unsigned long *events,
- struct notifier_block *nb)
-{
- struct vfio_container *container;
- struct vfio_iommu_driver *driver;
- int ret;
-
- lockdep_assert_held_read(&group->group_rwsem);
-
- container = group->container;
- driver = container->iommu_driver;
- if (likely(driver && driver->ops->register_notifier))
- ret = driver->ops->register_notifier(container->iommu_data,
- events, nb);
- else
- ret = -ENOTTY;
-
- return ret;
-}
-
-static int vfio_unregister_iommu_notifier(struct vfio_group *group,
- struct notifier_block *nb)
-{
- struct vfio_container *container;
- struct vfio_iommu_driver *driver;
- int ret;
-
- lockdep_assert_held_read(&group->group_rwsem);
-
- container = group->container;
- driver = container->iommu_driver;
- if (likely(driver && driver->ops->unregister_notifier))
- ret = driver->ops->unregister_notifier(container->iommu_data,
- nb);
- else
- ret = -ENOTTY;
-
- return ret;
-}
-
-int vfio_register_notifier(struct vfio_device *device,
- enum vfio_notify_type type, unsigned long *events,
- struct notifier_block *nb)
-{
- struct vfio_group *group = device->group;
- int ret;
-
- if (!nb || !events || (*events == 0) ||
- !vfio_assert_device_open(device))
- return -EINVAL;
-
- switch (type) {
- case VFIO_IOMMU_NOTIFY:
- ret = vfio_register_iommu_notifier(group, events, nb);
- break;
- default:
- ret = -EINVAL;
- }
- return ret;
-}
-EXPORT_SYMBOL(vfio_register_notifier);
-
-int vfio_unregister_notifier(struct vfio_device *device,
- enum vfio_notify_type type,
- struct notifier_block *nb)
-{
- struct vfio_group *group = device->group;
- int ret;
-
- if (!nb || !vfio_assert_device_open(device))
- return -EINVAL;
-
- switch (type) {
- case VFIO_IOMMU_NOTIFY:
- ret = vfio_unregister_iommu_notifier(group, nb);
- break;
- default:
- ret = -EINVAL;
- }
- return ret;
-}
-EXPORT_SYMBOL(vfio_unregister_notifier);
-
/*
* Module/class support
*/
diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
index a6713022115155..cb2e4e9baa8fe8 100644
--- a/drivers/vfio/vfio.h
+++ b/drivers/vfio/vfio.h
@@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
VFIO_IOMMU_CONTAINER_CLOSE = 0,
};
+/* events for register_notifier() */
+enum {
+ VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
+};
+
/**
* struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
*/
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index aa888cc517578e..b76623e3b92fca 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -44,6 +44,7 @@ struct vfio_device {
unsigned int open_count;
struct completion comp;
struct list_head group_next;
+ struct notifier_block iommu_nb;
};
/**
@@ -60,6 +61,8 @@ struct vfio_device {
* @match: Optional device name match callback (return: 0 for no-match, >0 for
* match, -errno for abort (ex. match with insufficient or incorrect
* additional args)
+ * @dma_unmap: Called when userspace unmaps IOVA from the container
+ * this device is attached to.
* @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
* @migration_set_state: Optional callback to change the migration state for
* devices that support migration. It's mandatory for
@@ -85,6 +88,7 @@ struct vfio_device_ops {
int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
void (*request)(struct vfio_device *vdev, unsigned int count);
int (*match)(struct vfio_device *vdev, char *buf);
+ void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
int (*device_feature)(struct vfio_device *device, u32 flags,
void __user *arg, size_t argsz);
struct file *(*migration_set_state)(
@@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn,
extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
void *data, size_t len, bool write);
-/* each type has independent events */
-enum vfio_notify_type {
- VFIO_IOMMU_NOTIFY = 0,
-};
-
-/* events for VFIO_IOMMU_NOTIFY */
-#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
-
-extern int vfio_register_notifier(struct vfio_device *device,
- enum vfio_notify_type type,
- unsigned long *required_events,
- struct notifier_block *nb);
-extern int vfio_unregister_notifier(struct vfio_device *device,
- enum vfio_notify_type type,
- struct notifier_block *nb);
-
-
/*
* Sub-module helpers
*/
--
2.36.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
@ 2022-06-07 23:02 ` Jason Gunthorpe
-1 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-07 23:02 UTC (permalink / raw)
To: Alexander Gordeev, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Christoph Hellwig
Instead of bouncing the function call to the driver op through a blocking
notifier just have the iommu layer call it directly.
Register each device that is being attached to the iommu with the lower
driver which then threads them on a linked list and calls the appropriate
driver op at the right time.
Currently the only use is if dma_unmap() is defined.
Also, fully lock all the debugging tests on the pinning path that a
dma_unmap is registered.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/vfio/vfio.c | 42 ++++---------
drivers/vfio/vfio.h | 14 ++---
drivers/vfio/vfio_iommu_type1.c | 103 ++++++++++++++++++++------------
include/linux/vfio.h | 2 +-
4 files changed, 83 insertions(+), 78 deletions(-)
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index f005b644ab9e69..065b57e601bff7 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -619,6 +619,9 @@ EXPORT_SYMBOL_GPL(vfio_register_group_dev);
*/
int vfio_register_emulated_iommu_dev(struct vfio_device *device)
{
+ if (WARN_ON(!device->ops->dma_unmap))
+ return -EINVAL;
+
return __vfio_register_dev(device,
vfio_noiommu_group_alloc(device->dev, VFIO_EMULATED_IOMMU));
}
@@ -1077,17 +1080,6 @@ static void vfio_device_unassign_container(struct vfio_device *device)
up_write(&device->group->group_rwsem);
}
-static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
- void *data)
-{
- struct vfio_device *vfio_device =
- container_of(nb, struct vfio_device, iommu_nb);
- struct vfio_iommu_type1_dma_unmap *unmap = data;
-
- vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
- return NOTIFY_OK;
-}
-
static struct file *vfio_device_open(struct vfio_device *device)
{
struct vfio_iommu_driver *iommu_driver;
@@ -1123,15 +1115,9 @@ static struct file *vfio_device_open(struct vfio_device *device)
}
iommu_driver = device->group->container->iommu_driver;
- if (device->ops->dma_unmap && iommu_driver &&
- iommu_driver->ops->register_notifier) {
- unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
-
- device->iommu_nb.notifier_call = vfio_iommu_notifier;
- iommu_driver->ops->register_notifier(
- device->group->container->iommu_data, &events,
- &device->iommu_nb);
- }
+ if (iommu_driver && iommu_driver->ops->register_device)
+ iommu_driver->ops->register_device(
+ device->group->container->iommu_data, device);
up_read(&device->group->group_rwsem);
}
@@ -1171,11 +1157,9 @@ static struct file *vfio_device_open(struct vfio_device *device)
device->ops->close_device(device);
iommu_driver = device->group->container->iommu_driver;
- if (device->ops->dma_unmap && iommu_driver &&
- iommu_driver->ops->register_notifier)
- iommu_driver->ops->unregister_notifier(
- device->group->container->iommu_data,
- &device->iommu_nb);
+ if (iommu_driver && iommu_driver->ops->register_device)
+ iommu_driver->ops->unregister_device(
+ device->group->container->iommu_data, device);
}
err_undo_count:
device->open_count--;
@@ -1380,11 +1364,9 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep)
device->ops->close_device(device);
iommu_driver = device->group->container->iommu_driver;
- if (device->ops->dma_unmap && iommu_driver &&
- iommu_driver->ops->register_notifier)
- iommu_driver->ops->unregister_notifier(
- device->group->container->iommu_data,
- &device->iommu_nb);
+ if (iommu_driver && iommu_driver->ops->unregister_device)
+ iommu_driver->ops->unregister_device(
+ device->group->container->iommu_data, device);
up_read(&device->group->group_rwsem);
device->open_count--;
if (device->open_count == 0)
diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
index cb2e4e9baa8fe8..4a7db1f3c33e7e 100644
--- a/drivers/vfio/vfio.h
+++ b/drivers/vfio/vfio.h
@@ -33,11 +33,6 @@ enum vfio_iommu_notify_type {
VFIO_IOMMU_CONTAINER_CLOSE = 0,
};
-/* events for register_notifier() */
-enum {
- VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
-};
-
/**
* struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
*/
@@ -60,11 +55,10 @@ struct vfio_iommu_driver_ops {
unsigned long *phys_pfn);
int (*unpin_pages)(void *iommu_data,
unsigned long *user_pfn, int npage);
- int (*register_notifier)(void *iommu_data,
- unsigned long *events,
- struct notifier_block *nb);
- int (*unregister_notifier)(void *iommu_data,
- struct notifier_block *nb);
+ void (*register_device)(void *iommu_data,
+ struct vfio_device *vdev);
+ void (*unregister_device)(void *iommu_data,
+ struct vfio_device *vdev);
int (*dma_rw)(void *iommu_data, dma_addr_t user_iova,
void *data, size_t count, bool write);
struct iommu_domain *(*group_iommu_domain)(void *iommu_data,
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index c13b9290e35759..4ddb1f1abd238b 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -67,7 +67,8 @@ struct vfio_iommu {
struct list_head iova_list;
struct mutex lock;
struct rb_root dma_list;
- struct blocking_notifier_head notifier;
+ struct list_head device_list;
+ struct mutex device_list_lock;
unsigned int dma_avail;
unsigned int vaddr_invalid_count;
uint64_t pgsize_bitmap;
@@ -865,8 +866,8 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
}
}
- /* Fail if notifier list is empty */
- if (!iommu->notifier.head) {
+ /* Fail if no dma_umap notifier is registered */
+ if (list_empty(&iommu->device_list)) {
ret = -EINVAL;
goto pin_done;
}
@@ -1287,6 +1288,35 @@ static int verify_bitmap_size(uint64_t npages, uint64_t bitmap_size)
return 0;
}
+/*
+ * Notify VFIO drivers using vfio_register_emulated_iommu_dev() to invalidate
+ * and unmap iovas within the range we're about to unmap. Drivers MUST unpin
+ * pages in response to an invalidation.
+ */
+static void vfio_notify_dma_unmap(struct vfio_iommu *iommu,
+ struct vfio_dma *dma)
+{
+ struct vfio_device *device;
+
+ if (list_empty(&iommu->device_list))
+ return;
+
+ /*
+ * The device is expected to call vfio_unpin_pages() for any IOVA it has
+ * pinned within the range. Since vfio_unpin_pages() will eventually
+ * call back down to this code and try to obtain the iommu->lock we must
+ * drop it.
+ */
+ mutex_lock(&iommu->device_list_lock);
+ mutex_unlock(&iommu->lock);
+
+ list_for_each_entry(device, &iommu->device_list, iommu_entry)
+ device->ops->dma_unmap(device, dma->iova, dma->size);
+
+ mutex_unlock(&iommu->device_list_lock);
+ mutex_lock(&iommu->lock);
+}
+
static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
struct vfio_iommu_type1_dma_unmap *unmap,
struct vfio_bitmap *bitmap)
@@ -1406,8 +1436,6 @@ static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
}
if (!RB_EMPTY_ROOT(&dma->pfn_list)) {
- struct vfio_iommu_type1_dma_unmap nb_unmap;
-
if (dma_last == dma) {
BUG_ON(++retries > 10);
} else {
@@ -1415,20 +1443,7 @@ static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
retries = 0;
}
- nb_unmap.iova = dma->iova;
- nb_unmap.size = dma->size;
-
- /*
- * Notify anyone (mdev vendor drivers) to invalidate and
- * unmap iovas within the range we're about to unmap.
- * Vendor drivers MUST unpin pages in response to an
- * invalidation.
- */
- mutex_unlock(&iommu->lock);
- blocking_notifier_call_chain(&iommu->notifier,
- VFIO_IOMMU_NOTIFY_DMA_UNMAP,
- &nb_unmap);
- mutex_lock(&iommu->lock);
+ vfio_notify_dma_unmap(iommu, dma);
goto again;
}
@@ -2478,7 +2493,7 @@ static void vfio_iommu_type1_detach_group(void *iommu_data,
if (list_empty(&iommu->emulated_iommu_groups) &&
list_empty(&iommu->domain_list)) {
- WARN_ON(iommu->notifier.head);
+ WARN_ON(!list_empty(&iommu->device_list));
vfio_iommu_unmap_unpin_all(iommu);
}
goto detach_group_done;
@@ -2510,7 +2525,8 @@ static void vfio_iommu_type1_detach_group(void *iommu_data,
if (list_empty(&domain->group_list)) {
if (list_is_singular(&iommu->domain_list)) {
if (list_empty(&iommu->emulated_iommu_groups)) {
- WARN_ON(iommu->notifier.head);
+ WARN_ON(!list_empty(
+ &iommu->device_list));
vfio_iommu_unmap_unpin_all(iommu);
} else {
vfio_iommu_unmap_unpin_reaccount(iommu);
@@ -2571,7 +2587,8 @@ static void *vfio_iommu_type1_open(unsigned long arg)
iommu->dma_avail = dma_entry_limit;
iommu->container_open = true;
mutex_init(&iommu->lock);
- BLOCKING_INIT_NOTIFIER_HEAD(&iommu->notifier);
+ mutex_init(&iommu->device_list_lock);
+ INIT_LIST_HEAD(&iommu->device_list);
init_waitqueue_head(&iommu->vaddr_wait);
iommu->pgsize_bitmap = PAGE_MASK;
INIT_LIST_HEAD(&iommu->emulated_iommu_groups);
@@ -3008,28 +3025,40 @@ static long vfio_iommu_type1_ioctl(void *iommu_data,
}
}
-static int vfio_iommu_type1_register_notifier(void *iommu_data,
- unsigned long *events,
- struct notifier_block *nb)
+static void vfio_iommu_type1_register_device(void *iommu_data,
+ struct vfio_device *vdev)
{
struct vfio_iommu *iommu = iommu_data;
- /* clear known events */
- *events &= ~VFIO_IOMMU_NOTIFY_DMA_UNMAP;
-
- /* refuse to register if still events remaining */
- if (*events)
- return -EINVAL;
+ if (!vdev->ops->dma_unmap)
+ return;
- return blocking_notifier_chain_register(&iommu->notifier, nb);
+ /*
+ * list_empty(&iommu->device_list) is tested under the iommu->lock while
+ * iteration for dma_unmap must be done under the device_list_lock.
+ * Holding both locks here allows avoiding the device_list_lock in
+ * several fast paths. See vfio_notify_dma_unmap()
+ */
+ mutex_lock(&iommu->lock);
+ mutex_lock(&iommu->device_list_lock);
+ list_add(&vdev->iommu_entry, &iommu->device_list);
+ mutex_unlock(&iommu->device_list_lock);
+ mutex_unlock(&iommu->lock);
}
-static int vfio_iommu_type1_unregister_notifier(void *iommu_data,
- struct notifier_block *nb)
+static void vfio_iommu_type1_unregister_device(void *iommu_data,
+ struct vfio_device *vdev)
{
struct vfio_iommu *iommu = iommu_data;
- return blocking_notifier_chain_unregister(&iommu->notifier, nb);
+ if (!vdev->ops->dma_unmap)
+ return;
+
+ mutex_lock(&iommu->lock);
+ mutex_lock(&iommu->device_list_lock);
+ list_del(&vdev->iommu_entry);
+ mutex_unlock(&iommu->device_list_lock);
+ mutex_unlock(&iommu->lock);
}
static int vfio_iommu_type1_dma_rw_chunk(struct vfio_iommu *iommu,
@@ -3163,8 +3192,8 @@ static const struct vfio_iommu_driver_ops vfio_iommu_driver_ops_type1 = {
.detach_group = vfio_iommu_type1_detach_group,
.pin_pages = vfio_iommu_type1_pin_pages,
.unpin_pages = vfio_iommu_type1_unpin_pages,
- .register_notifier = vfio_iommu_type1_register_notifier,
- .unregister_notifier = vfio_iommu_type1_unregister_notifier,
+ .register_device = vfio_iommu_type1_register_device,
+ .unregister_device = vfio_iommu_type1_unregister_device,
.dma_rw = vfio_iommu_type1_dma_rw,
.group_iommu_domain = vfio_iommu_type1_group_iommu_domain,
.notify = vfio_iommu_type1_notify,
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index b76623e3b92fca..c22d3f1e13b66c 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -44,7 +44,7 @@ struct vfio_device {
unsigned int open_count;
struct completion comp;
struct list_head group_next;
- struct notifier_block iommu_nb;
+ struct list_head iommu_entry;
};
/**
--
2.36.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Intel-gfx] [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
@ 2022-06-07 23:02 ` Jason Gunthorpe
0 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-07 23:02 UTC (permalink / raw)
To: Alexander Gordeev, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Christoph Hellwig
Instead of bouncing the function call to the driver op through a blocking
notifier just have the iommu layer call it directly.
Register each device that is being attached to the iommu with the lower
driver which then threads them on a linked list and calls the appropriate
driver op at the right time.
Currently the only use is if dma_unmap() is defined.
Also, fully lock all the debugging tests on the pinning path that a
dma_unmap is registered.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/vfio/vfio.c | 42 ++++---------
drivers/vfio/vfio.h | 14 ++---
drivers/vfio/vfio_iommu_type1.c | 103 ++++++++++++++++++++------------
include/linux/vfio.h | 2 +-
4 files changed, 83 insertions(+), 78 deletions(-)
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index f005b644ab9e69..065b57e601bff7 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -619,6 +619,9 @@ EXPORT_SYMBOL_GPL(vfio_register_group_dev);
*/
int vfio_register_emulated_iommu_dev(struct vfio_device *device)
{
+ if (WARN_ON(!device->ops->dma_unmap))
+ return -EINVAL;
+
return __vfio_register_dev(device,
vfio_noiommu_group_alloc(device->dev, VFIO_EMULATED_IOMMU));
}
@@ -1077,17 +1080,6 @@ static void vfio_device_unassign_container(struct vfio_device *device)
up_write(&device->group->group_rwsem);
}
-static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
- void *data)
-{
- struct vfio_device *vfio_device =
- container_of(nb, struct vfio_device, iommu_nb);
- struct vfio_iommu_type1_dma_unmap *unmap = data;
-
- vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
- return NOTIFY_OK;
-}
-
static struct file *vfio_device_open(struct vfio_device *device)
{
struct vfio_iommu_driver *iommu_driver;
@@ -1123,15 +1115,9 @@ static struct file *vfio_device_open(struct vfio_device *device)
}
iommu_driver = device->group->container->iommu_driver;
- if (device->ops->dma_unmap && iommu_driver &&
- iommu_driver->ops->register_notifier) {
- unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
-
- device->iommu_nb.notifier_call = vfio_iommu_notifier;
- iommu_driver->ops->register_notifier(
- device->group->container->iommu_data, &events,
- &device->iommu_nb);
- }
+ if (iommu_driver && iommu_driver->ops->register_device)
+ iommu_driver->ops->register_device(
+ device->group->container->iommu_data, device);
up_read(&device->group->group_rwsem);
}
@@ -1171,11 +1157,9 @@ static struct file *vfio_device_open(struct vfio_device *device)
device->ops->close_device(device);
iommu_driver = device->group->container->iommu_driver;
- if (device->ops->dma_unmap && iommu_driver &&
- iommu_driver->ops->register_notifier)
- iommu_driver->ops->unregister_notifier(
- device->group->container->iommu_data,
- &device->iommu_nb);
+ if (iommu_driver && iommu_driver->ops->register_device)
+ iommu_driver->ops->unregister_device(
+ device->group->container->iommu_data, device);
}
err_undo_count:
device->open_count--;
@@ -1380,11 +1364,9 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep)
device->ops->close_device(device);
iommu_driver = device->group->container->iommu_driver;
- if (device->ops->dma_unmap && iommu_driver &&
- iommu_driver->ops->register_notifier)
- iommu_driver->ops->unregister_notifier(
- device->group->container->iommu_data,
- &device->iommu_nb);
+ if (iommu_driver && iommu_driver->ops->unregister_device)
+ iommu_driver->ops->unregister_device(
+ device->group->container->iommu_data, device);
up_read(&device->group->group_rwsem);
device->open_count--;
if (device->open_count == 0)
diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
index cb2e4e9baa8fe8..4a7db1f3c33e7e 100644
--- a/drivers/vfio/vfio.h
+++ b/drivers/vfio/vfio.h
@@ -33,11 +33,6 @@ enum vfio_iommu_notify_type {
VFIO_IOMMU_CONTAINER_CLOSE = 0,
};
-/* events for register_notifier() */
-enum {
- VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
-};
-
/**
* struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
*/
@@ -60,11 +55,10 @@ struct vfio_iommu_driver_ops {
unsigned long *phys_pfn);
int (*unpin_pages)(void *iommu_data,
unsigned long *user_pfn, int npage);
- int (*register_notifier)(void *iommu_data,
- unsigned long *events,
- struct notifier_block *nb);
- int (*unregister_notifier)(void *iommu_data,
- struct notifier_block *nb);
+ void (*register_device)(void *iommu_data,
+ struct vfio_device *vdev);
+ void (*unregister_device)(void *iommu_data,
+ struct vfio_device *vdev);
int (*dma_rw)(void *iommu_data, dma_addr_t user_iova,
void *data, size_t count, bool write);
struct iommu_domain *(*group_iommu_domain)(void *iommu_data,
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index c13b9290e35759..4ddb1f1abd238b 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -67,7 +67,8 @@ struct vfio_iommu {
struct list_head iova_list;
struct mutex lock;
struct rb_root dma_list;
- struct blocking_notifier_head notifier;
+ struct list_head device_list;
+ struct mutex device_list_lock;
unsigned int dma_avail;
unsigned int vaddr_invalid_count;
uint64_t pgsize_bitmap;
@@ -865,8 +866,8 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
}
}
- /* Fail if notifier list is empty */
- if (!iommu->notifier.head) {
+ /* Fail if no dma_umap notifier is registered */
+ if (list_empty(&iommu->device_list)) {
ret = -EINVAL;
goto pin_done;
}
@@ -1287,6 +1288,35 @@ static int verify_bitmap_size(uint64_t npages, uint64_t bitmap_size)
return 0;
}
+/*
+ * Notify VFIO drivers using vfio_register_emulated_iommu_dev() to invalidate
+ * and unmap iovas within the range we're about to unmap. Drivers MUST unpin
+ * pages in response to an invalidation.
+ */
+static void vfio_notify_dma_unmap(struct vfio_iommu *iommu,
+ struct vfio_dma *dma)
+{
+ struct vfio_device *device;
+
+ if (list_empty(&iommu->device_list))
+ return;
+
+ /*
+ * The device is expected to call vfio_unpin_pages() for any IOVA it has
+ * pinned within the range. Since vfio_unpin_pages() will eventually
+ * call back down to this code and try to obtain the iommu->lock we must
+ * drop it.
+ */
+ mutex_lock(&iommu->device_list_lock);
+ mutex_unlock(&iommu->lock);
+
+ list_for_each_entry(device, &iommu->device_list, iommu_entry)
+ device->ops->dma_unmap(device, dma->iova, dma->size);
+
+ mutex_unlock(&iommu->device_list_lock);
+ mutex_lock(&iommu->lock);
+}
+
static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
struct vfio_iommu_type1_dma_unmap *unmap,
struct vfio_bitmap *bitmap)
@@ -1406,8 +1436,6 @@ static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
}
if (!RB_EMPTY_ROOT(&dma->pfn_list)) {
- struct vfio_iommu_type1_dma_unmap nb_unmap;
-
if (dma_last == dma) {
BUG_ON(++retries > 10);
} else {
@@ -1415,20 +1443,7 @@ static int vfio_dma_do_unmap(struct vfio_iommu *iommu,
retries = 0;
}
- nb_unmap.iova = dma->iova;
- nb_unmap.size = dma->size;
-
- /*
- * Notify anyone (mdev vendor drivers) to invalidate and
- * unmap iovas within the range we're about to unmap.
- * Vendor drivers MUST unpin pages in response to an
- * invalidation.
- */
- mutex_unlock(&iommu->lock);
- blocking_notifier_call_chain(&iommu->notifier,
- VFIO_IOMMU_NOTIFY_DMA_UNMAP,
- &nb_unmap);
- mutex_lock(&iommu->lock);
+ vfio_notify_dma_unmap(iommu, dma);
goto again;
}
@@ -2478,7 +2493,7 @@ static void vfio_iommu_type1_detach_group(void *iommu_data,
if (list_empty(&iommu->emulated_iommu_groups) &&
list_empty(&iommu->domain_list)) {
- WARN_ON(iommu->notifier.head);
+ WARN_ON(!list_empty(&iommu->device_list));
vfio_iommu_unmap_unpin_all(iommu);
}
goto detach_group_done;
@@ -2510,7 +2525,8 @@ static void vfio_iommu_type1_detach_group(void *iommu_data,
if (list_empty(&domain->group_list)) {
if (list_is_singular(&iommu->domain_list)) {
if (list_empty(&iommu->emulated_iommu_groups)) {
- WARN_ON(iommu->notifier.head);
+ WARN_ON(!list_empty(
+ &iommu->device_list));
vfio_iommu_unmap_unpin_all(iommu);
} else {
vfio_iommu_unmap_unpin_reaccount(iommu);
@@ -2571,7 +2587,8 @@ static void *vfio_iommu_type1_open(unsigned long arg)
iommu->dma_avail = dma_entry_limit;
iommu->container_open = true;
mutex_init(&iommu->lock);
- BLOCKING_INIT_NOTIFIER_HEAD(&iommu->notifier);
+ mutex_init(&iommu->device_list_lock);
+ INIT_LIST_HEAD(&iommu->device_list);
init_waitqueue_head(&iommu->vaddr_wait);
iommu->pgsize_bitmap = PAGE_MASK;
INIT_LIST_HEAD(&iommu->emulated_iommu_groups);
@@ -3008,28 +3025,40 @@ static long vfio_iommu_type1_ioctl(void *iommu_data,
}
}
-static int vfio_iommu_type1_register_notifier(void *iommu_data,
- unsigned long *events,
- struct notifier_block *nb)
+static void vfio_iommu_type1_register_device(void *iommu_data,
+ struct vfio_device *vdev)
{
struct vfio_iommu *iommu = iommu_data;
- /* clear known events */
- *events &= ~VFIO_IOMMU_NOTIFY_DMA_UNMAP;
-
- /* refuse to register if still events remaining */
- if (*events)
- return -EINVAL;
+ if (!vdev->ops->dma_unmap)
+ return;
- return blocking_notifier_chain_register(&iommu->notifier, nb);
+ /*
+ * list_empty(&iommu->device_list) is tested under the iommu->lock while
+ * iteration for dma_unmap must be done under the device_list_lock.
+ * Holding both locks here allows avoiding the device_list_lock in
+ * several fast paths. See vfio_notify_dma_unmap()
+ */
+ mutex_lock(&iommu->lock);
+ mutex_lock(&iommu->device_list_lock);
+ list_add(&vdev->iommu_entry, &iommu->device_list);
+ mutex_unlock(&iommu->device_list_lock);
+ mutex_unlock(&iommu->lock);
}
-static int vfio_iommu_type1_unregister_notifier(void *iommu_data,
- struct notifier_block *nb)
+static void vfio_iommu_type1_unregister_device(void *iommu_data,
+ struct vfio_device *vdev)
{
struct vfio_iommu *iommu = iommu_data;
- return blocking_notifier_chain_unregister(&iommu->notifier, nb);
+ if (!vdev->ops->dma_unmap)
+ return;
+
+ mutex_lock(&iommu->lock);
+ mutex_lock(&iommu->device_list_lock);
+ list_del(&vdev->iommu_entry);
+ mutex_unlock(&iommu->device_list_lock);
+ mutex_unlock(&iommu->lock);
}
static int vfio_iommu_type1_dma_rw_chunk(struct vfio_iommu *iommu,
@@ -3163,8 +3192,8 @@ static const struct vfio_iommu_driver_ops vfio_iommu_driver_ops_type1 = {
.detach_group = vfio_iommu_type1_detach_group,
.pin_pages = vfio_iommu_type1_pin_pages,
.unpin_pages = vfio_iommu_type1_unpin_pages,
- .register_notifier = vfio_iommu_type1_register_notifier,
- .unregister_notifier = vfio_iommu_type1_unregister_notifier,
+ .register_device = vfio_iommu_type1_register_device,
+ .unregister_device = vfio_iommu_type1_unregister_device,
.dma_rw = vfio_iommu_type1_dma_rw,
.group_iommu_domain = vfio_iommu_type1_group_iommu_domain,
.notify = vfio_iommu_type1_notify,
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index b76623e3b92fca..c22d3f1e13b66c 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -44,7 +44,7 @@ struct vfio_device {
unsigned int open_count;
struct completion comp;
struct list_head group_next;
- struct notifier_block iommu_nb;
+ struct list_head iommu_entry;
};
/**
--
2.36.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* RE: [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
@ 2022-06-08 3:37 ` Tian, Kevin
-1 siblings, 0 replies; 34+ messages in thread
From: Tian, Kevin @ 2022-06-08 3:37 UTC (permalink / raw)
To: Jason Gunthorpe, Alexander Gordeev, David Airlie, Tony Krowiak,
Alex Williamson, Christian Borntraeger, Cornelia Huck,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Jason Herne, Joonas Lahtinen, kvm, linux-s390,
Matthew Rosato, Peter Oberparleiter, Halil Pasic, Vivi, Rodrigo,
Sven Schnelle, Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang,
Wang, Zhi A
Cc: Christoph Hellwig
> From: Jason Gunthorpe
> Sent: Wednesday, June 8, 2022 7:02 AM
>
> Instead of having drivers register the notifier with explicit code just
> have them provide a dma_unmap callback op in their driver ops and rely on
> the core code to wire it up.
>
> Suggested-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> ---
> drivers/gpu/drm/i915/gvt/gvt.h | 1 -
> drivers/gpu/drm/i915/gvt/kvmgt.c | 75 ++++-----------
> drivers/s390/cio/vfio_ccw_ops.c | 41 ++-------
> drivers/s390/cio/vfio_ccw_private.h | 1 -
> drivers/s390/crypto/vfio_ap_ops.c | 53 ++---------
> drivers/s390/crypto/vfio_ap_private.h | 3 -
> drivers/vfio/vfio.c | 126 +++++++++-----------------
> drivers/vfio/vfio.h | 5 +
> include/linux/vfio.h | 21 +----
> 9 files changed, 87 insertions(+), 239 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
> index aee1a45da74bcb..705689e6401197 100644
> --- a/drivers/gpu/drm/i915/gvt/gvt.h
> +++ b/drivers/gpu/drm/i915/gvt/gvt.h
> @@ -226,7 +226,6 @@ struct intel_vgpu {
> unsigned long nr_cache_entries;
> struct mutex cache_lock;
>
> - struct notifier_block iommu_notifier;
> atomic_t released;
>
> struct kvm_page_track_notifier_node track_node;
> diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c
> b/drivers/gpu/drm/i915/gvt/kvmgt.c
> index e2f6c56ab3420c..ecd5bb37b63a2a 100644
> --- a/drivers/gpu/drm/i915/gvt/kvmgt.c
> +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
> @@ -729,34 +729,25 @@ int intel_gvt_set_edid(struct intel_vgpu *vgpu, int
> port_num)
> return ret;
> }
>
> -static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
> - unsigned long action, void *data)
> +static void intel_vgpu_dma_unmap(struct vfio_device *vfio_dev, u64 iova,
> + u64 length)
> {
> - struct intel_vgpu *vgpu =
> - container_of(nb, struct intel_vgpu, iommu_notifier);
> -
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> - struct gvt_dma *entry;
> - unsigned long iov_pfn, end_iov_pfn;
> + struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> + struct gvt_dma *entry;
> + u64 iov_pfn = iova >> PAGE_SHIFT;
> + u64 end_iov_pfn = iov_pfn + length / PAGE_SIZE;
>
> - iov_pfn = unmap->iova >> PAGE_SHIFT;
> - end_iov_pfn = iov_pfn + unmap->size / PAGE_SIZE;
> + mutex_lock(&vgpu->cache_lock);
> + for (; iov_pfn < end_iov_pfn; iov_pfn++) {
> + entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
> + if (!entry)
> + continue;
>
> - mutex_lock(&vgpu->cache_lock);
> - for (; iov_pfn < end_iov_pfn; iov_pfn++) {
> - entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
> - if (!entry)
> - continue;
> -
> - gvt_dma_unmap_page(vgpu, entry->gfn, entry-
> >dma_addr,
> - entry->size);
> - __gvt_cache_remove_entry(vgpu, entry);
> - }
> - mutex_unlock(&vgpu->cache_lock);
> + gvt_dma_unmap_page(vgpu, entry->gfn, entry->dma_addr,
> + entry->size);
> + __gvt_cache_remove_entry(vgpu, entry);
> }
> -
> - return NOTIFY_OK;
> + mutex_unlock(&vgpu->cache_lock);
> }
>
> static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu)
> @@ -783,36 +774,20 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu
> *vgpu)
> static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
> {
> struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> - unsigned long events;
> - int ret;
> -
> - vgpu->iommu_notifier.notifier_call = intel_vgpu_iommu_notifier;
>
> - events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> - ret = vfio_register_notifier(vfio_dev, VFIO_IOMMU_NOTIFY, &events,
> - &vgpu->iommu_notifier);
> - if (ret != 0) {
> - gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n",
> - ret);
> - goto out;
> - }
> -
> - ret = -EEXIST;
> if (vgpu->attached)
> - goto undo_iommu;
> + return -EEXIST;
>
> - ret = -ESRCH;
> if (!vgpu->vfio_device.kvm ||
> vgpu->vfio_device.kvm->mm != current->mm) {
> gvt_vgpu_err("KVM is required to use Intel vGPU\n");
> - goto undo_iommu;
> + return -ESRCH;
> }
>
> kvm_get_kvm(vgpu->vfio_device.kvm);
>
> - ret = -EEXIST;
> if (__kvmgt_vgpu_exist(vgpu))
> - goto undo_iommu;
> + return -EEXIST;
>
> vgpu->attached = true;
>
> @@ -831,12 +806,6 @@ static int intel_vgpu_open_device(struct vfio_device
> *vfio_dev)
>
> atomic_set(&vgpu->released, 0);
> return 0;
> -
> -undo_iommu:
> - vfio_unregister_notifier(vfio_dev, VFIO_IOMMU_NOTIFY,
> - &vgpu->iommu_notifier);
> -out:
> - return ret;
> }
>
> static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
> @@ -853,8 +822,6 @@ static void
> intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
> static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
> {
> struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> - struct drm_i915_private *i915 = vgpu->gvt->gt->i915;
> - int ret;
>
> if (!vgpu->attached)
> return;
> @@ -864,11 +831,6 @@ static void intel_vgpu_close_device(struct
> vfio_device *vfio_dev)
>
> intel_gvt_release_vgpu(vgpu);
>
> - ret = vfio_unregister_notifier(&vgpu->vfio_device,
> VFIO_IOMMU_NOTIFY,
> - &vgpu->iommu_notifier);
> - drm_WARN(&i915->drm, ret,
> - "vfio_unregister_notifier for iommu failed: %d\n", ret);
> -
> debugfs_remove(debugfs_lookup(KVMGT_DEBUGFS_FILENAME,
> vgpu->debugfs));
>
> kvm_page_track_unregister_notifier(vgpu->vfio_device.kvm,
> @@ -1610,6 +1572,7 @@ static const struct vfio_device_ops
> intel_vgpu_dev_ops = {
> .write = intel_vgpu_write,
> .mmap = intel_vgpu_mmap,
> .ioctl = intel_vgpu_ioctl,
> + .dma_unmap = intel_vgpu_dma_unmap,
> };
>
> static int intel_vgpu_probe(struct mdev_device *mdev)
> diff --git a/drivers/s390/cio/vfio_ccw_ops.c
> b/drivers/s390/cio/vfio_ccw_ops.c
> index b49e2e9db2dc6f..09e0ce7b72324c 100644
> --- a/drivers/s390/cio/vfio_ccw_ops.c
> +++ b/drivers/s390/cio/vfio_ccw_ops.c
> @@ -44,31 +44,19 @@ static int vfio_ccw_mdev_reset(struct
> vfio_ccw_private *private)
> return ret;
> }
>
> -static int vfio_ccw_mdev_notifier(struct notifier_block *nb,
> - unsigned long action,
> - void *data)
> +static void vfio_ccw_dma_unmap(struct vfio_device *vdev, u64 iova, u64
> length)
> {
> struct vfio_ccw_private *private =
> - container_of(nb, struct vfio_ccw_private, nb);
> -
> - /*
> - * Vendor drivers MUST unpin pages in response to an
> - * invalidation.
> - */
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> -
> - if (!cp_iova_pinned(&private->cp, unmap->iova))
> - return NOTIFY_OK;
> + container_of(vdev, struct vfio_ccw_private, vdev);
>
> - if (vfio_ccw_mdev_reset(private))
> - return NOTIFY_BAD;
> + /* Drivers MUST unpin pages in response to an invalidation. */
> + if (!cp_iova_pinned(&private->cp, iova))
> + return;
>
> - cp_free(&private->cp);
> - return NOTIFY_OK;
> - }
> + if (vfio_ccw_mdev_reset(private))
> + return;
>
> - return NOTIFY_DONE;
> + cp_free(&private->cp);
> }
>
> static ssize_t name_show(struct mdev_type *mtype,
> @@ -178,19 +166,11 @@ static int vfio_ccw_mdev_open_device(struct
> vfio_device *vdev)
> {
> struct vfio_ccw_private *private =
> container_of(vdev, struct vfio_ccw_private, vdev);
> - unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> int ret;
>
> - private->nb.notifier_call = vfio_ccw_mdev_notifier;
> -
> - ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY,
> - &events, &private->nb);
> - if (ret)
> - return ret;
> -
> ret = vfio_ccw_register_async_dev_regions(private);
> if (ret)
> - goto out_unregister;
> + return ret;
>
> ret = vfio_ccw_register_schib_dev_regions(private);
> if (ret)
> @@ -204,7 +184,6 @@ static int vfio_ccw_mdev_open_device(struct
> vfio_device *vdev)
>
> out_unregister:
> vfio_ccw_unregister_dev_regions(private);
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
> return ret;
> }
>
> @@ -222,7 +201,6 @@ static void vfio_ccw_mdev_close_device(struct
> vfio_device *vdev)
>
> cp_free(&private->cp);
> vfio_ccw_unregister_dev_regions(private);
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
> }
>
> static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private
> *private,
> @@ -645,6 +623,7 @@ static const struct vfio_device_ops vfio_ccw_dev_ops
> = {
> .write = vfio_ccw_mdev_write,
> .ioctl = vfio_ccw_mdev_ioctl,
> .request = vfio_ccw_mdev_request,
> + .dma_unmap = vfio_ccw_dma_unmap,
> };
>
> struct mdev_driver vfio_ccw_mdev_driver = {
> diff --git a/drivers/s390/cio/vfio_ccw_private.h
> b/drivers/s390/cio/vfio_ccw_private.h
> index 7272eb78861244..2627791c9006d4 100644
> --- a/drivers/s390/cio/vfio_ccw_private.h
> +++ b/drivers/s390/cio/vfio_ccw_private.h
> @@ -98,7 +98,6 @@ struct vfio_ccw_private {
> struct completion *completion;
> atomic_t avail;
> struct mdev_device *mdev;
> - struct notifier_block nb;
> struct ccw_io_region *io_region;
> struct mutex io_mutex;
> struct vfio_ccw_region *region;
> diff --git a/drivers/s390/crypto/vfio_ap_ops.c
> b/drivers/s390/crypto/vfio_ap_ops.c
> index a7d2a95796d360..bb1a1677c5c230 100644
> --- a/drivers/s390/crypto/vfio_ap_ops.c
> +++ b/drivers/s390/crypto/vfio_ap_ops.c
> @@ -1226,34 +1226,14 @@ static int vfio_ap_mdev_set_kvm(struct
> ap_matrix_mdev *matrix_mdev,
> return 0;
> }
>
> -/**
> - * vfio_ap_mdev_iommu_notifier - IOMMU notifier callback
> - *
> - * @nb: The notifier block
> - * @action: Action to be taken
> - * @data: data associated with the request
> - *
> - * For an UNMAP request, unpin the guest IOVA (the NIB guest address we
> - * pinned before). Other requests are ignored.
> - *
> - * Return: for an UNMAP request, NOFITY_OK; otherwise NOTIFY_DONE.
> - */
> -static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb,
> - unsigned long action, void *data)
> +static void vfio_ap_mdev_dma_unmap(struct vfio_device *vdev, u64 iova,
> + u64 length)
> {
> - struct ap_matrix_mdev *matrix_mdev;
> -
> - matrix_mdev = container_of(nb, struct ap_matrix_mdev,
> iommu_notifier);
> -
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> - unsigned long g_pfn = unmap->iova >> PAGE_SHIFT;
> -
> - vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
> - return NOTIFY_OK;
> - }
> + struct ap_matrix_mdev *matrix_mdev =
> + container_of(vdev, struct ap_matrix_mdev, vdev);
> + unsigned long g_pfn = iova >> PAGE_SHIFT;
>
> - return NOTIFY_DONE;
> + vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
> }
>
> /**
> @@ -1380,27 +1360,11 @@ static int vfio_ap_mdev_open_device(struct
> vfio_device *vdev)
> {
> struct ap_matrix_mdev *matrix_mdev =
> container_of(vdev, struct ap_matrix_mdev, vdev);
> - unsigned long events;
> - int ret;
>
> if (!vdev->kvm)
> return -EINVAL;
>
> - ret = vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
> - if (ret)
> - return ret;
> -
> - matrix_mdev->iommu_notifier.notifier_call =
> vfio_ap_mdev_iommu_notifier;
> - events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> - ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY, &events,
> - &matrix_mdev->iommu_notifier);
> - if (ret)
> - goto err_kvm;
> - return 0;
> -
> -err_kvm:
> - vfio_ap_mdev_unset_kvm(matrix_mdev);
> - return ret;
> + return vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
> }
>
> static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
> @@ -1408,8 +1372,6 @@ static void vfio_ap_mdev_close_device(struct
> vfio_device *vdev)
> struct ap_matrix_mdev *matrix_mdev =
> container_of(vdev, struct ap_matrix_mdev, vdev);
>
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY,
> - &matrix_mdev->iommu_notifier);
> vfio_ap_mdev_unset_kvm(matrix_mdev);
> }
>
> @@ -1461,6 +1423,7 @@ static const struct vfio_device_ops
> vfio_ap_matrix_dev_ops = {
> .open_device = vfio_ap_mdev_open_device,
> .close_device = vfio_ap_mdev_close_device,
> .ioctl = vfio_ap_mdev_ioctl,
> + .dma_unmap = vfio_ap_mdev_dma_unmap,
> };
>
> static struct mdev_driver vfio_ap_matrix_driver = {
> diff --git a/drivers/s390/crypto/vfio_ap_private.h
> b/drivers/s390/crypto/vfio_ap_private.h
> index a26efd804d0df3..abb59d59f81b20 100644
> --- a/drivers/s390/crypto/vfio_ap_private.h
> +++ b/drivers/s390/crypto/vfio_ap_private.h
> @@ -81,8 +81,6 @@ struct ap_matrix {
> * @node: allows the ap_matrix_mdev struct to be added to a list
> * @matrix: the adapters, usage domains and control domains assigned
> to the
> * mediated matrix device.
> - * @iommu_notifier: notifier block used for specifying callback function for
> - * handling the VFIO_IOMMU_NOTIFY_DMA_UNMAP even
> * @kvm: the struct holding guest's state
> * @pqap_hook: the function pointer to the interception handler for
> the
> * PQAP(AQIC) instruction.
> @@ -92,7 +90,6 @@ struct ap_matrix_mdev {
> struct vfio_device vdev;
> struct list_head node;
> struct ap_matrix matrix;
> - struct notifier_block iommu_notifier;
> struct kvm *kvm;
> crypto_hook pqap_hook;
> struct mdev_device *mdev;
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 61e71c1154be67..f005b644ab9e69 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct
> vfio_device *device)
> up_write(&device->group->group_rwsem);
> }
>
> +static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long
> action,
> + void *data)
> +{
> + struct vfio_device *vfio_device =
> + container_of(nb, struct vfio_device, iommu_nb);
> + struct vfio_iommu_type1_dma_unmap *unmap = data;
> +
> + vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap-
> >size);
> + return NOTIFY_OK;
> +}
> +
> static struct file *vfio_device_open(struct vfio_device *device)
> {
> + struct vfio_iommu_driver *iommu_driver;
> struct file *filep;
> int ret;
>
> @@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct
> vfio_device *device)
> if (ret)
> goto err_undo_count;
> }
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier) {
> + unsigned long events =
> VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> +
> + device->iommu_nb.notifier_call =
> vfio_iommu_notifier;
> + iommu_driver->ops->register_notifier(
> + device->group->container->iommu_data,
> &events,
> + &device->iommu_nb);
> + }
> +
> up_read(&device->group->group_rwsem);
> }
> mutex_unlock(&device->dev_set->lock);
> @@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct
> vfio_device *device)
> err_close_device:
> mutex_lock(&device->dev_set->lock);
> down_read(&device->group->group_rwsem);
> - if (device->open_count == 1 && device->ops->close_device)
> + if (device->open_count == 1 && device->ops->close_device) {
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> + }
> err_undo_count:
> device->open_count--;
> if (device->open_count == 0 && device->kvm)
> @@ -1339,12 +1371,20 @@ static const struct file_operations
> vfio_group_fops = {
> static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> {
> struct vfio_device *device = filep->private_data;
> + struct vfio_iommu_driver *iommu_driver;
>
> mutex_lock(&device->dev_set->lock);
> vfio_assert_device_open(device);
> down_read(&device->group->group_rwsem);
> if (device->open_count == 1 && device->ops->close_device)
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> up_read(&device->group->group_rwsem);
> device->open_count--;
> if (device->open_count == 0)
> @@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device,
> dma_addr_t user_iova, void *data,
> }
> EXPORT_SYMBOL(vfio_dma_rw);
>
> -static int vfio_register_iommu_notifier(struct vfio_group *group,
> - unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->register_notifier))
> - ret = driver->ops->register_notifier(container->iommu_data,
> - events, nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -static int vfio_unregister_iommu_notifier(struct vfio_group *group,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->unregister_notifier))
> - ret = driver->ops->unregister_notifier(container-
> >iommu_data,
> - nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type, unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !events || (*events == 0) ||
> - !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_register_iommu_notifier(group, events, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_register_notifier);
> -
> -int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_unregister_iommu_notifier(group, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_unregister_notifier);
> -
> /*
> * Module/class support
> */
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index a6713022115155..cb2e4e9baa8fe8 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
> VFIO_IOMMU_CONTAINER_CLOSE = 0,
> };
>
> +/* events for register_notifier() */
> +enum {
> + VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> +};
> +
> /**
> * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> */
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index aa888cc517578e..b76623e3b92fca 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -44,6 +44,7 @@ struct vfio_device {
> unsigned int open_count;
> struct completion comp;
> struct list_head group_next;
> + struct notifier_block iommu_nb;
> };
>
> /**
> @@ -60,6 +61,8 @@ struct vfio_device {
> * @match: Optional device name match callback (return: 0 for no-match, >0
> for
> * match, -errno for abort (ex. match with insufficient or incorrect
> * additional args)
> + * @dma_unmap: Called when userspace unmaps IOVA from the container
> + * this device is attached to.
> * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
> * @migration_set_state: Optional callback to change the migration state for
> * devices that support migration. It's mandatory for
> @@ -85,6 +88,7 @@ struct vfio_device_ops {
> int (*mmap)(struct vfio_device *vdev, struct vm_area_struct
> *vma);
> void (*request)(struct vfio_device *vdev, unsigned int count);
> int (*match)(struct vfio_device *vdev, char *buf);
> + void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64
> length);
> int (*device_feature)(struct vfio_device *device, u32 flags,
> void __user *arg, size_t argsz);
> struct file *(*migration_set_state)(
> @@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device
> *device, unsigned long *user_pfn,
> extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
> void *data, size_t len, bool write);
>
> -/* each type has independent events */
> -enum vfio_notify_type {
> - VFIO_IOMMU_NOTIFY = 0,
> -};
> -
> -/* events for VFIO_IOMMU_NOTIFY */
> -#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
> -
> -extern int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - unsigned long *required_events,
> - struct notifier_block *nb);
> -extern int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb);
> -
> -
> /*
> * Sub-module helpers
> */
> --
> 2.36.1
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Intel-gfx] [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
@ 2022-06-08 3:37 ` Tian, Kevin
0 siblings, 0 replies; 34+ messages in thread
From: Tian, Kevin @ 2022-06-08 3:37 UTC (permalink / raw)
To: Jason Gunthorpe, Alexander Gordeev, David Airlie, Tony Krowiak,
Alex Williamson, Christian Borntraeger, Cornelia Huck,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Jason Herne, Joonas Lahtinen, kvm, linux-s390,
Matthew Rosato, Peter Oberparleiter, Halil Pasic, Vivi, Rodrigo,
Sven Schnelle, Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang,
Wang, Zhi A
Cc: Christoph Hellwig
> From: Jason Gunthorpe
> Sent: Wednesday, June 8, 2022 7:02 AM
>
> Instead of having drivers register the notifier with explicit code just
> have them provide a dma_unmap callback op in their driver ops and rely on
> the core code to wire it up.
>
> Suggested-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> ---
> drivers/gpu/drm/i915/gvt/gvt.h | 1 -
> drivers/gpu/drm/i915/gvt/kvmgt.c | 75 ++++-----------
> drivers/s390/cio/vfio_ccw_ops.c | 41 ++-------
> drivers/s390/cio/vfio_ccw_private.h | 1 -
> drivers/s390/crypto/vfio_ap_ops.c | 53 ++---------
> drivers/s390/crypto/vfio_ap_private.h | 3 -
> drivers/vfio/vfio.c | 126 +++++++++-----------------
> drivers/vfio/vfio.h | 5 +
> include/linux/vfio.h | 21 +----
> 9 files changed, 87 insertions(+), 239 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
> index aee1a45da74bcb..705689e6401197 100644
> --- a/drivers/gpu/drm/i915/gvt/gvt.h
> +++ b/drivers/gpu/drm/i915/gvt/gvt.h
> @@ -226,7 +226,6 @@ struct intel_vgpu {
> unsigned long nr_cache_entries;
> struct mutex cache_lock;
>
> - struct notifier_block iommu_notifier;
> atomic_t released;
>
> struct kvm_page_track_notifier_node track_node;
> diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c
> b/drivers/gpu/drm/i915/gvt/kvmgt.c
> index e2f6c56ab3420c..ecd5bb37b63a2a 100644
> --- a/drivers/gpu/drm/i915/gvt/kvmgt.c
> +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
> @@ -729,34 +729,25 @@ int intel_gvt_set_edid(struct intel_vgpu *vgpu, int
> port_num)
> return ret;
> }
>
> -static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
> - unsigned long action, void *data)
> +static void intel_vgpu_dma_unmap(struct vfio_device *vfio_dev, u64 iova,
> + u64 length)
> {
> - struct intel_vgpu *vgpu =
> - container_of(nb, struct intel_vgpu, iommu_notifier);
> -
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> - struct gvt_dma *entry;
> - unsigned long iov_pfn, end_iov_pfn;
> + struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> + struct gvt_dma *entry;
> + u64 iov_pfn = iova >> PAGE_SHIFT;
> + u64 end_iov_pfn = iov_pfn + length / PAGE_SIZE;
>
> - iov_pfn = unmap->iova >> PAGE_SHIFT;
> - end_iov_pfn = iov_pfn + unmap->size / PAGE_SIZE;
> + mutex_lock(&vgpu->cache_lock);
> + for (; iov_pfn < end_iov_pfn; iov_pfn++) {
> + entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
> + if (!entry)
> + continue;
>
> - mutex_lock(&vgpu->cache_lock);
> - for (; iov_pfn < end_iov_pfn; iov_pfn++) {
> - entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
> - if (!entry)
> - continue;
> -
> - gvt_dma_unmap_page(vgpu, entry->gfn, entry-
> >dma_addr,
> - entry->size);
> - __gvt_cache_remove_entry(vgpu, entry);
> - }
> - mutex_unlock(&vgpu->cache_lock);
> + gvt_dma_unmap_page(vgpu, entry->gfn, entry->dma_addr,
> + entry->size);
> + __gvt_cache_remove_entry(vgpu, entry);
> }
> -
> - return NOTIFY_OK;
> + mutex_unlock(&vgpu->cache_lock);
> }
>
> static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu)
> @@ -783,36 +774,20 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu
> *vgpu)
> static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
> {
> struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> - unsigned long events;
> - int ret;
> -
> - vgpu->iommu_notifier.notifier_call = intel_vgpu_iommu_notifier;
>
> - events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> - ret = vfio_register_notifier(vfio_dev, VFIO_IOMMU_NOTIFY, &events,
> - &vgpu->iommu_notifier);
> - if (ret != 0) {
> - gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n",
> - ret);
> - goto out;
> - }
> -
> - ret = -EEXIST;
> if (vgpu->attached)
> - goto undo_iommu;
> + return -EEXIST;
>
> - ret = -ESRCH;
> if (!vgpu->vfio_device.kvm ||
> vgpu->vfio_device.kvm->mm != current->mm) {
> gvt_vgpu_err("KVM is required to use Intel vGPU\n");
> - goto undo_iommu;
> + return -ESRCH;
> }
>
> kvm_get_kvm(vgpu->vfio_device.kvm);
>
> - ret = -EEXIST;
> if (__kvmgt_vgpu_exist(vgpu))
> - goto undo_iommu;
> + return -EEXIST;
>
> vgpu->attached = true;
>
> @@ -831,12 +806,6 @@ static int intel_vgpu_open_device(struct vfio_device
> *vfio_dev)
>
> atomic_set(&vgpu->released, 0);
> return 0;
> -
> -undo_iommu:
> - vfio_unregister_notifier(vfio_dev, VFIO_IOMMU_NOTIFY,
> - &vgpu->iommu_notifier);
> -out:
> - return ret;
> }
>
> static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
> @@ -853,8 +822,6 @@ static void
> intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
> static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
> {
> struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> - struct drm_i915_private *i915 = vgpu->gvt->gt->i915;
> - int ret;
>
> if (!vgpu->attached)
> return;
> @@ -864,11 +831,6 @@ static void intel_vgpu_close_device(struct
> vfio_device *vfio_dev)
>
> intel_gvt_release_vgpu(vgpu);
>
> - ret = vfio_unregister_notifier(&vgpu->vfio_device,
> VFIO_IOMMU_NOTIFY,
> - &vgpu->iommu_notifier);
> - drm_WARN(&i915->drm, ret,
> - "vfio_unregister_notifier for iommu failed: %d\n", ret);
> -
> debugfs_remove(debugfs_lookup(KVMGT_DEBUGFS_FILENAME,
> vgpu->debugfs));
>
> kvm_page_track_unregister_notifier(vgpu->vfio_device.kvm,
> @@ -1610,6 +1572,7 @@ static const struct vfio_device_ops
> intel_vgpu_dev_ops = {
> .write = intel_vgpu_write,
> .mmap = intel_vgpu_mmap,
> .ioctl = intel_vgpu_ioctl,
> + .dma_unmap = intel_vgpu_dma_unmap,
> };
>
> static int intel_vgpu_probe(struct mdev_device *mdev)
> diff --git a/drivers/s390/cio/vfio_ccw_ops.c
> b/drivers/s390/cio/vfio_ccw_ops.c
> index b49e2e9db2dc6f..09e0ce7b72324c 100644
> --- a/drivers/s390/cio/vfio_ccw_ops.c
> +++ b/drivers/s390/cio/vfio_ccw_ops.c
> @@ -44,31 +44,19 @@ static int vfio_ccw_mdev_reset(struct
> vfio_ccw_private *private)
> return ret;
> }
>
> -static int vfio_ccw_mdev_notifier(struct notifier_block *nb,
> - unsigned long action,
> - void *data)
> +static void vfio_ccw_dma_unmap(struct vfio_device *vdev, u64 iova, u64
> length)
> {
> struct vfio_ccw_private *private =
> - container_of(nb, struct vfio_ccw_private, nb);
> -
> - /*
> - * Vendor drivers MUST unpin pages in response to an
> - * invalidation.
> - */
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> -
> - if (!cp_iova_pinned(&private->cp, unmap->iova))
> - return NOTIFY_OK;
> + container_of(vdev, struct vfio_ccw_private, vdev);
>
> - if (vfio_ccw_mdev_reset(private))
> - return NOTIFY_BAD;
> + /* Drivers MUST unpin pages in response to an invalidation. */
> + if (!cp_iova_pinned(&private->cp, iova))
> + return;
>
> - cp_free(&private->cp);
> - return NOTIFY_OK;
> - }
> + if (vfio_ccw_mdev_reset(private))
> + return;
>
> - return NOTIFY_DONE;
> + cp_free(&private->cp);
> }
>
> static ssize_t name_show(struct mdev_type *mtype,
> @@ -178,19 +166,11 @@ static int vfio_ccw_mdev_open_device(struct
> vfio_device *vdev)
> {
> struct vfio_ccw_private *private =
> container_of(vdev, struct vfio_ccw_private, vdev);
> - unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> int ret;
>
> - private->nb.notifier_call = vfio_ccw_mdev_notifier;
> -
> - ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY,
> - &events, &private->nb);
> - if (ret)
> - return ret;
> -
> ret = vfio_ccw_register_async_dev_regions(private);
> if (ret)
> - goto out_unregister;
> + return ret;
>
> ret = vfio_ccw_register_schib_dev_regions(private);
> if (ret)
> @@ -204,7 +184,6 @@ static int vfio_ccw_mdev_open_device(struct
> vfio_device *vdev)
>
> out_unregister:
> vfio_ccw_unregister_dev_regions(private);
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
> return ret;
> }
>
> @@ -222,7 +201,6 @@ static void vfio_ccw_mdev_close_device(struct
> vfio_device *vdev)
>
> cp_free(&private->cp);
> vfio_ccw_unregister_dev_regions(private);
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
> }
>
> static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private
> *private,
> @@ -645,6 +623,7 @@ static const struct vfio_device_ops vfio_ccw_dev_ops
> = {
> .write = vfio_ccw_mdev_write,
> .ioctl = vfio_ccw_mdev_ioctl,
> .request = vfio_ccw_mdev_request,
> + .dma_unmap = vfio_ccw_dma_unmap,
> };
>
> struct mdev_driver vfio_ccw_mdev_driver = {
> diff --git a/drivers/s390/cio/vfio_ccw_private.h
> b/drivers/s390/cio/vfio_ccw_private.h
> index 7272eb78861244..2627791c9006d4 100644
> --- a/drivers/s390/cio/vfio_ccw_private.h
> +++ b/drivers/s390/cio/vfio_ccw_private.h
> @@ -98,7 +98,6 @@ struct vfio_ccw_private {
> struct completion *completion;
> atomic_t avail;
> struct mdev_device *mdev;
> - struct notifier_block nb;
> struct ccw_io_region *io_region;
> struct mutex io_mutex;
> struct vfio_ccw_region *region;
> diff --git a/drivers/s390/crypto/vfio_ap_ops.c
> b/drivers/s390/crypto/vfio_ap_ops.c
> index a7d2a95796d360..bb1a1677c5c230 100644
> --- a/drivers/s390/crypto/vfio_ap_ops.c
> +++ b/drivers/s390/crypto/vfio_ap_ops.c
> @@ -1226,34 +1226,14 @@ static int vfio_ap_mdev_set_kvm(struct
> ap_matrix_mdev *matrix_mdev,
> return 0;
> }
>
> -/**
> - * vfio_ap_mdev_iommu_notifier - IOMMU notifier callback
> - *
> - * @nb: The notifier block
> - * @action: Action to be taken
> - * @data: data associated with the request
> - *
> - * For an UNMAP request, unpin the guest IOVA (the NIB guest address we
> - * pinned before). Other requests are ignored.
> - *
> - * Return: for an UNMAP request, NOFITY_OK; otherwise NOTIFY_DONE.
> - */
> -static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb,
> - unsigned long action, void *data)
> +static void vfio_ap_mdev_dma_unmap(struct vfio_device *vdev, u64 iova,
> + u64 length)
> {
> - struct ap_matrix_mdev *matrix_mdev;
> -
> - matrix_mdev = container_of(nb, struct ap_matrix_mdev,
> iommu_notifier);
> -
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> - unsigned long g_pfn = unmap->iova >> PAGE_SHIFT;
> -
> - vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
> - return NOTIFY_OK;
> - }
> + struct ap_matrix_mdev *matrix_mdev =
> + container_of(vdev, struct ap_matrix_mdev, vdev);
> + unsigned long g_pfn = iova >> PAGE_SHIFT;
>
> - return NOTIFY_DONE;
> + vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
> }
>
> /**
> @@ -1380,27 +1360,11 @@ static int vfio_ap_mdev_open_device(struct
> vfio_device *vdev)
> {
> struct ap_matrix_mdev *matrix_mdev =
> container_of(vdev, struct ap_matrix_mdev, vdev);
> - unsigned long events;
> - int ret;
>
> if (!vdev->kvm)
> return -EINVAL;
>
> - ret = vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
> - if (ret)
> - return ret;
> -
> - matrix_mdev->iommu_notifier.notifier_call =
> vfio_ap_mdev_iommu_notifier;
> - events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> - ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY, &events,
> - &matrix_mdev->iommu_notifier);
> - if (ret)
> - goto err_kvm;
> - return 0;
> -
> -err_kvm:
> - vfio_ap_mdev_unset_kvm(matrix_mdev);
> - return ret;
> + return vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
> }
>
> static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
> @@ -1408,8 +1372,6 @@ static void vfio_ap_mdev_close_device(struct
> vfio_device *vdev)
> struct ap_matrix_mdev *matrix_mdev =
> container_of(vdev, struct ap_matrix_mdev, vdev);
>
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY,
> - &matrix_mdev->iommu_notifier);
> vfio_ap_mdev_unset_kvm(matrix_mdev);
> }
>
> @@ -1461,6 +1423,7 @@ static const struct vfio_device_ops
> vfio_ap_matrix_dev_ops = {
> .open_device = vfio_ap_mdev_open_device,
> .close_device = vfio_ap_mdev_close_device,
> .ioctl = vfio_ap_mdev_ioctl,
> + .dma_unmap = vfio_ap_mdev_dma_unmap,
> };
>
> static struct mdev_driver vfio_ap_matrix_driver = {
> diff --git a/drivers/s390/crypto/vfio_ap_private.h
> b/drivers/s390/crypto/vfio_ap_private.h
> index a26efd804d0df3..abb59d59f81b20 100644
> --- a/drivers/s390/crypto/vfio_ap_private.h
> +++ b/drivers/s390/crypto/vfio_ap_private.h
> @@ -81,8 +81,6 @@ struct ap_matrix {
> * @node: allows the ap_matrix_mdev struct to be added to a list
> * @matrix: the adapters, usage domains and control domains assigned
> to the
> * mediated matrix device.
> - * @iommu_notifier: notifier block used for specifying callback function for
> - * handling the VFIO_IOMMU_NOTIFY_DMA_UNMAP even
> * @kvm: the struct holding guest's state
> * @pqap_hook: the function pointer to the interception handler for
> the
> * PQAP(AQIC) instruction.
> @@ -92,7 +90,6 @@ struct ap_matrix_mdev {
> struct vfio_device vdev;
> struct list_head node;
> struct ap_matrix matrix;
> - struct notifier_block iommu_notifier;
> struct kvm *kvm;
> crypto_hook pqap_hook;
> struct mdev_device *mdev;
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 61e71c1154be67..f005b644ab9e69 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct
> vfio_device *device)
> up_write(&device->group->group_rwsem);
> }
>
> +static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long
> action,
> + void *data)
> +{
> + struct vfio_device *vfio_device =
> + container_of(nb, struct vfio_device, iommu_nb);
> + struct vfio_iommu_type1_dma_unmap *unmap = data;
> +
> + vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap-
> >size);
> + return NOTIFY_OK;
> +}
> +
> static struct file *vfio_device_open(struct vfio_device *device)
> {
> + struct vfio_iommu_driver *iommu_driver;
> struct file *filep;
> int ret;
>
> @@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct
> vfio_device *device)
> if (ret)
> goto err_undo_count;
> }
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier) {
> + unsigned long events =
> VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> +
> + device->iommu_nb.notifier_call =
> vfio_iommu_notifier;
> + iommu_driver->ops->register_notifier(
> + device->group->container->iommu_data,
> &events,
> + &device->iommu_nb);
> + }
> +
> up_read(&device->group->group_rwsem);
> }
> mutex_unlock(&device->dev_set->lock);
> @@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct
> vfio_device *device)
> err_close_device:
> mutex_lock(&device->dev_set->lock);
> down_read(&device->group->group_rwsem);
> - if (device->open_count == 1 && device->ops->close_device)
> + if (device->open_count == 1 && device->ops->close_device) {
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> + }
> err_undo_count:
> device->open_count--;
> if (device->open_count == 0 && device->kvm)
> @@ -1339,12 +1371,20 @@ static const struct file_operations
> vfio_group_fops = {
> static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> {
> struct vfio_device *device = filep->private_data;
> + struct vfio_iommu_driver *iommu_driver;
>
> mutex_lock(&device->dev_set->lock);
> vfio_assert_device_open(device);
> down_read(&device->group->group_rwsem);
> if (device->open_count == 1 && device->ops->close_device)
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> up_read(&device->group->group_rwsem);
> device->open_count--;
> if (device->open_count == 0)
> @@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device,
> dma_addr_t user_iova, void *data,
> }
> EXPORT_SYMBOL(vfio_dma_rw);
>
> -static int vfio_register_iommu_notifier(struct vfio_group *group,
> - unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->register_notifier))
> - ret = driver->ops->register_notifier(container->iommu_data,
> - events, nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -static int vfio_unregister_iommu_notifier(struct vfio_group *group,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->unregister_notifier))
> - ret = driver->ops->unregister_notifier(container-
> >iommu_data,
> - nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type, unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !events || (*events == 0) ||
> - !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_register_iommu_notifier(group, events, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_register_notifier);
> -
> -int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_unregister_iommu_notifier(group, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_unregister_notifier);
> -
> /*
> * Module/class support
> */
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index a6713022115155..cb2e4e9baa8fe8 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
> VFIO_IOMMU_CONTAINER_CLOSE = 0,
> };
>
> +/* events for register_notifier() */
> +enum {
> + VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> +};
> +
> /**
> * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> */
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index aa888cc517578e..b76623e3b92fca 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -44,6 +44,7 @@ struct vfio_device {
> unsigned int open_count;
> struct completion comp;
> struct list_head group_next;
> + struct notifier_block iommu_nb;
> };
>
> /**
> @@ -60,6 +61,8 @@ struct vfio_device {
> * @match: Optional device name match callback (return: 0 for no-match, >0
> for
> * match, -errno for abort (ex. match with insufficient or incorrect
> * additional args)
> + * @dma_unmap: Called when userspace unmaps IOVA from the container
> + * this device is attached to.
> * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
> * @migration_set_state: Optional callback to change the migration state for
> * devices that support migration. It's mandatory for
> @@ -85,6 +88,7 @@ struct vfio_device_ops {
> int (*mmap)(struct vfio_device *vdev, struct vm_area_struct
> *vma);
> void (*request)(struct vfio_device *vdev, unsigned int count);
> int (*match)(struct vfio_device *vdev, char *buf);
> + void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64
> length);
> int (*device_feature)(struct vfio_device *device, u32 flags,
> void __user *arg, size_t argsz);
> struct file *(*migration_set_state)(
> @@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device
> *device, unsigned long *user_pfn,
> extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
> void *data, size_t len, bool write);
>
> -/* each type has independent events */
> -enum vfio_notify_type {
> - VFIO_IOMMU_NOTIFY = 0,
> -};
> -
> -/* events for VFIO_IOMMU_NOTIFY */
> -#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
> -
> -extern int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - unsigned long *required_events,
> - struct notifier_block *nb);
> -extern int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb);
> -
> -
> /*
> * Sub-module helpers
> */
> --
> 2.36.1
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
@ 2022-06-08 3:47 ` Tian, Kevin
-1 siblings, 0 replies; 34+ messages in thread
From: Tian, Kevin @ 2022-06-08 3:47 UTC (permalink / raw)
To: Jason Gunthorpe, Alexander Gordeev, David Airlie, Tony Krowiak,
Alex Williamson, Christian Borntraeger, Cornelia Huck,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Jason Herne, Joonas Lahtinen, kvm, linux-s390,
Matthew Rosato, Peter Oberparleiter, Halil Pasic, Vivi, Rodrigo,
Sven Schnelle, Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang,
Wang, Zhi A
Cc: Christoph Hellwig
> From: Jason Gunthorpe
> Sent: Wednesday, June 8, 2022 7:02 AM
>
> Instead of bouncing the function call to the driver op through a blocking
> notifier just have the iommu layer call it directly.
>
> Register each device that is being attached to the iommu with the lower
> driver which then threads them on a linked list and calls the appropriate
> driver op at the right time.
>
> Currently the only use is if dma_unmap() is defined.
>
> Also, fully lock all the debugging tests on the pinning path that a
> dma_unmap is registered.
>
...
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> diff --git a/drivers/vfio/vfio_iommu_type1.c
> b/drivers/vfio/vfio_iommu_type1.c
> index c13b9290e35759..4ddb1f1abd238b 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -67,7 +67,8 @@ struct vfio_iommu {
> struct list_head iova_list;
> struct mutex lock;
> struct rb_root dma_list;
> - struct blocking_notifier_head notifier;
> + struct list_head device_list;
> + struct mutex device_list_lock;
emulated_device_list or unmap_device_list, to be more accurate?
btw a few lines downward there is already a emulated_iommu_groups.
the two sounds a bit overlapping given mdev's iommu group is faked
and singleton. Wonder whether it's cleaner to just reuse the existing field...
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Intel-gfx] [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
@ 2022-06-08 3:47 ` Tian, Kevin
0 siblings, 0 replies; 34+ messages in thread
From: Tian, Kevin @ 2022-06-08 3:47 UTC (permalink / raw)
To: Jason Gunthorpe, Alexander Gordeev, David Airlie, Tony Krowiak,
Alex Williamson, Christian Borntraeger, Cornelia Huck,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Jason Herne, Joonas Lahtinen, kvm, linux-s390,
Matthew Rosato, Peter Oberparleiter, Halil Pasic, Vivi, Rodrigo,
Sven Schnelle, Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang,
Wang, Zhi A
Cc: Christoph Hellwig
> From: Jason Gunthorpe
> Sent: Wednesday, June 8, 2022 7:02 AM
>
> Instead of bouncing the function call to the driver op through a blocking
> notifier just have the iommu layer call it directly.
>
> Register each device that is being attached to the iommu with the lower
> driver which then threads them on a linked list and calls the appropriate
> driver op at the right time.
>
> Currently the only use is if dma_unmap() is defined.
>
> Also, fully lock all the debugging tests on the pinning path that a
> dma_unmap is registered.
>
...
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> diff --git a/drivers/vfio/vfio_iommu_type1.c
> b/drivers/vfio/vfio_iommu_type1.c
> index c13b9290e35759..4ddb1f1abd238b 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -67,7 +67,8 @@ struct vfio_iommu {
> struct list_head iova_list;
> struct mutex lock;
> struct rb_root dma_list;
> - struct blocking_notifier_head notifier;
> + struct list_head device_list;
> + struct mutex device_list_lock;
emulated_device_list or unmap_device_list, to be more accurate?
btw a few lines downward there is already a emulated_iommu_groups.
the two sounds a bit overlapping given mdev's iommu group is faked
and singleton. Wonder whether it's cleaner to just reuse the existing field...
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Intel-gfx] [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
@ 2022-06-08 5:33 ` Christoph Hellwig
-1 siblings, 0 replies; 34+ messages in thread
From: Christoph Hellwig @ 2022-06-08 5:33 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Jason Herne, Eric Farman,
Vasily Gorbik, Heiko Carstens, Harald Freudenberger,
Rodrigo Vivi, intel-gvt-dev, Tony Krowiak, Cornelia Huck,
Peter Oberparleiter, Sven Schnelle
Looks good:
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
@ 2022-06-08 5:33 ` Christoph Hellwig
0 siblings, 0 replies; 34+ messages in thread
From: Christoph Hellwig @ 2022-06-08 5:33 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alexander Gordeev, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang,
Christoph Hellwig
Looks good:
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 34+ messages in thread
* [Intel-gfx] ✗ Fi.CI.BAT: failure for Remove the VFIO_IOMMU_NOTIFY_DMA_UNMAP notifier (rev2)
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
` (2 preceding siblings ...)
(?)
@ 2022-06-08 5:38 ` Patchwork
-1 siblings, 0 replies; 34+ messages in thread
From: Patchwork @ 2022-06-08 5:38 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: intel-gfx
[-- Attachment #1: Type: text/plain, Size: 11430 bytes --]
== Series Details ==
Series: Remove the VFIO_IOMMU_NOTIFY_DMA_UNMAP notifier (rev2)
URL : https://patchwork.freedesktop.org/series/104793/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_11732 -> Patchwork_104793v2
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_104793v2 absolutely need to be
verified manually.
If you think the reported changes have nothing to do with the changes
introduced in Patchwork_104793v2, please notify your bug team to allow them
to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/index.html
Participating hosts (44 -> 46)
------------------------------
Additional (3): fi-kbl-soraka bat-dg2-9 bat-adlp-6
Missing (1): fi-bdw-samus
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_104793v2:
### IGT changes ###
#### Possible regressions ####
* igt@i915_selftest@live@memory_region:
- fi-bdw-5557u: [PASS][1] -> [INCOMPLETE][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/fi-bdw-5557u/igt@i915_selftest@live@memory_region.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-bdw-5557u/igt@i915_selftest@live@memory_region.html
Known issues
------------
Here are the changes found in Patchwork_104793v2 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_huc_copy@huc-copy:
- fi-kbl-soraka: NOTRUN -> [SKIP][3] ([fdo#109271] / [i915#2190])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-kbl-soraka/igt@gem_huc_copy@huc-copy.html
* igt@gem_lmem_swapping@basic:
- fi-kbl-soraka: NOTRUN -> [SKIP][4] ([fdo#109271] / [i915#4613]) +3 similar issues
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-kbl-soraka/igt@gem_lmem_swapping@basic.html
* igt@i915_selftest@live@gem_migrate:
- fi-bdw-5557u: [PASS][5] -> [INCOMPLETE][6] ([i915#5716])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/fi-bdw-5557u/igt@i915_selftest@live@gem_migrate.html
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-bdw-5557u/igt@i915_selftest@live@gem_migrate.html
* igt@i915_selftest@live@gt_engines:
- bat-dg1-6: [PASS][7] -> [INCOMPLETE][8] ([i915#4418])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/bat-dg1-6/igt@i915_selftest@live@gt_engines.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/bat-dg1-6/igt@i915_selftest@live@gt_engines.html
- bat-dg1-5: [PASS][9] -> [INCOMPLETE][10] ([i915#4418])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/bat-dg1-5/igt@i915_selftest@live@gt_engines.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/bat-dg1-5/igt@i915_selftest@live@gt_engines.html
* igt@i915_selftest@live@gt_pm:
- fi-kbl-soraka: NOTRUN -> [DMESG-FAIL][11] ([i915#1886])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-kbl-soraka/igt@i915_selftest@live@gt_pm.html
* igt@i915_selftest@live@hangcheck:
- fi-hsw-4770: [PASS][12] -> [INCOMPLETE][13] ([i915#3303] / [i915#4785])
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/fi-hsw-4770/igt@i915_selftest@live@hangcheck.html
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-hsw-4770/igt@i915_selftest@live@hangcheck.html
- fi-hsw-g3258: [PASS][14] -> [INCOMPLETE][15] ([i915#3303] / [i915#4785])
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/fi-hsw-g3258/igt@i915_selftest@live@hangcheck.html
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-hsw-g3258/igt@i915_selftest@live@hangcheck.html
* igt@i915_suspend@basic-s2idle-without-i915:
- fi-bdw-gvtdvm: NOTRUN -> [INCOMPLETE][16] ([i915#4817])
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-bdw-gvtdvm/igt@i915_suspend@basic-s2idle-without-i915.html
* igt@kms_chamelium@common-hpd-after-suspend:
- fi-pnv-d510: NOTRUN -> [SKIP][17] ([fdo#109271])
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-pnv-d510/igt@kms_chamelium@common-hpd-after-suspend.html
* igt@kms_chamelium@hdmi-hpd-fast:
- fi-kbl-soraka: NOTRUN -> [SKIP][18] ([fdo#109271] / [fdo#111827]) +7 similar issues
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-kbl-soraka/igt@kms_chamelium@hdmi-hpd-fast.html
* igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic:
- fi-kbl-soraka: NOTRUN -> [SKIP][19] ([fdo#109271]) +9 similar issues
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-kbl-soraka/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic.html
* igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d:
- fi-kbl-soraka: NOTRUN -> [SKIP][20] ([fdo#109271] / [i915#533])
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-kbl-soraka/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d.html
* igt@runner@aborted:
- fi-hsw-4770: NOTRUN -> [FAIL][21] ([fdo#109271] / [i915#4312] / [i915#5594])
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-hsw-4770/igt@runner@aborted.html
- fi-hsw-g3258: NOTRUN -> [FAIL][22] ([fdo#109271] / [i915#4312])
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-hsw-g3258/igt@runner@aborted.html
#### Possible fixes ####
* igt@i915_pm_rpm@module-reload:
- fi-cfl-8109u: [DMESG-FAIL][23] ([i915#62]) -> [PASS][24]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/fi-cfl-8109u/igt@i915_pm_rpm@module-reload.html
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-cfl-8109u/igt@i915_pm_rpm@module-reload.html
* igt@i915_selftest@live@gem:
- fi-bdw-gvtdvm: [DMESG-FAIL][25] -> [PASS][26]
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/fi-bdw-gvtdvm/igt@i915_selftest@live@gem.html
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-bdw-gvtdvm/igt@i915_selftest@live@gem.html
* igt@i915_selftest@live@late_gt_pm:
- fi-cfl-8109u: [DMESG-WARN][27] ([i915#5904]) -> [PASS][28] +11 similar issues
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/fi-cfl-8109u/igt@i915_selftest@live@late_gt_pm.html
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-cfl-8109u/igt@i915_selftest@live@late_gt_pm.html
* igt@i915_selftest@live@requests:
- fi-pnv-d510: [DMESG-FAIL][29] ([i915#4528]) -> [PASS][30]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/fi-pnv-d510/igt@i915_selftest@live@requests.html
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-pnv-d510/igt@i915_selftest@live@requests.html
* igt@i915_suspend@basic-s2idle-without-i915:
- fi-cfl-8109u: [DMESG-WARN][31] ([i915#5904] / [i915#62]) -> [PASS][32]
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/fi-cfl-8109u/igt@i915_suspend@basic-s2idle-without-i915.html
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-cfl-8109u/igt@i915_suspend@basic-s2idle-without-i915.html
* igt@kms_frontbuffer_tracking@basic:
- fi-cfl-8109u: [DMESG-WARN][33] ([i915#62]) -> [PASS][34] +15 similar issues
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_11732/fi-cfl-8109u/igt@kms_frontbuffer_tracking@basic.html
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/fi-cfl-8109u/igt@kms_frontbuffer_tracking@basic.html
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
[fdo#109295]: https://bugs.freedesktop.org/show_bug.cgi?id=109295
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
[i915#1155]: https://gitlab.freedesktop.org/drm/intel/issues/1155
[i915#1886]: https://gitlab.freedesktop.org/drm/intel/issues/1886
[i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
[i915#2190]: https://gitlab.freedesktop.org/drm/intel/issues/2190
[i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
[i915#3291]: https://gitlab.freedesktop.org/drm/intel/issues/3291
[i915#3301]: https://gitlab.freedesktop.org/drm/intel/issues/3301
[i915#3303]: https://gitlab.freedesktop.org/drm/intel/issues/3303
[i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
[i915#3595]: https://gitlab.freedesktop.org/drm/intel/issues/3595
[i915#3708]: https://gitlab.freedesktop.org/drm/intel/issues/3708
[i915#4077]: https://gitlab.freedesktop.org/drm/intel/issues/4077
[i915#4079]: https://gitlab.freedesktop.org/drm/intel/issues/4079
[i915#4083]: https://gitlab.freedesktop.org/drm/intel/issues/4083
[i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103
[i915#4212]: https://gitlab.freedesktop.org/drm/intel/issues/4212
[i915#4213]: https://gitlab.freedesktop.org/drm/intel/issues/4213
[i915#4215]: https://gitlab.freedesktop.org/drm/intel/issues/4215
[i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
[i915#4418]: https://gitlab.freedesktop.org/drm/intel/issues/4418
[i915#4528]: https://gitlab.freedesktop.org/drm/intel/issues/4528
[i915#4579]: https://gitlab.freedesktop.org/drm/intel/issues/4579
[i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
[i915#4785]: https://gitlab.freedesktop.org/drm/intel/issues/4785
[i915#4817]: https://gitlab.freedesktop.org/drm/intel/issues/4817
[i915#4873]: https://gitlab.freedesktop.org/drm/intel/issues/4873
[i915#5174]: https://gitlab.freedesktop.org/drm/intel/issues/5174
[i915#5190]: https://gitlab.freedesktop.org/drm/intel/issues/5190
[i915#5274]: https://gitlab.freedesktop.org/drm/intel/issues/5274
[i915#533]: https://gitlab.freedesktop.org/drm/intel/issues/533
[i915#5594]: https://gitlab.freedesktop.org/drm/intel/issues/5594
[i915#5716]: https://gitlab.freedesktop.org/drm/intel/issues/5716
[i915#5763]: https://gitlab.freedesktop.org/drm/intel/issues/5763
[i915#5885]: https://gitlab.freedesktop.org/drm/intel/issues/5885
[i915#5903]: https://gitlab.freedesktop.org/drm/intel/issues/5903
[i915#5904]: https://gitlab.freedesktop.org/drm/intel/issues/5904
[i915#62]: https://gitlab.freedesktop.org/drm/intel/issues/62
Build changes
-------------
* Linux: CI_DRM_11732 -> Patchwork_104793v2
CI-20190529: 20190529
CI_DRM_11732: ea6c9f614fb2d6a5da6d417cdce31d925786275c @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6512: 5efc64e9b93a2df85581ea425ee7d0f435855f13 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_104793v2: ea6c9f614fb2d6a5da6d417cdce31d925786275c @ git://anongit.freedesktop.org/gfx-ci/linux
### Linux commits
93b6c68e6e55 vfio: Replace the iommu notifier with a device list
47572caa1fa5 vfio: Replace the DMA unmapping notifier with a callback
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_104793v2/index.html
[-- Attachment #2: Type: text/html, Size: 11752 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
2022-06-08 3:47 ` [Intel-gfx] " Tian, Kevin
(?)
@ 2022-06-08 11:26 ` Jason Gunthorpe
-1 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-08 11:26 UTC (permalink / raw)
To: Tian, Kevin
Cc: Alexander Gordeev, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Vivi, Rodrigo, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Wang, Zhi A,
Christoph Hellwig
On Wed, Jun 08, 2022 at 03:47:12AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe
> > Sent: Wednesday, June 8, 2022 7:02 AM
> >
> > Instead of bouncing the function call to the driver op through a blocking
> > notifier just have the iommu layer call it directly.
> >
> > Register each device that is being attached to the iommu with the lower
> > driver which then threads them on a linked list and calls the appropriate
> > driver op at the right time.
> >
> > Currently the only use is if dma_unmap() is defined.
> >
> > Also, fully lock all the debugging tests on the pinning path that a
> > dma_unmap is registered.
> >
> ...
> > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> > diff --git a/drivers/vfio/vfio_iommu_type1.c
> > b/drivers/vfio/vfio_iommu_type1.c
> > index c13b9290e35759..4ddb1f1abd238b 100644
> > +++ b/drivers/vfio/vfio_iommu_type1.c
> > @@ -67,7 +67,8 @@ struct vfio_iommu {
> > struct list_head iova_list;
> > struct mutex lock;
> > struct rb_root dma_list;
> > - struct blocking_notifier_head notifier;
> > + struct list_head device_list;
> > + struct mutex device_list_lock;
>
> emulated_device_list or unmap_device_list, to be more accurate?
I don't think it is worth getting too specific here, the fact it only
has dma_unmap devices is a bit of a shortcut to save some work.
> btw a few lines downward there is already a emulated_iommu_groups.
> the two sounds a bit overlapping given mdev's iommu group is faked
> and singleton. Wonder whether it's cleaner to just reuse the existing field...
The locking is more complicated on this since we still have check
every device in the group if it is opened or not while launching the
callback and prevent it from opening/closing while the callback is
running.
Since I plan to delete the dirty tracking which will drop the
emulated_iommu_groups too I would leave it like this.
Jason
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
@ 2022-06-08 11:26 ` Jason Gunthorpe
0 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-08 11:26 UTC (permalink / raw)
To: Tian, Kevin
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Wang, Zhi A, Jason Herne,
Eric Farman, Vasily Gorbik, Heiko Carstens, Alex Williamson,
Harald Freudenberger, Vivi, Rodrigo, intel-gvt-dev, Tony Krowiak,
Tvrtko Ursulin, Cornelia Huck, Peter Oberparleiter,
Sven Schnelle
On Wed, Jun 08, 2022 at 03:47:12AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe
> > Sent: Wednesday, June 8, 2022 7:02 AM
> >
> > Instead of bouncing the function call to the driver op through a blocking
> > notifier just have the iommu layer call it directly.
> >
> > Register each device that is being attached to the iommu with the lower
> > driver which then threads them on a linked list and calls the appropriate
> > driver op at the right time.
> >
> > Currently the only use is if dma_unmap() is defined.
> >
> > Also, fully lock all the debugging tests on the pinning path that a
> > dma_unmap is registered.
> >
> ...
> > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> > diff --git a/drivers/vfio/vfio_iommu_type1.c
> > b/drivers/vfio/vfio_iommu_type1.c
> > index c13b9290e35759..4ddb1f1abd238b 100644
> > +++ b/drivers/vfio/vfio_iommu_type1.c
> > @@ -67,7 +67,8 @@ struct vfio_iommu {
> > struct list_head iova_list;
> > struct mutex lock;
> > struct rb_root dma_list;
> > - struct blocking_notifier_head notifier;
> > + struct list_head device_list;
> > + struct mutex device_list_lock;
>
> emulated_device_list or unmap_device_list, to be more accurate?
I don't think it is worth getting too specific here, the fact it only
has dma_unmap devices is a bit of a shortcut to save some work.
> btw a few lines downward there is already a emulated_iommu_groups.
> the two sounds a bit overlapping given mdev's iommu group is faked
> and singleton. Wonder whether it's cleaner to just reuse the existing field...
The locking is more complicated on this since we still have check
every device in the group if it is opened or not while launching the
callback and prevent it from opening/closing while the callback is
running.
Since I plan to delete the dirty tracking which will drop the
emulated_iommu_groups too I would leave it like this.
Jason
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Intel-gfx] [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
@ 2022-06-08 11:26 ` Jason Gunthorpe
0 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-08 11:26 UTC (permalink / raw)
To: Tian, Kevin
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Jason Herne, Eric Farman,
Vasily Gorbik, Heiko Carstens, Harald Freudenberger, Vivi,
Rodrigo, intel-gvt-dev, Tony Krowiak, Cornelia Huck,
Peter Oberparleiter, Sven Schnelle
On Wed, Jun 08, 2022 at 03:47:12AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe
> > Sent: Wednesday, June 8, 2022 7:02 AM
> >
> > Instead of bouncing the function call to the driver op through a blocking
> > notifier just have the iommu layer call it directly.
> >
> > Register each device that is being attached to the iommu with the lower
> > driver which then threads them on a linked list and calls the appropriate
> > driver op at the right time.
> >
> > Currently the only use is if dma_unmap() is defined.
> >
> > Also, fully lock all the debugging tests on the pinning path that a
> > dma_unmap is registered.
> >
> ...
> > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> > diff --git a/drivers/vfio/vfio_iommu_type1.c
> > b/drivers/vfio/vfio_iommu_type1.c
> > index c13b9290e35759..4ddb1f1abd238b 100644
> > +++ b/drivers/vfio/vfio_iommu_type1.c
> > @@ -67,7 +67,8 @@ struct vfio_iommu {
> > struct list_head iova_list;
> > struct mutex lock;
> > struct rb_root dma_list;
> > - struct blocking_notifier_head notifier;
> > + struct list_head device_list;
> > + struct mutex device_list_lock;
>
> emulated_device_list or unmap_device_list, to be more accurate?
I don't think it is worth getting too specific here, the fact it only
has dma_unmap devices is a bit of a shortcut to save some work.
> btw a few lines downward there is already a emulated_iommu_groups.
> the two sounds a bit overlapping given mdev's iommu group is faked
> and singleton. Wonder whether it's cleaner to just reuse the existing field...
The locking is more complicated on this since we still have check
every device in the group if it is opened or not while launching the
callback and prevent it from opening/closing while the callback is
running.
Since I plan to delete the dirty tracking which will drop the
emulated_iommu_groups too I would leave it like this.
Jason
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
(?)
(?)
@ 2022-06-08 15:50 ` Eric Farman
2022-06-09 14:43 ` Jason Gunthorpe
-1 siblings, 1 reply; 34+ messages in thread
From: Eric Farman @ 2022-06-08 15:50 UTC (permalink / raw)
To: Jason Gunthorpe, Alexander Gordeev, David Airlie, Tony Krowiak,
Alex Williamson, Christian Borntraeger, Cornelia Huck,
Daniel Vetter, dri-devel, Harald Freudenberger, Vasily Gorbik,
Heiko Carstens, intel-gfx, intel-gvt-dev, Jani Nikula,
Jason Herne, Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Christoph Hellwig
On Tue, 2022-06-07 at 20:02 -0300, Jason Gunthorpe wrote:
> Instead of having drivers register the notifier with explicit code
> just
> have them provide a dma_unmap callback op in their driver ops and
> rely on
> the core code to wire it up.
>
> Suggested-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
> drivers/gpu/drm/i915/gvt/gvt.h | 1 -
> drivers/gpu/drm/i915/gvt/kvmgt.c | 75 ++++-----------
> drivers/s390/cio/vfio_ccw_ops.c | 41 ++-------
> drivers/s390/cio/vfio_ccw_private.h | 1 -
> drivers/s390/crypto/vfio_ap_ops.c | 53 ++---------
> drivers/s390/crypto/vfio_ap_private.h | 3 -
> drivers/vfio/vfio.c | 126 +++++++++---------------
> --
> drivers/vfio/vfio.h | 5 +
> include/linux/vfio.h | 21 +----
> 9 files changed, 87 insertions(+), 239 deletions(-)
>
>
...snip...
> diff --git a/drivers/s390/cio/vfio_ccw_private.h
> b/drivers/s390/cio/vfio_ccw_private.h
> index 7272eb78861244..2627791c9006d4 100644
> --- a/drivers/s390/cio/vfio_ccw_private.h
> +++ b/drivers/s390/cio/vfio_ccw_private.h
> @@ -98,7 +98,6 @@ struct vfio_ccw_private {
> struct completion *completion;
> atomic_t avail;
> struct mdev_device *mdev;
> - struct notifier_block nb;
Could you also remove this from the comment block above the struct?
Besides that, this is fine for -ccw.
Reviewed-by: Eric Farman <farman@linux.ibm.com>
> struct ccw_io_region *io_region;
> struct mutex io_mutex;
> struct vfio_ccw_region *region;
>
...snip...
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
2022-06-08 11:26 ` Jason Gunthorpe
(?)
@ 2022-06-08 23:50 ` Tian, Kevin
-1 siblings, 0 replies; 34+ messages in thread
From: Tian, Kevin @ 2022-06-08 23:50 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alexander Gordeev, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Vivi, Rodrigo, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Wang, Zhi A,
Christoph Hellwig
> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, June 8, 2022 7:27 PM
>
> On Wed, Jun 08, 2022 at 03:47:12AM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe
> > > Sent: Wednesday, June 8, 2022 7:02 AM
> > >
> > > Instead of bouncing the function call to the driver op through a blocking
> > > notifier just have the iommu layer call it directly.
> > >
> > > Register each device that is being attached to the iommu with the lower
> > > driver which then threads them on a linked list and calls the appropriate
> > > driver op at the right time.
> > >
> > > Currently the only use is if dma_unmap() is defined.
> > >
> > > Also, fully lock all the debugging tests on the pinning path that a
> > > dma_unmap is registered.
> > >
> > ...
> > > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> > > diff --git a/drivers/vfio/vfio_iommu_type1.c
> > > b/drivers/vfio/vfio_iommu_type1.c
> > > index c13b9290e35759..4ddb1f1abd238b 100644
> > > +++ b/drivers/vfio/vfio_iommu_type1.c
> > > @@ -67,7 +67,8 @@ struct vfio_iommu {
> > > struct list_head iova_list;
> > > struct mutex lock;
> > > struct rb_root dma_list;
> > > - struct blocking_notifier_head notifier;
> > > + struct list_head device_list;
> > > + struct mutex device_list_lock;
> >
> > emulated_device_list or unmap_device_list, to be more accurate?
>
> I don't think it is worth getting too specific here, the fact it only
> has dma_unmap devices is a bit of a shortcut to save some work.
>
> > btw a few lines downward there is already a emulated_iommu_groups.
> > the two sounds a bit overlapping given mdev's iommu group is faked
> > and singleton. Wonder whether it's cleaner to just reuse the existing field...
>
> The locking is more complicated on this since we still have check
> every device in the group if it is opened or not while launching the
> callback and prevent it from opening/closing while the callback is
> running.
>
> Since I plan to delete the dirty tracking which will drop the
> emulated_iommu_groups too I would leave it like this.
>
Make sense.
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
@ 2022-06-08 23:50 ` Tian, Kevin
0 siblings, 0 replies; 34+ messages in thread
From: Tian, Kevin @ 2022-06-08 23:50 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Wang, Zhi A, Jason Herne,
Eric Farman, Vasily Gorbik, Heiko Carstens, Alex Williamson,
Harald Freudenberger, Vivi, Rodrigo, intel-gvt-dev, Tony Krowiak,
Tvrtko Ursulin, Cornelia Huck, Peter Oberparleiter,
Sven Schnelle
> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, June 8, 2022 7:27 PM
>
> On Wed, Jun 08, 2022 at 03:47:12AM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe
> > > Sent: Wednesday, June 8, 2022 7:02 AM
> > >
> > > Instead of bouncing the function call to the driver op through a blocking
> > > notifier just have the iommu layer call it directly.
> > >
> > > Register each device that is being attached to the iommu with the lower
> > > driver which then threads them on a linked list and calls the appropriate
> > > driver op at the right time.
> > >
> > > Currently the only use is if dma_unmap() is defined.
> > >
> > > Also, fully lock all the debugging tests on the pinning path that a
> > > dma_unmap is registered.
> > >
> > ...
> > > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> > > diff --git a/drivers/vfio/vfio_iommu_type1.c
> > > b/drivers/vfio/vfio_iommu_type1.c
> > > index c13b9290e35759..4ddb1f1abd238b 100644
> > > +++ b/drivers/vfio/vfio_iommu_type1.c
> > > @@ -67,7 +67,8 @@ struct vfio_iommu {
> > > struct list_head iova_list;
> > > struct mutex lock;
> > > struct rb_root dma_list;
> > > - struct blocking_notifier_head notifier;
> > > + struct list_head device_list;
> > > + struct mutex device_list_lock;
> >
> > emulated_device_list or unmap_device_list, to be more accurate?
>
> I don't think it is worth getting too specific here, the fact it only
> has dma_unmap devices is a bit of a shortcut to save some work.
>
> > btw a few lines downward there is already a emulated_iommu_groups.
> > the two sounds a bit overlapping given mdev's iommu group is faked
> > and singleton. Wonder whether it's cleaner to just reuse the existing field...
>
> The locking is more complicated on this since we still have check
> every device in the group if it is opened or not while launching the
> callback and prevent it from opening/closing while the callback is
> running.
>
> Since I plan to delete the dirty tracking which will drop the
> emulated_iommu_groups too I would leave it like this.
>
Make sense.
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Intel-gfx] [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
@ 2022-06-08 23:50 ` Tian, Kevin
0 siblings, 0 replies; 34+ messages in thread
From: Tian, Kevin @ 2022-06-08 23:50 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Jason Herne, Eric Farman,
Vasily Gorbik, Heiko Carstens, Harald Freudenberger, Vivi,
Rodrigo, intel-gvt-dev, Tony Krowiak, Cornelia Huck,
Peter Oberparleiter, Sven Schnelle
> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, June 8, 2022 7:27 PM
>
> On Wed, Jun 08, 2022 at 03:47:12AM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe
> > > Sent: Wednesday, June 8, 2022 7:02 AM
> > >
> > > Instead of bouncing the function call to the driver op through a blocking
> > > notifier just have the iommu layer call it directly.
> > >
> > > Register each device that is being attached to the iommu with the lower
> > > driver which then threads them on a linked list and calls the appropriate
> > > driver op at the right time.
> > >
> > > Currently the only use is if dma_unmap() is defined.
> > >
> > > Also, fully lock all the debugging tests on the pinning path that a
> > > dma_unmap is registered.
> > >
> > ...
> > > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> > > diff --git a/drivers/vfio/vfio_iommu_type1.c
> > > b/drivers/vfio/vfio_iommu_type1.c
> > > index c13b9290e35759..4ddb1f1abd238b 100644
> > > +++ b/drivers/vfio/vfio_iommu_type1.c
> > > @@ -67,7 +67,8 @@ struct vfio_iommu {
> > > struct list_head iova_list;
> > > struct mutex lock;
> > > struct rb_root dma_list;
> > > - struct blocking_notifier_head notifier;
> > > + struct list_head device_list;
> > > + struct mutex device_list_lock;
> >
> > emulated_device_list or unmap_device_list, to be more accurate?
>
> I don't think it is worth getting too specific here, the fact it only
> has dma_unmap devices is a bit of a shortcut to save some work.
>
> > btw a few lines downward there is already a emulated_iommu_groups.
> > the two sounds a bit overlapping given mdev's iommu group is faked
> > and singleton. Wonder whether it's cleaner to just reuse the existing field...
>
> The locking is more complicated on this since we still have check
> every device in the group if it is opened or not while launching the
> callback and prevent it from opening/closing while the callback is
> running.
>
> Since I plan to delete the dirty tracking which will drop the
> emulated_iommu_groups too I would leave it like this.
>
Make sense.
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
@ 2022-06-09 13:21 ` Tony Krowiak
-1 siblings, 0 replies; 34+ messages in thread
From: Tony Krowiak @ 2022-06-09 13:21 UTC (permalink / raw)
To: Jason Gunthorpe, Alexander Gordeev, David Airlie,
Alex Williamson, Christian Borntraeger, Cornelia Huck,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Jason Herne, Joonas Lahtinen, kvm, linux-s390,
Matthew Rosato, Peter Oberparleiter, Halil Pasic, Rodrigo Vivi,
Sven Schnelle, Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang,
Zhi Wang
Cc: Christoph Hellwig
Reviewed-by: Tony Krowiak <akrowiak@linux.ibm.com>
On 6/7/22 7:02 PM, Jason Gunthorpe wrote:
> Instead of having drivers register the notifier with explicit code just
> have them provide a dma_unmap callback op in their driver ops and rely on
> the core code to wire it up.
>
> Suggested-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
> drivers/gpu/drm/i915/gvt/gvt.h | 1 -
> drivers/gpu/drm/i915/gvt/kvmgt.c | 75 ++++-----------
> drivers/s390/cio/vfio_ccw_ops.c | 41 ++-------
> drivers/s390/cio/vfio_ccw_private.h | 1 -
> drivers/s390/crypto/vfio_ap_ops.c | 53 ++---------
> drivers/s390/crypto/vfio_ap_private.h | 3 -
> drivers/vfio/vfio.c | 126 +++++++++-----------------
> drivers/vfio/vfio.h | 5 +
> include/linux/vfio.h | 21 +----
> 9 files changed, 87 insertions(+), 239 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
> index aee1a45da74bcb..705689e6401197 100644
> --- a/drivers/gpu/drm/i915/gvt/gvt.h
> +++ b/drivers/gpu/drm/i915/gvt/gvt.h
> @@ -226,7 +226,6 @@ struct intel_vgpu {
> unsigned long nr_cache_entries;
> struct mutex cache_lock;
>
> - struct notifier_block iommu_notifier;
> atomic_t released;
>
> struct kvm_page_track_notifier_node track_node;
> diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
> index e2f6c56ab3420c..ecd5bb37b63a2a 100644
> --- a/drivers/gpu/drm/i915/gvt/kvmgt.c
> +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
> @@ -729,34 +729,25 @@ int intel_gvt_set_edid(struct intel_vgpu *vgpu, int port_num)
> return ret;
> }
>
> -static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
> - unsigned long action, void *data)
> +static void intel_vgpu_dma_unmap(struct vfio_device *vfio_dev, u64 iova,
> + u64 length)
> {
> - struct intel_vgpu *vgpu =
> - container_of(nb, struct intel_vgpu, iommu_notifier);
> -
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> - struct gvt_dma *entry;
> - unsigned long iov_pfn, end_iov_pfn;
> + struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> + struct gvt_dma *entry;
> + u64 iov_pfn = iova >> PAGE_SHIFT;
> + u64 end_iov_pfn = iov_pfn + length / PAGE_SIZE;
>
> - iov_pfn = unmap->iova >> PAGE_SHIFT;
> - end_iov_pfn = iov_pfn + unmap->size / PAGE_SIZE;
> + mutex_lock(&vgpu->cache_lock);
> + for (; iov_pfn < end_iov_pfn; iov_pfn++) {
> + entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
> + if (!entry)
> + continue;
>
> - mutex_lock(&vgpu->cache_lock);
> - for (; iov_pfn < end_iov_pfn; iov_pfn++) {
> - entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
> - if (!entry)
> - continue;
> -
> - gvt_dma_unmap_page(vgpu, entry->gfn, entry->dma_addr,
> - entry->size);
> - __gvt_cache_remove_entry(vgpu, entry);
> - }
> - mutex_unlock(&vgpu->cache_lock);
> + gvt_dma_unmap_page(vgpu, entry->gfn, entry->dma_addr,
> + entry->size);
> + __gvt_cache_remove_entry(vgpu, entry);
> }
> -
> - return NOTIFY_OK;
> + mutex_unlock(&vgpu->cache_lock);
> }
>
> static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu)
> @@ -783,36 +774,20 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu)
> static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
> {
> struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> - unsigned long events;
> - int ret;
> -
> - vgpu->iommu_notifier.notifier_call = intel_vgpu_iommu_notifier;
>
> - events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> - ret = vfio_register_notifier(vfio_dev, VFIO_IOMMU_NOTIFY, &events,
> - &vgpu->iommu_notifier);
> - if (ret != 0) {
> - gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n",
> - ret);
> - goto out;
> - }
> -
> - ret = -EEXIST;
> if (vgpu->attached)
> - goto undo_iommu;
> + return -EEXIST;
>
> - ret = -ESRCH;
> if (!vgpu->vfio_device.kvm ||
> vgpu->vfio_device.kvm->mm != current->mm) {
> gvt_vgpu_err("KVM is required to use Intel vGPU\n");
> - goto undo_iommu;
> + return -ESRCH;
> }
>
> kvm_get_kvm(vgpu->vfio_device.kvm);
>
> - ret = -EEXIST;
> if (__kvmgt_vgpu_exist(vgpu))
> - goto undo_iommu;
> + return -EEXIST;
>
> vgpu->attached = true;
>
> @@ -831,12 +806,6 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
>
> atomic_set(&vgpu->released, 0);
> return 0;
> -
> -undo_iommu:
> - vfio_unregister_notifier(vfio_dev, VFIO_IOMMU_NOTIFY,
> - &vgpu->iommu_notifier);
> -out:
> - return ret;
> }
>
> static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
> @@ -853,8 +822,6 @@ static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
> static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
> {
> struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> - struct drm_i915_private *i915 = vgpu->gvt->gt->i915;
> - int ret;
>
> if (!vgpu->attached)
> return;
> @@ -864,11 +831,6 @@ static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
>
> intel_gvt_release_vgpu(vgpu);
>
> - ret = vfio_unregister_notifier(&vgpu->vfio_device, VFIO_IOMMU_NOTIFY,
> - &vgpu->iommu_notifier);
> - drm_WARN(&i915->drm, ret,
> - "vfio_unregister_notifier for iommu failed: %d\n", ret);
> -
> debugfs_remove(debugfs_lookup(KVMGT_DEBUGFS_FILENAME, vgpu->debugfs));
>
> kvm_page_track_unregister_notifier(vgpu->vfio_device.kvm,
> @@ -1610,6 +1572,7 @@ static const struct vfio_device_ops intel_vgpu_dev_ops = {
> .write = intel_vgpu_write,
> .mmap = intel_vgpu_mmap,
> .ioctl = intel_vgpu_ioctl,
> + .dma_unmap = intel_vgpu_dma_unmap,
> };
>
> static int intel_vgpu_probe(struct mdev_device *mdev)
> diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c
> index b49e2e9db2dc6f..09e0ce7b72324c 100644
> --- a/drivers/s390/cio/vfio_ccw_ops.c
> +++ b/drivers/s390/cio/vfio_ccw_ops.c
> @@ -44,31 +44,19 @@ static int vfio_ccw_mdev_reset(struct vfio_ccw_private *private)
> return ret;
> }
>
> -static int vfio_ccw_mdev_notifier(struct notifier_block *nb,
> - unsigned long action,
> - void *data)
> +static void vfio_ccw_dma_unmap(struct vfio_device *vdev, u64 iova, u64 length)
> {
> struct vfio_ccw_private *private =
> - container_of(nb, struct vfio_ccw_private, nb);
> -
> - /*
> - * Vendor drivers MUST unpin pages in response to an
> - * invalidation.
> - */
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> -
> - if (!cp_iova_pinned(&private->cp, unmap->iova))
> - return NOTIFY_OK;
> + container_of(vdev, struct vfio_ccw_private, vdev);
>
> - if (vfio_ccw_mdev_reset(private))
> - return NOTIFY_BAD;
> + /* Drivers MUST unpin pages in response to an invalidation. */
> + if (!cp_iova_pinned(&private->cp, iova))
> + return;
>
> - cp_free(&private->cp);
> - return NOTIFY_OK;
> - }
> + if (vfio_ccw_mdev_reset(private))
> + return;
>
> - return NOTIFY_DONE;
> + cp_free(&private->cp);
> }
>
> static ssize_t name_show(struct mdev_type *mtype,
> @@ -178,19 +166,11 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev)
> {
> struct vfio_ccw_private *private =
> container_of(vdev, struct vfio_ccw_private, vdev);
> - unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> int ret;
>
> - private->nb.notifier_call = vfio_ccw_mdev_notifier;
> -
> - ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY,
> - &events, &private->nb);
> - if (ret)
> - return ret;
> -
> ret = vfio_ccw_register_async_dev_regions(private);
> if (ret)
> - goto out_unregister;
> + return ret;
>
> ret = vfio_ccw_register_schib_dev_regions(private);
> if (ret)
> @@ -204,7 +184,6 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev)
>
> out_unregister:
> vfio_ccw_unregister_dev_regions(private);
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
> return ret;
> }
>
> @@ -222,7 +201,6 @@ static void vfio_ccw_mdev_close_device(struct vfio_device *vdev)
>
> cp_free(&private->cp);
> vfio_ccw_unregister_dev_regions(private);
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
> }
>
> static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private *private,
> @@ -645,6 +623,7 @@ static const struct vfio_device_ops vfio_ccw_dev_ops = {
> .write = vfio_ccw_mdev_write,
> .ioctl = vfio_ccw_mdev_ioctl,
> .request = vfio_ccw_mdev_request,
> + .dma_unmap = vfio_ccw_dma_unmap,
> };
>
> struct mdev_driver vfio_ccw_mdev_driver = {
> diff --git a/drivers/s390/cio/vfio_ccw_private.h b/drivers/s390/cio/vfio_ccw_private.h
> index 7272eb78861244..2627791c9006d4 100644
> --- a/drivers/s390/cio/vfio_ccw_private.h
> +++ b/drivers/s390/cio/vfio_ccw_private.h
> @@ -98,7 +98,6 @@ struct vfio_ccw_private {
> struct completion *completion;
> atomic_t avail;
> struct mdev_device *mdev;
> - struct notifier_block nb;
> struct ccw_io_region *io_region;
> struct mutex io_mutex;
> struct vfio_ccw_region *region;
> diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
> index a7d2a95796d360..bb1a1677c5c230 100644
> --- a/drivers/s390/crypto/vfio_ap_ops.c
> +++ b/drivers/s390/crypto/vfio_ap_ops.c
> @@ -1226,34 +1226,14 @@ static int vfio_ap_mdev_set_kvm(struct ap_matrix_mdev *matrix_mdev,
> return 0;
> }
>
> -/**
> - * vfio_ap_mdev_iommu_notifier - IOMMU notifier callback
> - *
> - * @nb: The notifier block
> - * @action: Action to be taken
> - * @data: data associated with the request
> - *
> - * For an UNMAP request, unpin the guest IOVA (the NIB guest address we
> - * pinned before). Other requests are ignored.
> - *
> - * Return: for an UNMAP request, NOFITY_OK; otherwise NOTIFY_DONE.
> - */
> -static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb,
> - unsigned long action, void *data)
> +static void vfio_ap_mdev_dma_unmap(struct vfio_device *vdev, u64 iova,
> + u64 length)
> {
> - struct ap_matrix_mdev *matrix_mdev;
> -
> - matrix_mdev = container_of(nb, struct ap_matrix_mdev, iommu_notifier);
> -
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> - unsigned long g_pfn = unmap->iova >> PAGE_SHIFT;
> -
> - vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
> - return NOTIFY_OK;
> - }
> + struct ap_matrix_mdev *matrix_mdev =
> + container_of(vdev, struct ap_matrix_mdev, vdev);
> + unsigned long g_pfn = iova >> PAGE_SHIFT;
>
> - return NOTIFY_DONE;
> + vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
> }
>
> /**
> @@ -1380,27 +1360,11 @@ static int vfio_ap_mdev_open_device(struct vfio_device *vdev)
> {
> struct ap_matrix_mdev *matrix_mdev =
> container_of(vdev, struct ap_matrix_mdev, vdev);
> - unsigned long events;
> - int ret;
>
> if (!vdev->kvm)
> return -EINVAL;
>
> - ret = vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
> - if (ret)
> - return ret;
> -
> - matrix_mdev->iommu_notifier.notifier_call = vfio_ap_mdev_iommu_notifier;
> - events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> - ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY, &events,
> - &matrix_mdev->iommu_notifier);
> - if (ret)
> - goto err_kvm;
> - return 0;
> -
> -err_kvm:
> - vfio_ap_mdev_unset_kvm(matrix_mdev);
> - return ret;
> + return vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
> }
>
> static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
> @@ -1408,8 +1372,6 @@ static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
> struct ap_matrix_mdev *matrix_mdev =
> container_of(vdev, struct ap_matrix_mdev, vdev);
>
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY,
> - &matrix_mdev->iommu_notifier);
> vfio_ap_mdev_unset_kvm(matrix_mdev);
> }
>
> @@ -1461,6 +1423,7 @@ static const struct vfio_device_ops vfio_ap_matrix_dev_ops = {
> .open_device = vfio_ap_mdev_open_device,
> .close_device = vfio_ap_mdev_close_device,
> .ioctl = vfio_ap_mdev_ioctl,
> + .dma_unmap = vfio_ap_mdev_dma_unmap,
> };
>
> static struct mdev_driver vfio_ap_matrix_driver = {
> diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h
> index a26efd804d0df3..abb59d59f81b20 100644
> --- a/drivers/s390/crypto/vfio_ap_private.h
> +++ b/drivers/s390/crypto/vfio_ap_private.h
> @@ -81,8 +81,6 @@ struct ap_matrix {
> * @node: allows the ap_matrix_mdev struct to be added to a list
> * @matrix: the adapters, usage domains and control domains assigned to the
> * mediated matrix device.
> - * @iommu_notifier: notifier block used for specifying callback function for
> - * handling the VFIO_IOMMU_NOTIFY_DMA_UNMAP even
> * @kvm: the struct holding guest's state
> * @pqap_hook: the function pointer to the interception handler for the
> * PQAP(AQIC) instruction.
> @@ -92,7 +90,6 @@ struct ap_matrix_mdev {
> struct vfio_device vdev;
> struct list_head node;
> struct ap_matrix matrix;
> - struct notifier_block iommu_notifier;
> struct kvm *kvm;
> crypto_hook pqap_hook;
> struct mdev_device *mdev;
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 61e71c1154be67..f005b644ab9e69 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> up_write(&device->group->group_rwsem);
> }
>
> +static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> + void *data)
> +{
> + struct vfio_device *vfio_device =
> + container_of(nb, struct vfio_device, iommu_nb);
> + struct vfio_iommu_type1_dma_unmap *unmap = data;
> +
> + vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> + return NOTIFY_OK;
> +}
> +
> static struct file *vfio_device_open(struct vfio_device *device)
> {
> + struct vfio_iommu_driver *iommu_driver;
> struct file *filep;
> int ret;
>
> @@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct vfio_device *device)
> if (ret)
> goto err_undo_count;
> }
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier) {
> + unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> +
> + device->iommu_nb.notifier_call = vfio_iommu_notifier;
> + iommu_driver->ops->register_notifier(
> + device->group->container->iommu_data, &events,
> + &device->iommu_nb);
> + }
> +
> up_read(&device->group->group_rwsem);
> }
> mutex_unlock(&device->dev_set->lock);
> @@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct vfio_device *device)
> err_close_device:
> mutex_lock(&device->dev_set->lock);
> down_read(&device->group->group_rwsem);
> - if (device->open_count == 1 && device->ops->close_device)
> + if (device->open_count == 1 && device->ops->close_device) {
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> + }
> err_undo_count:
> device->open_count--;
> if (device->open_count == 0 && device->kvm)
> @@ -1339,12 +1371,20 @@ static const struct file_operations vfio_group_fops = {
> static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> {
> struct vfio_device *device = filep->private_data;
> + struct vfio_iommu_driver *iommu_driver;
>
> mutex_lock(&device->dev_set->lock);
> vfio_assert_device_open(device);
> down_read(&device->group->group_rwsem);
> if (device->open_count == 1 && device->ops->close_device)
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> up_read(&device->group->group_rwsem);
> device->open_count--;
> if (device->open_count == 0)
> @@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data,
> }
> EXPORT_SYMBOL(vfio_dma_rw);
>
> -static int vfio_register_iommu_notifier(struct vfio_group *group,
> - unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->register_notifier))
> - ret = driver->ops->register_notifier(container->iommu_data,
> - events, nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -static int vfio_unregister_iommu_notifier(struct vfio_group *group,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->unregister_notifier))
> - ret = driver->ops->unregister_notifier(container->iommu_data,
> - nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type, unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !events || (*events == 0) ||
> - !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_register_iommu_notifier(group, events, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_register_notifier);
> -
> -int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_unregister_iommu_notifier(group, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_unregister_notifier);
> -
> /*
> * Module/class support
> */
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index a6713022115155..cb2e4e9baa8fe8 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
> VFIO_IOMMU_CONTAINER_CLOSE = 0,
> };
>
> +/* events for register_notifier() */
> +enum {
> + VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> +};
> +
> /**
> * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> */
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index aa888cc517578e..b76623e3b92fca 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -44,6 +44,7 @@ struct vfio_device {
> unsigned int open_count;
> struct completion comp;
> struct list_head group_next;
> + struct notifier_block iommu_nb;
> };
>
> /**
> @@ -60,6 +61,8 @@ struct vfio_device {
> * @match: Optional device name match callback (return: 0 for no-match, >0 for
> * match, -errno for abort (ex. match with insufficient or incorrect
> * additional args)
> + * @dma_unmap: Called when userspace unmaps IOVA from the container
> + * this device is attached to.
> * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
> * @migration_set_state: Optional callback to change the migration state for
> * devices that support migration. It's mandatory for
> @@ -85,6 +88,7 @@ struct vfio_device_ops {
> int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
> void (*request)(struct vfio_device *vdev, unsigned int count);
> int (*match)(struct vfio_device *vdev, char *buf);
> + void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
> int (*device_feature)(struct vfio_device *device, u32 flags,
> void __user *arg, size_t argsz);
> struct file *(*migration_set_state)(
> @@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn,
> extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
> void *data, size_t len, bool write);
>
> -/* each type has independent events */
> -enum vfio_notify_type {
> - VFIO_IOMMU_NOTIFY = 0,
> -};
> -
> -/* events for VFIO_IOMMU_NOTIFY */
> -#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
> -
> -extern int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - unsigned long *required_events,
> - struct notifier_block *nb);
> -extern int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb);
> -
> -
> /*
> * Sub-module helpers
> */
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Intel-gfx] [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
@ 2022-06-09 13:21 ` Tony Krowiak
0 siblings, 0 replies; 34+ messages in thread
From: Tony Krowiak @ 2022-06-09 13:21 UTC (permalink / raw)
To: Jason Gunthorpe, Alexander Gordeev, David Airlie,
Alex Williamson, Christian Borntraeger, Cornelia Huck,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Jason Herne, Joonas Lahtinen, kvm, linux-s390,
Matthew Rosato, Peter Oberparleiter, Halil Pasic, Rodrigo Vivi,
Sven Schnelle, Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang,
Zhi Wang
Cc: Christoph Hellwig
Reviewed-by: Tony Krowiak <akrowiak@linux.ibm.com>
On 6/7/22 7:02 PM, Jason Gunthorpe wrote:
> Instead of having drivers register the notifier with explicit code just
> have them provide a dma_unmap callback op in their driver ops and rely on
> the core code to wire it up.
>
> Suggested-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
> drivers/gpu/drm/i915/gvt/gvt.h | 1 -
> drivers/gpu/drm/i915/gvt/kvmgt.c | 75 ++++-----------
> drivers/s390/cio/vfio_ccw_ops.c | 41 ++-------
> drivers/s390/cio/vfio_ccw_private.h | 1 -
> drivers/s390/crypto/vfio_ap_ops.c | 53 ++---------
> drivers/s390/crypto/vfio_ap_private.h | 3 -
> drivers/vfio/vfio.c | 126 +++++++++-----------------
> drivers/vfio/vfio.h | 5 +
> include/linux/vfio.h | 21 +----
> 9 files changed, 87 insertions(+), 239 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h
> index aee1a45da74bcb..705689e6401197 100644
> --- a/drivers/gpu/drm/i915/gvt/gvt.h
> +++ b/drivers/gpu/drm/i915/gvt/gvt.h
> @@ -226,7 +226,6 @@ struct intel_vgpu {
> unsigned long nr_cache_entries;
> struct mutex cache_lock;
>
> - struct notifier_block iommu_notifier;
> atomic_t released;
>
> struct kvm_page_track_notifier_node track_node;
> diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
> index e2f6c56ab3420c..ecd5bb37b63a2a 100644
> --- a/drivers/gpu/drm/i915/gvt/kvmgt.c
> +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
> @@ -729,34 +729,25 @@ int intel_gvt_set_edid(struct intel_vgpu *vgpu, int port_num)
> return ret;
> }
>
> -static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
> - unsigned long action, void *data)
> +static void intel_vgpu_dma_unmap(struct vfio_device *vfio_dev, u64 iova,
> + u64 length)
> {
> - struct intel_vgpu *vgpu =
> - container_of(nb, struct intel_vgpu, iommu_notifier);
> -
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> - struct gvt_dma *entry;
> - unsigned long iov_pfn, end_iov_pfn;
> + struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> + struct gvt_dma *entry;
> + u64 iov_pfn = iova >> PAGE_SHIFT;
> + u64 end_iov_pfn = iov_pfn + length / PAGE_SIZE;
>
> - iov_pfn = unmap->iova >> PAGE_SHIFT;
> - end_iov_pfn = iov_pfn + unmap->size / PAGE_SIZE;
> + mutex_lock(&vgpu->cache_lock);
> + for (; iov_pfn < end_iov_pfn; iov_pfn++) {
> + entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
> + if (!entry)
> + continue;
>
> - mutex_lock(&vgpu->cache_lock);
> - for (; iov_pfn < end_iov_pfn; iov_pfn++) {
> - entry = __gvt_cache_find_gfn(vgpu, iov_pfn);
> - if (!entry)
> - continue;
> -
> - gvt_dma_unmap_page(vgpu, entry->gfn, entry->dma_addr,
> - entry->size);
> - __gvt_cache_remove_entry(vgpu, entry);
> - }
> - mutex_unlock(&vgpu->cache_lock);
> + gvt_dma_unmap_page(vgpu, entry->gfn, entry->dma_addr,
> + entry->size);
> + __gvt_cache_remove_entry(vgpu, entry);
> }
> -
> - return NOTIFY_OK;
> + mutex_unlock(&vgpu->cache_lock);
> }
>
> static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu)
> @@ -783,36 +774,20 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu)
> static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
> {
> struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> - unsigned long events;
> - int ret;
> -
> - vgpu->iommu_notifier.notifier_call = intel_vgpu_iommu_notifier;
>
> - events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> - ret = vfio_register_notifier(vfio_dev, VFIO_IOMMU_NOTIFY, &events,
> - &vgpu->iommu_notifier);
> - if (ret != 0) {
> - gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n",
> - ret);
> - goto out;
> - }
> -
> - ret = -EEXIST;
> if (vgpu->attached)
> - goto undo_iommu;
> + return -EEXIST;
>
> - ret = -ESRCH;
> if (!vgpu->vfio_device.kvm ||
> vgpu->vfio_device.kvm->mm != current->mm) {
> gvt_vgpu_err("KVM is required to use Intel vGPU\n");
> - goto undo_iommu;
> + return -ESRCH;
> }
>
> kvm_get_kvm(vgpu->vfio_device.kvm);
>
> - ret = -EEXIST;
> if (__kvmgt_vgpu_exist(vgpu))
> - goto undo_iommu;
> + return -EEXIST;
>
> vgpu->attached = true;
>
> @@ -831,12 +806,6 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev)
>
> atomic_set(&vgpu->released, 0);
> return 0;
> -
> -undo_iommu:
> - vfio_unregister_notifier(vfio_dev, VFIO_IOMMU_NOTIFY,
> - &vgpu->iommu_notifier);
> -out:
> - return ret;
> }
>
> static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
> @@ -853,8 +822,6 @@ static void intel_vgpu_release_msi_eventfd_ctx(struct intel_vgpu *vgpu)
> static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
> {
> struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev);
> - struct drm_i915_private *i915 = vgpu->gvt->gt->i915;
> - int ret;
>
> if (!vgpu->attached)
> return;
> @@ -864,11 +831,6 @@ static void intel_vgpu_close_device(struct vfio_device *vfio_dev)
>
> intel_gvt_release_vgpu(vgpu);
>
> - ret = vfio_unregister_notifier(&vgpu->vfio_device, VFIO_IOMMU_NOTIFY,
> - &vgpu->iommu_notifier);
> - drm_WARN(&i915->drm, ret,
> - "vfio_unregister_notifier for iommu failed: %d\n", ret);
> -
> debugfs_remove(debugfs_lookup(KVMGT_DEBUGFS_FILENAME, vgpu->debugfs));
>
> kvm_page_track_unregister_notifier(vgpu->vfio_device.kvm,
> @@ -1610,6 +1572,7 @@ static const struct vfio_device_ops intel_vgpu_dev_ops = {
> .write = intel_vgpu_write,
> .mmap = intel_vgpu_mmap,
> .ioctl = intel_vgpu_ioctl,
> + .dma_unmap = intel_vgpu_dma_unmap,
> };
>
> static int intel_vgpu_probe(struct mdev_device *mdev)
> diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c
> index b49e2e9db2dc6f..09e0ce7b72324c 100644
> --- a/drivers/s390/cio/vfio_ccw_ops.c
> +++ b/drivers/s390/cio/vfio_ccw_ops.c
> @@ -44,31 +44,19 @@ static int vfio_ccw_mdev_reset(struct vfio_ccw_private *private)
> return ret;
> }
>
> -static int vfio_ccw_mdev_notifier(struct notifier_block *nb,
> - unsigned long action,
> - void *data)
> +static void vfio_ccw_dma_unmap(struct vfio_device *vdev, u64 iova, u64 length)
> {
> struct vfio_ccw_private *private =
> - container_of(nb, struct vfio_ccw_private, nb);
> -
> - /*
> - * Vendor drivers MUST unpin pages in response to an
> - * invalidation.
> - */
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> -
> - if (!cp_iova_pinned(&private->cp, unmap->iova))
> - return NOTIFY_OK;
> + container_of(vdev, struct vfio_ccw_private, vdev);
>
> - if (vfio_ccw_mdev_reset(private))
> - return NOTIFY_BAD;
> + /* Drivers MUST unpin pages in response to an invalidation. */
> + if (!cp_iova_pinned(&private->cp, iova))
> + return;
>
> - cp_free(&private->cp);
> - return NOTIFY_OK;
> - }
> + if (vfio_ccw_mdev_reset(private))
> + return;
>
> - return NOTIFY_DONE;
> + cp_free(&private->cp);
> }
>
> static ssize_t name_show(struct mdev_type *mtype,
> @@ -178,19 +166,11 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev)
> {
> struct vfio_ccw_private *private =
> container_of(vdev, struct vfio_ccw_private, vdev);
> - unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> int ret;
>
> - private->nb.notifier_call = vfio_ccw_mdev_notifier;
> -
> - ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY,
> - &events, &private->nb);
> - if (ret)
> - return ret;
> -
> ret = vfio_ccw_register_async_dev_regions(private);
> if (ret)
> - goto out_unregister;
> + return ret;
>
> ret = vfio_ccw_register_schib_dev_regions(private);
> if (ret)
> @@ -204,7 +184,6 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev)
>
> out_unregister:
> vfio_ccw_unregister_dev_regions(private);
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
> return ret;
> }
>
> @@ -222,7 +201,6 @@ static void vfio_ccw_mdev_close_device(struct vfio_device *vdev)
>
> cp_free(&private->cp);
> vfio_ccw_unregister_dev_regions(private);
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb);
> }
>
> static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private *private,
> @@ -645,6 +623,7 @@ static const struct vfio_device_ops vfio_ccw_dev_ops = {
> .write = vfio_ccw_mdev_write,
> .ioctl = vfio_ccw_mdev_ioctl,
> .request = vfio_ccw_mdev_request,
> + .dma_unmap = vfio_ccw_dma_unmap,
> };
>
> struct mdev_driver vfio_ccw_mdev_driver = {
> diff --git a/drivers/s390/cio/vfio_ccw_private.h b/drivers/s390/cio/vfio_ccw_private.h
> index 7272eb78861244..2627791c9006d4 100644
> --- a/drivers/s390/cio/vfio_ccw_private.h
> +++ b/drivers/s390/cio/vfio_ccw_private.h
> @@ -98,7 +98,6 @@ struct vfio_ccw_private {
> struct completion *completion;
> atomic_t avail;
> struct mdev_device *mdev;
> - struct notifier_block nb;
> struct ccw_io_region *io_region;
> struct mutex io_mutex;
> struct vfio_ccw_region *region;
> diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
> index a7d2a95796d360..bb1a1677c5c230 100644
> --- a/drivers/s390/crypto/vfio_ap_ops.c
> +++ b/drivers/s390/crypto/vfio_ap_ops.c
> @@ -1226,34 +1226,14 @@ static int vfio_ap_mdev_set_kvm(struct ap_matrix_mdev *matrix_mdev,
> return 0;
> }
>
> -/**
> - * vfio_ap_mdev_iommu_notifier - IOMMU notifier callback
> - *
> - * @nb: The notifier block
> - * @action: Action to be taken
> - * @data: data associated with the request
> - *
> - * For an UNMAP request, unpin the guest IOVA (the NIB guest address we
> - * pinned before). Other requests are ignored.
> - *
> - * Return: for an UNMAP request, NOFITY_OK; otherwise NOTIFY_DONE.
> - */
> -static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb,
> - unsigned long action, void *data)
> +static void vfio_ap_mdev_dma_unmap(struct vfio_device *vdev, u64 iova,
> + u64 length)
> {
> - struct ap_matrix_mdev *matrix_mdev;
> -
> - matrix_mdev = container_of(nb, struct ap_matrix_mdev, iommu_notifier);
> -
> - if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) {
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> - unsigned long g_pfn = unmap->iova >> PAGE_SHIFT;
> -
> - vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
> - return NOTIFY_OK;
> - }
> + struct ap_matrix_mdev *matrix_mdev =
> + container_of(vdev, struct ap_matrix_mdev, vdev);
> + unsigned long g_pfn = iova >> PAGE_SHIFT;
>
> - return NOTIFY_DONE;
> + vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1);
> }
>
> /**
> @@ -1380,27 +1360,11 @@ static int vfio_ap_mdev_open_device(struct vfio_device *vdev)
> {
> struct ap_matrix_mdev *matrix_mdev =
> container_of(vdev, struct ap_matrix_mdev, vdev);
> - unsigned long events;
> - int ret;
>
> if (!vdev->kvm)
> return -EINVAL;
>
> - ret = vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
> - if (ret)
> - return ret;
> -
> - matrix_mdev->iommu_notifier.notifier_call = vfio_ap_mdev_iommu_notifier;
> - events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> - ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY, &events,
> - &matrix_mdev->iommu_notifier);
> - if (ret)
> - goto err_kvm;
> - return 0;
> -
> -err_kvm:
> - vfio_ap_mdev_unset_kvm(matrix_mdev);
> - return ret;
> + return vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm);
> }
>
> static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
> @@ -1408,8 +1372,6 @@ static void vfio_ap_mdev_close_device(struct vfio_device *vdev)
> struct ap_matrix_mdev *matrix_mdev =
> container_of(vdev, struct ap_matrix_mdev, vdev);
>
> - vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY,
> - &matrix_mdev->iommu_notifier);
> vfio_ap_mdev_unset_kvm(matrix_mdev);
> }
>
> @@ -1461,6 +1423,7 @@ static const struct vfio_device_ops vfio_ap_matrix_dev_ops = {
> .open_device = vfio_ap_mdev_open_device,
> .close_device = vfio_ap_mdev_close_device,
> .ioctl = vfio_ap_mdev_ioctl,
> + .dma_unmap = vfio_ap_mdev_dma_unmap,
> };
>
> static struct mdev_driver vfio_ap_matrix_driver = {
> diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h
> index a26efd804d0df3..abb59d59f81b20 100644
> --- a/drivers/s390/crypto/vfio_ap_private.h
> +++ b/drivers/s390/crypto/vfio_ap_private.h
> @@ -81,8 +81,6 @@ struct ap_matrix {
> * @node: allows the ap_matrix_mdev struct to be added to a list
> * @matrix: the adapters, usage domains and control domains assigned to the
> * mediated matrix device.
> - * @iommu_notifier: notifier block used for specifying callback function for
> - * handling the VFIO_IOMMU_NOTIFY_DMA_UNMAP even
> * @kvm: the struct holding guest's state
> * @pqap_hook: the function pointer to the interception handler for the
> * PQAP(AQIC) instruction.
> @@ -92,7 +90,6 @@ struct ap_matrix_mdev {
> struct vfio_device vdev;
> struct list_head node;
> struct ap_matrix matrix;
> - struct notifier_block iommu_notifier;
> struct kvm *kvm;
> crypto_hook pqap_hook;
> struct mdev_device *mdev;
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 61e71c1154be67..f005b644ab9e69 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> up_write(&device->group->group_rwsem);
> }
>
> +static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> + void *data)
> +{
> + struct vfio_device *vfio_device =
> + container_of(nb, struct vfio_device, iommu_nb);
> + struct vfio_iommu_type1_dma_unmap *unmap = data;
> +
> + vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> + return NOTIFY_OK;
> +}
> +
> static struct file *vfio_device_open(struct vfio_device *device)
> {
> + struct vfio_iommu_driver *iommu_driver;
> struct file *filep;
> int ret;
>
> @@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct vfio_device *device)
> if (ret)
> goto err_undo_count;
> }
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier) {
> + unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> +
> + device->iommu_nb.notifier_call = vfio_iommu_notifier;
> + iommu_driver->ops->register_notifier(
> + device->group->container->iommu_data, &events,
> + &device->iommu_nb);
> + }
> +
> up_read(&device->group->group_rwsem);
> }
> mutex_unlock(&device->dev_set->lock);
> @@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct vfio_device *device)
> err_close_device:
> mutex_lock(&device->dev_set->lock);
> down_read(&device->group->group_rwsem);
> - if (device->open_count == 1 && device->ops->close_device)
> + if (device->open_count == 1 && device->ops->close_device) {
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> + }
> err_undo_count:
> device->open_count--;
> if (device->open_count == 0 && device->kvm)
> @@ -1339,12 +1371,20 @@ static const struct file_operations vfio_group_fops = {
> static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> {
> struct vfio_device *device = filep->private_data;
> + struct vfio_iommu_driver *iommu_driver;
>
> mutex_lock(&device->dev_set->lock);
> vfio_assert_device_open(device);
> down_read(&device->group->group_rwsem);
> if (device->open_count == 1 && device->ops->close_device)
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> up_read(&device->group->group_rwsem);
> device->open_count--;
> if (device->open_count == 0)
> @@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data,
> }
> EXPORT_SYMBOL(vfio_dma_rw);
>
> -static int vfio_register_iommu_notifier(struct vfio_group *group,
> - unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->register_notifier))
> - ret = driver->ops->register_notifier(container->iommu_data,
> - events, nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -static int vfio_unregister_iommu_notifier(struct vfio_group *group,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->unregister_notifier))
> - ret = driver->ops->unregister_notifier(container->iommu_data,
> - nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type, unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !events || (*events == 0) ||
> - !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_register_iommu_notifier(group, events, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_register_notifier);
> -
> -int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_unregister_iommu_notifier(group, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_unregister_notifier);
> -
> /*
> * Module/class support
> */
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index a6713022115155..cb2e4e9baa8fe8 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
> VFIO_IOMMU_CONTAINER_CLOSE = 0,
> };
>
> +/* events for register_notifier() */
> +enum {
> + VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> +};
> +
> /**
> * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> */
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index aa888cc517578e..b76623e3b92fca 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -44,6 +44,7 @@ struct vfio_device {
> unsigned int open_count;
> struct completion comp;
> struct list_head group_next;
> + struct notifier_block iommu_nb;
> };
>
> /**
> @@ -60,6 +61,8 @@ struct vfio_device {
> * @match: Optional device name match callback (return: 0 for no-match, >0 for
> * match, -errno for abort (ex. match with insufficient or incorrect
> * additional args)
> + * @dma_unmap: Called when userspace unmaps IOVA from the container
> + * this device is attached to.
> * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
> * @migration_set_state: Optional callback to change the migration state for
> * devices that support migration. It's mandatory for
> @@ -85,6 +88,7 @@ struct vfio_device_ops {
> int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
> void (*request)(struct vfio_device *vdev, unsigned int count);
> int (*match)(struct vfio_device *vdev, char *buf);
> + void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
> int (*device_feature)(struct vfio_device *device, u32 flags,
> void __user *arg, size_t argsz);
> struct file *(*migration_set_state)(
> @@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn,
> extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
> void *data, size_t len, bool write);
>
> -/* each type has independent events */
> -enum vfio_notify_type {
> - VFIO_IOMMU_NOTIFY = 0,
> -};
> -
> -/* events for VFIO_IOMMU_NOTIFY */
> -#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
> -
> -extern int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - unsigned long *required_events,
> - struct notifier_block *nb);
> -extern int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb);
> -
> -
> /*
> * Sub-module helpers
> */
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
2022-06-08 15:50 ` Eric Farman
2022-06-09 14:43 ` Jason Gunthorpe
@ 2022-06-09 14:43 ` Jason Gunthorpe
0 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-09 14:43 UTC (permalink / raw)
To: Eric Farman
Cc: Alexander Gordeev, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Harald Freudenberger, Vasily Gorbik, Heiko Carstens, intel-gfx,
intel-gvt-dev, Jani Nikula, Jason Herne, Joonas Lahtinen, kvm,
linux-s390, Matthew Rosato, Peter Oberparleiter, Halil Pasic,
Rodrigo Vivi, Sven Schnelle, Tvrtko Ursulin, Vineeth Vijayan,
Zhenyu Wang, Zhi Wang, Christoph Hellwig
On Wed, Jun 08, 2022 at 11:50:31AM -0400, Eric Farman wrote:
> > --- a/drivers/s390/cio/vfio_ccw_private.h
> > +++ b/drivers/s390/cio/vfio_ccw_private.h
> > @@ -98,7 +98,6 @@ struct vfio_ccw_private {
> > struct completion *completion;
> > atomic_t avail;
> > struct mdev_device *mdev;
> > - struct notifier_block nb;
>
> Could you also remove this from the comment block above the struct?
> Besides that, this is fine for -ccw.
Done, thanks
Jason
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
@ 2022-06-09 14:43 ` Jason Gunthorpe
0 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-09 14:43 UTC (permalink / raw)
To: Eric Farman
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Zhi Wang, Jason Herne,
Vasily Gorbik, Heiko Carstens, Alex Williamson,
Harald Freudenberger, Rodrigo Vivi, intel-gvt-dev, Tony Krowiak,
Tvrtko Ursulin, Cornelia Huck, Peter Oberparleiter,
Sven Schnelle
On Wed, Jun 08, 2022 at 11:50:31AM -0400, Eric Farman wrote:
> > --- a/drivers/s390/cio/vfio_ccw_private.h
> > +++ b/drivers/s390/cio/vfio_ccw_private.h
> > @@ -98,7 +98,6 @@ struct vfio_ccw_private {
> > struct completion *completion;
> > atomic_t avail;
> > struct mdev_device *mdev;
> > - struct notifier_block nb;
>
> Could you also remove this from the comment block above the struct?
> Besides that, this is fine for -ccw.
Done, thanks
Jason
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Intel-gfx] [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
@ 2022-06-09 14:43 ` Jason Gunthorpe
0 siblings, 0 replies; 34+ messages in thread
From: Jason Gunthorpe @ 2022-06-09 14:43 UTC (permalink / raw)
To: Eric Farman
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Jason Herne, Vasily Gorbik,
Heiko Carstens, Harald Freudenberger, Rodrigo Vivi,
intel-gvt-dev, Tony Krowiak, Cornelia Huck, Peter Oberparleiter,
Sven Schnelle
On Wed, Jun 08, 2022 at 11:50:31AM -0400, Eric Farman wrote:
> > --- a/drivers/s390/cio/vfio_ccw_private.h
> > +++ b/drivers/s390/cio/vfio_ccw_private.h
> > @@ -98,7 +98,6 @@ struct vfio_ccw_private {
> > struct completion *completion;
> > atomic_t avail;
> > struct mdev_device *mdev;
> > - struct notifier_block nb;
>
> Could you also remove this from the comment block above the struct?
> Besides that, this is fine for -ccw.
Done, thanks
Jason
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
(?)
@ 2022-06-17 22:42 ` Alex Williamson
-1 siblings, 0 replies; 34+ messages in thread
From: Alex Williamson @ 2022-06-17 22:42 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alexander Gordeev, David Airlie, Tony Krowiak,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang,
Christoph Hellwig
On Tue, 7 Jun 2022 20:02:11 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 61e71c1154be67..f005b644ab9e69 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> up_write(&device->group->group_rwsem);
> }
>
> +static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> + void *data)
> +{
> + struct vfio_device *vfio_device =
> + container_of(nb, struct vfio_device, iommu_nb);
> + struct vfio_iommu_type1_dma_unmap *unmap = data;
> +
> + vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> + return NOTIFY_OK;
> +}
> +
> static struct file *vfio_device_open(struct vfio_device *device)
> {
> + struct vfio_iommu_driver *iommu_driver;
> struct file *filep;
> int ret;
>
> @@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct vfio_device *device)
> if (ret)
> goto err_undo_count;
> }
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier) {
> + unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> +
> + device->iommu_nb.notifier_call = vfio_iommu_notifier;
> + iommu_driver->ops->register_notifier(
> + device->group->container->iommu_data, &events,
> + &device->iommu_nb);
> + }
> +
> up_read(&device->group->group_rwsem);
> }
> mutex_unlock(&device->dev_set->lock);
> @@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct vfio_device *device)
> err_close_device:
> mutex_lock(&device->dev_set->lock);
> down_read(&device->group->group_rwsem);
> - if (device->open_count == 1 && device->ops->close_device)
> + if (device->open_count == 1 && device->ops->close_device) {
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
Test for register_notifier callback...
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
use unregister_notifier callback. Same below.
> + }
> err_undo_count:
> device->open_count--;
> if (device->open_count == 0 && device->kvm)
> @@ -1339,12 +1371,20 @@ static const struct file_operations vfio_group_fops = {
> static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> {
> struct vfio_device *device = filep->private_data;
> + struct vfio_iommu_driver *iommu_driver;
>
> mutex_lock(&device->dev_set->lock);
> vfio_assert_device_open(device);
> down_read(&device->group->group_rwsem);
> if (device->open_count == 1 && device->ops->close_device)
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> up_read(&device->group->group_rwsem);
> device->open_count--;
> if (device->open_count == 0)
> @@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data,
> }
> EXPORT_SYMBOL(vfio_dma_rw);
>
> -static int vfio_register_iommu_notifier(struct vfio_group *group,
> - unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->register_notifier))
> - ret = driver->ops->register_notifier(container->iommu_data,
> - events, nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -static int vfio_unregister_iommu_notifier(struct vfio_group *group,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->unregister_notifier))
> - ret = driver->ops->unregister_notifier(container->iommu_data,
> - nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type, unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !events || (*events == 0) ||
> - !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_register_iommu_notifier(group, events, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_register_notifier);
> -
> -int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_unregister_iommu_notifier(group, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_unregister_notifier);
> -
> /*
> * Module/class support
> */
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index a6713022115155..cb2e4e9baa8fe8 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
> VFIO_IOMMU_CONTAINER_CLOSE = 0,
> };
>
> +/* events for register_notifier() */
> +enum {
> + VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> +};
Can't say I understand why this changed from BIT(0) to an enum, the
event mask is meant to be a bitfield. Using the notifier all the way
to the device was meant to avoid future callbacks on the device. If we
now have a dma_unmap on the device, should the whole infrastructure be
tailored to that one task? For example a dma_unmap_nb on the device,
{un}register_dma_unmap_notifier on the iommu ops,
vfio_dma_unmap_notifier, etc? Thanks,
Alex
> +
> /**
> * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> */
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index aa888cc517578e..b76623e3b92fca 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -44,6 +44,7 @@ struct vfio_device {
> unsigned int open_count;
> struct completion comp;
> struct list_head group_next;
> + struct notifier_block iommu_nb;
> };
>
> /**
> @@ -60,6 +61,8 @@ struct vfio_device {
> * @match: Optional device name match callback (return: 0 for no-match, >0 for
> * match, -errno for abort (ex. match with insufficient or incorrect
> * additional args)
> + * @dma_unmap: Called when userspace unmaps IOVA from the container
> + * this device is attached to.
> * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
> * @migration_set_state: Optional callback to change the migration state for
> * devices that support migration. It's mandatory for
> @@ -85,6 +88,7 @@ struct vfio_device_ops {
> int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
> void (*request)(struct vfio_device *vdev, unsigned int count);
> int (*match)(struct vfio_device *vdev, char *buf);
> + void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
> int (*device_feature)(struct vfio_device *device, u32 flags,
> void __user *arg, size_t argsz);
> struct file *(*migration_set_state)(
> @@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn,
> extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
> void *data, size_t len, bool write);
>
> -/* each type has independent events */
> -enum vfio_notify_type {
> - VFIO_IOMMU_NOTIFY = 0,
> -};
> -
> -/* events for VFIO_IOMMU_NOTIFY */
> -#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
> -
> -extern int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - unsigned long *required_events,
> - struct notifier_block *nb);
> -extern int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb);
> -
> -
> /*
> * Sub-module helpers
> */
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
@ 2022-06-17 22:42 ` Alex Williamson
0 siblings, 0 replies; 34+ messages in thread
From: Alex Williamson @ 2022-06-17 22:42 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Zhi Wang, Jason Herne,
Eric Farman, Vasily Gorbik, Heiko Carstens, Harald Freudenberger,
Rodrigo Vivi, intel-gvt-dev, Tony Krowiak, Tvrtko Ursulin,
Cornelia Huck, Peter Oberparleiter, Sven Schnelle
On Tue, 7 Jun 2022 20:02:11 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 61e71c1154be67..f005b644ab9e69 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> up_write(&device->group->group_rwsem);
> }
>
> +static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> + void *data)
> +{
> + struct vfio_device *vfio_device =
> + container_of(nb, struct vfio_device, iommu_nb);
> + struct vfio_iommu_type1_dma_unmap *unmap = data;
> +
> + vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> + return NOTIFY_OK;
> +}
> +
> static struct file *vfio_device_open(struct vfio_device *device)
> {
> + struct vfio_iommu_driver *iommu_driver;
> struct file *filep;
> int ret;
>
> @@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct vfio_device *device)
> if (ret)
> goto err_undo_count;
> }
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier) {
> + unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> +
> + device->iommu_nb.notifier_call = vfio_iommu_notifier;
> + iommu_driver->ops->register_notifier(
> + device->group->container->iommu_data, &events,
> + &device->iommu_nb);
> + }
> +
> up_read(&device->group->group_rwsem);
> }
> mutex_unlock(&device->dev_set->lock);
> @@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct vfio_device *device)
> err_close_device:
> mutex_lock(&device->dev_set->lock);
> down_read(&device->group->group_rwsem);
> - if (device->open_count == 1 && device->ops->close_device)
> + if (device->open_count == 1 && device->ops->close_device) {
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
Test for register_notifier callback...
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
use unregister_notifier callback. Same below.
> + }
> err_undo_count:
> device->open_count--;
> if (device->open_count == 0 && device->kvm)
> @@ -1339,12 +1371,20 @@ static const struct file_operations vfio_group_fops = {
> static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> {
> struct vfio_device *device = filep->private_data;
> + struct vfio_iommu_driver *iommu_driver;
>
> mutex_lock(&device->dev_set->lock);
> vfio_assert_device_open(device);
> down_read(&device->group->group_rwsem);
> if (device->open_count == 1 && device->ops->close_device)
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> up_read(&device->group->group_rwsem);
> device->open_count--;
> if (device->open_count == 0)
> @@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data,
> }
> EXPORT_SYMBOL(vfio_dma_rw);
>
> -static int vfio_register_iommu_notifier(struct vfio_group *group,
> - unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->register_notifier))
> - ret = driver->ops->register_notifier(container->iommu_data,
> - events, nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -static int vfio_unregister_iommu_notifier(struct vfio_group *group,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->unregister_notifier))
> - ret = driver->ops->unregister_notifier(container->iommu_data,
> - nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type, unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !events || (*events == 0) ||
> - !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_register_iommu_notifier(group, events, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_register_notifier);
> -
> -int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_unregister_iommu_notifier(group, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_unregister_notifier);
> -
> /*
> * Module/class support
> */
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index a6713022115155..cb2e4e9baa8fe8 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
> VFIO_IOMMU_CONTAINER_CLOSE = 0,
> };
>
> +/* events for register_notifier() */
> +enum {
> + VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> +};
Can't say I understand why this changed from BIT(0) to an enum, the
event mask is meant to be a bitfield. Using the notifier all the way
to the device was meant to avoid future callbacks on the device. If we
now have a dma_unmap on the device, should the whole infrastructure be
tailored to that one task? For example a dma_unmap_nb on the device,
{un}register_dma_unmap_notifier on the iommu ops,
vfio_dma_unmap_notifier, etc? Thanks,
Alex
> +
> /**
> * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> */
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index aa888cc517578e..b76623e3b92fca 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -44,6 +44,7 @@ struct vfio_device {
> unsigned int open_count;
> struct completion comp;
> struct list_head group_next;
> + struct notifier_block iommu_nb;
> };
>
> /**
> @@ -60,6 +61,8 @@ struct vfio_device {
> * @match: Optional device name match callback (return: 0 for no-match, >0 for
> * match, -errno for abort (ex. match with insufficient or incorrect
> * additional args)
> + * @dma_unmap: Called when userspace unmaps IOVA from the container
> + * this device is attached to.
> * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
> * @migration_set_state: Optional callback to change the migration state for
> * devices that support migration. It's mandatory for
> @@ -85,6 +88,7 @@ struct vfio_device_ops {
> int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
> void (*request)(struct vfio_device *vdev, unsigned int count);
> int (*match)(struct vfio_device *vdev, char *buf);
> + void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
> int (*device_feature)(struct vfio_device *device, u32 flags,
> void __user *arg, size_t argsz);
> struct file *(*migration_set_state)(
> @@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn,
> extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
> void *data, size_t len, bool write);
>
> -/* each type has independent events */
> -enum vfio_notify_type {
> - VFIO_IOMMU_NOTIFY = 0,
> -};
> -
> -/* events for VFIO_IOMMU_NOTIFY */
> -#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
> -
> -extern int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - unsigned long *required_events,
> - struct notifier_block *nb);
> -extern int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb);
> -
> -
> /*
> * Sub-module helpers
> */
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Intel-gfx] [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
@ 2022-06-17 22:42 ` Alex Williamson
0 siblings, 0 replies; 34+ messages in thread
From: Alex Williamson @ 2022-06-17 22:42 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Jason Herne, Eric Farman,
Vasily Gorbik, Heiko Carstens, Harald Freudenberger,
Rodrigo Vivi, intel-gvt-dev, Tony Krowiak, Cornelia Huck,
Peter Oberparleiter, Sven Schnelle
On Tue, 7 Jun 2022 20:02:11 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 61e71c1154be67..f005b644ab9e69 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> up_write(&device->group->group_rwsem);
> }
>
> +static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> + void *data)
> +{
> + struct vfio_device *vfio_device =
> + container_of(nb, struct vfio_device, iommu_nb);
> + struct vfio_iommu_type1_dma_unmap *unmap = data;
> +
> + vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> + return NOTIFY_OK;
> +}
> +
> static struct file *vfio_device_open(struct vfio_device *device)
> {
> + struct vfio_iommu_driver *iommu_driver;
> struct file *filep;
> int ret;
>
> @@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct vfio_device *device)
> if (ret)
> goto err_undo_count;
> }
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier) {
> + unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> +
> + device->iommu_nb.notifier_call = vfio_iommu_notifier;
> + iommu_driver->ops->register_notifier(
> + device->group->container->iommu_data, &events,
> + &device->iommu_nb);
> + }
> +
> up_read(&device->group->group_rwsem);
> }
> mutex_unlock(&device->dev_set->lock);
> @@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct vfio_device *device)
> err_close_device:
> mutex_lock(&device->dev_set->lock);
> down_read(&device->group->group_rwsem);
> - if (device->open_count == 1 && device->ops->close_device)
> + if (device->open_count == 1 && device->ops->close_device) {
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
Test for register_notifier callback...
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
use unregister_notifier callback. Same below.
> + }
> err_undo_count:
> device->open_count--;
> if (device->open_count == 0 && device->kvm)
> @@ -1339,12 +1371,20 @@ static const struct file_operations vfio_group_fops = {
> static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> {
> struct vfio_device *device = filep->private_data;
> + struct vfio_iommu_driver *iommu_driver;
>
> mutex_lock(&device->dev_set->lock);
> vfio_assert_device_open(device);
> down_read(&device->group->group_rwsem);
> if (device->open_count == 1 && device->ops->close_device)
> device->ops->close_device(device);
> +
> + iommu_driver = device->group->container->iommu_driver;
> + if (device->ops->dma_unmap && iommu_driver &&
> + iommu_driver->ops->register_notifier)
> + iommu_driver->ops->unregister_notifier(
> + device->group->container->iommu_data,
> + &device->iommu_nb);
> up_read(&device->group->group_rwsem);
> device->open_count--;
> if (device->open_count == 0)
> @@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data,
> }
> EXPORT_SYMBOL(vfio_dma_rw);
>
> -static int vfio_register_iommu_notifier(struct vfio_group *group,
> - unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->register_notifier))
> - ret = driver->ops->register_notifier(container->iommu_data,
> - events, nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -static int vfio_unregister_iommu_notifier(struct vfio_group *group,
> - struct notifier_block *nb)
> -{
> - struct vfio_container *container;
> - struct vfio_iommu_driver *driver;
> - int ret;
> -
> - lockdep_assert_held_read(&group->group_rwsem);
> -
> - container = group->container;
> - driver = container->iommu_driver;
> - if (likely(driver && driver->ops->unregister_notifier))
> - ret = driver->ops->unregister_notifier(container->iommu_data,
> - nb);
> - else
> - ret = -ENOTTY;
> -
> - return ret;
> -}
> -
> -int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type, unsigned long *events,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !events || (*events == 0) ||
> - !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_register_iommu_notifier(group, events, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_register_notifier);
> -
> -int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb)
> -{
> - struct vfio_group *group = device->group;
> - int ret;
> -
> - if (!nb || !vfio_assert_device_open(device))
> - return -EINVAL;
> -
> - switch (type) {
> - case VFIO_IOMMU_NOTIFY:
> - ret = vfio_unregister_iommu_notifier(group, nb);
> - break;
> - default:
> - ret = -EINVAL;
> - }
> - return ret;
> -}
> -EXPORT_SYMBOL(vfio_unregister_notifier);
> -
> /*
> * Module/class support
> */
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index a6713022115155..cb2e4e9baa8fe8 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
> VFIO_IOMMU_CONTAINER_CLOSE = 0,
> };
>
> +/* events for register_notifier() */
> +enum {
> + VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> +};
Can't say I understand why this changed from BIT(0) to an enum, the
event mask is meant to be a bitfield. Using the notifier all the way
to the device was meant to avoid future callbacks on the device. If we
now have a dma_unmap on the device, should the whole infrastructure be
tailored to that one task? For example a dma_unmap_nb on the device,
{un}register_dma_unmap_notifier on the iommu ops,
vfio_dma_unmap_notifier, etc? Thanks,
Alex
> +
> /**
> * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> */
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index aa888cc517578e..b76623e3b92fca 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -44,6 +44,7 @@ struct vfio_device {
> unsigned int open_count;
> struct completion comp;
> struct list_head group_next;
> + struct notifier_block iommu_nb;
> };
>
> /**
> @@ -60,6 +61,8 @@ struct vfio_device {
> * @match: Optional device name match callback (return: 0 for no-match, >0 for
> * match, -errno for abort (ex. match with insufficient or incorrect
> * additional args)
> + * @dma_unmap: Called when userspace unmaps IOVA from the container
> + * this device is attached to.
> * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
> * @migration_set_state: Optional callback to change the migration state for
> * devices that support migration. It's mandatory for
> @@ -85,6 +88,7 @@ struct vfio_device_ops {
> int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
> void (*request)(struct vfio_device *vdev, unsigned int count);
> int (*match)(struct vfio_device *vdev, char *buf);
> + void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
> int (*device_feature)(struct vfio_device *device, u32 flags,
> void __user *arg, size_t argsz);
> struct file *(*migration_set_state)(
> @@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn,
> extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
> void *data, size_t len, bool write);
>
> -/* each type has independent events */
> -enum vfio_notify_type {
> - VFIO_IOMMU_NOTIFY = 0,
> -};
> -
> -/* events for VFIO_IOMMU_NOTIFY */
> -#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
> -
> -extern int vfio_register_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - unsigned long *required_events,
> - struct notifier_block *nb);
> -extern int vfio_unregister_notifier(struct vfio_device *device,
> - enum vfio_notify_type type,
> - struct notifier_block *nb);
> -
> -
> /*
> * Sub-module helpers
> */
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
2022-06-17 22:42 ` Alex Williamson
(?)
@ 2022-06-17 22:47 ` Alex Williamson
-1 siblings, 0 replies; 34+ messages in thread
From: Alex Williamson @ 2022-06-17 22:47 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alexander Gordeev, David Airlie, Tony Krowiak,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang,
Christoph Hellwig
On Fri, 17 Jun 2022 16:42:30 -0600
Alex Williamson <alex.williamson@redhat.com> wrote:
> On Tue, 7 Jun 2022 20:02:11 -0300
> Jason Gunthorpe <jgg@nvidia.com> wrote:
> > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> > index 61e71c1154be67..f005b644ab9e69 100644
> > --- a/drivers/vfio/vfio.c
> > +++ b/drivers/vfio/vfio.c
> > @@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> > up_write(&device->group->group_rwsem);
> > }
> >
> > +static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> > + void *data)
> > +{
> > + struct vfio_device *vfio_device =
> > + container_of(nb, struct vfio_device, iommu_nb);
> > + struct vfio_iommu_type1_dma_unmap *unmap = data;
> > +
> > + vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> > + return NOTIFY_OK;
> > +}
> > +
> > static struct file *vfio_device_open(struct vfio_device *device)
> > {
> > + struct vfio_iommu_driver *iommu_driver;
> > struct file *filep;
> > int ret;
> >
> > @@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct vfio_device *device)
> > if (ret)
> > goto err_undo_count;
> > }
> > +
> > + iommu_driver = device->group->container->iommu_driver;
> > + if (device->ops->dma_unmap && iommu_driver &&
> > + iommu_driver->ops->register_notifier) {
> > + unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> > +
> > + device->iommu_nb.notifier_call = vfio_iommu_notifier;
> > + iommu_driver->ops->register_notifier(
> > + device->group->container->iommu_data, &events,
> > + &device->iommu_nb);
> > + }
> > +
> > up_read(&device->group->group_rwsem);
> > }
> > mutex_unlock(&device->dev_set->lock);
> > @@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct vfio_device *device)
> > err_close_device:
> > mutex_lock(&device->dev_set->lock);
> > down_read(&device->group->group_rwsem);
> > - if (device->open_count == 1 && device->ops->close_device)
> > + if (device->open_count == 1 && device->ops->close_device) {
> > device->ops->close_device(device);
> > +
> > + iommu_driver = device->group->container->iommu_driver;
> > + if (device->ops->dma_unmap && iommu_driver &&
> > + iommu_driver->ops->register_notifier)
>
> Test for register_notifier callback...
>
> > + iommu_driver->ops->unregister_notifier(
> > + device->group->container->iommu_data,
> > + &device->iommu_nb);
>
> use unregister_notifier callback. Same below.
>
> > + }
> > err_undo_count:
> > device->open_count--;
> > if (device->open_count == 0 && device->kvm)
> > @@ -1339,12 +1371,20 @@ static const struct file_operations vfio_group_fops = {
> > static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> > {
> > struct vfio_device *device = filep->private_data;
> > + struct vfio_iommu_driver *iommu_driver;
> >
> > mutex_lock(&device->dev_set->lock);
> > vfio_assert_device_open(device);
> > down_read(&device->group->group_rwsem);
> > if (device->open_count == 1 && device->ops->close_device)
> > device->ops->close_device(device);
> > +
> > + iommu_driver = device->group->container->iommu_driver;
> > + if (device->ops->dma_unmap && iommu_driver &&
> > + iommu_driver->ops->register_notifier)
> > + iommu_driver->ops->unregister_notifier(
> > + device->group->container->iommu_data,
> > + &device->iommu_nb);
> > up_read(&device->group->group_rwsem);
> > device->open_count--;
> > if (device->open_count == 0)
> > @@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data,
> > }
> > EXPORT_SYMBOL(vfio_dma_rw);
> >
> > -static int vfio_register_iommu_notifier(struct vfio_group *group,
> > - unsigned long *events,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_container *container;
> > - struct vfio_iommu_driver *driver;
> > - int ret;
> > -
> > - lockdep_assert_held_read(&group->group_rwsem);
> > -
> > - container = group->container;
> > - driver = container->iommu_driver;
> > - if (likely(driver && driver->ops->register_notifier))
> > - ret = driver->ops->register_notifier(container->iommu_data,
> > - events, nb);
> > - else
> > - ret = -ENOTTY;
> > -
> > - return ret;
> > -}
> > -
> > -static int vfio_unregister_iommu_notifier(struct vfio_group *group,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_container *container;
> > - struct vfio_iommu_driver *driver;
> > - int ret;
> > -
> > - lockdep_assert_held_read(&group->group_rwsem);
> > -
> > - container = group->container;
> > - driver = container->iommu_driver;
> > - if (likely(driver && driver->ops->unregister_notifier))
> > - ret = driver->ops->unregister_notifier(container->iommu_data,
> > - nb);
> > - else
> > - ret = -ENOTTY;
> > -
> > - return ret;
> > -}
> > -
> > -int vfio_register_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type, unsigned long *events,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_group *group = device->group;
> > - int ret;
> > -
> > - if (!nb || !events || (*events == 0) ||
> > - !vfio_assert_device_open(device))
> > - return -EINVAL;
> > -
> > - switch (type) {
> > - case VFIO_IOMMU_NOTIFY:
> > - ret = vfio_register_iommu_notifier(group, events, nb);
> > - break;
> > - default:
> > - ret = -EINVAL;
> > - }
> > - return ret;
> > -}
> > -EXPORT_SYMBOL(vfio_register_notifier);
> > -
> > -int vfio_unregister_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_group *group = device->group;
> > - int ret;
> > -
> > - if (!nb || !vfio_assert_device_open(device))
> > - return -EINVAL;
> > -
> > - switch (type) {
> > - case VFIO_IOMMU_NOTIFY:
> > - ret = vfio_unregister_iommu_notifier(group, nb);
> > - break;
> > - default:
> > - ret = -EINVAL;
> > - }
> > - return ret;
> > -}
> > -EXPORT_SYMBOL(vfio_unregister_notifier);
> > -
> > /*
> > * Module/class support
> > */
> > diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> > index a6713022115155..cb2e4e9baa8fe8 100644
> > --- a/drivers/vfio/vfio.h
> > +++ b/drivers/vfio/vfio.h
> > @@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
> > VFIO_IOMMU_CONTAINER_CLOSE = 0,
> > };
> >
> > +/* events for register_notifier() */
> > +enum {
> > + VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> > +};
>
> Can't say I understand why this changed from BIT(0) to an enum, the
> event mask is meant to be a bitfield. Using the notifier all the way
> to the device was meant to avoid future callbacks on the device. If we
> now have a dma_unmap on the device, should the whole infrastructure be
> tailored to that one task? For example a dma_unmap_nb on the device,
> {un}register_dma_unmap_notifier on the iommu ops,
> vfio_dma_unmap_notifier, etc? Thanks,
Ok, this all seems cleared up in the next patch, maybe there's a better
intermediate step, but not worth bike shedding. Thanks,
Alex
> > +
> > /**
> > * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> > */
> > diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> > index aa888cc517578e..b76623e3b92fca 100644
> > --- a/include/linux/vfio.h
> > +++ b/include/linux/vfio.h
> > @@ -44,6 +44,7 @@ struct vfio_device {
> > unsigned int open_count;
> > struct completion comp;
> > struct list_head group_next;
> > + struct notifier_block iommu_nb;
> > };
> >
> > /**
> > @@ -60,6 +61,8 @@ struct vfio_device {
> > * @match: Optional device name match callback (return: 0 for no-match, >0 for
> > * match, -errno for abort (ex. match with insufficient or incorrect
> > * additional args)
> > + * @dma_unmap: Called when userspace unmaps IOVA from the container
> > + * this device is attached to.
> > * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
> > * @migration_set_state: Optional callback to change the migration state for
> > * devices that support migration. It's mandatory for
> > @@ -85,6 +88,7 @@ struct vfio_device_ops {
> > int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
> > void (*request)(struct vfio_device *vdev, unsigned int count);
> > int (*match)(struct vfio_device *vdev, char *buf);
> > + void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
> > int (*device_feature)(struct vfio_device *device, u32 flags,
> > void __user *arg, size_t argsz);
> > struct file *(*migration_set_state)(
> > @@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn,
> > extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
> > void *data, size_t len, bool write);
> >
> > -/* each type has independent events */
> > -enum vfio_notify_type {
> > - VFIO_IOMMU_NOTIFY = 0,
> > -};
> > -
> > -/* events for VFIO_IOMMU_NOTIFY */
> > -#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
> > -
> > -extern int vfio_register_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type,
> > - unsigned long *required_events,
> > - struct notifier_block *nb);
> > -extern int vfio_unregister_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type,
> > - struct notifier_block *nb);
> > -
> > -
> > /*
> > * Sub-module helpers
> > */
>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
@ 2022-06-17 22:47 ` Alex Williamson
0 siblings, 0 replies; 34+ messages in thread
From: Alex Williamson @ 2022-06-17 22:47 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Zhi Wang, Jason Herne,
Eric Farman, Vasily Gorbik, Heiko Carstens, Harald Freudenberger,
Rodrigo Vivi, intel-gvt-dev, Tony Krowiak, Tvrtko Ursulin,
Cornelia Huck, Peter Oberparleiter, Sven Schnelle
On Fri, 17 Jun 2022 16:42:30 -0600
Alex Williamson <alex.williamson@redhat.com> wrote:
> On Tue, 7 Jun 2022 20:02:11 -0300
> Jason Gunthorpe <jgg@nvidia.com> wrote:
> > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> > index 61e71c1154be67..f005b644ab9e69 100644
> > --- a/drivers/vfio/vfio.c
> > +++ b/drivers/vfio/vfio.c
> > @@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> > up_write(&device->group->group_rwsem);
> > }
> >
> > +static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> > + void *data)
> > +{
> > + struct vfio_device *vfio_device =
> > + container_of(nb, struct vfio_device, iommu_nb);
> > + struct vfio_iommu_type1_dma_unmap *unmap = data;
> > +
> > + vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> > + return NOTIFY_OK;
> > +}
> > +
> > static struct file *vfio_device_open(struct vfio_device *device)
> > {
> > + struct vfio_iommu_driver *iommu_driver;
> > struct file *filep;
> > int ret;
> >
> > @@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct vfio_device *device)
> > if (ret)
> > goto err_undo_count;
> > }
> > +
> > + iommu_driver = device->group->container->iommu_driver;
> > + if (device->ops->dma_unmap && iommu_driver &&
> > + iommu_driver->ops->register_notifier) {
> > + unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> > +
> > + device->iommu_nb.notifier_call = vfio_iommu_notifier;
> > + iommu_driver->ops->register_notifier(
> > + device->group->container->iommu_data, &events,
> > + &device->iommu_nb);
> > + }
> > +
> > up_read(&device->group->group_rwsem);
> > }
> > mutex_unlock(&device->dev_set->lock);
> > @@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct vfio_device *device)
> > err_close_device:
> > mutex_lock(&device->dev_set->lock);
> > down_read(&device->group->group_rwsem);
> > - if (device->open_count == 1 && device->ops->close_device)
> > + if (device->open_count == 1 && device->ops->close_device) {
> > device->ops->close_device(device);
> > +
> > + iommu_driver = device->group->container->iommu_driver;
> > + if (device->ops->dma_unmap && iommu_driver &&
> > + iommu_driver->ops->register_notifier)
>
> Test for register_notifier callback...
>
> > + iommu_driver->ops->unregister_notifier(
> > + device->group->container->iommu_data,
> > + &device->iommu_nb);
>
> use unregister_notifier callback. Same below.
>
> > + }
> > err_undo_count:
> > device->open_count--;
> > if (device->open_count == 0 && device->kvm)
> > @@ -1339,12 +1371,20 @@ static const struct file_operations vfio_group_fops = {
> > static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> > {
> > struct vfio_device *device = filep->private_data;
> > + struct vfio_iommu_driver *iommu_driver;
> >
> > mutex_lock(&device->dev_set->lock);
> > vfio_assert_device_open(device);
> > down_read(&device->group->group_rwsem);
> > if (device->open_count == 1 && device->ops->close_device)
> > device->ops->close_device(device);
> > +
> > + iommu_driver = device->group->container->iommu_driver;
> > + if (device->ops->dma_unmap && iommu_driver &&
> > + iommu_driver->ops->register_notifier)
> > + iommu_driver->ops->unregister_notifier(
> > + device->group->container->iommu_data,
> > + &device->iommu_nb);
> > up_read(&device->group->group_rwsem);
> > device->open_count--;
> > if (device->open_count == 0)
> > @@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data,
> > }
> > EXPORT_SYMBOL(vfio_dma_rw);
> >
> > -static int vfio_register_iommu_notifier(struct vfio_group *group,
> > - unsigned long *events,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_container *container;
> > - struct vfio_iommu_driver *driver;
> > - int ret;
> > -
> > - lockdep_assert_held_read(&group->group_rwsem);
> > -
> > - container = group->container;
> > - driver = container->iommu_driver;
> > - if (likely(driver && driver->ops->register_notifier))
> > - ret = driver->ops->register_notifier(container->iommu_data,
> > - events, nb);
> > - else
> > - ret = -ENOTTY;
> > -
> > - return ret;
> > -}
> > -
> > -static int vfio_unregister_iommu_notifier(struct vfio_group *group,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_container *container;
> > - struct vfio_iommu_driver *driver;
> > - int ret;
> > -
> > - lockdep_assert_held_read(&group->group_rwsem);
> > -
> > - container = group->container;
> > - driver = container->iommu_driver;
> > - if (likely(driver && driver->ops->unregister_notifier))
> > - ret = driver->ops->unregister_notifier(container->iommu_data,
> > - nb);
> > - else
> > - ret = -ENOTTY;
> > -
> > - return ret;
> > -}
> > -
> > -int vfio_register_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type, unsigned long *events,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_group *group = device->group;
> > - int ret;
> > -
> > - if (!nb || !events || (*events == 0) ||
> > - !vfio_assert_device_open(device))
> > - return -EINVAL;
> > -
> > - switch (type) {
> > - case VFIO_IOMMU_NOTIFY:
> > - ret = vfio_register_iommu_notifier(group, events, nb);
> > - break;
> > - default:
> > - ret = -EINVAL;
> > - }
> > - return ret;
> > -}
> > -EXPORT_SYMBOL(vfio_register_notifier);
> > -
> > -int vfio_unregister_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_group *group = device->group;
> > - int ret;
> > -
> > - if (!nb || !vfio_assert_device_open(device))
> > - return -EINVAL;
> > -
> > - switch (type) {
> > - case VFIO_IOMMU_NOTIFY:
> > - ret = vfio_unregister_iommu_notifier(group, nb);
> > - break;
> > - default:
> > - ret = -EINVAL;
> > - }
> > - return ret;
> > -}
> > -EXPORT_SYMBOL(vfio_unregister_notifier);
> > -
> > /*
> > * Module/class support
> > */
> > diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> > index a6713022115155..cb2e4e9baa8fe8 100644
> > --- a/drivers/vfio/vfio.h
> > +++ b/drivers/vfio/vfio.h
> > @@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
> > VFIO_IOMMU_CONTAINER_CLOSE = 0,
> > };
> >
> > +/* events for register_notifier() */
> > +enum {
> > + VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> > +};
>
> Can't say I understand why this changed from BIT(0) to an enum, the
> event mask is meant to be a bitfield. Using the notifier all the way
> to the device was meant to avoid future callbacks on the device. If we
> now have a dma_unmap on the device, should the whole infrastructure be
> tailored to that one task? For example a dma_unmap_nb on the device,
> {un}register_dma_unmap_notifier on the iommu ops,
> vfio_dma_unmap_notifier, etc? Thanks,
Ok, this all seems cleared up in the next patch, maybe there's a better
intermediate step, but not worth bike shedding. Thanks,
Alex
> > +
> > /**
> > * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> > */
> > diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> > index aa888cc517578e..b76623e3b92fca 100644
> > --- a/include/linux/vfio.h
> > +++ b/include/linux/vfio.h
> > @@ -44,6 +44,7 @@ struct vfio_device {
> > unsigned int open_count;
> > struct completion comp;
> > struct list_head group_next;
> > + struct notifier_block iommu_nb;
> > };
> >
> > /**
> > @@ -60,6 +61,8 @@ struct vfio_device {
> > * @match: Optional device name match callback (return: 0 for no-match, >0 for
> > * match, -errno for abort (ex. match with insufficient or incorrect
> > * additional args)
> > + * @dma_unmap: Called when userspace unmaps IOVA from the container
> > + * this device is attached to.
> > * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
> > * @migration_set_state: Optional callback to change the migration state for
> > * devices that support migration. It's mandatory for
> > @@ -85,6 +88,7 @@ struct vfio_device_ops {
> > int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
> > void (*request)(struct vfio_device *vdev, unsigned int count);
> > int (*match)(struct vfio_device *vdev, char *buf);
> > + void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
> > int (*device_feature)(struct vfio_device *device, u32 flags,
> > void __user *arg, size_t argsz);
> > struct file *(*migration_set_state)(
> > @@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn,
> > extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
> > void *data, size_t len, bool write);
> >
> > -/* each type has independent events */
> > -enum vfio_notify_type {
> > - VFIO_IOMMU_NOTIFY = 0,
> > -};
> > -
> > -/* events for VFIO_IOMMU_NOTIFY */
> > -#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
> > -
> > -extern int vfio_register_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type,
> > - unsigned long *required_events,
> > - struct notifier_block *nb);
> > -extern int vfio_unregister_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type,
> > - struct notifier_block *nb);
> > -
> > -
> > /*
> > * Sub-module helpers
> > */
>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Intel-gfx] [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback
@ 2022-06-17 22:47 ` Alex Williamson
0 siblings, 0 replies; 34+ messages in thread
From: Alex Williamson @ 2022-06-17 22:47 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Jason Herne, Eric Farman,
Vasily Gorbik, Heiko Carstens, Harald Freudenberger,
Rodrigo Vivi, intel-gvt-dev, Tony Krowiak, Cornelia Huck,
Peter Oberparleiter, Sven Schnelle
On Fri, 17 Jun 2022 16:42:30 -0600
Alex Williamson <alex.williamson@redhat.com> wrote:
> On Tue, 7 Jun 2022 20:02:11 -0300
> Jason Gunthorpe <jgg@nvidia.com> wrote:
> > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> > index 61e71c1154be67..f005b644ab9e69 100644
> > --- a/drivers/vfio/vfio.c
> > +++ b/drivers/vfio/vfio.c
> > @@ -1077,8 +1077,20 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> > up_write(&device->group->group_rwsem);
> > }
> >
> > +static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> > + void *data)
> > +{
> > + struct vfio_device *vfio_device =
> > + container_of(nb, struct vfio_device, iommu_nb);
> > + struct vfio_iommu_type1_dma_unmap *unmap = data;
> > +
> > + vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> > + return NOTIFY_OK;
> > +}
> > +
> > static struct file *vfio_device_open(struct vfio_device *device)
> > {
> > + struct vfio_iommu_driver *iommu_driver;
> > struct file *filep;
> > int ret;
> >
> > @@ -1109,6 +1121,18 @@ static struct file *vfio_device_open(struct vfio_device *device)
> > if (ret)
> > goto err_undo_count;
> > }
> > +
> > + iommu_driver = device->group->container->iommu_driver;
> > + if (device->ops->dma_unmap && iommu_driver &&
> > + iommu_driver->ops->register_notifier) {
> > + unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> > +
> > + device->iommu_nb.notifier_call = vfio_iommu_notifier;
> > + iommu_driver->ops->register_notifier(
> > + device->group->container->iommu_data, &events,
> > + &device->iommu_nb);
> > + }
> > +
> > up_read(&device->group->group_rwsem);
> > }
> > mutex_unlock(&device->dev_set->lock);
> > @@ -1143,8 +1167,16 @@ static struct file *vfio_device_open(struct vfio_device *device)
> > err_close_device:
> > mutex_lock(&device->dev_set->lock);
> > down_read(&device->group->group_rwsem);
> > - if (device->open_count == 1 && device->ops->close_device)
> > + if (device->open_count == 1 && device->ops->close_device) {
> > device->ops->close_device(device);
> > +
> > + iommu_driver = device->group->container->iommu_driver;
> > + if (device->ops->dma_unmap && iommu_driver &&
> > + iommu_driver->ops->register_notifier)
>
> Test for register_notifier callback...
>
> > + iommu_driver->ops->unregister_notifier(
> > + device->group->container->iommu_data,
> > + &device->iommu_nb);
>
> use unregister_notifier callback. Same below.
>
> > + }
> > err_undo_count:
> > device->open_count--;
> > if (device->open_count == 0 && device->kvm)
> > @@ -1339,12 +1371,20 @@ static const struct file_operations vfio_group_fops = {
> > static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> > {
> > struct vfio_device *device = filep->private_data;
> > + struct vfio_iommu_driver *iommu_driver;
> >
> > mutex_lock(&device->dev_set->lock);
> > vfio_assert_device_open(device);
> > down_read(&device->group->group_rwsem);
> > if (device->open_count == 1 && device->ops->close_device)
> > device->ops->close_device(device);
> > +
> > + iommu_driver = device->group->container->iommu_driver;
> > + if (device->ops->dma_unmap && iommu_driver &&
> > + iommu_driver->ops->register_notifier)
> > + iommu_driver->ops->unregister_notifier(
> > + device->group->container->iommu_data,
> > + &device->iommu_nb);
> > up_read(&device->group->group_rwsem);
> > device->open_count--;
> > if (device->open_count == 0)
> > @@ -2027,90 +2067,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data,
> > }
> > EXPORT_SYMBOL(vfio_dma_rw);
> >
> > -static int vfio_register_iommu_notifier(struct vfio_group *group,
> > - unsigned long *events,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_container *container;
> > - struct vfio_iommu_driver *driver;
> > - int ret;
> > -
> > - lockdep_assert_held_read(&group->group_rwsem);
> > -
> > - container = group->container;
> > - driver = container->iommu_driver;
> > - if (likely(driver && driver->ops->register_notifier))
> > - ret = driver->ops->register_notifier(container->iommu_data,
> > - events, nb);
> > - else
> > - ret = -ENOTTY;
> > -
> > - return ret;
> > -}
> > -
> > -static int vfio_unregister_iommu_notifier(struct vfio_group *group,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_container *container;
> > - struct vfio_iommu_driver *driver;
> > - int ret;
> > -
> > - lockdep_assert_held_read(&group->group_rwsem);
> > -
> > - container = group->container;
> > - driver = container->iommu_driver;
> > - if (likely(driver && driver->ops->unregister_notifier))
> > - ret = driver->ops->unregister_notifier(container->iommu_data,
> > - nb);
> > - else
> > - ret = -ENOTTY;
> > -
> > - return ret;
> > -}
> > -
> > -int vfio_register_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type, unsigned long *events,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_group *group = device->group;
> > - int ret;
> > -
> > - if (!nb || !events || (*events == 0) ||
> > - !vfio_assert_device_open(device))
> > - return -EINVAL;
> > -
> > - switch (type) {
> > - case VFIO_IOMMU_NOTIFY:
> > - ret = vfio_register_iommu_notifier(group, events, nb);
> > - break;
> > - default:
> > - ret = -EINVAL;
> > - }
> > - return ret;
> > -}
> > -EXPORT_SYMBOL(vfio_register_notifier);
> > -
> > -int vfio_unregister_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type,
> > - struct notifier_block *nb)
> > -{
> > - struct vfio_group *group = device->group;
> > - int ret;
> > -
> > - if (!nb || !vfio_assert_device_open(device))
> > - return -EINVAL;
> > -
> > - switch (type) {
> > - case VFIO_IOMMU_NOTIFY:
> > - ret = vfio_unregister_iommu_notifier(group, nb);
> > - break;
> > - default:
> > - ret = -EINVAL;
> > - }
> > - return ret;
> > -}
> > -EXPORT_SYMBOL(vfio_unregister_notifier);
> > -
> > /*
> > * Module/class support
> > */
> > diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> > index a6713022115155..cb2e4e9baa8fe8 100644
> > --- a/drivers/vfio/vfio.h
> > +++ b/drivers/vfio/vfio.h
> > @@ -33,6 +33,11 @@ enum vfio_iommu_notify_type {
> > VFIO_IOMMU_CONTAINER_CLOSE = 0,
> > };
> >
> > +/* events for register_notifier() */
> > +enum {
> > + VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> > +};
>
> Can't say I understand why this changed from BIT(0) to an enum, the
> event mask is meant to be a bitfield. Using the notifier all the way
> to the device was meant to avoid future callbacks on the device. If we
> now have a dma_unmap on the device, should the whole infrastructure be
> tailored to that one task? For example a dma_unmap_nb on the device,
> {un}register_dma_unmap_notifier on the iommu ops,
> vfio_dma_unmap_notifier, etc? Thanks,
Ok, this all seems cleared up in the next patch, maybe there's a better
intermediate step, but not worth bike shedding. Thanks,
Alex
> > +
> > /**
> > * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> > */
> > diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> > index aa888cc517578e..b76623e3b92fca 100644
> > --- a/include/linux/vfio.h
> > +++ b/include/linux/vfio.h
> > @@ -44,6 +44,7 @@ struct vfio_device {
> > unsigned int open_count;
> > struct completion comp;
> > struct list_head group_next;
> > + struct notifier_block iommu_nb;
> > };
> >
> > /**
> > @@ -60,6 +61,8 @@ struct vfio_device {
> > * @match: Optional device name match callback (return: 0 for no-match, >0 for
> > * match, -errno for abort (ex. match with insufficient or incorrect
> > * additional args)
> > + * @dma_unmap: Called when userspace unmaps IOVA from the container
> > + * this device is attached to.
> > * @device_feature: Optional, fill in the VFIO_DEVICE_FEATURE ioctl
> > * @migration_set_state: Optional callback to change the migration state for
> > * devices that support migration. It's mandatory for
> > @@ -85,6 +88,7 @@ struct vfio_device_ops {
> > int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
> > void (*request)(struct vfio_device *vdev, unsigned int count);
> > int (*match)(struct vfio_device *vdev, char *buf);
> > + void (*dma_unmap)(struct vfio_device *vdev, u64 iova, u64 length);
> > int (*device_feature)(struct vfio_device *device, u32 flags,
> > void __user *arg, size_t argsz);
> > struct file *(*migration_set_state)(
> > @@ -154,23 +158,6 @@ extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn,
> > extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova,
> > void *data, size_t len, bool write);
> >
> > -/* each type has independent events */
> > -enum vfio_notify_type {
> > - VFIO_IOMMU_NOTIFY = 0,
> > -};
> > -
> > -/* events for VFIO_IOMMU_NOTIFY */
> > -#define VFIO_IOMMU_NOTIFY_DMA_UNMAP BIT(0)
> > -
> > -extern int vfio_register_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type,
> > - unsigned long *required_events,
> > - struct notifier_block *nb);
> > -extern int vfio_unregister_notifier(struct vfio_device *device,
> > - enum vfio_notify_type type,
> > - struct notifier_block *nb);
> > -
> > -
> > /*
> > * Sub-module helpers
> > */
>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
(?)
@ 2022-06-17 23:19 ` Alex Williamson
-1 siblings, 0 replies; 34+ messages in thread
From: Alex Williamson @ 2022-06-17 23:19 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alexander Gordeev, David Airlie, Tony Krowiak,
Christian Borntraeger, Cornelia Huck, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Jason Herne,
Joonas Lahtinen, kvm, linux-s390, Matthew Rosato,
Peter Oberparleiter, Halil Pasic, Rodrigo Vivi, Sven Schnelle,
Tvrtko Ursulin, Vineeth Vijayan, Zhenyu Wang, Zhi Wang,
Christoph Hellwig
On Tue, 7 Jun 2022 20:02:12 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:
> Instead of bouncing the function call to the driver op through a blocking
> notifier just have the iommu layer call it directly.
>
> Register each device that is being attached to the iommu with the lower
> driver which then threads them on a linked list and calls the appropriate
> driver op at the right time.
>
> Currently the only use is if dma_unmap() is defined.
>
> Also, fully lock all the debugging tests on the pinning path that a
> dma_unmap is registered.
>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
> drivers/vfio/vfio.c | 42 ++++---------
> drivers/vfio/vfio.h | 14 ++---
> drivers/vfio/vfio_iommu_type1.c | 103 ++++++++++++++++++++------------
> include/linux/vfio.h | 2 +-
> 4 files changed, 83 insertions(+), 78 deletions(-)
>
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index f005b644ab9e69..065b57e601bff7 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -619,6 +619,9 @@ EXPORT_SYMBOL_GPL(vfio_register_group_dev);
> */
> int vfio_register_emulated_iommu_dev(struct vfio_device *device)
> {
> + if (WARN_ON(!device->ops->dma_unmap))
> + return -EINVAL;
> +
> return __vfio_register_dev(device,
> vfio_noiommu_group_alloc(device->dev, VFIO_EMULATED_IOMMU));
> }
> @@ -1077,17 +1080,6 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> up_write(&device->group->group_rwsem);
> }
>
> -static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> - void *data)
> -{
> - struct vfio_device *vfio_device =
> - container_of(nb, struct vfio_device, iommu_nb);
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> -
> - vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> - return NOTIFY_OK;
> -}
> -
> static struct file *vfio_device_open(struct vfio_device *device)
> {
> struct vfio_iommu_driver *iommu_driver;
> @@ -1123,15 +1115,9 @@ static struct file *vfio_device_open(struct vfio_device *device)
> }
>
> iommu_driver = device->group->container->iommu_driver;
> - if (device->ops->dma_unmap && iommu_driver &&
> - iommu_driver->ops->register_notifier) {
> - unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> -
> - device->iommu_nb.notifier_call = vfio_iommu_notifier;
> - iommu_driver->ops->register_notifier(
> - device->group->container->iommu_data, &events,
> - &device->iommu_nb);
> - }
> + if (iommu_driver && iommu_driver->ops->register_device)
> + iommu_driver->ops->register_device(
> + device->group->container->iommu_data, device);
>
> up_read(&device->group->group_rwsem);
> }
> @@ -1171,11 +1157,9 @@ static struct file *vfio_device_open(struct vfio_device *device)
> device->ops->close_device(device);
>
> iommu_driver = device->group->container->iommu_driver;
> - if (device->ops->dma_unmap && iommu_driver &&
> - iommu_driver->ops->register_notifier)
> - iommu_driver->ops->unregister_notifier(
> - device->group->container->iommu_data,
> - &device->iommu_nb);
> + if (iommu_driver && iommu_driver->ops->register_device)
> + iommu_driver->ops->unregister_device(
> + device->group->container->iommu_data, device);
But let's fix this in the next respin too, ie. test register but call
unregister. Got it right below in this one.
> }
> err_undo_count:
> device->open_count--;
> @@ -1380,11 +1364,9 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> device->ops->close_device(device);
>
> iommu_driver = device->group->container->iommu_driver;
> - if (device->ops->dma_unmap && iommu_driver &&
> - iommu_driver->ops->register_notifier)
> - iommu_driver->ops->unregister_notifier(
> - device->group->container->iommu_data,
> - &device->iommu_nb);
> + if (iommu_driver && iommu_driver->ops->unregister_device)
> + iommu_driver->ops->unregister_device(
> + device->group->container->iommu_data, device);
> up_read(&device->group->group_rwsem);
> device->open_count--;
> if (device->open_count == 0)
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index cb2e4e9baa8fe8..4a7db1f3c33e7e 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -33,11 +33,6 @@ enum vfio_iommu_notify_type {
> VFIO_IOMMU_CONTAINER_CLOSE = 0,
> };
>
> -/* events for register_notifier() */
> -enum {
> - VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> -};
> -
> /**
> * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> */
> @@ -60,11 +55,10 @@ struct vfio_iommu_driver_ops {
> unsigned long *phys_pfn);
> int (*unpin_pages)(void *iommu_data,
> unsigned long *user_pfn, int npage);
> - int (*register_notifier)(void *iommu_data,
> - unsigned long *events,
> - struct notifier_block *nb);
> - int (*unregister_notifier)(void *iommu_data,
> - struct notifier_block *nb);
> + void (*register_device)(void *iommu_data,
> + struct vfio_device *vdev);
> + void (*unregister_device)(void *iommu_data,
> + struct vfio_device *vdev);
> int (*dma_rw)(void *iommu_data, dma_addr_t user_iova,
> void *data, size_t count, bool write);
> struct iommu_domain *(*group_iommu_domain)(void *iommu_data,
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index c13b9290e35759..4ddb1f1abd238b 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -67,7 +67,8 @@ struct vfio_iommu {
> struct list_head iova_list;
> struct mutex lock;
> struct rb_root dma_list;
> - struct blocking_notifier_head notifier;
> + struct list_head device_list;
> + struct mutex device_list_lock;
> unsigned int dma_avail;
> unsigned int vaddr_invalid_count;
> uint64_t pgsize_bitmap;
> @@ -865,8 +866,8 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
> }
> }
>
> - /* Fail if notifier list is empty */
> - if (!iommu->notifier.head) {
> + /* Fail if no dma_umap notifier is registered */
No "notifier" anymore.
Should we even get here if this list is empty? Seems like we can
restrict page pinning to devices supporting unmap_dma now and this
could be a WARN_ON. Thanks,
Alex
> + if (list_empty(&iommu->device_list)) {
> ret = -EINVAL;
> goto pin_done;
> }
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
@ 2022-06-17 23:19 ` Alex Williamson
0 siblings, 0 replies; 34+ messages in thread
From: Alex Williamson @ 2022-06-17 23:19 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Zhi Wang, Jason Herne,
Eric Farman, Vasily Gorbik, Heiko Carstens, Harald Freudenberger,
Rodrigo Vivi, intel-gvt-dev, Tony Krowiak, Tvrtko Ursulin,
Cornelia Huck, Peter Oberparleiter, Sven Schnelle
On Tue, 7 Jun 2022 20:02:12 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:
> Instead of bouncing the function call to the driver op through a blocking
> notifier just have the iommu layer call it directly.
>
> Register each device that is being attached to the iommu with the lower
> driver which then threads them on a linked list and calls the appropriate
> driver op at the right time.
>
> Currently the only use is if dma_unmap() is defined.
>
> Also, fully lock all the debugging tests on the pinning path that a
> dma_unmap is registered.
>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
> drivers/vfio/vfio.c | 42 ++++---------
> drivers/vfio/vfio.h | 14 ++---
> drivers/vfio/vfio_iommu_type1.c | 103 ++++++++++++++++++++------------
> include/linux/vfio.h | 2 +-
> 4 files changed, 83 insertions(+), 78 deletions(-)
>
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index f005b644ab9e69..065b57e601bff7 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -619,6 +619,9 @@ EXPORT_SYMBOL_GPL(vfio_register_group_dev);
> */
> int vfio_register_emulated_iommu_dev(struct vfio_device *device)
> {
> + if (WARN_ON(!device->ops->dma_unmap))
> + return -EINVAL;
> +
> return __vfio_register_dev(device,
> vfio_noiommu_group_alloc(device->dev, VFIO_EMULATED_IOMMU));
> }
> @@ -1077,17 +1080,6 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> up_write(&device->group->group_rwsem);
> }
>
> -static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> - void *data)
> -{
> - struct vfio_device *vfio_device =
> - container_of(nb, struct vfio_device, iommu_nb);
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> -
> - vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> - return NOTIFY_OK;
> -}
> -
> static struct file *vfio_device_open(struct vfio_device *device)
> {
> struct vfio_iommu_driver *iommu_driver;
> @@ -1123,15 +1115,9 @@ static struct file *vfio_device_open(struct vfio_device *device)
> }
>
> iommu_driver = device->group->container->iommu_driver;
> - if (device->ops->dma_unmap && iommu_driver &&
> - iommu_driver->ops->register_notifier) {
> - unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> -
> - device->iommu_nb.notifier_call = vfio_iommu_notifier;
> - iommu_driver->ops->register_notifier(
> - device->group->container->iommu_data, &events,
> - &device->iommu_nb);
> - }
> + if (iommu_driver && iommu_driver->ops->register_device)
> + iommu_driver->ops->register_device(
> + device->group->container->iommu_data, device);
>
> up_read(&device->group->group_rwsem);
> }
> @@ -1171,11 +1157,9 @@ static struct file *vfio_device_open(struct vfio_device *device)
> device->ops->close_device(device);
>
> iommu_driver = device->group->container->iommu_driver;
> - if (device->ops->dma_unmap && iommu_driver &&
> - iommu_driver->ops->register_notifier)
> - iommu_driver->ops->unregister_notifier(
> - device->group->container->iommu_data,
> - &device->iommu_nb);
> + if (iommu_driver && iommu_driver->ops->register_device)
> + iommu_driver->ops->unregister_device(
> + device->group->container->iommu_data, device);
But let's fix this in the next respin too, ie. test register but call
unregister. Got it right below in this one.
> }
> err_undo_count:
> device->open_count--;
> @@ -1380,11 +1364,9 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> device->ops->close_device(device);
>
> iommu_driver = device->group->container->iommu_driver;
> - if (device->ops->dma_unmap && iommu_driver &&
> - iommu_driver->ops->register_notifier)
> - iommu_driver->ops->unregister_notifier(
> - device->group->container->iommu_data,
> - &device->iommu_nb);
> + if (iommu_driver && iommu_driver->ops->unregister_device)
> + iommu_driver->ops->unregister_device(
> + device->group->container->iommu_data, device);
> up_read(&device->group->group_rwsem);
> device->open_count--;
> if (device->open_count == 0)
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index cb2e4e9baa8fe8..4a7db1f3c33e7e 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -33,11 +33,6 @@ enum vfio_iommu_notify_type {
> VFIO_IOMMU_CONTAINER_CLOSE = 0,
> };
>
> -/* events for register_notifier() */
> -enum {
> - VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> -};
> -
> /**
> * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> */
> @@ -60,11 +55,10 @@ struct vfio_iommu_driver_ops {
> unsigned long *phys_pfn);
> int (*unpin_pages)(void *iommu_data,
> unsigned long *user_pfn, int npage);
> - int (*register_notifier)(void *iommu_data,
> - unsigned long *events,
> - struct notifier_block *nb);
> - int (*unregister_notifier)(void *iommu_data,
> - struct notifier_block *nb);
> + void (*register_device)(void *iommu_data,
> + struct vfio_device *vdev);
> + void (*unregister_device)(void *iommu_data,
> + struct vfio_device *vdev);
> int (*dma_rw)(void *iommu_data, dma_addr_t user_iova,
> void *data, size_t count, bool write);
> struct iommu_domain *(*group_iommu_domain)(void *iommu_data,
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index c13b9290e35759..4ddb1f1abd238b 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -67,7 +67,8 @@ struct vfio_iommu {
> struct list_head iova_list;
> struct mutex lock;
> struct rb_root dma_list;
> - struct blocking_notifier_head notifier;
> + struct list_head device_list;
> + struct mutex device_list_lock;
> unsigned int dma_avail;
> unsigned int vaddr_invalid_count;
> uint64_t pgsize_bitmap;
> @@ -865,8 +866,8 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
> }
> }
>
> - /* Fail if notifier list is empty */
> - if (!iommu->notifier.head) {
> + /* Fail if no dma_umap notifier is registered */
No "notifier" anymore.
Should we even get here if this list is empty? Seems like we can
restrict page pinning to devices supporting unmap_dma now and this
could be a WARN_ON. Thanks,
Alex
> + if (list_empty(&iommu->device_list)) {
> ret = -EINVAL;
> goto pin_done;
> }
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Intel-gfx] [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list
@ 2022-06-17 23:19 ` Alex Williamson
0 siblings, 0 replies; 34+ messages in thread
From: Alex Williamson @ 2022-06-17 23:19 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, David Airlie, dri-devel, Vineeth Vijayan, Alexander Gordeev,
Christoph Hellwig, linux-s390, Matthew Rosato, Halil Pasic,
Christian Borntraeger, intel-gfx, Jason Herne, Eric Farman,
Vasily Gorbik, Heiko Carstens, Harald Freudenberger,
Rodrigo Vivi, intel-gvt-dev, Tony Krowiak, Cornelia Huck,
Peter Oberparleiter, Sven Schnelle
On Tue, 7 Jun 2022 20:02:12 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:
> Instead of bouncing the function call to the driver op through a blocking
> notifier just have the iommu layer call it directly.
>
> Register each device that is being attached to the iommu with the lower
> driver which then threads them on a linked list and calls the appropriate
> driver op at the right time.
>
> Currently the only use is if dma_unmap() is defined.
>
> Also, fully lock all the debugging tests on the pinning path that a
> dma_unmap is registered.
>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
> drivers/vfio/vfio.c | 42 ++++---------
> drivers/vfio/vfio.h | 14 ++---
> drivers/vfio/vfio_iommu_type1.c | 103 ++++++++++++++++++++------------
> include/linux/vfio.h | 2 +-
> 4 files changed, 83 insertions(+), 78 deletions(-)
>
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index f005b644ab9e69..065b57e601bff7 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -619,6 +619,9 @@ EXPORT_SYMBOL_GPL(vfio_register_group_dev);
> */
> int vfio_register_emulated_iommu_dev(struct vfio_device *device)
> {
> + if (WARN_ON(!device->ops->dma_unmap))
> + return -EINVAL;
> +
> return __vfio_register_dev(device,
> vfio_noiommu_group_alloc(device->dev, VFIO_EMULATED_IOMMU));
> }
> @@ -1077,17 +1080,6 @@ static void vfio_device_unassign_container(struct vfio_device *device)
> up_write(&device->group->group_rwsem);
> }
>
> -static int vfio_iommu_notifier(struct notifier_block *nb, unsigned long action,
> - void *data)
> -{
> - struct vfio_device *vfio_device =
> - container_of(nb, struct vfio_device, iommu_nb);
> - struct vfio_iommu_type1_dma_unmap *unmap = data;
> -
> - vfio_device->ops->dma_unmap(vfio_device, unmap->iova, unmap->size);
> - return NOTIFY_OK;
> -}
> -
> static struct file *vfio_device_open(struct vfio_device *device)
> {
> struct vfio_iommu_driver *iommu_driver;
> @@ -1123,15 +1115,9 @@ static struct file *vfio_device_open(struct vfio_device *device)
> }
>
> iommu_driver = device->group->container->iommu_driver;
> - if (device->ops->dma_unmap && iommu_driver &&
> - iommu_driver->ops->register_notifier) {
> - unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
> -
> - device->iommu_nb.notifier_call = vfio_iommu_notifier;
> - iommu_driver->ops->register_notifier(
> - device->group->container->iommu_data, &events,
> - &device->iommu_nb);
> - }
> + if (iommu_driver && iommu_driver->ops->register_device)
> + iommu_driver->ops->register_device(
> + device->group->container->iommu_data, device);
>
> up_read(&device->group->group_rwsem);
> }
> @@ -1171,11 +1157,9 @@ static struct file *vfio_device_open(struct vfio_device *device)
> device->ops->close_device(device);
>
> iommu_driver = device->group->container->iommu_driver;
> - if (device->ops->dma_unmap && iommu_driver &&
> - iommu_driver->ops->register_notifier)
> - iommu_driver->ops->unregister_notifier(
> - device->group->container->iommu_data,
> - &device->iommu_nb);
> + if (iommu_driver && iommu_driver->ops->register_device)
> + iommu_driver->ops->unregister_device(
> + device->group->container->iommu_data, device);
But let's fix this in the next respin too, ie. test register but call
unregister. Got it right below in this one.
> }
> err_undo_count:
> device->open_count--;
> @@ -1380,11 +1364,9 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> device->ops->close_device(device);
>
> iommu_driver = device->group->container->iommu_driver;
> - if (device->ops->dma_unmap && iommu_driver &&
> - iommu_driver->ops->register_notifier)
> - iommu_driver->ops->unregister_notifier(
> - device->group->container->iommu_data,
> - &device->iommu_nb);
> + if (iommu_driver && iommu_driver->ops->unregister_device)
> + iommu_driver->ops->unregister_device(
> + device->group->container->iommu_data, device);
> up_read(&device->group->group_rwsem);
> device->open_count--;
> if (device->open_count == 0)
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index cb2e4e9baa8fe8..4a7db1f3c33e7e 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -33,11 +33,6 @@ enum vfio_iommu_notify_type {
> VFIO_IOMMU_CONTAINER_CLOSE = 0,
> };
>
> -/* events for register_notifier() */
> -enum {
> - VFIO_IOMMU_NOTIFY_DMA_UNMAP = 1,
> -};
> -
> /**
> * struct vfio_iommu_driver_ops - VFIO IOMMU driver callbacks
> */
> @@ -60,11 +55,10 @@ struct vfio_iommu_driver_ops {
> unsigned long *phys_pfn);
> int (*unpin_pages)(void *iommu_data,
> unsigned long *user_pfn, int npage);
> - int (*register_notifier)(void *iommu_data,
> - unsigned long *events,
> - struct notifier_block *nb);
> - int (*unregister_notifier)(void *iommu_data,
> - struct notifier_block *nb);
> + void (*register_device)(void *iommu_data,
> + struct vfio_device *vdev);
> + void (*unregister_device)(void *iommu_data,
> + struct vfio_device *vdev);
> int (*dma_rw)(void *iommu_data, dma_addr_t user_iova,
> void *data, size_t count, bool write);
> struct iommu_domain *(*group_iommu_domain)(void *iommu_data,
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index c13b9290e35759..4ddb1f1abd238b 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -67,7 +67,8 @@ struct vfio_iommu {
> struct list_head iova_list;
> struct mutex lock;
> struct rb_root dma_list;
> - struct blocking_notifier_head notifier;
> + struct list_head device_list;
> + struct mutex device_list_lock;
> unsigned int dma_avail;
> unsigned int vaddr_invalid_count;
> uint64_t pgsize_bitmap;
> @@ -865,8 +866,8 @@ static int vfio_iommu_type1_pin_pages(void *iommu_data,
> }
> }
>
> - /* Fail if notifier list is empty */
> - if (!iommu->notifier.head) {
> + /* Fail if no dma_umap notifier is registered */
No "notifier" anymore.
Should we even get here if this list is empty? Seems like we can
restrict page pinning to devices supporting unmap_dma now and this
could be a WARN_ON. Thanks,
Alex
> + if (list_empty(&iommu->device_list)) {
> ret = -EINVAL;
> goto pin_done;
> }
^ permalink raw reply [flat|nested] 34+ messages in thread
end of thread, other threads:[~2022-06-21 11:06 UTC | newest]
Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-07 23:02 [PATCH v2 0/2] Remove the VFIO_IOMMU_NOTIFY_DMA_UNMAP notifier Jason Gunthorpe
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
2022-06-07 23:02 ` [PATCH v2 1/2] vfio: Replace the DMA unmapping notifier with a callback Jason Gunthorpe
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
2022-06-08 3:37 ` Tian, Kevin
2022-06-08 3:37 ` [Intel-gfx] " Tian, Kevin
2022-06-08 15:50 ` Eric Farman
2022-06-09 14:43 ` Jason Gunthorpe
2022-06-09 14:43 ` [Intel-gfx] " Jason Gunthorpe
2022-06-09 14:43 ` Jason Gunthorpe
2022-06-09 13:21 ` Tony Krowiak
2022-06-09 13:21 ` [Intel-gfx] " Tony Krowiak
2022-06-17 22:42 ` Alex Williamson
2022-06-17 22:42 ` [Intel-gfx] " Alex Williamson
2022-06-17 22:42 ` Alex Williamson
2022-06-17 22:47 ` Alex Williamson
2022-06-17 22:47 ` [Intel-gfx] " Alex Williamson
2022-06-17 22:47 ` Alex Williamson
2022-06-07 23:02 ` [PATCH v2 2/2] vfio: Replace the iommu notifier with a device list Jason Gunthorpe
2022-06-07 23:02 ` [Intel-gfx] " Jason Gunthorpe
2022-06-08 3:47 ` Tian, Kevin
2022-06-08 3:47 ` [Intel-gfx] " Tian, Kevin
2022-06-08 11:26 ` Jason Gunthorpe
2022-06-08 11:26 ` [Intel-gfx] " Jason Gunthorpe
2022-06-08 11:26 ` Jason Gunthorpe
2022-06-08 23:50 ` Tian, Kevin
2022-06-08 23:50 ` [Intel-gfx] " Tian, Kevin
2022-06-08 23:50 ` Tian, Kevin
2022-06-08 5:33 ` [Intel-gfx] " Christoph Hellwig
2022-06-08 5:33 ` Christoph Hellwig
2022-06-17 23:19 ` Alex Williamson
2022-06-17 23:19 ` [Intel-gfx] " Alex Williamson
2022-06-17 23:19 ` Alex Williamson
2022-06-08 5:38 ` [Intel-gfx] ✗ Fi.CI.BAT: failure for Remove the VFIO_IOMMU_NOTIFY_DMA_UNMAP notifier (rev2) Patchwork
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.