* [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-23 23:02 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:02 UTC (permalink / raw)
To: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc,
linux-s390, Peter Oberparleiter, Halil Pasic, Pierre Morel,
Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Raj, Ashok, Dan Williams, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
Prologue
========
This is series #3 in part of a larger work that arose from the minor
remark that the mdev_parent_ops indirection shim is useless and
complicates things.
It applies on top of Alex's current tree and requires the prior two
series.
This series achieves the removal of vfio_mdev.c. The future patches are all
focused on leveraging the changes made in the prior series to simplify the
API and device operation.
A preview of the future series's is here:
https://github.com/jgunthorpe/linux/pull/3/commits
========
The mdev bus's core part for managing the lifecycle of devices is mostly
as one would expect for a driver core bus subsystem.
However instead of having a normal 'struct device_driver' and binding the
actual mdev drivers through the standard driver core mechanisms it open
codes this with the struct mdev_parent_ops and provides a single driver
that shims between the VFIO core and the actual device driver.
Make every one of the mdev drivers implement an actual struct mdev_driver
and directly call vfio_register_group_dev() in the probe() function for
the mdev.
Squash what is left of the mdev_parent_ops into the mdev_driver and remap
create(), remove() and mdev_attr_groups to their driver core
equivalents. Arrange to bind the created mdev_device to the mdev_driver
that is provided by the end driver.
The actual execution flow doesn't change much, eg what was
parent_ops->create is now device_driver->probe and it is called at almost
the exact same time - except under the normal control of the driver core.
This allows deleting the entire mdev_drvdata, and tidying some of the
sysfs. Many places in the drivers start using container_of()
This cleanly splits the mdev sysfs GUID lifecycle management stuff from
the vfio_device implementation part, the only VFIO special part of mdev
that remains is the mdev specific iommu intervention.
Thanks,
Jason
Jason Gunthorpe (12):
vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
vfio/mdev: Allow the mdev_parent_ops to specify the device driver to
bind
vfio/mtty: Convert to use vfio_register_group_dev()
vfio/mdpy: Convert to use vfio_register_group_dev()
vfio/mbochs: Convert to use vfio_register_group_dev()
vfio/ap_ops: Convert to use vfio_register_group_dev()
vfio/ccw: Convert to use vfio_register_group_dev()
vfio/gvt: Convert to use vfio_register_group_dev()
vfio/mdev: Remove mdev_parent_ops dev_attr_groups
vfio/mdev: Remove mdev_parent_ops
vfio/mdev: Use the driver core to create the 'remove' file
vfio/mdev: Remove mdev drvdata
.../driver-api/vfio-mediated-device.rst | 55 ++---
Documentation/s390/vfio-ap.rst | 1 -
arch/s390/Kconfig | 2 +-
drivers/gpu/drm/i915/Kconfig | 2 +-
drivers/gpu/drm/i915/gvt/kvmgt.c | 210 +++++++++--------
drivers/s390/cio/vfio_ccw_drv.c | 21 +-
drivers/s390/cio/vfio_ccw_ops.c | 136 ++++++-----
drivers/s390/cio/vfio_ccw_private.h | 5 +
drivers/s390/crypto/vfio_ap_ops.c | 138 ++++++-----
drivers/s390/crypto/vfio_ap_private.h | 2 +
drivers/vfio/mdev/Kconfig | 7 -
drivers/vfio/mdev/Makefile | 1 -
drivers/vfio/mdev/mdev_core.c | 65 ++++--
drivers/vfio/mdev/mdev_driver.c | 10 +-
drivers/vfio/mdev/mdev_private.h | 4 +-
drivers/vfio/mdev/mdev_sysfs.c | 37 ++-
drivers/vfio/mdev/vfio_mdev.c | 180 ---------------
drivers/vfio/vfio.c | 6 +-
include/linux/mdev.h | 86 +------
include/linux/vfio.h | 4 +
samples/Kconfig | 6 +-
samples/vfio-mdev/mbochs.c | 166 +++++++------
samples/vfio-mdev/mdpy.c | 162 +++++++------
samples/vfio-mdev/mtty.c | 218 +++++++-----------
24 files changed, 649 insertions(+), 875 deletions(-)
delete mode 100644 drivers/vfio/mdev/vfio_mdev.c
--
2.31.1
^ permalink raw reply [flat|nested] 74+ messages in thread
* [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-23 23:02 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:02 UTC (permalink / raw)
To: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc,
linux-s390, Peter Oberparleiter, Halil Pasic, Pierre Morel,
Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
Prologue
========
This is series #3 in part of a larger work that arose from the minor
remark that the mdev_parent_ops indirection shim is useless and
complicates things.
It applies on top of Alex's current tree and requires the prior two
series.
This series achieves the removal of vfio_mdev.c. The future patches are all
focused on leveraging the changes made in the prior series to simplify the
API and device operation.
A preview of the future series's is here:
https://github.com/jgunthorpe/linux/pull/3/commits
========
The mdev bus's core part for managing the lifecycle of devices is mostly
as one would expect for a driver core bus subsystem.
However instead of having a normal 'struct device_driver' and binding the
actual mdev drivers through the standard driver core mechanisms it open
codes this with the struct mdev_parent_ops and provides a single driver
that shims between the VFIO core and the actual device driver.
Make every one of the mdev drivers implement an actual struct mdev_driver
and directly call vfio_register_group_dev() in the probe() function for
the mdev.
Squash what is left of the mdev_parent_ops into the mdev_driver and remap
create(), remove() and mdev_attr_groups to their driver core
equivalents. Arrange to bind the created mdev_device to the mdev_driver
that is provided by the end driver.
The actual execution flow doesn't change much, eg what was
parent_ops->create is now device_driver->probe and it is called at almost
the exact same time - except under the normal control of the driver core.
This allows deleting the entire mdev_drvdata, and tidying some of the
sysfs. Many places in the drivers start using container_of()
This cleanly splits the mdev sysfs GUID lifecycle management stuff from
the vfio_device implementation part, the only VFIO special part of mdev
that remains is the mdev specific iommu intervention.
Thanks,
Jason
Jason Gunthorpe (12):
vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
vfio/mdev: Allow the mdev_parent_ops to specify the device driver to
bind
vfio/mtty: Convert to use vfio_register_group_dev()
vfio/mdpy: Convert to use vfio_register_group_dev()
vfio/mbochs: Convert to use vfio_register_group_dev()
vfio/ap_ops: Convert to use vfio_register_group_dev()
vfio/ccw: Convert to use vfio_register_group_dev()
vfio/gvt: Convert to use vfio_register_group_dev()
vfio/mdev: Remove mdev_parent_ops dev_attr_groups
vfio/mdev: Remove mdev_parent_ops
vfio/mdev: Use the driver core to create the 'remove' file
vfio/mdev: Remove mdev drvdata
.../driver-api/vfio-mediated-device.rst | 55 ++---
Documentation/s390/vfio-ap.rst | 1 -
arch/s390/Kconfig | 2 +-
drivers/gpu/drm/i915/Kconfig | 2 +-
drivers/gpu/drm/i915/gvt/kvmgt.c | 210 +++++++++--------
drivers/s390/cio/vfio_ccw_drv.c | 21 +-
drivers/s390/cio/vfio_ccw_ops.c | 136 ++++++-----
drivers/s390/cio/vfio_ccw_private.h | 5 +
drivers/s390/crypto/vfio_ap_ops.c | 138 ++++++-----
drivers/s390/crypto/vfio_ap_private.h | 2 +
drivers/vfio/mdev/Kconfig | 7 -
drivers/vfio/mdev/Makefile | 1 -
drivers/vfio/mdev/mdev_core.c | 65 ++++--
drivers/vfio/mdev/mdev_driver.c | 10 +-
drivers/vfio/mdev/mdev_private.h | 4 +-
drivers/vfio/mdev/mdev_sysfs.c | 37 ++-
drivers/vfio/mdev/vfio_mdev.c | 180 ---------------
drivers/vfio/vfio.c | 6 +-
include/linux/mdev.h | 86 +------
include/linux/vfio.h | 4 +
samples/Kconfig | 6 +-
samples/vfio-mdev/mbochs.c | 166 +++++++------
samples/vfio-mdev/mdpy.c | 162 +++++++------
samples/vfio-mdev/mtty.c | 218 +++++++-----------
24 files changed, 649 insertions(+), 875 deletions(-)
delete mode 100644 drivers/vfio/mdev/vfio_mdev.c
--
2.31.1
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 74+ messages in thread
* [Intel-gfx] [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-23 23:02 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:02 UTC (permalink / raw)
To: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc,
linux-s390, Peter Oberparleiter, Halil Pasic, Pierre Morel,
Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
Prologue
========
This is series #3 in part of a larger work that arose from the minor
remark that the mdev_parent_ops indirection shim is useless and
complicates things.
It applies on top of Alex's current tree and requires the prior two
series.
This series achieves the removal of vfio_mdev.c. The future patches are all
focused on leveraging the changes made in the prior series to simplify the
API and device operation.
A preview of the future series's is here:
https://github.com/jgunthorpe/linux/pull/3/commits
========
The mdev bus's core part for managing the lifecycle of devices is mostly
as one would expect for a driver core bus subsystem.
However instead of having a normal 'struct device_driver' and binding the
actual mdev drivers through the standard driver core mechanisms it open
codes this with the struct mdev_parent_ops and provides a single driver
that shims between the VFIO core and the actual device driver.
Make every one of the mdev drivers implement an actual struct mdev_driver
and directly call vfio_register_group_dev() in the probe() function for
the mdev.
Squash what is left of the mdev_parent_ops into the mdev_driver and remap
create(), remove() and mdev_attr_groups to their driver core
equivalents. Arrange to bind the created mdev_device to the mdev_driver
that is provided by the end driver.
The actual execution flow doesn't change much, eg what was
parent_ops->create is now device_driver->probe and it is called at almost
the exact same time - except under the normal control of the driver core.
This allows deleting the entire mdev_drvdata, and tidying some of the
sysfs. Many places in the drivers start using container_of()
This cleanly splits the mdev sysfs GUID lifecycle management stuff from
the vfio_device implementation part, the only VFIO special part of mdev
that remains is the mdev specific iommu intervention.
Thanks,
Jason
Jason Gunthorpe (12):
vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
vfio/mdev: Allow the mdev_parent_ops to specify the device driver to
bind
vfio/mtty: Convert to use vfio_register_group_dev()
vfio/mdpy: Convert to use vfio_register_group_dev()
vfio/mbochs: Convert to use vfio_register_group_dev()
vfio/ap_ops: Convert to use vfio_register_group_dev()
vfio/ccw: Convert to use vfio_register_group_dev()
vfio/gvt: Convert to use vfio_register_group_dev()
vfio/mdev: Remove mdev_parent_ops dev_attr_groups
vfio/mdev: Remove mdev_parent_ops
vfio/mdev: Use the driver core to create the 'remove' file
vfio/mdev: Remove mdev drvdata
.../driver-api/vfio-mediated-device.rst | 55 ++---
Documentation/s390/vfio-ap.rst | 1 -
arch/s390/Kconfig | 2 +-
drivers/gpu/drm/i915/Kconfig | 2 +-
drivers/gpu/drm/i915/gvt/kvmgt.c | 210 +++++++++--------
drivers/s390/cio/vfio_ccw_drv.c | 21 +-
drivers/s390/cio/vfio_ccw_ops.c | 136 ++++++-----
drivers/s390/cio/vfio_ccw_private.h | 5 +
drivers/s390/crypto/vfio_ap_ops.c | 138 ++++++-----
drivers/s390/crypto/vfio_ap_private.h | 2 +
drivers/vfio/mdev/Kconfig | 7 -
drivers/vfio/mdev/Makefile | 1 -
drivers/vfio/mdev/mdev_core.c | 65 ++++--
drivers/vfio/mdev/mdev_driver.c | 10 +-
drivers/vfio/mdev/mdev_private.h | 4 +-
drivers/vfio/mdev/mdev_sysfs.c | 37 ++-
drivers/vfio/mdev/vfio_mdev.c | 180 ---------------
drivers/vfio/vfio.c | 6 +-
include/linux/mdev.h | 86 +------
include/linux/vfio.h | 4 +
samples/Kconfig | 6 +-
samples/vfio-mdev/mbochs.c | 166 +++++++------
samples/vfio-mdev/mdpy.c | 162 +++++++------
samples/vfio-mdev/mtty.c | 218 +++++++-----------
24 files changed, 649 insertions(+), 875 deletions(-)
delete mode 100644 drivers/vfio/mdev/vfio_mdev.c
--
2.31.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
2021-04-23 23:02 ` Jason Gunthorpe
(?)
@ 2021-04-23 23:02 ` Jason Gunthorpe
-1 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:02 UTC (permalink / raw)
To: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Vasily Gorbik, Heiko Carstens,
intel-gfx, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Halil Pasic, Pierre Morel, Rodrigo Vivi
Cc: Raj, Ashok, Dan Williams, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
For some reason the vfio_mdev shim mdev_driver has its own module and
kconfig. As the next patch requires access to it from mdev.ko merge the
two modules together and remove VFIO_MDEV_DEVICE.
A later patch deletes this driver entirely.
This also fixes a misuse of kconfig in the samples which prevented the
samples from being built in.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
Documentation/s390/vfio-ap.rst | 1 -
arch/s390/Kconfig | 2 +-
drivers/gpu/drm/i915/Kconfig | 2 +-
drivers/vfio/mdev/Kconfig | 7 -------
drivers/vfio/mdev/Makefile | 3 +--
drivers/vfio/mdev/mdev_core.c | 16 ++++++++++++++--
drivers/vfio/mdev/mdev_private.h | 2 ++
drivers/vfio/mdev/vfio_mdev.c | 24 +-----------------------
samples/Kconfig | 6 +++---
9 files changed, 23 insertions(+), 40 deletions(-)
diff --git a/Documentation/s390/vfio-ap.rst b/Documentation/s390/vfio-ap.rst
index e15436599086b7..f57ae621f33e89 100644
--- a/Documentation/s390/vfio-ap.rst
+++ b/Documentation/s390/vfio-ap.rst
@@ -514,7 +514,6 @@ These are the steps:
* S390_AP_IOMMU
* VFIO
* VFIO_MDEV
- * VFIO_MDEV_DEVICE
* KVM
If using make menuconfig select the following to build the vfio_ap module::
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index c1ff874e6c2e63..dc7928e37fa409 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -773,7 +773,7 @@ config VFIO_CCW
config VFIO_AP
def_tristate n
prompt "VFIO support for AP devices"
- depends on S390_AP_IOMMU && VFIO_MDEV_DEVICE && KVM
+ depends on S390_AP_IOMMU && VFIO_MDEV && KVM
depends on ZCRYPT
help
This driver grants access to Adjunct Processor (AP) devices
diff --git a/drivers/gpu/drm/i915/Kconfig b/drivers/gpu/drm/i915/Kconfig
index 483e9ff8ca1d23..388bc41aa1a75b 100644
--- a/drivers/gpu/drm/i915/Kconfig
+++ b/drivers/gpu/drm/i915/Kconfig
@@ -125,7 +125,7 @@ config DRM_I915_GVT_KVMGT
tristate "Enable KVM/VFIO support for Intel GVT-g"
depends on DRM_I915_GVT
depends on KVM
- depends on VFIO_MDEV && VFIO_MDEV_DEVICE
+ depends on VFIO_MDEV
default n
help
Choose this option if you want to enable KVMGT support for
diff --git a/drivers/vfio/mdev/Kconfig b/drivers/vfio/mdev/Kconfig
index 5da27f2100f9bd..763c877a1318bc 100644
--- a/drivers/vfio/mdev/Kconfig
+++ b/drivers/vfio/mdev/Kconfig
@@ -9,10 +9,3 @@ config VFIO_MDEV
See Documentation/driver-api/vfio-mediated-device.rst for more details.
If you don't know what do here, say N.
-
-config VFIO_MDEV_DEVICE
- tristate "VFIO driver for Mediated devices"
- depends on VFIO && VFIO_MDEV
- default n
- help
- VFIO based driver for Mediated devices.
diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
index 101516fdf3753e..ff9ecd80212503 100644
--- a/drivers/vfio/mdev/Makefile
+++ b/drivers/vfio/mdev/Makefile
@@ -1,6 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
-mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
+mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
obj-$(CONFIG_VFIO_MDEV) += mdev.o
-obj-$(CONFIG_VFIO_MDEV_DEVICE) += vfio_mdev.o
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index 2a85d6fcb7ddd0..ff8c1a84516698 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -360,11 +360,24 @@ int mdev_device_remove(struct mdev_device *mdev)
static int __init mdev_init(void)
{
- return mdev_bus_register();
+ int rc;
+
+ rc = mdev_bus_register();
+ if (rc)
+ return rc;
+ rc = mdev_register_driver(&vfio_mdev_driver);
+ if (rc)
+ goto err_bus;
+ return 0;
+err_bus:
+ mdev_bus_unregister();
+ return rc;
}
static void __exit mdev_exit(void)
{
+ mdev_unregister_driver(&vfio_mdev_driver);
+
if (mdev_bus_compat_class)
class_compat_unregister(mdev_bus_compat_class);
@@ -378,4 +391,3 @@ MODULE_VERSION(DRIVER_VERSION);
MODULE_LICENSE("GPL v2");
MODULE_AUTHOR(DRIVER_AUTHOR);
MODULE_DESCRIPTION(DRIVER_DESC);
-MODULE_SOFTDEP("post: vfio_mdev");
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
index a656cfe0346c33..5461b67582289f 100644
--- a/drivers/vfio/mdev/mdev_private.h
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -37,6 +37,8 @@ struct mdev_type {
#define to_mdev_type(_kobj) \
container_of(_kobj, struct mdev_type, kobj)
+extern struct mdev_driver vfio_mdev_driver;
+
int parent_create_sysfs_files(struct mdev_parent *parent);
void parent_remove_sysfs_files(struct mdev_parent *parent);
diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
index 922729071c5a8e..d5b4eede47c1a5 100644
--- a/drivers/vfio/mdev/vfio_mdev.c
+++ b/drivers/vfio/mdev/vfio_mdev.c
@@ -17,10 +17,6 @@
#include "mdev_private.h"
-#define DRIVER_VERSION "0.1"
-#define DRIVER_AUTHOR "NVIDIA Corporation"
-#define DRIVER_DESC "VFIO based driver for Mediated device"
-
static int vfio_mdev_open(struct vfio_device *core_vdev)
{
struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
@@ -151,7 +147,7 @@ static void vfio_mdev_remove(struct mdev_device *mdev)
kfree(vdev);
}
-static struct mdev_driver vfio_mdev_driver = {
+struct mdev_driver vfio_mdev_driver = {
.driver = {
.name = "vfio_mdev",
.owner = THIS_MODULE,
@@ -160,21 +156,3 @@ static struct mdev_driver vfio_mdev_driver = {
.probe = vfio_mdev_probe,
.remove = vfio_mdev_remove,
};
-
-static int __init vfio_mdev_init(void)
-{
- return mdev_register_driver(&vfio_mdev_driver);
-}
-
-static void __exit vfio_mdev_exit(void)
-{
- mdev_unregister_driver(&vfio_mdev_driver);
-}
-
-module_init(vfio_mdev_init)
-module_exit(vfio_mdev_exit)
-
-MODULE_VERSION(DRIVER_VERSION);
-MODULE_LICENSE("GPL v2");
-MODULE_AUTHOR(DRIVER_AUTHOR);
-MODULE_DESCRIPTION(DRIVER_DESC);
diff --git a/samples/Kconfig b/samples/Kconfig
index e76cdfc50e257d..2a4876e2ce0d03 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -147,14 +147,14 @@ config SAMPLE_UHID
config SAMPLE_VFIO_MDEV_MTTY
tristate "Build VFIO mtty example mediated device sample code -- loadable modules only"
- depends on VFIO_MDEV_DEVICE && m
+ depends on VFIO_MDEV
help
Build a virtual tty sample driver for use as a VFIO
mediated device
config SAMPLE_VFIO_MDEV_MDPY
tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
- depends on VFIO_MDEV_DEVICE && m
+ depends on VFIO_MDEV
help
Build a virtual display sample driver for use as a VFIO
mediated device. It is a simple framebuffer and supports
@@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
config SAMPLE_VFIO_MDEV_MBOCHS
tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
- depends on VFIO_MDEV_DEVICE && m
+ depends on VFIO_MDEV
select DMA_SHARED_BUFFER
help
Build a virtual display sample driver for use as a VFIO
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
@ 2021-04-23 23:02 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:02 UTC (permalink / raw)
To: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Vasily Gorbik, Heiko Carstens,
intel-gfx, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Halil Pasic, Pierre Morel, Rodrigo Vivi
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
For some reason the vfio_mdev shim mdev_driver has its own module and
kconfig. As the next patch requires access to it from mdev.ko merge the
two modules together and remove VFIO_MDEV_DEVICE.
A later patch deletes this driver entirely.
This also fixes a misuse of kconfig in the samples which prevented the
samples from being built in.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
Documentation/s390/vfio-ap.rst | 1 -
arch/s390/Kconfig | 2 +-
drivers/gpu/drm/i915/Kconfig | 2 +-
drivers/vfio/mdev/Kconfig | 7 -------
drivers/vfio/mdev/Makefile | 3 +--
drivers/vfio/mdev/mdev_core.c | 16 ++++++++++++++--
drivers/vfio/mdev/mdev_private.h | 2 ++
drivers/vfio/mdev/vfio_mdev.c | 24 +-----------------------
samples/Kconfig | 6 +++---
9 files changed, 23 insertions(+), 40 deletions(-)
diff --git a/Documentation/s390/vfio-ap.rst b/Documentation/s390/vfio-ap.rst
index e15436599086b7..f57ae621f33e89 100644
--- a/Documentation/s390/vfio-ap.rst
+++ b/Documentation/s390/vfio-ap.rst
@@ -514,7 +514,6 @@ These are the steps:
* S390_AP_IOMMU
* VFIO
* VFIO_MDEV
- * VFIO_MDEV_DEVICE
* KVM
If using make menuconfig select the following to build the vfio_ap module::
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index c1ff874e6c2e63..dc7928e37fa409 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -773,7 +773,7 @@ config VFIO_CCW
config VFIO_AP
def_tristate n
prompt "VFIO support for AP devices"
- depends on S390_AP_IOMMU && VFIO_MDEV_DEVICE && KVM
+ depends on S390_AP_IOMMU && VFIO_MDEV && KVM
depends on ZCRYPT
help
This driver grants access to Adjunct Processor (AP) devices
diff --git a/drivers/gpu/drm/i915/Kconfig b/drivers/gpu/drm/i915/Kconfig
index 483e9ff8ca1d23..388bc41aa1a75b 100644
--- a/drivers/gpu/drm/i915/Kconfig
+++ b/drivers/gpu/drm/i915/Kconfig
@@ -125,7 +125,7 @@ config DRM_I915_GVT_KVMGT
tristate "Enable KVM/VFIO support for Intel GVT-g"
depends on DRM_I915_GVT
depends on KVM
- depends on VFIO_MDEV && VFIO_MDEV_DEVICE
+ depends on VFIO_MDEV
default n
help
Choose this option if you want to enable KVMGT support for
diff --git a/drivers/vfio/mdev/Kconfig b/drivers/vfio/mdev/Kconfig
index 5da27f2100f9bd..763c877a1318bc 100644
--- a/drivers/vfio/mdev/Kconfig
+++ b/drivers/vfio/mdev/Kconfig
@@ -9,10 +9,3 @@ config VFIO_MDEV
See Documentation/driver-api/vfio-mediated-device.rst for more details.
If you don't know what do here, say N.
-
-config VFIO_MDEV_DEVICE
- tristate "VFIO driver for Mediated devices"
- depends on VFIO && VFIO_MDEV
- default n
- help
- VFIO based driver for Mediated devices.
diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
index 101516fdf3753e..ff9ecd80212503 100644
--- a/drivers/vfio/mdev/Makefile
+++ b/drivers/vfio/mdev/Makefile
@@ -1,6 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
-mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
+mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
obj-$(CONFIG_VFIO_MDEV) += mdev.o
-obj-$(CONFIG_VFIO_MDEV_DEVICE) += vfio_mdev.o
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index 2a85d6fcb7ddd0..ff8c1a84516698 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -360,11 +360,24 @@ int mdev_device_remove(struct mdev_device *mdev)
static int __init mdev_init(void)
{
- return mdev_bus_register();
+ int rc;
+
+ rc = mdev_bus_register();
+ if (rc)
+ return rc;
+ rc = mdev_register_driver(&vfio_mdev_driver);
+ if (rc)
+ goto err_bus;
+ return 0;
+err_bus:
+ mdev_bus_unregister();
+ return rc;
}
static void __exit mdev_exit(void)
{
+ mdev_unregister_driver(&vfio_mdev_driver);
+
if (mdev_bus_compat_class)
class_compat_unregister(mdev_bus_compat_class);
@@ -378,4 +391,3 @@ MODULE_VERSION(DRIVER_VERSION);
MODULE_LICENSE("GPL v2");
MODULE_AUTHOR(DRIVER_AUTHOR);
MODULE_DESCRIPTION(DRIVER_DESC);
-MODULE_SOFTDEP("post: vfio_mdev");
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
index a656cfe0346c33..5461b67582289f 100644
--- a/drivers/vfio/mdev/mdev_private.h
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -37,6 +37,8 @@ struct mdev_type {
#define to_mdev_type(_kobj) \
container_of(_kobj, struct mdev_type, kobj)
+extern struct mdev_driver vfio_mdev_driver;
+
int parent_create_sysfs_files(struct mdev_parent *parent);
void parent_remove_sysfs_files(struct mdev_parent *parent);
diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
index 922729071c5a8e..d5b4eede47c1a5 100644
--- a/drivers/vfio/mdev/vfio_mdev.c
+++ b/drivers/vfio/mdev/vfio_mdev.c
@@ -17,10 +17,6 @@
#include "mdev_private.h"
-#define DRIVER_VERSION "0.1"
-#define DRIVER_AUTHOR "NVIDIA Corporation"
-#define DRIVER_DESC "VFIO based driver for Mediated device"
-
static int vfio_mdev_open(struct vfio_device *core_vdev)
{
struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
@@ -151,7 +147,7 @@ static void vfio_mdev_remove(struct mdev_device *mdev)
kfree(vdev);
}
-static struct mdev_driver vfio_mdev_driver = {
+struct mdev_driver vfio_mdev_driver = {
.driver = {
.name = "vfio_mdev",
.owner = THIS_MODULE,
@@ -160,21 +156,3 @@ static struct mdev_driver vfio_mdev_driver = {
.probe = vfio_mdev_probe,
.remove = vfio_mdev_remove,
};
-
-static int __init vfio_mdev_init(void)
-{
- return mdev_register_driver(&vfio_mdev_driver);
-}
-
-static void __exit vfio_mdev_exit(void)
-{
- mdev_unregister_driver(&vfio_mdev_driver);
-}
-
-module_init(vfio_mdev_init)
-module_exit(vfio_mdev_exit)
-
-MODULE_VERSION(DRIVER_VERSION);
-MODULE_LICENSE("GPL v2");
-MODULE_AUTHOR(DRIVER_AUTHOR);
-MODULE_DESCRIPTION(DRIVER_DESC);
diff --git a/samples/Kconfig b/samples/Kconfig
index e76cdfc50e257d..2a4876e2ce0d03 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -147,14 +147,14 @@ config SAMPLE_UHID
config SAMPLE_VFIO_MDEV_MTTY
tristate "Build VFIO mtty example mediated device sample code -- loadable modules only"
- depends on VFIO_MDEV_DEVICE && m
+ depends on VFIO_MDEV
help
Build a virtual tty sample driver for use as a VFIO
mediated device
config SAMPLE_VFIO_MDEV_MDPY
tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
- depends on VFIO_MDEV_DEVICE && m
+ depends on VFIO_MDEV
help
Build a virtual display sample driver for use as a VFIO
mediated device. It is a simple framebuffer and supports
@@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
config SAMPLE_VFIO_MDEV_MBOCHS
tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
- depends on VFIO_MDEV_DEVICE && m
+ depends on VFIO_MDEV
select DMA_SHARED_BUFFER
help
Build a virtual display sample driver for use as a VFIO
--
2.31.1
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [Intel-gfx] [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
@ 2021-04-23 23:02 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:02 UTC (permalink / raw)
To: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Vasily Gorbik, Heiko Carstens,
intel-gfx, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Halil Pasic, Pierre Morel, Rodrigo Vivi
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
For some reason the vfio_mdev shim mdev_driver has its own module and
kconfig. As the next patch requires access to it from mdev.ko merge the
two modules together and remove VFIO_MDEV_DEVICE.
A later patch deletes this driver entirely.
This also fixes a misuse of kconfig in the samples which prevented the
samples from being built in.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
Documentation/s390/vfio-ap.rst | 1 -
arch/s390/Kconfig | 2 +-
drivers/gpu/drm/i915/Kconfig | 2 +-
drivers/vfio/mdev/Kconfig | 7 -------
drivers/vfio/mdev/Makefile | 3 +--
drivers/vfio/mdev/mdev_core.c | 16 ++++++++++++++--
drivers/vfio/mdev/mdev_private.h | 2 ++
drivers/vfio/mdev/vfio_mdev.c | 24 +-----------------------
samples/Kconfig | 6 +++---
9 files changed, 23 insertions(+), 40 deletions(-)
diff --git a/Documentation/s390/vfio-ap.rst b/Documentation/s390/vfio-ap.rst
index e15436599086b7..f57ae621f33e89 100644
--- a/Documentation/s390/vfio-ap.rst
+++ b/Documentation/s390/vfio-ap.rst
@@ -514,7 +514,6 @@ These are the steps:
* S390_AP_IOMMU
* VFIO
* VFIO_MDEV
- * VFIO_MDEV_DEVICE
* KVM
If using make menuconfig select the following to build the vfio_ap module::
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index c1ff874e6c2e63..dc7928e37fa409 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -773,7 +773,7 @@ config VFIO_CCW
config VFIO_AP
def_tristate n
prompt "VFIO support for AP devices"
- depends on S390_AP_IOMMU && VFIO_MDEV_DEVICE && KVM
+ depends on S390_AP_IOMMU && VFIO_MDEV && KVM
depends on ZCRYPT
help
This driver grants access to Adjunct Processor (AP) devices
diff --git a/drivers/gpu/drm/i915/Kconfig b/drivers/gpu/drm/i915/Kconfig
index 483e9ff8ca1d23..388bc41aa1a75b 100644
--- a/drivers/gpu/drm/i915/Kconfig
+++ b/drivers/gpu/drm/i915/Kconfig
@@ -125,7 +125,7 @@ config DRM_I915_GVT_KVMGT
tristate "Enable KVM/VFIO support for Intel GVT-g"
depends on DRM_I915_GVT
depends on KVM
- depends on VFIO_MDEV && VFIO_MDEV_DEVICE
+ depends on VFIO_MDEV
default n
help
Choose this option if you want to enable KVMGT support for
diff --git a/drivers/vfio/mdev/Kconfig b/drivers/vfio/mdev/Kconfig
index 5da27f2100f9bd..763c877a1318bc 100644
--- a/drivers/vfio/mdev/Kconfig
+++ b/drivers/vfio/mdev/Kconfig
@@ -9,10 +9,3 @@ config VFIO_MDEV
See Documentation/driver-api/vfio-mediated-device.rst for more details.
If you don't know what do here, say N.
-
-config VFIO_MDEV_DEVICE
- tristate "VFIO driver for Mediated devices"
- depends on VFIO && VFIO_MDEV
- default n
- help
- VFIO based driver for Mediated devices.
diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
index 101516fdf3753e..ff9ecd80212503 100644
--- a/drivers/vfio/mdev/Makefile
+++ b/drivers/vfio/mdev/Makefile
@@ -1,6 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
-mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
+mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
obj-$(CONFIG_VFIO_MDEV) += mdev.o
-obj-$(CONFIG_VFIO_MDEV_DEVICE) += vfio_mdev.o
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index 2a85d6fcb7ddd0..ff8c1a84516698 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -360,11 +360,24 @@ int mdev_device_remove(struct mdev_device *mdev)
static int __init mdev_init(void)
{
- return mdev_bus_register();
+ int rc;
+
+ rc = mdev_bus_register();
+ if (rc)
+ return rc;
+ rc = mdev_register_driver(&vfio_mdev_driver);
+ if (rc)
+ goto err_bus;
+ return 0;
+err_bus:
+ mdev_bus_unregister();
+ return rc;
}
static void __exit mdev_exit(void)
{
+ mdev_unregister_driver(&vfio_mdev_driver);
+
if (mdev_bus_compat_class)
class_compat_unregister(mdev_bus_compat_class);
@@ -378,4 +391,3 @@ MODULE_VERSION(DRIVER_VERSION);
MODULE_LICENSE("GPL v2");
MODULE_AUTHOR(DRIVER_AUTHOR);
MODULE_DESCRIPTION(DRIVER_DESC);
-MODULE_SOFTDEP("post: vfio_mdev");
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
index a656cfe0346c33..5461b67582289f 100644
--- a/drivers/vfio/mdev/mdev_private.h
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -37,6 +37,8 @@ struct mdev_type {
#define to_mdev_type(_kobj) \
container_of(_kobj, struct mdev_type, kobj)
+extern struct mdev_driver vfio_mdev_driver;
+
int parent_create_sysfs_files(struct mdev_parent *parent);
void parent_remove_sysfs_files(struct mdev_parent *parent);
diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
index 922729071c5a8e..d5b4eede47c1a5 100644
--- a/drivers/vfio/mdev/vfio_mdev.c
+++ b/drivers/vfio/mdev/vfio_mdev.c
@@ -17,10 +17,6 @@
#include "mdev_private.h"
-#define DRIVER_VERSION "0.1"
-#define DRIVER_AUTHOR "NVIDIA Corporation"
-#define DRIVER_DESC "VFIO based driver for Mediated device"
-
static int vfio_mdev_open(struct vfio_device *core_vdev)
{
struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
@@ -151,7 +147,7 @@ static void vfio_mdev_remove(struct mdev_device *mdev)
kfree(vdev);
}
-static struct mdev_driver vfio_mdev_driver = {
+struct mdev_driver vfio_mdev_driver = {
.driver = {
.name = "vfio_mdev",
.owner = THIS_MODULE,
@@ -160,21 +156,3 @@ static struct mdev_driver vfio_mdev_driver = {
.probe = vfio_mdev_probe,
.remove = vfio_mdev_remove,
};
-
-static int __init vfio_mdev_init(void)
-{
- return mdev_register_driver(&vfio_mdev_driver);
-}
-
-static void __exit vfio_mdev_exit(void)
-{
- mdev_unregister_driver(&vfio_mdev_driver);
-}
-
-module_init(vfio_mdev_init)
-module_exit(vfio_mdev_exit)
-
-MODULE_VERSION(DRIVER_VERSION);
-MODULE_LICENSE("GPL v2");
-MODULE_AUTHOR(DRIVER_AUTHOR);
-MODULE_DESCRIPTION(DRIVER_DESC);
diff --git a/samples/Kconfig b/samples/Kconfig
index e76cdfc50e257d..2a4876e2ce0d03 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -147,14 +147,14 @@ config SAMPLE_UHID
config SAMPLE_VFIO_MDEV_MTTY
tristate "Build VFIO mtty example mediated device sample code -- loadable modules only"
- depends on VFIO_MDEV_DEVICE && m
+ depends on VFIO_MDEV
help
Build a virtual tty sample driver for use as a VFIO
mediated device
config SAMPLE_VFIO_MDEV_MDPY
tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
- depends on VFIO_MDEV_DEVICE && m
+ depends on VFIO_MDEV
help
Build a virtual display sample driver for use as a VFIO
mediated device. It is a simple framebuffer and supports
@@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
config SAMPLE_VFIO_MDEV_MBOCHS
tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
- depends on VFIO_MDEV_DEVICE && m
+ depends on VFIO_MDEV
select DMA_SHARED_BUFFER
help
Build a virtual display sample driver for use as a VFIO
--
2.31.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 02/12] vfio/mdev: Allow the mdev_parent_ops to specify the device driver to bind
2021-04-23 23:02 ` Jason Gunthorpe
` (2 preceding siblings ...)
(?)
@ 2021-04-23 23:02 ` Jason Gunthorpe
2021-04-26 14:02 ` Christoph Hellwig
-1 siblings, 1 reply; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:02 UTC (permalink / raw)
To: Alex Williamson, Cornelia Huck, kvm, Kirti Wankhede
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
This allows a mdev driver to opt out of using vfio_mdev.c, instead the
driver will provide a 'struct mdev_driver' and register directly with the
driver core.
Much of mdev_parent_ops becomes unused in this mode:
- create()/remove() are done via the mdev_driver probe()/remove()
- mdev_attr_groups becomes mdev_driver driver.dev_groups
- Wrapper function callbacks are replaced with the same ones from
struct vfio_device_ops
Following patches convert all the drivers.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/vfio/mdev/mdev_core.c | 64 ++++++++++++++++++++++++++++-----
drivers/vfio/mdev/mdev_driver.c | 17 ++++++++-
include/linux/mdev.h | 3 ++
3 files changed, 75 insertions(+), 9 deletions(-)
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index ff8c1a84516698..51b8a9fcf866ad 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -94,9 +94,11 @@ static void mdev_device_remove_common(struct mdev_device *mdev)
mdev_remove_sysfs_files(mdev);
device_del(&mdev->dev);
lockdep_assert_held(&parent->unreg_sem);
- ret = parent->ops->remove(mdev);
- if (ret)
- dev_err(&mdev->dev, "Remove failed: err=%d\n", ret);
+ if (parent->ops->remove) {
+ ret = parent->ops->remove(mdev);
+ if (ret)
+ dev_err(&mdev->dev, "Remove failed: err=%d\n", ret);
+ }
/* Balances with device_initialize() */
put_device(&mdev->dev);
@@ -127,7 +129,9 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
char *envp[] = { env_string, NULL };
/* check for mandatory ops */
- if (!ops || !ops->create || !ops->remove || !ops->supported_type_groups)
+ if (!ops || !ops->supported_type_groups)
+ return -EINVAL;
+ if (!ops->device_driver && (!ops->create || !ops->remove))
return -EINVAL;
dev = get_device(dev);
@@ -251,6 +255,43 @@ static void mdev_device_release(struct device *dev)
kfree(mdev);
}
+/*
+ * mdev drivers can refuse to bind during probe(), in this case we want to fail
+ * the creation of the mdev all the way back to sysfs. This is a weird model
+ * that doesn't fit in the driver core well, nor does it seem to appear any
+ * place else in the kernel, so use a simple hack.
+ */
+static int mdev_bind_driver(struct mdev_device *mdev)
+{
+ struct mdev_driver *drv = mdev->type->parent->ops->device_driver;
+ int ret;
+
+ if (!drv)
+ drv = &vfio_mdev_driver;
+
+ while (1) {
+ device_lock(&mdev->dev);
+ if (mdev->dev.driver == &drv->driver) {
+ ret = 0;
+ goto out_unlock;
+ }
+ if (mdev->probe_err) {
+ ret = mdev->probe_err;
+ goto out_unlock;
+ }
+ device_unlock(&mdev->dev);
+ ret = device_attach(&mdev->dev);
+ if (ret)
+ return ret;
+ mdev->probe_err = -EINVAL;
+ }
+ return 0;
+
+out_unlock:
+ device_unlock(&mdev->dev);
+ return ret;
+}
+
int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
{
int ret;
@@ -296,14 +337,20 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
goto out_put_device;
}
- ret = parent->ops->create(mdev);
- if (ret)
- goto out_unlock;
+ if (parent->ops->create) {
+ ret = parent->ops->create(mdev);
+ if (ret)
+ goto out_unlock;
+ }
ret = device_add(&mdev->dev);
if (ret)
goto out_remove;
+ ret = mdev_bind_driver(mdev);
+ if (ret)
+ goto out_del;
+
ret = mdev_create_sysfs_files(mdev);
if (ret)
goto out_del;
@@ -317,7 +364,8 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
out_del:
device_del(&mdev->dev);
out_remove:
- parent->ops->remove(mdev);
+ if (parent->ops->remove)
+ parent->ops->remove(mdev);
out_unlock:
up_read(&parent->unreg_sem);
out_put_device:
diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
index 041699571b7e55..6e96c023d7823d 100644
--- a/drivers/vfio/mdev/mdev_driver.c
+++ b/drivers/vfio/mdev/mdev_driver.c
@@ -49,7 +49,7 @@ static int mdev_probe(struct device *dev)
return ret;
if (drv->probe) {
- ret = drv->probe(mdev);
+ ret = mdev->probe_err = drv->probe(mdev);
if (ret)
mdev_detach_iommu(mdev);
}
@@ -71,10 +71,25 @@ static int mdev_remove(struct device *dev)
return 0;
}
+static int mdev_match(struct device *dev, struct device_driver *drv)
+{
+ struct mdev_device *mdev = to_mdev_device(dev);
+ struct mdev_driver *target = mdev->type->parent->ops->device_driver;
+
+ /*
+ * The ops specify the device driver to connect, fall back to the old
+ * shim driver if the driver hasn't been converted.
+ */
+ if (!target)
+ target = &vfio_mdev_driver;
+ return drv == &target->driver;
+}
+
struct bus_type mdev_bus_type = {
.name = "mdev",
.probe = mdev_probe,
.remove = mdev_remove,
+ .match = mdev_match,
};
EXPORT_SYMBOL_GPL(mdev_bus_type);
diff --git a/include/linux/mdev.h b/include/linux/mdev.h
index 1fb34ea394ad46..49cc4f65120d57 100644
--- a/include/linux/mdev.h
+++ b/include/linux/mdev.h
@@ -19,6 +19,7 @@ struct mdev_device {
struct list_head next;
struct mdev_type *type;
struct device *iommu_device;
+ int probe_err;
bool active;
};
@@ -55,6 +56,7 @@ struct device *mtype_get_parent_dev(struct mdev_type *mtype);
* register the device to mdev module.
*
* @owner: The module owner.
+ * @device_driver: Which device driver to probe() on newly created devices
* @dev_attr_groups: Attributes of the parent device.
* @mdev_attr_groups: Attributes of the mediated device.
* @supported_type_groups: Attributes to define supported types. It is mandatory
@@ -103,6 +105,7 @@ struct device *mtype_get_parent_dev(struct mdev_type *mtype);
**/
struct mdev_parent_ops {
struct module *owner;
+ struct mdev_driver *device_driver;
const struct attribute_group **dev_attr_groups;
const struct attribute_group **mdev_attr_groups;
struct attribute_group **supported_type_groups;
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 03/12] vfio/mtty: Convert to use vfio_register_group_dev()
2021-04-23 23:02 ` Jason Gunthorpe
` (3 preceding siblings ...)
(?)
@ 2021-04-23 23:03 ` Jason Gunthorpe
2021-04-26 14:06 ` Christoph Hellwig
-1 siblings, 1 reply; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: kvm, Kirti Wankhede
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
This is straightforward conversion, the mdev_state is actually serving as
the vfio_device and we can replace all the mdev_get_drvdata()'s and the
wonky dead code with a simple container_of()
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
samples/vfio-mdev/mtty.c | 185 ++++++++++++++++++---------------------
1 file changed, 83 insertions(+), 102 deletions(-)
diff --git a/samples/vfio-mdev/mtty.c b/samples/vfio-mdev/mtty.c
index b9b24be4abdab7..d2a168420b775d 100644
--- a/samples/vfio-mdev/mtty.c
+++ b/samples/vfio-mdev/mtty.c
@@ -127,6 +127,7 @@ struct serial_port {
/* State of each mdev device */
struct mdev_state {
+ struct vfio_device vdev;
int irq_fd;
struct eventfd_ctx *intx_evtfd;
struct eventfd_ctx *msi_evtfd;
@@ -150,6 +151,8 @@ static const struct file_operations vd_fops = {
.owner = THIS_MODULE,
};
+static const struct vfio_device_ops mtty_dev_ops;
+
/* function prototypes */
static int mtty_trigger_interrupt(struct mdev_state *mdev_state);
@@ -631,22 +634,15 @@ static void mdev_read_base(struct mdev_state *mdev_state)
}
}
-static ssize_t mdev_access(struct mdev_device *mdev, u8 *buf, size_t count,
+static ssize_t mdev_access(struct mdev_state *mdev_state, u8 *buf, size_t count,
loff_t pos, bool is_write)
{
- struct mdev_state *mdev_state;
unsigned int index;
loff_t offset;
int ret = 0;
- if (!mdev || !buf)
- return -EINVAL;
-
- mdev_state = mdev_get_drvdata(mdev);
- if (!mdev_state) {
- pr_err("%s mdev_state not found\n", __func__);
+ if (!buf)
return -EINVAL;
- }
mutex_lock(&mdev_state->ops_lock);
@@ -708,15 +704,18 @@ static ssize_t mdev_access(struct mdev_device *mdev, u8 *buf, size_t count,
return ret;
}
-static int mtty_create(struct mdev_device *mdev)
+static int mtty_probe(struct mdev_device *mdev)
{
struct mdev_state *mdev_state;
int nr_ports = mdev_get_type_group_id(mdev) + 1;
+ int ret;
mdev_state = kzalloc(sizeof(struct mdev_state), GFP_KERNEL);
if (mdev_state == NULL)
return -ENOMEM;
+ vfio_init_group_dev(&mdev_state->vdev, &mdev->dev, &mtty_dev_ops);
+
mdev_state->nr_ports = nr_ports;
mdev_state->irq_index = -1;
mdev_state->s[0].max_fifo_size = MAX_FIFO_SIZE;
@@ -731,7 +730,6 @@ static int mtty_create(struct mdev_device *mdev)
mutex_init(&mdev_state->ops_lock);
mdev_state->mdev = mdev;
- mdev_set_drvdata(mdev, mdev_state);
mtty_create_config_space(mdev_state);
@@ -739,50 +737,40 @@ static int mtty_create(struct mdev_device *mdev)
list_add(&mdev_state->next, &mdev_devices_list);
mutex_unlock(&mdev_list_lock);
+ ret = vfio_register_group_dev(&mdev_state->vdev);
+ if (ret) {
+ kfree(mdev_state);
+ return ret;
+ }
+ dev_set_drvdata(&mdev->dev, mdev_state);
return 0;
}
-static int mtty_remove(struct mdev_device *mdev)
+static void mtty_remove(struct mdev_device *mdev)
{
- struct mdev_state *mds, *tmp_mds;
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
- int ret = -EINVAL;
+ struct mdev_state *mdev_state = dev_get_drvdata(&mdev->dev);
+ vfio_unregister_group_dev(&mdev_state->vdev);
mutex_lock(&mdev_list_lock);
- list_for_each_entry_safe(mds, tmp_mds, &mdev_devices_list, next) {
- if (mdev_state == mds) {
- list_del(&mdev_state->next);
- mdev_set_drvdata(mdev, NULL);
- kfree(mdev_state->vconfig);
- kfree(mdev_state);
- ret = 0;
- break;
- }
- }
+ list_del(&mdev_state->next);
mutex_unlock(&mdev_list_lock);
- return ret;
+ kfree(mdev_state->vconfig);
+ kfree(mdev_state);
}
-static int mtty_reset(struct mdev_device *mdev)
+static int mtty_reset(struct mdev_state *mdev_stte)
{
- struct mdev_state *mdev_state;
-
- if (!mdev)
- return -EINVAL;
-
- mdev_state = mdev_get_drvdata(mdev);
- if (!mdev_state)
- return -EINVAL;
-
pr_info("%s: called\n", __func__);
return 0;
}
-static ssize_t mtty_read(struct mdev_device *mdev, char __user *buf,
+static ssize_t mtty_read(struct vfio_device *vdev, char __user *buf,
size_t count, loff_t *ppos)
{
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
unsigned int done = 0;
int ret;
@@ -792,7 +780,7 @@ static ssize_t mtty_read(struct mdev_device *mdev, char __user *buf,
if (count >= 4 && !(*ppos % 4)) {
u32 val;
- ret = mdev_access(mdev, (u8 *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (u8 *)&val, sizeof(val),
*ppos, false);
if (ret <= 0)
goto read_err;
@@ -804,7 +792,7 @@ static ssize_t mtty_read(struct mdev_device *mdev, char __user *buf,
} else if (count >= 2 && !(*ppos % 2)) {
u16 val;
- ret = mdev_access(mdev, (u8 *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (u8 *)&val, sizeof(val),
*ppos, false);
if (ret <= 0)
goto read_err;
@@ -816,7 +804,7 @@ static ssize_t mtty_read(struct mdev_device *mdev, char __user *buf,
} else {
u8 val;
- ret = mdev_access(mdev, (u8 *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (u8 *)&val, sizeof(val),
*ppos, false);
if (ret <= 0)
goto read_err;
@@ -839,9 +827,11 @@ static ssize_t mtty_read(struct mdev_device *mdev, char __user *buf,
return -EFAULT;
}
-static ssize_t mtty_write(struct mdev_device *mdev, const char __user *buf,
+static ssize_t mtty_write(struct vfio_device *vdev, const char __user *buf,
size_t count, loff_t *ppos)
{
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
unsigned int done = 0;
int ret;
@@ -854,7 +844,7 @@ static ssize_t mtty_write(struct mdev_device *mdev, const char __user *buf,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = mdev_access(mdev, (u8 *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (u8 *)&val, sizeof(val),
*ppos, true);
if (ret <= 0)
goto write_err;
@@ -866,7 +856,7 @@ static ssize_t mtty_write(struct mdev_device *mdev, const char __user *buf,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = mdev_access(mdev, (u8 *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (u8 *)&val, sizeof(val),
*ppos, true);
if (ret <= 0)
goto write_err;
@@ -878,7 +868,7 @@ static ssize_t mtty_write(struct mdev_device *mdev, const char __user *buf,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = mdev_access(mdev, (u8 *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (u8 *)&val, sizeof(val),
*ppos, true);
if (ret <= 0)
goto write_err;
@@ -896,19 +886,11 @@ static ssize_t mtty_write(struct mdev_device *mdev, const char __user *buf,
return -EFAULT;
}
-static int mtty_set_irqs(struct mdev_device *mdev, uint32_t flags,
+static int mtty_set_irqs(struct mdev_state *mdev_state, uint32_t flags,
unsigned int index, unsigned int start,
unsigned int count, void *data)
{
int ret = 0;
- struct mdev_state *mdev_state;
-
- if (!mdev)
- return -EINVAL;
-
- mdev_state = mdev_get_drvdata(mdev);
- if (!mdev_state)
- return -EINVAL;
mutex_lock(&mdev_state->ops_lock);
switch (index) {
@@ -1024,21 +1006,13 @@ static int mtty_trigger_interrupt(struct mdev_state *mdev_state)
return ret;
}
-static int mtty_get_region_info(struct mdev_device *mdev,
+static int mtty_get_region_info(struct mdev_state *mdev_state,
struct vfio_region_info *region_info,
u16 *cap_type_id, void **cap_type)
{
unsigned int size = 0;
- struct mdev_state *mdev_state;
u32 bar_index;
- if (!mdev)
- return -EINVAL;
-
- mdev_state = mdev_get_drvdata(mdev);
- if (!mdev_state)
- return -EINVAL;
-
bar_index = region_info->index;
if (bar_index >= VFIO_PCI_NUM_REGIONS)
return -EINVAL;
@@ -1073,8 +1047,7 @@ static int mtty_get_region_info(struct mdev_device *mdev,
return 0;
}
-static int mtty_get_irq_info(struct mdev_device *mdev,
- struct vfio_irq_info *irq_info)
+static int mtty_get_irq_info(struct vfio_irq_info *irq_info)
{
switch (irq_info->index) {
case VFIO_PCI_INTX_IRQ_INDEX:
@@ -1098,8 +1071,7 @@ static int mtty_get_irq_info(struct mdev_device *mdev,
return 0;
}
-static int mtty_get_device_info(struct mdev_device *mdev,
- struct vfio_device_info *dev_info)
+static int mtty_get_device_info(struct vfio_device_info *dev_info)
{
dev_info->flags = VFIO_DEVICE_FLAGS_PCI;
dev_info->num_regions = VFIO_PCI_NUM_REGIONS;
@@ -1108,19 +1080,13 @@ static int mtty_get_device_info(struct mdev_device *mdev,
return 0;
}
-static long mtty_ioctl(struct mdev_device *mdev, unsigned int cmd,
+static long mtty_ioctl(struct vfio_device *vdev, unsigned int cmd,
unsigned long arg)
{
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
int ret = 0;
unsigned long minsz;
- struct mdev_state *mdev_state;
-
- if (!mdev)
- return -EINVAL;
-
- mdev_state = mdev_get_drvdata(mdev);
- if (!mdev_state)
- return -ENODEV;
switch (cmd) {
case VFIO_DEVICE_GET_INFO:
@@ -1135,7 +1101,7 @@ static long mtty_ioctl(struct mdev_device *mdev, unsigned int cmd,
if (info.argsz < minsz)
return -EINVAL;
- ret = mtty_get_device_info(mdev, &info);
+ ret = mtty_get_device_info(&info);
if (ret)
return ret;
@@ -1160,7 +1126,7 @@ static long mtty_ioctl(struct mdev_device *mdev, unsigned int cmd,
if (info.argsz < minsz)
return -EINVAL;
- ret = mtty_get_region_info(mdev, &info, &cap_type_id,
+ ret = mtty_get_region_info(mdev_state, &info, &cap_type_id,
&cap_type);
if (ret)
return ret;
@@ -1184,7 +1150,7 @@ static long mtty_ioctl(struct mdev_device *mdev, unsigned int cmd,
(info.index >= mdev_state->dev_info.num_irqs))
return -EINVAL;
- ret = mtty_get_irq_info(mdev, &info);
+ ret = mtty_get_irq_info(&info);
if (ret)
return ret;
@@ -1218,25 +1184,25 @@ static long mtty_ioctl(struct mdev_device *mdev, unsigned int cmd,
return PTR_ERR(data);
}
- ret = mtty_set_irqs(mdev, hdr.flags, hdr.index, hdr.start,
+ ret = mtty_set_irqs(mdev_state, hdr.flags, hdr.index, hdr.start,
hdr.count, data);
kfree(ptr);
return ret;
}
case VFIO_DEVICE_RESET:
- return mtty_reset(mdev);
+ return mtty_reset(mdev_state);
}
return -ENOTTY;
}
-static int mtty_open(struct mdev_device *mdev)
+static int mtty_open(struct vfio_device *vdev)
{
pr_info("%s\n", __func__);
return 0;
}
-static void mtty_close(struct mdev_device *mdev)
+static void mtty_close(struct vfio_device *mdev)
{
pr_info("%s\n", __func__);
}
@@ -1351,18 +1317,31 @@ static struct attribute_group *mdev_type_groups[] = {
NULL,
};
+static const struct vfio_device_ops mtty_dev_ops = {
+ .name = "vfio-mdev",
+ .open = mtty_open,
+ .release = mtty_close,
+ .read = mtty_read,
+ .write = mtty_write,
+ .ioctl = mtty_ioctl,
+};
+
+static struct mdev_driver mtty_driver = {
+ .driver = {
+ .name = "mtty",
+ .owner = THIS_MODULE,
+ .mod_name = KBUILD_MODNAME,
+ .dev_groups = mdev_dev_groups,
+ },
+ .probe = mtty_probe,
+ .remove = mtty_remove,
+};
+
static const struct mdev_parent_ops mdev_fops = {
.owner = THIS_MODULE,
+ .device_driver = &mtty_driver,
.dev_attr_groups = mtty_dev_groups,
- .mdev_attr_groups = mdev_dev_groups,
.supported_type_groups = mdev_type_groups,
- .create = mtty_create,
- .remove = mtty_remove,
- .open = mtty_open,
- .release = mtty_close,
- .read = mtty_read,
- .write = mtty_write,
- .ioctl = mtty_ioctl,
};
static void mtty_device_release(struct device *dev)
@@ -1393,12 +1372,16 @@ static int __init mtty_dev_init(void)
pr_info("major_number:%d\n", MAJOR(mtty_dev.vd_devt));
+ ret = mdev_register_driver(&mtty_driver);
+ if (ret)
+ goto err_cdev;
+
mtty_dev.vd_class = class_create(THIS_MODULE, MTTY_CLASS_NAME);
if (IS_ERR(mtty_dev.vd_class)) {
pr_err("Error: failed to register mtty_dev class\n");
ret = PTR_ERR(mtty_dev.vd_class);
- goto failed1;
+ goto err_driver;
}
mtty_dev.dev.class = mtty_dev.vd_class;
@@ -1407,28 +1390,25 @@ static int __init mtty_dev_init(void)
ret = device_register(&mtty_dev.dev);
if (ret)
- goto failed2;
+ goto err_class;
ret = mdev_register_device(&mtty_dev.dev, &mdev_fops);
if (ret)
- goto failed3;
+ goto err_device;
mutex_init(&mdev_list_lock);
INIT_LIST_HEAD(&mdev_devices_list);
+ return 0;
- goto all_done;
-
-failed3:
-
+err_device:
device_unregister(&mtty_dev.dev);
-failed2:
+err_class:
class_destroy(mtty_dev.vd_class);
-
-failed1:
+err_driver:
+ mdev_unregister_driver(&mtty_driver);
+err_cdev:
cdev_del(&mtty_dev.vd_cdev);
unregister_chrdev_region(mtty_dev.vd_devt, MINORMASK + 1);
-
-all_done:
return ret;
}
@@ -1439,6 +1419,7 @@ static void __exit mtty_dev_exit(void)
device_unregister(&mtty_dev.dev);
idr_destroy(&mtty_dev.vd_idr);
+ mdev_unregister_driver(&mtty_driver);
cdev_del(&mtty_dev.vd_cdev);
unregister_chrdev_region(mtty_dev.vd_devt, MINORMASK + 1);
class_destroy(mtty_dev.vd_class);
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 04/12] vfio/mdpy: Convert to use vfio_register_group_dev()
2021-04-23 23:02 ` Jason Gunthorpe
` (4 preceding siblings ...)
(?)
@ 2021-04-23 23:03 ` Jason Gunthorpe
2021-04-26 14:06 ` Christoph Hellwig
-1 siblings, 1 reply; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: kvm, Kirti Wankhede
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
This is straightforward conversion, the mdev_state is actually serving as
the vfio_device and we can replace all the mdev_get_drvdata()'s and the
wonky dead code with a simple container_of().
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
samples/vfio-mdev/mdpy.c | 159 ++++++++++++++++++++++-----------------
1 file changed, 88 insertions(+), 71 deletions(-)
diff --git a/samples/vfio-mdev/mdpy.c b/samples/vfio-mdev/mdpy.c
index 885b88ea20e234..82638de333330d 100644
--- a/samples/vfio-mdev/mdpy.c
+++ b/samples/vfio-mdev/mdpy.c
@@ -85,9 +85,11 @@ static struct class *mdpy_class;
static struct cdev mdpy_cdev;
static struct device mdpy_dev;
static u32 mdpy_count;
+static const struct vfio_device_ops mdpy_dev_ops;
/* State of each mdev device */
struct mdev_state {
+ struct vfio_device vdev;
u8 *vconfig;
u32 bar_mask;
struct mutex ops_lock;
@@ -162,11 +164,9 @@ static void handle_pci_cfg_write(struct mdev_state *mdev_state, u16 offset,
}
}
-static ssize_t mdev_access(struct mdev_device *mdev, char *buf, size_t count,
- loff_t pos, bool is_write)
+static ssize_t mdev_access(struct mdev_state *mdev_state, char *buf,
+ size_t count, loff_t pos, bool is_write)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
- struct device *dev = mdev_dev(mdev);
int ret = 0;
mutex_lock(&mdev_state->ops_lock);
@@ -187,8 +187,9 @@ static ssize_t mdev_access(struct mdev_device *mdev, char *buf, size_t count,
memcpy(buf, mdev_state->memblk, count);
} else {
- dev_info(dev, "%s: %s @0x%llx (unhandled)\n",
- __func__, is_write ? "WR" : "RD", pos);
+ dev_info(mdev_state->vdev.dev,
+ "%s: %s @0x%llx (unhandled)\n", __func__,
+ is_write ? "WR" : "RD", pos);
ret = -1;
goto accessfailed;
}
@@ -202,9 +203,8 @@ static ssize_t mdev_access(struct mdev_device *mdev, char *buf, size_t count,
return ret;
}
-static int mdpy_reset(struct mdev_device *mdev)
+static int mdpy_reset(struct mdev_state *mdev_state)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
u32 stride, i;
/* initialize with gray gradient */
@@ -216,13 +216,14 @@ static int mdpy_reset(struct mdev_device *mdev)
return 0;
}
-static int mdpy_create(struct mdev_device *mdev)
+static int mdpy_probe(struct mdev_device *mdev)
{
const struct mdpy_type *type =
&mdpy_types[mdev_get_type_group_id(mdev)];
struct device *dev = mdev_dev(mdev);
struct mdev_state *mdev_state;
u32 fbsize;
+ int ret;
if (mdpy_count >= max_devices)
return -ENOMEM;
@@ -230,6 +231,7 @@ static int mdpy_create(struct mdev_device *mdev)
mdev_state = kzalloc(sizeof(struct mdev_state), GFP_KERNEL);
if (mdev_state == NULL)
return -ENOMEM;
+ vfio_init_group_dev(&mdev_state->vdev, &mdev->dev, &mdpy_dev_ops);
mdev_state->vconfig = kzalloc(MDPY_CONFIG_SPACE_SIZE, GFP_KERNEL);
if (mdev_state->vconfig == NULL) {
@@ -250,36 +252,41 @@ static int mdpy_create(struct mdev_device *mdev)
mutex_init(&mdev_state->ops_lock);
mdev_state->mdev = mdev;
- mdev_set_drvdata(mdev, mdev_state);
-
mdev_state->type = type;
mdev_state->memsize = fbsize;
mdpy_create_config_space(mdev_state);
- mdpy_reset(mdev);
+ mdpy_reset(mdev_state);
mdpy_count++;
+
+ ret = vfio_register_group_dev(&mdev_state->vdev);
+ if (ret) {
+ kfree(mdev_state);
+ return ret;
+ }
+ dev_set_drvdata(&mdev->dev, mdev_state);
return 0;
}
-static int mdpy_remove(struct mdev_device *mdev)
+static void mdpy_remove(struct mdev_device *mdev)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
- struct device *dev = mdev_dev(mdev);
+ struct mdev_state *mdev_state = dev_get_drvdata(&mdev->dev);
- dev_info(dev, "%s\n", __func__);
+ dev_info(&mdev->dev, "%s\n", __func__);
- mdev_set_drvdata(mdev, NULL);
+ vfio_unregister_group_dev(&mdev_state->vdev);
vfree(mdev_state->memblk);
kfree(mdev_state->vconfig);
kfree(mdev_state);
mdpy_count--;
- return 0;
}
-static ssize_t mdpy_read(struct mdev_device *mdev, char __user *buf,
+static ssize_t mdpy_read(struct vfio_device *vdev, char __user *buf,
size_t count, loff_t *ppos)
{
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
unsigned int done = 0;
int ret;
@@ -289,8 +296,8 @@ static ssize_t mdpy_read(struct mdev_device *mdev, char __user *buf,
if (count >= 4 && !(*ppos % 4)) {
u32 val;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
- *ppos, false);
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
+ *ppos, false);
if (ret <= 0)
goto read_err;
@@ -301,7 +308,7 @@ static ssize_t mdpy_read(struct mdev_device *mdev, char __user *buf,
} else if (count >= 2 && !(*ppos % 2)) {
u16 val;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, false);
if (ret <= 0)
goto read_err;
@@ -313,7 +320,7 @@ static ssize_t mdpy_read(struct mdev_device *mdev, char __user *buf,
} else {
u8 val;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, false);
if (ret <= 0)
goto read_err;
@@ -336,9 +343,11 @@ static ssize_t mdpy_read(struct mdev_device *mdev, char __user *buf,
return -EFAULT;
}
-static ssize_t mdpy_write(struct mdev_device *mdev, const char __user *buf,
+static ssize_t mdpy_write(struct vfio_device *vdev, const char __user *buf,
size_t count, loff_t *ppos)
{
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
unsigned int done = 0;
int ret;
@@ -351,7 +360,7 @@ static ssize_t mdpy_write(struct mdev_device *mdev, const char __user *buf,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, true);
if (ret <= 0)
goto write_err;
@@ -363,7 +372,7 @@ static ssize_t mdpy_write(struct mdev_device *mdev, const char __user *buf,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, true);
if (ret <= 0)
goto write_err;
@@ -375,7 +384,7 @@ static ssize_t mdpy_write(struct mdev_device *mdev, const char __user *buf,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, true);
if (ret <= 0)
goto write_err;
@@ -393,9 +402,10 @@ static ssize_t mdpy_write(struct mdev_device *mdev, const char __user *buf,
return -EFAULT;
}
-static int mdpy_mmap(struct mdev_device *mdev, struct vm_area_struct *vma)
+static int mdpy_mmap(struct vfio_device *vdev, struct vm_area_struct *vma)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
if (vma->vm_pgoff != MDPY_MEMORY_BAR_OFFSET >> PAGE_SHIFT)
return -EINVAL;
@@ -411,16 +421,10 @@ static int mdpy_mmap(struct mdev_device *mdev, struct vm_area_struct *vma)
vma->vm_end - vma->vm_start);
}
-static int mdpy_get_region_info(struct mdev_device *mdev,
+static int mdpy_get_region_info(struct mdev_state *mdev_state,
struct vfio_region_info *region_info,
u16 *cap_type_id, void **cap_type)
{
- struct mdev_state *mdev_state;
-
- mdev_state = mdev_get_drvdata(mdev);
- if (!mdev_state)
- return -EINVAL;
-
if (region_info->index >= VFIO_PCI_NUM_REGIONS &&
region_info->index != MDPY_DISPLAY_REGION)
return -EINVAL;
@@ -449,15 +453,13 @@ static int mdpy_get_region_info(struct mdev_device *mdev,
return 0;
}
-static int mdpy_get_irq_info(struct mdev_device *mdev,
- struct vfio_irq_info *irq_info)
+static int mdpy_get_irq_info(struct vfio_irq_info *irq_info)
{
irq_info->count = 0;
return 0;
}
-static int mdpy_get_device_info(struct mdev_device *mdev,
- struct vfio_device_info *dev_info)
+static int mdpy_get_device_info(struct vfio_device_info *dev_info)
{
dev_info->flags = VFIO_DEVICE_FLAGS_PCI;
dev_info->num_regions = VFIO_PCI_NUM_REGIONS;
@@ -465,11 +467,9 @@ static int mdpy_get_device_info(struct mdev_device *mdev,
return 0;
}
-static int mdpy_query_gfx_plane(struct mdev_device *mdev,
+static int mdpy_query_gfx_plane(struct mdev_state *mdev_state,
struct vfio_device_gfx_plane_info *plane)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
-
if (plane->flags & VFIO_GFX_PLANE_TYPE_PROBE) {
if (plane->flags == (VFIO_GFX_PLANE_TYPE_PROBE |
VFIO_GFX_PLANE_TYPE_REGION))
@@ -498,14 +498,13 @@ static int mdpy_query_gfx_plane(struct mdev_device *mdev,
return 0;
}
-static long mdpy_ioctl(struct mdev_device *mdev, unsigned int cmd,
+static long mdpy_ioctl(struct vfio_device *vdev, unsigned int cmd,
unsigned long arg)
{
int ret = 0;
unsigned long minsz;
- struct mdev_state *mdev_state;
-
- mdev_state = mdev_get_drvdata(mdev);
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
switch (cmd) {
case VFIO_DEVICE_GET_INFO:
@@ -520,7 +519,7 @@ static long mdpy_ioctl(struct mdev_device *mdev, unsigned int cmd,
if (info.argsz < minsz)
return -EINVAL;
- ret = mdpy_get_device_info(mdev, &info);
+ ret = mdpy_get_device_info(&info);
if (ret)
return ret;
@@ -545,7 +544,7 @@ static long mdpy_ioctl(struct mdev_device *mdev, unsigned int cmd,
if (info.argsz < minsz)
return -EINVAL;
- ret = mdpy_get_region_info(mdev, &info, &cap_type_id,
+ ret = mdpy_get_region_info(mdev_state, &info, &cap_type_id,
&cap_type);
if (ret)
return ret;
@@ -569,7 +568,7 @@ static long mdpy_ioctl(struct mdev_device *mdev, unsigned int cmd,
(info.index >= mdev_state->dev_info.num_irqs))
return -EINVAL;
- ret = mdpy_get_irq_info(mdev, &info);
+ ret = mdpy_get_irq_info(&info);
if (ret)
return ret;
@@ -592,7 +591,7 @@ static long mdpy_ioctl(struct mdev_device *mdev, unsigned int cmd,
if (plane.argsz < minsz)
return -EINVAL;
- ret = mdpy_query_gfx_plane(mdev, &plane);
+ ret = mdpy_query_gfx_plane(mdev_state, &plane);
if (ret)
return ret;
@@ -606,12 +605,12 @@ static long mdpy_ioctl(struct mdev_device *mdev, unsigned int cmd,
return -EINVAL;
case VFIO_DEVICE_RESET:
- return mdpy_reset(mdev);
+ return mdpy_reset(mdev_state);
}
return -ENOTTY;
}
-static int mdpy_open(struct mdev_device *mdev)
+static int mdpy_open(struct vfio_device *vdev)
{
if (!try_module_get(THIS_MODULE))
return -ENODEV;
@@ -619,7 +618,7 @@ static int mdpy_open(struct mdev_device *mdev)
return 0;
}
-static void mdpy_close(struct mdev_device *mdev)
+static void mdpy_close(struct vfio_device *vdev)
{
module_put(THIS_MODULE);
}
@@ -628,8 +627,7 @@ static ssize_t
resolution_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct mdev_device *mdev = mdev_from_dev(dev);
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
+ struct mdev_state *mdev_state = dev_get_drvdata(dev);
return sprintf(buf, "%dx%d\n",
mdev_state->type->width,
@@ -719,18 +717,30 @@ static struct attribute_group *mdev_type_groups[] = {
NULL,
};
+static const struct vfio_device_ops mdpy_dev_ops = {
+ .open = mdpy_open,
+ .release = mdpy_close,
+ .read = mdpy_read,
+ .write = mdpy_write,
+ .ioctl = mdpy_ioctl,
+ .mmap = mdpy_mmap,
+};
+
+static struct mdev_driver mdpy_driver = {
+ .driver = {
+ .name = "mdpy",
+ .owner = THIS_MODULE,
+ .mod_name = KBUILD_MODNAME,
+ .dev_groups = mdev_dev_groups,
+ },
+ .probe = mdpy_probe,
+ .remove = mdpy_remove,
+};
+
static const struct mdev_parent_ops mdev_fops = {
.owner = THIS_MODULE,
- .mdev_attr_groups = mdev_dev_groups,
+ .device_driver = &mdpy_driver,
.supported_type_groups = mdev_type_groups,
- .create = mdpy_create,
- .remove = mdpy_remove,
- .open = mdpy_open,
- .release = mdpy_close,
- .read = mdpy_read,
- .write = mdpy_write,
- .ioctl = mdpy_ioctl,
- .mmap = mdpy_mmap,
};
static const struct file_operations vd_fops = {
@@ -755,11 +765,15 @@ static int __init mdpy_dev_init(void)
cdev_add(&mdpy_cdev, mdpy_devt, MINORMASK + 1);
pr_info("%s: major %d\n", __func__, MAJOR(mdpy_devt));
+ ret = mdev_register_driver(&mdpy_driver);
+ if (ret)
+ goto err_cdev;
+
mdpy_class = class_create(THIS_MODULE, MDPY_CLASS_NAME);
if (IS_ERR(mdpy_class)) {
pr_err("Error: failed to register mdpy_dev class\n");
ret = PTR_ERR(mdpy_class);
- goto failed1;
+ goto err_driver;
}
mdpy_dev.class = mdpy_class;
mdpy_dev.release = mdpy_device_release;
@@ -767,19 +781,21 @@ static int __init mdpy_dev_init(void)
ret = device_register(&mdpy_dev);
if (ret)
- goto failed2;
+ goto err_class;
ret = mdev_register_device(&mdpy_dev, &mdev_fops);
if (ret)
- goto failed3;
+ goto err_device;
return 0;
-failed3:
+err_device:
device_unregister(&mdpy_dev);
-failed2:
+err_class:
class_destroy(mdpy_class);
-failed1:
+err_driver:
+ mdev_unregister_driver(&mdpy_driver);
+err_cdev:
cdev_del(&mdpy_cdev);
unregister_chrdev_region(mdpy_devt, MINORMASK + 1);
return ret;
@@ -791,6 +807,7 @@ static void __exit mdpy_dev_exit(void)
mdev_unregister_device(&mdpy_dev);
device_unregister(&mdpy_dev);
+ mdev_unregister_driver(&mdpy_driver);
cdev_del(&mdpy_cdev);
unregister_chrdev_region(mdpy_devt, MINORMASK + 1);
class_destroy(mdpy_class);
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 05/12] vfio/mbochs: Convert to use vfio_register_group_dev()
2021-04-23 23:02 ` Jason Gunthorpe
` (5 preceding siblings ...)
(?)
@ 2021-04-23 23:03 ` Jason Gunthorpe
2021-04-26 14:06 ` Christoph Hellwig
-1 siblings, 1 reply; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: kvm, Kirti Wankhede
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
This is straightforward conversion, the mdev_state is actually serving as
the vfio_device and we can replace all the mdev_get_drvdata()'s and the
wonky dead code with a simple container_of().
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
samples/vfio-mdev/mbochs.c | 163 +++++++++++++++++++++----------------
1 file changed, 91 insertions(+), 72 deletions(-)
diff --git a/samples/vfio-mdev/mbochs.c b/samples/vfio-mdev/mbochs.c
index 861c76914e7639..e18821a8a6beb8 100644
--- a/samples/vfio-mdev/mbochs.c
+++ b/samples/vfio-mdev/mbochs.c
@@ -130,6 +130,7 @@ static struct class *mbochs_class;
static struct cdev mbochs_cdev;
static struct device mbochs_dev;
static int mbochs_used_mbytes;
+static const struct vfio_device_ops mbochs_dev_ops;
struct vfio_region_info_ext {
struct vfio_region_info base;
@@ -160,6 +161,7 @@ struct mbochs_dmabuf {
/* State of each mdev device */
struct mdev_state {
+ struct vfio_device vdev;
u8 *vconfig;
u64 bar_mask[3];
u32 memory_bar_mask;
@@ -425,11 +427,9 @@ static void handle_edid_blob(struct mdev_state *mdev_state, u16 offset,
memcpy(buf, mdev_state->edid_blob + offset, count);
}
-static ssize_t mdev_access(struct mdev_device *mdev, char *buf, size_t count,
- loff_t pos, bool is_write)
+static ssize_t mdev_access(struct mdev_state *mdev_state, char *buf,
+ size_t count, loff_t pos, bool is_write)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
- struct device *dev = mdev_dev(mdev);
struct page *pg;
loff_t poff;
char *map;
@@ -478,7 +478,7 @@ static ssize_t mdev_access(struct mdev_device *mdev, char *buf, size_t count,
put_page(pg);
} else {
- dev_dbg(dev, "%s: %s @0x%llx (unhandled)\n",
+ dev_dbg(mdev_state->vdev.dev, "%s: %s @0x%llx (unhandled)\n",
__func__, is_write ? "WR" : "RD", pos);
ret = -1;
goto accessfailed;
@@ -493,9 +493,8 @@ static ssize_t mdev_access(struct mdev_device *mdev, char *buf, size_t count,
return ret;
}
-static int mbochs_reset(struct mdev_device *mdev)
+static int mbochs_reset(struct mdev_state *mdev_state)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
u32 size64k = mdev_state->memsize / (64 * 1024);
int i;
@@ -506,12 +505,13 @@ static int mbochs_reset(struct mdev_device *mdev)
return 0;
}
-static int mbochs_create(struct mdev_device *mdev)
+static int mbochs_probe(struct mdev_device *mdev)
{
const struct mbochs_type *type =
&mbochs_types[mdev_get_type_group_id(mdev)];
struct device *dev = mdev_dev(mdev);
struct mdev_state *mdev_state;
+ int ret = -ENOMEM;
if (!type)
type = &mbochs_types[0];
@@ -521,6 +521,7 @@ static int mbochs_create(struct mdev_device *mdev)
mdev_state = kzalloc(sizeof(struct mdev_state), GFP_KERNEL);
if (mdev_state == NULL)
return -ENOMEM;
+ vfio_init_group_dev(&mdev_state->vdev, &mdev->dev, &mbochs_dev_ops);
mdev_state->vconfig = kzalloc(MBOCHS_CONFIG_SPACE_SIZE, GFP_KERNEL);
if (mdev_state->vconfig == NULL)
@@ -539,7 +540,6 @@ static int mbochs_create(struct mdev_device *mdev)
mutex_init(&mdev_state->ops_lock);
mdev_state->mdev = mdev;
- mdev_set_drvdata(mdev, mdev_state);
INIT_LIST_HEAD(&mdev_state->dmabufs);
mdev_state->next_id = 1;
@@ -549,32 +549,38 @@ static int mbochs_create(struct mdev_device *mdev)
mdev_state->edid_regs.edid_offset = MBOCHS_EDID_BLOB_OFFSET;
mdev_state->edid_regs.edid_max_size = sizeof(mdev_state->edid_blob);
mbochs_create_config_space(mdev_state);
- mbochs_reset(mdev);
+ mbochs_reset(mdev_state);
mbochs_used_mbytes += type->mbytes;
+
+ ret = vfio_register_group_dev(&mdev_state->vdev);
+ if (ret)
+ goto err_mem;
+ dev_set_drvdata(&mdev->dev, mdev_state);
return 0;
err_mem:
kfree(mdev_state->vconfig);
kfree(mdev_state);
- return -ENOMEM;
+ return ret;
}
-static int mbochs_remove(struct mdev_device *mdev)
+static void mbochs_remove(struct mdev_device *mdev)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
+ struct mdev_state *mdev_state = dev_get_drvdata(&mdev->dev);
mbochs_used_mbytes -= mdev_state->type->mbytes;
- mdev_set_drvdata(mdev, NULL);
+ vfio_unregister_group_dev(&mdev_state->vdev);
kfree(mdev_state->pages);
kfree(mdev_state->vconfig);
kfree(mdev_state);
- return 0;
}
-static ssize_t mbochs_read(struct mdev_device *mdev, char __user *buf,
+static ssize_t mbochs_read(struct vfio_device *vdev, char __user *buf,
size_t count, loff_t *ppos)
{
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
unsigned int done = 0;
int ret;
@@ -584,7 +590,7 @@ static ssize_t mbochs_read(struct mdev_device *mdev, char __user *buf,
if (count >= 4 && !(*ppos % 4)) {
u32 val;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, false);
if (ret <= 0)
goto read_err;
@@ -596,7 +602,7 @@ static ssize_t mbochs_read(struct mdev_device *mdev, char __user *buf,
} else if (count >= 2 && !(*ppos % 2)) {
u16 val;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, false);
if (ret <= 0)
goto read_err;
@@ -608,7 +614,7 @@ static ssize_t mbochs_read(struct mdev_device *mdev, char __user *buf,
} else {
u8 val;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, false);
if (ret <= 0)
goto read_err;
@@ -631,9 +637,11 @@ static ssize_t mbochs_read(struct mdev_device *mdev, char __user *buf,
return -EFAULT;
}
-static ssize_t mbochs_write(struct mdev_device *mdev, const char __user *buf,
+static ssize_t mbochs_write(struct vfio_device *vdev, const char __user *buf,
size_t count, loff_t *ppos)
{
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
unsigned int done = 0;
int ret;
@@ -646,7 +654,7 @@ static ssize_t mbochs_write(struct mdev_device *mdev, const char __user *buf,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, true);
if (ret <= 0)
goto write_err;
@@ -658,7 +666,7 @@ static ssize_t mbochs_write(struct mdev_device *mdev, const char __user *buf,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, true);
if (ret <= 0)
goto write_err;
@@ -670,7 +678,7 @@ static ssize_t mbochs_write(struct mdev_device *mdev, const char __user *buf,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = mdev_access(mdev, (char *)&val, sizeof(val),
+ ret = mdev_access(mdev_state, (char *)&val, sizeof(val),
*ppos, true);
if (ret <= 0)
goto write_err;
@@ -756,9 +764,10 @@ static const struct vm_operations_struct mbochs_region_vm_ops = {
.fault = mbochs_region_vm_fault,
};
-static int mbochs_mmap(struct mdev_device *mdev, struct vm_area_struct *vma)
+static int mbochs_mmap(struct vfio_device *vdev, struct vm_area_struct *vma)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
if (vma->vm_pgoff != MBOCHS_MEMORY_BAR_OFFSET >> PAGE_SHIFT)
return -EINVAL;
@@ -965,7 +974,7 @@ mbochs_dmabuf_find_by_id(struct mdev_state *mdev_state, u32 id)
static int mbochs_dmabuf_export(struct mbochs_dmabuf *dmabuf)
{
struct mdev_state *mdev_state = dmabuf->mdev_state;
- struct device *dev = mdev_dev(mdev_state->mdev);
+ struct device *dev = mdev_state->vdev.dev;
DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
struct dma_buf *buf;
@@ -993,15 +1002,10 @@ static int mbochs_dmabuf_export(struct mbochs_dmabuf *dmabuf)
return 0;
}
-static int mbochs_get_region_info(struct mdev_device *mdev,
+static int mbochs_get_region_info(struct mdev_state *mdev_state,
struct vfio_region_info_ext *ext)
{
struct vfio_region_info *region_info = &ext->base;
- struct mdev_state *mdev_state;
-
- mdev_state = mdev_get_drvdata(mdev);
- if (!mdev_state)
- return -EINVAL;
if (region_info->index >= MBOCHS_NUM_REGIONS)
return -EINVAL;
@@ -1049,15 +1053,13 @@ static int mbochs_get_region_info(struct mdev_device *mdev,
return 0;
}
-static int mbochs_get_irq_info(struct mdev_device *mdev,
- struct vfio_irq_info *irq_info)
+static int mbochs_get_irq_info(struct vfio_irq_info *irq_info)
{
irq_info->count = 0;
return 0;
}
-static int mbochs_get_device_info(struct mdev_device *mdev,
- struct vfio_device_info *dev_info)
+static int mbochs_get_device_info(struct vfio_device_info *dev_info)
{
dev_info->flags = VFIO_DEVICE_FLAGS_PCI;
dev_info->num_regions = MBOCHS_NUM_REGIONS;
@@ -1065,11 +1067,9 @@ static int mbochs_get_device_info(struct mdev_device *mdev,
return 0;
}
-static int mbochs_query_gfx_plane(struct mdev_device *mdev,
+static int mbochs_query_gfx_plane(struct mdev_state *mdev_state,
struct vfio_device_gfx_plane_info *plane)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
- struct device *dev = mdev_dev(mdev);
struct mbochs_dmabuf *dmabuf;
struct mbochs_mode mode;
int ret;
@@ -1123,18 +1123,16 @@ static int mbochs_query_gfx_plane(struct mdev_device *mdev,
done:
if (plane->drm_plane_type == DRM_PLANE_TYPE_PRIMARY &&
mdev_state->active_id != plane->dmabuf_id) {
- dev_dbg(dev, "%s: primary: %d => %d\n", __func__,
- mdev_state->active_id, plane->dmabuf_id);
+ dev_dbg(mdev_state->vdev.dev, "%s: primary: %d => %d\n",
+ __func__, mdev_state->active_id, plane->dmabuf_id);
mdev_state->active_id = plane->dmabuf_id;
}
mutex_unlock(&mdev_state->ops_lock);
return 0;
}
-static int mbochs_get_gfx_dmabuf(struct mdev_device *mdev,
- u32 id)
+static int mbochs_get_gfx_dmabuf(struct mdev_state *mdev_state, u32 id)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
struct mbochs_dmabuf *dmabuf;
mutex_lock(&mdev_state->ops_lock);
@@ -1156,9 +1154,11 @@ static int mbochs_get_gfx_dmabuf(struct mdev_device *mdev,
return dma_buf_fd(dmabuf->buf, 0);
}
-static long mbochs_ioctl(struct mdev_device *mdev, unsigned int cmd,
- unsigned long arg)
+static long mbochs_ioctl(struct vfio_device *vdev, unsigned int cmd,
+ unsigned long arg)
{
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
int ret = 0;
unsigned long minsz, outsz;
@@ -1175,7 +1175,7 @@ static long mbochs_ioctl(struct mdev_device *mdev, unsigned int cmd,
if (info.argsz < minsz)
return -EINVAL;
- ret = mbochs_get_device_info(mdev, &info);
+ ret = mbochs_get_device_info(&info);
if (ret)
return ret;
@@ -1199,7 +1199,7 @@ static long mbochs_ioctl(struct mdev_device *mdev, unsigned int cmd,
if (outsz > sizeof(info))
return -EINVAL;
- ret = mbochs_get_region_info(mdev, &info);
+ ret = mbochs_get_region_info(mdev_state, &info);
if (ret)
return ret;
@@ -1222,7 +1222,7 @@ static long mbochs_ioctl(struct mdev_device *mdev, unsigned int cmd,
(info.index >= VFIO_PCI_NUM_IRQS))
return -EINVAL;
- ret = mbochs_get_irq_info(mdev, &info);
+ ret = mbochs_get_irq_info(&info);
if (ret)
return ret;
@@ -1245,7 +1245,7 @@ static long mbochs_ioctl(struct mdev_device *mdev, unsigned int cmd,
if (plane.argsz < minsz)
return -EINVAL;
- ret = mbochs_query_gfx_plane(mdev, &plane);
+ ret = mbochs_query_gfx_plane(mdev_state, &plane);
if (ret)
return ret;
@@ -1262,19 +1262,19 @@ static long mbochs_ioctl(struct mdev_device *mdev, unsigned int cmd,
if (get_user(dmabuf_id, (__u32 __user *)arg))
return -EFAULT;
- return mbochs_get_gfx_dmabuf(mdev, dmabuf_id);
+ return mbochs_get_gfx_dmabuf(mdev_state, dmabuf_id);
}
case VFIO_DEVICE_SET_IRQS:
return -EINVAL;
case VFIO_DEVICE_RESET:
- return mbochs_reset(mdev);
+ return mbochs_reset(mdev_state);
}
return -ENOTTY;
}
-static int mbochs_open(struct mdev_device *mdev)
+static int mbochs_open(struct vfio_device *vdev)
{
if (!try_module_get(THIS_MODULE))
return -ENODEV;
@@ -1282,9 +1282,10 @@ static int mbochs_open(struct mdev_device *mdev)
return 0;
}
-static void mbochs_close(struct mdev_device *mdev)
+static void mbochs_close(struct vfio_device *vdev)
{
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
+ struct mdev_state *mdev_state =
+ container_of(vdev, struct mdev_state, vdev);
struct mbochs_dmabuf *dmabuf, *tmp;
mutex_lock(&mdev_state->ops_lock);
@@ -1308,8 +1309,7 @@ static ssize_t
memory_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct mdev_device *mdev = mdev_from_dev(dev);
- struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
+ struct mdev_state *mdev_state = dev_get_drvdata(dev);
return sprintf(buf, "%d MB\n", mdev_state->type->mbytes);
}
@@ -1400,18 +1400,30 @@ static struct attribute_group *mdev_type_groups[] = {
NULL,
};
+static const struct vfio_device_ops mbochs_dev_ops = {
+ .open = mbochs_open,
+ .release = mbochs_close,
+ .read = mbochs_read,
+ .write = mbochs_write,
+ .ioctl = mbochs_ioctl,
+ .mmap = mbochs_mmap,
+};
+
+static struct mdev_driver mbochs_driver = {
+ .driver = {
+ .name = "mbochs",
+ .owner = THIS_MODULE,
+ .mod_name = KBUILD_MODNAME,
+ .dev_groups = mdev_dev_groups,
+ },
+ .probe = mbochs_probe,
+ .remove = mbochs_remove,
+};
+
static const struct mdev_parent_ops mdev_fops = {
.owner = THIS_MODULE,
- .mdev_attr_groups = mdev_dev_groups,
+ .device_driver = &mbochs_driver,
.supported_type_groups = mdev_type_groups,
- .create = mbochs_create,
- .remove = mbochs_remove,
- .open = mbochs_open,
- .release = mbochs_close,
- .read = mbochs_read,
- .write = mbochs_write,
- .ioctl = mbochs_ioctl,
- .mmap = mbochs_mmap,
};
static const struct file_operations vd_fops = {
@@ -1436,11 +1448,15 @@ static int __init mbochs_dev_init(void)
cdev_add(&mbochs_cdev, mbochs_devt, MINORMASK + 1);
pr_info("%s: major %d\n", __func__, MAJOR(mbochs_devt));
+ ret = mdev_register_driver(&mbochs_driver);
+ if (ret)
+ goto err_cdev;
+
mbochs_class = class_create(THIS_MODULE, MBOCHS_CLASS_NAME);
if (IS_ERR(mbochs_class)) {
pr_err("Error: failed to register mbochs_dev class\n");
ret = PTR_ERR(mbochs_class);
- goto failed1;
+ goto err_driver;
}
mbochs_dev.class = mbochs_class;
mbochs_dev.release = mbochs_device_release;
@@ -1448,19 +1464,21 @@ static int __init mbochs_dev_init(void)
ret = device_register(&mbochs_dev);
if (ret)
- goto failed2;
+ goto err_class;
ret = mdev_register_device(&mbochs_dev, &mdev_fops);
if (ret)
- goto failed3;
+ goto err_device;
return 0;
-failed3:
+err_device:
device_unregister(&mbochs_dev);
-failed2:
+err_class:
class_destroy(mbochs_class);
-failed1:
+err_driver:
+ mdev_unregister_driver(&mbochs_driver);
+err_cdev:
cdev_del(&mbochs_cdev);
unregister_chrdev_region(mbochs_devt, MINORMASK + 1);
return ret;
@@ -1472,6 +1490,7 @@ static void __exit mbochs_dev_exit(void)
mdev_unregister_device(&mbochs_dev);
device_unregister(&mbochs_dev);
+ mdev_unregister_driver(&mbochs_driver);
cdev_del(&mbochs_cdev);
unregister_chrdev_region(mbochs_devt, MINORMASK + 1);
class_destroy(mbochs_class);
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-04-23 23:02 ` Jason Gunthorpe
` (6 preceding siblings ...)
(?)
@ 2021-04-23 23:03 ` Jason Gunthorpe
2021-04-26 14:07 ` Christoph Hellwig
2021-04-26 17:48 ` Cornelia Huck
-1 siblings, 2 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: Tony Krowiak, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
This is straightforward conversion, the ap_matrix_mdev is actually serving
as the vfio_device and we can replace all the mdev_get_drvdata()'s with a
simple container_of().
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/s390/crypto/vfio_ap_ops.c | 137 ++++++++++++++++----------
drivers/s390/crypto/vfio_ap_private.h | 2 +
2 files changed, 89 insertions(+), 50 deletions(-)
diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
index 0ce00c9311d378..79872c857dd522 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -24,8 +24,9 @@
#define VFIO_AP_MDEV_TYPE_HWVIRT "passthrough"
#define VFIO_AP_MDEV_NAME_HWVIRT "VFIO AP Passthrough Device"
-static int vfio_ap_mdev_reset_queues(struct mdev_device *mdev);
+static int vfio_ap_mdev_reset_queues(struct ap_matrix_mdev *matrix_mdev);
static struct vfio_ap_queue *vfio_ap_find_queue(int apqn);
+static const struct vfio_device_ops vfio_ap_matrix_dev_ops;
static int match_apqn(struct device *dev, const void *data)
{
@@ -322,48 +323,63 @@ static void vfio_ap_matrix_init(struct ap_config_info *info,
matrix->adm_max = info->apxa ? info->Nd : 15;
}
-static int vfio_ap_mdev_create(struct mdev_device *mdev)
+static int vfio_ap_mdev_probe(struct mdev_device *mdev)
{
struct ap_matrix_mdev *matrix_mdev;
+ int ret;
if ((atomic_dec_if_positive(&matrix_dev->available_instances) < 0))
return -EPERM;
matrix_mdev = kzalloc(sizeof(*matrix_mdev), GFP_KERNEL);
if (!matrix_mdev) {
- atomic_inc(&matrix_dev->available_instances);
- return -ENOMEM;
+ ret = -ENOMEM;
+ goto err_atomic;
}
+ vfio_init_group_dev(&matrix_mdev->vdev, &mdev->dev,
+ &vfio_ap_matrix_dev_ops);
matrix_mdev->mdev = mdev;
vfio_ap_matrix_init(&matrix_dev->info, &matrix_mdev->matrix);
- mdev_set_drvdata(mdev, matrix_mdev);
matrix_mdev->pqap_hook.hook = handle_pqap;
matrix_mdev->pqap_hook.owner = THIS_MODULE;
mutex_lock(&matrix_dev->lock);
list_add(&matrix_mdev->node, &matrix_dev->mdev_list);
mutex_unlock(&matrix_dev->lock);
+ ret = vfio_register_group_dev(&matrix_mdev->vdev);
+ if (ret)
+ goto err_list;
+ dev_set_drvdata(&mdev->dev, matrix_mdev);
return 0;
+
+err_list:
+ mutex_lock(&matrix_dev->lock);
+ list_del(&matrix_mdev->node);
+ mutex_unlock(&matrix_dev->lock);
+ kfree(matrix_mdev);
+err_atomic:
+ atomic_inc(&matrix_dev->available_instances);
+ return ret;
}
-static int vfio_ap_mdev_remove(struct mdev_device *mdev)
+static void vfio_ap_mdev_remove(struct mdev_device *mdev)
{
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(&mdev->dev);
- if (matrix_mdev->kvm)
- return -EBUSY;
+ /* FIXME: Remove isn't allowed to fail */
+ if (WARN_ON(matrix_mdev->kvm))
+ return;
+
+ vfio_unregister_group_dev(&matrix_mdev->vdev);
mutex_lock(&matrix_dev->lock);
- vfio_ap_mdev_reset_queues(mdev);
+ vfio_ap_mdev_reset_queues(matrix_mdev);
list_del(&matrix_mdev->node);
mutex_unlock(&matrix_dev->lock);
kfree(matrix_mdev);
- mdev_set_drvdata(mdev, NULL);
atomic_inc(&matrix_dev->available_instances);
-
- return 0;
}
static ssize_t name_show(struct mdev_type *mtype,
@@ -605,8 +621,7 @@ static ssize_t assign_adapter_store(struct device *dev,
{
int ret;
unsigned long apid;
- struct mdev_device *mdev = mdev_from_dev(dev);
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(dev);
/* If the guest is running, disallow assignment of adapter */
if (matrix_mdev->kvm)
@@ -671,8 +686,7 @@ static ssize_t unassign_adapter_store(struct device *dev,
{
int ret;
unsigned long apid;
- struct mdev_device *mdev = mdev_from_dev(dev);
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(dev);
/* If the guest is running, disallow un-assignment of adapter */
if (matrix_mdev->kvm)
@@ -751,8 +765,7 @@ static ssize_t assign_domain_store(struct device *dev,
{
int ret;
unsigned long apqi;
- struct mdev_device *mdev = mdev_from_dev(dev);
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(dev);
unsigned long max_apqi = matrix_mdev->matrix.aqm_max;
/* If the guest is running, disallow assignment of domain */
@@ -813,8 +826,7 @@ static ssize_t unassign_domain_store(struct device *dev,
{
int ret;
unsigned long apqi;
- struct mdev_device *mdev = mdev_from_dev(dev);
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(dev);
/* If the guest is running, disallow un-assignment of domain */
if (matrix_mdev->kvm)
@@ -857,8 +869,7 @@ static ssize_t assign_control_domain_store(struct device *dev,
{
int ret;
unsigned long id;
- struct mdev_device *mdev = mdev_from_dev(dev);
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(dev);
/* If the guest is running, disallow assignment of control domain */
if (matrix_mdev->kvm)
@@ -906,8 +917,7 @@ static ssize_t unassign_control_domain_store(struct device *dev,
{
int ret;
unsigned long domid;
- struct mdev_device *mdev = mdev_from_dev(dev);
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(dev);
unsigned long max_domid = matrix_mdev->matrix.adm_max;
/* If the guest is running, disallow un-assignment of control domain */
@@ -936,8 +946,7 @@ static ssize_t control_domains_show(struct device *dev,
int nchars = 0;
int n;
char *bufpos = buf;
- struct mdev_device *mdev = mdev_from_dev(dev);
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(dev);
unsigned long max_domid = matrix_mdev->matrix.adm_max;
mutex_lock(&matrix_dev->lock);
@@ -955,8 +964,7 @@ static DEVICE_ATTR_RO(control_domains);
static ssize_t matrix_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct mdev_device *mdev = mdev_from_dev(dev);
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(dev);
char *bufpos = buf;
unsigned long apid;
unsigned long apqi;
@@ -1085,7 +1093,7 @@ static void vfio_ap_mdev_unset_kvm(struct ap_matrix_mdev *matrix_mdev)
{
kvm_arch_crypto_clear_masks(matrix_mdev->kvm);
matrix_mdev->kvm->arch.crypto.pqap_hook = NULL;
- vfio_ap_mdev_reset_queues(matrix_mdev->mdev);
+ vfio_ap_mdev_reset_queues(matrix_mdev);
kvm_put_kvm(matrix_mdev->kvm);
matrix_mdev->kvm = NULL;
}
@@ -1195,13 +1203,12 @@ int vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q,
return ret;
}
-static int vfio_ap_mdev_reset_queues(struct mdev_device *mdev)
+static int vfio_ap_mdev_reset_queues(struct ap_matrix_mdev *matrix_mdev)
{
int ret;
int rc = 0;
unsigned long apid, apqi;
struct vfio_ap_queue *q;
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
for_each_set_bit_inv(apid, matrix_mdev->matrix.apm,
matrix_mdev->matrix.apm_max + 1) {
@@ -1222,9 +1229,10 @@ static int vfio_ap_mdev_reset_queues(struct mdev_device *mdev)
return rc;
}
-static int vfio_ap_mdev_open(struct mdev_device *mdev)
+static int vfio_ap_mdev_open(struct vfio_device *vdev)
{
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev =
+ container_of(vdev, struct ap_matrix_mdev, vdev);
unsigned long events;
int ret;
@@ -1235,7 +1243,7 @@ static int vfio_ap_mdev_open(struct mdev_device *mdev)
matrix_mdev->group_notifier.notifier_call = vfio_ap_mdev_group_notifier;
events = VFIO_GROUP_NOTIFY_SET_KVM;
- ret = vfio_register_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY,
+ ret = vfio_register_notifier(vdev->dev, VFIO_GROUP_NOTIFY,
&events, &matrix_mdev->group_notifier);
if (ret) {
module_put(THIS_MODULE);
@@ -1244,29 +1252,30 @@ static int vfio_ap_mdev_open(struct mdev_device *mdev)
matrix_mdev->iommu_notifier.notifier_call = vfio_ap_mdev_iommu_notifier;
events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
- ret = vfio_register_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+ ret = vfio_register_notifier(vdev->dev, VFIO_IOMMU_NOTIFY,
&events, &matrix_mdev->iommu_notifier);
if (!ret)
return ret;
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY,
+ vfio_unregister_notifier(vdev->dev, VFIO_GROUP_NOTIFY,
&matrix_mdev->group_notifier);
module_put(THIS_MODULE);
return ret;
}
-static void vfio_ap_mdev_release(struct mdev_device *mdev)
+static void vfio_ap_mdev_release(struct vfio_device *vdev)
{
- struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
+ struct ap_matrix_mdev *matrix_mdev =
+ container_of(vdev, struct ap_matrix_mdev, vdev);
mutex_lock(&matrix_dev->lock);
if (matrix_mdev->kvm)
vfio_ap_mdev_unset_kvm(matrix_mdev);
mutex_unlock(&matrix_dev->lock);
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+ vfio_unregister_notifier(vdev->dev, VFIO_IOMMU_NOTIFY,
&matrix_mdev->iommu_notifier);
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY,
+ vfio_unregister_notifier(vdev->dev, VFIO_GROUP_NOTIFY,
&matrix_mdev->group_notifier);
module_put(THIS_MODULE);
}
@@ -1291,9 +1300,11 @@ static int vfio_ap_mdev_get_device_info(unsigned long arg)
return copy_to_user((void __user *)arg, &info, minsz) ? -EFAULT : 0;
}
-static ssize_t vfio_ap_mdev_ioctl(struct mdev_device *mdev,
+static ssize_t vfio_ap_mdev_ioctl(struct vfio_device *vdev,
unsigned int cmd, unsigned long arg)
{
+ struct ap_matrix_mdev *matrix_mdev =
+ container_of(vdev, struct ap_matrix_mdev, vdev);
int ret;
mutex_lock(&matrix_dev->lock);
@@ -1302,7 +1313,7 @@ static ssize_t vfio_ap_mdev_ioctl(struct mdev_device *mdev,
ret = vfio_ap_mdev_get_device_info(arg);
break;
case VFIO_DEVICE_RESET:
- ret = vfio_ap_mdev_reset_queues(mdev);
+ ret = vfio_ap_mdev_reset_queues(matrix_mdev);
break;
default:
ret = -EOPNOTSUPP;
@@ -1313,25 +1324,51 @@ static ssize_t vfio_ap_mdev_ioctl(struct mdev_device *mdev,
return ret;
}
+static const struct vfio_device_ops vfio_ap_matrix_dev_ops = {
+ .open = vfio_ap_mdev_open,
+ .release = vfio_ap_mdev_release,
+ .ioctl = vfio_ap_mdev_ioctl,
+};
+
+static struct mdev_driver vfio_ap_matrix_driver = {
+ .driver = {
+ .name = "vfio_ap_mdev",
+ .owner = THIS_MODULE,
+ .mod_name = KBUILD_MODNAME,
+ .dev_groups = vfio_ap_mdev_attr_groups,
+ },
+ .probe = vfio_ap_mdev_probe,
+ .remove = vfio_ap_mdev_remove,
+};
+
static const struct mdev_parent_ops vfio_ap_matrix_ops = {
.owner = THIS_MODULE,
+ .device_driver = &vfio_ap_matrix_driver,
.supported_type_groups = vfio_ap_mdev_type_groups,
- .mdev_attr_groups = vfio_ap_mdev_attr_groups,
- .create = vfio_ap_mdev_create,
- .remove = vfio_ap_mdev_remove,
- .open = vfio_ap_mdev_open,
- .release = vfio_ap_mdev_release,
- .ioctl = vfio_ap_mdev_ioctl,
};
int vfio_ap_mdev_register(void)
{
+ int ret;
+
atomic_set(&matrix_dev->available_instances, MAX_ZDEV_ENTRIES_EXT);
- return mdev_register_device(&matrix_dev->device, &vfio_ap_matrix_ops);
+ ret = mdev_register_driver(&vfio_ap_matrix_driver);
+ if (ret)
+ return ret;
+
+ ret = mdev_register_device(&matrix_dev->device, &vfio_ap_matrix_ops);
+ if (ret)
+ goto err_driver;
+ return 0;
+
+err_driver:
+ mdev_unregister_driver(&vfio_ap_matrix_driver);
+ return ret;
}
void vfio_ap_mdev_unregister(void)
{
mdev_unregister_device(&matrix_dev->device);
+ mdev_unregister_driver(&vfio_ap_matrix_driver);
}
diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h
index 28e9d998976820..b95ba674f60b1b 100644
--- a/drivers/s390/crypto/vfio_ap_private.h
+++ b/drivers/s390/crypto/vfio_ap_private.h
@@ -18,6 +18,7 @@
#include <linux/delay.h>
#include <linux/mutex.h>
#include <linux/kvm_host.h>
+#include <linux/vfio.h>
#include "ap_bus.h"
@@ -79,6 +80,7 @@ struct ap_matrix {
* @kvm: the struct holding guest's state
*/
struct ap_matrix_mdev {
+ struct vfio_device vdev;
struct list_head node;
struct ap_matrix matrix;
struct notifier_block group_notifier;
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 07/12] vfio/ccw: Convert to use vfio_register_group_dev()
2021-04-23 23:02 ` Jason Gunthorpe
` (7 preceding siblings ...)
(?)
@ 2021-04-23 23:03 ` Jason Gunthorpe
-1 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: Christian Borntraeger, Cornelia Huck, Eric Farman, Vasily Gorbik,
Heiko Carstens, kvm, linux-s390, Peter Oberparleiter,
Halil Pasic, Vineeth Vijayan
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
This is more complicated because vfio_ccw is sharing the vfio_device
between both the mdev_device and its vfio_device and the css_driver.
The mdev is a singleton, and the reason for this sharing appears to be to
allow the extra css_driver function callbacks to be delivered to the
vfio_device.
This keeps things as they were, with the css_driver allocating the
singleton, not the mdev_driver, this is pretty confusing. I'm also
uncertain how the lifetime model for the mdev works in the css_driver
callbacks.
At this point embed the vfio_device in the vfio_ccw_private and
instantiate it as a vfio_device when the mdev probes. The drvdata of both
the css_device and the mdev_device point at the private, and container_of
is used to get it back from the vfio_device.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/s390/cio/vfio_ccw_drv.c | 21 +++--
drivers/s390/cio/vfio_ccw_ops.c | 135 +++++++++++++++-------------
drivers/s390/cio/vfio_ccw_private.h | 5 ++
3 files changed, 94 insertions(+), 67 deletions(-)
diff --git a/drivers/s390/cio/vfio_ccw_drv.c b/drivers/s390/cio/vfio_ccw_drv.c
index 8c625b530035f5..55c4876dfd139d 100644
--- a/drivers/s390/cio/vfio_ccw_drv.c
+++ b/drivers/s390/cio/vfio_ccw_drv.c
@@ -442,7 +442,7 @@ static int __init vfio_ccw_sch_init(void)
vfio_ccw_work_q = create_singlethread_workqueue("vfio-ccw");
if (!vfio_ccw_work_q) {
ret = -ENOMEM;
- goto out_err;
+ goto out_regions;
}
vfio_ccw_io_region = kmem_cache_create_usercopy("vfio_ccw_io_region",
@@ -451,7 +451,7 @@ static int __init vfio_ccw_sch_init(void)
sizeof(struct ccw_io_region), NULL);
if (!vfio_ccw_io_region) {
ret = -ENOMEM;
- goto out_err;
+ goto out_regions;
}
vfio_ccw_cmd_region = kmem_cache_create_usercopy("vfio_ccw_cmd_region",
@@ -460,7 +460,7 @@ static int __init vfio_ccw_sch_init(void)
sizeof(struct ccw_cmd_region), NULL);
if (!vfio_ccw_cmd_region) {
ret = -ENOMEM;
- goto out_err;
+ goto out_regions;
}
vfio_ccw_schib_region = kmem_cache_create_usercopy("vfio_ccw_schib_region",
@@ -470,7 +470,7 @@ static int __init vfio_ccw_sch_init(void)
if (!vfio_ccw_schib_region) {
ret = -ENOMEM;
- goto out_err;
+ goto out_regions;
}
vfio_ccw_crw_region = kmem_cache_create_usercopy("vfio_ccw_crw_region",
@@ -480,19 +480,25 @@ static int __init vfio_ccw_sch_init(void)
if (!vfio_ccw_crw_region) {
ret = -ENOMEM;
- goto out_err;
+ goto out_regions;
}
+ ret = mdev_register_driver(&vfio_ccw_mdev_driver);
+ if (ret)
+ goto out_regions;
+
isc_register(VFIO_CCW_ISC);
ret = css_driver_register(&vfio_ccw_sch_driver);
if (ret) {
isc_unregister(VFIO_CCW_ISC);
- goto out_err;
+ goto out_driver;
}
return ret;
-out_err:
+out_driver:
+ mdev_unregister_driver(&vfio_ccw_mdev_driver);
+out_regions:
vfio_ccw_destroy_regions();
destroy_workqueue(vfio_ccw_work_q);
vfio_ccw_debug_exit();
@@ -501,6 +507,7 @@ static int __init vfio_ccw_sch_init(void)
static void __exit vfio_ccw_sch_exit(void)
{
+ mdev_unregister_driver(&vfio_ccw_mdev_driver);
css_driver_unregister(&vfio_ccw_sch_driver);
isc_unregister(VFIO_CCW_ISC);
vfio_ccw_destroy_regions();
diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c
index 491a64c61fff1a..0fcf46031d3821 100644
--- a/drivers/s390/cio/vfio_ccw_ops.c
+++ b/drivers/s390/cio/vfio_ccw_ops.c
@@ -17,13 +17,13 @@
#include "vfio_ccw_private.h"
-static int vfio_ccw_mdev_reset(struct mdev_device *mdev)
+static const struct vfio_device_ops vfio_ccw_dev_ops;
+
+static int vfio_ccw_mdev_reset(struct vfio_ccw_private *private)
{
- struct vfio_ccw_private *private;
struct subchannel *sch;
int ret;
- private = dev_get_drvdata(mdev_parent_dev(mdev));
sch = private->sch;
/*
* TODO:
@@ -61,7 +61,7 @@ static int vfio_ccw_mdev_notifier(struct notifier_block *nb,
if (!cp_iova_pinned(&private->cp, unmap->iova))
return NOTIFY_OK;
- if (vfio_ccw_mdev_reset(private->mdev))
+ if (vfio_ccw_mdev_reset(private))
return NOTIFY_BAD;
cp_free(&private->cp);
@@ -113,10 +113,11 @@ static struct attribute_group *mdev_type_groups[] = {
NULL,
};
-static int vfio_ccw_mdev_create(struct mdev_device *mdev)
+static int vfio_ccw_mdev_probe(struct mdev_device *mdev)
{
struct vfio_ccw_private *private =
dev_get_drvdata(mdev_parent_dev(mdev));
+ int ret;
if (private->state == VFIO_CCW_STATE_NOT_OPER)
return -ENODEV;
@@ -124,6 +125,10 @@ static int vfio_ccw_mdev_create(struct mdev_device *mdev)
if (atomic_dec_if_positive(&private->avail) < 0)
return -EPERM;
+ memset(&private->vdev, 0, sizeof(private->vdev));
+ vfio_init_group_dev(&private->vdev, &mdev->dev,
+ &vfio_ccw_dev_ops);
+
private->mdev = mdev;
private->state = VFIO_CCW_STATE_IDLE;
@@ -132,19 +137,28 @@ static int vfio_ccw_mdev_create(struct mdev_device *mdev)
private->sch->schid.ssid,
private->sch->schid.sch_no);
+ ret = vfio_register_group_dev(&private->vdev);
+ if (ret)
+ goto err_atomic;
+ dev_set_drvdata(&mdev->dev, private);
return 0;
+
+err_atomic:
+ atomic_inc(&private->avail);
+ return ret;
}
-static int vfio_ccw_mdev_remove(struct mdev_device *mdev)
+static void vfio_ccw_mdev_remove(struct mdev_device *mdev)
{
- struct vfio_ccw_private *private =
- dev_get_drvdata(mdev_parent_dev(mdev));
+ struct vfio_ccw_private *private = dev_get_drvdata(&mdev->dev);
VFIO_CCW_MSG_EVENT(2, "mdev %pUl, sch %x.%x.%04x: remove\n",
mdev_uuid(mdev), private->sch->schid.cssid,
private->sch->schid.ssid,
private->sch->schid.sch_no);
+ vfio_unregister_group_dev(&private->vdev);
+
if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
(private->state != VFIO_CCW_STATE_STANDBY)) {
if (!vfio_ccw_sch_quiesce(private->sch))
@@ -155,20 +169,18 @@ static int vfio_ccw_mdev_remove(struct mdev_device *mdev)
cp_free(&private->cp);
private->mdev = NULL;
atomic_inc(&private->avail);
-
- return 0;
}
-static int vfio_ccw_mdev_open(struct mdev_device *mdev)
+static int vfio_ccw_mdev_open(struct vfio_device *vdev)
{
struct vfio_ccw_private *private =
- dev_get_drvdata(mdev_parent_dev(mdev));
+ container_of(vdev, struct vfio_ccw_private, vdev);
unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
int ret;
private->nb.notifier_call = vfio_ccw_mdev_notifier;
- ret = vfio_register_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+ ret = vfio_register_notifier(vdev->dev, VFIO_IOMMU_NOTIFY,
&events, &private->nb);
if (ret)
return ret;
@@ -189,27 +201,26 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
out_unregister:
vfio_ccw_unregister_dev_regions(private);
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+ vfio_unregister_notifier(vdev->dev, VFIO_IOMMU_NOTIFY,
&private->nb);
return ret;
}
-static void vfio_ccw_mdev_release(struct mdev_device *mdev)
+static void vfio_ccw_mdev_release(struct vfio_device *vdev)
{
struct vfio_ccw_private *private =
- dev_get_drvdata(mdev_parent_dev(mdev));
+ container_of(vdev, struct vfio_ccw_private, vdev);
if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
(private->state != VFIO_CCW_STATE_STANDBY)) {
- if (!vfio_ccw_mdev_reset(mdev))
+ if (!vfio_ccw_mdev_reset(private))
private->state = VFIO_CCW_STATE_STANDBY;
/* The state will be NOT_OPER on error. */
}
cp_free(&private->cp);
vfio_ccw_unregister_dev_regions(private);
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
- &private->nb);
+ vfio_unregister_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, &private->nb);
}
static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private *private,
@@ -233,15 +244,14 @@ static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private *private,
return ret;
}
-static ssize_t vfio_ccw_mdev_read(struct mdev_device *mdev,
+static ssize_t vfio_ccw_mdev_read(struct vfio_device *vdev,
char __user *buf,
size_t count,
loff_t *ppos)
{
+ struct vfio_ccw_private *private =
+ container_of(vdev, struct vfio_ccw_private, vdev);
unsigned int index = VFIO_CCW_OFFSET_TO_INDEX(*ppos);
- struct vfio_ccw_private *private;
-
- private = dev_get_drvdata(mdev_parent_dev(mdev));
if (index >= VFIO_CCW_NUM_REGIONS + private->num_regions)
return -EINVAL;
@@ -288,15 +298,14 @@ static ssize_t vfio_ccw_mdev_write_io_region(struct vfio_ccw_private *private,
return ret;
}
-static ssize_t vfio_ccw_mdev_write(struct mdev_device *mdev,
+static ssize_t vfio_ccw_mdev_write(struct vfio_device *vdev,
const char __user *buf,
size_t count,
loff_t *ppos)
{
+ struct vfio_ccw_private *private =
+ container_of(vdev, struct vfio_ccw_private, vdev);
unsigned int index = VFIO_CCW_OFFSET_TO_INDEX(*ppos);
- struct vfio_ccw_private *private;
-
- private = dev_get_drvdata(mdev_parent_dev(mdev));
if (index >= VFIO_CCW_NUM_REGIONS + private->num_regions)
return -EINVAL;
@@ -313,12 +322,9 @@ static ssize_t vfio_ccw_mdev_write(struct mdev_device *mdev,
return -EINVAL;
}
-static int vfio_ccw_mdev_get_device_info(struct vfio_device_info *info,
- struct mdev_device *mdev)
+static int vfio_ccw_mdev_get_device_info(struct vfio_ccw_private *private,
+ struct vfio_device_info *info)
{
- struct vfio_ccw_private *private;
-
- private = dev_get_drvdata(mdev_parent_dev(mdev));
info->flags = VFIO_DEVICE_FLAGS_CCW | VFIO_DEVICE_FLAGS_RESET;
info->num_regions = VFIO_CCW_NUM_REGIONS + private->num_regions;
info->num_irqs = VFIO_CCW_NUM_IRQS;
@@ -326,14 +332,12 @@ static int vfio_ccw_mdev_get_device_info(struct vfio_device_info *info,
return 0;
}
-static int vfio_ccw_mdev_get_region_info(struct vfio_region_info *info,
- struct mdev_device *mdev,
+static int vfio_ccw_mdev_get_region_info(struct vfio_ccw_private *private,
+ struct vfio_region_info *info,
unsigned long arg)
{
- struct vfio_ccw_private *private;
int i;
- private = dev_get_drvdata(mdev_parent_dev(mdev));
switch (info->index) {
case VFIO_CCW_CONFIG_REGION_INDEX:
info->offset = 0;
@@ -408,19 +412,16 @@ static int vfio_ccw_mdev_get_irq_info(struct vfio_irq_info *info)
return 0;
}
-static int vfio_ccw_mdev_set_irqs(struct mdev_device *mdev,
+static int vfio_ccw_mdev_set_irqs(struct vfio_ccw_private *private,
uint32_t flags,
uint32_t index,
void __user *data)
{
- struct vfio_ccw_private *private;
struct eventfd_ctx **ctx;
if (!(flags & VFIO_IRQ_SET_ACTION_TRIGGER))
return -EINVAL;
- private = dev_get_drvdata(mdev_parent_dev(mdev));
-
switch (index) {
case VFIO_CCW_IO_IRQ_INDEX:
ctx = &private->io_trigger;
@@ -522,10 +523,12 @@ void vfio_ccw_unregister_dev_regions(struct vfio_ccw_private *private)
private->region = NULL;
}
-static ssize_t vfio_ccw_mdev_ioctl(struct mdev_device *mdev,
+static ssize_t vfio_ccw_mdev_ioctl(struct vfio_device *vdev,
unsigned int cmd,
unsigned long arg)
{
+ struct vfio_ccw_private *private =
+ container_of(vdev, struct vfio_ccw_private, vdev);
int ret = 0;
unsigned long minsz;
@@ -542,7 +545,7 @@ static ssize_t vfio_ccw_mdev_ioctl(struct mdev_device *mdev,
if (info.argsz < minsz)
return -EINVAL;
- ret = vfio_ccw_mdev_get_device_info(&info, mdev);
+ ret = vfio_ccw_mdev_get_device_info(private, &info);
if (ret)
return ret;
@@ -560,7 +563,7 @@ static ssize_t vfio_ccw_mdev_ioctl(struct mdev_device *mdev,
if (info.argsz < minsz)
return -EINVAL;
- ret = vfio_ccw_mdev_get_region_info(&info, mdev, arg);
+ ret = vfio_ccw_mdev_get_region_info(private, &info, arg);
if (ret)
return ret;
@@ -605,47 +608,59 @@ static ssize_t vfio_ccw_mdev_ioctl(struct mdev_device *mdev,
return ret;
data = (void __user *)(arg + minsz);
- return vfio_ccw_mdev_set_irqs(mdev, hdr.flags, hdr.index, data);
+ return vfio_ccw_mdev_set_irqs(private, hdr.flags, hdr.index,
+ data);
}
case VFIO_DEVICE_RESET:
- return vfio_ccw_mdev_reset(mdev);
+ return vfio_ccw_mdev_reset(private);
default:
return -ENOTTY;
}
}
/* Request removal of the device*/
-static void vfio_ccw_mdev_request(struct mdev_device *mdev, unsigned int count)
+static void vfio_ccw_mdev_request(struct vfio_device *vdev, unsigned int count)
{
- struct vfio_ccw_private *private = dev_get_drvdata(mdev_parent_dev(mdev));
-
- if (!private)
- return;
+ struct vfio_ccw_private *private =
+ container_of(vdev, struct vfio_ccw_private, vdev);
+ struct device *dev = private->vdev.dev;
if (private->req_trigger) {
if (!(count % 10))
- dev_notice_ratelimited(mdev_dev(private->mdev),
+ dev_notice_ratelimited(dev,
"Relaying device request to user (#%u)\n",
count);
eventfd_signal(private->req_trigger, 1);
} else if (count == 0) {
- dev_notice(mdev_dev(private->mdev),
+ dev_notice(dev,
"No device request channel registered, blocked until released by user\n");
}
}
+static const struct vfio_device_ops vfio_ccw_dev_ops = {
+ .open = vfio_ccw_mdev_open,
+ .release = vfio_ccw_mdev_release,
+ .read = vfio_ccw_mdev_read,
+ .write = vfio_ccw_mdev_write,
+ .ioctl = vfio_ccw_mdev_ioctl,
+ .request = vfio_ccw_mdev_request,
+};
+
+struct mdev_driver vfio_ccw_mdev_driver = {
+ .driver = {
+ .name = "vfio_ccw_mdev",
+ .owner = THIS_MODULE,
+ .mod_name = KBUILD_MODNAME,
+ },
+ .probe = vfio_ccw_mdev_probe,
+ .remove = vfio_ccw_mdev_remove,
+};
+
static const struct mdev_parent_ops vfio_ccw_mdev_ops = {
.owner = THIS_MODULE,
+ .device_driver = &vfio_ccw_mdev_driver,
.supported_type_groups = mdev_type_groups,
- .create = vfio_ccw_mdev_create,
- .remove = vfio_ccw_mdev_remove,
- .open = vfio_ccw_mdev_open,
- .release = vfio_ccw_mdev_release,
- .read = vfio_ccw_mdev_read,
- .write = vfio_ccw_mdev_write,
- .ioctl = vfio_ccw_mdev_ioctl,
- .request = vfio_ccw_mdev_request,
};
int vfio_ccw_mdev_reg(struct subchannel *sch)
diff --git a/drivers/s390/cio/vfio_ccw_private.h b/drivers/s390/cio/vfio_ccw_private.h
index b2c762eb42b9bb..7272eb78861244 100644
--- a/drivers/s390/cio/vfio_ccw_private.h
+++ b/drivers/s390/cio/vfio_ccw_private.h
@@ -17,6 +17,7 @@
#include <linux/eventfd.h>
#include <linux/workqueue.h>
#include <linux/vfio_ccw.h>
+#include <linux/vfio.h>
#include <asm/crw.h>
#include <asm/debug.h>
@@ -67,6 +68,7 @@ struct vfio_ccw_crw {
/**
* struct vfio_ccw_private
+ * @vdev: Embedded VFIO device
* @sch: pointer to the subchannel
* @state: internal state of the device
* @completion: synchronization helper of the I/O completion
@@ -90,6 +92,7 @@ struct vfio_ccw_crw {
* @crw_work: work for deferral process of CRW handling
*/
struct vfio_ccw_private {
+ struct vfio_device vdev;
struct subchannel *sch;
int state;
struct completion *completion;
@@ -121,6 +124,8 @@ extern void vfio_ccw_mdev_unreg(struct subchannel *sch);
extern int vfio_ccw_sch_quiesce(struct subchannel *sch);
+extern struct mdev_driver vfio_ccw_mdev_driver;
+
/*
* States of the device statemachine.
*/
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 08/12] vfio/gvt: Convert to use vfio_register_group_dev()
2021-04-23 23:02 ` Jason Gunthorpe
(?)
@ 2021-04-23 23:03 ` Jason Gunthorpe
-1 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: David Airlie, Alex Williamson, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, intel-gfx, intel-gvt-dev, Jani Nikula,
Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc, Rodrigo Vivi,
Zhenyu Wang, Zhi Wang
Cc: Raj, Ashok, Dan Williams, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
While there is a confusing mess of pointers and structs in this driver,
the struct kvmgt_vdev (which in turn is 1:1 with a struct intel_vgpu) is
what holds the vfio_device. Replace all the drvdata's and weird
derivations of vgpu and vdev with container_of() or vdev->vgpu.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
.../driver-api/vfio-mediated-device.rst | 19 --
drivers/gpu/drm/i915/gvt/kvmgt.c | 208 ++++++++++--------
drivers/vfio/mdev/Makefile | 2 +-
drivers/vfio/mdev/mdev_core.c | 47 +---
drivers/vfio/mdev/mdev_driver.c | 11 +-
drivers/vfio/mdev/mdev_private.h | 2 -
drivers/vfio/mdev/vfio_mdev.c | 158 -------------
drivers/vfio/vfio.c | 6 +-
include/linux/mdev.h | 52 -----
include/linux/vfio.h | 4 +
10 files changed, 126 insertions(+), 383 deletions(-)
delete mode 100644 drivers/vfio/mdev/vfio_mdev.c
diff --git a/Documentation/driver-api/vfio-mediated-device.rst b/Documentation/driver-api/vfio-mediated-device.rst
index 1779b85f014e2f..5f866b17c93e69 100644
--- a/Documentation/driver-api/vfio-mediated-device.rst
+++ b/Documentation/driver-api/vfio-mediated-device.rst
@@ -137,25 +137,6 @@ The structures in the mdev_parent_ops structure are as follows:
* mdev_attr_groups: attributes of the mediated device
* supported_config: attributes to define supported configurations
-The functions in the mdev_parent_ops structure are as follows:
-
-* create: allocate basic resources in a driver for a mediated device
-* remove: free resources in a driver when a mediated device is destroyed
-
-(Note that mdev-core provides no implicit serialization of create/remove
-callbacks per mdev parent device, per mdev type, or any other categorization.
-Vendor drivers are expected to be fully asynchronous in this respect or
-provide their own internal resource protection.)
-
-The callbacks in the mdev_parent_ops structure are as follows:
-
-* open: open callback of mediated device
-* close: close callback of mediated device
-* ioctl: ioctl callback of mediated device
-* read : read emulation callback
-* write: write emulation callback
-* mmap: mmap emulation callback
-
A driver should use the mdev_parent_ops structure in the function call to
register itself with the mdev core driver::
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 6bf176e8426e63..85ef300087e091 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -50,6 +50,7 @@
#include "gvt.h"
static const struct intel_gvt_ops *intel_gvt_ops;
+static const struct vfio_device_ops intel_vgpu_dev_ops;
/* helper macros copied from vfio-pci */
#define VFIO_PCI_OFFSET_SHIFT 40
@@ -109,8 +110,8 @@ struct gvt_dma {
};
struct kvmgt_vdev {
+ struct vfio_device vfio_device;
struct intel_vgpu *vgpu;
- struct mdev_device *mdev;
struct vfio_region *region;
int num_regions;
struct eventfd_ctx *intx_trigger;
@@ -130,7 +131,6 @@ struct kvmgt_vdev {
struct kvm *kvm;
struct work_struct release_work;
atomic_t released;
- struct vfio_device *vfio_device;
struct vfio_group *vfio_group;
};
@@ -144,7 +144,7 @@ static inline bool handle_valid(unsigned long handle)
return !!(handle & ~0xff);
}
-static int kvmgt_guest_init(struct mdev_device *mdev);
+static int kvmgt_guest_init(struct kvmgt_vdev *vdev);
static void intel_vgpu_release_work(struct work_struct *work);
static bool kvmgt_guest_exit(struct kvmgt_guest_info *info);
@@ -611,12 +611,7 @@ static int kvmgt_get_vfio_device(void *p_vgpu)
struct intel_vgpu *vgpu = (struct intel_vgpu *)p_vgpu;
struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
- vdev->vfio_device = vfio_device_get_from_dev(
- mdev_dev(vdev->mdev));
- if (!vdev->vfio_device) {
- gvt_vgpu_err("failed to get vfio device\n");
- return -ENODEV;
- }
+ vfio_device_get(&vdev->vfio_device);
return 0;
}
@@ -683,16 +678,14 @@ static void kvmgt_put_vfio_device(void *vgpu)
{
struct kvmgt_vdev *vdev = kvmgt_vdev((struct intel_vgpu *)vgpu);
- if (WARN_ON(!vdev->vfio_device))
- return;
-
- vfio_device_put(vdev->vfio_device);
+ vfio_device_put(&vdev->vfio_device);
}
-static int intel_vgpu_create(struct mdev_device *mdev)
+static int intel_vgpu_probe(struct mdev_device *mdev)
{
struct intel_vgpu *vgpu = NULL;
struct intel_vgpu_type *type;
+ struct kvmgt_vdev *vdev;
struct device *pdev;
void *gvt;
int ret;
@@ -702,40 +695,40 @@ static int intel_vgpu_create(struct mdev_device *mdev)
type = intel_gvt_ops->gvt_find_vgpu_type(gvt,
mdev_get_type_group_id(mdev));
- if (!type) {
- ret = -EINVAL;
- goto out;
- }
+ if (!type)
+ return -EINVAL;
vgpu = intel_gvt_ops->vgpu_create(gvt, type);
if (IS_ERR_OR_NULL(vgpu)) {
- ret = vgpu == NULL ? -EFAULT : PTR_ERR(vgpu);
gvt_err("failed to create intel vgpu: %d\n", ret);
- goto out;
+ return vgpu == NULL ? -EFAULT : PTR_ERR(vgpu);
}
- INIT_WORK(&kvmgt_vdev(vgpu)->release_work, intel_vgpu_release_work);
+ vdev = kvmgt_vdev(vgpu);
+ INIT_WORK(&vdev->release_work, intel_vgpu_release_work);
+ vfio_init_group_dev(&vdev->vfio_device, &mdev->dev,
+ &intel_vgpu_dev_ops);
- kvmgt_vdev(vgpu)->mdev = mdev;
- mdev_set_drvdata(mdev, vgpu);
+ ret = vfio_register_group_dev(&vdev->vfio_device);
+ if (ret) {
+ intel_gvt_ops->vgpu_destroy(vgpu);
+ return ret;
+ }
+ dev_set_drvdata(&mdev->dev, vdev);
gvt_dbg_core("intel_vgpu_create succeeded for mdev: %s\n",
dev_name(mdev_dev(mdev)));
- ret = 0;
-
-out:
- return ret;
+ return 0;
}
-static int intel_vgpu_remove(struct mdev_device *mdev)
+static void intel_vgpu_remove(struct mdev_device *mdev)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
-
- if (handle_valid(vgpu->handle))
- return -EBUSY;
+ struct kvmgt_vdev *vdev = dev_get_drvdata(&mdev->dev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
+ if (WARN_ON(handle_valid(vgpu->handle)))
+ return;
intel_gvt_ops->vgpu_destroy(vgpu);
- return 0;
}
static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
@@ -788,10 +781,11 @@ static int intel_vgpu_group_notifier(struct notifier_block *nb,
return NOTIFY_OK;
}
-static int intel_vgpu_open(struct mdev_device *mdev)
+static int intel_vgpu_open(struct vfio_device *vfio_dev)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
- struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned long events;
int ret;
struct vfio_group *vfio_group;
@@ -800,7 +794,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
vdev->group_notifier.notifier_call = intel_vgpu_group_notifier;
events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
- ret = vfio_register_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, &events,
+ ret = vfio_register_notifier(vfio_dev->dev, VFIO_IOMMU_NOTIFY, &events,
&vdev->iommu_notifier);
if (ret != 0) {
gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n",
@@ -809,7 +803,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
}
events = VFIO_GROUP_NOTIFY_SET_KVM;
- ret = vfio_register_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, &events,
+ ret = vfio_register_notifier(vfio_dev->dev, VFIO_GROUP_NOTIFY, &events,
&vdev->group_notifier);
if (ret != 0) {
gvt_vgpu_err("vfio_register_notifier for group failed: %d\n",
@@ -817,7 +811,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
goto undo_iommu;
}
- vfio_group = vfio_group_get_external_user_from_dev(mdev_dev(mdev));
+ vfio_group = vfio_group_get_external_user_from_dev(vfio_dev->dev);
if (IS_ERR_OR_NULL(vfio_group)) {
ret = !vfio_group ? -EFAULT : PTR_ERR(vfio_group);
gvt_vgpu_err("vfio_group_get_external_user_from_dev failed\n");
@@ -833,11 +827,11 @@ static int intel_vgpu_open(struct mdev_device *mdev)
goto undo_group;
}
- ret = kvmgt_guest_init(mdev);
+ ret = kvmgt_guest_init(vdev);
if (ret)
goto undo_group;
- intel_gvt_ops->vgpu_activate(vgpu);
+ intel_gvt_ops->vgpu_activate(vdev->vgpu);
atomic_set(&vdev->released, 0);
return ret;
@@ -847,11 +841,11 @@ static int intel_vgpu_open(struct mdev_device *mdev)
vdev->vfio_group = NULL;
undo_register:
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY,
+ vfio_unregister_notifier(vfio_dev->dev, VFIO_GROUP_NOTIFY,
&vdev->group_notifier);
undo_iommu:
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+ vfio_unregister_notifier(vfio_dev->dev, VFIO_IOMMU_NOTIFY,
&vdev->iommu_notifier);
out:
return ret;
@@ -884,12 +878,12 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
intel_gvt_ops->vgpu_release(vgpu);
- ret = vfio_unregister_notifier(mdev_dev(vdev->mdev), VFIO_IOMMU_NOTIFY,
+ ret = vfio_unregister_notifier(vdev->vfio_device.dev, VFIO_IOMMU_NOTIFY,
&vdev->iommu_notifier);
drm_WARN(&i915->drm, ret,
"vfio_unregister_notifier for iommu failed: %d\n", ret);
- ret = vfio_unregister_notifier(mdev_dev(vdev->mdev), VFIO_GROUP_NOTIFY,
+ ret = vfio_unregister_notifier(vdev->vfio_device.dev, VFIO_GROUP_NOTIFY,
&vdev->group_notifier);
drm_WARN(&i915->drm, ret,
"vfio_unregister_notifier for group failed: %d\n", ret);
@@ -907,11 +901,12 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
vgpu->handle = 0;
}
-static void intel_vgpu_release(struct mdev_device *mdev)
+static void intel_vgpu_release(struct vfio_device *vfio_dev)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
- __intel_vgpu_release(vgpu);
+ __intel_vgpu_release(vdev->vgpu);
}
static void intel_vgpu_release_work(struct work_struct *work)
@@ -997,11 +992,10 @@ static int intel_vgpu_aperture_rw(struct intel_vgpu *vgpu, u64 off,
return 0;
}
-static ssize_t intel_vgpu_rw(struct mdev_device *mdev, char *buf,
+static ssize_t intel_vgpu_rw(struct kvmgt_vdev *vdev, char *buf,
size_t count, loff_t *ppos, bool is_write)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
- struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
u64 pos = *ppos & VFIO_PCI_OFFSET_MASK;
int ret = -EINVAL;
@@ -1047,9 +1041,9 @@ static ssize_t intel_vgpu_rw(struct mdev_device *mdev, char *buf,
return ret == 0 ? count : ret;
}
-static bool gtt_entry(struct mdev_device *mdev, loff_t *ppos)
+static bool gtt_entry(struct kvmgt_vdev *vdev, loff_t *ppos)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
struct intel_gvt *gvt = vgpu->gvt;
int offset;
@@ -1066,9 +1060,11 @@ static bool gtt_entry(struct mdev_device *mdev, loff_t *ppos)
true : false;
}
-static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
+static ssize_t intel_vgpu_read(struct vfio_device *vfio_dev, char __user *buf,
size_t count, loff_t *ppos)
{
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
unsigned int done = 0;
int ret;
@@ -1077,10 +1073,10 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
/* Only support GGTT entry 8 bytes read */
if (count >= 8 && !(*ppos % 8) &&
- gtt_entry(mdev, ppos)) {
+ gtt_entry(vdev, ppos)) {
u64 val;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, false);
if (ret <= 0)
goto read_err;
@@ -1092,7 +1088,7 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
} else if (count >= 4 && !(*ppos % 4)) {
u32 val;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, false);
if (ret <= 0)
goto read_err;
@@ -1104,7 +1100,7 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
} else if (count >= 2 && !(*ppos % 2)) {
u16 val;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, false);
if (ret <= 0)
goto read_err;
@@ -1116,7 +1112,7 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
} else {
u8 val;
- ret = intel_vgpu_rw(mdev, &val, sizeof(val), ppos,
+ ret = intel_vgpu_rw(vdev, &val, sizeof(val), ppos,
false);
if (ret <= 0)
goto read_err;
@@ -1139,10 +1135,12 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
return -EFAULT;
}
-static ssize_t intel_vgpu_write(struct mdev_device *mdev,
+static ssize_t intel_vgpu_write(struct vfio_device *vfio_dev,
const char __user *buf,
size_t count, loff_t *ppos)
{
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
unsigned int done = 0;
int ret;
@@ -1151,13 +1149,13 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
/* Only support GGTT entry 8 bytes write */
if (count >= 8 && !(*ppos % 8) &&
- gtt_entry(mdev, ppos)) {
+ gtt_entry(vdev, ppos)) {
u64 val;
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, true);
if (ret <= 0)
goto write_err;
@@ -1169,7 +1167,7 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, true);
if (ret <= 0)
goto write_err;
@@ -1181,7 +1179,7 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, (char *)&val,
+ ret = intel_vgpu_rw(vdev, (char *)&val,
sizeof(val), ppos, true);
if (ret <= 0)
goto write_err;
@@ -1193,7 +1191,7 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, &val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, &val, sizeof(val),
ppos, true);
if (ret <= 0)
goto write_err;
@@ -1212,13 +1210,16 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
return -EFAULT;
}
-static int intel_vgpu_mmap(struct mdev_device *mdev, struct vm_area_struct *vma)
+static int intel_vgpu_mmap(struct vfio_device *vfio_dev,
+ struct vm_area_struct *vma)
{
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
unsigned int index;
u64 virtaddr;
unsigned long req_size, pgoff, req_start;
pgprot_t pg_prot;
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
index = vma->vm_pgoff >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
if (index >= VFIO_PCI_ROM_REGION_INDEX)
@@ -1341,11 +1342,12 @@ static int intel_vgpu_set_irqs(struct intel_vgpu *vgpu, u32 flags,
return func(vgpu, index, start, count, flags, data);
}
-static long intel_vgpu_ioctl(struct mdev_device *mdev, unsigned int cmd,
+static long intel_vgpu_ioctl(struct vfio_device *vfio_dev, unsigned int cmd,
unsigned long arg)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
- struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned long minsz;
gvt_dbg_core("vgpu%d ioctl, cmd: %d\n", vgpu->id, cmd);
@@ -1624,14 +1626,10 @@ static ssize_t
vgpu_id_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct mdev_device *mdev = mdev_from_dev(dev);
+ struct kvmgt_vdev *vdev = dev_get_drvdata(dev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
- if (mdev) {
- struct intel_vgpu *vgpu = (struct intel_vgpu *)
- mdev_get_drvdata(mdev);
- return sprintf(buf, "%d\n", vgpu->id);
- }
- return sprintf(buf, "\n");
+ return sprintf(buf, "%d\n", vgpu->id);
}
static DEVICE_ATTR_RO(vgpu_id);
@@ -1651,18 +1649,28 @@ static const struct attribute_group *intel_vgpu_groups[] = {
NULL,
};
-static struct mdev_parent_ops intel_vgpu_ops = {
- .mdev_attr_groups = intel_vgpu_groups,
- .create = intel_vgpu_create,
- .remove = intel_vgpu_remove,
+static const struct vfio_device_ops intel_vgpu_dev_ops = {
+ .open = intel_vgpu_open,
+ .release = intel_vgpu_release,
+ .read = intel_vgpu_read,
+ .write = intel_vgpu_write,
+ .mmap = intel_vgpu_mmap,
+ .ioctl = intel_vgpu_ioctl,
+};
- .open = intel_vgpu_open,
- .release = intel_vgpu_release,
+static struct mdev_driver intel_vgpu_mdev_driver = {
+ .driver = {
+ .name = "intel_vgpu_mdev",
+ .owner = THIS_MODULE,
+ .mod_name = KBUILD_MODNAME,
+ .dev_groups = intel_vgpu_groups,
+ },
+ .probe = intel_vgpu_probe,
+ .remove = intel_vgpu_remove,
+};
- .read = intel_vgpu_read,
- .write = intel_vgpu_write,
- .mmap = intel_vgpu_mmap,
- .ioctl = intel_vgpu_ioctl,
+static struct mdev_parent_ops intel_vgpu_ops = {
+ .device_driver = &intel_vgpu_mdev_driver,
};
static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
@@ -1806,18 +1814,12 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu, struct kvm *kvm)
return ret;
}
-static int kvmgt_guest_init(struct mdev_device *mdev)
+static int kvmgt_guest_init(struct kvmgt_vdev *vdev)
{
struct kvmgt_guest_info *info;
- struct intel_vgpu *vgpu;
- struct kvmgt_vdev *vdev;
+ struct intel_vgpu *vgpu = vdev->vgpu;
struct kvm *kvm;
- vgpu = mdev_get_drvdata(mdev);
- if (handle_valid(vgpu->handle))
- return -EEXIST;
-
- vdev = kvmgt_vdev(vgpu);
kvm = vdev->kvm;
if (!kvm || kvm->mm != current->mm) {
gvt_vgpu_err("KVM is required to use Intel vGPU\n");
@@ -2125,13 +2127,25 @@ static const struct intel_gvt_mpt kvmgt_mpt = {
static int __init kvmgt_init(void)
{
- if (intel_gvt_register_hypervisor(&kvmgt_mpt) < 0)
- return -ENODEV;
+ int ret;
+
+ ret = mdev_register_driver(&intel_vgpu_mdev_driver);
+ if (ret)
+ return ret;
+
+ if (intel_gvt_register_hypervisor(&kvmgt_mpt) < 0) {
+ ret = -ENODEV;
+ goto err_driver;
+ }
return 0;
+err_driver:
+ mdev_unregister_driver(&intel_vgpu_mdev_driver);
+ return ret;
}
static void __exit kvmgt_exit(void)
{
+ mdev_unregister_driver(&intel_vgpu_mdev_driver);
intel_gvt_unregister_hypervisor();
}
diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
index ff9ecd80212503..7c236ba1b90eb1 100644
--- a/drivers/vfio/mdev/Makefile
+++ b/drivers/vfio/mdev/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
-mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
+mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
obj-$(CONFIG_VFIO_MDEV) += mdev.o
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index 51b8a9fcf866ad..f95d01b57fb168 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -89,17 +89,10 @@ void mdev_release_parent(struct kref *kref)
static void mdev_device_remove_common(struct mdev_device *mdev)
{
struct mdev_parent *parent = mdev->type->parent;
- int ret;
mdev_remove_sysfs_files(mdev);
device_del(&mdev->dev);
lockdep_assert_held(&parent->unreg_sem);
- if (parent->ops->remove) {
- ret = parent->ops->remove(mdev);
- if (ret)
- dev_err(&mdev->dev, "Remove failed: err=%d\n", ret);
- }
-
/* Balances with device_initialize() */
put_device(&mdev->dev);
}
@@ -131,17 +124,13 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
/* check for mandatory ops */
if (!ops || !ops->supported_type_groups)
return -EINVAL;
- if (!ops->device_driver && (!ops->create || !ops->remove))
+ if (!ops->device_driver)
return -EINVAL;
dev = get_device(dev);
if (!dev)
return -EINVAL;
- /* Not mandatory, but its absence could be a problem */
- if (!ops->request)
- dev_info(dev, "Driver cannot be asked to release device\n");
-
mutex_lock(&parent_list_lock);
/* Check for duplicate */
@@ -263,15 +252,12 @@ static void mdev_device_release(struct device *dev)
*/
static int mdev_bind_driver(struct mdev_device *mdev)
{
- struct mdev_driver *drv = mdev->type->parent->ops->device_driver;
int ret;
- if (!drv)
- drv = &vfio_mdev_driver;
-
while (1) {
device_lock(&mdev->dev);
- if (mdev->dev.driver == &drv->driver) {
+ if (mdev->dev.driver ==
+ &mdev->type->parent->ops->device_driver->driver) {
ret = 0;
goto out_unlock;
}
@@ -337,15 +323,9 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
goto out_put_device;
}
- if (parent->ops->create) {
- ret = parent->ops->create(mdev);
- if (ret)
- goto out_unlock;
- }
-
ret = device_add(&mdev->dev);
if (ret)
- goto out_remove;
+ goto out_unlock;
ret = mdev_bind_driver(mdev);
if (ret)
@@ -363,9 +343,6 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
out_del:
device_del(&mdev->dev);
-out_remove:
- if (parent->ops->remove)
- parent->ops->remove(mdev);
out_unlock:
up_read(&parent->unreg_sem);
out_put_device:
@@ -408,27 +385,13 @@ int mdev_device_remove(struct mdev_device *mdev)
static int __init mdev_init(void)
{
- int rc;
-
- rc = mdev_bus_register();
- if (rc)
- return rc;
- rc = mdev_register_driver(&vfio_mdev_driver);
- if (rc)
- goto err_bus;
- return 0;
-err_bus:
- mdev_bus_unregister();
- return rc;
+ return mdev_bus_register();
}
static void __exit mdev_exit(void)
{
- mdev_unregister_driver(&vfio_mdev_driver);
-
if (mdev_bus_compat_class)
class_compat_unregister(mdev_bus_compat_class);
-
mdev_bus_unregister();
}
diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
index 6e96c023d7823d..0012a9ee7cb0a4 100644
--- a/drivers/vfio/mdev/mdev_driver.c
+++ b/drivers/vfio/mdev/mdev_driver.c
@@ -74,15 +74,8 @@ static int mdev_remove(struct device *dev)
static int mdev_match(struct device *dev, struct device_driver *drv)
{
struct mdev_device *mdev = to_mdev_device(dev);
- struct mdev_driver *target = mdev->type->parent->ops->device_driver;
-
- /*
- * The ops specify the device driver to connect, fall back to the old
- * shim driver if the driver hasn't been converted.
- */
- if (!target)
- target = &vfio_mdev_driver;
- return drv == &target->driver;
+
+ return drv == &mdev->type->parent->ops->device_driver->driver;
}
struct bus_type mdev_bus_type = {
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
index 5461b67582289f..a656cfe0346c33 100644
--- a/drivers/vfio/mdev/mdev_private.h
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -37,8 +37,6 @@ struct mdev_type {
#define to_mdev_type(_kobj) \
container_of(_kobj, struct mdev_type, kobj)
-extern struct mdev_driver vfio_mdev_driver;
-
int parent_create_sysfs_files(struct mdev_parent *parent);
void parent_remove_sysfs_files(struct mdev_parent *parent);
diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
deleted file mode 100644
index d5b4eede47c1a5..00000000000000
--- a/drivers/vfio/mdev/vfio_mdev.c
+++ /dev/null
@@ -1,158 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * VFIO based driver for Mediated device
- *
- * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
- * Author: Neo Jia <cjia@nvidia.com>
- * Kirti Wankhede <kwankhede@nvidia.com>
- */
-
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/device.h>
-#include <linux/kernel.h>
-#include <linux/slab.h>
-#include <linux/vfio.h>
-#include <linux/mdev.h>
-
-#include "mdev_private.h"
-
-static int vfio_mdev_open(struct vfio_device *core_vdev)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- int ret;
-
- if (unlikely(!parent->ops->open))
- return -EINVAL;
-
- if (!try_module_get(THIS_MODULE))
- return -ENODEV;
-
- ret = parent->ops->open(mdev);
- if (ret)
- module_put(THIS_MODULE);
-
- return ret;
-}
-
-static void vfio_mdev_release(struct vfio_device *core_vdev)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (likely(parent->ops->release))
- parent->ops->release(mdev);
-
- module_put(THIS_MODULE);
-}
-
-static long vfio_mdev_unlocked_ioctl(struct vfio_device *core_vdev,
- unsigned int cmd, unsigned long arg)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->ioctl))
- return -EINVAL;
-
- return parent->ops->ioctl(mdev, cmd, arg);
-}
-
-static ssize_t vfio_mdev_read(struct vfio_device *core_vdev, char __user *buf,
- size_t count, loff_t *ppos)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->read))
- return -EINVAL;
-
- return parent->ops->read(mdev, buf, count, ppos);
-}
-
-static ssize_t vfio_mdev_write(struct vfio_device *core_vdev,
- const char __user *buf, size_t count,
- loff_t *ppos)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->write))
- return -EINVAL;
-
- return parent->ops->write(mdev, buf, count, ppos);
-}
-
-static int vfio_mdev_mmap(struct vfio_device *core_vdev,
- struct vm_area_struct *vma)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->mmap))
- return -EINVAL;
-
- return parent->ops->mmap(mdev, vma);
-}
-
-static void vfio_mdev_request(struct vfio_device *core_vdev, unsigned int count)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (parent->ops->request)
- parent->ops->request(mdev, count);
- else if (count == 0)
- dev_notice(mdev_dev(mdev),
- "No mdev vendor driver request callback support, blocked until released by user\n");
-}
-
-static const struct vfio_device_ops vfio_mdev_dev_ops = {
- .name = "vfio-mdev",
- .open = vfio_mdev_open,
- .release = vfio_mdev_release,
- .ioctl = vfio_mdev_unlocked_ioctl,
- .read = vfio_mdev_read,
- .write = vfio_mdev_write,
- .mmap = vfio_mdev_mmap,
- .request = vfio_mdev_request,
-};
-
-static int vfio_mdev_probe(struct mdev_device *mdev)
-{
- struct vfio_device *vdev;
- int ret;
-
- vdev = kzalloc(sizeof(*vdev), GFP_KERNEL);
- if (!vdev)
- return -ENOMEM;
-
- vfio_init_group_dev(vdev, &mdev->dev, &vfio_mdev_dev_ops);
- ret = vfio_register_group_dev(vdev);
- if (ret) {
- kfree(vdev);
- return ret;
- }
- dev_set_drvdata(&mdev->dev, vdev);
- return 0;
-}
-
-static void vfio_mdev_remove(struct mdev_device *mdev)
-{
- struct vfio_device *vdev = dev_get_drvdata(&mdev->dev);
-
- vfio_unregister_group_dev(vdev);
- kfree(vdev);
-}
-
-struct mdev_driver vfio_mdev_driver = {
- .driver = {
- .name = "vfio_mdev",
- .owner = THIS_MODULE,
- .mod_name = KBUILD_MODNAME,
- },
- .probe = vfio_mdev_probe,
- .remove = vfio_mdev_remove,
-};
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 5e631c359ef23c..59bbdf6634f934 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -747,7 +747,7 @@ void vfio_init_group_dev(struct vfio_device *device, struct device *dev,
device->dev = dev;
device->ops = ops;
}
-EXPORT_SYMBOL_GPL(vfio_init_group_dev);
+EXPORT_SYMBOL(vfio_init_group_dev);
int vfio_register_group_dev(struct vfio_device *device)
{
@@ -796,7 +796,7 @@ int vfio_register_group_dev(struct vfio_device *device)
return 0;
}
-EXPORT_SYMBOL_GPL(vfio_register_group_dev);
+EXPORT_SYMBOL(vfio_register_group_dev);
/**
* Get a reference to the vfio_device for a device. Even if the
@@ -927,7 +927,7 @@ void vfio_unregister_group_dev(struct vfio_device *device)
/* Matches the get in vfio_register_group_dev() */
vfio_group_put(group);
}
-EXPORT_SYMBOL_GPL(vfio_unregister_group_dev);
+EXPORT_SYMBOL(vfio_unregister_group_dev);
/**
* VFIO base fd, /dev/vfio/vfio
diff --git a/include/linux/mdev.h b/include/linux/mdev.h
index 49cc4f65120d57..ea48c401e4fa63 100644
--- a/include/linux/mdev.h
+++ b/include/linux/mdev.h
@@ -61,45 +61,6 @@ struct device *mtype_get_parent_dev(struct mdev_type *mtype);
* @mdev_attr_groups: Attributes of the mediated device.
* @supported_type_groups: Attributes to define supported types. It is mandatory
* to provide supported types.
- * @create: Called to allocate basic resources in parent device's
- * driver for a particular mediated device. It is
- * mandatory to provide create ops.
- * @mdev: mdev_device structure on of mediated device
- * that is being created
- * Returns integer: success (0) or error (< 0)
- * @remove: Called to free resources in parent device's driver for
- * a mediated device. It is mandatory to provide 'remove'
- * ops.
- * @mdev: mdev_device device structure which is being
- * destroyed
- * Returns integer: success (0) or error (< 0)
- * @open: Open mediated device.
- * @mdev: mediated device.
- * Returns integer: success (0) or error (< 0)
- * @release: release mediated device
- * @mdev: mediated device.
- * @read: Read emulation callback
- * @mdev: mediated device structure
- * @buf: read buffer
- * @count: number of bytes to read
- * @ppos: address.
- * Retuns number on bytes read on success or error.
- * @write: Write emulation callback
- * @mdev: mediated device structure
- * @buf: write buffer
- * @count: number of bytes to be written
- * @ppos: address.
- * Retuns number on bytes written on success or error.
- * @ioctl: IOCTL callback
- * @mdev: mediated device structure
- * @cmd: ioctl command
- * @arg: arguments to ioctl
- * @mmap: mmap callback
- * @mdev: mediated device structure
- * @vma: vma structure
- * @request: request callback to release device
- * @mdev: mediated device structure
- * @count: request sequence number
* Parent device that support mediated device should be registered with mdev
* module with mdev_parent_ops structure.
**/
@@ -109,19 +70,6 @@ struct mdev_parent_ops {
const struct attribute_group **dev_attr_groups;
const struct attribute_group **mdev_attr_groups;
struct attribute_group **supported_type_groups;
-
- int (*create)(struct mdev_device *mdev);
- int (*remove)(struct mdev_device *mdev);
- int (*open)(struct mdev_device *mdev);
- void (*release)(struct mdev_device *mdev);
- ssize_t (*read)(struct mdev_device *mdev, char __user *buf,
- size_t count, loff_t *ppos);
- ssize_t (*write)(struct mdev_device *mdev, const char __user *buf,
- size_t count, loff_t *ppos);
- long (*ioctl)(struct mdev_device *mdev, unsigned int cmd,
- unsigned long arg);
- int (*mmap)(struct mdev_device *mdev, struct vm_area_struct *vma);
- void (*request)(struct mdev_device *mdev, unsigned int count);
};
/* interface for exporting mdev supported type attributes */
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index a2c5b30e1763ba..c5e08be4c56395 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -64,6 +64,10 @@ void vfio_init_group_dev(struct vfio_device *device, struct device *dev,
int vfio_register_group_dev(struct vfio_device *device);
void vfio_unregister_group_dev(struct vfio_device *device);
extern struct vfio_device *vfio_device_get_from_dev(struct device *dev);
+static inline void vfio_device_get(struct vfio_device *device)
+{
+ refcount_inc(&device->refcount);
+}
extern void vfio_device_put(struct vfio_device *device);
/* events for the backend driver notify callback */
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 08/12] vfio/gvt: Convert to use vfio_register_group_dev()
@ 2021-04-23 23:03 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: David Airlie, Alex Williamson, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, intel-gfx, intel-gvt-dev, Jani Nikula,
Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc, Rodrigo Vivi,
Zhenyu Wang, Zhi Wang
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
While there is a confusing mess of pointers and structs in this driver,
the struct kvmgt_vdev (which in turn is 1:1 with a struct intel_vgpu) is
what holds the vfio_device. Replace all the drvdata's and weird
derivations of vgpu and vdev with container_of() or vdev->vgpu.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
.../driver-api/vfio-mediated-device.rst | 19 --
drivers/gpu/drm/i915/gvt/kvmgt.c | 208 ++++++++++--------
drivers/vfio/mdev/Makefile | 2 +-
drivers/vfio/mdev/mdev_core.c | 47 +---
drivers/vfio/mdev/mdev_driver.c | 11 +-
drivers/vfio/mdev/mdev_private.h | 2 -
drivers/vfio/mdev/vfio_mdev.c | 158 -------------
drivers/vfio/vfio.c | 6 +-
include/linux/mdev.h | 52 -----
include/linux/vfio.h | 4 +
10 files changed, 126 insertions(+), 383 deletions(-)
delete mode 100644 drivers/vfio/mdev/vfio_mdev.c
diff --git a/Documentation/driver-api/vfio-mediated-device.rst b/Documentation/driver-api/vfio-mediated-device.rst
index 1779b85f014e2f..5f866b17c93e69 100644
--- a/Documentation/driver-api/vfio-mediated-device.rst
+++ b/Documentation/driver-api/vfio-mediated-device.rst
@@ -137,25 +137,6 @@ The structures in the mdev_parent_ops structure are as follows:
* mdev_attr_groups: attributes of the mediated device
* supported_config: attributes to define supported configurations
-The functions in the mdev_parent_ops structure are as follows:
-
-* create: allocate basic resources in a driver for a mediated device
-* remove: free resources in a driver when a mediated device is destroyed
-
-(Note that mdev-core provides no implicit serialization of create/remove
-callbacks per mdev parent device, per mdev type, or any other categorization.
-Vendor drivers are expected to be fully asynchronous in this respect or
-provide their own internal resource protection.)
-
-The callbacks in the mdev_parent_ops structure are as follows:
-
-* open: open callback of mediated device
-* close: close callback of mediated device
-* ioctl: ioctl callback of mediated device
-* read : read emulation callback
-* write: write emulation callback
-* mmap: mmap emulation callback
-
A driver should use the mdev_parent_ops structure in the function call to
register itself with the mdev core driver::
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 6bf176e8426e63..85ef300087e091 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -50,6 +50,7 @@
#include "gvt.h"
static const struct intel_gvt_ops *intel_gvt_ops;
+static const struct vfio_device_ops intel_vgpu_dev_ops;
/* helper macros copied from vfio-pci */
#define VFIO_PCI_OFFSET_SHIFT 40
@@ -109,8 +110,8 @@ struct gvt_dma {
};
struct kvmgt_vdev {
+ struct vfio_device vfio_device;
struct intel_vgpu *vgpu;
- struct mdev_device *mdev;
struct vfio_region *region;
int num_regions;
struct eventfd_ctx *intx_trigger;
@@ -130,7 +131,6 @@ struct kvmgt_vdev {
struct kvm *kvm;
struct work_struct release_work;
atomic_t released;
- struct vfio_device *vfio_device;
struct vfio_group *vfio_group;
};
@@ -144,7 +144,7 @@ static inline bool handle_valid(unsigned long handle)
return !!(handle & ~0xff);
}
-static int kvmgt_guest_init(struct mdev_device *mdev);
+static int kvmgt_guest_init(struct kvmgt_vdev *vdev);
static void intel_vgpu_release_work(struct work_struct *work);
static bool kvmgt_guest_exit(struct kvmgt_guest_info *info);
@@ -611,12 +611,7 @@ static int kvmgt_get_vfio_device(void *p_vgpu)
struct intel_vgpu *vgpu = (struct intel_vgpu *)p_vgpu;
struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
- vdev->vfio_device = vfio_device_get_from_dev(
- mdev_dev(vdev->mdev));
- if (!vdev->vfio_device) {
- gvt_vgpu_err("failed to get vfio device\n");
- return -ENODEV;
- }
+ vfio_device_get(&vdev->vfio_device);
return 0;
}
@@ -683,16 +678,14 @@ static void kvmgt_put_vfio_device(void *vgpu)
{
struct kvmgt_vdev *vdev = kvmgt_vdev((struct intel_vgpu *)vgpu);
- if (WARN_ON(!vdev->vfio_device))
- return;
-
- vfio_device_put(vdev->vfio_device);
+ vfio_device_put(&vdev->vfio_device);
}
-static int intel_vgpu_create(struct mdev_device *mdev)
+static int intel_vgpu_probe(struct mdev_device *mdev)
{
struct intel_vgpu *vgpu = NULL;
struct intel_vgpu_type *type;
+ struct kvmgt_vdev *vdev;
struct device *pdev;
void *gvt;
int ret;
@@ -702,40 +695,40 @@ static int intel_vgpu_create(struct mdev_device *mdev)
type = intel_gvt_ops->gvt_find_vgpu_type(gvt,
mdev_get_type_group_id(mdev));
- if (!type) {
- ret = -EINVAL;
- goto out;
- }
+ if (!type)
+ return -EINVAL;
vgpu = intel_gvt_ops->vgpu_create(gvt, type);
if (IS_ERR_OR_NULL(vgpu)) {
- ret = vgpu == NULL ? -EFAULT : PTR_ERR(vgpu);
gvt_err("failed to create intel vgpu: %d\n", ret);
- goto out;
+ return vgpu == NULL ? -EFAULT : PTR_ERR(vgpu);
}
- INIT_WORK(&kvmgt_vdev(vgpu)->release_work, intel_vgpu_release_work);
+ vdev = kvmgt_vdev(vgpu);
+ INIT_WORK(&vdev->release_work, intel_vgpu_release_work);
+ vfio_init_group_dev(&vdev->vfio_device, &mdev->dev,
+ &intel_vgpu_dev_ops);
- kvmgt_vdev(vgpu)->mdev = mdev;
- mdev_set_drvdata(mdev, vgpu);
+ ret = vfio_register_group_dev(&vdev->vfio_device);
+ if (ret) {
+ intel_gvt_ops->vgpu_destroy(vgpu);
+ return ret;
+ }
+ dev_set_drvdata(&mdev->dev, vdev);
gvt_dbg_core("intel_vgpu_create succeeded for mdev: %s\n",
dev_name(mdev_dev(mdev)));
- ret = 0;
-
-out:
- return ret;
+ return 0;
}
-static int intel_vgpu_remove(struct mdev_device *mdev)
+static void intel_vgpu_remove(struct mdev_device *mdev)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
-
- if (handle_valid(vgpu->handle))
- return -EBUSY;
+ struct kvmgt_vdev *vdev = dev_get_drvdata(&mdev->dev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
+ if (WARN_ON(handle_valid(vgpu->handle)))
+ return;
intel_gvt_ops->vgpu_destroy(vgpu);
- return 0;
}
static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
@@ -788,10 +781,11 @@ static int intel_vgpu_group_notifier(struct notifier_block *nb,
return NOTIFY_OK;
}
-static int intel_vgpu_open(struct mdev_device *mdev)
+static int intel_vgpu_open(struct vfio_device *vfio_dev)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
- struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned long events;
int ret;
struct vfio_group *vfio_group;
@@ -800,7 +794,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
vdev->group_notifier.notifier_call = intel_vgpu_group_notifier;
events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
- ret = vfio_register_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, &events,
+ ret = vfio_register_notifier(vfio_dev->dev, VFIO_IOMMU_NOTIFY, &events,
&vdev->iommu_notifier);
if (ret != 0) {
gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n",
@@ -809,7 +803,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
}
events = VFIO_GROUP_NOTIFY_SET_KVM;
- ret = vfio_register_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, &events,
+ ret = vfio_register_notifier(vfio_dev->dev, VFIO_GROUP_NOTIFY, &events,
&vdev->group_notifier);
if (ret != 0) {
gvt_vgpu_err("vfio_register_notifier for group failed: %d\n",
@@ -817,7 +811,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
goto undo_iommu;
}
- vfio_group = vfio_group_get_external_user_from_dev(mdev_dev(mdev));
+ vfio_group = vfio_group_get_external_user_from_dev(vfio_dev->dev);
if (IS_ERR_OR_NULL(vfio_group)) {
ret = !vfio_group ? -EFAULT : PTR_ERR(vfio_group);
gvt_vgpu_err("vfio_group_get_external_user_from_dev failed\n");
@@ -833,11 +827,11 @@ static int intel_vgpu_open(struct mdev_device *mdev)
goto undo_group;
}
- ret = kvmgt_guest_init(mdev);
+ ret = kvmgt_guest_init(vdev);
if (ret)
goto undo_group;
- intel_gvt_ops->vgpu_activate(vgpu);
+ intel_gvt_ops->vgpu_activate(vdev->vgpu);
atomic_set(&vdev->released, 0);
return ret;
@@ -847,11 +841,11 @@ static int intel_vgpu_open(struct mdev_device *mdev)
vdev->vfio_group = NULL;
undo_register:
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY,
+ vfio_unregister_notifier(vfio_dev->dev, VFIO_GROUP_NOTIFY,
&vdev->group_notifier);
undo_iommu:
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+ vfio_unregister_notifier(vfio_dev->dev, VFIO_IOMMU_NOTIFY,
&vdev->iommu_notifier);
out:
return ret;
@@ -884,12 +878,12 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
intel_gvt_ops->vgpu_release(vgpu);
- ret = vfio_unregister_notifier(mdev_dev(vdev->mdev), VFIO_IOMMU_NOTIFY,
+ ret = vfio_unregister_notifier(vdev->vfio_device.dev, VFIO_IOMMU_NOTIFY,
&vdev->iommu_notifier);
drm_WARN(&i915->drm, ret,
"vfio_unregister_notifier for iommu failed: %d\n", ret);
- ret = vfio_unregister_notifier(mdev_dev(vdev->mdev), VFIO_GROUP_NOTIFY,
+ ret = vfio_unregister_notifier(vdev->vfio_device.dev, VFIO_GROUP_NOTIFY,
&vdev->group_notifier);
drm_WARN(&i915->drm, ret,
"vfio_unregister_notifier for group failed: %d\n", ret);
@@ -907,11 +901,12 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
vgpu->handle = 0;
}
-static void intel_vgpu_release(struct mdev_device *mdev)
+static void intel_vgpu_release(struct vfio_device *vfio_dev)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
- __intel_vgpu_release(vgpu);
+ __intel_vgpu_release(vdev->vgpu);
}
static void intel_vgpu_release_work(struct work_struct *work)
@@ -997,11 +992,10 @@ static int intel_vgpu_aperture_rw(struct intel_vgpu *vgpu, u64 off,
return 0;
}
-static ssize_t intel_vgpu_rw(struct mdev_device *mdev, char *buf,
+static ssize_t intel_vgpu_rw(struct kvmgt_vdev *vdev, char *buf,
size_t count, loff_t *ppos, bool is_write)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
- struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
u64 pos = *ppos & VFIO_PCI_OFFSET_MASK;
int ret = -EINVAL;
@@ -1047,9 +1041,9 @@ static ssize_t intel_vgpu_rw(struct mdev_device *mdev, char *buf,
return ret == 0 ? count : ret;
}
-static bool gtt_entry(struct mdev_device *mdev, loff_t *ppos)
+static bool gtt_entry(struct kvmgt_vdev *vdev, loff_t *ppos)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
struct intel_gvt *gvt = vgpu->gvt;
int offset;
@@ -1066,9 +1060,11 @@ static bool gtt_entry(struct mdev_device *mdev, loff_t *ppos)
true : false;
}
-static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
+static ssize_t intel_vgpu_read(struct vfio_device *vfio_dev, char __user *buf,
size_t count, loff_t *ppos)
{
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
unsigned int done = 0;
int ret;
@@ -1077,10 +1073,10 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
/* Only support GGTT entry 8 bytes read */
if (count >= 8 && !(*ppos % 8) &&
- gtt_entry(mdev, ppos)) {
+ gtt_entry(vdev, ppos)) {
u64 val;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, false);
if (ret <= 0)
goto read_err;
@@ -1092,7 +1088,7 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
} else if (count >= 4 && !(*ppos % 4)) {
u32 val;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, false);
if (ret <= 0)
goto read_err;
@@ -1104,7 +1100,7 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
} else if (count >= 2 && !(*ppos % 2)) {
u16 val;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, false);
if (ret <= 0)
goto read_err;
@@ -1116,7 +1112,7 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
} else {
u8 val;
- ret = intel_vgpu_rw(mdev, &val, sizeof(val), ppos,
+ ret = intel_vgpu_rw(vdev, &val, sizeof(val), ppos,
false);
if (ret <= 0)
goto read_err;
@@ -1139,10 +1135,12 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
return -EFAULT;
}
-static ssize_t intel_vgpu_write(struct mdev_device *mdev,
+static ssize_t intel_vgpu_write(struct vfio_device *vfio_dev,
const char __user *buf,
size_t count, loff_t *ppos)
{
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
unsigned int done = 0;
int ret;
@@ -1151,13 +1149,13 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
/* Only support GGTT entry 8 bytes write */
if (count >= 8 && !(*ppos % 8) &&
- gtt_entry(mdev, ppos)) {
+ gtt_entry(vdev, ppos)) {
u64 val;
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, true);
if (ret <= 0)
goto write_err;
@@ -1169,7 +1167,7 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, true);
if (ret <= 0)
goto write_err;
@@ -1181,7 +1179,7 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, (char *)&val,
+ ret = intel_vgpu_rw(vdev, (char *)&val,
sizeof(val), ppos, true);
if (ret <= 0)
goto write_err;
@@ -1193,7 +1191,7 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, &val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, &val, sizeof(val),
ppos, true);
if (ret <= 0)
goto write_err;
@@ -1212,13 +1210,16 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
return -EFAULT;
}
-static int intel_vgpu_mmap(struct mdev_device *mdev, struct vm_area_struct *vma)
+static int intel_vgpu_mmap(struct vfio_device *vfio_dev,
+ struct vm_area_struct *vma)
{
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
unsigned int index;
u64 virtaddr;
unsigned long req_size, pgoff, req_start;
pgprot_t pg_prot;
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
index = vma->vm_pgoff >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
if (index >= VFIO_PCI_ROM_REGION_INDEX)
@@ -1341,11 +1342,12 @@ static int intel_vgpu_set_irqs(struct intel_vgpu *vgpu, u32 flags,
return func(vgpu, index, start, count, flags, data);
}
-static long intel_vgpu_ioctl(struct mdev_device *mdev, unsigned int cmd,
+static long intel_vgpu_ioctl(struct vfio_device *vfio_dev, unsigned int cmd,
unsigned long arg)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
- struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned long minsz;
gvt_dbg_core("vgpu%d ioctl, cmd: %d\n", vgpu->id, cmd);
@@ -1624,14 +1626,10 @@ static ssize_t
vgpu_id_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct mdev_device *mdev = mdev_from_dev(dev);
+ struct kvmgt_vdev *vdev = dev_get_drvdata(dev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
- if (mdev) {
- struct intel_vgpu *vgpu = (struct intel_vgpu *)
- mdev_get_drvdata(mdev);
- return sprintf(buf, "%d\n", vgpu->id);
- }
- return sprintf(buf, "\n");
+ return sprintf(buf, "%d\n", vgpu->id);
}
static DEVICE_ATTR_RO(vgpu_id);
@@ -1651,18 +1649,28 @@ static const struct attribute_group *intel_vgpu_groups[] = {
NULL,
};
-static struct mdev_parent_ops intel_vgpu_ops = {
- .mdev_attr_groups = intel_vgpu_groups,
- .create = intel_vgpu_create,
- .remove = intel_vgpu_remove,
+static const struct vfio_device_ops intel_vgpu_dev_ops = {
+ .open = intel_vgpu_open,
+ .release = intel_vgpu_release,
+ .read = intel_vgpu_read,
+ .write = intel_vgpu_write,
+ .mmap = intel_vgpu_mmap,
+ .ioctl = intel_vgpu_ioctl,
+};
- .open = intel_vgpu_open,
- .release = intel_vgpu_release,
+static struct mdev_driver intel_vgpu_mdev_driver = {
+ .driver = {
+ .name = "intel_vgpu_mdev",
+ .owner = THIS_MODULE,
+ .mod_name = KBUILD_MODNAME,
+ .dev_groups = intel_vgpu_groups,
+ },
+ .probe = intel_vgpu_probe,
+ .remove = intel_vgpu_remove,
+};
- .read = intel_vgpu_read,
- .write = intel_vgpu_write,
- .mmap = intel_vgpu_mmap,
- .ioctl = intel_vgpu_ioctl,
+static struct mdev_parent_ops intel_vgpu_ops = {
+ .device_driver = &intel_vgpu_mdev_driver,
};
static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
@@ -1806,18 +1814,12 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu, struct kvm *kvm)
return ret;
}
-static int kvmgt_guest_init(struct mdev_device *mdev)
+static int kvmgt_guest_init(struct kvmgt_vdev *vdev)
{
struct kvmgt_guest_info *info;
- struct intel_vgpu *vgpu;
- struct kvmgt_vdev *vdev;
+ struct intel_vgpu *vgpu = vdev->vgpu;
struct kvm *kvm;
- vgpu = mdev_get_drvdata(mdev);
- if (handle_valid(vgpu->handle))
- return -EEXIST;
-
- vdev = kvmgt_vdev(vgpu);
kvm = vdev->kvm;
if (!kvm || kvm->mm != current->mm) {
gvt_vgpu_err("KVM is required to use Intel vGPU\n");
@@ -2125,13 +2127,25 @@ static const struct intel_gvt_mpt kvmgt_mpt = {
static int __init kvmgt_init(void)
{
- if (intel_gvt_register_hypervisor(&kvmgt_mpt) < 0)
- return -ENODEV;
+ int ret;
+
+ ret = mdev_register_driver(&intel_vgpu_mdev_driver);
+ if (ret)
+ return ret;
+
+ if (intel_gvt_register_hypervisor(&kvmgt_mpt) < 0) {
+ ret = -ENODEV;
+ goto err_driver;
+ }
return 0;
+err_driver:
+ mdev_unregister_driver(&intel_vgpu_mdev_driver);
+ return ret;
}
static void __exit kvmgt_exit(void)
{
+ mdev_unregister_driver(&intel_vgpu_mdev_driver);
intel_gvt_unregister_hypervisor();
}
diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
index ff9ecd80212503..7c236ba1b90eb1 100644
--- a/drivers/vfio/mdev/Makefile
+++ b/drivers/vfio/mdev/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
-mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
+mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
obj-$(CONFIG_VFIO_MDEV) += mdev.o
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index 51b8a9fcf866ad..f95d01b57fb168 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -89,17 +89,10 @@ void mdev_release_parent(struct kref *kref)
static void mdev_device_remove_common(struct mdev_device *mdev)
{
struct mdev_parent *parent = mdev->type->parent;
- int ret;
mdev_remove_sysfs_files(mdev);
device_del(&mdev->dev);
lockdep_assert_held(&parent->unreg_sem);
- if (parent->ops->remove) {
- ret = parent->ops->remove(mdev);
- if (ret)
- dev_err(&mdev->dev, "Remove failed: err=%d\n", ret);
- }
-
/* Balances with device_initialize() */
put_device(&mdev->dev);
}
@@ -131,17 +124,13 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
/* check for mandatory ops */
if (!ops || !ops->supported_type_groups)
return -EINVAL;
- if (!ops->device_driver && (!ops->create || !ops->remove))
+ if (!ops->device_driver)
return -EINVAL;
dev = get_device(dev);
if (!dev)
return -EINVAL;
- /* Not mandatory, but its absence could be a problem */
- if (!ops->request)
- dev_info(dev, "Driver cannot be asked to release device\n");
-
mutex_lock(&parent_list_lock);
/* Check for duplicate */
@@ -263,15 +252,12 @@ static void mdev_device_release(struct device *dev)
*/
static int mdev_bind_driver(struct mdev_device *mdev)
{
- struct mdev_driver *drv = mdev->type->parent->ops->device_driver;
int ret;
- if (!drv)
- drv = &vfio_mdev_driver;
-
while (1) {
device_lock(&mdev->dev);
- if (mdev->dev.driver == &drv->driver) {
+ if (mdev->dev.driver ==
+ &mdev->type->parent->ops->device_driver->driver) {
ret = 0;
goto out_unlock;
}
@@ -337,15 +323,9 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
goto out_put_device;
}
- if (parent->ops->create) {
- ret = parent->ops->create(mdev);
- if (ret)
- goto out_unlock;
- }
-
ret = device_add(&mdev->dev);
if (ret)
- goto out_remove;
+ goto out_unlock;
ret = mdev_bind_driver(mdev);
if (ret)
@@ -363,9 +343,6 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
out_del:
device_del(&mdev->dev);
-out_remove:
- if (parent->ops->remove)
- parent->ops->remove(mdev);
out_unlock:
up_read(&parent->unreg_sem);
out_put_device:
@@ -408,27 +385,13 @@ int mdev_device_remove(struct mdev_device *mdev)
static int __init mdev_init(void)
{
- int rc;
-
- rc = mdev_bus_register();
- if (rc)
- return rc;
- rc = mdev_register_driver(&vfio_mdev_driver);
- if (rc)
- goto err_bus;
- return 0;
-err_bus:
- mdev_bus_unregister();
- return rc;
+ return mdev_bus_register();
}
static void __exit mdev_exit(void)
{
- mdev_unregister_driver(&vfio_mdev_driver);
-
if (mdev_bus_compat_class)
class_compat_unregister(mdev_bus_compat_class);
-
mdev_bus_unregister();
}
diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
index 6e96c023d7823d..0012a9ee7cb0a4 100644
--- a/drivers/vfio/mdev/mdev_driver.c
+++ b/drivers/vfio/mdev/mdev_driver.c
@@ -74,15 +74,8 @@ static int mdev_remove(struct device *dev)
static int mdev_match(struct device *dev, struct device_driver *drv)
{
struct mdev_device *mdev = to_mdev_device(dev);
- struct mdev_driver *target = mdev->type->parent->ops->device_driver;
-
- /*
- * The ops specify the device driver to connect, fall back to the old
- * shim driver if the driver hasn't been converted.
- */
- if (!target)
- target = &vfio_mdev_driver;
- return drv == &target->driver;
+
+ return drv == &mdev->type->parent->ops->device_driver->driver;
}
struct bus_type mdev_bus_type = {
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
index 5461b67582289f..a656cfe0346c33 100644
--- a/drivers/vfio/mdev/mdev_private.h
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -37,8 +37,6 @@ struct mdev_type {
#define to_mdev_type(_kobj) \
container_of(_kobj, struct mdev_type, kobj)
-extern struct mdev_driver vfio_mdev_driver;
-
int parent_create_sysfs_files(struct mdev_parent *parent);
void parent_remove_sysfs_files(struct mdev_parent *parent);
diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
deleted file mode 100644
index d5b4eede47c1a5..00000000000000
--- a/drivers/vfio/mdev/vfio_mdev.c
+++ /dev/null
@@ -1,158 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * VFIO based driver for Mediated device
- *
- * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
- * Author: Neo Jia <cjia@nvidia.com>
- * Kirti Wankhede <kwankhede@nvidia.com>
- */
-
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/device.h>
-#include <linux/kernel.h>
-#include <linux/slab.h>
-#include <linux/vfio.h>
-#include <linux/mdev.h>
-
-#include "mdev_private.h"
-
-static int vfio_mdev_open(struct vfio_device *core_vdev)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- int ret;
-
- if (unlikely(!parent->ops->open))
- return -EINVAL;
-
- if (!try_module_get(THIS_MODULE))
- return -ENODEV;
-
- ret = parent->ops->open(mdev);
- if (ret)
- module_put(THIS_MODULE);
-
- return ret;
-}
-
-static void vfio_mdev_release(struct vfio_device *core_vdev)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (likely(parent->ops->release))
- parent->ops->release(mdev);
-
- module_put(THIS_MODULE);
-}
-
-static long vfio_mdev_unlocked_ioctl(struct vfio_device *core_vdev,
- unsigned int cmd, unsigned long arg)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->ioctl))
- return -EINVAL;
-
- return parent->ops->ioctl(mdev, cmd, arg);
-}
-
-static ssize_t vfio_mdev_read(struct vfio_device *core_vdev, char __user *buf,
- size_t count, loff_t *ppos)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->read))
- return -EINVAL;
-
- return parent->ops->read(mdev, buf, count, ppos);
-}
-
-static ssize_t vfio_mdev_write(struct vfio_device *core_vdev,
- const char __user *buf, size_t count,
- loff_t *ppos)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->write))
- return -EINVAL;
-
- return parent->ops->write(mdev, buf, count, ppos);
-}
-
-static int vfio_mdev_mmap(struct vfio_device *core_vdev,
- struct vm_area_struct *vma)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->mmap))
- return -EINVAL;
-
- return parent->ops->mmap(mdev, vma);
-}
-
-static void vfio_mdev_request(struct vfio_device *core_vdev, unsigned int count)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (parent->ops->request)
- parent->ops->request(mdev, count);
- else if (count == 0)
- dev_notice(mdev_dev(mdev),
- "No mdev vendor driver request callback support, blocked until released by user\n");
-}
-
-static const struct vfio_device_ops vfio_mdev_dev_ops = {
- .name = "vfio-mdev",
- .open = vfio_mdev_open,
- .release = vfio_mdev_release,
- .ioctl = vfio_mdev_unlocked_ioctl,
- .read = vfio_mdev_read,
- .write = vfio_mdev_write,
- .mmap = vfio_mdev_mmap,
- .request = vfio_mdev_request,
-};
-
-static int vfio_mdev_probe(struct mdev_device *mdev)
-{
- struct vfio_device *vdev;
- int ret;
-
- vdev = kzalloc(sizeof(*vdev), GFP_KERNEL);
- if (!vdev)
- return -ENOMEM;
-
- vfio_init_group_dev(vdev, &mdev->dev, &vfio_mdev_dev_ops);
- ret = vfio_register_group_dev(vdev);
- if (ret) {
- kfree(vdev);
- return ret;
- }
- dev_set_drvdata(&mdev->dev, vdev);
- return 0;
-}
-
-static void vfio_mdev_remove(struct mdev_device *mdev)
-{
- struct vfio_device *vdev = dev_get_drvdata(&mdev->dev);
-
- vfio_unregister_group_dev(vdev);
- kfree(vdev);
-}
-
-struct mdev_driver vfio_mdev_driver = {
- .driver = {
- .name = "vfio_mdev",
- .owner = THIS_MODULE,
- .mod_name = KBUILD_MODNAME,
- },
- .probe = vfio_mdev_probe,
- .remove = vfio_mdev_remove,
-};
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 5e631c359ef23c..59bbdf6634f934 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -747,7 +747,7 @@ void vfio_init_group_dev(struct vfio_device *device, struct device *dev,
device->dev = dev;
device->ops = ops;
}
-EXPORT_SYMBOL_GPL(vfio_init_group_dev);
+EXPORT_SYMBOL(vfio_init_group_dev);
int vfio_register_group_dev(struct vfio_device *device)
{
@@ -796,7 +796,7 @@ int vfio_register_group_dev(struct vfio_device *device)
return 0;
}
-EXPORT_SYMBOL_GPL(vfio_register_group_dev);
+EXPORT_SYMBOL(vfio_register_group_dev);
/**
* Get a reference to the vfio_device for a device. Even if the
@@ -927,7 +927,7 @@ void vfio_unregister_group_dev(struct vfio_device *device)
/* Matches the get in vfio_register_group_dev() */
vfio_group_put(group);
}
-EXPORT_SYMBOL_GPL(vfio_unregister_group_dev);
+EXPORT_SYMBOL(vfio_unregister_group_dev);
/**
* VFIO base fd, /dev/vfio/vfio
diff --git a/include/linux/mdev.h b/include/linux/mdev.h
index 49cc4f65120d57..ea48c401e4fa63 100644
--- a/include/linux/mdev.h
+++ b/include/linux/mdev.h
@@ -61,45 +61,6 @@ struct device *mtype_get_parent_dev(struct mdev_type *mtype);
* @mdev_attr_groups: Attributes of the mediated device.
* @supported_type_groups: Attributes to define supported types. It is mandatory
* to provide supported types.
- * @create: Called to allocate basic resources in parent device's
- * driver for a particular mediated device. It is
- * mandatory to provide create ops.
- * @mdev: mdev_device structure on of mediated device
- * that is being created
- * Returns integer: success (0) or error (< 0)
- * @remove: Called to free resources in parent device's driver for
- * a mediated device. It is mandatory to provide 'remove'
- * ops.
- * @mdev: mdev_device device structure which is being
- * destroyed
- * Returns integer: success (0) or error (< 0)
- * @open: Open mediated device.
- * @mdev: mediated device.
- * Returns integer: success (0) or error (< 0)
- * @release: release mediated device
- * @mdev: mediated device.
- * @read: Read emulation callback
- * @mdev: mediated device structure
- * @buf: read buffer
- * @count: number of bytes to read
- * @ppos: address.
- * Retuns number on bytes read on success or error.
- * @write: Write emulation callback
- * @mdev: mediated device structure
- * @buf: write buffer
- * @count: number of bytes to be written
- * @ppos: address.
- * Retuns number on bytes written on success or error.
- * @ioctl: IOCTL callback
- * @mdev: mediated device structure
- * @cmd: ioctl command
- * @arg: arguments to ioctl
- * @mmap: mmap callback
- * @mdev: mediated device structure
- * @vma: vma structure
- * @request: request callback to release device
- * @mdev: mediated device structure
- * @count: request sequence number
* Parent device that support mediated device should be registered with mdev
* module with mdev_parent_ops structure.
**/
@@ -109,19 +70,6 @@ struct mdev_parent_ops {
const struct attribute_group **dev_attr_groups;
const struct attribute_group **mdev_attr_groups;
struct attribute_group **supported_type_groups;
-
- int (*create)(struct mdev_device *mdev);
- int (*remove)(struct mdev_device *mdev);
- int (*open)(struct mdev_device *mdev);
- void (*release)(struct mdev_device *mdev);
- ssize_t (*read)(struct mdev_device *mdev, char __user *buf,
- size_t count, loff_t *ppos);
- ssize_t (*write)(struct mdev_device *mdev, const char __user *buf,
- size_t count, loff_t *ppos);
- long (*ioctl)(struct mdev_device *mdev, unsigned int cmd,
- unsigned long arg);
- int (*mmap)(struct mdev_device *mdev, struct vm_area_struct *vma);
- void (*request)(struct mdev_device *mdev, unsigned int count);
};
/* interface for exporting mdev supported type attributes */
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index a2c5b30e1763ba..c5e08be4c56395 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -64,6 +64,10 @@ void vfio_init_group_dev(struct vfio_device *device, struct device *dev,
int vfio_register_group_dev(struct vfio_device *device);
void vfio_unregister_group_dev(struct vfio_device *device);
extern struct vfio_device *vfio_device_get_from_dev(struct device *dev);
+static inline void vfio_device_get(struct vfio_device *device)
+{
+ refcount_inc(&device->refcount);
+}
extern void vfio_device_put(struct vfio_device *device);
/* events for the backend driver notify callback */
--
2.31.1
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [Intel-gfx] [PATCH 08/12] vfio/gvt: Convert to use vfio_register_group_dev()
@ 2021-04-23 23:03 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: David Airlie, Alex Williamson, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, intel-gfx, intel-gvt-dev, Jani Nikula,
Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc, Rodrigo Vivi,
Zhenyu Wang, Zhi Wang
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
While there is a confusing mess of pointers and structs in this driver,
the struct kvmgt_vdev (which in turn is 1:1 with a struct intel_vgpu) is
what holds the vfio_device. Replace all the drvdata's and weird
derivations of vgpu and vdev with container_of() or vdev->vgpu.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
.../driver-api/vfio-mediated-device.rst | 19 --
drivers/gpu/drm/i915/gvt/kvmgt.c | 208 ++++++++++--------
drivers/vfio/mdev/Makefile | 2 +-
drivers/vfio/mdev/mdev_core.c | 47 +---
drivers/vfio/mdev/mdev_driver.c | 11 +-
drivers/vfio/mdev/mdev_private.h | 2 -
drivers/vfio/mdev/vfio_mdev.c | 158 -------------
drivers/vfio/vfio.c | 6 +-
include/linux/mdev.h | 52 -----
include/linux/vfio.h | 4 +
10 files changed, 126 insertions(+), 383 deletions(-)
delete mode 100644 drivers/vfio/mdev/vfio_mdev.c
diff --git a/Documentation/driver-api/vfio-mediated-device.rst b/Documentation/driver-api/vfio-mediated-device.rst
index 1779b85f014e2f..5f866b17c93e69 100644
--- a/Documentation/driver-api/vfio-mediated-device.rst
+++ b/Documentation/driver-api/vfio-mediated-device.rst
@@ -137,25 +137,6 @@ The structures in the mdev_parent_ops structure are as follows:
* mdev_attr_groups: attributes of the mediated device
* supported_config: attributes to define supported configurations
-The functions in the mdev_parent_ops structure are as follows:
-
-* create: allocate basic resources in a driver for a mediated device
-* remove: free resources in a driver when a mediated device is destroyed
-
-(Note that mdev-core provides no implicit serialization of create/remove
-callbacks per mdev parent device, per mdev type, or any other categorization.
-Vendor drivers are expected to be fully asynchronous in this respect or
-provide their own internal resource protection.)
-
-The callbacks in the mdev_parent_ops structure are as follows:
-
-* open: open callback of mediated device
-* close: close callback of mediated device
-* ioctl: ioctl callback of mediated device
-* read : read emulation callback
-* write: write emulation callback
-* mmap: mmap emulation callback
-
A driver should use the mdev_parent_ops structure in the function call to
register itself with the mdev core driver::
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 6bf176e8426e63..85ef300087e091 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -50,6 +50,7 @@
#include "gvt.h"
static const struct intel_gvt_ops *intel_gvt_ops;
+static const struct vfio_device_ops intel_vgpu_dev_ops;
/* helper macros copied from vfio-pci */
#define VFIO_PCI_OFFSET_SHIFT 40
@@ -109,8 +110,8 @@ struct gvt_dma {
};
struct kvmgt_vdev {
+ struct vfio_device vfio_device;
struct intel_vgpu *vgpu;
- struct mdev_device *mdev;
struct vfio_region *region;
int num_regions;
struct eventfd_ctx *intx_trigger;
@@ -130,7 +131,6 @@ struct kvmgt_vdev {
struct kvm *kvm;
struct work_struct release_work;
atomic_t released;
- struct vfio_device *vfio_device;
struct vfio_group *vfio_group;
};
@@ -144,7 +144,7 @@ static inline bool handle_valid(unsigned long handle)
return !!(handle & ~0xff);
}
-static int kvmgt_guest_init(struct mdev_device *mdev);
+static int kvmgt_guest_init(struct kvmgt_vdev *vdev);
static void intel_vgpu_release_work(struct work_struct *work);
static bool kvmgt_guest_exit(struct kvmgt_guest_info *info);
@@ -611,12 +611,7 @@ static int kvmgt_get_vfio_device(void *p_vgpu)
struct intel_vgpu *vgpu = (struct intel_vgpu *)p_vgpu;
struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
- vdev->vfio_device = vfio_device_get_from_dev(
- mdev_dev(vdev->mdev));
- if (!vdev->vfio_device) {
- gvt_vgpu_err("failed to get vfio device\n");
- return -ENODEV;
- }
+ vfio_device_get(&vdev->vfio_device);
return 0;
}
@@ -683,16 +678,14 @@ static void kvmgt_put_vfio_device(void *vgpu)
{
struct kvmgt_vdev *vdev = kvmgt_vdev((struct intel_vgpu *)vgpu);
- if (WARN_ON(!vdev->vfio_device))
- return;
-
- vfio_device_put(vdev->vfio_device);
+ vfio_device_put(&vdev->vfio_device);
}
-static int intel_vgpu_create(struct mdev_device *mdev)
+static int intel_vgpu_probe(struct mdev_device *mdev)
{
struct intel_vgpu *vgpu = NULL;
struct intel_vgpu_type *type;
+ struct kvmgt_vdev *vdev;
struct device *pdev;
void *gvt;
int ret;
@@ -702,40 +695,40 @@ static int intel_vgpu_create(struct mdev_device *mdev)
type = intel_gvt_ops->gvt_find_vgpu_type(gvt,
mdev_get_type_group_id(mdev));
- if (!type) {
- ret = -EINVAL;
- goto out;
- }
+ if (!type)
+ return -EINVAL;
vgpu = intel_gvt_ops->vgpu_create(gvt, type);
if (IS_ERR_OR_NULL(vgpu)) {
- ret = vgpu == NULL ? -EFAULT : PTR_ERR(vgpu);
gvt_err("failed to create intel vgpu: %d\n", ret);
- goto out;
+ return vgpu == NULL ? -EFAULT : PTR_ERR(vgpu);
}
- INIT_WORK(&kvmgt_vdev(vgpu)->release_work, intel_vgpu_release_work);
+ vdev = kvmgt_vdev(vgpu);
+ INIT_WORK(&vdev->release_work, intel_vgpu_release_work);
+ vfio_init_group_dev(&vdev->vfio_device, &mdev->dev,
+ &intel_vgpu_dev_ops);
- kvmgt_vdev(vgpu)->mdev = mdev;
- mdev_set_drvdata(mdev, vgpu);
+ ret = vfio_register_group_dev(&vdev->vfio_device);
+ if (ret) {
+ intel_gvt_ops->vgpu_destroy(vgpu);
+ return ret;
+ }
+ dev_set_drvdata(&mdev->dev, vdev);
gvt_dbg_core("intel_vgpu_create succeeded for mdev: %s\n",
dev_name(mdev_dev(mdev)));
- ret = 0;
-
-out:
- return ret;
+ return 0;
}
-static int intel_vgpu_remove(struct mdev_device *mdev)
+static void intel_vgpu_remove(struct mdev_device *mdev)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
-
- if (handle_valid(vgpu->handle))
- return -EBUSY;
+ struct kvmgt_vdev *vdev = dev_get_drvdata(&mdev->dev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
+ if (WARN_ON(handle_valid(vgpu->handle)))
+ return;
intel_gvt_ops->vgpu_destroy(vgpu);
- return 0;
}
static int intel_vgpu_iommu_notifier(struct notifier_block *nb,
@@ -788,10 +781,11 @@ static int intel_vgpu_group_notifier(struct notifier_block *nb,
return NOTIFY_OK;
}
-static int intel_vgpu_open(struct mdev_device *mdev)
+static int intel_vgpu_open(struct vfio_device *vfio_dev)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
- struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned long events;
int ret;
struct vfio_group *vfio_group;
@@ -800,7 +794,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
vdev->group_notifier.notifier_call = intel_vgpu_group_notifier;
events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
- ret = vfio_register_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, &events,
+ ret = vfio_register_notifier(vfio_dev->dev, VFIO_IOMMU_NOTIFY, &events,
&vdev->iommu_notifier);
if (ret != 0) {
gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n",
@@ -809,7 +803,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
}
events = VFIO_GROUP_NOTIFY_SET_KVM;
- ret = vfio_register_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, &events,
+ ret = vfio_register_notifier(vfio_dev->dev, VFIO_GROUP_NOTIFY, &events,
&vdev->group_notifier);
if (ret != 0) {
gvt_vgpu_err("vfio_register_notifier for group failed: %d\n",
@@ -817,7 +811,7 @@ static int intel_vgpu_open(struct mdev_device *mdev)
goto undo_iommu;
}
- vfio_group = vfio_group_get_external_user_from_dev(mdev_dev(mdev));
+ vfio_group = vfio_group_get_external_user_from_dev(vfio_dev->dev);
if (IS_ERR_OR_NULL(vfio_group)) {
ret = !vfio_group ? -EFAULT : PTR_ERR(vfio_group);
gvt_vgpu_err("vfio_group_get_external_user_from_dev failed\n");
@@ -833,11 +827,11 @@ static int intel_vgpu_open(struct mdev_device *mdev)
goto undo_group;
}
- ret = kvmgt_guest_init(mdev);
+ ret = kvmgt_guest_init(vdev);
if (ret)
goto undo_group;
- intel_gvt_ops->vgpu_activate(vgpu);
+ intel_gvt_ops->vgpu_activate(vdev->vgpu);
atomic_set(&vdev->released, 0);
return ret;
@@ -847,11 +841,11 @@ static int intel_vgpu_open(struct mdev_device *mdev)
vdev->vfio_group = NULL;
undo_register:
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY,
+ vfio_unregister_notifier(vfio_dev->dev, VFIO_GROUP_NOTIFY,
&vdev->group_notifier);
undo_iommu:
- vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+ vfio_unregister_notifier(vfio_dev->dev, VFIO_IOMMU_NOTIFY,
&vdev->iommu_notifier);
out:
return ret;
@@ -884,12 +878,12 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
intel_gvt_ops->vgpu_release(vgpu);
- ret = vfio_unregister_notifier(mdev_dev(vdev->mdev), VFIO_IOMMU_NOTIFY,
+ ret = vfio_unregister_notifier(vdev->vfio_device.dev, VFIO_IOMMU_NOTIFY,
&vdev->iommu_notifier);
drm_WARN(&i915->drm, ret,
"vfio_unregister_notifier for iommu failed: %d\n", ret);
- ret = vfio_unregister_notifier(mdev_dev(vdev->mdev), VFIO_GROUP_NOTIFY,
+ ret = vfio_unregister_notifier(vdev->vfio_device.dev, VFIO_GROUP_NOTIFY,
&vdev->group_notifier);
drm_WARN(&i915->drm, ret,
"vfio_unregister_notifier for group failed: %d\n", ret);
@@ -907,11 +901,12 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu)
vgpu->handle = 0;
}
-static void intel_vgpu_release(struct mdev_device *mdev)
+static void intel_vgpu_release(struct vfio_device *vfio_dev)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
- __intel_vgpu_release(vgpu);
+ __intel_vgpu_release(vdev->vgpu);
}
static void intel_vgpu_release_work(struct work_struct *work)
@@ -997,11 +992,10 @@ static int intel_vgpu_aperture_rw(struct intel_vgpu *vgpu, u64 off,
return 0;
}
-static ssize_t intel_vgpu_rw(struct mdev_device *mdev, char *buf,
+static ssize_t intel_vgpu_rw(struct kvmgt_vdev *vdev, char *buf,
size_t count, loff_t *ppos, bool is_write)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
- struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
u64 pos = *ppos & VFIO_PCI_OFFSET_MASK;
int ret = -EINVAL;
@@ -1047,9 +1041,9 @@ static ssize_t intel_vgpu_rw(struct mdev_device *mdev, char *buf,
return ret == 0 ? count : ret;
}
-static bool gtt_entry(struct mdev_device *mdev, loff_t *ppos)
+static bool gtt_entry(struct kvmgt_vdev *vdev, loff_t *ppos)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
struct intel_gvt *gvt = vgpu->gvt;
int offset;
@@ -1066,9 +1060,11 @@ static bool gtt_entry(struct mdev_device *mdev, loff_t *ppos)
true : false;
}
-static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
+static ssize_t intel_vgpu_read(struct vfio_device *vfio_dev, char __user *buf,
size_t count, loff_t *ppos)
{
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
unsigned int done = 0;
int ret;
@@ -1077,10 +1073,10 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
/* Only support GGTT entry 8 bytes read */
if (count >= 8 && !(*ppos % 8) &&
- gtt_entry(mdev, ppos)) {
+ gtt_entry(vdev, ppos)) {
u64 val;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, false);
if (ret <= 0)
goto read_err;
@@ -1092,7 +1088,7 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
} else if (count >= 4 && !(*ppos % 4)) {
u32 val;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, false);
if (ret <= 0)
goto read_err;
@@ -1104,7 +1100,7 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
} else if (count >= 2 && !(*ppos % 2)) {
u16 val;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, false);
if (ret <= 0)
goto read_err;
@@ -1116,7 +1112,7 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
} else {
u8 val;
- ret = intel_vgpu_rw(mdev, &val, sizeof(val), ppos,
+ ret = intel_vgpu_rw(vdev, &val, sizeof(val), ppos,
false);
if (ret <= 0)
goto read_err;
@@ -1139,10 +1135,12 @@ static ssize_t intel_vgpu_read(struct mdev_device *mdev, char __user *buf,
return -EFAULT;
}
-static ssize_t intel_vgpu_write(struct mdev_device *mdev,
+static ssize_t intel_vgpu_write(struct vfio_device *vfio_dev,
const char __user *buf,
size_t count, loff_t *ppos)
{
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
unsigned int done = 0;
int ret;
@@ -1151,13 +1149,13 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
/* Only support GGTT entry 8 bytes write */
if (count >= 8 && !(*ppos % 8) &&
- gtt_entry(mdev, ppos)) {
+ gtt_entry(vdev, ppos)) {
u64 val;
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, true);
if (ret <= 0)
goto write_err;
@@ -1169,7 +1167,7 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, (char *)&val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, (char *)&val, sizeof(val),
ppos, true);
if (ret <= 0)
goto write_err;
@@ -1181,7 +1179,7 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, (char *)&val,
+ ret = intel_vgpu_rw(vdev, (char *)&val,
sizeof(val), ppos, true);
if (ret <= 0)
goto write_err;
@@ -1193,7 +1191,7 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
if (copy_from_user(&val, buf, sizeof(val)))
goto write_err;
- ret = intel_vgpu_rw(mdev, &val, sizeof(val),
+ ret = intel_vgpu_rw(vdev, &val, sizeof(val),
ppos, true);
if (ret <= 0)
goto write_err;
@@ -1212,13 +1210,16 @@ static ssize_t intel_vgpu_write(struct mdev_device *mdev,
return -EFAULT;
}
-static int intel_vgpu_mmap(struct mdev_device *mdev, struct vm_area_struct *vma)
+static int intel_vgpu_mmap(struct vfio_device *vfio_dev,
+ struct vm_area_struct *vma)
{
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
unsigned int index;
u64 virtaddr;
unsigned long req_size, pgoff, req_start;
pgprot_t pg_prot;
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
index = vma->vm_pgoff >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
if (index >= VFIO_PCI_ROM_REGION_INDEX)
@@ -1341,11 +1342,12 @@ static int intel_vgpu_set_irqs(struct intel_vgpu *vgpu, u32 flags,
return func(vgpu, index, start, count, flags, data);
}
-static long intel_vgpu_ioctl(struct mdev_device *mdev, unsigned int cmd,
+static long intel_vgpu_ioctl(struct vfio_device *vfio_dev, unsigned int cmd,
unsigned long arg)
{
- struct intel_vgpu *vgpu = mdev_get_drvdata(mdev);
- struct kvmgt_vdev *vdev = kvmgt_vdev(vgpu);
+ struct kvmgt_vdev *vdev =
+ container_of(vfio_dev, struct kvmgt_vdev, vfio_device);
+ struct intel_vgpu *vgpu = vdev->vgpu;
unsigned long minsz;
gvt_dbg_core("vgpu%d ioctl, cmd: %d\n", vgpu->id, cmd);
@@ -1624,14 +1626,10 @@ static ssize_t
vgpu_id_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct mdev_device *mdev = mdev_from_dev(dev);
+ struct kvmgt_vdev *vdev = dev_get_drvdata(dev);
+ struct intel_vgpu *vgpu = vdev->vgpu;
- if (mdev) {
- struct intel_vgpu *vgpu = (struct intel_vgpu *)
- mdev_get_drvdata(mdev);
- return sprintf(buf, "%d\n", vgpu->id);
- }
- return sprintf(buf, "\n");
+ return sprintf(buf, "%d\n", vgpu->id);
}
static DEVICE_ATTR_RO(vgpu_id);
@@ -1651,18 +1649,28 @@ static const struct attribute_group *intel_vgpu_groups[] = {
NULL,
};
-static struct mdev_parent_ops intel_vgpu_ops = {
- .mdev_attr_groups = intel_vgpu_groups,
- .create = intel_vgpu_create,
- .remove = intel_vgpu_remove,
+static const struct vfio_device_ops intel_vgpu_dev_ops = {
+ .open = intel_vgpu_open,
+ .release = intel_vgpu_release,
+ .read = intel_vgpu_read,
+ .write = intel_vgpu_write,
+ .mmap = intel_vgpu_mmap,
+ .ioctl = intel_vgpu_ioctl,
+};
- .open = intel_vgpu_open,
- .release = intel_vgpu_release,
+static struct mdev_driver intel_vgpu_mdev_driver = {
+ .driver = {
+ .name = "intel_vgpu_mdev",
+ .owner = THIS_MODULE,
+ .mod_name = KBUILD_MODNAME,
+ .dev_groups = intel_vgpu_groups,
+ },
+ .probe = intel_vgpu_probe,
+ .remove = intel_vgpu_remove,
+};
- .read = intel_vgpu_read,
- .write = intel_vgpu_write,
- .mmap = intel_vgpu_mmap,
- .ioctl = intel_vgpu_ioctl,
+static struct mdev_parent_ops intel_vgpu_ops = {
+ .device_driver = &intel_vgpu_mdev_driver,
};
static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
@@ -1806,18 +1814,12 @@ static bool __kvmgt_vgpu_exist(struct intel_vgpu *vgpu, struct kvm *kvm)
return ret;
}
-static int kvmgt_guest_init(struct mdev_device *mdev)
+static int kvmgt_guest_init(struct kvmgt_vdev *vdev)
{
struct kvmgt_guest_info *info;
- struct intel_vgpu *vgpu;
- struct kvmgt_vdev *vdev;
+ struct intel_vgpu *vgpu = vdev->vgpu;
struct kvm *kvm;
- vgpu = mdev_get_drvdata(mdev);
- if (handle_valid(vgpu->handle))
- return -EEXIST;
-
- vdev = kvmgt_vdev(vgpu);
kvm = vdev->kvm;
if (!kvm || kvm->mm != current->mm) {
gvt_vgpu_err("KVM is required to use Intel vGPU\n");
@@ -2125,13 +2127,25 @@ static const struct intel_gvt_mpt kvmgt_mpt = {
static int __init kvmgt_init(void)
{
- if (intel_gvt_register_hypervisor(&kvmgt_mpt) < 0)
- return -ENODEV;
+ int ret;
+
+ ret = mdev_register_driver(&intel_vgpu_mdev_driver);
+ if (ret)
+ return ret;
+
+ if (intel_gvt_register_hypervisor(&kvmgt_mpt) < 0) {
+ ret = -ENODEV;
+ goto err_driver;
+ }
return 0;
+err_driver:
+ mdev_unregister_driver(&intel_vgpu_mdev_driver);
+ return ret;
}
static void __exit kvmgt_exit(void)
{
+ mdev_unregister_driver(&intel_vgpu_mdev_driver);
intel_gvt_unregister_hypervisor();
}
diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
index ff9ecd80212503..7c236ba1b90eb1 100644
--- a/drivers/vfio/mdev/Makefile
+++ b/drivers/vfio/mdev/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0-only
-mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
+mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
obj-$(CONFIG_VFIO_MDEV) += mdev.o
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index 51b8a9fcf866ad..f95d01b57fb168 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -89,17 +89,10 @@ void mdev_release_parent(struct kref *kref)
static void mdev_device_remove_common(struct mdev_device *mdev)
{
struct mdev_parent *parent = mdev->type->parent;
- int ret;
mdev_remove_sysfs_files(mdev);
device_del(&mdev->dev);
lockdep_assert_held(&parent->unreg_sem);
- if (parent->ops->remove) {
- ret = parent->ops->remove(mdev);
- if (ret)
- dev_err(&mdev->dev, "Remove failed: err=%d\n", ret);
- }
-
/* Balances with device_initialize() */
put_device(&mdev->dev);
}
@@ -131,17 +124,13 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
/* check for mandatory ops */
if (!ops || !ops->supported_type_groups)
return -EINVAL;
- if (!ops->device_driver && (!ops->create || !ops->remove))
+ if (!ops->device_driver)
return -EINVAL;
dev = get_device(dev);
if (!dev)
return -EINVAL;
- /* Not mandatory, but its absence could be a problem */
- if (!ops->request)
- dev_info(dev, "Driver cannot be asked to release device\n");
-
mutex_lock(&parent_list_lock);
/* Check for duplicate */
@@ -263,15 +252,12 @@ static void mdev_device_release(struct device *dev)
*/
static int mdev_bind_driver(struct mdev_device *mdev)
{
- struct mdev_driver *drv = mdev->type->parent->ops->device_driver;
int ret;
- if (!drv)
- drv = &vfio_mdev_driver;
-
while (1) {
device_lock(&mdev->dev);
- if (mdev->dev.driver == &drv->driver) {
+ if (mdev->dev.driver ==
+ &mdev->type->parent->ops->device_driver->driver) {
ret = 0;
goto out_unlock;
}
@@ -337,15 +323,9 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
goto out_put_device;
}
- if (parent->ops->create) {
- ret = parent->ops->create(mdev);
- if (ret)
- goto out_unlock;
- }
-
ret = device_add(&mdev->dev);
if (ret)
- goto out_remove;
+ goto out_unlock;
ret = mdev_bind_driver(mdev);
if (ret)
@@ -363,9 +343,6 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
out_del:
device_del(&mdev->dev);
-out_remove:
- if (parent->ops->remove)
- parent->ops->remove(mdev);
out_unlock:
up_read(&parent->unreg_sem);
out_put_device:
@@ -408,27 +385,13 @@ int mdev_device_remove(struct mdev_device *mdev)
static int __init mdev_init(void)
{
- int rc;
-
- rc = mdev_bus_register();
- if (rc)
- return rc;
- rc = mdev_register_driver(&vfio_mdev_driver);
- if (rc)
- goto err_bus;
- return 0;
-err_bus:
- mdev_bus_unregister();
- return rc;
+ return mdev_bus_register();
}
static void __exit mdev_exit(void)
{
- mdev_unregister_driver(&vfio_mdev_driver);
-
if (mdev_bus_compat_class)
class_compat_unregister(mdev_bus_compat_class);
-
mdev_bus_unregister();
}
diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
index 6e96c023d7823d..0012a9ee7cb0a4 100644
--- a/drivers/vfio/mdev/mdev_driver.c
+++ b/drivers/vfio/mdev/mdev_driver.c
@@ -74,15 +74,8 @@ static int mdev_remove(struct device *dev)
static int mdev_match(struct device *dev, struct device_driver *drv)
{
struct mdev_device *mdev = to_mdev_device(dev);
- struct mdev_driver *target = mdev->type->parent->ops->device_driver;
-
- /*
- * The ops specify the device driver to connect, fall back to the old
- * shim driver if the driver hasn't been converted.
- */
- if (!target)
- target = &vfio_mdev_driver;
- return drv == &target->driver;
+
+ return drv == &mdev->type->parent->ops->device_driver->driver;
}
struct bus_type mdev_bus_type = {
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
index 5461b67582289f..a656cfe0346c33 100644
--- a/drivers/vfio/mdev/mdev_private.h
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -37,8 +37,6 @@ struct mdev_type {
#define to_mdev_type(_kobj) \
container_of(_kobj, struct mdev_type, kobj)
-extern struct mdev_driver vfio_mdev_driver;
-
int parent_create_sysfs_files(struct mdev_parent *parent);
void parent_remove_sysfs_files(struct mdev_parent *parent);
diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
deleted file mode 100644
index d5b4eede47c1a5..00000000000000
--- a/drivers/vfio/mdev/vfio_mdev.c
+++ /dev/null
@@ -1,158 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * VFIO based driver for Mediated device
- *
- * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
- * Author: Neo Jia <cjia@nvidia.com>
- * Kirti Wankhede <kwankhede@nvidia.com>
- */
-
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/device.h>
-#include <linux/kernel.h>
-#include <linux/slab.h>
-#include <linux/vfio.h>
-#include <linux/mdev.h>
-
-#include "mdev_private.h"
-
-static int vfio_mdev_open(struct vfio_device *core_vdev)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- int ret;
-
- if (unlikely(!parent->ops->open))
- return -EINVAL;
-
- if (!try_module_get(THIS_MODULE))
- return -ENODEV;
-
- ret = parent->ops->open(mdev);
- if (ret)
- module_put(THIS_MODULE);
-
- return ret;
-}
-
-static void vfio_mdev_release(struct vfio_device *core_vdev)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (likely(parent->ops->release))
- parent->ops->release(mdev);
-
- module_put(THIS_MODULE);
-}
-
-static long vfio_mdev_unlocked_ioctl(struct vfio_device *core_vdev,
- unsigned int cmd, unsigned long arg)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->ioctl))
- return -EINVAL;
-
- return parent->ops->ioctl(mdev, cmd, arg);
-}
-
-static ssize_t vfio_mdev_read(struct vfio_device *core_vdev, char __user *buf,
- size_t count, loff_t *ppos)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->read))
- return -EINVAL;
-
- return parent->ops->read(mdev, buf, count, ppos);
-}
-
-static ssize_t vfio_mdev_write(struct vfio_device *core_vdev,
- const char __user *buf, size_t count,
- loff_t *ppos)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->write))
- return -EINVAL;
-
- return parent->ops->write(mdev, buf, count, ppos);
-}
-
-static int vfio_mdev_mmap(struct vfio_device *core_vdev,
- struct vm_area_struct *vma)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (unlikely(!parent->ops->mmap))
- return -EINVAL;
-
- return parent->ops->mmap(mdev, vma);
-}
-
-static void vfio_mdev_request(struct vfio_device *core_vdev, unsigned int count)
-{
- struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
- struct mdev_parent *parent = mdev->type->parent;
-
- if (parent->ops->request)
- parent->ops->request(mdev, count);
- else if (count == 0)
- dev_notice(mdev_dev(mdev),
- "No mdev vendor driver request callback support, blocked until released by user\n");
-}
-
-static const struct vfio_device_ops vfio_mdev_dev_ops = {
- .name = "vfio-mdev",
- .open = vfio_mdev_open,
- .release = vfio_mdev_release,
- .ioctl = vfio_mdev_unlocked_ioctl,
- .read = vfio_mdev_read,
- .write = vfio_mdev_write,
- .mmap = vfio_mdev_mmap,
- .request = vfio_mdev_request,
-};
-
-static int vfio_mdev_probe(struct mdev_device *mdev)
-{
- struct vfio_device *vdev;
- int ret;
-
- vdev = kzalloc(sizeof(*vdev), GFP_KERNEL);
- if (!vdev)
- return -ENOMEM;
-
- vfio_init_group_dev(vdev, &mdev->dev, &vfio_mdev_dev_ops);
- ret = vfio_register_group_dev(vdev);
- if (ret) {
- kfree(vdev);
- return ret;
- }
- dev_set_drvdata(&mdev->dev, vdev);
- return 0;
-}
-
-static void vfio_mdev_remove(struct mdev_device *mdev)
-{
- struct vfio_device *vdev = dev_get_drvdata(&mdev->dev);
-
- vfio_unregister_group_dev(vdev);
- kfree(vdev);
-}
-
-struct mdev_driver vfio_mdev_driver = {
- .driver = {
- .name = "vfio_mdev",
- .owner = THIS_MODULE,
- .mod_name = KBUILD_MODNAME,
- },
- .probe = vfio_mdev_probe,
- .remove = vfio_mdev_remove,
-};
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 5e631c359ef23c..59bbdf6634f934 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -747,7 +747,7 @@ void vfio_init_group_dev(struct vfio_device *device, struct device *dev,
device->dev = dev;
device->ops = ops;
}
-EXPORT_SYMBOL_GPL(vfio_init_group_dev);
+EXPORT_SYMBOL(vfio_init_group_dev);
int vfio_register_group_dev(struct vfio_device *device)
{
@@ -796,7 +796,7 @@ int vfio_register_group_dev(struct vfio_device *device)
return 0;
}
-EXPORT_SYMBOL_GPL(vfio_register_group_dev);
+EXPORT_SYMBOL(vfio_register_group_dev);
/**
* Get a reference to the vfio_device for a device. Even if the
@@ -927,7 +927,7 @@ void vfio_unregister_group_dev(struct vfio_device *device)
/* Matches the get in vfio_register_group_dev() */
vfio_group_put(group);
}
-EXPORT_SYMBOL_GPL(vfio_unregister_group_dev);
+EXPORT_SYMBOL(vfio_unregister_group_dev);
/**
* VFIO base fd, /dev/vfio/vfio
diff --git a/include/linux/mdev.h b/include/linux/mdev.h
index 49cc4f65120d57..ea48c401e4fa63 100644
--- a/include/linux/mdev.h
+++ b/include/linux/mdev.h
@@ -61,45 +61,6 @@ struct device *mtype_get_parent_dev(struct mdev_type *mtype);
* @mdev_attr_groups: Attributes of the mediated device.
* @supported_type_groups: Attributes to define supported types. It is mandatory
* to provide supported types.
- * @create: Called to allocate basic resources in parent device's
- * driver for a particular mediated device. It is
- * mandatory to provide create ops.
- * @mdev: mdev_device structure on of mediated device
- * that is being created
- * Returns integer: success (0) or error (< 0)
- * @remove: Called to free resources in parent device's driver for
- * a mediated device. It is mandatory to provide 'remove'
- * ops.
- * @mdev: mdev_device device structure which is being
- * destroyed
- * Returns integer: success (0) or error (< 0)
- * @open: Open mediated device.
- * @mdev: mediated device.
- * Returns integer: success (0) or error (< 0)
- * @release: release mediated device
- * @mdev: mediated device.
- * @read: Read emulation callback
- * @mdev: mediated device structure
- * @buf: read buffer
- * @count: number of bytes to read
- * @ppos: address.
- * Retuns number on bytes read on success or error.
- * @write: Write emulation callback
- * @mdev: mediated device structure
- * @buf: write buffer
- * @count: number of bytes to be written
- * @ppos: address.
- * Retuns number on bytes written on success or error.
- * @ioctl: IOCTL callback
- * @mdev: mediated device structure
- * @cmd: ioctl command
- * @arg: arguments to ioctl
- * @mmap: mmap callback
- * @mdev: mediated device structure
- * @vma: vma structure
- * @request: request callback to release device
- * @mdev: mediated device structure
- * @count: request sequence number
* Parent device that support mediated device should be registered with mdev
* module with mdev_parent_ops structure.
**/
@@ -109,19 +70,6 @@ struct mdev_parent_ops {
const struct attribute_group **dev_attr_groups;
const struct attribute_group **mdev_attr_groups;
struct attribute_group **supported_type_groups;
-
- int (*create)(struct mdev_device *mdev);
- int (*remove)(struct mdev_device *mdev);
- int (*open)(struct mdev_device *mdev);
- void (*release)(struct mdev_device *mdev);
- ssize_t (*read)(struct mdev_device *mdev, char __user *buf,
- size_t count, loff_t *ppos);
- ssize_t (*write)(struct mdev_device *mdev, const char __user *buf,
- size_t count, loff_t *ppos);
- long (*ioctl)(struct mdev_device *mdev, unsigned int cmd,
- unsigned long arg);
- int (*mmap)(struct mdev_device *mdev, struct vm_area_struct *vma);
- void (*request)(struct mdev_device *mdev, unsigned int count);
};
/* interface for exporting mdev supported type attributes */
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index a2c5b30e1763ba..c5e08be4c56395 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -64,6 +64,10 @@ void vfio_init_group_dev(struct vfio_device *device, struct device *dev,
int vfio_register_group_dev(struct vfio_device *device);
void vfio_unregister_group_dev(struct vfio_device *device);
extern struct vfio_device *vfio_device_get_from_dev(struct device *dev);
+static inline void vfio_device_get(struct vfio_device *device)
+{
+ refcount_inc(&device->refcount);
+}
extern void vfio_device_put(struct vfio_device *device);
/* events for the backend driver notify callback */
--
2.31.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 09/12] vfio/mdev: Remove mdev_parent_ops dev_attr_groups
2021-04-23 23:02 ` Jason Gunthorpe
` (9 preceding siblings ...)
(?)
@ 2021-04-23 23:03 ` Jason Gunthorpe
2021-04-26 14:14 ` Christoph Hellwig
-1 siblings, 1 reply; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: Alex Williamson, Cornelia Huck, kvm, Kirti Wankhede
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
This is only used by one sample to print a fixed string that is pointless.
In general, having a device driver attach sysfs attributes to the parent
is horrific. This should never happen, and always leads to some kind of
liftime bug as it become very difficult for the sysfs attribute to go back
to any data owned by the device driver.
Remove the general mechanism to create this abuse.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/vfio/mdev/mdev_sysfs.c | 12 ++----------
include/linux/mdev.h | 2 --
samples/vfio-mdev/mtty.c | 30 +-----------------------------
3 files changed, 3 insertions(+), 41 deletions(-)
diff --git a/drivers/vfio/mdev/mdev_sysfs.c b/drivers/vfio/mdev/mdev_sysfs.c
index f5cf1931c54e48..66eef08833a4ef 100644
--- a/drivers/vfio/mdev/mdev_sysfs.c
+++ b/drivers/vfio/mdev/mdev_sysfs.c
@@ -197,7 +197,6 @@ void parent_remove_sysfs_files(struct mdev_parent *parent)
remove_mdev_supported_type(type);
}
- sysfs_remove_groups(&parent->dev->kobj, parent->ops->dev_attr_groups);
kset_unregister(parent->mdev_types_kset);
}
@@ -213,17 +212,10 @@ int parent_create_sysfs_files(struct mdev_parent *parent)
INIT_LIST_HEAD(&parent->type_list);
- ret = sysfs_create_groups(&parent->dev->kobj,
- parent->ops->dev_attr_groups);
- if (ret)
- goto create_err;
-
ret = add_mdev_supported_type_groups(parent);
if (ret)
- sysfs_remove_groups(&parent->dev->kobj,
- parent->ops->dev_attr_groups);
- else
- return ret;
+ goto create_err;
+ return 0;
create_err:
kset_unregister(parent->mdev_types_kset);
diff --git a/include/linux/mdev.h b/include/linux/mdev.h
index ea48c401e4fa63..fd9fe1dcf0e230 100644
--- a/include/linux/mdev.h
+++ b/include/linux/mdev.h
@@ -57,7 +57,6 @@ struct device *mtype_get_parent_dev(struct mdev_type *mtype);
*
* @owner: The module owner.
* @device_driver: Which device driver to probe() on newly created devices
- * @dev_attr_groups: Attributes of the parent device.
* @mdev_attr_groups: Attributes of the mediated device.
* @supported_type_groups: Attributes to define supported types. It is mandatory
* to provide supported types.
@@ -67,7 +66,6 @@ struct device *mtype_get_parent_dev(struct mdev_type *mtype);
struct mdev_parent_ops {
struct module *owner;
struct mdev_driver *device_driver;
- const struct attribute_group **dev_attr_groups;
const struct attribute_group **mdev_attr_groups;
struct attribute_group **supported_type_groups;
};
diff --git a/samples/vfio-mdev/mtty.c b/samples/vfio-mdev/mtty.c
index d2a168420b775d..31eec76bc553ce 100644
--- a/samples/vfio-mdev/mtty.c
+++ b/samples/vfio-mdev/mtty.c
@@ -1207,38 +1207,11 @@ static void mtty_close(struct vfio_device *mdev)
pr_info("%s\n", __func__);
}
-static ssize_t
-sample_mtty_dev_show(struct device *dev, struct device_attribute *attr,
- char *buf)
-{
- return sprintf(buf, "This is phy device\n");
-}
-
-static DEVICE_ATTR_RO(sample_mtty_dev);
-
-static struct attribute *mtty_dev_attrs[] = {
- &dev_attr_sample_mtty_dev.attr,
- NULL,
-};
-
-static const struct attribute_group mtty_dev_group = {
- .name = "mtty_dev",
- .attrs = mtty_dev_attrs,
-};
-
-static const struct attribute_group *mtty_dev_groups[] = {
- &mtty_dev_group,
- NULL,
-};
-
static ssize_t
sample_mdev_dev_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- if (mdev_from_dev(dev))
- return sprintf(buf, "This is MDEV %s\n", dev_name(dev));
-
- return sprintf(buf, "\n");
+ return sprintf(buf, "This is MDEV %s\n", dev_name(dev));
}
static DEVICE_ATTR_RO(sample_mdev_dev);
@@ -1340,7 +1313,6 @@ static struct mdev_driver mtty_driver = {
static const struct mdev_parent_ops mdev_fops = {
.owner = THIS_MODULE,
.device_driver = &mtty_driver,
- .dev_attr_groups = mtty_dev_groups,
.supported_type_groups = mdev_type_groups,
};
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 10/12] vfio/mdev: Remove mdev_parent_ops
2021-04-23 23:02 ` Jason Gunthorpe
(?)
@ 2021-04-23 23:03 ` Jason Gunthorpe
-1 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc,
linux-s390, Peter Oberparleiter, Halil Pasic, Pierre Morel,
Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Raj, Ashok, Dan Williams, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
The last useful member in this struct is the supported_type_groups, move
it to the mdev_driver and delete mdev_parent_ops.
Replace it with mdev_driver as an argument to mdev_register_device()
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
.../driver-api/vfio-mediated-device.rst | 36 +++++++------------
drivers/gpu/drm/i915/gvt/kvmgt.c | 8 ++---
drivers/s390/cio/vfio_ccw_ops.c | 7 +---
drivers/s390/crypto/vfio_ap_ops.c | 9 ++---
drivers/vfio/mdev/mdev_core.c | 13 +++----
drivers/vfio/mdev/mdev_driver.c | 2 +-
drivers/vfio/mdev/mdev_private.h | 2 +-
drivers/vfio/mdev/mdev_sysfs.c | 6 ++--
include/linux/mdev.h | 24 +++----------
samples/vfio-mdev/mbochs.c | 9 ++---
samples/vfio-mdev/mdpy.c | 9 ++---
samples/vfio-mdev/mtty.c | 9 ++---
12 files changed, 38 insertions(+), 96 deletions(-)
diff --git a/Documentation/driver-api/vfio-mediated-device.rst b/Documentation/driver-api/vfio-mediated-device.rst
index 5f866b17c93e69..b7cf357243d269 100644
--- a/Documentation/driver-api/vfio-mediated-device.rst
+++ b/Documentation/driver-api/vfio-mediated-device.rst
@@ -93,7 +93,7 @@ interfaces:
Registration Interface for a Mediated Bus Driver
------------------------------------------------
-The registration interface for a mediated bus driver provides the following
+The registration interface for a mediated device driver provides the following
structure to represent a mediated device's driver::
/*
@@ -105,6 +105,7 @@ structure to represent a mediated device's driver::
struct mdev_driver {
int (*probe) (struct mdev_device *dev);
void (*remove) (struct mdev_device *dev);
+ struct attribute_group **supported_type_groups;
struct device_driver driver;
};
@@ -119,35 +120,24 @@ to register and unregister itself with the core driver:
extern void mdev_unregister_driver(struct mdev_driver *drv);
-The mediated bus driver is responsible for adding mediated devices to the VFIO
-group when devices are bound to the driver and removing mediated devices from
-the VFIO when devices are unbound from the driver.
+The mediated bus driver's probe function should create a vfio_device on top of
+the mdev_device and connect it to an appropriate implementation of vfio_device_ops.
-
-Physical Device Driver Interface
---------------------------------
-
-The physical device driver interface provides the mdev_parent_ops[3] structure
-to define the APIs to manage work in the mediated core driver that is related
-to the physical device.
-
-The structures in the mdev_parent_ops structure are as follows:
-
-* dev_attr_groups: attributes of the parent device
-* mdev_attr_groups: attributes of the mediated device
-* supported_config: attributes to define supported configurations
-
-A driver should use the mdev_parent_ops structure in the function call to
-register itself with the mdev core driver::
+When a driver wants to add the GUID creation sysfs to an existing device it has
+probe'd to then it should call:
extern int mdev_register_device(struct device *dev,
- const struct mdev_parent_ops *ops);
+ struct mdev_driver *mdev_driver);
+
+This will provide the 'mdev_supported_types/XX/create' files which can then be used
+to trigger the creation of a mdev_device. The created mdev_device will be attached
+to the specified driver.
-However, the mdev_parent_ops structure is not required in the function call
-that a driver should use to unregister itself with the mdev core driver::
+When the driver needs to remove itself it calls:
extern void mdev_unregister_device(struct device *dev);
+Which will unbind and destroy all the created mdevs and remove the sysfs files.
Mediated Device Management Interface Through sysfs
==================================================
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 85ef300087e091..02089efd15bb92 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1669,10 +1669,6 @@ static struct mdev_driver intel_vgpu_mdev_driver = {
.remove = intel_vgpu_remove,
};
-static struct mdev_parent_ops intel_vgpu_ops = {
- .device_driver = &intel_vgpu_mdev_driver,
-};
-
static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
{
struct attribute_group **kvm_vgpu_type_groups;
@@ -1680,9 +1676,9 @@ static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
intel_gvt_ops = ops;
if (!intel_gvt_ops->get_gvt_attrs(&kvm_vgpu_type_groups))
return -EFAULT;
- intel_vgpu_ops.supported_type_groups = kvm_vgpu_type_groups;
+ intel_vgpu_mdev_driver.supported_type_groups = kvm_vgpu_type_groups;
- return mdev_register_device(dev, &intel_vgpu_ops);
+ return mdev_register_device(dev, &intel_vgpu_mdev_driver);
}
static void kvmgt_host_exit(struct device *dev)
diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c
index 0fcf46031d3821..161697529dcc41 100644
--- a/drivers/s390/cio/vfio_ccw_ops.c
+++ b/drivers/s390/cio/vfio_ccw_ops.c
@@ -655,17 +655,12 @@ struct mdev_driver vfio_ccw_mdev_driver = {
},
.probe = vfio_ccw_mdev_probe,
.remove = vfio_ccw_mdev_remove,
-};
-
-static const struct mdev_parent_ops vfio_ccw_mdev_ops = {
- .owner = THIS_MODULE,
- .device_driver = &vfio_ccw_mdev_driver,
.supported_type_groups = mdev_type_groups,
};
int vfio_ccw_mdev_reg(struct subchannel *sch)
{
- return mdev_register_device(&sch->dev, &vfio_ccw_mdev_ops);
+ return mdev_register_device(&sch->dev, &vfio_ccw_mdev_driver);
}
void vfio_ccw_mdev_unreg(struct subchannel *sch)
diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
index 79872c857dd522..92789257c87639 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -1339,12 +1339,7 @@ static struct mdev_driver vfio_ap_matrix_driver = {
},
.probe = vfio_ap_mdev_probe,
.remove = vfio_ap_mdev_remove,
-};
-
-static const struct mdev_parent_ops vfio_ap_matrix_ops = {
- .owner = THIS_MODULE,
- .device_driver = &vfio_ap_matrix_driver,
- .supported_type_groups = vfio_ap_mdev_type_groups,
+ .supported_type_groups = vfio_ap_mdev_type_groups,
};
int vfio_ap_mdev_register(void)
@@ -1357,7 +1352,7 @@ int vfio_ap_mdev_register(void)
if (ret)
return ret;
- ret = mdev_register_device(&matrix_dev->device, &vfio_ap_matrix_ops);
+ ret = mdev_register_device(&matrix_dev->device, &vfio_ap_matrix_driver);
if (ret)
goto err_driver;
return 0;
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index f95d01b57fb168..7e918241de10cc 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -109,12 +109,12 @@ static int mdev_device_remove_cb(struct device *dev, void *data)
/*
* mdev_register_device : Register a device
* @dev: device structure representing parent device.
- * @ops: Parent device operation structure to be registered.
+ * @mdev_driver: Device driver to bind to the newly created mdev
*
* Add device to list of registered parent devices.
* Returns a negative value on error, otherwise 0.
*/
-int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
+int mdev_register_device(struct device *dev, struct mdev_driver *mdev_driver)
{
int ret;
struct mdev_parent *parent;
@@ -122,9 +122,7 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
char *envp[] = { env_string, NULL };
/* check for mandatory ops */
- if (!ops || !ops->supported_type_groups)
- return -EINVAL;
- if (!ops->device_driver)
+ if (!mdev_driver->supported_type_groups)
return -EINVAL;
dev = get_device(dev);
@@ -151,7 +149,7 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
init_rwsem(&parent->unreg_sem);
parent->dev = dev;
- parent->ops = ops;
+ parent->mdev_driver = mdev_driver;
if (!mdev_bus_compat_class) {
mdev_bus_compat_class = class_compat_register("mdev_bus");
@@ -257,7 +255,7 @@ static int mdev_bind_driver(struct mdev_device *mdev)
while (1) {
device_lock(&mdev->dev);
if (mdev->dev.driver ==
- &mdev->type->parent->ops->device_driver->driver) {
+ &mdev->type->parent->mdev_driver->driver) {
ret = 0;
goto out_unlock;
}
@@ -304,7 +302,6 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
mdev->dev.parent = parent->dev;
mdev->dev.bus = &mdev_bus_type;
mdev->dev.release = mdev_device_release;
- mdev->dev.groups = parent->ops->mdev_attr_groups;
mdev->type = type;
/* Pairs with the put in mdev_device_release() */
kobject_get(&type->kobj);
diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
index 0012a9ee7cb0a4..12091e32afa396 100644
--- a/drivers/vfio/mdev/mdev_driver.c
+++ b/drivers/vfio/mdev/mdev_driver.c
@@ -75,7 +75,7 @@ static int mdev_match(struct device *dev, struct device_driver *drv)
{
struct mdev_device *mdev = to_mdev_device(dev);
- return drv == &mdev->type->parent->ops->device_driver->driver;
+ return drv == &mdev->type->parent->mdev_driver->driver;
}
struct bus_type mdev_bus_type = {
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
index a656cfe0346c33..839567d059a07d 100644
--- a/drivers/vfio/mdev/mdev_private.h
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -15,7 +15,7 @@ void mdev_bus_unregister(void);
struct mdev_parent {
struct device *dev;
- const struct mdev_parent_ops *ops;
+ const struct mdev_driver *mdev_driver;
struct kref ref;
struct list_head next;
struct kset *mdev_types_kset;
diff --git a/drivers/vfio/mdev/mdev_sysfs.c b/drivers/vfio/mdev/mdev_sysfs.c
index 66eef08833a4ef..5a3873d1a275ae 100644
--- a/drivers/vfio/mdev/mdev_sysfs.c
+++ b/drivers/vfio/mdev/mdev_sysfs.c
@@ -97,7 +97,7 @@ static struct mdev_type *add_mdev_supported_type(struct mdev_parent *parent,
{
struct mdev_type *type;
struct attribute_group *group =
- parent->ops->supported_type_groups[type_group_id];
+ parent->mdev_driver->supported_type_groups[type_group_id];
int ret;
if (!group->name) {
@@ -154,7 +154,7 @@ static struct mdev_type *add_mdev_supported_type(struct mdev_parent *parent,
static void remove_mdev_supported_type(struct mdev_type *type)
{
struct attribute_group *group =
- type->parent->ops->supported_type_groups[type->type_group_id];
+ type->parent->mdev_driver->supported_type_groups[type->type_group_id];
sysfs_remove_files(&type->kobj,
(const struct attribute **)group->attrs);
@@ -168,7 +168,7 @@ static int add_mdev_supported_type_groups(struct mdev_parent *parent)
{
int i;
- for (i = 0; parent->ops->supported_type_groups[i]; i++) {
+ for (i = 0; parent->mdev_driver->supported_type_groups[i]; i++) {
struct mdev_type *type;
type = add_mdev_supported_type(parent, i);
diff --git a/include/linux/mdev.h b/include/linux/mdev.h
index fd9fe1dcf0e230..af807c77c1e0f5 100644
--- a/include/linux/mdev.h
+++ b/include/linux/mdev.h
@@ -51,25 +51,6 @@ unsigned int mdev_get_type_group_id(struct mdev_device *mdev);
unsigned int mtype_get_type_group_id(struct mdev_type *mtype);
struct device *mtype_get_parent_dev(struct mdev_type *mtype);
-/**
- * struct mdev_parent_ops - Structure to be registered for each parent device to
- * register the device to mdev module.
- *
- * @owner: The module owner.
- * @device_driver: Which device driver to probe() on newly created devices
- * @mdev_attr_groups: Attributes of the mediated device.
- * @supported_type_groups: Attributes to define supported types. It is mandatory
- * to provide supported types.
- * Parent device that support mediated device should be registered with mdev
- * module with mdev_parent_ops structure.
- **/
-struct mdev_parent_ops {
- struct module *owner;
- struct mdev_driver *device_driver;
- const struct attribute_group **mdev_attr_groups;
- struct attribute_group **supported_type_groups;
-};
-
/* interface for exporting mdev supported type attributes */
struct mdev_type_attribute {
struct attribute attr;
@@ -94,12 +75,15 @@ struct mdev_type_attribute mdev_type_attr_##_name = \
* struct mdev_driver - Mediated device driver
* @probe: called when new device created
* @remove: called when device removed
+ * @supported_type_groups: Attributes to define supported types. It is mandatory
+ * to provide supported types.
* @driver: device driver structure
*
**/
struct mdev_driver {
int (*probe)(struct mdev_device *dev);
void (*remove)(struct mdev_device *dev);
+ struct attribute_group **supported_type_groups;
struct device_driver driver;
};
@@ -118,7 +102,7 @@ static inline const guid_t *mdev_uuid(struct mdev_device *mdev)
extern struct bus_type mdev_bus_type;
-int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops);
+int mdev_register_device(struct device *dev, struct mdev_driver *mdev_driver);
void mdev_unregister_device(struct device *dev);
int mdev_register_driver(struct mdev_driver *drv);
diff --git a/samples/vfio-mdev/mbochs.c b/samples/vfio-mdev/mbochs.c
index e18821a8a6beb8..c76ceec584b41b 100644
--- a/samples/vfio-mdev/mbochs.c
+++ b/samples/vfio-mdev/mbochs.c
@@ -1418,12 +1418,7 @@ static struct mdev_driver mbochs_driver = {
},
.probe = mbochs_probe,
.remove = mbochs_remove,
-};
-
-static const struct mdev_parent_ops mdev_fops = {
- .owner = THIS_MODULE,
- .device_driver = &mbochs_driver,
- .supported_type_groups = mdev_type_groups,
+ .supported_type_groups = mdev_type_groups,
};
static const struct file_operations vd_fops = {
@@ -1466,7 +1461,7 @@ static int __init mbochs_dev_init(void)
if (ret)
goto err_class;
- ret = mdev_register_device(&mbochs_dev, &mdev_fops);
+ ret = mdev_register_device(&mbochs_dev, &mbochs_driver);
if (ret)
goto err_device;
diff --git a/samples/vfio-mdev/mdpy.c b/samples/vfio-mdev/mdpy.c
index 82638de333330d..c22b2c808d132d 100644
--- a/samples/vfio-mdev/mdpy.c
+++ b/samples/vfio-mdev/mdpy.c
@@ -735,12 +735,7 @@ static struct mdev_driver mdpy_driver = {
},
.probe = mdpy_probe,
.remove = mdpy_remove,
-};
-
-static const struct mdev_parent_ops mdev_fops = {
- .owner = THIS_MODULE,
- .device_driver = &mdpy_driver,
- .supported_type_groups = mdev_type_groups,
+ .supported_type_groups = mdev_type_groups,
};
static const struct file_operations vd_fops = {
@@ -783,7 +778,7 @@ static int __init mdpy_dev_init(void)
if (ret)
goto err_class;
- ret = mdev_register_device(&mdpy_dev, &mdev_fops);
+ ret = mdev_register_device(&mdpy_dev, &mdpy_driver);
if (ret)
goto err_device;
diff --git a/samples/vfio-mdev/mtty.c b/samples/vfio-mdev/mtty.c
index 31eec76bc553ce..87f5ba12a230e3 100644
--- a/samples/vfio-mdev/mtty.c
+++ b/samples/vfio-mdev/mtty.c
@@ -1308,12 +1308,7 @@ static struct mdev_driver mtty_driver = {
},
.probe = mtty_probe,
.remove = mtty_remove,
-};
-
-static const struct mdev_parent_ops mdev_fops = {
- .owner = THIS_MODULE,
- .device_driver = &mtty_driver,
- .supported_type_groups = mdev_type_groups,
+ .supported_type_groups = mdev_type_groups,
};
static void mtty_device_release(struct device *dev)
@@ -1364,7 +1359,7 @@ static int __init mtty_dev_init(void)
if (ret)
goto err_class;
- ret = mdev_register_device(&mtty_dev.dev, &mdev_fops);
+ ret = mdev_register_device(&mtty_dev.dev, &mtty_driver);
if (ret)
goto err_device;
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 10/12] vfio/mdev: Remove mdev_parent_ops
@ 2021-04-23 23:03 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc,
linux-s390, Peter Oberparleiter, Halil Pasic, Pierre Morel,
Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
The last useful member in this struct is the supported_type_groups, move
it to the mdev_driver and delete mdev_parent_ops.
Replace it with mdev_driver as an argument to mdev_register_device()
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
.../driver-api/vfio-mediated-device.rst | 36 +++++++------------
drivers/gpu/drm/i915/gvt/kvmgt.c | 8 ++---
drivers/s390/cio/vfio_ccw_ops.c | 7 +---
drivers/s390/crypto/vfio_ap_ops.c | 9 ++---
drivers/vfio/mdev/mdev_core.c | 13 +++----
drivers/vfio/mdev/mdev_driver.c | 2 +-
drivers/vfio/mdev/mdev_private.h | 2 +-
drivers/vfio/mdev/mdev_sysfs.c | 6 ++--
include/linux/mdev.h | 24 +++----------
samples/vfio-mdev/mbochs.c | 9 ++---
samples/vfio-mdev/mdpy.c | 9 ++---
samples/vfio-mdev/mtty.c | 9 ++---
12 files changed, 38 insertions(+), 96 deletions(-)
diff --git a/Documentation/driver-api/vfio-mediated-device.rst b/Documentation/driver-api/vfio-mediated-device.rst
index 5f866b17c93e69..b7cf357243d269 100644
--- a/Documentation/driver-api/vfio-mediated-device.rst
+++ b/Documentation/driver-api/vfio-mediated-device.rst
@@ -93,7 +93,7 @@ interfaces:
Registration Interface for a Mediated Bus Driver
------------------------------------------------
-The registration interface for a mediated bus driver provides the following
+The registration interface for a mediated device driver provides the following
structure to represent a mediated device's driver::
/*
@@ -105,6 +105,7 @@ structure to represent a mediated device's driver::
struct mdev_driver {
int (*probe) (struct mdev_device *dev);
void (*remove) (struct mdev_device *dev);
+ struct attribute_group **supported_type_groups;
struct device_driver driver;
};
@@ -119,35 +120,24 @@ to register and unregister itself with the core driver:
extern void mdev_unregister_driver(struct mdev_driver *drv);
-The mediated bus driver is responsible for adding mediated devices to the VFIO
-group when devices are bound to the driver and removing mediated devices from
-the VFIO when devices are unbound from the driver.
+The mediated bus driver's probe function should create a vfio_device on top of
+the mdev_device and connect it to an appropriate implementation of vfio_device_ops.
-
-Physical Device Driver Interface
---------------------------------
-
-The physical device driver interface provides the mdev_parent_ops[3] structure
-to define the APIs to manage work in the mediated core driver that is related
-to the physical device.
-
-The structures in the mdev_parent_ops structure are as follows:
-
-* dev_attr_groups: attributes of the parent device
-* mdev_attr_groups: attributes of the mediated device
-* supported_config: attributes to define supported configurations
-
-A driver should use the mdev_parent_ops structure in the function call to
-register itself with the mdev core driver::
+When a driver wants to add the GUID creation sysfs to an existing device it has
+probe'd to then it should call:
extern int mdev_register_device(struct device *dev,
- const struct mdev_parent_ops *ops);
+ struct mdev_driver *mdev_driver);
+
+This will provide the 'mdev_supported_types/XX/create' files which can then be used
+to trigger the creation of a mdev_device. The created mdev_device will be attached
+to the specified driver.
-However, the mdev_parent_ops structure is not required in the function call
-that a driver should use to unregister itself with the mdev core driver::
+When the driver needs to remove itself it calls:
extern void mdev_unregister_device(struct device *dev);
+Which will unbind and destroy all the created mdevs and remove the sysfs files.
Mediated Device Management Interface Through sysfs
==================================================
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 85ef300087e091..02089efd15bb92 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1669,10 +1669,6 @@ static struct mdev_driver intel_vgpu_mdev_driver = {
.remove = intel_vgpu_remove,
};
-static struct mdev_parent_ops intel_vgpu_ops = {
- .device_driver = &intel_vgpu_mdev_driver,
-};
-
static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
{
struct attribute_group **kvm_vgpu_type_groups;
@@ -1680,9 +1676,9 @@ static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
intel_gvt_ops = ops;
if (!intel_gvt_ops->get_gvt_attrs(&kvm_vgpu_type_groups))
return -EFAULT;
- intel_vgpu_ops.supported_type_groups = kvm_vgpu_type_groups;
+ intel_vgpu_mdev_driver.supported_type_groups = kvm_vgpu_type_groups;
- return mdev_register_device(dev, &intel_vgpu_ops);
+ return mdev_register_device(dev, &intel_vgpu_mdev_driver);
}
static void kvmgt_host_exit(struct device *dev)
diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c
index 0fcf46031d3821..161697529dcc41 100644
--- a/drivers/s390/cio/vfio_ccw_ops.c
+++ b/drivers/s390/cio/vfio_ccw_ops.c
@@ -655,17 +655,12 @@ struct mdev_driver vfio_ccw_mdev_driver = {
},
.probe = vfio_ccw_mdev_probe,
.remove = vfio_ccw_mdev_remove,
-};
-
-static const struct mdev_parent_ops vfio_ccw_mdev_ops = {
- .owner = THIS_MODULE,
- .device_driver = &vfio_ccw_mdev_driver,
.supported_type_groups = mdev_type_groups,
};
int vfio_ccw_mdev_reg(struct subchannel *sch)
{
- return mdev_register_device(&sch->dev, &vfio_ccw_mdev_ops);
+ return mdev_register_device(&sch->dev, &vfio_ccw_mdev_driver);
}
void vfio_ccw_mdev_unreg(struct subchannel *sch)
diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
index 79872c857dd522..92789257c87639 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -1339,12 +1339,7 @@ static struct mdev_driver vfio_ap_matrix_driver = {
},
.probe = vfio_ap_mdev_probe,
.remove = vfio_ap_mdev_remove,
-};
-
-static const struct mdev_parent_ops vfio_ap_matrix_ops = {
- .owner = THIS_MODULE,
- .device_driver = &vfio_ap_matrix_driver,
- .supported_type_groups = vfio_ap_mdev_type_groups,
+ .supported_type_groups = vfio_ap_mdev_type_groups,
};
int vfio_ap_mdev_register(void)
@@ -1357,7 +1352,7 @@ int vfio_ap_mdev_register(void)
if (ret)
return ret;
- ret = mdev_register_device(&matrix_dev->device, &vfio_ap_matrix_ops);
+ ret = mdev_register_device(&matrix_dev->device, &vfio_ap_matrix_driver);
if (ret)
goto err_driver;
return 0;
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index f95d01b57fb168..7e918241de10cc 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -109,12 +109,12 @@ static int mdev_device_remove_cb(struct device *dev, void *data)
/*
* mdev_register_device : Register a device
* @dev: device structure representing parent device.
- * @ops: Parent device operation structure to be registered.
+ * @mdev_driver: Device driver to bind to the newly created mdev
*
* Add device to list of registered parent devices.
* Returns a negative value on error, otherwise 0.
*/
-int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
+int mdev_register_device(struct device *dev, struct mdev_driver *mdev_driver)
{
int ret;
struct mdev_parent *parent;
@@ -122,9 +122,7 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
char *envp[] = { env_string, NULL };
/* check for mandatory ops */
- if (!ops || !ops->supported_type_groups)
- return -EINVAL;
- if (!ops->device_driver)
+ if (!mdev_driver->supported_type_groups)
return -EINVAL;
dev = get_device(dev);
@@ -151,7 +149,7 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
init_rwsem(&parent->unreg_sem);
parent->dev = dev;
- parent->ops = ops;
+ parent->mdev_driver = mdev_driver;
if (!mdev_bus_compat_class) {
mdev_bus_compat_class = class_compat_register("mdev_bus");
@@ -257,7 +255,7 @@ static int mdev_bind_driver(struct mdev_device *mdev)
while (1) {
device_lock(&mdev->dev);
if (mdev->dev.driver ==
- &mdev->type->parent->ops->device_driver->driver) {
+ &mdev->type->parent->mdev_driver->driver) {
ret = 0;
goto out_unlock;
}
@@ -304,7 +302,6 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
mdev->dev.parent = parent->dev;
mdev->dev.bus = &mdev_bus_type;
mdev->dev.release = mdev_device_release;
- mdev->dev.groups = parent->ops->mdev_attr_groups;
mdev->type = type;
/* Pairs with the put in mdev_device_release() */
kobject_get(&type->kobj);
diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
index 0012a9ee7cb0a4..12091e32afa396 100644
--- a/drivers/vfio/mdev/mdev_driver.c
+++ b/drivers/vfio/mdev/mdev_driver.c
@@ -75,7 +75,7 @@ static int mdev_match(struct device *dev, struct device_driver *drv)
{
struct mdev_device *mdev = to_mdev_device(dev);
- return drv == &mdev->type->parent->ops->device_driver->driver;
+ return drv == &mdev->type->parent->mdev_driver->driver;
}
struct bus_type mdev_bus_type = {
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
index a656cfe0346c33..839567d059a07d 100644
--- a/drivers/vfio/mdev/mdev_private.h
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -15,7 +15,7 @@ void mdev_bus_unregister(void);
struct mdev_parent {
struct device *dev;
- const struct mdev_parent_ops *ops;
+ const struct mdev_driver *mdev_driver;
struct kref ref;
struct list_head next;
struct kset *mdev_types_kset;
diff --git a/drivers/vfio/mdev/mdev_sysfs.c b/drivers/vfio/mdev/mdev_sysfs.c
index 66eef08833a4ef..5a3873d1a275ae 100644
--- a/drivers/vfio/mdev/mdev_sysfs.c
+++ b/drivers/vfio/mdev/mdev_sysfs.c
@@ -97,7 +97,7 @@ static struct mdev_type *add_mdev_supported_type(struct mdev_parent *parent,
{
struct mdev_type *type;
struct attribute_group *group =
- parent->ops->supported_type_groups[type_group_id];
+ parent->mdev_driver->supported_type_groups[type_group_id];
int ret;
if (!group->name) {
@@ -154,7 +154,7 @@ static struct mdev_type *add_mdev_supported_type(struct mdev_parent *parent,
static void remove_mdev_supported_type(struct mdev_type *type)
{
struct attribute_group *group =
- type->parent->ops->supported_type_groups[type->type_group_id];
+ type->parent->mdev_driver->supported_type_groups[type->type_group_id];
sysfs_remove_files(&type->kobj,
(const struct attribute **)group->attrs);
@@ -168,7 +168,7 @@ static int add_mdev_supported_type_groups(struct mdev_parent *parent)
{
int i;
- for (i = 0; parent->ops->supported_type_groups[i]; i++) {
+ for (i = 0; parent->mdev_driver->supported_type_groups[i]; i++) {
struct mdev_type *type;
type = add_mdev_supported_type(parent, i);
diff --git a/include/linux/mdev.h b/include/linux/mdev.h
index fd9fe1dcf0e230..af807c77c1e0f5 100644
--- a/include/linux/mdev.h
+++ b/include/linux/mdev.h
@@ -51,25 +51,6 @@ unsigned int mdev_get_type_group_id(struct mdev_device *mdev);
unsigned int mtype_get_type_group_id(struct mdev_type *mtype);
struct device *mtype_get_parent_dev(struct mdev_type *mtype);
-/**
- * struct mdev_parent_ops - Structure to be registered for each parent device to
- * register the device to mdev module.
- *
- * @owner: The module owner.
- * @device_driver: Which device driver to probe() on newly created devices
- * @mdev_attr_groups: Attributes of the mediated device.
- * @supported_type_groups: Attributes to define supported types. It is mandatory
- * to provide supported types.
- * Parent device that support mediated device should be registered with mdev
- * module with mdev_parent_ops structure.
- **/
-struct mdev_parent_ops {
- struct module *owner;
- struct mdev_driver *device_driver;
- const struct attribute_group **mdev_attr_groups;
- struct attribute_group **supported_type_groups;
-};
-
/* interface for exporting mdev supported type attributes */
struct mdev_type_attribute {
struct attribute attr;
@@ -94,12 +75,15 @@ struct mdev_type_attribute mdev_type_attr_##_name = \
* struct mdev_driver - Mediated device driver
* @probe: called when new device created
* @remove: called when device removed
+ * @supported_type_groups: Attributes to define supported types. It is mandatory
+ * to provide supported types.
* @driver: device driver structure
*
**/
struct mdev_driver {
int (*probe)(struct mdev_device *dev);
void (*remove)(struct mdev_device *dev);
+ struct attribute_group **supported_type_groups;
struct device_driver driver;
};
@@ -118,7 +102,7 @@ static inline const guid_t *mdev_uuid(struct mdev_device *mdev)
extern struct bus_type mdev_bus_type;
-int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops);
+int mdev_register_device(struct device *dev, struct mdev_driver *mdev_driver);
void mdev_unregister_device(struct device *dev);
int mdev_register_driver(struct mdev_driver *drv);
diff --git a/samples/vfio-mdev/mbochs.c b/samples/vfio-mdev/mbochs.c
index e18821a8a6beb8..c76ceec584b41b 100644
--- a/samples/vfio-mdev/mbochs.c
+++ b/samples/vfio-mdev/mbochs.c
@@ -1418,12 +1418,7 @@ static struct mdev_driver mbochs_driver = {
},
.probe = mbochs_probe,
.remove = mbochs_remove,
-};
-
-static const struct mdev_parent_ops mdev_fops = {
- .owner = THIS_MODULE,
- .device_driver = &mbochs_driver,
- .supported_type_groups = mdev_type_groups,
+ .supported_type_groups = mdev_type_groups,
};
static const struct file_operations vd_fops = {
@@ -1466,7 +1461,7 @@ static int __init mbochs_dev_init(void)
if (ret)
goto err_class;
- ret = mdev_register_device(&mbochs_dev, &mdev_fops);
+ ret = mdev_register_device(&mbochs_dev, &mbochs_driver);
if (ret)
goto err_device;
diff --git a/samples/vfio-mdev/mdpy.c b/samples/vfio-mdev/mdpy.c
index 82638de333330d..c22b2c808d132d 100644
--- a/samples/vfio-mdev/mdpy.c
+++ b/samples/vfio-mdev/mdpy.c
@@ -735,12 +735,7 @@ static struct mdev_driver mdpy_driver = {
},
.probe = mdpy_probe,
.remove = mdpy_remove,
-};
-
-static const struct mdev_parent_ops mdev_fops = {
- .owner = THIS_MODULE,
- .device_driver = &mdpy_driver,
- .supported_type_groups = mdev_type_groups,
+ .supported_type_groups = mdev_type_groups,
};
static const struct file_operations vd_fops = {
@@ -783,7 +778,7 @@ static int __init mdpy_dev_init(void)
if (ret)
goto err_class;
- ret = mdev_register_device(&mdpy_dev, &mdev_fops);
+ ret = mdev_register_device(&mdpy_dev, &mdpy_driver);
if (ret)
goto err_device;
diff --git a/samples/vfio-mdev/mtty.c b/samples/vfio-mdev/mtty.c
index 31eec76bc553ce..87f5ba12a230e3 100644
--- a/samples/vfio-mdev/mtty.c
+++ b/samples/vfio-mdev/mtty.c
@@ -1308,12 +1308,7 @@ static struct mdev_driver mtty_driver = {
},
.probe = mtty_probe,
.remove = mtty_remove,
-};
-
-static const struct mdev_parent_ops mdev_fops = {
- .owner = THIS_MODULE,
- .device_driver = &mtty_driver,
- .supported_type_groups = mdev_type_groups,
+ .supported_type_groups = mdev_type_groups,
};
static void mtty_device_release(struct device *dev)
@@ -1364,7 +1359,7 @@ static int __init mtty_dev_init(void)
if (ret)
goto err_class;
- ret = mdev_register_device(&mtty_dev.dev, &mdev_fops);
+ ret = mdev_register_device(&mtty_dev.dev, &mtty_driver);
if (ret)
goto err_device;
--
2.31.1
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [Intel-gfx] [PATCH 10/12] vfio/mdev: Remove mdev_parent_ops
@ 2021-04-23 23:03 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc,
linux-s390, Peter Oberparleiter, Halil Pasic, Pierre Morel,
Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang, Zhi Wang
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
The last useful member in this struct is the supported_type_groups, move
it to the mdev_driver and delete mdev_parent_ops.
Replace it with mdev_driver as an argument to mdev_register_device()
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
.../driver-api/vfio-mediated-device.rst | 36 +++++++------------
drivers/gpu/drm/i915/gvt/kvmgt.c | 8 ++---
drivers/s390/cio/vfio_ccw_ops.c | 7 +---
drivers/s390/crypto/vfio_ap_ops.c | 9 ++---
drivers/vfio/mdev/mdev_core.c | 13 +++----
drivers/vfio/mdev/mdev_driver.c | 2 +-
drivers/vfio/mdev/mdev_private.h | 2 +-
drivers/vfio/mdev/mdev_sysfs.c | 6 ++--
include/linux/mdev.h | 24 +++----------
samples/vfio-mdev/mbochs.c | 9 ++---
samples/vfio-mdev/mdpy.c | 9 ++---
samples/vfio-mdev/mtty.c | 9 ++---
12 files changed, 38 insertions(+), 96 deletions(-)
diff --git a/Documentation/driver-api/vfio-mediated-device.rst b/Documentation/driver-api/vfio-mediated-device.rst
index 5f866b17c93e69..b7cf357243d269 100644
--- a/Documentation/driver-api/vfio-mediated-device.rst
+++ b/Documentation/driver-api/vfio-mediated-device.rst
@@ -93,7 +93,7 @@ interfaces:
Registration Interface for a Mediated Bus Driver
------------------------------------------------
-The registration interface for a mediated bus driver provides the following
+The registration interface for a mediated device driver provides the following
structure to represent a mediated device's driver::
/*
@@ -105,6 +105,7 @@ structure to represent a mediated device's driver::
struct mdev_driver {
int (*probe) (struct mdev_device *dev);
void (*remove) (struct mdev_device *dev);
+ struct attribute_group **supported_type_groups;
struct device_driver driver;
};
@@ -119,35 +120,24 @@ to register and unregister itself with the core driver:
extern void mdev_unregister_driver(struct mdev_driver *drv);
-The mediated bus driver is responsible for adding mediated devices to the VFIO
-group when devices are bound to the driver and removing mediated devices from
-the VFIO when devices are unbound from the driver.
+The mediated bus driver's probe function should create a vfio_device on top of
+the mdev_device and connect it to an appropriate implementation of vfio_device_ops.
-
-Physical Device Driver Interface
---------------------------------
-
-The physical device driver interface provides the mdev_parent_ops[3] structure
-to define the APIs to manage work in the mediated core driver that is related
-to the physical device.
-
-The structures in the mdev_parent_ops structure are as follows:
-
-* dev_attr_groups: attributes of the parent device
-* mdev_attr_groups: attributes of the mediated device
-* supported_config: attributes to define supported configurations
-
-A driver should use the mdev_parent_ops structure in the function call to
-register itself with the mdev core driver::
+When a driver wants to add the GUID creation sysfs to an existing device it has
+probe'd to then it should call:
extern int mdev_register_device(struct device *dev,
- const struct mdev_parent_ops *ops);
+ struct mdev_driver *mdev_driver);
+
+This will provide the 'mdev_supported_types/XX/create' files which can then be used
+to trigger the creation of a mdev_device. The created mdev_device will be attached
+to the specified driver.
-However, the mdev_parent_ops structure is not required in the function call
-that a driver should use to unregister itself with the mdev core driver::
+When the driver needs to remove itself it calls:
extern void mdev_unregister_device(struct device *dev);
+Which will unbind and destroy all the created mdevs and remove the sysfs files.
Mediated Device Management Interface Through sysfs
==================================================
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 85ef300087e091..02089efd15bb92 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1669,10 +1669,6 @@ static struct mdev_driver intel_vgpu_mdev_driver = {
.remove = intel_vgpu_remove,
};
-static struct mdev_parent_ops intel_vgpu_ops = {
- .device_driver = &intel_vgpu_mdev_driver,
-};
-
static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
{
struct attribute_group **kvm_vgpu_type_groups;
@@ -1680,9 +1676,9 @@ static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)
intel_gvt_ops = ops;
if (!intel_gvt_ops->get_gvt_attrs(&kvm_vgpu_type_groups))
return -EFAULT;
- intel_vgpu_ops.supported_type_groups = kvm_vgpu_type_groups;
+ intel_vgpu_mdev_driver.supported_type_groups = kvm_vgpu_type_groups;
- return mdev_register_device(dev, &intel_vgpu_ops);
+ return mdev_register_device(dev, &intel_vgpu_mdev_driver);
}
static void kvmgt_host_exit(struct device *dev)
diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c
index 0fcf46031d3821..161697529dcc41 100644
--- a/drivers/s390/cio/vfio_ccw_ops.c
+++ b/drivers/s390/cio/vfio_ccw_ops.c
@@ -655,17 +655,12 @@ struct mdev_driver vfio_ccw_mdev_driver = {
},
.probe = vfio_ccw_mdev_probe,
.remove = vfio_ccw_mdev_remove,
-};
-
-static const struct mdev_parent_ops vfio_ccw_mdev_ops = {
- .owner = THIS_MODULE,
- .device_driver = &vfio_ccw_mdev_driver,
.supported_type_groups = mdev_type_groups,
};
int vfio_ccw_mdev_reg(struct subchannel *sch)
{
- return mdev_register_device(&sch->dev, &vfio_ccw_mdev_ops);
+ return mdev_register_device(&sch->dev, &vfio_ccw_mdev_driver);
}
void vfio_ccw_mdev_unreg(struct subchannel *sch)
diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
index 79872c857dd522..92789257c87639 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -1339,12 +1339,7 @@ static struct mdev_driver vfio_ap_matrix_driver = {
},
.probe = vfio_ap_mdev_probe,
.remove = vfio_ap_mdev_remove,
-};
-
-static const struct mdev_parent_ops vfio_ap_matrix_ops = {
- .owner = THIS_MODULE,
- .device_driver = &vfio_ap_matrix_driver,
- .supported_type_groups = vfio_ap_mdev_type_groups,
+ .supported_type_groups = vfio_ap_mdev_type_groups,
};
int vfio_ap_mdev_register(void)
@@ -1357,7 +1352,7 @@ int vfio_ap_mdev_register(void)
if (ret)
return ret;
- ret = mdev_register_device(&matrix_dev->device, &vfio_ap_matrix_ops);
+ ret = mdev_register_device(&matrix_dev->device, &vfio_ap_matrix_driver);
if (ret)
goto err_driver;
return 0;
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index f95d01b57fb168..7e918241de10cc 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -109,12 +109,12 @@ static int mdev_device_remove_cb(struct device *dev, void *data)
/*
* mdev_register_device : Register a device
* @dev: device structure representing parent device.
- * @ops: Parent device operation structure to be registered.
+ * @mdev_driver: Device driver to bind to the newly created mdev
*
* Add device to list of registered parent devices.
* Returns a negative value on error, otherwise 0.
*/
-int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
+int mdev_register_device(struct device *dev, struct mdev_driver *mdev_driver)
{
int ret;
struct mdev_parent *parent;
@@ -122,9 +122,7 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
char *envp[] = { env_string, NULL };
/* check for mandatory ops */
- if (!ops || !ops->supported_type_groups)
- return -EINVAL;
- if (!ops->device_driver)
+ if (!mdev_driver->supported_type_groups)
return -EINVAL;
dev = get_device(dev);
@@ -151,7 +149,7 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops)
init_rwsem(&parent->unreg_sem);
parent->dev = dev;
- parent->ops = ops;
+ parent->mdev_driver = mdev_driver;
if (!mdev_bus_compat_class) {
mdev_bus_compat_class = class_compat_register("mdev_bus");
@@ -257,7 +255,7 @@ static int mdev_bind_driver(struct mdev_device *mdev)
while (1) {
device_lock(&mdev->dev);
if (mdev->dev.driver ==
- &mdev->type->parent->ops->device_driver->driver) {
+ &mdev->type->parent->mdev_driver->driver) {
ret = 0;
goto out_unlock;
}
@@ -304,7 +302,6 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
mdev->dev.parent = parent->dev;
mdev->dev.bus = &mdev_bus_type;
mdev->dev.release = mdev_device_release;
- mdev->dev.groups = parent->ops->mdev_attr_groups;
mdev->type = type;
/* Pairs with the put in mdev_device_release() */
kobject_get(&type->kobj);
diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
index 0012a9ee7cb0a4..12091e32afa396 100644
--- a/drivers/vfio/mdev/mdev_driver.c
+++ b/drivers/vfio/mdev/mdev_driver.c
@@ -75,7 +75,7 @@ static int mdev_match(struct device *dev, struct device_driver *drv)
{
struct mdev_device *mdev = to_mdev_device(dev);
- return drv == &mdev->type->parent->ops->device_driver->driver;
+ return drv == &mdev->type->parent->mdev_driver->driver;
}
struct bus_type mdev_bus_type = {
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
index a656cfe0346c33..839567d059a07d 100644
--- a/drivers/vfio/mdev/mdev_private.h
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -15,7 +15,7 @@ void mdev_bus_unregister(void);
struct mdev_parent {
struct device *dev;
- const struct mdev_parent_ops *ops;
+ const struct mdev_driver *mdev_driver;
struct kref ref;
struct list_head next;
struct kset *mdev_types_kset;
diff --git a/drivers/vfio/mdev/mdev_sysfs.c b/drivers/vfio/mdev/mdev_sysfs.c
index 66eef08833a4ef..5a3873d1a275ae 100644
--- a/drivers/vfio/mdev/mdev_sysfs.c
+++ b/drivers/vfio/mdev/mdev_sysfs.c
@@ -97,7 +97,7 @@ static struct mdev_type *add_mdev_supported_type(struct mdev_parent *parent,
{
struct mdev_type *type;
struct attribute_group *group =
- parent->ops->supported_type_groups[type_group_id];
+ parent->mdev_driver->supported_type_groups[type_group_id];
int ret;
if (!group->name) {
@@ -154,7 +154,7 @@ static struct mdev_type *add_mdev_supported_type(struct mdev_parent *parent,
static void remove_mdev_supported_type(struct mdev_type *type)
{
struct attribute_group *group =
- type->parent->ops->supported_type_groups[type->type_group_id];
+ type->parent->mdev_driver->supported_type_groups[type->type_group_id];
sysfs_remove_files(&type->kobj,
(const struct attribute **)group->attrs);
@@ -168,7 +168,7 @@ static int add_mdev_supported_type_groups(struct mdev_parent *parent)
{
int i;
- for (i = 0; parent->ops->supported_type_groups[i]; i++) {
+ for (i = 0; parent->mdev_driver->supported_type_groups[i]; i++) {
struct mdev_type *type;
type = add_mdev_supported_type(parent, i);
diff --git a/include/linux/mdev.h b/include/linux/mdev.h
index fd9fe1dcf0e230..af807c77c1e0f5 100644
--- a/include/linux/mdev.h
+++ b/include/linux/mdev.h
@@ -51,25 +51,6 @@ unsigned int mdev_get_type_group_id(struct mdev_device *mdev);
unsigned int mtype_get_type_group_id(struct mdev_type *mtype);
struct device *mtype_get_parent_dev(struct mdev_type *mtype);
-/**
- * struct mdev_parent_ops - Structure to be registered for each parent device to
- * register the device to mdev module.
- *
- * @owner: The module owner.
- * @device_driver: Which device driver to probe() on newly created devices
- * @mdev_attr_groups: Attributes of the mediated device.
- * @supported_type_groups: Attributes to define supported types. It is mandatory
- * to provide supported types.
- * Parent device that support mediated device should be registered with mdev
- * module with mdev_parent_ops structure.
- **/
-struct mdev_parent_ops {
- struct module *owner;
- struct mdev_driver *device_driver;
- const struct attribute_group **mdev_attr_groups;
- struct attribute_group **supported_type_groups;
-};
-
/* interface for exporting mdev supported type attributes */
struct mdev_type_attribute {
struct attribute attr;
@@ -94,12 +75,15 @@ struct mdev_type_attribute mdev_type_attr_##_name = \
* struct mdev_driver - Mediated device driver
* @probe: called when new device created
* @remove: called when device removed
+ * @supported_type_groups: Attributes to define supported types. It is mandatory
+ * to provide supported types.
* @driver: device driver structure
*
**/
struct mdev_driver {
int (*probe)(struct mdev_device *dev);
void (*remove)(struct mdev_device *dev);
+ struct attribute_group **supported_type_groups;
struct device_driver driver;
};
@@ -118,7 +102,7 @@ static inline const guid_t *mdev_uuid(struct mdev_device *mdev)
extern struct bus_type mdev_bus_type;
-int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops);
+int mdev_register_device(struct device *dev, struct mdev_driver *mdev_driver);
void mdev_unregister_device(struct device *dev);
int mdev_register_driver(struct mdev_driver *drv);
diff --git a/samples/vfio-mdev/mbochs.c b/samples/vfio-mdev/mbochs.c
index e18821a8a6beb8..c76ceec584b41b 100644
--- a/samples/vfio-mdev/mbochs.c
+++ b/samples/vfio-mdev/mbochs.c
@@ -1418,12 +1418,7 @@ static struct mdev_driver mbochs_driver = {
},
.probe = mbochs_probe,
.remove = mbochs_remove,
-};
-
-static const struct mdev_parent_ops mdev_fops = {
- .owner = THIS_MODULE,
- .device_driver = &mbochs_driver,
- .supported_type_groups = mdev_type_groups,
+ .supported_type_groups = mdev_type_groups,
};
static const struct file_operations vd_fops = {
@@ -1466,7 +1461,7 @@ static int __init mbochs_dev_init(void)
if (ret)
goto err_class;
- ret = mdev_register_device(&mbochs_dev, &mdev_fops);
+ ret = mdev_register_device(&mbochs_dev, &mbochs_driver);
if (ret)
goto err_device;
diff --git a/samples/vfio-mdev/mdpy.c b/samples/vfio-mdev/mdpy.c
index 82638de333330d..c22b2c808d132d 100644
--- a/samples/vfio-mdev/mdpy.c
+++ b/samples/vfio-mdev/mdpy.c
@@ -735,12 +735,7 @@ static struct mdev_driver mdpy_driver = {
},
.probe = mdpy_probe,
.remove = mdpy_remove,
-};
-
-static const struct mdev_parent_ops mdev_fops = {
- .owner = THIS_MODULE,
- .device_driver = &mdpy_driver,
- .supported_type_groups = mdev_type_groups,
+ .supported_type_groups = mdev_type_groups,
};
static const struct file_operations vd_fops = {
@@ -783,7 +778,7 @@ static int __init mdpy_dev_init(void)
if (ret)
goto err_class;
- ret = mdev_register_device(&mdpy_dev, &mdev_fops);
+ ret = mdev_register_device(&mdpy_dev, &mdpy_driver);
if (ret)
goto err_device;
diff --git a/samples/vfio-mdev/mtty.c b/samples/vfio-mdev/mtty.c
index 31eec76bc553ce..87f5ba12a230e3 100644
--- a/samples/vfio-mdev/mtty.c
+++ b/samples/vfio-mdev/mtty.c
@@ -1308,12 +1308,7 @@ static struct mdev_driver mtty_driver = {
},
.probe = mtty_probe,
.remove = mtty_remove,
-};
-
-static const struct mdev_parent_ops mdev_fops = {
- .owner = THIS_MODULE,
- .device_driver = &mtty_driver,
- .supported_type_groups = mdev_type_groups,
+ .supported_type_groups = mdev_type_groups,
};
static void mtty_device_release(struct device *dev)
@@ -1364,7 +1359,7 @@ static int __init mtty_dev_init(void)
if (ret)
goto err_class;
- ret = mdev_register_device(&mtty_dev.dev, &mdev_fops);
+ ret = mdev_register_device(&mtty_dev.dev, &mtty_driver);
if (ret)
goto err_device;
--
2.31.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 11/12] vfio/mdev: Use the driver core to create the 'remove' file
2021-04-23 23:02 ` Jason Gunthorpe
` (11 preceding siblings ...)
(?)
@ 2021-04-23 23:03 ` Jason Gunthorpe
2021-04-26 14:20 ` Christoph Hellwig
-1 siblings, 1 reply; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: Alex Williamson, Cornelia Huck, kvm, Kirti Wankhede
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
The device creator is supposed to use the dev.groups value to add sysfs
files before device_add is called, not call sysfs_create_files() after
device_add() returns. This creates a race with uevent delivery where the
extra attribute will not be visible.
This was being done because the groups had been co-opted by the mdev
driver, now that prior patches have moved the driver's groups to the
struct device_driver the dev.group is properly free for use here.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
drivers/vfio/mdev/mdev_core.c | 1 +
drivers/vfio/mdev/mdev_private.h | 2 ++
drivers/vfio/mdev/mdev_sysfs.c | 19 ++++++++++---------
3 files changed, 13 insertions(+), 9 deletions(-)
diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
index 7e918241de10cc..93d0955ba993f9 100644
--- a/drivers/vfio/mdev/mdev_core.c
+++ b/drivers/vfio/mdev/mdev_core.c
@@ -302,6 +302,7 @@ int mdev_device_create(struct mdev_type *type, const guid_t *uuid)
mdev->dev.parent = parent->dev;
mdev->dev.bus = &mdev_bus_type;
mdev->dev.release = mdev_device_release;
+ mdev->dev.groups = mdev_device_groups;
mdev->type = type;
/* Pairs with the put in mdev_device_release() */
kobject_get(&type->kobj);
diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h
index 839567d059a07d..c6944d3eaf78fa 100644
--- a/drivers/vfio/mdev/mdev_private.h
+++ b/drivers/vfio/mdev/mdev_private.h
@@ -32,6 +32,8 @@ struct mdev_type {
unsigned int type_group_id;
};
+extern const struct attribute_group *mdev_device_groups[];
+
#define to_mdev_type_attr(_attr) \
container_of(_attr, struct mdev_type_attribute, attr)
#define to_mdev_type(_kobj) \
diff --git a/drivers/vfio/mdev/mdev_sysfs.c b/drivers/vfio/mdev/mdev_sysfs.c
index 5a3873d1a275ae..0ccfeb3dda2455 100644
--- a/drivers/vfio/mdev/mdev_sysfs.c
+++ b/drivers/vfio/mdev/mdev_sysfs.c
@@ -244,11 +244,20 @@ static ssize_t remove_store(struct device *dev, struct device_attribute *attr,
static DEVICE_ATTR_WO(remove);
-static const struct attribute *mdev_device_attrs[] = {
+static struct attribute *mdev_device_attrs[] = {
&dev_attr_remove.attr,
NULL,
};
+static const struct attribute_group mdev_device_group = {
+ .attrs = mdev_device_attrs,
+};
+
+const struct attribute_group *mdev_device_groups[] = {
+ &mdev_device_group,
+ NULL
+};
+
int mdev_create_sysfs_files(struct mdev_device *mdev)
{
struct mdev_type *type = mdev->type;
@@ -262,15 +271,8 @@ int mdev_create_sysfs_files(struct mdev_device *mdev)
ret = sysfs_create_link(kobj, &type->kobj, "mdev_type");
if (ret)
goto type_link_failed;
-
- ret = sysfs_create_files(kobj, mdev_device_attrs);
- if (ret)
- goto create_files_failed;
-
return ret;
-create_files_failed:
- sysfs_remove_link(kobj, "mdev_type");
type_link_failed:
sysfs_remove_link(mdev->type->devices_kobj, dev_name(&mdev->dev));
return ret;
@@ -280,7 +282,6 @@ void mdev_remove_sysfs_files(struct mdev_device *mdev)
{
struct kobject *kobj = &mdev->dev.kobj;
- sysfs_remove_files(kobj, mdev_device_attrs);
sysfs_remove_link(kobj, "mdev_type");
sysfs_remove_link(mdev->type->devices_kobj, dev_name(&mdev->dev));
}
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* [PATCH 12/12] vfio/mdev: Remove mdev drvdata
2021-04-23 23:02 ` Jason Gunthorpe
` (12 preceding siblings ...)
(?)
@ 2021-04-23 23:03 ` Jason Gunthorpe
2021-04-26 14:20 ` Christoph Hellwig
-1 siblings, 1 reply; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-23 23:03 UTC (permalink / raw)
To: kvm, Kirti Wankhede
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
This is no longer used, remove it.
All usages were moved over to either use container_of() from a vfio_device
or to use dev_drvdata() directly on the mdev.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
include/linux/mdev.h | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/include/linux/mdev.h b/include/linux/mdev.h
index af807c77c1e0f5..2c7267f1356d78 100644
--- a/include/linux/mdev.h
+++ b/include/linux/mdev.h
@@ -15,7 +15,6 @@ struct mdev_type;
struct mdev_device {
struct device dev;
guid_t uuid;
- void *driver_data;
struct list_head next;
struct mdev_type *type;
struct device *iommu_device;
@@ -87,14 +86,6 @@ struct mdev_driver {
struct device_driver driver;
};
-static inline void *mdev_get_drvdata(struct mdev_device *mdev)
-{
- return mdev->driver_data;
-}
-static inline void mdev_set_drvdata(struct mdev_device *mdev, void *data)
-{
- mdev->driver_data = data;
-}
static inline const guid_t *mdev_uuid(struct mdev_device *mdev)
{
return &mdev->uuid;
--
2.31.1
^ permalink raw reply related [flat|nested] 74+ messages in thread
* Re: [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
2021-04-23 23:02 ` Jason Gunthorpe
(?)
@ 2021-04-24 0:08 ` Randy Dunlap
-1 siblings, 0 replies; 74+ messages in thread
From: Randy Dunlap @ 2021-04-24 0:08 UTC (permalink / raw)
To: Jason Gunthorpe, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Vasily Gorbik, Heiko Carstens,
intel-gfx, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Halil Pasic, Pierre Morel, Rodrigo Vivi
Cc: Raj, Ashok, Dan Williams, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
On 4/23/21 4:02 PM, Jason Gunthorpe wrote:
> @@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
>
> config SAMPLE_VFIO_MDEV_MBOCHS
> tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
You can drop the ending of the prompt string.
> - depends on VFIO_MDEV_DEVICE && m
> + depends on VFIO_MDEV
> select DMA_SHARED_BUFFER
> help
> Build a virtual display sample driver for use as a VFIO
--
~Randy
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
@ 2021-04-24 0:08 ` Randy Dunlap
0 siblings, 0 replies; 74+ messages in thread
From: Randy Dunlap @ 2021-04-24 0:08 UTC (permalink / raw)
To: Jason Gunthorpe, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Vasily Gorbik, Heiko Carstens,
intel-gfx, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Halil Pasic, Pierre Morel, Rodrigo Vivi
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
On 4/23/21 4:02 PM, Jason Gunthorpe wrote:
> @@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
>
> config SAMPLE_VFIO_MDEV_MBOCHS
> tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
You can drop the ending of the prompt string.
> - depends on VFIO_MDEV_DEVICE && m
> + depends on VFIO_MDEV
> select DMA_SHARED_BUFFER
> help
> Build a virtual display sample driver for use as a VFIO
--
~Randy
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
@ 2021-04-24 0:08 ` Randy Dunlap
0 siblings, 0 replies; 74+ messages in thread
From: Randy Dunlap @ 2021-04-24 0:08 UTC (permalink / raw)
To: Jason Gunthorpe, David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Vasily Gorbik, Heiko Carstens,
intel-gfx, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Halil Pasic, Pierre Morel, Rodrigo Vivi
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
On 4/23/21 4:02 PM, Jason Gunthorpe wrote:
> @@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
>
> config SAMPLE_VFIO_MDEV_MBOCHS
> tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
You can drop the ending of the prompt string.
> - depends on VFIO_MDEV_DEVICE && m
> + depends on VFIO_MDEV
> select DMA_SHARED_BUFFER
> help
> Build a virtual display sample driver for use as a VFIO
--
~Randy
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 02/12] vfio/mdev: Allow the mdev_parent_ops to specify the device driver to bind
2021-04-23 23:02 ` [PATCH 02/12] vfio/mdev: Allow the mdev_parent_ops to specify the device driver to bind Jason Gunthorpe
@ 2021-04-26 14:02 ` Christoph Hellwig
2021-04-26 14:11 ` Jason Gunthorpe
0 siblings, 1 reply; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:02 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alex Williamson, Cornelia Huck, kvm, Kirti Wankhede, Raj, Ashok,
Dan Williams, Daniel Vetter, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
On Fri, Apr 23, 2021 at 08:02:59PM -0300, Jason Gunthorpe wrote:
> +/*
> + * mdev drivers can refuse to bind during probe(), in this case we want to fail
> + * the creation of the mdev all the way back to sysfs. This is a weird model
> + * that doesn't fit in the driver core well, nor does it seem to appear any
> + * place else in the kernel, so use a simple hack.
> + */
> +static int mdev_bind_driver(struct mdev_device *mdev)
> +{
> + struct mdev_driver *drv = mdev->type->parent->ops->device_driver;
> + int ret;
> +
> + if (!drv)
> + drv = &vfio_mdev_driver;
> +
> + while (1) {
> + device_lock(&mdev->dev);
> + if (mdev->dev.driver == &drv->driver) {
> + ret = 0;
> + goto out_unlock;
> + }
> + if (mdev->probe_err) {
> + ret = mdev->probe_err;
> + goto out_unlock;
> + }
> + device_unlock(&mdev->dev);
> + ret = device_attach(&mdev->dev);
> + if (ret)
> + return ret;
> + mdev->probe_err = -EINVAL;
> + }
> + return 0;
> +
> +out_unlock:
> + device_unlock(&mdev->dev);
> + return ret;
> +}
This looks strange to me, and I think by open coding
device_attach we could do much better here, something like:
static int mdev_bind_driver(struct mdev_device *mdev)
{
struct mdev_driver *drv = mdev->type->parent->ops->device_driver;
int ret = -EINVAL;
if (!drv)
drv = &vfio_mdev_driver;
device_lock(&mdev->dev);
if (WARN_ON_ONCE(device_is_bound(dev)))
goto out_unlock;
if (mdev->dev.p->dead)
goto out_unlock;
mdev->dev.driver = &drv->driver;
ret = device_bind_driver(&mdev->dev);
if (ret)
mdev->dev.driver = NULL;
out_unlock:
device_unlock(&mdev->dev);
return ret;
}
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 03/12] vfio/mtty: Convert to use vfio_register_group_dev()
2021-04-23 23:03 ` [PATCH 03/12] vfio/mtty: Convert to use vfio_register_group_dev() Jason Gunthorpe
@ 2021-04-26 14:06 ` Christoph Hellwig
0 siblings, 0 replies; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:06 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, Kirti Wankhede, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Fri, Apr 23, 2021 at 08:03:00PM -0300, Jason Gunthorpe wrote:
> This is straightforward conversion, the mdev_state is actually serving as
> the vfio_device and we can replace all the mdev_get_drvdata()'s and the
> wonky dead code with a simple container_of()
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 04/12] vfio/mdpy: Convert to use vfio_register_group_dev()
2021-04-23 23:03 ` [PATCH 04/12] vfio/mdpy: " Jason Gunthorpe
@ 2021-04-26 14:06 ` Christoph Hellwig
0 siblings, 0 replies; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:06 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, Kirti Wankhede, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Fri, Apr 23, 2021 at 08:03:01PM -0300, Jason Gunthorpe wrote:
> This is straightforward conversion, the mdev_state is actually serving as
> the vfio_device and we can replace all the mdev_get_drvdata()'s and the
> wonky dead code with a simple container_of().
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 05/12] vfio/mbochs: Convert to use vfio_register_group_dev()
2021-04-23 23:03 ` [PATCH 05/12] vfio/mbochs: " Jason Gunthorpe
@ 2021-04-26 14:06 ` Christoph Hellwig
0 siblings, 0 replies; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:06 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, Kirti Wankhede, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Fri, Apr 23, 2021 at 08:03:02PM -0300, Jason Gunthorpe wrote:
> This is straightforward conversion, the mdev_state is actually serving as
> the vfio_device and we can replace all the mdev_get_drvdata()'s and the
> wonky dead code with a simple container_of().
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-04-23 23:03 ` [PATCH 06/12] vfio/ap_ops: " Jason Gunthorpe
@ 2021-04-26 14:07 ` Christoph Hellwig
2021-04-26 17:48 ` Cornelia Huck
1 sibling, 0 replies; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:07 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Tony Krowiak, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Fri, Apr 23, 2021 at 08:03:03PM -0300, Jason Gunthorpe wrote:
> This is straightforward conversion, the ap_matrix_mdev is actually serving
> as the vfio_device and we can replace all the mdev_get_drvdata()'s with a
> simple container_of().
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 02/12] vfio/mdev: Allow the mdev_parent_ops to specify the device driver to bind
2021-04-26 14:02 ` Christoph Hellwig
@ 2021-04-26 14:11 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 14:11 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Alex Williamson, Cornelia Huck, kvm, Kirti Wankhede, Raj, Ashok,
Dan Williams, Daniel Vetter, Leon Romanovsky, Max Gurtovoy,
Tarun Gupta
On Mon, Apr 26, 2021 at 04:02:57PM +0200, Christoph Hellwig wrote:
> On Fri, Apr 23, 2021 at 08:02:59PM -0300, Jason Gunthorpe wrote:
> > +/*
> > + * mdev drivers can refuse to bind during probe(), in this case we want to fail
> > + * the creation of the mdev all the way back to sysfs. This is a weird model
> > + * that doesn't fit in the driver core well, nor does it seem to appear any
> > + * place else in the kernel, so use a simple hack.
> > + */
> > +static int mdev_bind_driver(struct mdev_device *mdev)
> > +{
> > + struct mdev_driver *drv = mdev->type->parent->ops->device_driver;
> > + int ret;
> > +
> > + if (!drv)
> > + drv = &vfio_mdev_driver;
> > +
> > + while (1) {
> > + device_lock(&mdev->dev);
> > + if (mdev->dev.driver == &drv->driver) {
> > + ret = 0;
> > + goto out_unlock;
> > + }
> > + if (mdev->probe_err) {
> > + ret = mdev->probe_err;
> > + goto out_unlock;
> > + }
> > + device_unlock(&mdev->dev);
> > + ret = device_attach(&mdev->dev);
> > + if (ret)
> > + return ret;
> > + mdev->probe_err = -EINVAL;
> > + }
> > + return 0;
> > +
> > +out_unlock:
> > + device_unlock(&mdev->dev);
> > + return ret;
> > +}
>
> This looks strange to me, and I think by open coding
> device_attach we could do much better here, something like:
I look at this for a long time, it is strange.
> static int mdev_bind_driver(struct mdev_device *mdev)
> {
> struct mdev_driver *drv = mdev->type->parent->ops->device_driver;
> int ret = -EINVAL;
>
> if (!drv)
> drv = &vfio_mdev_driver;
>
> device_lock(&mdev->dev);
> if (WARN_ON_ONCE(device_is_bound(dev)))
> goto out_unlock;
> if (mdev->dev.p->dead)
> goto out_unlock;
'p' is private to the driver core so we can't touch it here
> mdev->dev.driver = &drv->driver;
> ret = device_bind_driver(&mdev->dev);
It is really counter intuitive but device_bind_driver() doesn't
actually call probe, or do a lot of other essential stuff.
As far as I can see the driver core has three different ways to bind
drivers:
- The normal 'really_probe()' path with all the bells and whistles.
- You can set dev.driver before calling device_add() and related
- You can call device_bind_driver() 'somehow'.
The later two completely skip all the really_probe() stuff, so things
like devm and more become broken. They also don't call probe(), that
is up to the caller. They seem only usable in very niche special
cases, unfortunately.
Some callers open code the probe() but then they have ordering
problems with the sysfs and other little issues.
In this case 99% of the time the driver will already be bound here and
this routine does nothing - the only case I worried about about is
some kind of defered probe by default which calling device_attach()
will defeat.
Thanks,
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 08/12] vfio/gvt: Convert to use vfio_register_group_dev()
2021-04-23 23:03 ` Jason Gunthorpe
@ 2021-04-26 14:13 ` Christoph Hellwig
-1 siblings, 0 replies; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:13 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: David Airlie, Alex Williamson, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, intel-gfx, intel-gvt-dev, Jani Nikula,
Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc, Rodrigo Vivi,
Zhenyu Wang, Zhi Wang, Raj, Ashok, Dan Williams,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta
> diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
> index ff9ecd80212503..7c236ba1b90eb1 100644
> --- a/drivers/vfio/mdev/Makefile
> +++ b/drivers/vfio/mdev/Makefile
> @@ -1,5 +1,5 @@
> # SPDX-License-Identifier: GPL-2.0-only
>
> -mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
> +mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
>
> obj-$(CONFIG_VFIO_MDEV) += mdev.o
> diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
> index 51b8a9fcf866ad..f95d01b57fb168 100644
> --- a/drivers/vfio/mdev/mdev_core.c
> +++ b/drivers/vfio/mdev/mdev_core.c
I think all these mdev core changes belong into a separate commit with a
separate commit log.
> static int __init mdev_init(void)
> {
> - int rc;
> -
> - rc = mdev_bus_register();
> - if (rc)
> - return rc;
> - rc = mdev_register_driver(&vfio_mdev_driver);
> - if (rc)
> - goto err_bus;
> - return 0;
> -err_bus:
> - mdev_bus_unregister();
> - return rc;
> + return mdev_bus_register();
Weird indentation. But I think it would be best to just kill off the
mdev_init wrapper anyway.
> static void __exit mdev_exit(void)
> {
> - mdev_unregister_driver(&vfio_mdev_driver);
> -
> if (mdev_bus_compat_class)
> class_compat_unregister(mdev_bus_compat_class);
> -
> mdev_bus_unregister();
> }
Same here.
> diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
> index 6e96c023d7823d..0012a9ee7cb0a4 100644
> --- a/drivers/vfio/mdev/mdev_driver.c
> +++ b/drivers/vfio/mdev/mdev_driver.c
> @@ -74,15 +74,8 @@ static int mdev_remove(struct device *dev)
> static int mdev_match(struct device *dev, struct device_driver *drv)
> {
> struct mdev_device *mdev = to_mdev_device(dev);
> +
> + return drv == &mdev->type->parent->ops->device_driver->driver;
> }
Btw, I think we don't even need ->match with the switch to use
device_bind_driver that I suggested.
> -EXPORT_SYMBOL_GPL(vfio_init_group_dev);
> +EXPORT_SYMBOL(vfio_init_group_dev);
> -EXPORT_SYMBOL_GPL(vfio_register_group_dev);
> +EXPORT_SYMBOL(vfio_register_group_dev);
> -EXPORT_SYMBOL_GPL(vfio_unregister_group_dev);
> +EXPORT_SYMBOL(vfio_unregister_group_dev);
Err, no. vfio should remain EXPORT_SYMBOL_GPL, just because the weird
mdev "GPL condom" that should never have been merged in that form went away.
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 08/12] vfio/gvt: Convert to use vfio_register_group_dev()
@ 2021-04-26 14:13 ` Christoph Hellwig
0 siblings, 0 replies; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:13 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Leon Romanovsky, Christoph Hellwig, Raj, Ashok, Jonathan Corbet,
Tarun Gupta, intel-gfx, Dan Williams, intel-gvt-dev,
Max Gurtovoy, Cornelia Huck
> diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
> index ff9ecd80212503..7c236ba1b90eb1 100644
> --- a/drivers/vfio/mdev/Makefile
> +++ b/drivers/vfio/mdev/Makefile
> @@ -1,5 +1,5 @@
> # SPDX-License-Identifier: GPL-2.0-only
>
> -mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
> +mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
>
> obj-$(CONFIG_VFIO_MDEV) += mdev.o
> diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
> index 51b8a9fcf866ad..f95d01b57fb168 100644
> --- a/drivers/vfio/mdev/mdev_core.c
> +++ b/drivers/vfio/mdev/mdev_core.c
I think all these mdev core changes belong into a separate commit with a
separate commit log.
> static int __init mdev_init(void)
> {
> - int rc;
> -
> - rc = mdev_bus_register();
> - if (rc)
> - return rc;
> - rc = mdev_register_driver(&vfio_mdev_driver);
> - if (rc)
> - goto err_bus;
> - return 0;
> -err_bus:
> - mdev_bus_unregister();
> - return rc;
> + return mdev_bus_register();
Weird indentation. But I think it would be best to just kill off the
mdev_init wrapper anyway.
> static void __exit mdev_exit(void)
> {
> - mdev_unregister_driver(&vfio_mdev_driver);
> -
> if (mdev_bus_compat_class)
> class_compat_unregister(mdev_bus_compat_class);
> -
> mdev_bus_unregister();
> }
Same here.
> diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
> index 6e96c023d7823d..0012a9ee7cb0a4 100644
> --- a/drivers/vfio/mdev/mdev_driver.c
> +++ b/drivers/vfio/mdev/mdev_driver.c
> @@ -74,15 +74,8 @@ static int mdev_remove(struct device *dev)
> static int mdev_match(struct device *dev, struct device_driver *drv)
> {
> struct mdev_device *mdev = to_mdev_device(dev);
> +
> + return drv == &mdev->type->parent->ops->device_driver->driver;
> }
Btw, I think we don't even need ->match with the switch to use
device_bind_driver that I suggested.
> -EXPORT_SYMBOL_GPL(vfio_init_group_dev);
> +EXPORT_SYMBOL(vfio_init_group_dev);
> -EXPORT_SYMBOL_GPL(vfio_register_group_dev);
> +EXPORT_SYMBOL(vfio_register_group_dev);
> -EXPORT_SYMBOL_GPL(vfio_unregister_group_dev);
> +EXPORT_SYMBOL(vfio_unregister_group_dev);
Err, no. vfio should remain EXPORT_SYMBOL_GPL, just because the weird
mdev "GPL condom" that should never have been merged in that form went away.
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 09/12] vfio/mdev: Remove mdev_parent_ops dev_attr_groups
2021-04-23 23:03 ` [PATCH 09/12] vfio/mdev: Remove mdev_parent_ops dev_attr_groups Jason Gunthorpe
@ 2021-04-26 14:14 ` Christoph Hellwig
0 siblings, 0 replies; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:14 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alex Williamson, Cornelia Huck, kvm, Kirti Wankhede, Raj, Ashok,
Dan Williams, Daniel Vetter, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
On Fri, Apr 23, 2021 at 08:03:06PM -0300, Jason Gunthorpe wrote:
> This is only used by one sample to print a fixed string that is pointless.
>
> In general, having a device driver attach sysfs attributes to the parent
> is horrific. This should never happen, and always leads to some kind of
> liftime bug as it become very difficult for the sysfs attribute to go back
> to any data owned by the device driver.
>
> Remove the general mechanism to create this abuse.
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 10/12] vfio/mdev: Remove mdev_parent_ops
2021-04-23 23:03 ` Jason Gunthorpe
@ 2021-04-26 14:19 ` Christoph Hellwig
-1 siblings, 0 replies; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:19 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc,
linux-s390, Peter Oberparleiter, Halil Pasic, Pierre Morel,
Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang, Zhi Wang, Raj, Ashok,
Dan Williams, Christoph Hellwig, Leon Romanovsky, Max Gurtovoy,
Tarun Gupta
> +The mediated bus driver's probe function should create a vfio_device on top of
> +the mdev_device and connect it to an appropriate implementation of vfio_device_ops.
Overly long line.
> +This will provide the 'mdev_supported_types/XX/create' files which can then be used
> +to trigger the creation of a mdev_device. The created mdev_device will be attached
Two more.
Otherwise looks good:
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 10/12] vfio/mdev: Remove mdev_parent_ops
@ 2021-04-26 14:19 ` Christoph Hellwig
0 siblings, 0 replies; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:19 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Vineeth Vijayan, Leon Romanovsky, Christoph Hellwig, linux-s390,
Raj, Ashok, Jonathan Corbet, Halil Pasic, Christian Borntraeger,
Tarun Gupta, intel-gfx, Max Gurtovoy, Eric Farman, Vasily Gorbik,
Heiko Carstens, Harald Freudenberger, Dan Williams,
intel-gvt-dev, Tony Krowiak, Pierre Morel, Cornelia Huck,
Peter Oberparleiter
> +The mediated bus driver's probe function should create a vfio_device on top of
> +the mdev_device and connect it to an appropriate implementation of vfio_device_ops.
Overly long line.
> +This will provide the 'mdev_supported_types/XX/create' files which can then be used
> +to trigger the creation of a mdev_device. The created mdev_device will be attached
Two more.
Otherwise looks good:
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 11/12] vfio/mdev: Use the driver core to create the 'remove' file
2021-04-23 23:03 ` [PATCH 11/12] vfio/mdev: Use the driver core to create the 'remove' file Jason Gunthorpe
@ 2021-04-26 14:20 ` Christoph Hellwig
2021-04-26 19:07 ` Jason Gunthorpe
0 siblings, 1 reply; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:20 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alex Williamson, Cornelia Huck, kvm, Kirti Wankhede, Raj, Ashok,
Dan Williams, Daniel Vetter, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
> diff --git a/drivers/vfio/mdev/mdev_sysfs.c b/drivers/vfio/mdev/mdev_sysfs.c
> index 5a3873d1a275ae..0ccfeb3dda2455 100644
> --- a/drivers/vfio/mdev/mdev_sysfs.c
> +++ b/drivers/vfio/mdev/mdev_sysfs.c
> @@ -244,11 +244,20 @@ static ssize_t remove_store(struct device *dev, struct device_attribute *attr,
>
> static DEVICE_ATTR_WO(remove);
>
> -static const struct attribute *mdev_device_attrs[] = {
> +static struct attribute *mdev_device_attrs[] = {
Why does this lose the const?
Otherwise looks good:
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 12/12] vfio/mdev: Remove mdev drvdata
2021-04-23 23:03 ` [PATCH 12/12] vfio/mdev: Remove mdev drvdata Jason Gunthorpe
@ 2021-04-26 14:20 ` Christoph Hellwig
0 siblings, 0 replies; 74+ messages in thread
From: Christoph Hellwig @ 2021-04-26 14:20 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, Kirti Wankhede, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Fri, Apr 23, 2021 at 08:03:09PM -0300, Jason Gunthorpe wrote:
> This is no longer used, remove it.
>
> All usages were moved over to either use container_of() from a vfio_device
> or to use dev_drvdata() directly on the mdev.
>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 08/12] vfio/gvt: Convert to use vfio_register_group_dev()
2021-04-26 14:13 ` [Intel-gfx] " Christoph Hellwig
(?)
@ 2021-04-26 15:44 ` Jason Gunthorpe
-1 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 15:44 UTC (permalink / raw)
To: Christoph Hellwig
Cc: David Airlie, Alex Williamson, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, intel-gfx, intel-gvt-dev, Jani Nikula,
Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc, Rodrigo Vivi,
Zhenyu Wang, Zhi Wang, Raj, Ashok, Dan Williams, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
On Mon, Apr 26, 2021 at 04:13:55PM +0200, Christoph Hellwig wrote:
> > diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
> > index ff9ecd80212503..7c236ba1b90eb1 100644
> > +++ b/drivers/vfio/mdev/Makefile
> > @@ -1,5 +1,5 @@
> > # SPDX-License-Identifier: GPL-2.0-only
> >
> > -mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
> > +mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
> >
> > obj-$(CONFIG_VFIO_MDEV) += mdev.o
> > diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
> > index 51b8a9fcf866ad..f95d01b57fb168 100644
> > +++ b/drivers/vfio/mdev/mdev_core.c
>
> I think all these mdev core changes belong into a separate commit with a
> separate commit log.
Gah, they were split, I must have flubbed up a rebase on Friday :\
commit daeb9dd3a152e21d11960805b55e34967987e8cf
vfio/mdev: Remove vfio_mdev.c
Now that all mdev drivers directly create their own mdev_device driver and
directly register with the vfio core's vfio_device_ops this is all dead
code.
Delete vfio_mdev.c and the mdev_parent_ops members that are connected to
it.
Preserve VFIO's design of allowing mdev drivers to be !GPL by allowing the
three functions that replace this module for !GPL usage. This goes along
with the other 19 symbols that are already marked !GPL in VFIO.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
I'll fix it
> > static int __init mdev_init(void)
> > {
> > - int rc;
> > -
> > - rc = mdev_bus_register();
> > - if (rc)
> > - return rc;
> > - rc = mdev_register_driver(&vfio_mdev_driver);
> > - if (rc)
> > - goto err_bus;
> > - return 0;
> > -err_bus:
> > - mdev_bus_unregister();
> > - return rc;
> > + return mdev_bus_register();
>
> Weird indentation. But I think it would be best to just kill off the
> mdev_init wrapper anyway.
Oh, right good point
> > diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
> > index 6e96c023d7823d..0012a9ee7cb0a4 100644
> > +++ b/drivers/vfio/mdev/mdev_driver.c
> > @@ -74,15 +74,8 @@ static int mdev_remove(struct device *dev)
> > static int mdev_match(struct device *dev, struct device_driver *drv)
> > {
> > struct mdev_device *mdev = to_mdev_device(dev);
> > +
> > + return drv == &mdev->type->parent->ops->device_driver->driver;
> > }
>
> Btw, I think we don't even need ->match with the switch to use
> device_bind_driver that I suggested.
See my other email for why it is like this..
> > -EXPORT_SYMBOL_GPL(vfio_init_group_dev);
> > +EXPORT_SYMBOL(vfio_init_group_dev);
>
> > -EXPORT_SYMBOL_GPL(vfio_register_group_dev);
> > +EXPORT_SYMBOL(vfio_register_group_dev);
>
> > -EXPORT_SYMBOL_GPL(vfio_unregister_group_dev);
> > +EXPORT_SYMBOL(vfio_unregister_group_dev);
>
> Err, no. vfio should remain EXPORT_SYMBOL_GPL, just because the weird
> mdev "GPL condom" that should never have been merged in that form went away.
VFIO is already !GPL - there are 19 symbols supporting this
today. What happened here is that this patch make all of those symbols
unusable !GPL by changing how registration works so you can't get the
vfio_device argument to use with the API family.
So, either the two registration functions need to be !GPL to make the
other 19 symbols make sense, or the entire !GPL needs to be ripped
out. The lost commit message above was explaining this.
Since it is predominately !GPL today, I'd prefer a discussion on
changing VFIO to be GPL only to be in its own patch proposing removing
all 22 !GPL symbols. Those are always fun threads..
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 08/12] vfio/gvt: Convert to use vfio_register_group_dev()
@ 2021-04-26 15:44 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 15:44 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Max Gurtovoy, Raj, Ashok, kvm, Jonathan Corbet, David Airlie,
Leon Romanovsky, intel-gfx, Cornelia Huck, linux-doc,
Kirti Wankhede, dri-devel, Alex Williamson, Rodrigo Vivi,
Dan Williams, intel-gvt-dev, Zhi Wang, Tarun Gupta
On Mon, Apr 26, 2021 at 04:13:55PM +0200, Christoph Hellwig wrote:
> > diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
> > index ff9ecd80212503..7c236ba1b90eb1 100644
> > +++ b/drivers/vfio/mdev/Makefile
> > @@ -1,5 +1,5 @@
> > # SPDX-License-Identifier: GPL-2.0-only
> >
> > -mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
> > +mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
> >
> > obj-$(CONFIG_VFIO_MDEV) += mdev.o
> > diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
> > index 51b8a9fcf866ad..f95d01b57fb168 100644
> > +++ b/drivers/vfio/mdev/mdev_core.c
>
> I think all these mdev core changes belong into a separate commit with a
> separate commit log.
Gah, they were split, I must have flubbed up a rebase on Friday :\
commit daeb9dd3a152e21d11960805b55e34967987e8cf
vfio/mdev: Remove vfio_mdev.c
Now that all mdev drivers directly create their own mdev_device driver and
directly register with the vfio core's vfio_device_ops this is all dead
code.
Delete vfio_mdev.c and the mdev_parent_ops members that are connected to
it.
Preserve VFIO's design of allowing mdev drivers to be !GPL by allowing the
three functions that replace this module for !GPL usage. This goes along
with the other 19 symbols that are already marked !GPL in VFIO.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
I'll fix it
> > static int __init mdev_init(void)
> > {
> > - int rc;
> > -
> > - rc = mdev_bus_register();
> > - if (rc)
> > - return rc;
> > - rc = mdev_register_driver(&vfio_mdev_driver);
> > - if (rc)
> > - goto err_bus;
> > - return 0;
> > -err_bus:
> > - mdev_bus_unregister();
> > - return rc;
> > + return mdev_bus_register();
>
> Weird indentation. But I think it would be best to just kill off the
> mdev_init wrapper anyway.
Oh, right good point
> > diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
> > index 6e96c023d7823d..0012a9ee7cb0a4 100644
> > +++ b/drivers/vfio/mdev/mdev_driver.c
> > @@ -74,15 +74,8 @@ static int mdev_remove(struct device *dev)
> > static int mdev_match(struct device *dev, struct device_driver *drv)
> > {
> > struct mdev_device *mdev = to_mdev_device(dev);
> > +
> > + return drv == &mdev->type->parent->ops->device_driver->driver;
> > }
>
> Btw, I think we don't even need ->match with the switch to use
> device_bind_driver that I suggested.
See my other email for why it is like this..
> > -EXPORT_SYMBOL_GPL(vfio_init_group_dev);
> > +EXPORT_SYMBOL(vfio_init_group_dev);
>
> > -EXPORT_SYMBOL_GPL(vfio_register_group_dev);
> > +EXPORT_SYMBOL(vfio_register_group_dev);
>
> > -EXPORT_SYMBOL_GPL(vfio_unregister_group_dev);
> > +EXPORT_SYMBOL(vfio_unregister_group_dev);
>
> Err, no. vfio should remain EXPORT_SYMBOL_GPL, just because the weird
> mdev "GPL condom" that should never have been merged in that form went away.
VFIO is already !GPL - there are 19 symbols supporting this
today. What happened here is that this patch make all of those symbols
unusable !GPL by changing how registration works so you can't get the
vfio_device argument to use with the API family.
So, either the two registration functions need to be !GPL to make the
other 19 symbols make sense, or the entire !GPL needs to be ripped
out. The lost commit message above was explaining this.
Since it is predominately !GPL today, I'd prefer a discussion on
changing VFIO to be GPL only to be in its own patch proposing removing
all 22 !GPL symbols. Those are always fun threads..
Jason
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 08/12] vfio/gvt: Convert to use vfio_register_group_dev()
@ 2021-04-26 15:44 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 15:44 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Max Gurtovoy, Raj, Ashok, kvm, Jonathan Corbet, David Airlie,
Leon Romanovsky, intel-gfx, Cornelia Huck, linux-doc,
Kirti Wankhede, dri-devel, Dan Williams, intel-gvt-dev,
Tarun Gupta
On Mon, Apr 26, 2021 at 04:13:55PM +0200, Christoph Hellwig wrote:
> > diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
> > index ff9ecd80212503..7c236ba1b90eb1 100644
> > +++ b/drivers/vfio/mdev/Makefile
> > @@ -1,5 +1,5 @@
> > # SPDX-License-Identifier: GPL-2.0-only
> >
> > -mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o vfio_mdev.o
> > +mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
> >
> > obj-$(CONFIG_VFIO_MDEV) += mdev.o
> > diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c
> > index 51b8a9fcf866ad..f95d01b57fb168 100644
> > +++ b/drivers/vfio/mdev/mdev_core.c
>
> I think all these mdev core changes belong into a separate commit with a
> separate commit log.
Gah, they were split, I must have flubbed up a rebase on Friday :\
commit daeb9dd3a152e21d11960805b55e34967987e8cf
vfio/mdev: Remove vfio_mdev.c
Now that all mdev drivers directly create their own mdev_device driver and
directly register with the vfio core's vfio_device_ops this is all dead
code.
Delete vfio_mdev.c and the mdev_parent_ops members that are connected to
it.
Preserve VFIO's design of allowing mdev drivers to be !GPL by allowing the
three functions that replace this module for !GPL usage. This goes along
with the other 19 symbols that are already marked !GPL in VFIO.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
I'll fix it
> > static int __init mdev_init(void)
> > {
> > - int rc;
> > -
> > - rc = mdev_bus_register();
> > - if (rc)
> > - return rc;
> > - rc = mdev_register_driver(&vfio_mdev_driver);
> > - if (rc)
> > - goto err_bus;
> > - return 0;
> > -err_bus:
> > - mdev_bus_unregister();
> > - return rc;
> > + return mdev_bus_register();
>
> Weird indentation. But I think it would be best to just kill off the
> mdev_init wrapper anyway.
Oh, right good point
> > diff --git a/drivers/vfio/mdev/mdev_driver.c b/drivers/vfio/mdev/mdev_driver.c
> > index 6e96c023d7823d..0012a9ee7cb0a4 100644
> > +++ b/drivers/vfio/mdev/mdev_driver.c
> > @@ -74,15 +74,8 @@ static int mdev_remove(struct device *dev)
> > static int mdev_match(struct device *dev, struct device_driver *drv)
> > {
> > struct mdev_device *mdev = to_mdev_device(dev);
> > +
> > + return drv == &mdev->type->parent->ops->device_driver->driver;
> > }
>
> Btw, I think we don't even need ->match with the switch to use
> device_bind_driver that I suggested.
See my other email for why it is like this..
> > -EXPORT_SYMBOL_GPL(vfio_init_group_dev);
> > +EXPORT_SYMBOL(vfio_init_group_dev);
>
> > -EXPORT_SYMBOL_GPL(vfio_register_group_dev);
> > +EXPORT_SYMBOL(vfio_register_group_dev);
>
> > -EXPORT_SYMBOL_GPL(vfio_unregister_group_dev);
> > +EXPORT_SYMBOL(vfio_unregister_group_dev);
>
> Err, no. vfio should remain EXPORT_SYMBOL_GPL, just because the weird
> mdev "GPL condom" that should never have been merged in that form went away.
VFIO is already !GPL - there are 19 symbols supporting this
today. What happened here is that this patch make all of those symbols
unusable !GPL by changing how registration works so you can't get the
vfio_device argument to use with the API family.
So, either the two registration functions need to be !GPL to make the
other 19 symbols make sense, or the entire !GPL needs to be ripped
out. The lost commit message above was explaining this.
Since it is predominately !GPL today, I'd prefer a discussion on
changing VFIO to be GPL only to be in its own patch proposing removing
all 22 !GPL symbols. Those are always fun threads..
Jason
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
2021-04-23 23:02 ` Jason Gunthorpe
(?)
@ 2021-04-26 16:43 ` Christian Borntraeger
-1 siblings, 0 replies; 74+ messages in thread
From: Christian Borntraeger @ 2021-04-26 16:43 UTC (permalink / raw)
To: Jason Gunthorpe, David Airlie, Tony Krowiak, Alex Williamson,
Cornelia Huck, Jonathan Corbet, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Joonas Lahtinen, kvm,
Kirti Wankhede, linux-doc, linux-s390, Peter Oberparleiter,
Halil Pasic, Pierre Morel, Rodrigo Vivi, Vineeth Vijayan,
Zhenyu Wang, Zhi Wang
Cc: Raj, Ashok, Dan Williams, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
On 24.04.21 01:02, Jason Gunthorpe wrote:
> Prologue
> ========
>
> This is series #3 in part of a larger work that arose from the minor
> remark that the mdev_parent_ops indirection shim is useless and
> complicates things.
>
> It applies on top of Alex's current tree and requires the prior two
> series.
Do you have a tree somewhere?
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-26 16:43 ` Christian Borntraeger
0 siblings, 0 replies; 74+ messages in thread
From: Christian Borntraeger @ 2021-04-26 16:43 UTC (permalink / raw)
To: Jason Gunthorpe, David Airlie, Tony Krowiak, Alex Williamson,
Cornelia Huck, Jonathan Corbet, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Joonas Lahtinen, kvm,
Kirti Wankhede, linux-doc, linux-s390, Peter Oberparleiter,
Halil Pasic, Pierre Morel, Rodrigo Vivi, Vineeth Vijayan,
Zhenyu Wang, Zhi Wang
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
On 24.04.21 01:02, Jason Gunthorpe wrote:
> Prologue
> ========
>
> This is series #3 in part of a larger work that arose from the minor
> remark that the mdev_parent_ops indirection shim is useless and
> complicates things.
>
> It applies on top of Alex's current tree and requires the prior two
> series.
Do you have a tree somewhere?
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-26 16:43 ` Christian Borntraeger
0 siblings, 0 replies; 74+ messages in thread
From: Christian Borntraeger @ 2021-04-26 16:43 UTC (permalink / raw)
To: Jason Gunthorpe, David Airlie, Tony Krowiak, Alex Williamson,
Cornelia Huck, Jonathan Corbet, Daniel Vetter, dri-devel,
Eric Farman, Harald Freudenberger, Vasily Gorbik, Heiko Carstens,
intel-gfx, intel-gvt-dev, Jani Nikula, Joonas Lahtinen, kvm,
Kirti Wankhede, linux-doc, linux-s390, Peter Oberparleiter,
Halil Pasic, Pierre Morel, Rodrigo Vivi, Vineeth Vijayan,
Zhenyu Wang, Zhi Wang
Cc: Max Gurtovoy, Raj, Ashok, Tarun Gupta, Dan Williams,
Leon Romanovsky, Christoph Hellwig
On 24.04.21 01:02, Jason Gunthorpe wrote:
> Prologue
> ========
>
> This is series #3 in part of a larger work that arose from the minor
> remark that the mdev_parent_ops indirection shim is useless and
> complicates things.
>
> It applies on top of Alex's current tree and requires the prior two
> series.
Do you have a tree somewhere?
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
2021-04-26 16:43 ` Christian Borntraeger
(?)
@ 2021-04-26 17:42 ` Jason Gunthorpe
-1 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 17:42 UTC (permalink / raw)
To: Christian Borntraeger
Cc: David Airlie, Tony Krowiak, Alex Williamson, Cornelia Huck,
Jonathan Corbet, Daniel Vetter, dri-devel, Eric Farman,
Harald Freudenberger, Vasily Gorbik, Heiko Carstens, intel-gfx,
intel-gvt-dev, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Peter Oberparleiter, Halil Pasic,
Pierre Morel, Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang,
Zhi Wang, Raj, Ashok, Dan Williams, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Mon, Apr 26, 2021 at 06:43:14PM +0200, Christian Borntraeger wrote:
> On 24.04.21 01:02, Jason Gunthorpe wrote:
> > Prologue
> > ========
> >
> > This is series #3 in part of a larger work that arose from the minor
> > remark that the mdev_parent_ops indirection shim is useless and
> > complicates things.
> >
> > It applies on top of Alex's current tree and requires the prior two
> > series.
>
> Do you have a tree somewhere?
[..]
> > A preview of the future series's is here:
> > https://github.com/jgunthorpe/linux/pull/3/commits
Has everything, you'll want to go to:
cover-letter: Remove vfio_mdev.c, mdev_parent_ops and more
As there are additional WIPs in that tree.
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-26 17:42 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 17:42 UTC (permalink / raw)
To: Christian Borntraeger
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Vineeth Vijayan, Leon Romanovsky, Christoph Hellwig, linux-s390,
Raj, Ashok, Jonathan Corbet, Halil Pasic, Tarun Gupta, intel-gfx,
Zhi Wang, Max Gurtovoy, Eric Farman, Vasily Gorbik,
Heiko Carstens, Alex Williamson, Harald Freudenberger,
Rodrigo Vivi, Dan Williams, intel-gvt-dev, Tony Krowiak,
Pierre Morel, Cornelia Huck, Peter Oberparleiter
On Mon, Apr 26, 2021 at 06:43:14PM +0200, Christian Borntraeger wrote:
> On 24.04.21 01:02, Jason Gunthorpe wrote:
> > Prologue
> > ========
> >
> > This is series #3 in part of a larger work that arose from the minor
> > remark that the mdev_parent_ops indirection shim is useless and
> > complicates things.
> >
> > It applies on top of Alex's current tree and requires the prior two
> > series.
>
> Do you have a tree somewhere?
[..]
> > A preview of the future series's is here:
> > https://github.com/jgunthorpe/linux/pull/3/commits
Has everything, you'll want to go to:
cover-letter: Remove vfio_mdev.c, mdev_parent_ops and more
As there are additional WIPs in that tree.
Jason
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-26 17:42 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 17:42 UTC (permalink / raw)
To: Christian Borntraeger
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Vineeth Vijayan, Leon Romanovsky, Christoph Hellwig, linux-s390,
Raj, Ashok, Jonathan Corbet, Halil Pasic, Tarun Gupta, intel-gfx,
Max Gurtovoy, Eric Farman, Vasily Gorbik, Heiko Carstens,
Harald Freudenberger, Dan Williams, intel-gvt-dev, Tony Krowiak,
Pierre Morel, Cornelia Huck, Peter Oberparleiter
On Mon, Apr 26, 2021 at 06:43:14PM +0200, Christian Borntraeger wrote:
> On 24.04.21 01:02, Jason Gunthorpe wrote:
> > Prologue
> > ========
> >
> > This is series #3 in part of a larger work that arose from the minor
> > remark that the mdev_parent_ops indirection shim is useless and
> > complicates things.
> >
> > It applies on top of Alex's current tree and requires the prior two
> > series.
>
> Do you have a tree somewhere?
[..]
> > A preview of the future series's is here:
> > https://github.com/jgunthorpe/linux/pull/3/commits
Has everything, you'll want to go to:
cover-letter: Remove vfio_mdev.c, mdev_parent_ops and more
As there are additional WIPs in that tree.
Jason
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-04-23 23:03 ` [PATCH 06/12] vfio/ap_ops: " Jason Gunthorpe
2021-04-26 14:07 ` Christoph Hellwig
@ 2021-04-26 17:48 ` Cornelia Huck
2021-04-26 18:10 ` Jason Gunthorpe
` (2 more replies)
1 sibling, 3 replies; 74+ messages in thread
From: Cornelia Huck @ 2021-04-26 17:48 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Tony Krowiak, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Fri, 23 Apr 2021 20:03:03 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:
> This is straightforward conversion, the ap_matrix_mdev is actually serving
> as the vfio_device and we can replace all the mdev_get_drvdata()'s with a
> simple container_of().
>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
> drivers/s390/crypto/vfio_ap_ops.c | 137 ++++++++++++++++----------
> drivers/s390/crypto/vfio_ap_private.h | 2 +
> 2 files changed, 89 insertions(+), 50 deletions(-)
>
(...)
> -static int vfio_ap_mdev_remove(struct mdev_device *mdev)
> +static void vfio_ap_mdev_remove(struct mdev_device *mdev)
> {
> - struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
> + struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(&mdev->dev);
>
> - if (matrix_mdev->kvm)
> - return -EBUSY;
> + /* FIXME: Remove isn't allowed to fail */
> + if (WARN_ON(matrix_mdev->kvm))
> + return;
This is a pre-existing problem, but the rework now makes it more
obvious.
Previously, the mdev code would only print a warning and then continue
with device removal, even if a ->remove() callback returned an error.
Now, it's quite clear that we'll end up in a weird half-dead state.
IIRC, the check for matrix_mdev->kvm is intended to protect against
ripping out the device under a running guest (I think it needs to
manipulate some crypto control blocks?)
So my question for the vfio-ap maintainers is: Can we actually end up
in this case? If yes, is there any way to gracefully shut down the
device?
> +
> + vfio_unregister_group_dev(&matrix_mdev->vdev);
>
> mutex_lock(&matrix_dev->lock);
> - vfio_ap_mdev_reset_queues(mdev);
> + vfio_ap_mdev_reset_queues(matrix_mdev);
> list_del(&matrix_mdev->node);
> mutex_unlock(&matrix_dev->lock);
>
> kfree(matrix_mdev);
> - mdev_set_drvdata(mdev, NULL);
> atomic_inc(&matrix_dev->available_instances);
> -
> - return 0;
> }
(...)
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-04-26 17:48 ` Cornelia Huck
@ 2021-04-26 18:10 ` Jason Gunthorpe
2021-04-26 23:41 ` Halil Pasic
2021-05-03 20:14 ` Tony Krowiak
2 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 18:10 UTC (permalink / raw)
To: Cornelia Huck
Cc: Tony Krowiak, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Mon, Apr 26, 2021 at 07:48:59PM +0200, Cornelia Huck wrote:
> On Fri, 23 Apr 2021 20:03:03 -0300
> Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> > This is straightforward conversion, the ap_matrix_mdev is actually serving
> > as the vfio_device and we can replace all the mdev_get_drvdata()'s with a
> > simple container_of().
> >
> > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> > drivers/s390/crypto/vfio_ap_ops.c | 137 ++++++++++++++++----------
> > drivers/s390/crypto/vfio_ap_private.h | 2 +
> > 2 files changed, 89 insertions(+), 50 deletions(-)
> >
>
> (...)
>
> > -static int vfio_ap_mdev_remove(struct mdev_device *mdev)
> > +static void vfio_ap_mdev_remove(struct mdev_device *mdev)
> > {
> > - struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
> > + struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(&mdev->dev);
> >
> > - if (matrix_mdev->kvm)
> > - return -EBUSY;
> > + /* FIXME: Remove isn't allowed to fail */
> > + if (WARN_ON(matrix_mdev->kvm))
> > + return;
>
> This is a pre-existing problem, but the rework now makes it more
> obvious.
>
> Previously, the mdev code would only print a warning and then continue
> with device removal, even if a ->remove() callback returned an
> error.
This does mostly the same, the warning was just moved from
mdev_device_remove_common() to here and changed to a WARN_ON() because
it means we are permanently leaking kernel memory. I think in this
case the vfio_device is not deleted - though I could re-order this to
make that happen.
> Now, it's quite clear that we'll end up in a weird half-dead state.
I don't think it changes, after we print the WARN_ON we return to the
driver core which does the same put_device()/etc as
mdev_device_remove_common() was doing.
> IIRC, the check for matrix_mdev->kvm is intended to protect against
> ripping out the device under a running guest (I think it needs to
> manipulate some crypto control blocks?)
In that case it is missing locking too.
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
2021-04-24 0:08 ` Randy Dunlap
(?)
@ 2021-04-26 18:26 ` Jason Gunthorpe
-1 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 18:26 UTC (permalink / raw)
To: Randy Dunlap
Cc: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Vasily Gorbik, Heiko Carstens,
intel-gfx, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Halil Pasic, Pierre Morel, Rodrigo Vivi,
Raj, Ashok, Dan Williams, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
On Fri, Apr 23, 2021 at 05:08:10PM -0700, Randy Dunlap wrote:
> On 4/23/21 4:02 PM, Jason Gunthorpe wrote:
> > @@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
> >
> > config SAMPLE_VFIO_MDEV_MBOCHS
> > tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
>
> You can drop the ending of the prompt string.
Hum, I see this whole sample kconfig file is filled with this '&& m'
pattern, I wonder if there is a reason?
I think I will put the '&& m' back, I thought it was some kconfig
misunderstanding as it is very strange to see a naked '&& M'.
Thanks
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
@ 2021-04-26 18:26 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 18:26 UTC (permalink / raw)
To: Randy Dunlap
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Leon Romanovsky, Christoph Hellwig, linux-s390, Raj, Ashok,
Jonathan Corbet, Halil Pasic, Christian Borntraeger, Tarun Gupta,
intel-gfx, Max Gurtovoy, Vasily Gorbik, Heiko Carstens,
Alex Williamson, Rodrigo Vivi, Dan Williams, Tony Krowiak,
Pierre Morel, Cornelia Huck
On Fri, Apr 23, 2021 at 05:08:10PM -0700, Randy Dunlap wrote:
> On 4/23/21 4:02 PM, Jason Gunthorpe wrote:
> > @@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
> >
> > config SAMPLE_VFIO_MDEV_MBOCHS
> > tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
>
> You can drop the ending of the prompt string.
Hum, I see this whole sample kconfig file is filled with this '&& m'
pattern, I wonder if there is a reason?
I think I will put the '&& m' back, I thought it was some kconfig
misunderstanding as it is very strange to see a naked '&& M'.
Thanks
Jason
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
@ 2021-04-26 18:26 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 18:26 UTC (permalink / raw)
To: Randy Dunlap
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Leon Romanovsky, Christoph Hellwig, linux-s390, Raj, Ashok,
Jonathan Corbet, Halil Pasic, Christian Borntraeger, Tarun Gupta,
intel-gfx, Max Gurtovoy, Vasily Gorbik, Heiko Carstens,
Dan Williams, Tony Krowiak, Pierre Morel, Cornelia Huck
On Fri, Apr 23, 2021 at 05:08:10PM -0700, Randy Dunlap wrote:
> On 4/23/21 4:02 PM, Jason Gunthorpe wrote:
> > @@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
> >
> > config SAMPLE_VFIO_MDEV_MBOCHS
> > tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
>
> You can drop the ending of the prompt string.
Hum, I see this whole sample kconfig file is filled with this '&& m'
pattern, I wonder if there is a reason?
I think I will put the '&& m' back, I thought it was some kconfig
misunderstanding as it is very strange to see a naked '&& M'.
Thanks
Jason
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 10/12] vfio/mdev: Remove mdev_parent_ops
2021-04-26 14:19 ` [Intel-gfx] " Christoph Hellwig
(?)
@ 2021-04-26 18:33 ` Jason Gunthorpe
-1 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 18:33 UTC (permalink / raw)
To: Christoph Hellwig
Cc: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Eric Farman, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, intel-gfx, intel-gvt-dev,
Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede, linux-doc,
linux-s390, Peter Oberparleiter, Halil Pasic, Pierre Morel,
Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang, Zhi Wang, Raj, Ashok,
Dan Williams, Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Mon, Apr 26, 2021 at 04:19:11PM +0200, Christoph Hellwig wrote:
> > +The mediated bus driver's probe function should create a vfio_device on top of
> > +the mdev_device and connect it to an appropriate implementation of vfio_device_ops.
>
> Overly long line.
>
> > +This will provide the 'mdev_supported_types/XX/create' files which can then be used
> > +to trigger the creation of a mdev_device. The created mdev_device will be attached
>
> Two more.
Got it, thanks
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 10/12] vfio/mdev: Remove mdev_parent_ops
@ 2021-04-26 18:33 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 18:33 UTC (permalink / raw)
To: Christoph Hellwig
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Vineeth Vijayan, Leon Romanovsky, linux-s390, Raj, Ashok,
Jonathan Corbet, Halil Pasic, Christian Borntraeger, Tarun Gupta,
intel-gfx, Zhi Wang, Max Gurtovoy, Eric Farman, Vasily Gorbik,
Heiko Carstens, Alex Williamson, Harald Freudenberger,
Rodrigo Vivi, Dan Williams, intel-gvt-dev, Tony Krowiak,
Pierre Morel, Cornelia Huck, Peter Oberparleiter
On Mon, Apr 26, 2021 at 04:19:11PM +0200, Christoph Hellwig wrote:
> > +The mediated bus driver's probe function should create a vfio_device on top of
> > +the mdev_device and connect it to an appropriate implementation of vfio_device_ops.
>
> Overly long line.
>
> > +This will provide the 'mdev_supported_types/XX/create' files which can then be used
> > +to trigger the creation of a mdev_device. The created mdev_device will be attached
>
> Two more.
Got it, thanks
Jason
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 10/12] vfio/mdev: Remove mdev_parent_ops
@ 2021-04-26 18:33 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 18:33 UTC (permalink / raw)
To: Christoph Hellwig
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Vineeth Vijayan, Leon Romanovsky, linux-s390, Raj, Ashok,
Jonathan Corbet, Halil Pasic, Christian Borntraeger, Tarun Gupta,
intel-gfx, Max Gurtovoy, Eric Farman, Vasily Gorbik,
Heiko Carstens, Harald Freudenberger, Dan Williams,
intel-gvt-dev, Tony Krowiak, Pierre Morel, Cornelia Huck,
Peter Oberparleiter
On Mon, Apr 26, 2021 at 04:19:11PM +0200, Christoph Hellwig wrote:
> > +The mediated bus driver's probe function should create a vfio_device on top of
> > +the mdev_device and connect it to an appropriate implementation of vfio_device_ops.
>
> Overly long line.
>
> > +This will provide the 'mdev_supported_types/XX/create' files which can then be used
> > +to trigger the creation of a mdev_device. The created mdev_device will be attached
>
> Two more.
Got it, thanks
Jason
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 11/12] vfio/mdev: Use the driver core to create the 'remove' file
2021-04-26 14:20 ` Christoph Hellwig
@ 2021-04-26 19:07 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-26 19:07 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Alex Williamson, Cornelia Huck, kvm, Kirti Wankhede, Raj, Ashok,
Dan Williams, Daniel Vetter, Leon Romanovsky, Max Gurtovoy,
Tarun Gupta
On Mon, Apr 26, 2021 at 04:20:11PM +0200, Christoph Hellwig wrote:
> > diff --git a/drivers/vfio/mdev/mdev_sysfs.c b/drivers/vfio/mdev/mdev_sysfs.c
> > index 5a3873d1a275ae..0ccfeb3dda2455 100644
> > +++ b/drivers/vfio/mdev/mdev_sysfs.c
> > @@ -244,11 +244,20 @@ static ssize_t remove_store(struct device *dev, struct device_attribute *attr,
> >
> > static DEVICE_ATTR_WO(remove);
> >
> > -static const struct attribute *mdev_device_attrs[] = {
> > +static struct attribute *mdev_device_attrs[] = {
>
> Why does this lose the const?
Due to the way the driver core sets up it structs:
drivers/vfio/mdev/mdev_sysfs.c:273:11: error: initialization of ‘struct attribute **’ from incompatible pointer type ‘const struct attribute **’ [-Werror=incompatible-pointer-types]
273 | .attrs = mdev_device_attrs,
struct attribute_group {
[..]
struct attribute **attrs;
Thanks,
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
2021-04-26 18:26 ` Jason Gunthorpe
(?)
@ 2021-04-26 19:11 ` Randy Dunlap
-1 siblings, 0 replies; 74+ messages in thread
From: Randy Dunlap @ 2021-04-26 19:11 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: David Airlie, Tony Krowiak, Alex Williamson,
Christian Borntraeger, Cornelia Huck, Jonathan Corbet,
Daniel Vetter, dri-devel, Vasily Gorbik, Heiko Carstens,
intel-gfx, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Halil Pasic, Pierre Morel, Rodrigo Vivi,
Raj, Ashok, Dan Williams, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
On 4/26/21 11:26 AM, Jason Gunthorpe wrote:
> On Fri, Apr 23, 2021 at 05:08:10PM -0700, Randy Dunlap wrote:
>> On 4/23/21 4:02 PM, Jason Gunthorpe wrote:
>>> @@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
>>>
>>> config SAMPLE_VFIO_MDEV_MBOCHS
>>> tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
>>
>> You can drop the ending of the prompt string.
>
> Hum, I see this whole sample kconfig file is filled with this '&& m'
> pattern, I wonder if there is a reason?
>
> I think I will put the '&& m' back, I thought it was some kconfig
> misunderstanding as it is very strange to see a naked '&& M'.
It just limits those kconfig items to being =m or not set,
i.e., even though they are tristate, setting to =y is not
allowed. I guess the thinking is that samples don't need to
reside in system memory for very long. However, if you want
this one to be capable of =y, like your patch, you can still
remove the end of the prompt string.
--
~Randy
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
@ 2021-04-26 19:11 ` Randy Dunlap
0 siblings, 0 replies; 74+ messages in thread
From: Randy Dunlap @ 2021-04-26 19:11 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Leon Romanovsky, Christoph Hellwig, linux-s390, Raj, Ashok,
Jonathan Corbet, Halil Pasic, Christian Borntraeger, Tarun Gupta,
intel-gfx, Max Gurtovoy, Vasily Gorbik, Heiko Carstens,
Alex Williamson, Rodrigo Vivi, Dan Williams, Tony Krowiak,
Pierre Morel, Cornelia Huck
On 4/26/21 11:26 AM, Jason Gunthorpe wrote:
> On Fri, Apr 23, 2021 at 05:08:10PM -0700, Randy Dunlap wrote:
>> On 4/23/21 4:02 PM, Jason Gunthorpe wrote:
>>> @@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
>>>
>>> config SAMPLE_VFIO_MDEV_MBOCHS
>>> tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
>>
>> You can drop the ending of the prompt string.
>
> Hum, I see this whole sample kconfig file is filled with this '&& m'
> pattern, I wonder if there is a reason?
>
> I think I will put the '&& m' back, I thought it was some kconfig
> misunderstanding as it is very strange to see a naked '&& M'.
It just limits those kconfig items to being =m or not set,
i.e., even though they are tristate, setting to =y is not
allowed. I guess the thinking is that samples don't need to
reside in system memory for very long. However, if you want
this one to be capable of =y, like your patch, you can still
remove the end of the prompt string.
--
~Randy
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE
@ 2021-04-26 19:11 ` Randy Dunlap
0 siblings, 0 replies; 74+ messages in thread
From: Randy Dunlap @ 2021-04-26 19:11 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Leon Romanovsky, Christoph Hellwig, linux-s390, Raj, Ashok,
Jonathan Corbet, Halil Pasic, Christian Borntraeger, Tarun Gupta,
intel-gfx, Max Gurtovoy, Vasily Gorbik, Heiko Carstens,
Dan Williams, Tony Krowiak, Pierre Morel, Cornelia Huck
On 4/26/21 11:26 AM, Jason Gunthorpe wrote:
> On Fri, Apr 23, 2021 at 05:08:10PM -0700, Randy Dunlap wrote:
>> On 4/23/21 4:02 PM, Jason Gunthorpe wrote:
>>> @@ -171,7 +171,7 @@ config SAMPLE_VFIO_MDEV_MDPY_FB
>>>
>>> config SAMPLE_VFIO_MDEV_MBOCHS
>>> tristate "Build VFIO mdpy example mediated device sample code -- loadable modules only"
>>
>> You can drop the ending of the prompt string.
>
> Hum, I see this whole sample kconfig file is filled with this '&& m'
> pattern, I wonder if there is a reason?
>
> I think I will put the '&& m' back, I thought it was some kconfig
> misunderstanding as it is very strange to see a naked '&& M'.
It just limits those kconfig items to being =m or not set,
i.e., even though they are tristate, setting to =y is not
allowed. I guess the thinking is that samples don't need to
reside in system memory for very long. However, if you want
this one to be capable of =y, like your patch, you can still
remove the end of the prompt string.
--
~Randy
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-04-26 17:48 ` Cornelia Huck
2021-04-26 18:10 ` Jason Gunthorpe
@ 2021-04-26 23:41 ` Halil Pasic
2021-05-03 20:14 ` Tony Krowiak
2 siblings, 0 replies; 74+ messages in thread
From: Halil Pasic @ 2021-04-26 23:41 UTC (permalink / raw)
To: Cornelia Huck
Cc: Jason Gunthorpe, Tony Krowiak, Christian Borntraeger,
Harald Freudenberger, Vasily Gorbik, Heiko Carstens, linux-s390,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Mon, 26 Apr 2021 19:48:59 +0200
Cornelia Huck <cohuck@redhat.com> wrote:
> So my question for the vfio-ap maintainers is: Can we actually end up
> in this case? If yes, is there any way to gracefully shut down the
> device?
@Tony: Can you take this one?
Regards,
Halil
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
2021-04-26 17:42 ` Jason Gunthorpe
(?)
@ 2021-04-27 7:33 ` Christian Borntraeger
-1 siblings, 0 replies; 74+ messages in thread
From: Christian Borntraeger @ 2021-04-27 7:33 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: David Airlie, Tony Krowiak, Alex Williamson, Cornelia Huck,
Jonathan Corbet, Daniel Vetter, dri-devel, Eric Farman,
Harald Freudenberger, Vasily Gorbik, Heiko Carstens, intel-gfx,
intel-gvt-dev, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Peter Oberparleiter, Halil Pasic,
Pierre Morel, Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang,
Zhi Wang, Raj, Ashok, Dan Williams, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On 26.04.21 19:42, Jason Gunthorpe wrote:
> On Mon, Apr 26, 2021 at 06:43:14PM +0200, Christian Borntraeger wrote:
>> On 24.04.21 01:02, Jason Gunthorpe wrote:
>>> Prologue
>>> ========
>>>
>>> This is series #3 in part of a larger work that arose from the minor
>>> remark that the mdev_parent_ops indirection shim is useless and
>>> complicates things.
>>>
>>> It applies on top of Alex's current tree and requires the prior two
>>> series.
>>
>> Do you have a tree somewhere?
>
> [..]
>>> A preview of the future series's is here:
>>> https://github.com/jgunthorpe/linux/pull/3/commits
>
> Has everything, you'll want to go to:
> cover-letter: Remove vfio_mdev.c, mdev_parent_ops and more
>
> As there are additional WIPs in that tree.
I gave this a quick spin on s390x vfio-ap and it seems to work ok.
This is really just a quick test, but no obvious problem.
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-27 7:33 ` Christian Borntraeger
0 siblings, 0 replies; 74+ messages in thread
From: Christian Borntraeger @ 2021-04-27 7:33 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Vineeth Vijayan, Leon Romanovsky, Christoph Hellwig, linux-s390,
Raj, Ashok, Jonathan Corbet, Halil Pasic, Tarun Gupta, intel-gfx,
Zhi Wang, Max Gurtovoy, Eric Farman, Vasily Gorbik,
Heiko Carstens, Alex Williamson, Harald Freudenberger,
Rodrigo Vivi, Dan Williams, intel-gvt-dev, Tony Krowiak,
Pierre Morel, Cornelia Huck, Peter Oberparleiter
On 26.04.21 19:42, Jason Gunthorpe wrote:
> On Mon, Apr 26, 2021 at 06:43:14PM +0200, Christian Borntraeger wrote:
>> On 24.04.21 01:02, Jason Gunthorpe wrote:
>>> Prologue
>>> ========
>>>
>>> This is series #3 in part of a larger work that arose from the minor
>>> remark that the mdev_parent_ops indirection shim is useless and
>>> complicates things.
>>>
>>> It applies on top of Alex's current tree and requires the prior two
>>> series.
>>
>> Do you have a tree somewhere?
>
> [..]
>>> A preview of the future series's is here:
>>> https://github.com/jgunthorpe/linux/pull/3/commits
>
> Has everything, you'll want to go to:
> cover-letter: Remove vfio_mdev.c, mdev_parent_ops and more
>
> As there are additional WIPs in that tree.
I gave this a quick spin on s390x vfio-ap and it seems to work ok.
This is really just a quick test, but no obvious problem.
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-27 7:33 ` Christian Borntraeger
0 siblings, 0 replies; 74+ messages in thread
From: Christian Borntraeger @ 2021-04-27 7:33 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Vineeth Vijayan, Leon Romanovsky, Christoph Hellwig, linux-s390,
Raj, Ashok, Jonathan Corbet, Halil Pasic, Tarun Gupta, intel-gfx,
Max Gurtovoy, Eric Farman, Vasily Gorbik, Heiko Carstens,
Harald Freudenberger, Dan Williams, intel-gvt-dev, Tony Krowiak,
Pierre Morel, Cornelia Huck, Peter Oberparleiter
On 26.04.21 19:42, Jason Gunthorpe wrote:
> On Mon, Apr 26, 2021 at 06:43:14PM +0200, Christian Borntraeger wrote:
>> On 24.04.21 01:02, Jason Gunthorpe wrote:
>>> Prologue
>>> ========
>>>
>>> This is series #3 in part of a larger work that arose from the minor
>>> remark that the mdev_parent_ops indirection shim is useless and
>>> complicates things.
>>>
>>> It applies on top of Alex's current tree and requires the prior two
>>> series.
>>
>> Do you have a tree somewhere?
>
> [..]
>>> A preview of the future series's is here:
>>> https://github.com/jgunthorpe/linux/pull/3/commits
>
> Has everything, you'll want to go to:
> cover-letter: Remove vfio_mdev.c, mdev_parent_ops and more
>
> As there are additional WIPs in that tree.
I gave this a quick spin on s390x vfio-ap and it seems to work ok.
This is really just a quick test, but no obvious problem.
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
2021-04-27 7:33 ` Christian Borntraeger
(?)
@ 2021-04-27 23:21 ` Jason Gunthorpe
-1 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-27 23:21 UTC (permalink / raw)
To: Christian Borntraeger
Cc: David Airlie, Tony Krowiak, Alex Williamson, Cornelia Huck,
Jonathan Corbet, Daniel Vetter, dri-devel, Eric Farman,
Harald Freudenberger, Vasily Gorbik, Heiko Carstens, intel-gfx,
intel-gvt-dev, Jani Nikula, Joonas Lahtinen, kvm, Kirti Wankhede,
linux-doc, linux-s390, Peter Oberparleiter, Halil Pasic,
Pierre Morel, Rodrigo Vivi, Vineeth Vijayan, Zhenyu Wang,
Zhi Wang, Raj, Ashok, Dan Williams, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Tue, Apr 27, 2021 at 09:33:56AM +0200, Christian Borntraeger wrote:
> On 26.04.21 19:42, Jason Gunthorpe wrote:
> > On Mon, Apr 26, 2021 at 06:43:14PM +0200, Christian Borntraeger wrote:
> > > On 24.04.21 01:02, Jason Gunthorpe wrote:
> > > > Prologue
> > > > ========
> > > >
> > > > This is series #3 in part of a larger work that arose from the minor
> > > > remark that the mdev_parent_ops indirection shim is useless and
> > > > complicates things.
> > > >
> > > > It applies on top of Alex's current tree and requires the prior two
> > > > series.
> > >
> > > Do you have a tree somewhere?
> >
> > [..]
> > > > A preview of the future series's is here:
> > > > https://github.com/jgunthorpe/linux/pull/3/commits
> >
> > Has everything, you'll want to go to:
> > cover-letter: Remove vfio_mdev.c, mdev_parent_ops and more
> >
> > As there are additional WIPs in that tree.
>
> I gave this a quick spin on s390x vfio-ap and it seems to work ok.
> This is really just a quick test, but no obvious problem.
Great! Thanks
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-27 23:21 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-27 23:21 UTC (permalink / raw)
To: Christian Borntraeger
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Vineeth Vijayan, Leon Romanovsky, Christoph Hellwig, linux-s390,
Raj, Ashok, Jonathan Corbet, Halil Pasic, Tarun Gupta, intel-gfx,
Zhi Wang, Max Gurtovoy, Eric Farman, Vasily Gorbik,
Heiko Carstens, Alex Williamson, Harald Freudenberger,
Rodrigo Vivi, Dan Williams, intel-gvt-dev, Tony Krowiak,
Pierre Morel, Cornelia Huck, Peter Oberparleiter
On Tue, Apr 27, 2021 at 09:33:56AM +0200, Christian Borntraeger wrote:
> On 26.04.21 19:42, Jason Gunthorpe wrote:
> > On Mon, Apr 26, 2021 at 06:43:14PM +0200, Christian Borntraeger wrote:
> > > On 24.04.21 01:02, Jason Gunthorpe wrote:
> > > > Prologue
> > > > ========
> > > >
> > > > This is series #3 in part of a larger work that arose from the minor
> > > > remark that the mdev_parent_ops indirection shim is useless and
> > > > complicates things.
> > > >
> > > > It applies on top of Alex's current tree and requires the prior two
> > > > series.
> > >
> > > Do you have a tree somewhere?
> >
> > [..]
> > > > A preview of the future series's is here:
> > > > https://github.com/jgunthorpe/linux/pull/3/commits
> >
> > Has everything, you'll want to go to:
> > cover-letter: Remove vfio_mdev.c, mdev_parent_ops and more
> >
> > As there are additional WIPs in that tree.
>
> I gave this a quick spin on s390x vfio-ap and it seems to work ok.
> This is really just a quick test, but no obvious problem.
Great! Thanks
Jason
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [Intel-gfx] [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more
@ 2021-04-27 23:21 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-04-27 23:21 UTC (permalink / raw)
To: Christian Borntraeger
Cc: kvm, linux-doc, David Airlie, dri-devel, Kirti Wankhede,
Vineeth Vijayan, Leon Romanovsky, Christoph Hellwig, linux-s390,
Raj, Ashok, Jonathan Corbet, Halil Pasic, Tarun Gupta, intel-gfx,
Max Gurtovoy, Eric Farman, Vasily Gorbik, Heiko Carstens,
Harald Freudenberger, Dan Williams, intel-gvt-dev, Tony Krowiak,
Pierre Morel, Cornelia Huck, Peter Oberparleiter
On Tue, Apr 27, 2021 at 09:33:56AM +0200, Christian Borntraeger wrote:
> On 26.04.21 19:42, Jason Gunthorpe wrote:
> > On Mon, Apr 26, 2021 at 06:43:14PM +0200, Christian Borntraeger wrote:
> > > On 24.04.21 01:02, Jason Gunthorpe wrote:
> > > > Prologue
> > > > ========
> > > >
> > > > This is series #3 in part of a larger work that arose from the minor
> > > > remark that the mdev_parent_ops indirection shim is useless and
> > > > complicates things.
> > > >
> > > > It applies on top of Alex's current tree and requires the prior two
> > > > series.
> > >
> > > Do you have a tree somewhere?
> >
> > [..]
> > > > A preview of the future series's is here:
> > > > https://github.com/jgunthorpe/linux/pull/3/commits
> >
> > Has everything, you'll want to go to:
> > cover-letter: Remove vfio_mdev.c, mdev_parent_ops and more
> >
> > As there are additional WIPs in that tree.
>
> I gave this a quick spin on s390x vfio-ap and it seems to work ok.
> This is really just a quick test, but no obvious problem.
Great! Thanks
Jason
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-04-26 17:48 ` Cornelia Huck
2021-04-26 18:10 ` Jason Gunthorpe
2021-04-26 23:41 ` Halil Pasic
@ 2021-05-03 20:14 ` Tony Krowiak
2021-05-03 20:33 ` Jason Gunthorpe
` (2 more replies)
2 siblings, 3 replies; 74+ messages in thread
From: Tony Krowiak @ 2021-05-03 20:14 UTC (permalink / raw)
To: Cornelia Huck, Jason Gunthorpe
Cc: Christian Borntraeger, Harald Freudenberger, Vasily Gorbik,
Heiko Carstens, linux-s390, Halil Pasic, Pierre Morel, Raj,
Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta, Jason J . Herne
On 4/26/21 1:48 PM, Cornelia Huck wrote:
> On Fri, 23 Apr 2021 20:03:03 -0300
> Jason Gunthorpe <jgg@nvidia.com> wrote:
>
>> This is straightforward conversion, the ap_matrix_mdev is actually serving
>> as the vfio_device and we can replace all the mdev_get_drvdata()'s with a
>> simple container_of().
>>
>> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
>> ---
>> drivers/s390/crypto/vfio_ap_ops.c | 137 ++++++++++++++++----------
>> drivers/s390/crypto/vfio_ap_private.h | 2 +
>> 2 files changed, 89 insertions(+), 50 deletions(-)
>>
> (...)
>
>> -static int vfio_ap_mdev_remove(struct mdev_device *mdev)
>> +static void vfio_ap_mdev_remove(struct mdev_device *mdev)
>> {
>> - struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
>> + struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(&mdev->dev);
>>
>> - if (matrix_mdev->kvm)
>> - return -EBUSY;
>> + /* FIXME: Remove isn't allowed to fail */
>> + if (WARN_ON(matrix_mdev->kvm))
>> + return;
> This is a pre-existing problem, but the rework now makes it more
> obvious.
I agree, I was not aware that returning a non-zero return code
from this callback did not return the -EBUSY to userspace
when the mdev is removed.
>
> Previously, the mdev code would only print a warning and then continue
> with device removal, even if a ->remove() callback returned an error.
> Now, it's quite clear that we'll end up in a weird half-dead state.
With the latest kernel from our tree, the remove hangs until the
guest is shutdown and the mdev fd is closed. During the hang, the
dmesg log has the following:
"No mdev vendor driver request callback support, blocked until released
by user"
So, it looks like nothing is done with the mdev until the fd for the
mdev is closed when the guest is shut down, at which time the
mdev is removed.
>
> IIRC, the check for matrix_mdev->kvm is intended to protect against
> ripping out the device under a running guest (I think it needs to
> manipulate some crypto control blocks?)
This is correct.
>
> So my question for the vfio-ap maintainers is: Can we actually end up
> in this case? If yes, is there any way to gracefully shut down the
> device?
This case will occur whenever a user removes the mdev
by echoing a '1' into the mdev's sysfs 'remove' attribute
file. I'm not sure it can be considered graceful to take away
all of the crypto devices from a guest while it is running,
but there is a way to process the remove callback without
leaving things in a "weird, half-dead state".
Up to this point, the onus for ensuring the proper procedure
is followed when managing pass-through crypto devices
for a KVM guest is left to the system administrator. In
other words, we don't prevent an admin from shooting
him/herself in the foot when doing things such as removing
an mdev while a KVM guest is using it. With this in mind,
I will handle this case in the follow-on patches implementing
dynamic AP configuration support for KVM guests.
>
>> +
>> + vfio_unregister_group_dev(&matrix_mdev->vdev);
>>
>> mutex_lock(&matrix_dev->lock);
>> - vfio_ap_mdev_reset_queues(mdev);
>> + vfio_ap_mdev_reset_queues(matrix_mdev);
>> list_del(&matrix_mdev->node);
>> mutex_unlock(&matrix_dev->lock);
>>
>> kfree(matrix_mdev);
>> - mdev_set_drvdata(mdev, NULL);
>> atomic_inc(&matrix_dev->available_instances);
>> -
>> - return 0;
>> }
> (...)
>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-05-03 20:14 ` Tony Krowiak
@ 2021-05-03 20:33 ` Jason Gunthorpe
2021-05-04 13:58 ` Tony Krowiak
2021-05-04 15:30 ` Cornelia Huck
2021-05-05 16:28 ` Tony Krowiak
2 siblings, 1 reply; 74+ messages in thread
From: Jason Gunthorpe @ 2021-05-03 20:33 UTC (permalink / raw)
To: Tony Krowiak
Cc: Cornelia Huck, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta,
Jason J . Herne
On Mon, May 03, 2021 at 04:14:43PM -0400, Tony Krowiak wrote:
> This case will occur whenever a user removes the mdev
> by echoing a '1' into the mdev's sysfs 'remove' attribute
> file. I'm not sure it can be considered graceful to take away
> all of the crypto devices from a guest while it is running,
> but there is a way to process the remove callback without
> leaving things in a "weird, half-dead state".
It is acceptable to just sleep here until whatever user controlled
condition is resolved.
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-05-03 20:33 ` Jason Gunthorpe
@ 2021-05-04 13:58 ` Tony Krowiak
2021-05-04 16:04 ` Jason Gunthorpe
0 siblings, 1 reply; 74+ messages in thread
From: Tony Krowiak @ 2021-05-04 13:58 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Cornelia Huck, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta,
Jason J . Herne
On 5/3/21 4:33 PM, Jason Gunthorpe wrote:
> On Mon, May 03, 2021 at 04:14:43PM -0400, Tony Krowiak wrote:
>
>> This case will occur whenever a user removes the mdev
>> by echoing a '1' into the mdev's sysfs 'remove' attribute
>> file. I'm not sure it can be considered graceful to take away
>> all of the crypto devices from a guest while it is running,
>> but there is a way to process the remove callback without
>> leaving things in a "weird, half-dead state".
> It is acceptable to just sleep here until whatever user controlled
> condition is resolved.
>
> Jason
I suppose we could do that, but the user that tried to remove
the mdev via its sysfs 'remove' attribute will be left sitting
there wondering why the operation didn't complete. That
could result in leaving the user hanging in perpetuity.
IMHO, the callback should continue to return an int and
the caller should display the error if a non-zero rc is
returned.
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-05-03 20:14 ` Tony Krowiak
2021-05-03 20:33 ` Jason Gunthorpe
@ 2021-05-04 15:30 ` Cornelia Huck
2021-05-05 12:30 ` Tony Krowiak
2021-05-05 16:28 ` Tony Krowiak
2 siblings, 1 reply; 74+ messages in thread
From: Cornelia Huck @ 2021-05-04 15:30 UTC (permalink / raw)
To: Tony Krowiak
Cc: Jason Gunthorpe, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta,
Jason J . Herne
On Mon, 3 May 2021 16:14:43 -0400
Tony Krowiak <akrowiak@linux.ibm.com> wrote:
> On 4/26/21 1:48 PM, Cornelia Huck wrote:
> > On Fri, 23 Apr 2021 20:03:03 -0300
> > Jason Gunthorpe <jgg@nvidia.com> wrote:
> >
> >> This is straightforward conversion, the ap_matrix_mdev is actually serving
> >> as the vfio_device and we can replace all the mdev_get_drvdata()'s with a
> >> simple container_of().
> >>
> >> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> >> ---
> >> drivers/s390/crypto/vfio_ap_ops.c | 137 ++++++++++++++++----------
> >> drivers/s390/crypto/vfio_ap_private.h | 2 +
> >> 2 files changed, 89 insertions(+), 50 deletions(-)
> >>
> > (...)
> >
> >> -static int vfio_ap_mdev_remove(struct mdev_device *mdev)
> >> +static void vfio_ap_mdev_remove(struct mdev_device *mdev)
> >> {
> >> - struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
> >> + struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(&mdev->dev);
> >>
> >> - if (matrix_mdev->kvm)
> >> - return -EBUSY;
> >> + /* FIXME: Remove isn't allowed to fail */
> >> + if (WARN_ON(matrix_mdev->kvm))
> >> + return;
> > This is a pre-existing problem, but the rework now makes it more
> > obvious.
>
> I agree, I was not aware that returning a non-zero return code
> from this callback did not return the -EBUSY to userspace
> when the mdev is removed.
>
> >
> > Previously, the mdev code would only print a warning and then continue
> > with device removal, even if a ->remove() callback returned an error.
> > Now, it's quite clear that we'll end up in a weird half-dead state.
>
> With the latest kernel from our tree, the remove hangs until the
> guest is shutdown and the mdev fd is closed. During the hang, the
> dmesg log has the following:
>
> "No mdev vendor driver request callback support, blocked until released
> by user"
>
> So, it looks like nothing is done with the mdev until the fd for the
> mdev is closed when the guest is shut down, at which time the
> mdev is removed.
You probably want to wire up the request callback and notify userspace.
What happens today if the device in QEMU is removed via device_del?
Does that already clean up things properly?
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-05-04 13:58 ` Tony Krowiak
@ 2021-05-04 16:04 ` Jason Gunthorpe
2021-05-05 13:07 ` Tony Krowiak
0 siblings, 1 reply; 74+ messages in thread
From: Jason Gunthorpe @ 2021-05-04 16:04 UTC (permalink / raw)
To: Tony Krowiak
Cc: Cornelia Huck, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta,
Jason J . Herne
On Tue, May 04, 2021 at 09:58:45AM -0400, Tony Krowiak wrote:
>
>
> On 5/3/21 4:33 PM, Jason Gunthorpe wrote:
> > On Mon, May 03, 2021 at 04:14:43PM -0400, Tony Krowiak wrote:
> >
> > > This case will occur whenever a user removes the mdev
> > > by echoing a '1' into the mdev's sysfs 'remove' attribute
> > > file. I'm not sure it can be considered graceful to take away
> > > all of the crypto devices from a guest while it is running,
> > > but there is a way to process the remove callback without
> > > leaving things in a "weird, half-dead state".
> > It is acceptable to just sleep here until whatever user controlled
> > condition is resolved.
> >
> > Jason
>
> I suppose we could do that, but the user that tried to remove
> the mdev via its sysfs 'remove' attribute will be left sitting
> there wondering why the operation didn't complete. That
> could result in leaving the user hanging in perpetuity.
Yes.
If the driver can't implement a disconnection then that is
unavoidable. What it does today by leaking memory under user control
is not acceptable.
> IMHO, the callback should continue to return an int and
> the caller should display the error if a non-zero rc is
> returned.
Nope, there is a reason removal is not allowed to fail.. sysfs remove
isn't the only reason the mdev driver could be destroyed, the
underlying physical device could be unplugged or other things.
Drivers need to implement a proper remove.
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-05-04 15:30 ` Cornelia Huck
@ 2021-05-05 12:30 ` Tony Krowiak
2021-05-05 17:47 ` Jason Gunthorpe
0 siblings, 1 reply; 74+ messages in thread
From: Tony Krowiak @ 2021-05-05 12:30 UTC (permalink / raw)
To: Cornelia Huck
Cc: Jason Gunthorpe, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta,
Jason J . Herne
On 5/4/21 11:30 AM, Cornelia Huck wrote:
> On Mon, 3 May 2021 16:14:43 -0400
> Tony Krowiak <akrowiak@linux.ibm.com> wrote:
>
>> On 4/26/21 1:48 PM, Cornelia Huck wrote:
>>> On Fri, 23 Apr 2021 20:03:03 -0300
>>> Jason Gunthorpe <jgg@nvidia.com> wrote:
>>>
>>>> This is straightforward conversion, the ap_matrix_mdev is actually serving
>>>> as the vfio_device and we can replace all the mdev_get_drvdata()'s with a
>>>> simple container_of().
>>>>
>>>> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
>>>> ---
>>>> drivers/s390/crypto/vfio_ap_ops.c | 137 ++++++++++++++++----------
>>>> drivers/s390/crypto/vfio_ap_private.h | 2 +
>>>> 2 files changed, 89 insertions(+), 50 deletions(-)
>>>>
>>> (...)
>>>
>>>> -static int vfio_ap_mdev_remove(struct mdev_device *mdev)
>>>> +static void vfio_ap_mdev_remove(struct mdev_device *mdev)
>>>> {
>>>> - struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
>>>> + struct ap_matrix_mdev *matrix_mdev = dev_get_drvdata(&mdev->dev);
>>>>
>>>> - if (matrix_mdev->kvm)
>>>> - return -EBUSY;
>>>> + /* FIXME: Remove isn't allowed to fail */
>>>> + if (WARN_ON(matrix_mdev->kvm))
>>>> + return;
>>> This is a pre-existing problem, but the rework now makes it more
>>> obvious.
>> I agree, I was not aware that returning a non-zero return code
>> from this callback did not return the -EBUSY to userspace
>> when the mdev is removed.
>>
>>> Previously, the mdev code would only print a warning and then continue
>>> with device removal, even if a ->remove() callback returned an error.
>>> Now, it's quite clear that we'll end up in a weird half-dead state.
>> With the latest kernel from our tree, the remove hangs until the
>> guest is shutdown and the mdev fd is closed. During the hang, the
>> dmesg log has the following:
>>
>> "No mdev vendor driver request callback support, blocked until released
>> by user"
>>
>> So, it looks like nothing is done with the mdev until the fd for the
>> mdev is closed when the guest is shut down, at which time the
>> mdev is removed.
> You probably want to wire up the request callback and notify userspace.
Not sure what you mean by this, but I also don't think it matters.
After coding up the fix for this and testing it, I've learned that
if a user attempts to remove an mdev via the sysfs 'remove'
attribute while the mdev fd is still open (i.e., in use by the guest),
the mdev remove callback is not invoked until the fd is closed
(i.e., the guest is shut down). During that time, the mdev is
physically removed from sysfs so no further actions can be taken
on it; but, since the remove callback has yet to be called, the
guest will have access to the AP resources provided by the
mdev during that time. I also tested detaching the mdev device from the
guest
(i.e., virsh detach-device) while the mdev was in the process of
being removed and this resulted in allowing the remove to
progress due to the mdev fd getting closed when it is detached.
>
> What happens today if the device in QEMU is removed via device_del?
> Does that already clean up things properly?
Hot plug/unplug is already supported, so yes, things get cleaned up
properly when the mdev fd is closed. This is handled by the mdev
release callback.
>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-05-04 16:04 ` Jason Gunthorpe
@ 2021-05-05 13:07 ` Tony Krowiak
0 siblings, 0 replies; 74+ messages in thread
From: Tony Krowiak @ 2021-05-05 13:07 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Cornelia Huck, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta,
Jason J . Herne
On 5/4/21 12:04 PM, Jason Gunthorpe wrote:
> On Tue, May 04, 2021 at 09:58:45AM -0400, Tony Krowiak wrote:
>>
>> On 5/3/21 4:33 PM, Jason Gunthorpe wrote:
>>> On Mon, May 03, 2021 at 04:14:43PM -0400, Tony Krowiak wrote:
>>>
>>>> This case will occur whenever a user removes the mdev
>>>> by echoing a '1' into the mdev's sysfs 'remove' attribute
>>>> file. I'm not sure it can be considered graceful to take away
>>>> all of the crypto devices from a guest while it is running,
>>>> but there is a way to process the remove callback without
>>>> leaving things in a "weird, half-dead state".
>>> It is acceptable to just sleep here until whatever user controlled
>>> condition is resolved.
>>>
>>> Jason
>> I suppose we could do that, but the user that tried to remove
>> the mdev via its sysfs 'remove' attribute will be left sitting
>> there wondering why the operation didn't complete. That
>> could result in leaving the user hanging in perpetuity.
> Yes.
>
> If the driver can't implement a disconnection then that is
> unavoidable. What it does today by leaking memory under user control
> is not acceptable.
Based upon my observations of the behavior during a removal
of the mdev, memory is not leaked. If the fd for the mdev is
open when an attempt is made to remove it, the operation
will hang until the mdev fd is closed which happens when
either the guest is shut down or the mdev device is detached from
the guest. When the fd is closed, the mdev release callback is
invoked which nullifies the KVM pointer, so when the remove callback
is subsequently invoked, the mdev resources will be cleaned up.
Of course, I imagine there are other possibilities
for how an mdev can be removed, but in the normal course of
events, memory will not be leaked.
>
>> IMHO, the callback should continue to return an int and
>> the caller should display the error if a non-zero rc is
>> returned.
> Nope, there is a reason removal is not allowed to fail.. sysfs remove
> isn't the only reason the mdev driver could be destroyed, the
> underlying physical device could be unplugged or other things.
That may be true with other devices, but the matrix device is
not a real, physical device. Its sole purpose in life is to provide
an anchor for the mdev devices used to provide AP resources
to a guest; however, I get your point.
>
> Drivers need to implement a proper remove.
>
> Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-05-03 20:14 ` Tony Krowiak
2021-05-03 20:33 ` Jason Gunthorpe
2021-05-04 15:30 ` Cornelia Huck
@ 2021-05-05 16:28 ` Tony Krowiak
2 siblings, 0 replies; 74+ messages in thread
From: Tony Krowiak @ 2021-05-05 16:28 UTC (permalink / raw)
To: Cornelia Huck, Jason Gunthorpe
Cc: Christian Borntraeger, Harald Freudenberger, Vasily Gorbik,
Heiko Carstens, linux-s390, Halil Pasic, Pierre Morel, Raj,
Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta, Jason J . Herne
On 5/3/21 4:14 PM, Tony Krowiak wrote:
...
>
> Up to this point, the onus for ensuring the proper procedure
> is followed when managing pass-through crypto devices
> for a KVM guest is left to the system administrator. In
> other words, we don't prevent an admin from shooting
> him/herself in the foot when doing things such as removing
> an mdev while a KVM guest is using it. With this in mind,
> I will handle this case in the follow-on patches implementing
> dynamic AP configuration support for KVM guests.
Change of plans: Based upon my exchanges with Jason, I have
decided to create a patch specifically to do the cleanup in
the mdev remove callback regardless of whether a KVM guest
is using the mdev or not.
>
>>
>
^ permalink raw reply [flat|nested] 74+ messages in thread
* Re: [PATCH 06/12] vfio/ap_ops: Convert to use vfio_register_group_dev()
2021-05-05 12:30 ` Tony Krowiak
@ 2021-05-05 17:47 ` Jason Gunthorpe
0 siblings, 0 replies; 74+ messages in thread
From: Jason Gunthorpe @ 2021-05-05 17:47 UTC (permalink / raw)
To: Tony Krowiak
Cc: Cornelia Huck, Christian Borntraeger, Harald Freudenberger,
Vasily Gorbik, Heiko Carstens, linux-s390, Halil Pasic,
Pierre Morel, Raj, Ashok, Dan Williams, Daniel Vetter,
Christoph Hellwig, Leon Romanovsky, Max Gurtovoy, Tarun Gupta,
Jason J . Herne
On Wed, May 05, 2021 at 08:30:54AM -0400, Tony Krowiak wrote:
> On 5/4/21 11:30 AM, Cornelia Huck wrote:
> > > So, it looks like nothing is done with the mdev until the fd for the
> > > mdev is closed when the guest is shut down, at which time the
> > > mdev is removed.
> > You probably want to wire up the request callback and notify userspace.
>
> Not sure what you mean by this, but I also don't think it matters.
request is triggered by the vfio core to expedite closing of the FD in
userspace.
> After coding up the fix for this and testing it, I've learned that
> if a user attempts to remove an mdev via the sysfs 'remove'
> attribute while the mdev fd is still open (i.e., in use by the
> guest), the mdev remove callback is not invoked until the fd is
> closed
Right the vfio_del_group_dev() called along this path and it blocks
until all FD related accesses are prevented. After it returns no ops
callback can be invoked.
> (i.e., the guest is shut down). During that time, the mdev is
> physically removed from sysfs so no further actions can be taken
> on it;
sysfs files are removed before we get to the remove callback - this is
part of why remove cannot fail, the damage is already done.
Jason
^ permalink raw reply [flat|nested] 74+ messages in thread
end of thread, other threads:[~2021-05-05 17:49 UTC | newest]
Thread overview: 74+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-23 23:02 [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more Jason Gunthorpe
2021-04-23 23:02 ` [Intel-gfx] " Jason Gunthorpe
2021-04-23 23:02 ` Jason Gunthorpe
2021-04-23 23:02 ` [PATCH 01/12] vfio/mdev: Remove CONFIG_VFIO_MDEV_DEVICE Jason Gunthorpe
2021-04-23 23:02 ` [Intel-gfx] " Jason Gunthorpe
2021-04-23 23:02 ` Jason Gunthorpe
2021-04-24 0:08 ` Randy Dunlap
2021-04-24 0:08 ` [Intel-gfx] " Randy Dunlap
2021-04-24 0:08 ` Randy Dunlap
2021-04-26 18:26 ` Jason Gunthorpe
2021-04-26 18:26 ` [Intel-gfx] " Jason Gunthorpe
2021-04-26 18:26 ` Jason Gunthorpe
2021-04-26 19:11 ` Randy Dunlap
2021-04-26 19:11 ` [Intel-gfx] " Randy Dunlap
2021-04-26 19:11 ` Randy Dunlap
2021-04-23 23:02 ` [PATCH 02/12] vfio/mdev: Allow the mdev_parent_ops to specify the device driver to bind Jason Gunthorpe
2021-04-26 14:02 ` Christoph Hellwig
2021-04-26 14:11 ` Jason Gunthorpe
2021-04-23 23:03 ` [PATCH 03/12] vfio/mtty: Convert to use vfio_register_group_dev() Jason Gunthorpe
2021-04-26 14:06 ` Christoph Hellwig
2021-04-23 23:03 ` [PATCH 04/12] vfio/mdpy: " Jason Gunthorpe
2021-04-26 14:06 ` Christoph Hellwig
2021-04-23 23:03 ` [PATCH 05/12] vfio/mbochs: " Jason Gunthorpe
2021-04-26 14:06 ` Christoph Hellwig
2021-04-23 23:03 ` [PATCH 06/12] vfio/ap_ops: " Jason Gunthorpe
2021-04-26 14:07 ` Christoph Hellwig
2021-04-26 17:48 ` Cornelia Huck
2021-04-26 18:10 ` Jason Gunthorpe
2021-04-26 23:41 ` Halil Pasic
2021-05-03 20:14 ` Tony Krowiak
2021-05-03 20:33 ` Jason Gunthorpe
2021-05-04 13:58 ` Tony Krowiak
2021-05-04 16:04 ` Jason Gunthorpe
2021-05-05 13:07 ` Tony Krowiak
2021-05-04 15:30 ` Cornelia Huck
2021-05-05 12:30 ` Tony Krowiak
2021-05-05 17:47 ` Jason Gunthorpe
2021-05-05 16:28 ` Tony Krowiak
2021-04-23 23:03 ` [PATCH 07/12] vfio/ccw: " Jason Gunthorpe
2021-04-23 23:03 ` [PATCH 08/12] vfio/gvt: " Jason Gunthorpe
2021-04-23 23:03 ` [Intel-gfx] " Jason Gunthorpe
2021-04-23 23:03 ` Jason Gunthorpe
2021-04-26 14:13 ` Christoph Hellwig
2021-04-26 14:13 ` [Intel-gfx] " Christoph Hellwig
2021-04-26 15:44 ` Jason Gunthorpe
2021-04-26 15:44 ` [Intel-gfx] " Jason Gunthorpe
2021-04-26 15:44 ` Jason Gunthorpe
2021-04-23 23:03 ` [PATCH 09/12] vfio/mdev: Remove mdev_parent_ops dev_attr_groups Jason Gunthorpe
2021-04-26 14:14 ` Christoph Hellwig
2021-04-23 23:03 ` [PATCH 10/12] vfio/mdev: Remove mdev_parent_ops Jason Gunthorpe
2021-04-23 23:03 ` [Intel-gfx] " Jason Gunthorpe
2021-04-23 23:03 ` Jason Gunthorpe
2021-04-26 14:19 ` Christoph Hellwig
2021-04-26 14:19 ` [Intel-gfx] " Christoph Hellwig
2021-04-26 18:33 ` Jason Gunthorpe
2021-04-26 18:33 ` [Intel-gfx] " Jason Gunthorpe
2021-04-26 18:33 ` Jason Gunthorpe
2021-04-23 23:03 ` [PATCH 11/12] vfio/mdev: Use the driver core to create the 'remove' file Jason Gunthorpe
2021-04-26 14:20 ` Christoph Hellwig
2021-04-26 19:07 ` Jason Gunthorpe
2021-04-23 23:03 ` [PATCH 12/12] vfio/mdev: Remove mdev drvdata Jason Gunthorpe
2021-04-26 14:20 ` Christoph Hellwig
2021-04-26 16:43 ` [PATCH 00/12] Remove vfio_mdev.c, mdev_parent_ops and more Christian Borntraeger
2021-04-26 16:43 ` [Intel-gfx] " Christian Borntraeger
2021-04-26 16:43 ` Christian Borntraeger
2021-04-26 17:42 ` Jason Gunthorpe
2021-04-26 17:42 ` [Intel-gfx] " Jason Gunthorpe
2021-04-26 17:42 ` Jason Gunthorpe
2021-04-27 7:33 ` Christian Borntraeger
2021-04-27 7:33 ` [Intel-gfx] " Christian Borntraeger
2021-04-27 7:33 ` Christian Borntraeger
2021-04-27 23:21 ` Jason Gunthorpe
2021-04-27 23:21 ` [Intel-gfx] " Jason Gunthorpe
2021-04-27 23:21 ` Jason Gunthorpe
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.