* [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior @ 2022-06-06 17:53 ` Alex Williamson 0 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-06 17:53 UTC (permalink / raw) To: maarten.lankhorst, mripard, tzimmermann, airlied, daniel Cc: Laszlo Ersek, Gerd Hoffmann, dri-devel, linux-kernel, kvm Users attempting to enable vfio PCI device assignment with a GPU will often block the default PCI driver from the device to avoid conflicts with the device initialization or release path. This means that vfio-pci is sometimes the first PCI driver to bind to the device. In the case of assigning the primary graphics device, low-level console drivers may still generate resource conflicts. Users often employ kernel command line arguments to disable conflicting drivers or perform unbinding in userspace to avoid this, but the actual solution is often distribution/kernel config specific based on the included drivers. We can instead allow vfio-pci to copy the behavior of drm_aperture_remove_conflicting_pci_framebuffers() in order to remove these low-level drivers with conflicting resources. vfio-pci is not however a DRM driver, nor does vfio-pci depend on DRM config options, thus we split out and export the necessary DRM apterture support and mirror the framebuffer and VGA support. I'd be happy to pull this series in through the vfio branch if approved by the DRM maintainers. Thanks, Alex --- Alex Williamson (2): drm/aperture: Split conflicting platform driver removal vfio/pci: Remove console drivers drivers/gpu/drm/drm_aperture.c | 33 +++++++++++++++++++++++--------- drivers/vfio/pci/vfio_pci_core.c | 17 ++++++++++++++++ include/drm/drm_aperture.h | 2 ++ 3 files changed, 43 insertions(+), 9 deletions(-) ^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior @ 2022-06-06 17:53 ` Alex Williamson 0 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-06 17:53 UTC (permalink / raw) To: maarten.lankhorst, mripard, tzimmermann, airlied, daniel Cc: kvm, Laszlo Ersek, Gerd Hoffmann, dri-devel, linux-kernel Users attempting to enable vfio PCI device assignment with a GPU will often block the default PCI driver from the device to avoid conflicts with the device initialization or release path. This means that vfio-pci is sometimes the first PCI driver to bind to the device. In the case of assigning the primary graphics device, low-level console drivers may still generate resource conflicts. Users often employ kernel command line arguments to disable conflicting drivers or perform unbinding in userspace to avoid this, but the actual solution is often distribution/kernel config specific based on the included drivers. We can instead allow vfio-pci to copy the behavior of drm_aperture_remove_conflicting_pci_framebuffers() in order to remove these low-level drivers with conflicting resources. vfio-pci is not however a DRM driver, nor does vfio-pci depend on DRM config options, thus we split out and export the necessary DRM apterture support and mirror the framebuffer and VGA support. I'd be happy to pull this series in through the vfio branch if approved by the DRM maintainers. Thanks, Alex --- Alex Williamson (2): drm/aperture: Split conflicting platform driver removal vfio/pci: Remove console drivers drivers/gpu/drm/drm_aperture.c | 33 +++++++++++++++++++++++--------- drivers/vfio/pci/vfio_pci_core.c | 17 ++++++++++++++++ include/drm/drm_aperture.h | 2 ++ 3 files changed, 43 insertions(+), 9 deletions(-) ^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH 1/2] drm/aperture: Split conflicting platform driver removal 2022-06-06 17:53 ` Alex Williamson @ 2022-06-06 17:53 ` Alex Williamson -1 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-06 17:53 UTC (permalink / raw) To: maarten.lankhorst, mripard, tzimmermann, airlied, daniel Cc: Laszlo Ersek, Laszlo Ersek, Gerd Hoffmann, dri-devel, linux-kernel, kvm Split the removal of platform drivers conflicting with PCI resources from drm_aperture_remove_conflicting_pci_framebuffers() to support both non-DRM callers and better modularity. This is intended to support the vfio-pci driver, which can acquire ownership of PCI graphics devices, but is not itself a DRM or FB driver, and therefore has no Kconfig dependencies. The remaining actions of drm_aperture_remove_conflicting_pci_framebuffers() are already exported from their respective subsystems, therefore this allows vfio-pci to separate each set of conflicts independently based on the configured features. Reported-by: Laszlo Ersek <lersek@redhat.com> Tested-by: Laszlo Ersek <lersek@redhat.com> Suggested-by: Gerd Hoffmann <kraxel@redhat.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com> --- drivers/gpu/drm/drm_aperture.c | 33 ++++++++++++++++++++++++--------- include/drm/drm_aperture.h | 2 ++ 2 files changed, 26 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/drm_aperture.c b/drivers/gpu/drm/drm_aperture.c index 74bd4a76b253..5b2500f7fb8b 100644 --- a/drivers/gpu/drm/drm_aperture.c +++ b/drivers/gpu/drm/drm_aperture.c @@ -313,6 +313,28 @@ int drm_aperture_remove_conflicting_framebuffers(resource_size_t base, resource_ } EXPORT_SYMBOL(drm_aperture_remove_conflicting_framebuffers); +/** + * drm_aperture_detach_platform_drivers - detach platform drivers conflicting with PCI device + * @pdev: PCI device + * + * This function detaches platform drivers with resource conflicts to the memory + * bars of the provided @pdev. + */ +void drm_aperture_detach_platform_drivers(struct pci_dev *pdev) +{ + resource_size_t base, size; + int bar; + + for (bar = 0; bar < PCI_STD_NUM_BARS; ++bar) { + if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) + continue; + base = pci_resource_start(pdev, bar); + size = pci_resource_len(pdev, bar); + drm_aperture_detach_drivers(base, size); + } +} +EXPORT_SYMBOL(drm_aperture_detach_platform_drivers); + /** * drm_aperture_remove_conflicting_pci_framebuffers - remove existing framebuffers for PCI devices * @pdev: PCI device @@ -328,16 +350,9 @@ EXPORT_SYMBOL(drm_aperture_remove_conflicting_framebuffers); int drm_aperture_remove_conflicting_pci_framebuffers(struct pci_dev *pdev, const struct drm_driver *req_driver) { - resource_size_t base, size; - int bar, ret = 0; + int ret = 0; - for (bar = 0; bar < PCI_STD_NUM_BARS; ++bar) { - if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) - continue; - base = pci_resource_start(pdev, bar); - size = pci_resource_len(pdev, bar); - drm_aperture_detach_drivers(base, size); - } + drm_aperture_detach_platform_drivers(pdev); /* * WARNING: Apparently we must kick fbdev drivers before vgacon, diff --git a/include/drm/drm_aperture.h b/include/drm/drm_aperture.h index 7096703c3949..53fd36fa258e 100644 --- a/include/drm/drm_aperture.h +++ b/include/drm/drm_aperture.h @@ -15,6 +15,8 @@ int devm_aperture_acquire_from_firmware(struct drm_device *dev, resource_size_t int drm_aperture_remove_conflicting_framebuffers(resource_size_t base, resource_size_t size, bool primary, const struct drm_driver *req_driver); +void drm_aperture_detach_platform_drivers(struct pci_dev *pdev); + int drm_aperture_remove_conflicting_pci_framebuffers(struct pci_dev *pdev, const struct drm_driver *req_driver); ^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 1/2] drm/aperture: Split conflicting platform driver removal @ 2022-06-06 17:53 ` Alex Williamson 0 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-06 17:53 UTC (permalink / raw) To: maarten.lankhorst, mripard, tzimmermann, airlied, daniel Cc: kvm, Laszlo Ersek, Gerd Hoffmann, dri-devel, linux-kernel Split the removal of platform drivers conflicting with PCI resources from drm_aperture_remove_conflicting_pci_framebuffers() to support both non-DRM callers and better modularity. This is intended to support the vfio-pci driver, which can acquire ownership of PCI graphics devices, but is not itself a DRM or FB driver, and therefore has no Kconfig dependencies. The remaining actions of drm_aperture_remove_conflicting_pci_framebuffers() are already exported from their respective subsystems, therefore this allows vfio-pci to separate each set of conflicts independently based on the configured features. Reported-by: Laszlo Ersek <lersek@redhat.com> Tested-by: Laszlo Ersek <lersek@redhat.com> Suggested-by: Gerd Hoffmann <kraxel@redhat.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com> --- drivers/gpu/drm/drm_aperture.c | 33 ++++++++++++++++++++++++--------- include/drm/drm_aperture.h | 2 ++ 2 files changed, 26 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/drm_aperture.c b/drivers/gpu/drm/drm_aperture.c index 74bd4a76b253..5b2500f7fb8b 100644 --- a/drivers/gpu/drm/drm_aperture.c +++ b/drivers/gpu/drm/drm_aperture.c @@ -313,6 +313,28 @@ int drm_aperture_remove_conflicting_framebuffers(resource_size_t base, resource_ } EXPORT_SYMBOL(drm_aperture_remove_conflicting_framebuffers); +/** + * drm_aperture_detach_platform_drivers - detach platform drivers conflicting with PCI device + * @pdev: PCI device + * + * This function detaches platform drivers with resource conflicts to the memory + * bars of the provided @pdev. + */ +void drm_aperture_detach_platform_drivers(struct pci_dev *pdev) +{ + resource_size_t base, size; + int bar; + + for (bar = 0; bar < PCI_STD_NUM_BARS; ++bar) { + if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) + continue; + base = pci_resource_start(pdev, bar); + size = pci_resource_len(pdev, bar); + drm_aperture_detach_drivers(base, size); + } +} +EXPORT_SYMBOL(drm_aperture_detach_platform_drivers); + /** * drm_aperture_remove_conflicting_pci_framebuffers - remove existing framebuffers for PCI devices * @pdev: PCI device @@ -328,16 +350,9 @@ EXPORT_SYMBOL(drm_aperture_remove_conflicting_framebuffers); int drm_aperture_remove_conflicting_pci_framebuffers(struct pci_dev *pdev, const struct drm_driver *req_driver) { - resource_size_t base, size; - int bar, ret = 0; + int ret = 0; - for (bar = 0; bar < PCI_STD_NUM_BARS; ++bar) { - if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) - continue; - base = pci_resource_start(pdev, bar); - size = pci_resource_len(pdev, bar); - drm_aperture_detach_drivers(base, size); - } + drm_aperture_detach_platform_drivers(pdev); /* * WARNING: Apparently we must kick fbdev drivers before vgacon, diff --git a/include/drm/drm_aperture.h b/include/drm/drm_aperture.h index 7096703c3949..53fd36fa258e 100644 --- a/include/drm/drm_aperture.h +++ b/include/drm/drm_aperture.h @@ -15,6 +15,8 @@ int devm_aperture_acquire_from_firmware(struct drm_device *dev, resource_size_t int drm_aperture_remove_conflicting_framebuffers(resource_size_t base, resource_size_t size, bool primary, const struct drm_driver *req_driver); +void drm_aperture_detach_platform_drivers(struct pci_dev *pdev); + int drm_aperture_remove_conflicting_pci_framebuffers(struct pci_dev *pdev, const struct drm_driver *req_driver); ^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 2/2] vfio/pci: Remove console drivers 2022-06-06 17:53 ` Alex Williamson @ 2022-06-06 17:53 ` Alex Williamson -1 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-06 17:53 UTC (permalink / raw) To: maarten.lankhorst, mripard, tzimmermann, airlied, daniel Cc: Laszlo Ersek, Laszlo Ersek, Gerd Hoffmann, dri-devel, linux-kernel, kvm Console drivers can create conflicts with PCI resources resulting in userspace getting mmap failures to memory BARs. This is especially evident when trying to re-use the system primary console for userspace drivers. Attempt to remove all nature of conflicting drivers as part of our VGA initialization. Reported-by: Laszlo Ersek <lersek@redhat.com> Tested-by: Laszlo Ersek <lersek@redhat.com> Suggested-by: Gerd Hoffmann <kraxel@redhat.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com> --- drivers/vfio/pci/vfio_pci_core.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index a0d69ddaf90d..e0cbcbc2aee1 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -13,6 +13,7 @@ #include <linux/device.h> #include <linux/eventfd.h> #include <linux/file.h> +#include <linux/fb.h> #include <linux/interrupt.h> #include <linux/iommu.h> #include <linux/module.h> @@ -29,6 +30,8 @@ #include <linux/vfio_pci_core.h> +#include <drm/drm_aperture.h> + #define DRIVER_AUTHOR "Alex Williamson <alex.williamson@redhat.com>" #define DRIVER_DESC "core driver for VFIO based PCI devices" @@ -1793,6 +1796,20 @@ static int vfio_pci_vga_init(struct vfio_pci_core_device *vdev) if (!vfio_pci_is_vga(pdev)) return 0; +#if IS_REACHABLE(CONFIG_DRM) + drm_aperture_detach_platform_drivers(pdev); +#endif + +#if IS_REACHABLE(CONFIG_FB) + ret = remove_conflicting_pci_framebuffers(pdev, vdev->vdev.ops->name); + if (ret) + return ret; +#endif + + ret = vga_remove_vgacon(pdev); + if (ret) + return ret; + ret = vga_client_register(pdev, vfio_pci_set_decode); if (ret) return ret; ^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 2/2] vfio/pci: Remove console drivers @ 2022-06-06 17:53 ` Alex Williamson 0 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-06 17:53 UTC (permalink / raw) To: maarten.lankhorst, mripard, tzimmermann, airlied, daniel Cc: kvm, Laszlo Ersek, Gerd Hoffmann, dri-devel, linux-kernel Console drivers can create conflicts with PCI resources resulting in userspace getting mmap failures to memory BARs. This is especially evident when trying to re-use the system primary console for userspace drivers. Attempt to remove all nature of conflicting drivers as part of our VGA initialization. Reported-by: Laszlo Ersek <lersek@redhat.com> Tested-by: Laszlo Ersek <lersek@redhat.com> Suggested-by: Gerd Hoffmann <kraxel@redhat.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com> --- drivers/vfio/pci/vfio_pci_core.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index a0d69ddaf90d..e0cbcbc2aee1 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -13,6 +13,7 @@ #include <linux/device.h> #include <linux/eventfd.h> #include <linux/file.h> +#include <linux/fb.h> #include <linux/interrupt.h> #include <linux/iommu.h> #include <linux/module.h> @@ -29,6 +30,8 @@ #include <linux/vfio_pci_core.h> +#include <drm/drm_aperture.h> + #define DRIVER_AUTHOR "Alex Williamson <alex.williamson@redhat.com>" #define DRIVER_DESC "core driver for VFIO based PCI devices" @@ -1793,6 +1796,20 @@ static int vfio_pci_vga_init(struct vfio_pci_core_device *vdev) if (!vfio_pci_is_vga(pdev)) return 0; +#if IS_REACHABLE(CONFIG_DRM) + drm_aperture_detach_platform_drivers(pdev); +#endif + +#if IS_REACHABLE(CONFIG_FB) + ret = remove_conflicting_pci_framebuffers(pdev, vdev->vdev.ops->name); + if (ret) + return ret; +#endif + + ret = vga_remove_vgacon(pdev); + if (ret) + return ret; + ret = vga_client_register(pdev, vfio_pci_set_decode); if (ret) return ret; ^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers 2022-06-06 17:53 ` Alex Williamson @ 2022-06-08 11:11 ` Thomas Zimmermann -1 siblings, 0 replies; 30+ messages in thread From: Thomas Zimmermann @ 2022-06-08 11:11 UTC (permalink / raw) To: Alex Williamson, maarten.lankhorst, mripard, airlied, daniel Cc: dri-devel, Laszlo Ersek, Gerd Hoffmann, kvm, linux-kernel [-- Attachment #1.1: Type: text/plain, Size: 3433 bytes --] Hi Alex Am 06.06.22 um 19:53 schrieb Alex Williamson: > Console drivers can create conflicts with PCI resources resulting in > userspace getting mmap failures to memory BARs. This is especially evident > when trying to re-use the system primary console for userspace drivers. > Attempt to remove all nature of conflicting drivers as part of our VGA > initialization. First a dumb question about your use case. You want to assign a PCI graphics card to a virtual machine and need to remove the generic driver from the framebuffer? > > Reported-by: Laszlo Ersek <lersek@redhat.com> > Tested-by: Laszlo Ersek <lersek@redhat.com> > Suggested-by: Gerd Hoffmann <kraxel@redhat.com> > Signed-off-by: Alex Williamson <alex.williamson@redhat.com> > --- > drivers/vfio/pci/vfio_pci_core.c | 17 +++++++++++++++++ > 1 file changed, 17 insertions(+) > > diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c > index a0d69ddaf90d..e0cbcbc2aee1 100644 > --- a/drivers/vfio/pci/vfio_pci_core.c > +++ b/drivers/vfio/pci/vfio_pci_core.c > @@ -13,6 +13,7 @@ > #include <linux/device.h> > #include <linux/eventfd.h> > #include <linux/file.h> > +#include <linux/fb.h> > #include <linux/interrupt.h> > #include <linux/iommu.h> > #include <linux/module.h> > @@ -29,6 +30,8 @@ > > #include <linux/vfio_pci_core.h> > > +#include <drm/drm_aperture.h> > + > #define DRIVER_AUTHOR "Alex Williamson <alex.williamson@redhat.com>" > #define DRIVER_DESC "core driver for VFIO based PCI devices" > > @@ -1793,6 +1796,20 @@ static int vfio_pci_vga_init(struct vfio_pci_core_device *vdev) > if (!vfio_pci_is_vga(pdev)) > return 0; > > +#if IS_REACHABLE(CONFIG_DRM) > + drm_aperture_detach_platform_drivers(pdev); > +#endif > + > +#if IS_REACHABLE(CONFIG_FB) > + ret = remove_conflicting_pci_framebuffers(pdev, vdev->vdev.ops->name); > + if (ret) > + return ret; > +#endif > + > + ret = vga_remove_vgacon(pdev); > + if (ret) > + return ret; > + You shouldn't have to copy any of the implementation of the aperture helpers. If you call drm_aperture_remove_conflicting_pci_framebuffers() it should work correctly. The only reason why it requires a DRM driver structure as second argument is for the driver's name. [1] And that name is only used for printing an info message. [2] The plan forward would be to drop patch 1 entirely. For patch 2, the most trivial workaround is to instanciate struct drm_driver here and set the name field to 'vdev->vdev.ops->name'. In the longer term, the aperture helpers will be moved out of DRM and into a more prominent location. That workaround will be cleaned up then. Alternatively, drm_aperture_remove_conflicting_pci_framebuffers() could be changed to accept the name string as second argument, but that's quite a bit of churn within the DRM code. Best regards Thomas [1] https://elixir.bootlin.com/linux/v5.18.2/source/drivers/gpu/drm/drm_aperture.c#L347 [2] https://elixir.bootlin.com/linux/v5.18.2/source/drivers/video/fbdev/core/fbmem.c#L1570 > ret = vga_client_register(pdev, vfio_pci_set_decode); > if (ret) > return ret; > > -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 840 bytes --] ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers @ 2022-06-08 11:11 ` Thomas Zimmermann 0 siblings, 0 replies; 30+ messages in thread From: Thomas Zimmermann @ 2022-06-08 11:11 UTC (permalink / raw) To: Alex Williamson, maarten.lankhorst, mripard, airlied, daniel Cc: kvm, Laszlo Ersek, Gerd Hoffmann, dri-devel, linux-kernel [-- Attachment #1.1: Type: text/plain, Size: 3433 bytes --] Hi Alex Am 06.06.22 um 19:53 schrieb Alex Williamson: > Console drivers can create conflicts with PCI resources resulting in > userspace getting mmap failures to memory BARs. This is especially evident > when trying to re-use the system primary console for userspace drivers. > Attempt to remove all nature of conflicting drivers as part of our VGA > initialization. First a dumb question about your use case. You want to assign a PCI graphics card to a virtual machine and need to remove the generic driver from the framebuffer? > > Reported-by: Laszlo Ersek <lersek@redhat.com> > Tested-by: Laszlo Ersek <lersek@redhat.com> > Suggested-by: Gerd Hoffmann <kraxel@redhat.com> > Signed-off-by: Alex Williamson <alex.williamson@redhat.com> > --- > drivers/vfio/pci/vfio_pci_core.c | 17 +++++++++++++++++ > 1 file changed, 17 insertions(+) > > diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c > index a0d69ddaf90d..e0cbcbc2aee1 100644 > --- a/drivers/vfio/pci/vfio_pci_core.c > +++ b/drivers/vfio/pci/vfio_pci_core.c > @@ -13,6 +13,7 @@ > #include <linux/device.h> > #include <linux/eventfd.h> > #include <linux/file.h> > +#include <linux/fb.h> > #include <linux/interrupt.h> > #include <linux/iommu.h> > #include <linux/module.h> > @@ -29,6 +30,8 @@ > > #include <linux/vfio_pci_core.h> > > +#include <drm/drm_aperture.h> > + > #define DRIVER_AUTHOR "Alex Williamson <alex.williamson@redhat.com>" > #define DRIVER_DESC "core driver for VFIO based PCI devices" > > @@ -1793,6 +1796,20 @@ static int vfio_pci_vga_init(struct vfio_pci_core_device *vdev) > if (!vfio_pci_is_vga(pdev)) > return 0; > > +#if IS_REACHABLE(CONFIG_DRM) > + drm_aperture_detach_platform_drivers(pdev); > +#endif > + > +#if IS_REACHABLE(CONFIG_FB) > + ret = remove_conflicting_pci_framebuffers(pdev, vdev->vdev.ops->name); > + if (ret) > + return ret; > +#endif > + > + ret = vga_remove_vgacon(pdev); > + if (ret) > + return ret; > + You shouldn't have to copy any of the implementation of the aperture helpers. If you call drm_aperture_remove_conflicting_pci_framebuffers() it should work correctly. The only reason why it requires a DRM driver structure as second argument is for the driver's name. [1] And that name is only used for printing an info message. [2] The plan forward would be to drop patch 1 entirely. For patch 2, the most trivial workaround is to instanciate struct drm_driver here and set the name field to 'vdev->vdev.ops->name'. In the longer term, the aperture helpers will be moved out of DRM and into a more prominent location. That workaround will be cleaned up then. Alternatively, drm_aperture_remove_conflicting_pci_framebuffers() could be changed to accept the name string as second argument, but that's quite a bit of churn within the DRM code. Best regards Thomas [1] https://elixir.bootlin.com/linux/v5.18.2/source/drivers/gpu/drm/drm_aperture.c#L347 [2] https://elixir.bootlin.com/linux/v5.18.2/source/drivers/video/fbdev/core/fbmem.c#L1570 > ret = vga_client_register(pdev, vfio_pci_set_decode); > if (ret) > return ret; > > -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 840 bytes --] ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers 2022-06-08 11:11 ` Thomas Zimmermann @ 2022-06-08 14:04 ` Alex Williamson -1 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-08 14:04 UTC (permalink / raw) To: Thomas Zimmermann Cc: maarten.lankhorst, mripard, airlied, daniel, dri-devel, Laszlo Ersek, Gerd Hoffmann, kvm, linux-kernel Hi Thomas, On Wed, 8 Jun 2022 13:11:21 +0200 Thomas Zimmermann <tzimmermann@suse.de> wrote: > Hi Alex > > Am 06.06.22 um 19:53 schrieb Alex Williamson: > > Console drivers can create conflicts with PCI resources resulting in > > userspace getting mmap failures to memory BARs. This is especially evident > > when trying to re-use the system primary console for userspace drivers. > > Attempt to remove all nature of conflicting drivers as part of our VGA > > initialization. > > First a dumb question about your use case. You want to assign a PCI > graphics card to a virtual machine and need to remove the generic driver > from the framebuffer? Exactly. > > Reported-by: Laszlo Ersek <lersek@redhat.com> > > Tested-by: Laszlo Ersek <lersek@redhat.com> > > Suggested-by: Gerd Hoffmann <kraxel@redhat.com> > > Signed-off-by: Alex Williamson <alex.williamson@redhat.com> > > --- > > drivers/vfio/pci/vfio_pci_core.c | 17 +++++++++++++++++ > > 1 file changed, 17 insertions(+) > > > > diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c > > index a0d69ddaf90d..e0cbcbc2aee1 100644 > > --- a/drivers/vfio/pci/vfio_pci_core.c > > +++ b/drivers/vfio/pci/vfio_pci_core.c > > @@ -13,6 +13,7 @@ > > #include <linux/device.h> > > #include <linux/eventfd.h> > > #include <linux/file.h> > > +#include <linux/fb.h> > > #include <linux/interrupt.h> > > #include <linux/iommu.h> > > #include <linux/module.h> > > @@ -29,6 +30,8 @@ > > > > #include <linux/vfio_pci_core.h> > > > > +#include <drm/drm_aperture.h> > > + > > #define DRIVER_AUTHOR "Alex Williamson <alex.williamson@redhat.com>" > > #define DRIVER_DESC "core driver for VFIO based PCI devices" > > > > @@ -1793,6 +1796,20 @@ static int vfio_pci_vga_init(struct vfio_pci_core_device *vdev) > > if (!vfio_pci_is_vga(pdev)) > > return 0; > > > > +#if IS_REACHABLE(CONFIG_DRM) > > + drm_aperture_detach_platform_drivers(pdev); > > +#endif > > + > > +#if IS_REACHABLE(CONFIG_FB) > > + ret = remove_conflicting_pci_framebuffers(pdev, vdev->vdev.ops->name); > > + if (ret) > > + return ret; > > +#endif > > + > > + ret = vga_remove_vgacon(pdev); > > + if (ret) > > + return ret; > > + > > You shouldn't have to copy any of the implementation of the aperture > helpers. > > If you call drm_aperture_remove_conflicting_pci_framebuffers() it should > work correctly. The only reason why it requires a DRM driver structure > as second argument is for the driver's name. [1] And that name is only > used for printing an info message. [2] vfio-pci is not dependent on CONFIG_DRM, therefore we need to open code this regardless. The only difference if we were to use the existing function would be something like: #if IS_REACHABLE(CONFIG_DRM) ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, &dummy_drm_driver); if (ret) return ret; #else #if IS_REACHABLE(CONFIG_FB) ret = remove_conflicting_pci_framebuffers(pdev, vdev->vdev.ops->name); if (ret) return ret; #endif ret = vga_remove_vgacon(pdev); if (ret) return ret; #endif It's also bad practice to create a dummy DRM driver struct with some assumption which fields are used. If the usage is later expanded, we'd only discover it by users getting segfaults. If DRM wanted to make such an API guarantee, then we shouldn't have commit 97c9bfe3f660 ("drm/aperture: Pass DRM driver structure instead of driver name"). > The plan forward would be to drop patch 1 entirely. > > For patch 2, the most trivial workaround is to instanciate struct > drm_driver here and set the name field to 'vdev->vdev.ops->name'. In the > longer term, the aperture helpers will be moved out of DRM and into a > more prominent location. That workaround will be cleaned up then. Trivial in execution, but as above, this is poor practice and should be avoided. > Alternatively, drm_aperture_remove_conflicting_pci_framebuffers() could > be changed to accept the name string as second argument, but that's > quite a bit of churn within the DRM code. The series as presented was exactly meant to provide the most correct solution with the least churn to the DRM code. The above referenced commit precludes us from calling the existing DRM function directly without resorting to poor practices of assuming the usage of the DRM driver struct. Using the existing DRM function also does not prevent us from open coding the remainder of the function to avoid a CONFIG_DRM dependency. Thanks, Alex ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers @ 2022-06-08 14:04 ` Alex Williamson 0 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-08 14:04 UTC (permalink / raw) To: Thomas Zimmermann Cc: kvm, airlied, linux-kernel, dri-devel, Laszlo Ersek, Gerd Hoffmann Hi Thomas, On Wed, 8 Jun 2022 13:11:21 +0200 Thomas Zimmermann <tzimmermann@suse.de> wrote: > Hi Alex > > Am 06.06.22 um 19:53 schrieb Alex Williamson: > > Console drivers can create conflicts with PCI resources resulting in > > userspace getting mmap failures to memory BARs. This is especially evident > > when trying to re-use the system primary console for userspace drivers. > > Attempt to remove all nature of conflicting drivers as part of our VGA > > initialization. > > First a dumb question about your use case. You want to assign a PCI > graphics card to a virtual machine and need to remove the generic driver > from the framebuffer? Exactly. > > Reported-by: Laszlo Ersek <lersek@redhat.com> > > Tested-by: Laszlo Ersek <lersek@redhat.com> > > Suggested-by: Gerd Hoffmann <kraxel@redhat.com> > > Signed-off-by: Alex Williamson <alex.williamson@redhat.com> > > --- > > drivers/vfio/pci/vfio_pci_core.c | 17 +++++++++++++++++ > > 1 file changed, 17 insertions(+) > > > > diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c > > index a0d69ddaf90d..e0cbcbc2aee1 100644 > > --- a/drivers/vfio/pci/vfio_pci_core.c > > +++ b/drivers/vfio/pci/vfio_pci_core.c > > @@ -13,6 +13,7 @@ > > #include <linux/device.h> > > #include <linux/eventfd.h> > > #include <linux/file.h> > > +#include <linux/fb.h> > > #include <linux/interrupt.h> > > #include <linux/iommu.h> > > #include <linux/module.h> > > @@ -29,6 +30,8 @@ > > > > #include <linux/vfio_pci_core.h> > > > > +#include <drm/drm_aperture.h> > > + > > #define DRIVER_AUTHOR "Alex Williamson <alex.williamson@redhat.com>" > > #define DRIVER_DESC "core driver for VFIO based PCI devices" > > > > @@ -1793,6 +1796,20 @@ static int vfio_pci_vga_init(struct vfio_pci_core_device *vdev) > > if (!vfio_pci_is_vga(pdev)) > > return 0; > > > > +#if IS_REACHABLE(CONFIG_DRM) > > + drm_aperture_detach_platform_drivers(pdev); > > +#endif > > + > > +#if IS_REACHABLE(CONFIG_FB) > > + ret = remove_conflicting_pci_framebuffers(pdev, vdev->vdev.ops->name); > > + if (ret) > > + return ret; > > +#endif > > + > > + ret = vga_remove_vgacon(pdev); > > + if (ret) > > + return ret; > > + > > You shouldn't have to copy any of the implementation of the aperture > helpers. > > If you call drm_aperture_remove_conflicting_pci_framebuffers() it should > work correctly. The only reason why it requires a DRM driver structure > as second argument is for the driver's name. [1] And that name is only > used for printing an info message. [2] vfio-pci is not dependent on CONFIG_DRM, therefore we need to open code this regardless. The only difference if we were to use the existing function would be something like: #if IS_REACHABLE(CONFIG_DRM) ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, &dummy_drm_driver); if (ret) return ret; #else #if IS_REACHABLE(CONFIG_FB) ret = remove_conflicting_pci_framebuffers(pdev, vdev->vdev.ops->name); if (ret) return ret; #endif ret = vga_remove_vgacon(pdev); if (ret) return ret; #endif It's also bad practice to create a dummy DRM driver struct with some assumption which fields are used. If the usage is later expanded, we'd only discover it by users getting segfaults. If DRM wanted to make such an API guarantee, then we shouldn't have commit 97c9bfe3f660 ("drm/aperture: Pass DRM driver structure instead of driver name"). > The plan forward would be to drop patch 1 entirely. > > For patch 2, the most trivial workaround is to instanciate struct > drm_driver here and set the name field to 'vdev->vdev.ops->name'. In the > longer term, the aperture helpers will be moved out of DRM and into a > more prominent location. That workaround will be cleaned up then. Trivial in execution, but as above, this is poor practice and should be avoided. > Alternatively, drm_aperture_remove_conflicting_pci_framebuffers() could > be changed to accept the name string as second argument, but that's > quite a bit of churn within the DRM code. The series as presented was exactly meant to provide the most correct solution with the least churn to the DRM code. The above referenced commit precludes us from calling the existing DRM function directly without resorting to poor practices of assuming the usage of the DRM driver struct. Using the existing DRM function also does not prevent us from open coding the remainder of the function to avoid a CONFIG_DRM dependency. Thanks, Alex ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers 2022-06-08 14:04 ` Alex Williamson @ 2022-06-09 9:13 ` Thomas Zimmermann -1 siblings, 0 replies; 30+ messages in thread From: Thomas Zimmermann @ 2022-06-09 9:13 UTC (permalink / raw) To: Alex Williamson Cc: kvm, airlied, linux-kernel, dri-devel, Laszlo Ersek, Gerd Hoffmann [-- Attachment #1.1.1: Type: text/plain, Size: 3607 bytes --] Hi Am 08.06.22 um 16:04 schrieb Alex Williamson: >> >> You shouldn't have to copy any of the implementation of the aperture >> helpers. >> >> If you call drm_aperture_remove_conflicting_pci_framebuffers() it should >> work correctly. The only reason why it requires a DRM driver structure >> as second argument is for the driver's name. [1] And that name is only >> used for printing an info message. [2] > > vfio-pci is not dependent on CONFIG_DRM, therefore we need to open code > this regardless. The only difference if we were to use the existing > function would be something like: > > #if IS_REACHABLE(CONFIG_DRM) > ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, &dummy_drm_driver); > if (ret) > return ret; > #else > #if IS_REACHABLE(CONFIG_FB) > ret = remove_conflicting_pci_framebuffers(pdev, vdev->vdev.ops->name); > if (ret) > return ret; > #endif > ret = vga_remove_vgacon(pdev); > if (ret) > return ret; > #endif > > It's also bad practice to create a dummy DRM driver struct with some > assumption which fields are used. If the usage is later expanded, we'd > only discover it by users getting segfaults. If DRM wanted to make > such an API guarantee, then we shouldn't have commit 97c9bfe3f660 > ("drm/aperture: Pass DRM driver structure instead of driver name"). What you're open coding within vfio is legacy code and we want to remove it as much as possible from the aperture helpers. Tying the helpers to DRM was in mistake in retrospective. We wanted something tailored to the needs of DRM. Now that we've seen quite a few corner cases in the interaction among graphics subsystems, we need something else. The order of creating devices and loading driver modules is crucial to making the hand-over between drivers work reliably. So far, this luckily has only been a problem in theory, but not practice. > >> The plan forward would be to drop patch 1 entirely. >> >> For patch 2, the most trivial workaround is to instanciate struct >> drm_driver here and set the name field to 'vdev->vdev.ops->name'. In the >> longer term, the aperture helpers will be moved out of DRM and into a >> more prominent location. That workaround will be cleaned up then. > > Trivial in execution, but as above, this is poor practice and should be > avoided. > >> Alternatively, drm_aperture_remove_conflicting_pci_framebuffers() could >> be changed to accept the name string as second argument, but that's >> quite a bit of churn within the DRM code. > > The series as presented was exactly meant to provide the most correct > solution with the least churn to the DRM code. The above referenced > commit precludes us from calling the existing DRM function directly > without resorting to poor practices of assuming the usage of the DRM > driver struct. Using the existing DRM function also does not prevent > us from open coding the remainder of the function to avoid a CONFIG_DRM > dependency. Thanks, Please have a look at the attached patch. It moves the aperture helpers to a location common to the various possible users (DRM, fbdev, vfio). The DRM interfaces remain untouched for now. The patch should provide what you need in vfio and also serve our future use cases for graphics drivers. If possible, please create your patch on top of it. Best regards Thomas > > Alex > -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev [-- Attachment #1.1.2: 0001-drm-Implement-DRM-aperture-helpers-under-video.patch --] [-- Type: text/x-patch, Size: 24431 bytes --] From ae8d8ebcf3884529ca1a9b659603a59271a8a553 Mon Sep 17 00:00:00 2001 From: Thomas Zimmermann <tzimmermann@suse.de> Date: Wed, 8 Jun 2022 20:52:50 +0200 Subject: [PATCH] drm: Implement DRM aperture helpers under video/ Implement DRM's aperture helpers under video/ for sharing with other sub-systems. Remove DRM-isms from the interface. The helpers track the ownership of framebuffer apertures and provide hand-over from firmware, such as EFI and VESA, to native graphics drivers. Other subsystems, such as fbdev and vfio, also have to maintain ownership of framebuffer apertures. Moving DRM's aperture helpers to a more public location allows all subsystems to interact with each other and share a common implementation. The aperture helpers are selected by the various firmware drivers within DRM and fbdev, and the VGA text-console driver. The original DRM interface is kept in place for use by DRM drivers. Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de> --- Documentation/driver-api/aperture.rst | 13 + Documentation/driver-api/index.rst | 1 + drivers/gpu/drm/drm_aperture.c | 174 +------------ drivers/gpu/drm/tiny/Kconfig | 1 + drivers/video/Kconfig | 6 + drivers/video/Makefile | 2 + drivers/video/aperture.c | 340 ++++++++++++++++++++++++++ drivers/video/console/Kconfig | 1 + drivers/video/fbdev/Kconfig | 7 +- include/linux/aperture.h | 56 +++++ 10 files changed, 435 insertions(+), 166 deletions(-) create mode 100644 Documentation/driver-api/aperture.rst create mode 100644 drivers/video/aperture.c create mode 100644 include/linux/aperture.h diff --git a/Documentation/driver-api/aperture.rst b/Documentation/driver-api/aperture.rst new file mode 100644 index 000000000000..d173f4e7a7d9 --- /dev/null +++ b/Documentation/driver-api/aperture.rst @@ -0,0 +1,13 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Managing Ownership of the Framebuffer Aperture +============================================== + +.. kernel-doc:: drivers/video/aperture.c + :doc: overview + +.. kernel-doc:: include/linux/aperture.h + :internal: + +.. kernel-doc:: drivers/video/aperture.c + :export: diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst index a7b0223e2886..d6d6e77d8afc 100644 --- a/Documentation/driver-api/index.rst +++ b/Documentation/driver-api/index.rst @@ -27,6 +27,7 @@ available subsections can be seen below. component message-based infiniband + aperture frame-buffer regulator reset diff --git a/drivers/gpu/drm/drm_aperture.c b/drivers/gpu/drm/drm_aperture.c index 74bd4a76b253..388a205bd023 100644 --- a/drivers/gpu/drm/drm_aperture.c +++ b/drivers/gpu/drm/drm_aperture.c @@ -1,14 +1,7 @@ // SPDX-License-Identifier: MIT -#include <linux/device.h> -#include <linux/fb.h> -#include <linux/list.h> -#include <linux/mutex.h> -#include <linux/pci.h> -#include <linux/platform_device.h> /* for firmware helpers */ -#include <linux/slab.h> -#include <linux/types.h> -#include <linux/vgaarb.h> +#include <linux/aperture.h> +#include <linux/platform_device.h> #include <drm/drm_aperture.h> #include <drm/drm_drv.h> @@ -126,92 +119,6 @@ * afterwards. */ -struct drm_aperture { - struct drm_device *dev; - resource_size_t base; - resource_size_t size; - struct list_head lh; - void (*detach)(struct drm_device *dev); -}; - -static LIST_HEAD(drm_apertures); -static DEFINE_MUTEX(drm_apertures_lock); - -static bool overlap(resource_size_t base1, resource_size_t end1, - resource_size_t base2, resource_size_t end2) -{ - return (base1 < end2) && (end1 > base2); -} - -static void devm_aperture_acquire_release(void *data) -{ - struct drm_aperture *ap = data; - bool detached = !ap->dev; - - if (detached) - return; - - mutex_lock(&drm_apertures_lock); - list_del(&ap->lh); - mutex_unlock(&drm_apertures_lock); -} - -static int devm_aperture_acquire(struct drm_device *dev, - resource_size_t base, resource_size_t size, - void (*detach)(struct drm_device *)) -{ - size_t end = base + size; - struct list_head *pos; - struct drm_aperture *ap; - - mutex_lock(&drm_apertures_lock); - - list_for_each(pos, &drm_apertures) { - ap = container_of(pos, struct drm_aperture, lh); - if (overlap(base, end, ap->base, ap->base + ap->size)) { - mutex_unlock(&drm_apertures_lock); - return -EBUSY; - } - } - - ap = devm_kzalloc(dev->dev, sizeof(*ap), GFP_KERNEL); - if (!ap) { - mutex_unlock(&drm_apertures_lock); - return -ENOMEM; - } - - ap->dev = dev; - ap->base = base; - ap->size = size; - ap->detach = detach; - INIT_LIST_HEAD(&ap->lh); - - list_add(&ap->lh, &drm_apertures); - - mutex_unlock(&drm_apertures_lock); - - return devm_add_action_or_reset(dev->dev, devm_aperture_acquire_release, ap); -} - -static void drm_aperture_detach_firmware(struct drm_device *dev) -{ - struct platform_device *pdev = to_platform_device(dev->dev); - - /* - * Remove the device from the device hierarchy. This is the right thing - * to do for firmware-based DRM drivers, such as EFI, VESA or VGA. After - * the new driver takes over the hardware, the firmware device's state - * will be lost. - * - * For non-platform devices, a new callback would be required. - * - * If the aperture helpers ever need to handle native drivers, this call - * would only have to unplug the DRM device, so that the hardware device - * stays around after detachment. - */ - platform_device_unregister(pdev); -} - /** * devm_aperture_acquire_from_firmware - Acquires ownership of a firmware framebuffer * on behalf of a DRM driver. @@ -239,39 +146,16 @@ static void drm_aperture_detach_firmware(struct drm_device *dev) int devm_aperture_acquire_from_firmware(struct drm_device *dev, resource_size_t base, resource_size_t size) { + struct platform_device *pdev; + if (drm_WARN_ON(dev, !dev_is_platform(dev->dev))) return -EINVAL; - return devm_aperture_acquire(dev, base, size, drm_aperture_detach_firmware); -} -EXPORT_SYMBOL(devm_aperture_acquire_from_firmware); - -static void drm_aperture_detach_drivers(resource_size_t base, resource_size_t size) -{ - resource_size_t end = base + size; - struct list_head *pos, *n; - - mutex_lock(&drm_apertures_lock); - - list_for_each_safe(pos, n, &drm_apertures) { - struct drm_aperture *ap = - container_of(pos, struct drm_aperture, lh); - struct drm_device *dev = ap->dev; - - if (WARN_ON_ONCE(!dev)) - continue; - - if (!overlap(base, end, ap->base, ap->base + ap->size)) - continue; + pdev = to_platform_device(dev->dev); - ap->dev = NULL; /* detach from device */ - list_del(&ap->lh); - - ap->detach(dev); - } - - mutex_unlock(&drm_apertures_lock); + return devm_acquire_aperture_of_platform_device(pdev, base, size); } +EXPORT_SYMBOL(devm_aperture_acquire_from_firmware); /** * drm_aperture_remove_conflicting_framebuffers - remove existing framebuffers in the given range @@ -289,27 +173,7 @@ static void drm_aperture_detach_drivers(resource_size_t base, resource_size_t si int drm_aperture_remove_conflicting_framebuffers(resource_size_t base, resource_size_t size, bool primary, const struct drm_driver *req_driver) { -#if IS_REACHABLE(CONFIG_FB) - struct apertures_struct *a; - int ret; - - a = alloc_apertures(1); - if (!a) - return -ENOMEM; - - a->ranges[0].base = base; - a->ranges[0].size = size; - - ret = remove_conflicting_framebuffers(a, req_driver->name, primary); - kfree(a); - - if (ret) - return ret; -#endif - - drm_aperture_detach_drivers(base, size); - - return 0; + return remove_conflicting_devices(base, size, primary, req_driver->name); } EXPORT_SYMBOL(drm_aperture_remove_conflicting_framebuffers); @@ -328,26 +192,6 @@ EXPORT_SYMBOL(drm_aperture_remove_conflicting_framebuffers); int drm_aperture_remove_conflicting_pci_framebuffers(struct pci_dev *pdev, const struct drm_driver *req_driver) { - resource_size_t base, size; - int bar, ret = 0; - - for (bar = 0; bar < PCI_STD_NUM_BARS; ++bar) { - if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) - continue; - base = pci_resource_start(pdev, bar); - size = pci_resource_len(pdev, bar); - drm_aperture_detach_drivers(base, size); - } - - /* - * WARNING: Apparently we must kick fbdev drivers before vgacon, - * otherwise the vga fbdev driver falls over. - */ -#if IS_REACHABLE(CONFIG_FB) - ret = remove_conflicting_pci_framebuffers(pdev, req_driver->name); -#endif - if (ret == 0) - ret = vga_remove_vgacon(pdev); - return ret; + return remove_conflicting_pci_devices(pdev, req_driver->name); } EXPORT_SYMBOL(drm_aperture_remove_conflicting_pci_framebuffers); diff --git a/drivers/gpu/drm/tiny/Kconfig b/drivers/gpu/drm/tiny/Kconfig index 627d637a1e7e..027cd87c3d0d 100644 --- a/drivers/gpu/drm/tiny/Kconfig +++ b/drivers/gpu/drm/tiny/Kconfig @@ -69,6 +69,7 @@ config DRM_PANEL_MIPI_DBI config DRM_SIMPLEDRM tristate "Simple framebuffer driver" depends on DRM && MMU + select APERTURE_HELPERS select DRM_GEM_SHMEM_HELPER select DRM_KMS_HELPER help diff --git a/drivers/video/Kconfig b/drivers/video/Kconfig index 427a993c7f57..c69b45f8c427 100644 --- a/drivers/video/Kconfig +++ b/drivers/video/Kconfig @@ -5,6 +5,12 @@ menu "Graphics support" +config APERTURE_HELPERS + bool + help + Support tracking and hand-over of aperture ownership. Required + for firmware graphics drivers. + if HAS_IOMEM config HAVE_FB_ATMEL diff --git a/drivers/video/Makefile b/drivers/video/Makefile index df7650adede9..5bb6b452cc83 100644 --- a/drivers/video/Makefile +++ b/drivers/video/Makefile @@ -1,4 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_APERTURE_HELPERS) += aperture.o obj-$(CONFIG_VGASTATE) += vgastate.o obj-$(CONFIG_HDMI) += hdmi.o diff --git a/drivers/video/aperture.c b/drivers/video/aperture.c new file mode 100644 index 000000000000..f0b877e9c256 --- /dev/null +++ b/drivers/video/aperture.c @@ -0,0 +1,340 @@ +// SPDX-License-Identifier: MIT + +#include <linux/aperture.h> +#include <linux/device.h> +#include <linux/fb.h> /* for old fbdev helpers */ +#include <linux/list.h> +#include <linux/mutex.h> +#include <linux/pci.h> +#include <linux/platform_device.h> +#include <linux/slab.h> +#include <linux/types.h> +#include <linux/vgaarb.h> + +/** + * DOC: overview + * + * A graphics device might be supported by different drivers, but only one + * driver can be active at any given time. Many systems load a generic + * graphics drivers, such as EFI-GOP or VESA, early during the boot process. + * During later boot stages, they replace the generic driver with a dedicated, + * hardware-specific driver. To take over the device the dedicated driver + * first has to remove the generic driver. Aperture functions manage + * ownership of framebuffer memory and hand-over between drivers. + * + * Graphics drivers should call remove_conflicting_devices() + * at the top of their probe function. The function removes any generic + * driver that is currently associated with the given framebuffer memory. + * If the framebuffer is located at PCI BAR 0, the rsp code looks as in the + * example given below. The cod assumes a DRM driver. + * + * .. code-block:: c + * + * static const struct drm_driver example_driver = { + * .name = "exampledrm", + * ... + * }; + * + * static int remove_conflicting_framebuffers(struct pci_dev *pdev) + * { + * bool primary = false; + * resource_size_t base, size; + * int ret; + * + * base = pci_resource_start(pdev, 0); + * size = pci_resource_len(pdev, 0); + * #ifdef CONFIG_X86 + * primary = pdev->resource[PCI_ROM_RESOURCE].flags & IORESOURCE_ROM_SHADOW; + * #endif + * + * return remove_conflicting_devices(base, size, primary, &example_driver->name); + * } + * + * static int probe(struct pci_dev *pdev) + * { + * int ret; + * + * // Remove any generic drivers... + * ret = remove_conflicting_framebuffers(pdev); + * if (ret) + * return ret; + * + * // ... and initialize the hardware. + * ... + * + * drm_dev_register(); + * + * return 0; + * } + * + * PCI device drivers can also call remove_conflicting_pci_devices() and let the + * function detect the apertures automatically. Device drivers without knowledge of + * the framebuffer's location shall call remove_all_conflicting_devices(), + * which removes all known devices. + * + * Drivers that are susceptible to being removed by other drivers, such as + * generic EFI or VESA drivers, have to register themselves as owners of their + * framebuffer apertures. Ownership of the framebuffer memory is achieved + * by calling devm_acquire_aperture_of_platform_device(). On success, the driver + * is the owner of the framebuffer range. The function fails if the + * framebuffer is already owned by another driver. See below for an example. + * + * .. code-block:: c + * + * static int acquire_framebuffers(struct drm_device *dev, struct platform_device *pdev) + * { + * resource_size_t base, size; + * + * mem = platform_get_resource(pdev, IORESOURCE_MEM, 0); + * if (!mem) + * return -EINVAL; + * base = mem->start; + * size = resource_size(mem); + * + * return devm_acquire_aperture_of_platform_device(pdev, base, size); + * } + * + * static int probe(struct platform_device *pdev) + * { + * struct drm_device *dev; + * int ret; + * + * // ... Initialize the device... + * dev = devm_drm_dev_alloc(); + * ... + * + * // ... and acquire ownership of the framebuffer. + * ret = acquire_framebuffers(dev, pdev); + * if (ret) + * return ret; + * + * drm_dev_register(dev, 0); + * + * return 0; + * } + * + * The generic driver is now subject to forced removal by other drivers. This + * only works for platform drivers that support hot unplugging. + * When a driver calls remove_conflicting_devices() et al + * for the registered framebuffer range, the aperture helpers call + * platform_device_unregister() and the generic driver unloads itself. It + * may not access the device's registers, framebuffer memory, ROM, etc + * afterwards. + */ + +struct dev_aperture { + struct device *dev; + resource_size_t base; + resource_size_t size; + struct list_head lh; + void (*detach)(struct device *dev); +}; + +static LIST_HEAD(apertures); +static DEFINE_MUTEX(apertures_lock); + +static bool overlap(resource_size_t base1, resource_size_t end1, + resource_size_t base2, resource_size_t end2) +{ + return (base1 < end2) && (end1 > base2); +} + +static void devm_aperture_acquire_release(void *data) +{ + struct dev_aperture *ap = data; + bool detached = !ap->dev; + + if (detached) + return; + + mutex_lock(&apertures_lock); + list_del(&ap->lh); + mutex_unlock(&apertures_lock); +} + +static int devm_aperture_acquire(struct device *dev, + resource_size_t base, resource_size_t size, + void (*detach)(struct device *)) +{ + size_t end = base + size; + struct list_head *pos; + struct dev_aperture *ap; + + mutex_lock(&apertures_lock); + + list_for_each(pos, &apertures) { + ap = container_of(pos, struct dev_aperture, lh); + if (overlap(base, end, ap->base, ap->base + ap->size)) { + mutex_unlock(&apertures_lock); + return -EBUSY; + } + } + + ap = devm_kzalloc(dev, sizeof(*ap), GFP_KERNEL); + if (!ap) { + mutex_unlock(&apertures_lock); + return -ENOMEM; + } + + ap->dev = dev; + ap->base = base; + ap->size = size; + ap->detach = detach; + INIT_LIST_HEAD(&ap->lh); + + list_add(&ap->lh, &apertures); + + mutex_unlock(&apertures_lock); + + return devm_add_action_or_reset(dev, devm_aperture_acquire_release, ap); +} + +static void detach_platform_device(struct device *dev) +{ + struct platform_device *pdev = to_platform_device(dev); + + /* + * Remove the device from the device hierarchy. This is the right thing + * to do for firmware-based DRM drivers, such as EFI, VESA or VGA. After + * the new driver takes over the hardware, the firmware device's state + * will be lost. + * + * For non-platform devices, a new callback would be required. + * + * If the aperture helpers ever need to handle native drivers, this call + * would only have to unplug the DRM device, so that the hardware device + * stays around after detachment. + */ + platform_device_unregister(pdev); +} + +/** + * devm_acquire_aperture_of_platform_device - Acquires ownership of an aperture + * on behalf of a platform device. + * @pdev: the platform device to own the aperture + * @base: the aperture's byte offset in physical memory + * @size: the aperture size in bytes + * + * Installs the given device as the new owner of the aperture. The function + * expects the aperture to be provided by a platform device. If another + * driver takes over ownership of the aperture, aperture helpers will then + * unregister the platform device automatically. All acquired apertures are + * released automatically when the underlying device goes away. + * + * The function fails if the aperture, or parts of it, is currently + * owned by another device. To evict current owners, callers should use + * remove_conflicting_devices() et al. before calling this function. + * + * Returns: + * 0 on success, or a negative errno value otherwise. + */ +int devm_acquire_aperture_of_platform_device(struct platform_device *pdev, + resource_size_t base, + resource_size_t size) +{ + return devm_aperture_acquire(&pdev->dev, base, size, detach_platform_device); +} +EXPORT_SYMBOL(devm_acquire_aperture_of_platform_device); + +static void detach_devices(resource_size_t base, resource_size_t size) +{ + resource_size_t end = base + size; + struct list_head *pos, *n; + + mutex_lock(&apertures_lock); + + list_for_each_safe(pos, n, &apertures) { + struct dev_aperture *ap = container_of(pos, struct dev_aperture, lh); + struct device *dev = ap->dev; + + if (WARN_ON_ONCE(!dev)) + continue; + + if (!overlap(base, end, ap->base, ap->base + ap->size)) + continue; + + ap->dev = NULL; /* detach from device */ + list_del(&ap->lh); + + ap->detach(dev); + } + + mutex_unlock(&apertures_lock); +} + +/** + * remove_conflicting_devices - remove devices in the given range + * @base: the aperture's base address in physical memory + * @size: aperture size in bytes + * @primary: also kick vga16fb if present; only relevant for VGA devices + * @name: a descriptive name of the requesting driver + * + * This function removes devices that own apertures within @base and @size. + * + * Returns: + * 0 on success, or a negative errno code otherwise + */ +int remove_conflicting_devices(resource_size_t base, resource_size_t size, bool primary, + const char *name) +{ +#if IS_REACHABLE(CONFIG_FB) + struct apertures_struct *a; + int ret; + + a = alloc_apertures(1); + if (!a) + return -ENOMEM; + + a->ranges[0].base = base; + a->ranges[0].size = size; + + ret = remove_conflicting_framebuffers(a, name, primary); + kfree(a); + + if (ret) + return ret; +#endif + + detach_devices(base, size); + + return 0; +} +EXPORT_SYMBOL(remove_conflicting_devices); + +/** + * remove_conflicting_pci_devices - remove existing framebuffers for PCI devices + * @pdev: PCI device + * @name: a descriptive name of the requesting driver + * + * This function removes devices that own apertures within any of @pdev's + * memory bars. The function assumes that PCI device with shadowed ROM + * drives a primary display and therefore kicks out vga16fb as well. + * + * Returns: + * 0 on success, or a negative errno code otherwise + */ +int remove_conflicting_pci_devices(struct pci_dev *pdev, const char *name) +{ + resource_size_t base, size; + int bar, ret = 0; + + for (bar = 0; bar < PCI_STD_NUM_BARS; ++bar) { + if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) + continue; + base = pci_resource_start(pdev, bar); + size = pci_resource_len(pdev, bar); + detach_devices(base, size); + } + + /* + * WARNING: Apparently we must kick fbdev drivers before vgacon, + * otherwise the vga fbdev driver falls over. + */ +#if IS_REACHABLE(CONFIG_FB) + ret = remove_conflicting_pci_framebuffers(pdev, name); +#endif + if (ret == 0) + ret = vga_remove_vgacon(pdev); + return ret; +} +EXPORT_SYMBOL(remove_conflicting_pci_devices); diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig index 40c50fa2dd70..7f3c44e1538b 100644 --- a/drivers/video/console/Kconfig +++ b/drivers/video/console/Kconfig @@ -10,6 +10,7 @@ config VGA_CONSOLE depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC && !SUPERH && \ (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \ !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !S390 && !UML + select APERTURE_HELPERS if (DRM || FB || VFIO_PCI) default y help Saying Y here will allow you to use Linux in text mode through a diff --git a/drivers/video/fbdev/Kconfig b/drivers/video/fbdev/Kconfig index bd849013f16f..7b398e7e3cc8 100644 --- a/drivers/video/fbdev/Kconfig +++ b/drivers/video/fbdev/Kconfig @@ -455,6 +455,7 @@ config FB_ATARI config FB_OF bool "Open Firmware frame buffer device support" depends on (FB = y) && PPC && (!PPC_PSERIES || PCI) + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT @@ -527,6 +528,7 @@ config FB_IMSTT config FB_VGA16 tristate "VGA 16-color graphics support" depends on FB && (X86 || PPC) + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT @@ -551,7 +553,7 @@ config FB_STI BIOS routines contained in a ROM chip in HP PA-RISC based machines. Enabling this option will implement the linux framebuffer device using calls to the STI BIOS routines for initialisation. - + If you enable this option, you will get a planar framebuffer device /dev/fb which will work on the most common HP graphic cards of the NGLE family, including the artist chips (in the 7xx and Bxxx series), @@ -617,6 +619,7 @@ config FB_UVESA config FB_VESA bool "VESA VGA graphics support" depends on (FB = y) && (X86 || COMPILE_TEST) + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT @@ -630,6 +633,7 @@ config FB_VESA config FB_EFI bool "EFI-based Framebuffer Support" depends on (FB = y) && !IA64 && EFI + select APERTURE_HELPERS select DRM_PANEL_ORIENTATION_QUIRKS select FB_CFB_FILLRECT select FB_CFB_COPYAREA @@ -2190,6 +2194,7 @@ config FB_SIMPLE tristate "Simple framebuffer support" depends on FB depends on !DRM_SIMPLEDRM + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT diff --git a/include/linux/aperture.h b/include/linux/aperture.h new file mode 100644 index 000000000000..0a206cdce606 --- /dev/null +++ b/include/linux/aperture.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: MIT */ + +#ifndef _LINUX_APERTURE_H_ +#define _LINUX_APERTURE_H_ + +#include <linux/types.h> + +struct pci_dev; +struct platform_device; + +#if defined(CONFIG_APERTURE_HELPERS) +int devm_acquire_aperture_of_platform_device(struct platform_device *pdev, + resource_size_t base, + resource_size_t size); + +int remove_conflicting_devices(resource_size_t base, resource_size_t size, + bool primary, const char *name); + +int remove_conflicting_pci_devices(struct pci_dev *pdev, const char *name); +#else +static inline int devm_acquire_aperture_of_platform_device(struct platform_device *pdev, + resource_size_t base, + resource_size_t size) +{ + return 0; +} + +static inline int remove_conflicting_devices(resource_size_t base, resource_size_t size, + bool primary, const char *name) +{ + return 0; +} + +static inline int remove_conflicting_pci_devices(struct pci_dev *pdev, const char *name) +{ + return 0; +} +#endif + +/** + * remove_all_conflicting_devices - remove all existing framebuffers + * @primary: also kick vga16fb if present; only relevant for VGA devices + * @name: a descriptive name of the requesting driver + * + * This function removes all graphics device drivers. Use this function on systems + * that can have their framebuffer located anywhere in memory. + * + * Returns: + * 0 on success, or a negative errno code otherwise + */ +static inline int remove_all_conflicting_devices(bool primary, const char *name) +{ + return remove_conflicting_devices(0, (resource_size_t)-1, primary, name); +} + +#endif -- 2.36.1 [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 840 bytes --] ^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers @ 2022-06-09 9:13 ` Thomas Zimmermann 0 siblings, 0 replies; 30+ messages in thread From: Thomas Zimmermann @ 2022-06-09 9:13 UTC (permalink / raw) To: Alex Williamson Cc: kvm, airlied, linux-kernel, dri-devel, Gerd Hoffmann, Laszlo Ersek [-- Attachment #1.1.1: Type: text/plain, Size: 3607 bytes --] Hi Am 08.06.22 um 16:04 schrieb Alex Williamson: >> >> You shouldn't have to copy any of the implementation of the aperture >> helpers. >> >> If you call drm_aperture_remove_conflicting_pci_framebuffers() it should >> work correctly. The only reason why it requires a DRM driver structure >> as second argument is for the driver's name. [1] And that name is only >> used for printing an info message. [2] > > vfio-pci is not dependent on CONFIG_DRM, therefore we need to open code > this regardless. The only difference if we were to use the existing > function would be something like: > > #if IS_REACHABLE(CONFIG_DRM) > ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, &dummy_drm_driver); > if (ret) > return ret; > #else > #if IS_REACHABLE(CONFIG_FB) > ret = remove_conflicting_pci_framebuffers(pdev, vdev->vdev.ops->name); > if (ret) > return ret; > #endif > ret = vga_remove_vgacon(pdev); > if (ret) > return ret; > #endif > > It's also bad practice to create a dummy DRM driver struct with some > assumption which fields are used. If the usage is later expanded, we'd > only discover it by users getting segfaults. If DRM wanted to make > such an API guarantee, then we shouldn't have commit 97c9bfe3f660 > ("drm/aperture: Pass DRM driver structure instead of driver name"). What you're open coding within vfio is legacy code and we want to remove it as much as possible from the aperture helpers. Tying the helpers to DRM was in mistake in retrospective. We wanted something tailored to the needs of DRM. Now that we've seen quite a few corner cases in the interaction among graphics subsystems, we need something else. The order of creating devices and loading driver modules is crucial to making the hand-over between drivers work reliably. So far, this luckily has only been a problem in theory, but not practice. > >> The plan forward would be to drop patch 1 entirely. >> >> For patch 2, the most trivial workaround is to instanciate struct >> drm_driver here and set the name field to 'vdev->vdev.ops->name'. In the >> longer term, the aperture helpers will be moved out of DRM and into a >> more prominent location. That workaround will be cleaned up then. > > Trivial in execution, but as above, this is poor practice and should be > avoided. > >> Alternatively, drm_aperture_remove_conflicting_pci_framebuffers() could >> be changed to accept the name string as second argument, but that's >> quite a bit of churn within the DRM code. > > The series as presented was exactly meant to provide the most correct > solution with the least churn to the DRM code. The above referenced > commit precludes us from calling the existing DRM function directly > without resorting to poor practices of assuming the usage of the DRM > driver struct. Using the existing DRM function also does not prevent > us from open coding the remainder of the function to avoid a CONFIG_DRM > dependency. Thanks, Please have a look at the attached patch. It moves the aperture helpers to a location common to the various possible users (DRM, fbdev, vfio). The DRM interfaces remain untouched for now. The patch should provide what you need in vfio and also serve our future use cases for graphics drivers. If possible, please create your patch on top of it. Best regards Thomas > > Alex > -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev [-- Attachment #1.1.2: 0001-drm-Implement-DRM-aperture-helpers-under-video.patch --] [-- Type: text/x-patch, Size: 24431 bytes --] From ae8d8ebcf3884529ca1a9b659603a59271a8a553 Mon Sep 17 00:00:00 2001 From: Thomas Zimmermann <tzimmermann@suse.de> Date: Wed, 8 Jun 2022 20:52:50 +0200 Subject: [PATCH] drm: Implement DRM aperture helpers under video/ Implement DRM's aperture helpers under video/ for sharing with other sub-systems. Remove DRM-isms from the interface. The helpers track the ownership of framebuffer apertures and provide hand-over from firmware, such as EFI and VESA, to native graphics drivers. Other subsystems, such as fbdev and vfio, also have to maintain ownership of framebuffer apertures. Moving DRM's aperture helpers to a more public location allows all subsystems to interact with each other and share a common implementation. The aperture helpers are selected by the various firmware drivers within DRM and fbdev, and the VGA text-console driver. The original DRM interface is kept in place for use by DRM drivers. Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de> --- Documentation/driver-api/aperture.rst | 13 + Documentation/driver-api/index.rst | 1 + drivers/gpu/drm/drm_aperture.c | 174 +------------ drivers/gpu/drm/tiny/Kconfig | 1 + drivers/video/Kconfig | 6 + drivers/video/Makefile | 2 + drivers/video/aperture.c | 340 ++++++++++++++++++++++++++ drivers/video/console/Kconfig | 1 + drivers/video/fbdev/Kconfig | 7 +- include/linux/aperture.h | 56 +++++ 10 files changed, 435 insertions(+), 166 deletions(-) create mode 100644 Documentation/driver-api/aperture.rst create mode 100644 drivers/video/aperture.c create mode 100644 include/linux/aperture.h diff --git a/Documentation/driver-api/aperture.rst b/Documentation/driver-api/aperture.rst new file mode 100644 index 000000000000..d173f4e7a7d9 --- /dev/null +++ b/Documentation/driver-api/aperture.rst @@ -0,0 +1,13 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Managing Ownership of the Framebuffer Aperture +============================================== + +.. kernel-doc:: drivers/video/aperture.c + :doc: overview + +.. kernel-doc:: include/linux/aperture.h + :internal: + +.. kernel-doc:: drivers/video/aperture.c + :export: diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst index a7b0223e2886..d6d6e77d8afc 100644 --- a/Documentation/driver-api/index.rst +++ b/Documentation/driver-api/index.rst @@ -27,6 +27,7 @@ available subsections can be seen below. component message-based infiniband + aperture frame-buffer regulator reset diff --git a/drivers/gpu/drm/drm_aperture.c b/drivers/gpu/drm/drm_aperture.c index 74bd4a76b253..388a205bd023 100644 --- a/drivers/gpu/drm/drm_aperture.c +++ b/drivers/gpu/drm/drm_aperture.c @@ -1,14 +1,7 @@ // SPDX-License-Identifier: MIT -#include <linux/device.h> -#include <linux/fb.h> -#include <linux/list.h> -#include <linux/mutex.h> -#include <linux/pci.h> -#include <linux/platform_device.h> /* for firmware helpers */ -#include <linux/slab.h> -#include <linux/types.h> -#include <linux/vgaarb.h> +#include <linux/aperture.h> +#include <linux/platform_device.h> #include <drm/drm_aperture.h> #include <drm/drm_drv.h> @@ -126,92 +119,6 @@ * afterwards. */ -struct drm_aperture { - struct drm_device *dev; - resource_size_t base; - resource_size_t size; - struct list_head lh; - void (*detach)(struct drm_device *dev); -}; - -static LIST_HEAD(drm_apertures); -static DEFINE_MUTEX(drm_apertures_lock); - -static bool overlap(resource_size_t base1, resource_size_t end1, - resource_size_t base2, resource_size_t end2) -{ - return (base1 < end2) && (end1 > base2); -} - -static void devm_aperture_acquire_release(void *data) -{ - struct drm_aperture *ap = data; - bool detached = !ap->dev; - - if (detached) - return; - - mutex_lock(&drm_apertures_lock); - list_del(&ap->lh); - mutex_unlock(&drm_apertures_lock); -} - -static int devm_aperture_acquire(struct drm_device *dev, - resource_size_t base, resource_size_t size, - void (*detach)(struct drm_device *)) -{ - size_t end = base + size; - struct list_head *pos; - struct drm_aperture *ap; - - mutex_lock(&drm_apertures_lock); - - list_for_each(pos, &drm_apertures) { - ap = container_of(pos, struct drm_aperture, lh); - if (overlap(base, end, ap->base, ap->base + ap->size)) { - mutex_unlock(&drm_apertures_lock); - return -EBUSY; - } - } - - ap = devm_kzalloc(dev->dev, sizeof(*ap), GFP_KERNEL); - if (!ap) { - mutex_unlock(&drm_apertures_lock); - return -ENOMEM; - } - - ap->dev = dev; - ap->base = base; - ap->size = size; - ap->detach = detach; - INIT_LIST_HEAD(&ap->lh); - - list_add(&ap->lh, &drm_apertures); - - mutex_unlock(&drm_apertures_lock); - - return devm_add_action_or_reset(dev->dev, devm_aperture_acquire_release, ap); -} - -static void drm_aperture_detach_firmware(struct drm_device *dev) -{ - struct platform_device *pdev = to_platform_device(dev->dev); - - /* - * Remove the device from the device hierarchy. This is the right thing - * to do for firmware-based DRM drivers, such as EFI, VESA or VGA. After - * the new driver takes over the hardware, the firmware device's state - * will be lost. - * - * For non-platform devices, a new callback would be required. - * - * If the aperture helpers ever need to handle native drivers, this call - * would only have to unplug the DRM device, so that the hardware device - * stays around after detachment. - */ - platform_device_unregister(pdev); -} - /** * devm_aperture_acquire_from_firmware - Acquires ownership of a firmware framebuffer * on behalf of a DRM driver. @@ -239,39 +146,16 @@ static void drm_aperture_detach_firmware(struct drm_device *dev) int devm_aperture_acquire_from_firmware(struct drm_device *dev, resource_size_t base, resource_size_t size) { + struct platform_device *pdev; + if (drm_WARN_ON(dev, !dev_is_platform(dev->dev))) return -EINVAL; - return devm_aperture_acquire(dev, base, size, drm_aperture_detach_firmware); -} -EXPORT_SYMBOL(devm_aperture_acquire_from_firmware); - -static void drm_aperture_detach_drivers(resource_size_t base, resource_size_t size) -{ - resource_size_t end = base + size; - struct list_head *pos, *n; - - mutex_lock(&drm_apertures_lock); - - list_for_each_safe(pos, n, &drm_apertures) { - struct drm_aperture *ap = - container_of(pos, struct drm_aperture, lh); - struct drm_device *dev = ap->dev; - - if (WARN_ON_ONCE(!dev)) - continue; - - if (!overlap(base, end, ap->base, ap->base + ap->size)) - continue; + pdev = to_platform_device(dev->dev); - ap->dev = NULL; /* detach from device */ - list_del(&ap->lh); - - ap->detach(dev); - } - - mutex_unlock(&drm_apertures_lock); + return devm_acquire_aperture_of_platform_device(pdev, base, size); } +EXPORT_SYMBOL(devm_aperture_acquire_from_firmware); /** * drm_aperture_remove_conflicting_framebuffers - remove existing framebuffers in the given range @@ -289,27 +173,7 @@ static void drm_aperture_detach_drivers(resource_size_t base, resource_size_t si int drm_aperture_remove_conflicting_framebuffers(resource_size_t base, resource_size_t size, bool primary, const struct drm_driver *req_driver) { -#if IS_REACHABLE(CONFIG_FB) - struct apertures_struct *a; - int ret; - - a = alloc_apertures(1); - if (!a) - return -ENOMEM; - - a->ranges[0].base = base; - a->ranges[0].size = size; - - ret = remove_conflicting_framebuffers(a, req_driver->name, primary); - kfree(a); - - if (ret) - return ret; -#endif - - drm_aperture_detach_drivers(base, size); - - return 0; + return remove_conflicting_devices(base, size, primary, req_driver->name); } EXPORT_SYMBOL(drm_aperture_remove_conflicting_framebuffers); @@ -328,26 +192,6 @@ EXPORT_SYMBOL(drm_aperture_remove_conflicting_framebuffers); int drm_aperture_remove_conflicting_pci_framebuffers(struct pci_dev *pdev, const struct drm_driver *req_driver) { - resource_size_t base, size; - int bar, ret = 0; - - for (bar = 0; bar < PCI_STD_NUM_BARS; ++bar) { - if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) - continue; - base = pci_resource_start(pdev, bar); - size = pci_resource_len(pdev, bar); - drm_aperture_detach_drivers(base, size); - } - - /* - * WARNING: Apparently we must kick fbdev drivers before vgacon, - * otherwise the vga fbdev driver falls over. - */ -#if IS_REACHABLE(CONFIG_FB) - ret = remove_conflicting_pci_framebuffers(pdev, req_driver->name); -#endif - if (ret == 0) - ret = vga_remove_vgacon(pdev); - return ret; + return remove_conflicting_pci_devices(pdev, req_driver->name); } EXPORT_SYMBOL(drm_aperture_remove_conflicting_pci_framebuffers); diff --git a/drivers/gpu/drm/tiny/Kconfig b/drivers/gpu/drm/tiny/Kconfig index 627d637a1e7e..027cd87c3d0d 100644 --- a/drivers/gpu/drm/tiny/Kconfig +++ b/drivers/gpu/drm/tiny/Kconfig @@ -69,6 +69,7 @@ config DRM_PANEL_MIPI_DBI config DRM_SIMPLEDRM tristate "Simple framebuffer driver" depends on DRM && MMU + select APERTURE_HELPERS select DRM_GEM_SHMEM_HELPER select DRM_KMS_HELPER help diff --git a/drivers/video/Kconfig b/drivers/video/Kconfig index 427a993c7f57..c69b45f8c427 100644 --- a/drivers/video/Kconfig +++ b/drivers/video/Kconfig @@ -5,6 +5,12 @@ menu "Graphics support" +config APERTURE_HELPERS + bool + help + Support tracking and hand-over of aperture ownership. Required + for firmware graphics drivers. + if HAS_IOMEM config HAVE_FB_ATMEL diff --git a/drivers/video/Makefile b/drivers/video/Makefile index df7650adede9..5bb6b452cc83 100644 --- a/drivers/video/Makefile +++ b/drivers/video/Makefile @@ -1,4 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_APERTURE_HELPERS) += aperture.o obj-$(CONFIG_VGASTATE) += vgastate.o obj-$(CONFIG_HDMI) += hdmi.o diff --git a/drivers/video/aperture.c b/drivers/video/aperture.c new file mode 100644 index 000000000000..f0b877e9c256 --- /dev/null +++ b/drivers/video/aperture.c @@ -0,0 +1,340 @@ +// SPDX-License-Identifier: MIT + +#include <linux/aperture.h> +#include <linux/device.h> +#include <linux/fb.h> /* for old fbdev helpers */ +#include <linux/list.h> +#include <linux/mutex.h> +#include <linux/pci.h> +#include <linux/platform_device.h> +#include <linux/slab.h> +#include <linux/types.h> +#include <linux/vgaarb.h> + +/** + * DOC: overview + * + * A graphics device might be supported by different drivers, but only one + * driver can be active at any given time. Many systems load a generic + * graphics drivers, such as EFI-GOP or VESA, early during the boot process. + * During later boot stages, they replace the generic driver with a dedicated, + * hardware-specific driver. To take over the device the dedicated driver + * first has to remove the generic driver. Aperture functions manage + * ownership of framebuffer memory and hand-over between drivers. + * + * Graphics drivers should call remove_conflicting_devices() + * at the top of their probe function. The function removes any generic + * driver that is currently associated with the given framebuffer memory. + * If the framebuffer is located at PCI BAR 0, the rsp code looks as in the + * example given below. The cod assumes a DRM driver. + * + * .. code-block:: c + * + * static const struct drm_driver example_driver = { + * .name = "exampledrm", + * ... + * }; + * + * static int remove_conflicting_framebuffers(struct pci_dev *pdev) + * { + * bool primary = false; + * resource_size_t base, size; + * int ret; + * + * base = pci_resource_start(pdev, 0); + * size = pci_resource_len(pdev, 0); + * #ifdef CONFIG_X86 + * primary = pdev->resource[PCI_ROM_RESOURCE].flags & IORESOURCE_ROM_SHADOW; + * #endif + * + * return remove_conflicting_devices(base, size, primary, &example_driver->name); + * } + * + * static int probe(struct pci_dev *pdev) + * { + * int ret; + * + * // Remove any generic drivers... + * ret = remove_conflicting_framebuffers(pdev); + * if (ret) + * return ret; + * + * // ... and initialize the hardware. + * ... + * + * drm_dev_register(); + * + * return 0; + * } + * + * PCI device drivers can also call remove_conflicting_pci_devices() and let the + * function detect the apertures automatically. Device drivers without knowledge of + * the framebuffer's location shall call remove_all_conflicting_devices(), + * which removes all known devices. + * + * Drivers that are susceptible to being removed by other drivers, such as + * generic EFI or VESA drivers, have to register themselves as owners of their + * framebuffer apertures. Ownership of the framebuffer memory is achieved + * by calling devm_acquire_aperture_of_platform_device(). On success, the driver + * is the owner of the framebuffer range. The function fails if the + * framebuffer is already owned by another driver. See below for an example. + * + * .. code-block:: c + * + * static int acquire_framebuffers(struct drm_device *dev, struct platform_device *pdev) + * { + * resource_size_t base, size; + * + * mem = platform_get_resource(pdev, IORESOURCE_MEM, 0); + * if (!mem) + * return -EINVAL; + * base = mem->start; + * size = resource_size(mem); + * + * return devm_acquire_aperture_of_platform_device(pdev, base, size); + * } + * + * static int probe(struct platform_device *pdev) + * { + * struct drm_device *dev; + * int ret; + * + * // ... Initialize the device... + * dev = devm_drm_dev_alloc(); + * ... + * + * // ... and acquire ownership of the framebuffer. + * ret = acquire_framebuffers(dev, pdev); + * if (ret) + * return ret; + * + * drm_dev_register(dev, 0); + * + * return 0; + * } + * + * The generic driver is now subject to forced removal by other drivers. This + * only works for platform drivers that support hot unplugging. + * When a driver calls remove_conflicting_devices() et al + * for the registered framebuffer range, the aperture helpers call + * platform_device_unregister() and the generic driver unloads itself. It + * may not access the device's registers, framebuffer memory, ROM, etc + * afterwards. + */ + +struct dev_aperture { + struct device *dev; + resource_size_t base; + resource_size_t size; + struct list_head lh; + void (*detach)(struct device *dev); +}; + +static LIST_HEAD(apertures); +static DEFINE_MUTEX(apertures_lock); + +static bool overlap(resource_size_t base1, resource_size_t end1, + resource_size_t base2, resource_size_t end2) +{ + return (base1 < end2) && (end1 > base2); +} + +static void devm_aperture_acquire_release(void *data) +{ + struct dev_aperture *ap = data; + bool detached = !ap->dev; + + if (detached) + return; + + mutex_lock(&apertures_lock); + list_del(&ap->lh); + mutex_unlock(&apertures_lock); +} + +static int devm_aperture_acquire(struct device *dev, + resource_size_t base, resource_size_t size, + void (*detach)(struct device *)) +{ + size_t end = base + size; + struct list_head *pos; + struct dev_aperture *ap; + + mutex_lock(&apertures_lock); + + list_for_each(pos, &apertures) { + ap = container_of(pos, struct dev_aperture, lh); + if (overlap(base, end, ap->base, ap->base + ap->size)) { + mutex_unlock(&apertures_lock); + return -EBUSY; + } + } + + ap = devm_kzalloc(dev, sizeof(*ap), GFP_KERNEL); + if (!ap) { + mutex_unlock(&apertures_lock); + return -ENOMEM; + } + + ap->dev = dev; + ap->base = base; + ap->size = size; + ap->detach = detach; + INIT_LIST_HEAD(&ap->lh); + + list_add(&ap->lh, &apertures); + + mutex_unlock(&apertures_lock); + + return devm_add_action_or_reset(dev, devm_aperture_acquire_release, ap); +} + +static void detach_platform_device(struct device *dev) +{ + struct platform_device *pdev = to_platform_device(dev); + + /* + * Remove the device from the device hierarchy. This is the right thing + * to do for firmware-based DRM drivers, such as EFI, VESA or VGA. After + * the new driver takes over the hardware, the firmware device's state + * will be lost. + * + * For non-platform devices, a new callback would be required. + * + * If the aperture helpers ever need to handle native drivers, this call + * would only have to unplug the DRM device, so that the hardware device + * stays around after detachment. + */ + platform_device_unregister(pdev); +} + +/** + * devm_acquire_aperture_of_platform_device - Acquires ownership of an aperture + * on behalf of a platform device. + * @pdev: the platform device to own the aperture + * @base: the aperture's byte offset in physical memory + * @size: the aperture size in bytes + * + * Installs the given device as the new owner of the aperture. The function + * expects the aperture to be provided by a platform device. If another + * driver takes over ownership of the aperture, aperture helpers will then + * unregister the platform device automatically. All acquired apertures are + * released automatically when the underlying device goes away. + * + * The function fails if the aperture, or parts of it, is currently + * owned by another device. To evict current owners, callers should use + * remove_conflicting_devices() et al. before calling this function. + * + * Returns: + * 0 on success, or a negative errno value otherwise. + */ +int devm_acquire_aperture_of_platform_device(struct platform_device *pdev, + resource_size_t base, + resource_size_t size) +{ + return devm_aperture_acquire(&pdev->dev, base, size, detach_platform_device); +} +EXPORT_SYMBOL(devm_acquire_aperture_of_platform_device); + +static void detach_devices(resource_size_t base, resource_size_t size) +{ + resource_size_t end = base + size; + struct list_head *pos, *n; + + mutex_lock(&apertures_lock); + + list_for_each_safe(pos, n, &apertures) { + struct dev_aperture *ap = container_of(pos, struct dev_aperture, lh); + struct device *dev = ap->dev; + + if (WARN_ON_ONCE(!dev)) + continue; + + if (!overlap(base, end, ap->base, ap->base + ap->size)) + continue; + + ap->dev = NULL; /* detach from device */ + list_del(&ap->lh); + + ap->detach(dev); + } + + mutex_unlock(&apertures_lock); +} + +/** + * remove_conflicting_devices - remove devices in the given range + * @base: the aperture's base address in physical memory + * @size: aperture size in bytes + * @primary: also kick vga16fb if present; only relevant for VGA devices + * @name: a descriptive name of the requesting driver + * + * This function removes devices that own apertures within @base and @size. + * + * Returns: + * 0 on success, or a negative errno code otherwise + */ +int remove_conflicting_devices(resource_size_t base, resource_size_t size, bool primary, + const char *name) +{ +#if IS_REACHABLE(CONFIG_FB) + struct apertures_struct *a; + int ret; + + a = alloc_apertures(1); + if (!a) + return -ENOMEM; + + a->ranges[0].base = base; + a->ranges[0].size = size; + + ret = remove_conflicting_framebuffers(a, name, primary); + kfree(a); + + if (ret) + return ret; +#endif + + detach_devices(base, size); + + return 0; +} +EXPORT_SYMBOL(remove_conflicting_devices); + +/** + * remove_conflicting_pci_devices - remove existing framebuffers for PCI devices + * @pdev: PCI device + * @name: a descriptive name of the requesting driver + * + * This function removes devices that own apertures within any of @pdev's + * memory bars. The function assumes that PCI device with shadowed ROM + * drives a primary display and therefore kicks out vga16fb as well. + * + * Returns: + * 0 on success, or a negative errno code otherwise + */ +int remove_conflicting_pci_devices(struct pci_dev *pdev, const char *name) +{ + resource_size_t base, size; + int bar, ret = 0; + + for (bar = 0; bar < PCI_STD_NUM_BARS; ++bar) { + if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) + continue; + base = pci_resource_start(pdev, bar); + size = pci_resource_len(pdev, bar); + detach_devices(base, size); + } + + /* + * WARNING: Apparently we must kick fbdev drivers before vgacon, + * otherwise the vga fbdev driver falls over. + */ +#if IS_REACHABLE(CONFIG_FB) + ret = remove_conflicting_pci_framebuffers(pdev, name); +#endif + if (ret == 0) + ret = vga_remove_vgacon(pdev); + return ret; +} +EXPORT_SYMBOL(remove_conflicting_pci_devices); diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig index 40c50fa2dd70..7f3c44e1538b 100644 --- a/drivers/video/console/Kconfig +++ b/drivers/video/console/Kconfig @@ -10,6 +10,7 @@ config VGA_CONSOLE depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC && !SUPERH && \ (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \ !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !S390 && !UML + select APERTURE_HELPERS if (DRM || FB || VFIO_PCI) default y help Saying Y here will allow you to use Linux in text mode through a diff --git a/drivers/video/fbdev/Kconfig b/drivers/video/fbdev/Kconfig index bd849013f16f..7b398e7e3cc8 100644 --- a/drivers/video/fbdev/Kconfig +++ b/drivers/video/fbdev/Kconfig @@ -455,6 +455,7 @@ config FB_ATARI config FB_OF bool "Open Firmware frame buffer device support" depends on (FB = y) && PPC && (!PPC_PSERIES || PCI) + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT @@ -527,6 +528,7 @@ config FB_IMSTT config FB_VGA16 tristate "VGA 16-color graphics support" depends on FB && (X86 || PPC) + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT @@ -551,7 +553,7 @@ config FB_STI BIOS routines contained in a ROM chip in HP PA-RISC based machines. Enabling this option will implement the linux framebuffer device using calls to the STI BIOS routines for initialisation. - + If you enable this option, you will get a planar framebuffer device /dev/fb which will work on the most common HP graphic cards of the NGLE family, including the artist chips (in the 7xx and Bxxx series), @@ -617,6 +619,7 @@ config FB_UVESA config FB_VESA bool "VESA VGA graphics support" depends on (FB = y) && (X86 || COMPILE_TEST) + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT @@ -630,6 +633,7 @@ config FB_VESA config FB_EFI bool "EFI-based Framebuffer Support" depends on (FB = y) && !IA64 && EFI + select APERTURE_HELPERS select DRM_PANEL_ORIENTATION_QUIRKS select FB_CFB_FILLRECT select FB_CFB_COPYAREA @@ -2190,6 +2194,7 @@ config FB_SIMPLE tristate "Simple framebuffer support" depends on FB depends on !DRM_SIMPLEDRM + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT diff --git a/include/linux/aperture.h b/include/linux/aperture.h new file mode 100644 index 000000000000..0a206cdce606 --- /dev/null +++ b/include/linux/aperture.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: MIT */ + +#ifndef _LINUX_APERTURE_H_ +#define _LINUX_APERTURE_H_ + +#include <linux/types.h> + +struct pci_dev; +struct platform_device; + +#if defined(CONFIG_APERTURE_HELPERS) +int devm_acquire_aperture_of_platform_device(struct platform_device *pdev, + resource_size_t base, + resource_size_t size); + +int remove_conflicting_devices(resource_size_t base, resource_size_t size, + bool primary, const char *name); + +int remove_conflicting_pci_devices(struct pci_dev *pdev, const char *name); +#else +static inline int devm_acquire_aperture_of_platform_device(struct platform_device *pdev, + resource_size_t base, + resource_size_t size) +{ + return 0; +} + +static inline int remove_conflicting_devices(resource_size_t base, resource_size_t size, + bool primary, const char *name) +{ + return 0; +} + +static inline int remove_conflicting_pci_devices(struct pci_dev *pdev, const char *name) +{ + return 0; +} +#endif + +/** + * remove_all_conflicting_devices - remove all existing framebuffers + * @primary: also kick vga16fb if present; only relevant for VGA devices + * @name: a descriptive name of the requesting driver + * + * This function removes all graphics device drivers. Use this function on systems + * that can have their framebuffer located anywhere in memory. + * + * Returns: + * 0 on success, or a negative errno code otherwise + */ +static inline int remove_all_conflicting_devices(bool primary, const char *name) +{ + return remove_conflicting_devices(0, (resource_size_t)-1, primary, name); +} + +#endif -- 2.36.1 [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 840 bytes --] ^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers 2022-06-09 9:13 ` Thomas Zimmermann @ 2022-06-09 21:41 ` Alex Williamson -1 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-09 21:41 UTC (permalink / raw) To: Thomas Zimmermann Cc: kvm, airlied, linux-kernel, dri-devel, Laszlo Ersek, Gerd Hoffmann On Thu, 9 Jun 2022 11:13:22 +0200 Thomas Zimmermann <tzimmermann@suse.de> wrote: > > Please have a look at the attached patch. It moves the aperture helpers > to a location common to the various possible users (DRM, fbdev, vfio). > The DRM interfaces remain untouched for now. The patch should provide > what you need in vfio and also serve our future use cases for graphics > drivers. If possible, please create your patch on top of it. Looks good to me, this of course makes the vfio change quite trivial. One change I'd request: diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig index 40c50fa2dd70..7f3c44e1538b 100644 --- a/drivers/video/console/Kconfig +++ b/drivers/video/console/Kconfig @@ -10,6 +10,7 @@ config VGA_CONSOLE depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC && !SUPERH && \ (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \ !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !S390 && !UML + select APERTURE_HELPERS if (DRM || FB || VFIO_PCI) default y help Saying Y here will allow you to use Linux in text mode through a This should be VFIO_PCI_CORE. Thanks, Alex ^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers @ 2022-06-09 21:41 ` Alex Williamson 0 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-09 21:41 UTC (permalink / raw) To: Thomas Zimmermann Cc: kvm, airlied, linux-kernel, dri-devel, Gerd Hoffmann, Laszlo Ersek On Thu, 9 Jun 2022 11:13:22 +0200 Thomas Zimmermann <tzimmermann@suse.de> wrote: > > Please have a look at the attached patch. It moves the aperture helpers > to a location common to the various possible users (DRM, fbdev, vfio). > The DRM interfaces remain untouched for now. The patch should provide > what you need in vfio and also serve our future use cases for graphics > drivers. If possible, please create your patch on top of it. Looks good to me, this of course makes the vfio change quite trivial. One change I'd request: diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig index 40c50fa2dd70..7f3c44e1538b 100644 --- a/drivers/video/console/Kconfig +++ b/drivers/video/console/Kconfig @@ -10,6 +10,7 @@ config VGA_CONSOLE depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC && !SUPERH && \ (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \ !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !S390 && !UML + select APERTURE_HELPERS if (DRM || FB || VFIO_PCI) default y help Saying Y here will allow you to use Linux in text mode through a This should be VFIO_PCI_CORE. Thanks, Alex ^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers 2022-06-09 21:41 ` Alex Williamson @ 2022-06-09 21:44 ` Alex Williamson -1 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-09 21:44 UTC (permalink / raw) To: Thomas Zimmermann Cc: kvm, airlied, linux-kernel, dri-devel, Laszlo Ersek, Gerd Hoffmann On Thu, 9 Jun 2022 15:41:02 -0600 Alex Williamson <alex.williamson@redhat.com> wrote: > On Thu, 9 Jun 2022 11:13:22 +0200 > Thomas Zimmermann <tzimmermann@suse.de> wrote: > > > > Please have a look at the attached patch. It moves the aperture helpers > > to a location common to the various possible users (DRM, fbdev, vfio). > > The DRM interfaces remain untouched for now. The patch should provide > > what you need in vfio and also serve our future use cases for graphics > > drivers. If possible, please create your patch on top of it. > > Looks good to me, this of course makes the vfio change quite trivial. > One change I'd request: > > diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig > index 40c50fa2dd70..7f3c44e1538b 100644 > --- a/drivers/video/console/Kconfig > +++ b/drivers/video/console/Kconfig > @@ -10,6 +10,7 @@ config VGA_CONSOLE > depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC && !SUPERH && \ > (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \ > !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !S390 && !UML > + select APERTURE_HELPERS if (DRM || FB || VFIO_PCI) > default y > help > Saying Y here will allow you to use Linux in text mode through a > > This should be VFIO_PCI_CORE. Thanks, Also, whatever tree this lands in, I'd appreciate a topic branch being made available so I can more easily get the vfio change in on the same release. Thanks, Alex ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers @ 2022-06-09 21:44 ` Alex Williamson 0 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-09 21:44 UTC (permalink / raw) To: Thomas Zimmermann Cc: kvm, airlied, linux-kernel, dri-devel, Gerd Hoffmann, Laszlo Ersek On Thu, 9 Jun 2022 15:41:02 -0600 Alex Williamson <alex.williamson@redhat.com> wrote: > On Thu, 9 Jun 2022 11:13:22 +0200 > Thomas Zimmermann <tzimmermann@suse.de> wrote: > > > > Please have a look at the attached patch. It moves the aperture helpers > > to a location common to the various possible users (DRM, fbdev, vfio). > > The DRM interfaces remain untouched for now. The patch should provide > > what you need in vfio and also serve our future use cases for graphics > > drivers. If possible, please create your patch on top of it. > > Looks good to me, this of course makes the vfio change quite trivial. > One change I'd request: > > diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig > index 40c50fa2dd70..7f3c44e1538b 100644 > --- a/drivers/video/console/Kconfig > +++ b/drivers/video/console/Kconfig > @@ -10,6 +10,7 @@ config VGA_CONSOLE > depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC && !SUPERH && \ > (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \ > !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !S390 && !UML > + select APERTURE_HELPERS if (DRM || FB || VFIO_PCI) > default y > help > Saying Y here will allow you to use Linux in text mode through a > > This should be VFIO_PCI_CORE. Thanks, Also, whatever tree this lands in, I'd appreciate a topic branch being made available so I can more easily get the vfio change in on the same release. Thanks, Alex ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers 2022-06-09 21:44 ` Alex Williamson (?) @ 2022-06-10 7:03 ` Thomas Zimmermann 2022-06-10 14:30 ` Alex Williamson -1 siblings, 1 reply; 30+ messages in thread From: Thomas Zimmermann @ 2022-06-10 7:03 UTC (permalink / raw) To: Alex Williamson Cc: kvm, airlied, linux-kernel, dri-devel, Gerd Hoffmann, Laszlo Ersek [-- Attachment #1.1.1: Type: text/plain, Size: 2087 bytes --] Hi Am 09.06.22 um 23:44 schrieb Alex Williamson: > On Thu, 9 Jun 2022 15:41:02 -0600 > Alex Williamson <alex.williamson@redhat.com> wrote: > >> On Thu, 9 Jun 2022 11:13:22 +0200 >> Thomas Zimmermann <tzimmermann@suse.de> wrote: >>> >>> Please have a look at the attached patch. It moves the aperture helpers >>> to a location common to the various possible users (DRM, fbdev, vfio). >>> The DRM interfaces remain untouched for now. The patch should provide >>> what you need in vfio and also serve our future use cases for graphics >>> drivers. If possible, please create your patch on top of it. >> >> Looks good to me, this of course makes the vfio change quite trivial. >> One change I'd request: >> >> diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig >> index 40c50fa2dd70..7f3c44e1538b 100644 >> --- a/drivers/video/console/Kconfig >> +++ b/drivers/video/console/Kconfig >> @@ -10,6 +10,7 @@ config VGA_CONSOLE >> depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC && !SUPERH && \ >> (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \ >> !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !S390 && !UML >> + select APERTURE_HELPERS if (DRM || FB || VFIO_PCI) >> default y >> help >> Saying Y here will allow you to use Linux in text mode through a >> >> This should be VFIO_PCI_CORE. Thanks, I attached an updated patch to this email. > > Also, whatever tree this lands in, I'd appreciate a topic branch being > made available so I can more easily get the vfio change in on the same > release. Thanks, You can add my patch to your series and merge it through vfio. You'd only have to cc dri-devel for the patch's review. I guess it's more important for vfio than DRM. We have no hurry on the DRM side, but v5.20 would be nice. Best regards Thomas > > Alex > -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev [-- Attachment #1.1.2: 0001-drm-Implement-DRM-aperture-helpers-under-video.patch --] [-- Type: text/x-patch, Size: 24436 bytes --] From 5e0293b7102c0772568ca490ef53b7e8775ed761 Mon Sep 17 00:00:00 2001 From: Thomas Zimmermann <tzimmermann@suse.de> Date: Wed, 8 Jun 2022 20:52:50 +0200 Subject: [PATCH] drm: Implement DRM aperture helpers under video/ Implement DRM's aperture helpers under video/ for sharing with other sub-systems. Remove DRM-isms from the interface. The helpers track the ownership of framebuffer apertures and provide hand-over from firmware, such as EFI and VESA, to native graphics drivers. Other subsystems, such as fbdev and vfio, also have to maintain ownership of framebuffer apertures. Moving DRM's aperture helpers to a more public location allows all subsystems to interact with each other and share a common implementation. The aperture helpers are selected by the various firmware drivers within DRM and fbdev, and the VGA text-console driver. The original DRM interface is kept in place for use by DRM drivers. Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de> --- Documentation/driver-api/aperture.rst | 13 + Documentation/driver-api/index.rst | 1 + drivers/gpu/drm/drm_aperture.c | 174 +------------ drivers/gpu/drm/tiny/Kconfig | 1 + drivers/video/Kconfig | 6 + drivers/video/Makefile | 2 + drivers/video/aperture.c | 340 ++++++++++++++++++++++++++ drivers/video/console/Kconfig | 1 + drivers/video/fbdev/Kconfig | 7 +- include/linux/aperture.h | 56 +++++ 10 files changed, 435 insertions(+), 166 deletions(-) create mode 100644 Documentation/driver-api/aperture.rst create mode 100644 drivers/video/aperture.c create mode 100644 include/linux/aperture.h diff --git a/Documentation/driver-api/aperture.rst b/Documentation/driver-api/aperture.rst new file mode 100644 index 000000000000..d173f4e7a7d9 --- /dev/null +++ b/Documentation/driver-api/aperture.rst @@ -0,0 +1,13 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Managing Ownership of the Framebuffer Aperture +============================================== + +.. kernel-doc:: drivers/video/aperture.c + :doc: overview + +.. kernel-doc:: include/linux/aperture.h + :internal: + +.. kernel-doc:: drivers/video/aperture.c + :export: diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst index a7b0223e2886..d6d6e77d8afc 100644 --- a/Documentation/driver-api/index.rst +++ b/Documentation/driver-api/index.rst @@ -27,6 +27,7 @@ available subsections can be seen below. component message-based infiniband + aperture frame-buffer regulator reset diff --git a/drivers/gpu/drm/drm_aperture.c b/drivers/gpu/drm/drm_aperture.c index 74bd4a76b253..388a205bd023 100644 --- a/drivers/gpu/drm/drm_aperture.c +++ b/drivers/gpu/drm/drm_aperture.c @@ -1,14 +1,7 @@ // SPDX-License-Identifier: MIT -#include <linux/device.h> -#include <linux/fb.h> -#include <linux/list.h> -#include <linux/mutex.h> -#include <linux/pci.h> -#include <linux/platform_device.h> /* for firmware helpers */ -#include <linux/slab.h> -#include <linux/types.h> -#include <linux/vgaarb.h> +#include <linux/aperture.h> +#include <linux/platform_device.h> #include <drm/drm_aperture.h> #include <drm/drm_drv.h> @@ -126,92 +119,6 @@ * afterwards. */ -struct drm_aperture { - struct drm_device *dev; - resource_size_t base; - resource_size_t size; - struct list_head lh; - void (*detach)(struct drm_device *dev); -}; - -static LIST_HEAD(drm_apertures); -static DEFINE_MUTEX(drm_apertures_lock); - -static bool overlap(resource_size_t base1, resource_size_t end1, - resource_size_t base2, resource_size_t end2) -{ - return (base1 < end2) && (end1 > base2); -} - -static void devm_aperture_acquire_release(void *data) -{ - struct drm_aperture *ap = data; - bool detached = !ap->dev; - - if (detached) - return; - - mutex_lock(&drm_apertures_lock); - list_del(&ap->lh); - mutex_unlock(&drm_apertures_lock); -} - -static int devm_aperture_acquire(struct drm_device *dev, - resource_size_t base, resource_size_t size, - void (*detach)(struct drm_device *)) -{ - size_t end = base + size; - struct list_head *pos; - struct drm_aperture *ap; - - mutex_lock(&drm_apertures_lock); - - list_for_each(pos, &drm_apertures) { - ap = container_of(pos, struct drm_aperture, lh); - if (overlap(base, end, ap->base, ap->base + ap->size)) { - mutex_unlock(&drm_apertures_lock); - return -EBUSY; - } - } - - ap = devm_kzalloc(dev->dev, sizeof(*ap), GFP_KERNEL); - if (!ap) { - mutex_unlock(&drm_apertures_lock); - return -ENOMEM; - } - - ap->dev = dev; - ap->base = base; - ap->size = size; - ap->detach = detach; - INIT_LIST_HEAD(&ap->lh); - - list_add(&ap->lh, &drm_apertures); - - mutex_unlock(&drm_apertures_lock); - - return devm_add_action_or_reset(dev->dev, devm_aperture_acquire_release, ap); -} - -static void drm_aperture_detach_firmware(struct drm_device *dev) -{ - struct platform_device *pdev = to_platform_device(dev->dev); - - /* - * Remove the device from the device hierarchy. This is the right thing - * to do for firmware-based DRM drivers, such as EFI, VESA or VGA. After - * the new driver takes over the hardware, the firmware device's state - * will be lost. - * - * For non-platform devices, a new callback would be required. - * - * If the aperture helpers ever need to handle native drivers, this call - * would only have to unplug the DRM device, so that the hardware device - * stays around after detachment. - */ - platform_device_unregister(pdev); -} - /** * devm_aperture_acquire_from_firmware - Acquires ownership of a firmware framebuffer * on behalf of a DRM driver. @@ -239,39 +146,16 @@ static void drm_aperture_detach_firmware(struct drm_device *dev) int devm_aperture_acquire_from_firmware(struct drm_device *dev, resource_size_t base, resource_size_t size) { + struct platform_device *pdev; + if (drm_WARN_ON(dev, !dev_is_platform(dev->dev))) return -EINVAL; - return devm_aperture_acquire(dev, base, size, drm_aperture_detach_firmware); -} -EXPORT_SYMBOL(devm_aperture_acquire_from_firmware); - -static void drm_aperture_detach_drivers(resource_size_t base, resource_size_t size) -{ - resource_size_t end = base + size; - struct list_head *pos, *n; - - mutex_lock(&drm_apertures_lock); - - list_for_each_safe(pos, n, &drm_apertures) { - struct drm_aperture *ap = - container_of(pos, struct drm_aperture, lh); - struct drm_device *dev = ap->dev; - - if (WARN_ON_ONCE(!dev)) - continue; - - if (!overlap(base, end, ap->base, ap->base + ap->size)) - continue; + pdev = to_platform_device(dev->dev); - ap->dev = NULL; /* detach from device */ - list_del(&ap->lh); - - ap->detach(dev); - } - - mutex_unlock(&drm_apertures_lock); + return devm_acquire_aperture_of_platform_device(pdev, base, size); } +EXPORT_SYMBOL(devm_aperture_acquire_from_firmware); /** * drm_aperture_remove_conflicting_framebuffers - remove existing framebuffers in the given range @@ -289,27 +173,7 @@ static void drm_aperture_detach_drivers(resource_size_t base, resource_size_t si int drm_aperture_remove_conflicting_framebuffers(resource_size_t base, resource_size_t size, bool primary, const struct drm_driver *req_driver) { -#if IS_REACHABLE(CONFIG_FB) - struct apertures_struct *a; - int ret; - - a = alloc_apertures(1); - if (!a) - return -ENOMEM; - - a->ranges[0].base = base; - a->ranges[0].size = size; - - ret = remove_conflicting_framebuffers(a, req_driver->name, primary); - kfree(a); - - if (ret) - return ret; -#endif - - drm_aperture_detach_drivers(base, size); - - return 0; + return remove_conflicting_devices(base, size, primary, req_driver->name); } EXPORT_SYMBOL(drm_aperture_remove_conflicting_framebuffers); @@ -328,26 +192,6 @@ EXPORT_SYMBOL(drm_aperture_remove_conflicting_framebuffers); int drm_aperture_remove_conflicting_pci_framebuffers(struct pci_dev *pdev, const struct drm_driver *req_driver) { - resource_size_t base, size; - int bar, ret = 0; - - for (bar = 0; bar < PCI_STD_NUM_BARS; ++bar) { - if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) - continue; - base = pci_resource_start(pdev, bar); - size = pci_resource_len(pdev, bar); - drm_aperture_detach_drivers(base, size); - } - - /* - * WARNING: Apparently we must kick fbdev drivers before vgacon, - * otherwise the vga fbdev driver falls over. - */ -#if IS_REACHABLE(CONFIG_FB) - ret = remove_conflicting_pci_framebuffers(pdev, req_driver->name); -#endif - if (ret == 0) - ret = vga_remove_vgacon(pdev); - return ret; + return remove_conflicting_pci_devices(pdev, req_driver->name); } EXPORT_SYMBOL(drm_aperture_remove_conflicting_pci_framebuffers); diff --git a/drivers/gpu/drm/tiny/Kconfig b/drivers/gpu/drm/tiny/Kconfig index 627d637a1e7e..027cd87c3d0d 100644 --- a/drivers/gpu/drm/tiny/Kconfig +++ b/drivers/gpu/drm/tiny/Kconfig @@ -69,6 +69,7 @@ config DRM_PANEL_MIPI_DBI config DRM_SIMPLEDRM tristate "Simple framebuffer driver" depends on DRM && MMU + select APERTURE_HELPERS select DRM_GEM_SHMEM_HELPER select DRM_KMS_HELPER help diff --git a/drivers/video/Kconfig b/drivers/video/Kconfig index 427a993c7f57..c69b45f8c427 100644 --- a/drivers/video/Kconfig +++ b/drivers/video/Kconfig @@ -5,6 +5,12 @@ menu "Graphics support" +config APERTURE_HELPERS + bool + help + Support tracking and hand-over of aperture ownership. Required + for firmware graphics drivers. + if HAS_IOMEM config HAVE_FB_ATMEL diff --git a/drivers/video/Makefile b/drivers/video/Makefile index df7650adede9..5bb6b452cc83 100644 --- a/drivers/video/Makefile +++ b/drivers/video/Makefile @@ -1,4 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_APERTURE_HELPERS) += aperture.o obj-$(CONFIG_VGASTATE) += vgastate.o obj-$(CONFIG_HDMI) += hdmi.o diff --git a/drivers/video/aperture.c b/drivers/video/aperture.c new file mode 100644 index 000000000000..f0b877e9c256 --- /dev/null +++ b/drivers/video/aperture.c @@ -0,0 +1,340 @@ +// SPDX-License-Identifier: MIT + +#include <linux/aperture.h> +#include <linux/device.h> +#include <linux/fb.h> /* for old fbdev helpers */ +#include <linux/list.h> +#include <linux/mutex.h> +#include <linux/pci.h> +#include <linux/platform_device.h> +#include <linux/slab.h> +#include <linux/types.h> +#include <linux/vgaarb.h> + +/** + * DOC: overview + * + * A graphics device might be supported by different drivers, but only one + * driver can be active at any given time. Many systems load a generic + * graphics drivers, such as EFI-GOP or VESA, early during the boot process. + * During later boot stages, they replace the generic driver with a dedicated, + * hardware-specific driver. To take over the device the dedicated driver + * first has to remove the generic driver. Aperture functions manage + * ownership of framebuffer memory and hand-over between drivers. + * + * Graphics drivers should call remove_conflicting_devices() + * at the top of their probe function. The function removes any generic + * driver that is currently associated with the given framebuffer memory. + * If the framebuffer is located at PCI BAR 0, the rsp code looks as in the + * example given below. The cod assumes a DRM driver. + * + * .. code-block:: c + * + * static const struct drm_driver example_driver = { + * .name = "exampledrm", + * ... + * }; + * + * static int remove_conflicting_framebuffers(struct pci_dev *pdev) + * { + * bool primary = false; + * resource_size_t base, size; + * int ret; + * + * base = pci_resource_start(pdev, 0); + * size = pci_resource_len(pdev, 0); + * #ifdef CONFIG_X86 + * primary = pdev->resource[PCI_ROM_RESOURCE].flags & IORESOURCE_ROM_SHADOW; + * #endif + * + * return remove_conflicting_devices(base, size, primary, &example_driver->name); + * } + * + * static int probe(struct pci_dev *pdev) + * { + * int ret; + * + * // Remove any generic drivers... + * ret = remove_conflicting_framebuffers(pdev); + * if (ret) + * return ret; + * + * // ... and initialize the hardware. + * ... + * + * drm_dev_register(); + * + * return 0; + * } + * + * PCI device drivers can also call remove_conflicting_pci_devices() and let the + * function detect the apertures automatically. Device drivers without knowledge of + * the framebuffer's location shall call remove_all_conflicting_devices(), + * which removes all known devices. + * + * Drivers that are susceptible to being removed by other drivers, such as + * generic EFI or VESA drivers, have to register themselves as owners of their + * framebuffer apertures. Ownership of the framebuffer memory is achieved + * by calling devm_acquire_aperture_of_platform_device(). On success, the driver + * is the owner of the framebuffer range. The function fails if the + * framebuffer is already owned by another driver. See below for an example. + * + * .. code-block:: c + * + * static int acquire_framebuffers(struct drm_device *dev, struct platform_device *pdev) + * { + * resource_size_t base, size; + * + * mem = platform_get_resource(pdev, IORESOURCE_MEM, 0); + * if (!mem) + * return -EINVAL; + * base = mem->start; + * size = resource_size(mem); + * + * return devm_acquire_aperture_of_platform_device(pdev, base, size); + * } + * + * static int probe(struct platform_device *pdev) + * { + * struct drm_device *dev; + * int ret; + * + * // ... Initialize the device... + * dev = devm_drm_dev_alloc(); + * ... + * + * // ... and acquire ownership of the framebuffer. + * ret = acquire_framebuffers(dev, pdev); + * if (ret) + * return ret; + * + * drm_dev_register(dev, 0); + * + * return 0; + * } + * + * The generic driver is now subject to forced removal by other drivers. This + * only works for platform drivers that support hot unplugging. + * When a driver calls remove_conflicting_devices() et al + * for the registered framebuffer range, the aperture helpers call + * platform_device_unregister() and the generic driver unloads itself. It + * may not access the device's registers, framebuffer memory, ROM, etc + * afterwards. + */ + +struct dev_aperture { + struct device *dev; + resource_size_t base; + resource_size_t size; + struct list_head lh; + void (*detach)(struct device *dev); +}; + +static LIST_HEAD(apertures); +static DEFINE_MUTEX(apertures_lock); + +static bool overlap(resource_size_t base1, resource_size_t end1, + resource_size_t base2, resource_size_t end2) +{ + return (base1 < end2) && (end1 > base2); +} + +static void devm_aperture_acquire_release(void *data) +{ + struct dev_aperture *ap = data; + bool detached = !ap->dev; + + if (detached) + return; + + mutex_lock(&apertures_lock); + list_del(&ap->lh); + mutex_unlock(&apertures_lock); +} + +static int devm_aperture_acquire(struct device *dev, + resource_size_t base, resource_size_t size, + void (*detach)(struct device *)) +{ + size_t end = base + size; + struct list_head *pos; + struct dev_aperture *ap; + + mutex_lock(&apertures_lock); + + list_for_each(pos, &apertures) { + ap = container_of(pos, struct dev_aperture, lh); + if (overlap(base, end, ap->base, ap->base + ap->size)) { + mutex_unlock(&apertures_lock); + return -EBUSY; + } + } + + ap = devm_kzalloc(dev, sizeof(*ap), GFP_KERNEL); + if (!ap) { + mutex_unlock(&apertures_lock); + return -ENOMEM; + } + + ap->dev = dev; + ap->base = base; + ap->size = size; + ap->detach = detach; + INIT_LIST_HEAD(&ap->lh); + + list_add(&ap->lh, &apertures); + + mutex_unlock(&apertures_lock); + + return devm_add_action_or_reset(dev, devm_aperture_acquire_release, ap); +} + +static void detach_platform_device(struct device *dev) +{ + struct platform_device *pdev = to_platform_device(dev); + + /* + * Remove the device from the device hierarchy. This is the right thing + * to do for firmware-based DRM drivers, such as EFI, VESA or VGA. After + * the new driver takes over the hardware, the firmware device's state + * will be lost. + * + * For non-platform devices, a new callback would be required. + * + * If the aperture helpers ever need to handle native drivers, this call + * would only have to unplug the DRM device, so that the hardware device + * stays around after detachment. + */ + platform_device_unregister(pdev); +} + +/** + * devm_acquire_aperture_of_platform_device - Acquires ownership of an aperture + * on behalf of a platform device. + * @pdev: the platform device to own the aperture + * @base: the aperture's byte offset in physical memory + * @size: the aperture size in bytes + * + * Installs the given device as the new owner of the aperture. The function + * expects the aperture to be provided by a platform device. If another + * driver takes over ownership of the aperture, aperture helpers will then + * unregister the platform device automatically. All acquired apertures are + * released automatically when the underlying device goes away. + * + * The function fails if the aperture, or parts of it, is currently + * owned by another device. To evict current owners, callers should use + * remove_conflicting_devices() et al. before calling this function. + * + * Returns: + * 0 on success, or a negative errno value otherwise. + */ +int devm_acquire_aperture_of_platform_device(struct platform_device *pdev, + resource_size_t base, + resource_size_t size) +{ + return devm_aperture_acquire(&pdev->dev, base, size, detach_platform_device); +} +EXPORT_SYMBOL(devm_acquire_aperture_of_platform_device); + +static void detach_devices(resource_size_t base, resource_size_t size) +{ + resource_size_t end = base + size; + struct list_head *pos, *n; + + mutex_lock(&apertures_lock); + + list_for_each_safe(pos, n, &apertures) { + struct dev_aperture *ap = container_of(pos, struct dev_aperture, lh); + struct device *dev = ap->dev; + + if (WARN_ON_ONCE(!dev)) + continue; + + if (!overlap(base, end, ap->base, ap->base + ap->size)) + continue; + + ap->dev = NULL; /* detach from device */ + list_del(&ap->lh); + + ap->detach(dev); + } + + mutex_unlock(&apertures_lock); +} + +/** + * remove_conflicting_devices - remove devices in the given range + * @base: the aperture's base address in physical memory + * @size: aperture size in bytes + * @primary: also kick vga16fb if present; only relevant for VGA devices + * @name: a descriptive name of the requesting driver + * + * This function removes devices that own apertures within @base and @size. + * + * Returns: + * 0 on success, or a negative errno code otherwise + */ +int remove_conflicting_devices(resource_size_t base, resource_size_t size, bool primary, + const char *name) +{ +#if IS_REACHABLE(CONFIG_FB) + struct apertures_struct *a; + int ret; + + a = alloc_apertures(1); + if (!a) + return -ENOMEM; + + a->ranges[0].base = base; + a->ranges[0].size = size; + + ret = remove_conflicting_framebuffers(a, name, primary); + kfree(a); + + if (ret) + return ret; +#endif + + detach_devices(base, size); + + return 0; +} +EXPORT_SYMBOL(remove_conflicting_devices); + +/** + * remove_conflicting_pci_devices - remove existing framebuffers for PCI devices + * @pdev: PCI device + * @name: a descriptive name of the requesting driver + * + * This function removes devices that own apertures within any of @pdev's + * memory bars. The function assumes that PCI device with shadowed ROM + * drives a primary display and therefore kicks out vga16fb as well. + * + * Returns: + * 0 on success, or a negative errno code otherwise + */ +int remove_conflicting_pci_devices(struct pci_dev *pdev, const char *name) +{ + resource_size_t base, size; + int bar, ret = 0; + + for (bar = 0; bar < PCI_STD_NUM_BARS; ++bar) { + if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) + continue; + base = pci_resource_start(pdev, bar); + size = pci_resource_len(pdev, bar); + detach_devices(base, size); + } + + /* + * WARNING: Apparently we must kick fbdev drivers before vgacon, + * otherwise the vga fbdev driver falls over. + */ +#if IS_REACHABLE(CONFIG_FB) + ret = remove_conflicting_pci_framebuffers(pdev, name); +#endif + if (ret == 0) + ret = vga_remove_vgacon(pdev); + return ret; +} +EXPORT_SYMBOL(remove_conflicting_pci_devices); diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig index 40c50fa2dd70..22cea5082ac4 100644 --- a/drivers/video/console/Kconfig +++ b/drivers/video/console/Kconfig @@ -10,6 +10,7 @@ config VGA_CONSOLE depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC && !SUPERH && \ (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \ !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !S390 && !UML + select APERTURE_HELPERS if (DRM || FB || VFIO_PCI_CORE) default y help Saying Y here will allow you to use Linux in text mode through a diff --git a/drivers/video/fbdev/Kconfig b/drivers/video/fbdev/Kconfig index bd849013f16f..7b398e7e3cc8 100644 --- a/drivers/video/fbdev/Kconfig +++ b/drivers/video/fbdev/Kconfig @@ -455,6 +455,7 @@ config FB_ATARI config FB_OF bool "Open Firmware frame buffer device support" depends on (FB = y) && PPC && (!PPC_PSERIES || PCI) + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT @@ -527,6 +528,7 @@ config FB_IMSTT config FB_VGA16 tristate "VGA 16-color graphics support" depends on FB && (X86 || PPC) + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT @@ -551,7 +553,7 @@ config FB_STI BIOS routines contained in a ROM chip in HP PA-RISC based machines. Enabling this option will implement the linux framebuffer device using calls to the STI BIOS routines for initialisation. - + If you enable this option, you will get a planar framebuffer device /dev/fb which will work on the most common HP graphic cards of the NGLE family, including the artist chips (in the 7xx and Bxxx series), @@ -617,6 +619,7 @@ config FB_UVESA config FB_VESA bool "VESA VGA graphics support" depends on (FB = y) && (X86 || COMPILE_TEST) + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT @@ -630,6 +633,7 @@ config FB_VESA config FB_EFI bool "EFI-based Framebuffer Support" depends on (FB = y) && !IA64 && EFI + select APERTURE_HELPERS select DRM_PANEL_ORIENTATION_QUIRKS select FB_CFB_FILLRECT select FB_CFB_COPYAREA @@ -2190,6 +2194,7 @@ config FB_SIMPLE tristate "Simple framebuffer support" depends on FB depends on !DRM_SIMPLEDRM + select APERTURE_HELPERS select FB_CFB_FILLRECT select FB_CFB_COPYAREA select FB_CFB_IMAGEBLIT diff --git a/include/linux/aperture.h b/include/linux/aperture.h new file mode 100644 index 000000000000..0a206cdce606 --- /dev/null +++ b/include/linux/aperture.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: MIT */ + +#ifndef _LINUX_APERTURE_H_ +#define _LINUX_APERTURE_H_ + +#include <linux/types.h> + +struct pci_dev; +struct platform_device; + +#if defined(CONFIG_APERTURE_HELPERS) +int devm_acquire_aperture_of_platform_device(struct platform_device *pdev, + resource_size_t base, + resource_size_t size); + +int remove_conflicting_devices(resource_size_t base, resource_size_t size, + bool primary, const char *name); + +int remove_conflicting_pci_devices(struct pci_dev *pdev, const char *name); +#else +static inline int devm_acquire_aperture_of_platform_device(struct platform_device *pdev, + resource_size_t base, + resource_size_t size) +{ + return 0; +} + +static inline int remove_conflicting_devices(resource_size_t base, resource_size_t size, + bool primary, const char *name) +{ + return 0; +} + +static inline int remove_conflicting_pci_devices(struct pci_dev *pdev, const char *name) +{ + return 0; +} +#endif + +/** + * remove_all_conflicting_devices - remove all existing framebuffers + * @primary: also kick vga16fb if present; only relevant for VGA devices + * @name: a descriptive name of the requesting driver + * + * This function removes all graphics device drivers. Use this function on systems + * that can have their framebuffer located anywhere in memory. + * + * Returns: + * 0 on success, or a negative errno code otherwise + */ +static inline int remove_all_conflicting_devices(bool primary, const char *name) +{ + return remove_conflicting_devices(0, (resource_size_t)-1, primary, name); +} + +#endif -- 2.36.1 [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 840 bytes --] ^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers 2022-06-10 7:03 ` Thomas Zimmermann @ 2022-06-10 14:30 ` Alex Williamson 0 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-10 14:30 UTC (permalink / raw) To: Thomas Zimmermann Cc: kvm, airlied, linux-kernel, dri-devel, Gerd Hoffmann, Laszlo Ersek On Fri, 10 Jun 2022 09:03:15 +0200 Thomas Zimmermann <tzimmermann@suse.de> wrote: > Hi > > Am 09.06.22 um 23:44 schrieb Alex Williamson: > > On Thu, 9 Jun 2022 15:41:02 -0600 > > Alex Williamson <alex.williamson@redhat.com> wrote: > > > >> On Thu, 9 Jun 2022 11:13:22 +0200 > >> Thomas Zimmermann <tzimmermann@suse.de> wrote: > >>> > >>> Please have a look at the attached patch. It moves the aperture helpers > >>> to a location common to the various possible users (DRM, fbdev, vfio). > >>> The DRM interfaces remain untouched for now. The patch should provide > >>> what you need in vfio and also serve our future use cases for graphics > >>> drivers. If possible, please create your patch on top of it. > >> > >> Looks good to me, this of course makes the vfio change quite trivial. > >> One change I'd request: > >> > >> diff --git a/drivers/video/console/Kconfig b/drivers/video/console/Kconfig > >> index 40c50fa2dd70..7f3c44e1538b 100644 > >> --- a/drivers/video/console/Kconfig > >> +++ b/drivers/video/console/Kconfig > >> @@ -10,6 +10,7 @@ config VGA_CONSOLE > >> depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC && !SUPERH && \ > >> (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \ > >> !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !S390 && !UML > >> + select APERTURE_HELPERS if (DRM || FB || VFIO_PCI) > >> default y > >> help > >> Saying Y here will allow you to use Linux in text mode through a > >> > >> This should be VFIO_PCI_CORE. Thanks, > > I attached an updated patch to this email. > > > > > Also, whatever tree this lands in, I'd appreciate a topic branch being > > made available so I can more easily get the vfio change in on the same > > release. Thanks, > > You can add my patch to your series and merge it through vfio. You'd > only have to cc dri-devel for the patch's review. I guess it's more > important for vfio than DRM. We have no hurry on the DRM side, but v5.20 > would be nice. Ok, I didn't realize you were offering the patch for me to post and merge. I'll do that. Thanks! Alex ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers 2022-06-08 11:11 ` Thomas Zimmermann @ 2022-06-08 15:37 ` Gerd Hoffmann -1 siblings, 0 replies; 30+ messages in thread From: Gerd Hoffmann @ 2022-06-08 15:37 UTC (permalink / raw) To: Thomas Zimmermann Cc: Alex Williamson, maarten.lankhorst, mripard, airlied, daniel, kvm, Laszlo Ersek, dri-devel, linux-kernel Hi, > You shouldn't have to copy any of the implementation of the aperture > helpers. That comes from the aperture helpers being part of drm ... > For patch 2, the most trivial workaround is to instanciate struct drm_driver > here and set the name field to 'vdev->vdev.ops->name'. In the longer term, > the aperture helpers will be moved out of DRM and into a more prominent > location. That workaround will be cleaned up then. ... but if the long-term plan is to clean that up properly anyway I don't see the point in bike shedding too much on the details of some temporary solution. > Alternatively, drm_aperture_remove_conflicting_pci_framebuffers() could be > changed to accept the name string as second argument, but that's quite a bit > of churn within the DRM code. Also pointless churn because you'll have the very same churn again when moving the aperture helpers out of drm. take care, Gerd ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 2/2] vfio/pci: Remove console drivers @ 2022-06-08 15:37 ` Gerd Hoffmann 0 siblings, 0 replies; 30+ messages in thread From: Gerd Hoffmann @ 2022-06-08 15:37 UTC (permalink / raw) To: Thomas Zimmermann Cc: kvm, airlied, linux-kernel, Alex Williamson, dri-devel, Laszlo Ersek Hi, > You shouldn't have to copy any of the implementation of the aperture > helpers. That comes from the aperture helpers being part of drm ... > For patch 2, the most trivial workaround is to instanciate struct drm_driver > here and set the name field to 'vdev->vdev.ops->name'. In the longer term, > the aperture helpers will be moved out of DRM and into a more prominent > location. That workaround will be cleaned up then. ... but if the long-term plan is to clean that up properly anyway I don't see the point in bike shedding too much on the details of some temporary solution. > Alternatively, drm_aperture_remove_conflicting_pci_framebuffers() could be > changed to accept the name string as second argument, but that's quite a bit > of churn within the DRM code. Also pointless churn because you'll have the very same churn again when moving the aperture helpers out of drm. take care, Gerd ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior 2022-06-06 17:53 ` Alex Williamson @ 2022-06-07 17:40 ` Javier Martinez Canillas -1 siblings, 0 replies; 30+ messages in thread From: Javier Martinez Canillas @ 2022-06-07 17:40 UTC (permalink / raw) To: Alex Williamson, maarten.lankhorst, mripard, tzimmermann, airlied, daniel Cc: dri-devel, Laszlo Ersek, Gerd Hoffmann, kvm, linux-kernel Hello Alex, On 6/6/22 19:53, Alex Williamson wrote: > Users attempting to enable vfio PCI device assignment with a GPU will > often block the default PCI driver from the device to avoid conflicts > with the device initialization or release path. This means that > vfio-pci is sometimes the first PCI driver to bind to the device. In > the case of assigning the primary graphics device, low-level console > drivers may still generate resource conflicts. Users often employ > kernel command line arguments to disable conflicting drivers or > perform unbinding in userspace to avoid this, but the actual solution > is often distribution/kernel config specific based on the included > drivers. > > We can instead allow vfio-pci to copy the behavior of > drm_aperture_remove_conflicting_pci_framebuffers() in order to remove > these low-level drivers with conflicting resources. vfio-pci is not > however a DRM driver, nor does vfio-pci depend on DRM config options, > thus we split out and export the necessary DRM apterture support and > mirror the framebuffer and VGA support. > > I'd be happy to pull this series in through the vfio branch if > approved by the DRM maintainers. Thanks, > I understand your issue but I really don't think that using this helper is the correct thing to do. We already have some races with the current aperture infrastructure As an example you can look at [0]. The agreement on the mentioned thread is that we want to unify the fbdev and DRM drivers apertures into a single list, and ideally moving all to the Linux device model to handle the removal of conflicting devices. That's why I don't feel that leaking the DRM aperture helper to another is desirable since it would make even harder to cleanup this later. But also, this issue isn't something that only affects graphic devices, right? AFAIU from [1] and [2], the same issue happens if a PCI device has to be bound to vfio-pci but already was bound to a host driver. The fact that DRM happens to have some infrastructure to remove devices that conflict with an aperture is just a coincidence. Since this is used to remove devices bound to drivers that make use of the firmware-provided system framebuffer. The series [0] mentioned above, adds a sysfb_disable() that disables the Generic System Framebuffer logic that is what registers the framebuffer devices that are bound to these generic video drivers. On disable, the devices registered by sysfb are also unregistered. Would be enough for your use case to use that helper function if it lands or do you really need to look at the apertures? That is, do you want to remove the {vesa,efi,simple}fb and simpledrm drivers or is there a need to also remove real fbdev and DRM drivers? [0]: https://lore.kernel.org/lkml/YnvrxICnisXU6I1y@ravnborg.org/T/ [1]: https://www.ibm.com/docs/en/linux-on-systems?topic=through-pci [2]: https://www.kernel.org/doc/Documentation/vfio.txt -- Best regards, Javier Martinez Canillas Linux Engineering Red Hat ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior @ 2022-06-07 17:40 ` Javier Martinez Canillas 0 siblings, 0 replies; 30+ messages in thread From: Javier Martinez Canillas @ 2022-06-07 17:40 UTC (permalink / raw) To: Alex Williamson, maarten.lankhorst, mripard, tzimmermann, airlied, daniel Cc: kvm, Laszlo Ersek, Gerd Hoffmann, dri-devel, linux-kernel Hello Alex, On 6/6/22 19:53, Alex Williamson wrote: > Users attempting to enable vfio PCI device assignment with a GPU will > often block the default PCI driver from the device to avoid conflicts > with the device initialization or release path. This means that > vfio-pci is sometimes the first PCI driver to bind to the device. In > the case of assigning the primary graphics device, low-level console > drivers may still generate resource conflicts. Users often employ > kernel command line arguments to disable conflicting drivers or > perform unbinding in userspace to avoid this, but the actual solution > is often distribution/kernel config specific based on the included > drivers. > > We can instead allow vfio-pci to copy the behavior of > drm_aperture_remove_conflicting_pci_framebuffers() in order to remove > these low-level drivers with conflicting resources. vfio-pci is not > however a DRM driver, nor does vfio-pci depend on DRM config options, > thus we split out and export the necessary DRM apterture support and > mirror the framebuffer and VGA support. > > I'd be happy to pull this series in through the vfio branch if > approved by the DRM maintainers. Thanks, > I understand your issue but I really don't think that using this helper is the correct thing to do. We already have some races with the current aperture infrastructure As an example you can look at [0]. The agreement on the mentioned thread is that we want to unify the fbdev and DRM drivers apertures into a single list, and ideally moving all to the Linux device model to handle the removal of conflicting devices. That's why I don't feel that leaking the DRM aperture helper to another is desirable since it would make even harder to cleanup this later. But also, this issue isn't something that only affects graphic devices, right? AFAIU from [1] and [2], the same issue happens if a PCI device has to be bound to vfio-pci but already was bound to a host driver. The fact that DRM happens to have some infrastructure to remove devices that conflict with an aperture is just a coincidence. Since this is used to remove devices bound to drivers that make use of the firmware-provided system framebuffer. The series [0] mentioned above, adds a sysfb_disable() that disables the Generic System Framebuffer logic that is what registers the framebuffer devices that are bound to these generic video drivers. On disable, the devices registered by sysfb are also unregistered. Would be enough for your use case to use that helper function if it lands or do you really need to look at the apertures? That is, do you want to remove the {vesa,efi,simple}fb and simpledrm drivers or is there a need to also remove real fbdev and DRM drivers? [0]: https://lore.kernel.org/lkml/YnvrxICnisXU6I1y@ravnborg.org/T/ [1]: https://www.ibm.com/docs/en/linux-on-systems?topic=through-pci [2]: https://www.kernel.org/doc/Documentation/vfio.txt -- Best regards, Javier Martinez Canillas Linux Engineering Red Hat ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior 2022-06-07 17:40 ` Javier Martinez Canillas @ 2022-06-07 21:01 ` Alex Williamson -1 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-07 21:01 UTC (permalink / raw) To: Javier Martinez Canillas Cc: tzimmermann, kvm, airlied, dri-devel, linux-kernel, Gerd Hoffmann, Laszlo Ersek On Tue, 7 Jun 2022 19:40:40 +0200 Javier Martinez Canillas <javierm@redhat.com> wrote: > Hello Alex, > > On 6/6/22 19:53, Alex Williamson wrote: > > Users attempting to enable vfio PCI device assignment with a GPU will > > often block the default PCI driver from the device to avoid conflicts > > with the device initialization or release path. This means that > > vfio-pci is sometimes the first PCI driver to bind to the device. In > > the case of assigning the primary graphics device, low-level console > > drivers may still generate resource conflicts. Users often employ > > kernel command line arguments to disable conflicting drivers or > > perform unbinding in userspace to avoid this, but the actual solution > > is often distribution/kernel config specific based on the included > > drivers. > > > > We can instead allow vfio-pci to copy the behavior of > > drm_aperture_remove_conflicting_pci_framebuffers() in order to remove > > these low-level drivers with conflicting resources. vfio-pci is not > > however a DRM driver, nor does vfio-pci depend on DRM config options, > > thus we split out and export the necessary DRM apterture support and > > mirror the framebuffer and VGA support. > > > > I'd be happy to pull this series in through the vfio branch if > > approved by the DRM maintainers. Thanks, > > > > I understand your issue but I really don't think that using this helper > is the correct thing to do. We already have some races with the current > aperture infrastructure As an example you can look at [0]. > > The agreement on the mentioned thread is that we want to unify the fbdev > and DRM drivers apertures into a single list, and ideally moving all to > the Linux device model to handle the removal of conflicting devices. > > That's why I don't feel that leaking the DRM aperture helper to another > is desirable since it would make even harder to cleanup this later. OTOH, this doesn't really make the problem worse and it identifies another stakeholder to a full solution. > But also, this issue isn't something that only affects graphic devices, > right? AFAIU from [1] and [2], the same issue happens if a PCI device > has to be bound to vfio-pci but already was bound to a host driver. When we're shuffling between PCI drivers, we expect the unbind of the previous driver to have released all the claimed resources. If there were a previously attached PCI graphics driver, then the code added in patch 2/ is simply redundant since that PCI graphics driver must have already performed similar actions. The primary use case of this series is where there is no previous PCI graphics driver and we have no visibility to platform drivers carving chunks of the PCI resources into different subsystems. AFAIK, this is unique to graphics devices. > The fact that DRM happens to have some infrastructure to remove devices > that conflict with an aperture is just a coincidence. Since this is used > to remove devices bound to drivers that make use of the firmware-provided > system framebuffer. It seems not so much a coincidence as an artifact of the exact problem both PCI graphics drivers and now vfio-pci face. We've created platform devices to manage sub-ranges of resources, where the actual parent of those resources is only discovered later and we don't automatically resolve the resource conflict between that parent device and these platform devices when binding the parent driver. > The series [0] mentioned above, adds a sysfb_disable() that disables the > Generic System Framebuffer logic that is what registers the framebuffer > devices that are bound to these generic video drivers. On disable, the > devices registered by sysfb are also unregistered. > > Would be enough for your use case to use that helper function if it lands > or do you really need to look at the apertures? That is, do you want to > remove the {vesa,efi,simple}fb and simpledrm drivers or is there a need > to also remove real fbdev and DRM drivers? It's not clear to me how this helps. I infer that sysfb_disable() is intended to be used by, for example, a PCI console driver which would be taking over the console and can therefore make a clear decision to end sysfb support. The vfio-pci driver is not a console driver so we can't make any sort of blind assertion regarding sysfb. We might be binding to a secondary graphics card which has no sysfb drivers attached. I'm a lot more comfortable wielding an interface that intends to disable drivers/devices relative to the resources of a given device rather than a blanket means to disable a subsystem. I wonder if the race issues aren't better solved by avoiding to create platform devices exposing resource conflicts with known devices, especially when those existing devices have drivers attached. Thanks, Alex ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior @ 2022-06-07 21:01 ` Alex Williamson 0 siblings, 0 replies; 30+ messages in thread From: Alex Williamson @ 2022-06-07 21:01 UTC (permalink / raw) To: Javier Martinez Canillas Cc: maarten.lankhorst, mripard, tzimmermann, airlied, daniel, kvm, Laszlo Ersek, Gerd Hoffmann, dri-devel, linux-kernel On Tue, 7 Jun 2022 19:40:40 +0200 Javier Martinez Canillas <javierm@redhat.com> wrote: > Hello Alex, > > On 6/6/22 19:53, Alex Williamson wrote: > > Users attempting to enable vfio PCI device assignment with a GPU will > > often block the default PCI driver from the device to avoid conflicts > > with the device initialization or release path. This means that > > vfio-pci is sometimes the first PCI driver to bind to the device. In > > the case of assigning the primary graphics device, low-level console > > drivers may still generate resource conflicts. Users often employ > > kernel command line arguments to disable conflicting drivers or > > perform unbinding in userspace to avoid this, but the actual solution > > is often distribution/kernel config specific based on the included > > drivers. > > > > We can instead allow vfio-pci to copy the behavior of > > drm_aperture_remove_conflicting_pci_framebuffers() in order to remove > > these low-level drivers with conflicting resources. vfio-pci is not > > however a DRM driver, nor does vfio-pci depend on DRM config options, > > thus we split out and export the necessary DRM apterture support and > > mirror the framebuffer and VGA support. > > > > I'd be happy to pull this series in through the vfio branch if > > approved by the DRM maintainers. Thanks, > > > > I understand your issue but I really don't think that using this helper > is the correct thing to do. We already have some races with the current > aperture infrastructure As an example you can look at [0]. > > The agreement on the mentioned thread is that we want to unify the fbdev > and DRM drivers apertures into a single list, and ideally moving all to > the Linux device model to handle the removal of conflicting devices. > > That's why I don't feel that leaking the DRM aperture helper to another > is desirable since it would make even harder to cleanup this later. OTOH, this doesn't really make the problem worse and it identifies another stakeholder to a full solution. > But also, this issue isn't something that only affects graphic devices, > right? AFAIU from [1] and [2], the same issue happens if a PCI device > has to be bound to vfio-pci but already was bound to a host driver. When we're shuffling between PCI drivers, we expect the unbind of the previous driver to have released all the claimed resources. If there were a previously attached PCI graphics driver, then the code added in patch 2/ is simply redundant since that PCI graphics driver must have already performed similar actions. The primary use case of this series is where there is no previous PCI graphics driver and we have no visibility to platform drivers carving chunks of the PCI resources into different subsystems. AFAIK, this is unique to graphics devices. > The fact that DRM happens to have some infrastructure to remove devices > that conflict with an aperture is just a coincidence. Since this is used > to remove devices bound to drivers that make use of the firmware-provided > system framebuffer. It seems not so much a coincidence as an artifact of the exact problem both PCI graphics drivers and now vfio-pci face. We've created platform devices to manage sub-ranges of resources, where the actual parent of those resources is only discovered later and we don't automatically resolve the resource conflict between that parent device and these platform devices when binding the parent driver. > The series [0] mentioned above, adds a sysfb_disable() that disables the > Generic System Framebuffer logic that is what registers the framebuffer > devices that are bound to these generic video drivers. On disable, the > devices registered by sysfb are also unregistered. > > Would be enough for your use case to use that helper function if it lands > or do you really need to look at the apertures? That is, do you want to > remove the {vesa,efi,simple}fb and simpledrm drivers or is there a need > to also remove real fbdev and DRM drivers? It's not clear to me how this helps. I infer that sysfb_disable() is intended to be used by, for example, a PCI console driver which would be taking over the console and can therefore make a clear decision to end sysfb support. The vfio-pci driver is not a console driver so we can't make any sort of blind assertion regarding sysfb. We might be binding to a secondary graphics card which has no sysfb drivers attached. I'm a lot more comfortable wielding an interface that intends to disable drivers/devices relative to the resources of a given device rather than a blanket means to disable a subsystem. I wonder if the race issues aren't better solved by avoiding to create platform devices exposing resource conflicts with known devices, especially when those existing devices have drivers attached. Thanks, Alex ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior 2022-06-07 17:40 ` Javier Martinez Canillas @ 2022-06-08 7:43 ` Gerd Hoffmann -1 siblings, 0 replies; 30+ messages in thread From: Gerd Hoffmann @ 2022-06-08 7:43 UTC (permalink / raw) To: Javier Martinez Canillas Cc: tzimmermann, kvm, airlied, linux-kernel, Alex Williamson, dri-devel, Laszlo Ersek Hi, > But also, this issue isn't something that only affects graphic devices, > right? AFAIU from [1] and [2], the same issue happens if a PCI device > has to be bound to vfio-pci but already was bound to a host driver. Nope. There is a standard procedure to bind and unbind pci drivers via sysfs, using /sys/bus/pci/drivers/$name/{bind,unbind}. > The fact that DRM happens to have some infrastructure to remove devices > that conflict with an aperture is just a coincidence. No. It's a consequence of firmware framebuffers not being linked to the pci device actually backing them, so some other way is needed to find and solve conflicts. > The series [0] mentioned above, adds a sysfb_disable() that disables the > Generic System Framebuffer logic that is what registers the framebuffer > devices that are bound to these generic video drivers. On disable, the > devices registered by sysfb are also unregistered. As Alex already mentioned this might not have the desired effect on systems with multiple GPUs (I think even without considering vfio-pci). > That is, do you want to remove the {vesa,efi,simple}fb and simpledrm > drivers or is there a need to also remove real fbdev and DRM drivers? Boot framebuffers are the problem because they are neither visible nor manageable in /sys/bus/pci. For real fbdev/drm drivers the standard pci unbind can be used. take care, Gerd ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior @ 2022-06-08 7:43 ` Gerd Hoffmann 0 siblings, 0 replies; 30+ messages in thread From: Gerd Hoffmann @ 2022-06-08 7:43 UTC (permalink / raw) To: Javier Martinez Canillas Cc: Alex Williamson, maarten.lankhorst, mripard, tzimmermann, airlied, daniel, kvm, Laszlo Ersek, dri-devel, linux-kernel Hi, > But also, this issue isn't something that only affects graphic devices, > right? AFAIU from [1] and [2], the same issue happens if a PCI device > has to be bound to vfio-pci but already was bound to a host driver. Nope. There is a standard procedure to bind and unbind pci drivers via sysfs, using /sys/bus/pci/drivers/$name/{bind,unbind}. > The fact that DRM happens to have some infrastructure to remove devices > that conflict with an aperture is just a coincidence. No. It's a consequence of firmware framebuffers not being linked to the pci device actually backing them, so some other way is needed to find and solve conflicts. > The series [0] mentioned above, adds a sysfb_disable() that disables the > Generic System Framebuffer logic that is what registers the framebuffer > devices that are bound to these generic video drivers. On disable, the > devices registered by sysfb are also unregistered. As Alex already mentioned this might not have the desired effect on systems with multiple GPUs (I think even without considering vfio-pci). > That is, do you want to remove the {vesa,efi,simple}fb and simpledrm > drivers or is there a need to also remove real fbdev and DRM drivers? Boot framebuffers are the problem because they are neither visible nor manageable in /sys/bus/pci. For real fbdev/drm drivers the standard pci unbind can be used. take care, Gerd ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior 2022-06-08 7:43 ` Gerd Hoffmann @ 2022-06-08 8:51 ` Javier Martinez Canillas -1 siblings, 0 replies; 30+ messages in thread From: Javier Martinez Canillas @ 2022-06-08 8:51 UTC (permalink / raw) To: Gerd Hoffmann Cc: tzimmermann, kvm, airlied, linux-kernel, Alex Williamson, dri-devel, Laszlo Ersek Hello Gerd and Alex, On 6/8/22 09:43, Gerd Hoffmann wrote: > Hi, > >> But also, this issue isn't something that only affects graphic devices, >> right? AFAIU from [1] and [2], the same issue happens if a PCI device >> has to be bound to vfio-pci but already was bound to a host driver. > > Nope. There is a standard procedure to bind and unbind pci drivers via > sysfs, using /sys/bus/pci/drivers/$name/{bind,unbind}. > Yes, but the cover letter says: "Users often employ kernel command line arguments to disable conflicting drivers or perform unbinding in userspace to avoid this" So I misunderstood that the goal was to avoid the need to do this via sysfs in user-space. I understand now that the problem is that for real PCI devices bound to a driver, you know the PCI device ID and bus so that you can use it, but with platform devices bound to drivers that just use a firmware-provided framebuffers you don't have that information to unbound. Because you could use the standard sysfs bind/unbind interface for this too, but don't have a way to know if the "simple-framebuffer" or "efi-framebuffer" is associated with a PCI device that you want to pass through or another one. The only information that could tell you that is the I/O memory resource that is associated with the platform device registered and that's why you want to use the drm_aperture_remove_conflicting_pci_framebuffers() helper. >> The fact that DRM happens to have some infrastructure to remove devices >> that conflict with an aperture is just a coincidence. > > No. It's a consequence of firmware framebuffers not being linked to the > pci device actually backing them, so some other way is needed to find > and solve conflicts. > Right, it's clear to me now. As mentioned I misunderstood your problem. >> The series [0] mentioned above, adds a sysfb_disable() that disables the >> Generic System Framebuffer logic that is what registers the framebuffer >> devices that are bound to these generic video drivers. On disable, the >> devices registered by sysfb are also unregistered. > > As Alex already mentioned this might not have the desired effect on > systems with multiple GPUs (I think even without considering vfio-pci). > That's correct, although the firmware framebuffer drivers are just a best effort to allow having some display output even if there's no real video driver (or if the user prevented them to load with "nomodeset"). We have talked about improving this, by unifying fbdev and DRM apertures in a single list that could track all the devices registered and their requested aperture so that all subsystems could use it. The reason why I was pushing back on using the DRM aperture helper is that it would make more complicated later to do this refactoring as more subsystems use the current API. But as Alex said, it wouldn't make the problem worse so I'm OK with this if others agree that's the correct thing to do. >> That is, do you want to remove the {vesa,efi,simple}fb and simpledrm >> drivers or is there a need to also remove real fbdev and DRM drivers? > > Boot framebuffers are the problem because they are neither visible nor > manageable in /sys/bus/pci. For real fbdev/drm drivers the standard pci > unbind can be used. > Yes. Honestly I believe all this should be handled by the Linux device model. That is, drivers could just do pci_request_region() / request_mem_region() and drivers that want to unbind another bound device could do something like pci_request_region_force() / request_mem_region_force() to kick them out. -- Best regards, Javier Martinez Canillas Linux Engineering Red Hat ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior @ 2022-06-08 8:51 ` Javier Martinez Canillas 0 siblings, 0 replies; 30+ messages in thread From: Javier Martinez Canillas @ 2022-06-08 8:51 UTC (permalink / raw) To: Gerd Hoffmann Cc: Alex Williamson, maarten.lankhorst, mripard, tzimmermann, airlied, daniel, kvm, Laszlo Ersek, dri-devel, linux-kernel Hello Gerd and Alex, On 6/8/22 09:43, Gerd Hoffmann wrote: > Hi, > >> But also, this issue isn't something that only affects graphic devices, >> right? AFAIU from [1] and [2], the same issue happens if a PCI device >> has to be bound to vfio-pci but already was bound to a host driver. > > Nope. There is a standard procedure to bind and unbind pci drivers via > sysfs, using /sys/bus/pci/drivers/$name/{bind,unbind}. > Yes, but the cover letter says: "Users often employ kernel command line arguments to disable conflicting drivers or perform unbinding in userspace to avoid this" So I misunderstood that the goal was to avoid the need to do this via sysfs in user-space. I understand now that the problem is that for real PCI devices bound to a driver, you know the PCI device ID and bus so that you can use it, but with platform devices bound to drivers that just use a firmware-provided framebuffers you don't have that information to unbound. Because you could use the standard sysfs bind/unbind interface for this too, but don't have a way to know if the "simple-framebuffer" or "efi-framebuffer" is associated with a PCI device that you want to pass through or another one. The only information that could tell you that is the I/O memory resource that is associated with the platform device registered and that's why you want to use the drm_aperture_remove_conflicting_pci_framebuffers() helper. >> The fact that DRM happens to have some infrastructure to remove devices >> that conflict with an aperture is just a coincidence. > > No. It's a consequence of firmware framebuffers not being linked to the > pci device actually backing them, so some other way is needed to find > and solve conflicts. > Right, it's clear to me now. As mentioned I misunderstood your problem. >> The series [0] mentioned above, adds a sysfb_disable() that disables the >> Generic System Framebuffer logic that is what registers the framebuffer >> devices that are bound to these generic video drivers. On disable, the >> devices registered by sysfb are also unregistered. > > As Alex already mentioned this might not have the desired effect on > systems with multiple GPUs (I think even without considering vfio-pci). > That's correct, although the firmware framebuffer drivers are just a best effort to allow having some display output even if there's no real video driver (or if the user prevented them to load with "nomodeset"). We have talked about improving this, by unifying fbdev and DRM apertures in a single list that could track all the devices registered and their requested aperture so that all subsystems could use it. The reason why I was pushing back on using the DRM aperture helper is that it would make more complicated later to do this refactoring as more subsystems use the current API. But as Alex said, it wouldn't make the problem worse so I'm OK with this if others agree that's the correct thing to do. >> That is, do you want to remove the {vesa,efi,simple}fb and simpledrm >> drivers or is there a need to also remove real fbdev and DRM drivers? > > Boot framebuffers are the problem because they are neither visible nor > manageable in /sys/bus/pci. For real fbdev/drm drivers the standard pci > unbind can be used. > Yes. Honestly I believe all this should be handled by the Linux device model. That is, drivers could just do pci_request_region() / request_mem_region() and drivers that want to unbind another bound device could do something like pci_request_region_force() / request_mem_region_force() to kick them out. -- Best regards, Javier Martinez Canillas Linux Engineering Red Hat ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior 2022-06-08 8:51 ` Javier Martinez Canillas @ 2022-06-08 9:11 ` Gerd Hoffmann -1 siblings, 0 replies; 30+ messages in thread From: Gerd Hoffmann @ 2022-06-08 9:11 UTC (permalink / raw) To: Javier Martinez Canillas Cc: tzimmermann, kvm, airlied, linux-kernel, Alex Williamson, dri-devel, Laszlo Ersek Hi, > >> But also, this issue isn't something that only affects graphic devices, > >> right? AFAIU from [1] and [2], the same issue happens if a PCI device > >> has to be bound to vfio-pci but already was bound to a host driver. > > > > Nope. There is a standard procedure to bind and unbind pci drivers via > > sysfs, using /sys/bus/pci/drivers/$name/{bind,unbind}. > > > > Yes, but the cover letter says: > > "Users often employ kernel command line arguments to disable conflicting > drivers or perform unbinding in userspace to avoid this" Thats helpful at times to deal with driver and/or hardware quirks. Example: Years ago drm drivers used to be horrible when it came to unbind, leaving oopses and panics left & right when you tried (luckily it works much better these days). [ leaving this here for completeness, snipping the remaining reply, noting that we are on the same page now ] thanks & take care, Gerd ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior @ 2022-06-08 9:11 ` Gerd Hoffmann 0 siblings, 0 replies; 30+ messages in thread From: Gerd Hoffmann @ 2022-06-08 9:11 UTC (permalink / raw) To: Javier Martinez Canillas Cc: Alex Williamson, maarten.lankhorst, mripard, tzimmermann, airlied, daniel, kvm, Laszlo Ersek, dri-devel, linux-kernel Hi, > >> But also, this issue isn't something that only affects graphic devices, > >> right? AFAIU from [1] and [2], the same issue happens if a PCI device > >> has to be bound to vfio-pci but already was bound to a host driver. > > > > Nope. There is a standard procedure to bind and unbind pci drivers via > > sysfs, using /sys/bus/pci/drivers/$name/{bind,unbind}. > > > > Yes, but the cover letter says: > > "Users often employ kernel command line arguments to disable conflicting > drivers or perform unbinding in userspace to avoid this" Thats helpful at times to deal with driver and/or hardware quirks. Example: Years ago drm drivers used to be horrible when it came to unbind, leaving oopses and panics left & right when you tried (luckily it works much better these days). [ leaving this here for completeness, snipping the remaining reply, noting that we are on the same page now ] thanks & take care, Gerd ^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2022-06-10 14:30 UTC | newest] Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-06-06 17:53 [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior Alex Williamson 2022-06-06 17:53 ` Alex Williamson 2022-06-06 17:53 ` [PATCH 1/2] drm/aperture: Split conflicting platform driver removal Alex Williamson 2022-06-06 17:53 ` Alex Williamson 2022-06-06 17:53 ` [PATCH 2/2] vfio/pci: Remove console drivers Alex Williamson 2022-06-06 17:53 ` Alex Williamson 2022-06-08 11:11 ` Thomas Zimmermann 2022-06-08 11:11 ` Thomas Zimmermann 2022-06-08 14:04 ` Alex Williamson 2022-06-08 14:04 ` Alex Williamson 2022-06-09 9:13 ` Thomas Zimmermann 2022-06-09 9:13 ` Thomas Zimmermann 2022-06-09 21:41 ` Alex Williamson 2022-06-09 21:41 ` Alex Williamson 2022-06-09 21:44 ` Alex Williamson 2022-06-09 21:44 ` Alex Williamson 2022-06-10 7:03 ` Thomas Zimmermann 2022-06-10 14:30 ` Alex Williamson 2022-06-08 15:37 ` Gerd Hoffmann 2022-06-08 15:37 ` Gerd Hoffmann 2022-06-07 17:40 ` [PATCH 0/2] Improve vfio-pci primary GPU assignment behavior Javier Martinez Canillas 2022-06-07 17:40 ` Javier Martinez Canillas 2022-06-07 21:01 ` Alex Williamson 2022-06-07 21:01 ` Alex Williamson 2022-06-08 7:43 ` Gerd Hoffmann 2022-06-08 7:43 ` Gerd Hoffmann 2022-06-08 8:51 ` Javier Martinez Canillas 2022-06-08 8:51 ` Javier Martinez Canillas 2022-06-08 9:11 ` Gerd Hoffmann 2022-06-08 9:11 ` Gerd Hoffmann
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.