[PATCH 0/7] drm/radeon: Convert to generic fbdev emulation

[PATCH 0/7] drm/radeon: Convert to generic fbdev emulation

[Thomas Zimmermann]

This patchset replaces radeon's framebuffer emulation with the generic
code from fb helpers. This reduces the driver size and complexity. As
a nice side effect, it also reduces memory pressure on low-memory devices.

Patches #1 and #2 add features to fbdev emulation to bring it on par with
radeons code.

Patch #4 adds internal pinning to vmap operations. This guarantess that
the framebuffer BO is not relocated while being updated.

Patch #5 and #6 replace radeon's framebuffer code with the generic helper
with shadow buffering. All userspace mappings interact with the shadow
buffer and the BO can be relocated an necessary. This allows to enable
32-bit colors on low-memory devices.

Patches #3 and #7 are merely cleanups.

Smoke tested with an AMD R5 (Oland) card by running the console and
switching to/from Xorg and weston.

Thomas Zimmermann (7):
  drm/fb-helper: Set framebuffer for vga-switcheroo clients
  drm/fb-helper: Add hint to enable VT switching during suspend/resume
  drm/radeon: Whitespace fixes
  drm/radeon: Pin buffers while they are vmap'ed
  drm/radeon: Replace framebuffer console with generic implementation
  drm/radeon: Use fbdev shadow fb
  drm/radeon: Move radeon_align_pitch() next to its only caller

 drivers/gpu/drm/drm_fb_helper.c         |  16 +-
 drivers/gpu/drm/radeon/radeon_device.c  |  11 +-
 drivers/gpu/drm/radeon/radeon_display.c |   2 -
 drivers/gpu/drm/radeon/radeon_drv.c     |   5 +-
 drivers/gpu/drm/radeon/radeon_fb.c      | 366 +-----------------------
 drivers/gpu/drm/radeon/radeon_gem.c     |  77 ++++-
 drivers/gpu/drm/radeon/radeon_kms.c     |  14 +-
 drivers/gpu/drm/radeon/radeon_mode.h    |  10 +-
 include/drm/drm_mode_config.h           |   7 +
 9 files changed, 111 insertions(+), 397 deletions(-)

--
2.29.2

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 0/7] drm/radeon: Convert to generic fbdev emulation

[Thomas Zimmermann]

This patchset replaces radeon's framebuffer emulation with the generic
code from fb helpers. This reduces the driver size and complexity. As
a nice side effect, it also reduces memory pressure on low-memory devices.

Patches #1 and #2 add features to fbdev emulation to bring it on par with
radeons code.

Patch #4 adds internal pinning to vmap operations. This guarantess that
the framebuffer BO is not relocated while being updated.

Patch #5 and #6 replace radeon's framebuffer code with the generic helper
with shadow buffering. All userspace mappings interact with the shadow
buffer and the BO can be relocated an necessary. This allows to enable
32-bit colors on low-memory devices.

Patches #3 and #7 are merely cleanups.

Smoke tested with an AMD R5 (Oland) card by running the console and
switching to/from Xorg and weston.

Thomas Zimmermann (7):
  drm/fb-helper: Set framebuffer for vga-switcheroo clients
  drm/fb-helper: Add hint to enable VT switching during suspend/resume
  drm/radeon: Whitespace fixes
  drm/radeon: Pin buffers while they are vmap'ed
  drm/radeon: Replace framebuffer console with generic implementation
  drm/radeon: Use fbdev shadow fb
  drm/radeon: Move radeon_align_pitch() next to its only caller

 drivers/gpu/drm/drm_fb_helper.c         |  16 +-
 drivers/gpu/drm/radeon/radeon_device.c  |  11 +-
 drivers/gpu/drm/radeon/radeon_display.c |   2 -
 drivers/gpu/drm/radeon/radeon_drv.c     |   5 +-
 drivers/gpu/drm/radeon/radeon_fb.c      | 366 +-----------------------
 drivers/gpu/drm/radeon/radeon_gem.c     |  77 ++++-
 drivers/gpu/drm/radeon/radeon_kms.c     |  14 +-
 drivers/gpu/drm/radeon/radeon_mode.h    |  10 +-
 include/drm/drm_mode_config.h           |   7 +
 9 files changed, 111 insertions(+), 397 deletions(-)

--
2.29.2

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 1/7] drm/fb-helper: Set framebuffer for vga-switcheroo clients

[Thomas Zimmermann]

Set the framebuffer info for devices that support vga switcheroo. For
other devices, this does nothing.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/drm_fb_helper.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index 01ba1da28511..d2ba404a23b3 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -33,8 +33,10 @@
 #include <linux/dma-buf.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/pci.h>
 #include <linux/slab.h>
 #include <linux/sysrq.h>
+#include <linux/vga_switcheroo.h>
 #include <linux/vmalloc.h>
 
 #include <drm/drm_atomic.h>
@@ -1940,6 +1942,8 @@ EXPORT_SYMBOL(drm_fb_helper_hotplug_event);
 void drm_fb_helper_lastclose(struct drm_device *dev)
 {
 	drm_fb_helper_restore_fbdev_mode_unlocked(dev->fb_helper);
+
+	vga_switcheroo_process_delayed_switch();
 }
 EXPORT_SYMBOL(drm_fb_helper_lastclose);
 
@@ -2311,6 +2315,10 @@ static int drm_fb_helper_generic_probe(struct drm_fb_helper *fb_helper,
 #endif
 	}
 
+	/* Set the fb info for vgaswitcheroo clients. Does nothing otherwise. */
+	if (dev_is_pci(dev->dev))
+		vga_switcheroo_client_fb_set(to_pci_dev(dev->dev), fbi);
+
 	return 0;
 }
 
-- 
2.29.2

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 1/7] drm/fb-helper: Set framebuffer for vga-switcheroo clients

[Thomas Zimmermann]

Set the framebuffer info for devices that support vga switcheroo. For
other devices, this does nothing.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/drm_fb_helper.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index 01ba1da28511..d2ba404a23b3 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -33,8 +33,10 @@
 #include <linux/dma-buf.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/pci.h>
 #include <linux/slab.h>
 #include <linux/sysrq.h>
+#include <linux/vga_switcheroo.h>
 #include <linux/vmalloc.h>
 
 #include <drm/drm_atomic.h>
@@ -1940,6 +1942,8 @@ EXPORT_SYMBOL(drm_fb_helper_hotplug_event);
 void drm_fb_helper_lastclose(struct drm_device *dev)
 {
 	drm_fb_helper_restore_fbdev_mode_unlocked(dev->fb_helper);
+
+	vga_switcheroo_process_delayed_switch();
 }
 EXPORT_SYMBOL(drm_fb_helper_lastclose);
 
@@ -2311,6 +2315,10 @@ static int drm_fb_helper_generic_probe(struct drm_fb_helper *fb_helper,
 #endif
 	}
 
+	/* Set the fb info for vgaswitcheroo clients. Does nothing otherwise. */
+	if (dev_is_pci(dev->dev))
+		vga_switcheroo_client_fb_set(to_pci_dev(dev->dev), fbi);
+
 	return 0;
 }
 
-- 
2.29.2

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 2/7] drm/fb-helper: Add hint to enable VT switching during suspend/resume

[Thomas Zimmermann]

Switching VTs during suspend/resume is required to reliably run radeon
with generic fbdev emulation.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/drm_fb_helper.c | 8 +++++---
 include/drm/drm_mode_config.h   | 7 +++++++
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index d2ba404a23b3..987cd781e62c 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -508,11 +508,11 @@ EXPORT_SYMBOL(drm_fb_helper_init);
  */
 struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper)
 {
-	struct device *dev = fb_helper->dev->dev;
+	struct drm_device *dev = fb_helper->dev;
 	struct fb_info *info;
 	int ret;
 
-	info = framebuffer_alloc(0, dev);
+	info = framebuffer_alloc(0, dev->dev);
 	if (!info)
 		return ERR_PTR(-ENOMEM);
 
@@ -535,7 +535,9 @@ struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper)
 	}
 
 	fb_helper->fbdev = info;
-	info->skip_vt_switch = true;
+
+	if (!dev->mode_config.require_vt_switch_fbdev)
+		info->skip_vt_switch = true;
 
 	return info;
 
diff --git a/include/drm/drm_mode_config.h b/include/drm/drm_mode_config.h
index ab424ddd7665..59e0796c6928 100644
--- a/include/drm/drm_mode_config.h
+++ b/include/drm/drm_mode_config.h
@@ -877,6 +877,13 @@ struct drm_mode_config {
 	 */
 	bool prefer_shadow_fbdev;
 
+	/**
+	 * @require_vt_switch_fbdev:
+	 *
+	 * Hint to framebuffer emulation to enable VT switching on suspend/resume.
+	 */
+	bool require_vt_switch_fbdev;
+
 	/**
 	 * @quirk_addfb_prefer_xbgr_30bpp:
 	 *
-- 
2.29.2

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 2/7] drm/fb-helper: Add hint to enable VT switching during suspend/resume

[Thomas Zimmermann]

Switching VTs during suspend/resume is required to reliably run radeon
with generic fbdev emulation.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/drm_fb_helper.c | 8 +++++---
 include/drm/drm_mode_config.h   | 7 +++++++
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index d2ba404a23b3..987cd781e62c 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -508,11 +508,11 @@ EXPORT_SYMBOL(drm_fb_helper_init);
  */
 struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper)
 {
-	struct device *dev = fb_helper->dev->dev;
+	struct drm_device *dev = fb_helper->dev;
 	struct fb_info *info;
 	int ret;
 
-	info = framebuffer_alloc(0, dev);
+	info = framebuffer_alloc(0, dev->dev);
 	if (!info)
 		return ERR_PTR(-ENOMEM);
 
@@ -535,7 +535,9 @@ struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper)
 	}
 
 	fb_helper->fbdev = info;
-	info->skip_vt_switch = true;
+
+	if (!dev->mode_config.require_vt_switch_fbdev)
+		info->skip_vt_switch = true;
 
 	return info;
 
diff --git a/include/drm/drm_mode_config.h b/include/drm/drm_mode_config.h
index ab424ddd7665..59e0796c6928 100644
--- a/include/drm/drm_mode_config.h
+++ b/include/drm/drm_mode_config.h
@@ -877,6 +877,13 @@ struct drm_mode_config {
 	 */
 	bool prefer_shadow_fbdev;
 
+	/**
+	 * @require_vt_switch_fbdev:
+	 *
+	 * Hint to framebuffer emulation to enable VT switching on suspend/resume.
+	 */
+	bool require_vt_switch_fbdev;
+
 	/**
 	 * @quirk_addfb_prefer_xbgr_30bpp:
 	 *
-- 
2.29.2

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 3/7] drm/radeon: Whitespace fixes

[Thomas Zimmermann]

Fixed to match kernel coding style.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/radeon/radeon_kms.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/radeon/radeon_kms.c b/drivers/gpu/drm/radeon/radeon_kms.c
index abb3bdd9ca25..75b038740ea8 100644
--- a/drivers/gpu/drm/radeon/radeon_kms.c
+++ b/drivers/gpu/drm/radeon/radeon_kms.c
@@ -74,7 +74,7 @@ void radeon_driver_unload_kms(struct drm_device *dev)
 	}
 
 	radeon_acpi_fini(rdev);
-	
+
 	radeon_modeset_fini(rdev);
 	radeon_device_fini(rdev);
 
-- 
2.29.2

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 3/7] drm/radeon: Whitespace fixes

[Thomas Zimmermann]

Fixed to match kernel coding style.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/radeon/radeon_kms.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/radeon/radeon_kms.c b/drivers/gpu/drm/radeon/radeon_kms.c
index abb3bdd9ca25..75b038740ea8 100644
--- a/drivers/gpu/drm/radeon/radeon_kms.c
+++ b/drivers/gpu/drm/radeon/radeon_kms.c
@@ -74,7 +74,7 @@ void radeon_driver_unload_kms(struct drm_device *dev)
 	}
 
 	radeon_acpi_fini(rdev);
-	
+
 	radeon_modeset_fini(rdev);
 	radeon_device_fini(rdev);
 
-- 
2.29.2

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

In order to avoid eviction of vmap'ed buffers, pin them in their GEM
object's vmap implementation. Unpin them in the vunmap implementation.
This is needed to make generic fbdev support work reliably. Without,
the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not
modify the BOs prime_shared_count, so a vmap-pinned BO does not count as
shared.

The actual pin location is not important as the vmap call returns
information on how to access the buffer. Callers that require a
specific location should explicitly pin the BO before vmapping it.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
 1 file changed, 49 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
index d2876ce3bc9e..eaf7fc9a7b07 100644
--- a/drivers/gpu/drm/radeon/radeon_gem.c
+++ b/drivers/gpu/drm/radeon/radeon_gem.c
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct radeon_device *rdev, int r)
 	return r;
 }
 
+static int radeon_gem_object_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+	static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+					   RADEON_GEM_DOMAIN_GTT |
+					   RADEON_GEM_DOMAIN_CPU;
+
+	struct radeon_bo *bo = gem_to_radeon_bo(obj);
+	int ret;
+
+	ret = radeon_bo_reserve(bo, false);
+	if (ret)
+		return ret;
+
+	/* pin buffer at its current location */
+	ret = radeon_bo_pin(bo, any_domain, NULL);
+	if (ret)
+		goto err_radeon_bo_unreserve;
+
+	ret = drm_gem_ttm_vmap(obj, map);
+	if (ret)
+		goto err_radeon_bo_unpin;
+
+	radeon_bo_unreserve(bo);
+
+	return 0;
+
+err_radeon_bo_unpin:
+	radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+	radeon_bo_unreserve(bo);
+	return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+	struct radeon_bo *bo = gem_to_radeon_bo(obj);
+	int ret;
+
+	ret = radeon_bo_reserve(bo, false);
+	if (ret)
+		return;
+
+	drm_gem_ttm_vunmap(obj, map);
+	radeon_bo_unpin(bo);
+	radeon_bo_unreserve(bo);
+}
+
 static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
 	.free = radeon_gem_object_free,
 	.open = radeon_gem_object_open,
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
 	.pin = radeon_gem_prime_pin,
 	.unpin = radeon_gem_prime_unpin,
 	.get_sg_table = radeon_gem_prime_get_sg_table,
-	.vmap = drm_gem_ttm_vmap,
-	.vunmap = drm_gem_ttm_vunmap,
+	.vmap = radeon_gem_object_vmap,
+	.vunmap = radeon_gem_object_vunmap,
 };
 
 /*
-- 
2.29.2

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

In order to avoid eviction of vmap'ed buffers, pin them in their GEM
object's vmap implementation. Unpin them in the vunmap implementation.
This is needed to make generic fbdev support work reliably. Without,
the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not
modify the BOs prime_shared_count, so a vmap-pinned BO does not count as
shared.

The actual pin location is not important as the vmap call returns
information on how to access the buffer. Callers that require a
specific location should explicitly pin the BO before vmapping it.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
 1 file changed, 49 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
index d2876ce3bc9e..eaf7fc9a7b07 100644
--- a/drivers/gpu/drm/radeon/radeon_gem.c
+++ b/drivers/gpu/drm/radeon/radeon_gem.c
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct radeon_device *rdev, int r)
 	return r;
 }
 
+static int radeon_gem_object_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+	static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+					   RADEON_GEM_DOMAIN_GTT |
+					   RADEON_GEM_DOMAIN_CPU;
+
+	struct radeon_bo *bo = gem_to_radeon_bo(obj);
+	int ret;
+
+	ret = radeon_bo_reserve(bo, false);
+	if (ret)
+		return ret;
+
+	/* pin buffer at its current location */
+	ret = radeon_bo_pin(bo, any_domain, NULL);
+	if (ret)
+		goto err_radeon_bo_unreserve;
+
+	ret = drm_gem_ttm_vmap(obj, map);
+	if (ret)
+		goto err_radeon_bo_unpin;
+
+	radeon_bo_unreserve(bo);
+
+	return 0;
+
+err_radeon_bo_unpin:
+	radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+	radeon_bo_unreserve(bo);
+	return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+	struct radeon_bo *bo = gem_to_radeon_bo(obj);
+	int ret;
+
+	ret = radeon_bo_reserve(bo, false);
+	if (ret)
+		return;
+
+	drm_gem_ttm_vunmap(obj, map);
+	radeon_bo_unpin(bo);
+	radeon_bo_unreserve(bo);
+}
+
 static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
 	.free = radeon_gem_object_free,
 	.open = radeon_gem_object_open,
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
 	.pin = radeon_gem_prime_pin,
 	.unpin = radeon_gem_prime_unpin,
 	.get_sg_table = radeon_gem_prime_get_sg_table,
-	.vmap = drm_gem_ttm_vmap,
-	.vunmap = drm_gem_ttm_vunmap,
+	.vmap = radeon_gem_object_vmap,
+	.vunmap = radeon_gem_object_vunmap,
 };
 
 /*
-- 
2.29.2

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 5/7] drm/radeon: Replace framebuffer console with generic implementation

[Thomas Zimmermann]

The fbdev code in radeon can be replaced by the generic fbdev helpers.
This allows for using fbdev shadow buffers and IGT testcases.

Generic fbdev acts like a regular DRM client and has to be initialized
after registering the DRM device. Change the driver initialization
accordingly. The release happens automatically during shutdown.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/radeon/radeon_display.c |   2 -
 drivers/gpu/drm/radeon/radeon_drv.c     |   5 +-
 drivers/gpu/drm/radeon/radeon_fb.c      | 337 +-----------------------
 drivers/gpu/drm/radeon/radeon_kms.c     |  12 -
 drivers/gpu/drm/radeon/radeon_mode.h    |   7 +-
 5 files changed, 16 insertions(+), 347 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_display.c b/drivers/gpu/drm/radeon/radeon_display.c
index b79686cf8bdb..a7bae33636c3 100644
--- a/drivers/gpu/drm/radeon/radeon_display.c
+++ b/drivers/gpu/drm/radeon/radeon_display.c
@@ -1638,7 +1638,6 @@ int radeon_modeset_init(struct radeon_device *rdev)
 	/* setup afmt */
 	radeon_afmt_init(rdev);
 
-	radeon_fbdev_init(rdev);
 	drm_kms_helper_poll_init(rdev->ddev);
 
 	/* do pm late init */
@@ -1653,7 +1652,6 @@ void radeon_modeset_fini(struct radeon_device *rdev)
 		drm_kms_helper_poll_fini(rdev->ddev);
 		radeon_hpd_fini(rdev);
 		drm_helper_force_disable_all(rdev->ddev);
-		radeon_fbdev_fini(rdev);
 		radeon_afmt_fini(rdev);
 		drm_mode_config_cleanup(rdev->ddev);
 		rdev->mode_info.mode_config_initialized = false;
diff --git a/drivers/gpu/drm/radeon/radeon_drv.c b/drivers/gpu/drm/radeon/radeon_drv.c
index bfadb799d6a3..63c3bf1a3a3c 100644
--- a/drivers/gpu/drm/radeon/radeon_drv.c
+++ b/drivers/gpu/drm/radeon/radeon_drv.c
@@ -114,7 +114,6 @@
 #define KMS_DRIVER_PATCHLEVEL	0
 int radeon_driver_load_kms(struct drm_device *dev, unsigned long flags);
 void radeon_driver_unload_kms(struct drm_device *dev);
-void radeon_driver_lastclose_kms(struct drm_device *dev);
 int radeon_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv);
 void radeon_driver_postclose_kms(struct drm_device *dev,
 				 struct drm_file *file_priv);
@@ -374,6 +373,8 @@ static int radeon_pci_probe(struct pci_dev *pdev,
 	if (ret)
 		goto err_agp;
 
+	radeon_fbdev_init(dev->dev_private);
+
 	return 0;
 
 err_agp:
@@ -635,7 +636,7 @@ static const struct drm_driver kms_driver = {
 	.load = radeon_driver_load_kms,
 	.open = radeon_driver_open_kms,
 	.postclose = radeon_driver_postclose_kms,
-	.lastclose = radeon_driver_lastclose_kms,
+	.lastclose = drm_fb_helper_lastclose,
 	.unload = radeon_driver_unload_kms,
 	.irq_preinstall = radeon_driver_irq_preinstall_kms,
 	.irq_postinstall = radeon_driver_irq_postinstall_kms,
diff --git a/drivers/gpu/drm/radeon/radeon_fb.c b/drivers/gpu/drm/radeon/radeon_fb.c
index fc4212633bdf..50fdc2aaa463 100644
--- a/drivers/gpu/drm/radeon/radeon_fb.c
+++ b/drivers/gpu/drm/radeon/radeon_fb.c
@@ -24,66 +24,12 @@
  *     David Airlie
  */
 
-#include <linux/module.h>
 #include <linux/pci.h>
-#include <linux/pm_runtime.h>
-#include <linux/slab.h>
-#include <linux/vga_switcheroo.h>
 
-#include <drm/drm_crtc.h>
-#include <drm/drm_crtc_helper.h>
 #include <drm/drm_fb_helper.h>
-#include <drm/drm_fourcc.h>
-#include <drm/radeon_drm.h>
 
 #include "radeon.h"
 
-/* object hierarchy -
- * this contains a helper + a radeon fb
- * the helper contains a pointer to radeon framebuffer baseclass.
- */
-struct radeon_fbdev {
-	struct drm_fb_helper helper; /* must be first */
-	struct drm_framebuffer fb;
-	struct radeon_device *rdev;
-};
-
-static int
-radeonfb_open(struct fb_info *info, int user)
-{
-	struct radeon_fbdev *rfbdev = info->par;
-	struct radeon_device *rdev = rfbdev->rdev;
-	int ret = pm_runtime_get_sync(rdev->ddev->dev);
-	if (ret < 0 && ret != -EACCES) {
-		pm_runtime_mark_last_busy(rdev->ddev->dev);
-		pm_runtime_put_autosuspend(rdev->ddev->dev);
-		return ret;
-	}
-	return 0;
-}
-
-static int
-radeonfb_release(struct fb_info *info, int user)
-{
-	struct radeon_fbdev *rfbdev = info->par;
-	struct radeon_device *rdev = rfbdev->rdev;
-
-	pm_runtime_mark_last_busy(rdev->ddev->dev);
-	pm_runtime_put_autosuspend(rdev->ddev->dev);
-	return 0;
-}
-
-static const struct fb_ops radeonfb_ops = {
-	.owner = THIS_MODULE,
-	DRM_FB_HELPER_DEFAULT_OPS,
-	.fb_open = radeonfb_open,
-	.fb_release = radeonfb_release,
-	.fb_fillrect = drm_fb_helper_cfb_fillrect,
-	.fb_copyarea = drm_fb_helper_cfb_copyarea,
-	.fb_imageblit = drm_fb_helper_cfb_imageblit,
-};
-
-
 int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tiled)
 {
 	int aligned = width;
@@ -108,294 +54,35 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 	return aligned * cpp;
 }
 
-static void radeonfb_destroy_pinned_object(struct drm_gem_object *gobj)
-{
-	struct radeon_bo *rbo = gem_to_radeon_bo(gobj);
-	int ret;
-
-	ret = radeon_bo_reserve(rbo, false);
-	if (likely(ret == 0)) {
-		radeon_bo_kunmap(rbo);
-		radeon_bo_unpin(rbo);
-		radeon_bo_unreserve(rbo);
-	}
-	drm_gem_object_put(gobj);
-}
-
-static int radeonfb_create_pinned_object(struct radeon_fbdev *rfbdev,
-					 struct drm_mode_fb_cmd2 *mode_cmd,
-					 struct drm_gem_object **gobj_p)
-{
-	const struct drm_format_info *info;
-	struct radeon_device *rdev = rfbdev->rdev;
-	struct drm_gem_object *gobj = NULL;
-	struct radeon_bo *rbo = NULL;
-	bool fb_tiled = false; /* useful for testing */
-	u32 tiling_flags = 0;
-	int ret;
-	int aligned_size, size;
-	int height = mode_cmd->height;
-	u32 cpp;
-
-	info = drm_get_format_info(rdev->ddev, mode_cmd);
-	cpp = info->cpp[0];
-
-	/* need to align pitch with crtc limits */
-	mode_cmd->pitches[0] = radeon_align_pitch(rdev, mode_cmd->width, cpp,
-						  fb_tiled);
-
-	if (rdev->family >= CHIP_R600)
-		height = ALIGN(mode_cmd->height, 8);
-	size = mode_cmd->pitches[0] * height;
-	aligned_size = ALIGN(size, PAGE_SIZE);
-	ret = radeon_gem_object_create(rdev, aligned_size, 0,
-				       RADEON_GEM_DOMAIN_VRAM,
-				       0, true, &gobj);
-	if (ret) {
-		pr_err("failed to allocate framebuffer (%d)\n", aligned_size);
-		return -ENOMEM;
-	}
-	rbo = gem_to_radeon_bo(gobj);
-
-	if (fb_tiled)
-		tiling_flags = RADEON_TILING_MACRO;
-
-#ifdef __BIG_ENDIAN
-	switch (cpp) {
-	case 4:
-		tiling_flags |= RADEON_TILING_SWAP_32BIT;
-		break;
-	case 2:
-		tiling_flags |= RADEON_TILING_SWAP_16BIT;
-	default:
-		break;
-	}
-#endif
-
-	if (tiling_flags) {
-		ret = radeon_bo_set_tiling_flags(rbo,
-						 tiling_flags | RADEON_TILING_SURFACE,
-						 mode_cmd->pitches[0]);
-		if (ret)
-			dev_err(rdev->dev, "FB failed to set tiling flags\n");
-	}
-
-
-	ret = radeon_bo_reserve(rbo, false);
-	if (unlikely(ret != 0))
-		goto out_unref;
-	/* Only 27 bit offset for legacy CRTC */
-	ret = radeon_bo_pin_restricted(rbo, RADEON_GEM_DOMAIN_VRAM,
-				       ASIC_IS_AVIVO(rdev) ? 0 : 1 << 27,
-				       NULL);
-	if (ret) {
-		radeon_bo_unreserve(rbo);
-		goto out_unref;
-	}
-	if (fb_tiled)
-		radeon_bo_check_tiling(rbo, 0, 0);
-	ret = radeon_bo_kmap(rbo, NULL);
-	radeon_bo_unreserve(rbo);
-	if (ret) {
-		goto out_unref;
-	}
-
-	*gobj_p = gobj;
-	return 0;
-out_unref:
-	radeonfb_destroy_pinned_object(gobj);
-	*gobj_p = NULL;
-	return ret;
-}
-
-static int radeonfb_create(struct drm_fb_helper *helper,
-			   struct drm_fb_helper_surface_size *sizes)
-{
-	struct radeon_fbdev *rfbdev =
-		container_of(helper, struct radeon_fbdev, helper);
-	struct radeon_device *rdev = rfbdev->rdev;
-	struct fb_info *info;
-	struct drm_framebuffer *fb = NULL;
-	struct drm_mode_fb_cmd2 mode_cmd;
-	struct drm_gem_object *gobj = NULL;
-	struct radeon_bo *rbo = NULL;
-	int ret;
-	unsigned long tmp;
-
-	mode_cmd.width = sizes->surface_width;
-	mode_cmd.height = sizes->surface_height;
-
-	/* avivo can't scanout real 24bpp */
-	if ((sizes->surface_bpp == 24) && ASIC_IS_AVIVO(rdev))
-		sizes->surface_bpp = 32;
-
-	mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp,
-							  sizes->surface_depth);
-
-	ret = radeonfb_create_pinned_object(rfbdev, &mode_cmd, &gobj);
-	if (ret) {
-		DRM_ERROR("failed to create fbcon object %d\n", ret);
-		return ret;
-	}
-
-	rbo = gem_to_radeon_bo(gobj);
-
-	/* okay we have an object now allocate the framebuffer */
-	info = drm_fb_helper_alloc_fbi(helper);
-	if (IS_ERR(info)) {
-		ret = PTR_ERR(info);
-		goto out;
-	}
-
-	/* radeon resume is fragile and needs a vt switch to help it along */
-	info->skip_vt_switch = false;
-
-	ret = radeon_framebuffer_init(rdev->ddev, &rfbdev->fb, &mode_cmd, gobj);
-	if (ret) {
-		DRM_ERROR("failed to initialize framebuffer %d\n", ret);
-		goto out;
-	}
-
-	fb = &rfbdev->fb;
-
-	/* setup helper */
-	rfbdev->helper.fb = fb;
-
-	memset_io(rbo->kptr, 0x0, radeon_bo_size(rbo));
-
-	info->fbops = &radeonfb_ops;
-
-	tmp = radeon_bo_gpu_offset(rbo) - rdev->mc.vram_start;
-	info->fix.smem_start = rdev->mc.aper_base + tmp;
-	info->fix.smem_len = radeon_bo_size(rbo);
-	info->screen_base = rbo->kptr;
-	info->screen_size = radeon_bo_size(rbo);
-
-	drm_fb_helper_fill_info(info, &rfbdev->helper, sizes);
-
-	/* setup aperture base/size for vesafb takeover */
-	info->apertures->ranges[0].base = rdev->ddev->mode_config.fb_base;
-	info->apertures->ranges[0].size = rdev->mc.aper_size;
-
-	/* Use default scratch pixmap (info->pixmap.flags = FB_PIXMAP_SYSTEM) */
-
-	if (info->screen_base == NULL) {
-		ret = -ENOSPC;
-		goto out;
-	}
-
-	DRM_INFO("fb mappable at 0x%lX\n",  info->fix.smem_start);
-	DRM_INFO("vram apper at 0x%lX\n",  (unsigned long)rdev->mc.aper_base);
-	DRM_INFO("size %lu\n", (unsigned long)radeon_bo_size(rbo));
-	DRM_INFO("fb depth is %d\n", fb->format->depth);
-	DRM_INFO("   pitch is %d\n", fb->pitches[0]);
-
-	vga_switcheroo_client_fb_set(rdev->ddev->pdev, info);
-	return 0;
-
-out:
-	if (rbo) {
-
-	}
-	if (fb && ret) {
-		drm_gem_object_put(gobj);
-		drm_framebuffer_unregister_private(fb);
-		drm_framebuffer_cleanup(fb);
-		kfree(fb);
-	}
-	return ret;
-}
-
-static int radeon_fbdev_destroy(struct drm_device *dev, struct radeon_fbdev *rfbdev)
-{
-	struct drm_framebuffer *fb = &rfbdev->fb;
-
-	drm_fb_helper_unregister_fbi(&rfbdev->helper);
-
-	if (fb->obj[0]) {
-		radeonfb_destroy_pinned_object(fb->obj[0]);
-		fb->obj[0] = NULL;
-		drm_framebuffer_unregister_private(fb);
-		drm_framebuffer_cleanup(fb);
-	}
-	drm_fb_helper_fini(&rfbdev->helper);
-
-	return 0;
-}
-
-static const struct drm_fb_helper_funcs radeon_fb_helper_funcs = {
-	.fb_probe = radeonfb_create,
-};
-
-int radeon_fbdev_init(struct radeon_device *rdev)
+void radeon_fbdev_init(struct radeon_device *rdev)
 {
-	struct radeon_fbdev *rfbdev;
+	struct drm_device *ddev = rdev->ddev;
 	int bpp_sel = 32;
-	int ret;
-
-	/* don't enable fbdev if no connectors */
-	if (list_empty(&rdev->ddev->mode_config.connector_list))
-		return 0;
 
 	/* select 8 bpp console on 8MB cards, or 16 bpp on RN50 or 32MB */
 	if (rdev->mc.real_vram_size <= (8*1024*1024))
 		bpp_sel = 8;
-	else if (ASIC_IS_RN50(rdev) ||
-		 rdev->mc.real_vram_size <= (32*1024*1024))
+	else if (ASIC_IS_RN50(rdev) || rdev->mc.real_vram_size <= (32*1024*1024))
 		bpp_sel = 16;
 
-	rfbdev = kzalloc(sizeof(struct radeon_fbdev), GFP_KERNEL);
-	if (!rfbdev)
-		return -ENOMEM;
-
-	rfbdev->rdev = rdev;
-	rdev->mode_info.rfbdev = rfbdev;
-
-	drm_fb_helper_prepare(rdev->ddev, &rfbdev->helper,
-			      &radeon_fb_helper_funcs);
-
-	ret = drm_fb_helper_init(rdev->ddev, &rfbdev->helper);
-	if (ret)
-		goto free;
-
-	/* disable all the possible outputs/crtcs before entering KMS mode */
-	drm_helper_disable_unused_functions(rdev->ddev);
-
-	ret = drm_fb_helper_initial_config(&rfbdev->helper, bpp_sel);
-	if (ret)
-		goto fini;
-
-	return 0;
-
-fini:
-	drm_fb_helper_fini(&rfbdev->helper);
-free:
-	kfree(rfbdev);
-	return ret;
-}
-
-void radeon_fbdev_fini(struct radeon_device *rdev)
-{
-	if (!rdev->mode_info.rfbdev)
-		return;
+	/* radeon resume is fragile and needs a vt switch to help it along */
+	ddev->mode_config.require_vt_switch_fbdev = true;
 
-	radeon_fbdev_destroy(rdev->ddev, rdev->mode_info.rfbdev);
-	kfree(rdev->mode_info.rfbdev);
-	rdev->mode_info.rfbdev = NULL;
+	drm_fbdev_generic_setup(ddev, bpp_sel);
 }
 
 void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state)
 {
-	if (rdev->mode_info.rfbdev)
-		drm_fb_helper_set_suspend(&rdev->mode_info.rfbdev->helper, state);
+	drm_fb_helper_set_suspend(rdev->ddev->fb_helper, state);
 }
 
 bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj)
 {
-	if (!rdev->mode_info.rfbdev)
+	if (!rdev->ddev->fb_helper)
+		return false;
+
+	if (gem_to_radeon_bo(rdev->ddev->fb_helper->buffer->gem) != robj)
 		return false;
 
-	if (robj == gem_to_radeon_bo(rdev->mode_info.rfbdev->fb.obj[0]))
-		return true;
-	return false;
+	return true;
 }
diff --git a/drivers/gpu/drm/radeon/radeon_kms.c b/drivers/gpu/drm/radeon/radeon_kms.c
index 75b038740ea8..76e18e589ccd 100644
--- a/drivers/gpu/drm/radeon/radeon_kms.c
+++ b/drivers/gpu/drm/radeon/radeon_kms.c
@@ -608,18 +608,6 @@ int radeon_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
 /*
  * Outdated mess for old drm with Xorg being in charge (void function now).
  */
-/**
- * radeon_driver_lastclose_kms - drm callback for last close
- *
- * @dev: drm dev pointer
- *
- * Switch vga_switcheroo state after last close (all asics).
- */
-void radeon_driver_lastclose_kms(struct drm_device *dev)
-{
-	drm_fb_helper_lastclose(dev);
-	vga_switcheroo_process_delayed_switch();
-}
 
 /**
  * radeon_driver_open_kms - drm callback for open
diff --git a/drivers/gpu/drm/radeon/radeon_mode.h b/drivers/gpu/drm/radeon/radeon_mode.h
index c7f223743d46..fe192ef0aafa 100644
--- a/drivers/gpu/drm/radeon/radeon_mode.h
+++ b/drivers/gpu/drm/radeon/radeon_mode.h
@@ -230,8 +230,6 @@ enum radeon_dvo_chip {
 	DVO_SIL1178,
 };
 
-struct radeon_fbdev;
-
 struct radeon_afmt {
 	bool enabled;
 	int offset;
@@ -268,8 +266,6 @@ struct radeon_mode_info {
 	struct edid *bios_hardcoded_edid;
 	int bios_hardcoded_edid_size;
 
-	/* pointer to fbdev info structure */
-	struct radeon_fbdev *rfbdev;
 	/* firmware flags */
 	u16 firmware_flags;
 	/* pointer to backlight encoder */
@@ -979,8 +975,7 @@ void dce4_program_fmt(struct drm_encoder *encoder);
 void dce8_program_fmt(struct drm_encoder *encoder);
 
 /* fbdev layer */
-int radeon_fbdev_init(struct radeon_device *rdev);
-void radeon_fbdev_fini(struct radeon_device *rdev);
+void radeon_fbdev_init(struct radeon_device *rdev);
 void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state);
 bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj);
 
-- 
2.29.2

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 5/7] drm/radeon: Replace framebuffer console with generic implementation

[Thomas Zimmermann]

The fbdev code in radeon can be replaced by the generic fbdev helpers.
This allows for using fbdev shadow buffers and IGT testcases.

Generic fbdev acts like a regular DRM client and has to be initialized
after registering the DRM device. Change the driver initialization
accordingly. The release happens automatically during shutdown.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/radeon/radeon_display.c |   2 -
 drivers/gpu/drm/radeon/radeon_drv.c     |   5 +-
 drivers/gpu/drm/radeon/radeon_fb.c      | 337 +-----------------------
 drivers/gpu/drm/radeon/radeon_kms.c     |  12 -
 drivers/gpu/drm/radeon/radeon_mode.h    |   7 +-
 5 files changed, 16 insertions(+), 347 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_display.c b/drivers/gpu/drm/radeon/radeon_display.c
index b79686cf8bdb..a7bae33636c3 100644
--- a/drivers/gpu/drm/radeon/radeon_display.c
+++ b/drivers/gpu/drm/radeon/radeon_display.c
@@ -1638,7 +1638,6 @@ int radeon_modeset_init(struct radeon_device *rdev)
 	/* setup afmt */
 	radeon_afmt_init(rdev);
 
-	radeon_fbdev_init(rdev);
 	drm_kms_helper_poll_init(rdev->ddev);
 
 	/* do pm late init */
@@ -1653,7 +1652,6 @@ void radeon_modeset_fini(struct radeon_device *rdev)
 		drm_kms_helper_poll_fini(rdev->ddev);
 		radeon_hpd_fini(rdev);
 		drm_helper_force_disable_all(rdev->ddev);
-		radeon_fbdev_fini(rdev);
 		radeon_afmt_fini(rdev);
 		drm_mode_config_cleanup(rdev->ddev);
 		rdev->mode_info.mode_config_initialized = false;
diff --git a/drivers/gpu/drm/radeon/radeon_drv.c b/drivers/gpu/drm/radeon/radeon_drv.c
index bfadb799d6a3..63c3bf1a3a3c 100644
--- a/drivers/gpu/drm/radeon/radeon_drv.c
+++ b/drivers/gpu/drm/radeon/radeon_drv.c
@@ -114,7 +114,6 @@
 #define KMS_DRIVER_PATCHLEVEL	0
 int radeon_driver_load_kms(struct drm_device *dev, unsigned long flags);
 void radeon_driver_unload_kms(struct drm_device *dev);
-void radeon_driver_lastclose_kms(struct drm_device *dev);
 int radeon_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv);
 void radeon_driver_postclose_kms(struct drm_device *dev,
 				 struct drm_file *file_priv);
@@ -374,6 +373,8 @@ static int radeon_pci_probe(struct pci_dev *pdev,
 	if (ret)
 		goto err_agp;
 
+	radeon_fbdev_init(dev->dev_private);
+
 	return 0;
 
 err_agp:
@@ -635,7 +636,7 @@ static const struct drm_driver kms_driver = {
 	.load = radeon_driver_load_kms,
 	.open = radeon_driver_open_kms,
 	.postclose = radeon_driver_postclose_kms,
-	.lastclose = radeon_driver_lastclose_kms,
+	.lastclose = drm_fb_helper_lastclose,
 	.unload = radeon_driver_unload_kms,
 	.irq_preinstall = radeon_driver_irq_preinstall_kms,
 	.irq_postinstall = radeon_driver_irq_postinstall_kms,
diff --git a/drivers/gpu/drm/radeon/radeon_fb.c b/drivers/gpu/drm/radeon/radeon_fb.c
index fc4212633bdf..50fdc2aaa463 100644
--- a/drivers/gpu/drm/radeon/radeon_fb.c
+++ b/drivers/gpu/drm/radeon/radeon_fb.c
@@ -24,66 +24,12 @@
  *     David Airlie
  */
 
-#include <linux/module.h>
 #include <linux/pci.h>
-#include <linux/pm_runtime.h>
-#include <linux/slab.h>
-#include <linux/vga_switcheroo.h>
 
-#include <drm/drm_crtc.h>
-#include <drm/drm_crtc_helper.h>
 #include <drm/drm_fb_helper.h>
-#include <drm/drm_fourcc.h>
-#include <drm/radeon_drm.h>
 
 #include "radeon.h"
 
-/* object hierarchy -
- * this contains a helper + a radeon fb
- * the helper contains a pointer to radeon framebuffer baseclass.
- */
-struct radeon_fbdev {
-	struct drm_fb_helper helper; /* must be first */
-	struct drm_framebuffer fb;
-	struct radeon_device *rdev;
-};
-
-static int
-radeonfb_open(struct fb_info *info, int user)
-{
-	struct radeon_fbdev *rfbdev = info->par;
-	struct radeon_device *rdev = rfbdev->rdev;
-	int ret = pm_runtime_get_sync(rdev->ddev->dev);
-	if (ret < 0 && ret != -EACCES) {
-		pm_runtime_mark_last_busy(rdev->ddev->dev);
-		pm_runtime_put_autosuspend(rdev->ddev->dev);
-		return ret;
-	}
-	return 0;
-}
-
-static int
-radeonfb_release(struct fb_info *info, int user)
-{
-	struct radeon_fbdev *rfbdev = info->par;
-	struct radeon_device *rdev = rfbdev->rdev;
-
-	pm_runtime_mark_last_busy(rdev->ddev->dev);
-	pm_runtime_put_autosuspend(rdev->ddev->dev);
-	return 0;
-}
-
-static const struct fb_ops radeonfb_ops = {
-	.owner = THIS_MODULE,
-	DRM_FB_HELPER_DEFAULT_OPS,
-	.fb_open = radeonfb_open,
-	.fb_release = radeonfb_release,
-	.fb_fillrect = drm_fb_helper_cfb_fillrect,
-	.fb_copyarea = drm_fb_helper_cfb_copyarea,
-	.fb_imageblit = drm_fb_helper_cfb_imageblit,
-};
-
-
 int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tiled)
 {
 	int aligned = width;
@@ -108,294 +54,35 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 	return aligned * cpp;
 }
 
-static void radeonfb_destroy_pinned_object(struct drm_gem_object *gobj)
-{
-	struct radeon_bo *rbo = gem_to_radeon_bo(gobj);
-	int ret;
-
-	ret = radeon_bo_reserve(rbo, false);
-	if (likely(ret == 0)) {
-		radeon_bo_kunmap(rbo);
-		radeon_bo_unpin(rbo);
-		radeon_bo_unreserve(rbo);
-	}
-	drm_gem_object_put(gobj);
-}
-
-static int radeonfb_create_pinned_object(struct radeon_fbdev *rfbdev,
-					 struct drm_mode_fb_cmd2 *mode_cmd,
-					 struct drm_gem_object **gobj_p)
-{
-	const struct drm_format_info *info;
-	struct radeon_device *rdev = rfbdev->rdev;
-	struct drm_gem_object *gobj = NULL;
-	struct radeon_bo *rbo = NULL;
-	bool fb_tiled = false; /* useful for testing */
-	u32 tiling_flags = 0;
-	int ret;
-	int aligned_size, size;
-	int height = mode_cmd->height;
-	u32 cpp;
-
-	info = drm_get_format_info(rdev->ddev, mode_cmd);
-	cpp = info->cpp[0];
-
-	/* need to align pitch with crtc limits */
-	mode_cmd->pitches[0] = radeon_align_pitch(rdev, mode_cmd->width, cpp,
-						  fb_tiled);
-
-	if (rdev->family >= CHIP_R600)
-		height = ALIGN(mode_cmd->height, 8);
-	size = mode_cmd->pitches[0] * height;
-	aligned_size = ALIGN(size, PAGE_SIZE);
-	ret = radeon_gem_object_create(rdev, aligned_size, 0,
-				       RADEON_GEM_DOMAIN_VRAM,
-				       0, true, &gobj);
-	if (ret) {
-		pr_err("failed to allocate framebuffer (%d)\n", aligned_size);
-		return -ENOMEM;
-	}
-	rbo = gem_to_radeon_bo(gobj);
-
-	if (fb_tiled)
-		tiling_flags = RADEON_TILING_MACRO;
-
-#ifdef __BIG_ENDIAN
-	switch (cpp) {
-	case 4:
-		tiling_flags |= RADEON_TILING_SWAP_32BIT;
-		break;
-	case 2:
-		tiling_flags |= RADEON_TILING_SWAP_16BIT;
-	default:
-		break;
-	}
-#endif
-
-	if (tiling_flags) {
-		ret = radeon_bo_set_tiling_flags(rbo,
-						 tiling_flags | RADEON_TILING_SURFACE,
-						 mode_cmd->pitches[0]);
-		if (ret)
-			dev_err(rdev->dev, "FB failed to set tiling flags\n");
-	}
-
-
-	ret = radeon_bo_reserve(rbo, false);
-	if (unlikely(ret != 0))
-		goto out_unref;
-	/* Only 27 bit offset for legacy CRTC */
-	ret = radeon_bo_pin_restricted(rbo, RADEON_GEM_DOMAIN_VRAM,
-				       ASIC_IS_AVIVO(rdev) ? 0 : 1 << 27,
-				       NULL);
-	if (ret) {
-		radeon_bo_unreserve(rbo);
-		goto out_unref;
-	}
-	if (fb_tiled)
-		radeon_bo_check_tiling(rbo, 0, 0);
-	ret = radeon_bo_kmap(rbo, NULL);
-	radeon_bo_unreserve(rbo);
-	if (ret) {
-		goto out_unref;
-	}
-
-	*gobj_p = gobj;
-	return 0;
-out_unref:
-	radeonfb_destroy_pinned_object(gobj);
-	*gobj_p = NULL;
-	return ret;
-}
-
-static int radeonfb_create(struct drm_fb_helper *helper,
-			   struct drm_fb_helper_surface_size *sizes)
-{
-	struct radeon_fbdev *rfbdev =
-		container_of(helper, struct radeon_fbdev, helper);
-	struct radeon_device *rdev = rfbdev->rdev;
-	struct fb_info *info;
-	struct drm_framebuffer *fb = NULL;
-	struct drm_mode_fb_cmd2 mode_cmd;
-	struct drm_gem_object *gobj = NULL;
-	struct radeon_bo *rbo = NULL;
-	int ret;
-	unsigned long tmp;
-
-	mode_cmd.width = sizes->surface_width;
-	mode_cmd.height = sizes->surface_height;
-
-	/* avivo can't scanout real 24bpp */
-	if ((sizes->surface_bpp == 24) && ASIC_IS_AVIVO(rdev))
-		sizes->surface_bpp = 32;
-
-	mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp,
-							  sizes->surface_depth);
-
-	ret = radeonfb_create_pinned_object(rfbdev, &mode_cmd, &gobj);
-	if (ret) {
-		DRM_ERROR("failed to create fbcon object %d\n", ret);
-		return ret;
-	}
-
-	rbo = gem_to_radeon_bo(gobj);
-
-	/* okay we have an object now allocate the framebuffer */
-	info = drm_fb_helper_alloc_fbi(helper);
-	if (IS_ERR(info)) {
-		ret = PTR_ERR(info);
-		goto out;
-	}
-
-	/* radeon resume is fragile and needs a vt switch to help it along */
-	info->skip_vt_switch = false;
-
-	ret = radeon_framebuffer_init(rdev->ddev, &rfbdev->fb, &mode_cmd, gobj);
-	if (ret) {
-		DRM_ERROR("failed to initialize framebuffer %d\n", ret);
-		goto out;
-	}
-
-	fb = &rfbdev->fb;
-
-	/* setup helper */
-	rfbdev->helper.fb = fb;
-
-	memset_io(rbo->kptr, 0x0, radeon_bo_size(rbo));
-
-	info->fbops = &radeonfb_ops;
-
-	tmp = radeon_bo_gpu_offset(rbo) - rdev->mc.vram_start;
-	info->fix.smem_start = rdev->mc.aper_base + tmp;
-	info->fix.smem_len = radeon_bo_size(rbo);
-	info->screen_base = rbo->kptr;
-	info->screen_size = radeon_bo_size(rbo);
-
-	drm_fb_helper_fill_info(info, &rfbdev->helper, sizes);
-
-	/* setup aperture base/size for vesafb takeover */
-	info->apertures->ranges[0].base = rdev->ddev->mode_config.fb_base;
-	info->apertures->ranges[0].size = rdev->mc.aper_size;
-
-	/* Use default scratch pixmap (info->pixmap.flags = FB_PIXMAP_SYSTEM) */
-
-	if (info->screen_base == NULL) {
-		ret = -ENOSPC;
-		goto out;
-	}
-
-	DRM_INFO("fb mappable at 0x%lX\n",  info->fix.smem_start);
-	DRM_INFO("vram apper at 0x%lX\n",  (unsigned long)rdev->mc.aper_base);
-	DRM_INFO("size %lu\n", (unsigned long)radeon_bo_size(rbo));
-	DRM_INFO("fb depth is %d\n", fb->format->depth);
-	DRM_INFO("   pitch is %d\n", fb->pitches[0]);
-
-	vga_switcheroo_client_fb_set(rdev->ddev->pdev, info);
-	return 0;
-
-out:
-	if (rbo) {
-
-	}
-	if (fb && ret) {
-		drm_gem_object_put(gobj);
-		drm_framebuffer_unregister_private(fb);
-		drm_framebuffer_cleanup(fb);
-		kfree(fb);
-	}
-	return ret;
-}
-
-static int radeon_fbdev_destroy(struct drm_device *dev, struct radeon_fbdev *rfbdev)
-{
-	struct drm_framebuffer *fb = &rfbdev->fb;
-
-	drm_fb_helper_unregister_fbi(&rfbdev->helper);
-
-	if (fb->obj[0]) {
-		radeonfb_destroy_pinned_object(fb->obj[0]);
-		fb->obj[0] = NULL;
-		drm_framebuffer_unregister_private(fb);
-		drm_framebuffer_cleanup(fb);
-	}
-	drm_fb_helper_fini(&rfbdev->helper);
-
-	return 0;
-}
-
-static const struct drm_fb_helper_funcs radeon_fb_helper_funcs = {
-	.fb_probe = radeonfb_create,
-};
-
-int radeon_fbdev_init(struct radeon_device *rdev)
+void radeon_fbdev_init(struct radeon_device *rdev)
 {
-	struct radeon_fbdev *rfbdev;
+	struct drm_device *ddev = rdev->ddev;
 	int bpp_sel = 32;
-	int ret;
-
-	/* don't enable fbdev if no connectors */
-	if (list_empty(&rdev->ddev->mode_config.connector_list))
-		return 0;
 
 	/* select 8 bpp console on 8MB cards, or 16 bpp on RN50 or 32MB */
 	if (rdev->mc.real_vram_size <= (8*1024*1024))
 		bpp_sel = 8;
-	else if (ASIC_IS_RN50(rdev) ||
-		 rdev->mc.real_vram_size <= (32*1024*1024))
+	else if (ASIC_IS_RN50(rdev) || rdev->mc.real_vram_size <= (32*1024*1024))
 		bpp_sel = 16;
 
-	rfbdev = kzalloc(sizeof(struct radeon_fbdev), GFP_KERNEL);
-	if (!rfbdev)
-		return -ENOMEM;
-
-	rfbdev->rdev = rdev;
-	rdev->mode_info.rfbdev = rfbdev;
-
-	drm_fb_helper_prepare(rdev->ddev, &rfbdev->helper,
-			      &radeon_fb_helper_funcs);
-
-	ret = drm_fb_helper_init(rdev->ddev, &rfbdev->helper);
-	if (ret)
-		goto free;
-
-	/* disable all the possible outputs/crtcs before entering KMS mode */
-	drm_helper_disable_unused_functions(rdev->ddev);
-
-	ret = drm_fb_helper_initial_config(&rfbdev->helper, bpp_sel);
-	if (ret)
-		goto fini;
-
-	return 0;
-
-fini:
-	drm_fb_helper_fini(&rfbdev->helper);
-free:
-	kfree(rfbdev);
-	return ret;
-}
-
-void radeon_fbdev_fini(struct radeon_device *rdev)
-{
-	if (!rdev->mode_info.rfbdev)
-		return;
+	/* radeon resume is fragile and needs a vt switch to help it along */
+	ddev->mode_config.require_vt_switch_fbdev = true;
 
-	radeon_fbdev_destroy(rdev->ddev, rdev->mode_info.rfbdev);
-	kfree(rdev->mode_info.rfbdev);
-	rdev->mode_info.rfbdev = NULL;
+	drm_fbdev_generic_setup(ddev, bpp_sel);
 }
 
 void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state)
 {
-	if (rdev->mode_info.rfbdev)
-		drm_fb_helper_set_suspend(&rdev->mode_info.rfbdev->helper, state);
+	drm_fb_helper_set_suspend(rdev->ddev->fb_helper, state);
 }
 
 bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj)
 {
-	if (!rdev->mode_info.rfbdev)
+	if (!rdev->ddev->fb_helper)
+		return false;
+
+	if (gem_to_radeon_bo(rdev->ddev->fb_helper->buffer->gem) != robj)
 		return false;
 
-	if (robj == gem_to_radeon_bo(rdev->mode_info.rfbdev->fb.obj[0]))
-		return true;
-	return false;
+	return true;
 }
diff --git a/drivers/gpu/drm/radeon/radeon_kms.c b/drivers/gpu/drm/radeon/radeon_kms.c
index 75b038740ea8..76e18e589ccd 100644
--- a/drivers/gpu/drm/radeon/radeon_kms.c
+++ b/drivers/gpu/drm/radeon/radeon_kms.c
@@ -608,18 +608,6 @@ int radeon_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
 /*
  * Outdated mess for old drm with Xorg being in charge (void function now).
  */
-/**
- * radeon_driver_lastclose_kms - drm callback for last close
- *
- * @dev: drm dev pointer
- *
- * Switch vga_switcheroo state after last close (all asics).
- */
-void radeon_driver_lastclose_kms(struct drm_device *dev)
-{
-	drm_fb_helper_lastclose(dev);
-	vga_switcheroo_process_delayed_switch();
-}
 
 /**
  * radeon_driver_open_kms - drm callback for open
diff --git a/drivers/gpu/drm/radeon/radeon_mode.h b/drivers/gpu/drm/radeon/radeon_mode.h
index c7f223743d46..fe192ef0aafa 100644
--- a/drivers/gpu/drm/radeon/radeon_mode.h
+++ b/drivers/gpu/drm/radeon/radeon_mode.h
@@ -230,8 +230,6 @@ enum radeon_dvo_chip {
 	DVO_SIL1178,
 };
 
-struct radeon_fbdev;
-
 struct radeon_afmt {
 	bool enabled;
 	int offset;
@@ -268,8 +266,6 @@ struct radeon_mode_info {
 	struct edid *bios_hardcoded_edid;
 	int bios_hardcoded_edid_size;
 
-	/* pointer to fbdev info structure */
-	struct radeon_fbdev *rfbdev;
 	/* firmware flags */
 	u16 firmware_flags;
 	/* pointer to backlight encoder */
@@ -979,8 +975,7 @@ void dce4_program_fmt(struct drm_encoder *encoder);
 void dce8_program_fmt(struct drm_encoder *encoder);
 
 /* fbdev layer */
-int radeon_fbdev_init(struct radeon_device *rdev);
-void radeon_fbdev_fini(struct radeon_device *rdev);
+void radeon_fbdev_init(struct radeon_device *rdev);
 void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state);
 bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj);
 
-- 
2.29.2

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 6/7] drm/radeon: Use fbdev shadow fb

[Thomas Zimmermann]

Fbdev framebuffer addresses are exported and mmap'ed to userspace, so the
framebuffer may not change its location. This creates memory pressure on
devices with little video RAM. Radeon mitigates this problem by reducing
the framebuffer's color depth on devices with 32 MiB or less.

Fully resolve the problem by enabling generic fbdev's shadow FB. All
userspace-visible memory is located in the shadow buffer. The underlying
BO can be evicted to system memory as needed, thus freeing video RAM for
more important uses.

Switching to a shadow framebuffer also allows to remove a special case in
BO handling, where the fbdev's BO was not evicted. Fbdev should now be
treated an any other framebuffer BO

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/radeon/radeon_device.c | 11 ++++-------
 drivers/gpu/drm/radeon/radeon_fb.c     | 17 ++---------------
 drivers/gpu/drm/radeon/radeon_mode.h   |  1 -
 3 files changed, 6 insertions(+), 23 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
index 266e3cbbd09b..87c14f6369ee 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -1605,13 +1605,10 @@ int radeon_suspend_kms(struct drm_device *dev, bool suspend,
 			continue;
 		}
 		robj = gem_to_radeon_bo(fb->obj[0]);
-		/* don't unpin kernel fb objects */
-		if (!radeon_fbdev_robj_is_fb(rdev, robj)) {
-			r = radeon_bo_reserve(robj, false);
-			if (r == 0) {
-				radeon_bo_unpin(robj);
-				radeon_bo_unreserve(robj);
-			}
+		r = radeon_bo_reserve(robj, false);
+		if (r == 0) {
+			radeon_bo_unpin(robj);
+			radeon_bo_unreserve(robj);
 		}
 	}
 	/* evict vram memory */
diff --git a/drivers/gpu/drm/radeon/radeon_fb.c b/drivers/gpu/drm/radeon/radeon_fb.c
index 50fdc2aaa463..c687d14a1b93 100644
--- a/drivers/gpu/drm/radeon/radeon_fb.c
+++ b/drivers/gpu/drm/radeon/radeon_fb.c
@@ -59,14 +59,12 @@ void radeon_fbdev_init(struct radeon_device *rdev)
 	struct drm_device *ddev = rdev->ddev;
 	int bpp_sel = 32;
 
-	/* select 8 bpp console on 8MB cards, or 16 bpp on RN50 or 32MB */
-	if (rdev->mc.real_vram_size <= (8*1024*1024))
-		bpp_sel = 8;
-	else if (ASIC_IS_RN50(rdev) || rdev->mc.real_vram_size <= (32*1024*1024))
+	if (ASIC_IS_RN50(rdev))
 		bpp_sel = 16;
 
 	/* radeon resume is fragile and needs a vt switch to help it along */
 	ddev->mode_config.require_vt_switch_fbdev = true;
+	ddev->mode_config.prefer_shadow_fbdev = true;
 
 	drm_fbdev_generic_setup(ddev, bpp_sel);
 }
@@ -75,14 +73,3 @@ void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state)
 {
 	drm_fb_helper_set_suspend(rdev->ddev->fb_helper, state);
 }
-
-bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj)
-{
-	if (!rdev->ddev->fb_helper)
-		return false;
-
-	if (gem_to_radeon_bo(rdev->ddev->fb_helper->buffer->gem) != robj)
-		return false;
-
-	return true;
-}
diff --git a/drivers/gpu/drm/radeon/radeon_mode.h b/drivers/gpu/drm/radeon/radeon_mode.h
index fe192ef0aafa..197423ad42c1 100644
--- a/drivers/gpu/drm/radeon/radeon_mode.h
+++ b/drivers/gpu/drm/radeon/radeon_mode.h
@@ -977,7 +977,6 @@ void dce8_program_fmt(struct drm_encoder *encoder);
 /* fbdev layer */
 void radeon_fbdev_init(struct radeon_device *rdev);
 void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state);
-bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj);
 
 void radeon_crtc_handle_vblank(struct radeon_device *rdev, int crtc_id);
 
-- 
2.29.2

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 6/7] drm/radeon: Use fbdev shadow fb

[Thomas Zimmermann]

Fbdev framebuffer addresses are exported and mmap'ed to userspace, so the
framebuffer may not change its location. This creates memory pressure on
devices with little video RAM. Radeon mitigates this problem by reducing
the framebuffer's color depth on devices with 32 MiB or less.

Fully resolve the problem by enabling generic fbdev's shadow FB. All
userspace-visible memory is located in the shadow buffer. The underlying
BO can be evicted to system memory as needed, thus freeing video RAM for
more important uses.

Switching to a shadow framebuffer also allows to remove a special case in
BO handling, where the fbdev's BO was not evicted. Fbdev should now be
treated an any other framebuffer BO

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/radeon/radeon_device.c | 11 ++++-------
 drivers/gpu/drm/radeon/radeon_fb.c     | 17 ++---------------
 drivers/gpu/drm/radeon/radeon_mode.h   |  1 -
 3 files changed, 6 insertions(+), 23 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
index 266e3cbbd09b..87c14f6369ee 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -1605,13 +1605,10 @@ int radeon_suspend_kms(struct drm_device *dev, bool suspend,
 			continue;
 		}
 		robj = gem_to_radeon_bo(fb->obj[0]);
-		/* don't unpin kernel fb objects */
-		if (!radeon_fbdev_robj_is_fb(rdev, robj)) {
-			r = radeon_bo_reserve(robj, false);
-			if (r == 0) {
-				radeon_bo_unpin(robj);
-				radeon_bo_unreserve(robj);
-			}
+		r = radeon_bo_reserve(robj, false);
+		if (r == 0) {
+			radeon_bo_unpin(robj);
+			radeon_bo_unreserve(robj);
 		}
 	}
 	/* evict vram memory */
diff --git a/drivers/gpu/drm/radeon/radeon_fb.c b/drivers/gpu/drm/radeon/radeon_fb.c
index 50fdc2aaa463..c687d14a1b93 100644
--- a/drivers/gpu/drm/radeon/radeon_fb.c
+++ b/drivers/gpu/drm/radeon/radeon_fb.c
@@ -59,14 +59,12 @@ void radeon_fbdev_init(struct radeon_device *rdev)
 	struct drm_device *ddev = rdev->ddev;
 	int bpp_sel = 32;
 
-	/* select 8 bpp console on 8MB cards, or 16 bpp on RN50 or 32MB */
-	if (rdev->mc.real_vram_size <= (8*1024*1024))
-		bpp_sel = 8;
-	else if (ASIC_IS_RN50(rdev) || rdev->mc.real_vram_size <= (32*1024*1024))
+	if (ASIC_IS_RN50(rdev))
 		bpp_sel = 16;
 
 	/* radeon resume is fragile and needs a vt switch to help it along */
 	ddev->mode_config.require_vt_switch_fbdev = true;
+	ddev->mode_config.prefer_shadow_fbdev = true;
 
 	drm_fbdev_generic_setup(ddev, bpp_sel);
 }
@@ -75,14 +73,3 @@ void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state)
 {
 	drm_fb_helper_set_suspend(rdev->ddev->fb_helper, state);
 }
-
-bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj)
-{
-	if (!rdev->ddev->fb_helper)
-		return false;
-
-	if (gem_to_radeon_bo(rdev->ddev->fb_helper->buffer->gem) != robj)
-		return false;
-
-	return true;
-}
diff --git a/drivers/gpu/drm/radeon/radeon_mode.h b/drivers/gpu/drm/radeon/radeon_mode.h
index fe192ef0aafa..197423ad42c1 100644
--- a/drivers/gpu/drm/radeon/radeon_mode.h
+++ b/drivers/gpu/drm/radeon/radeon_mode.h
@@ -977,7 +977,6 @@ void dce8_program_fmt(struct drm_encoder *encoder);
 /* fbdev layer */
 void radeon_fbdev_init(struct radeon_device *rdev);
 void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state);
-bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj);
 
 void radeon_crtc_handle_vblank(struct radeon_device *rdev, int crtc_id);
 
-- 
2.29.2

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 7/7] drm/radeon: Move radeon_align_pitch() next to its only caller

[Thomas Zimmermann]

Allows to declare the function as static. The tiled parameter is always
false, so it is being removed.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/radeon/radeon_fb.c   | 24 ------------------------
 drivers/gpu/drm/radeon/radeon_gem.c  | 26 +++++++++++++++++++++++++-
 drivers/gpu/drm/radeon/radeon_mode.h |  2 --
 3 files changed, 25 insertions(+), 27 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_fb.c b/drivers/gpu/drm/radeon/radeon_fb.c
index c687d14a1b93..0b89dd4f4af6 100644
--- a/drivers/gpu/drm/radeon/radeon_fb.c
+++ b/drivers/gpu/drm/radeon/radeon_fb.c
@@ -30,30 +30,6 @@
 
 #include "radeon.h"
 
-int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tiled)
-{
-	int aligned = width;
-	int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
-	int pitch_mask = 0;
-
-	switch (cpp) {
-	case 1:
-		pitch_mask = align_large ? 255 : 127;
-		break;
-	case 2:
-		pitch_mask = align_large ? 127 : 31;
-		break;
-	case 3:
-	case 4:
-		pitch_mask = align_large ? 63 : 15;
-		break;
-	}
-
-	aligned += pitch_mask;
-	aligned &= ~pitch_mask;
-	return aligned * cpp;
-}
-
 void radeon_fbdev_init(struct radeon_device *rdev)
 {
 	struct drm_device *ddev = rdev->ddev;
diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
index eaf7fc9a7b07..e19956c3309f 100644
--- a/drivers/gpu/drm/radeon/radeon_gem.c
+++ b/drivers/gpu/drm/radeon/radeon_gem.c
@@ -816,6 +816,30 @@ int radeon_gem_op_ioctl(struct drm_device *dev, void *data,
 	return r;
 }
 
+static int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp)
+{
+	int aligned = width;
+	int align_large = ASIC_IS_AVIVO(rdev);
+	int pitch_mask = 0;
+
+	switch (cpp) {
+	case 1:
+		pitch_mask = align_large ? 255 : 127;
+		break;
+	case 2:
+		pitch_mask = align_large ? 127 : 31;
+		break;
+	case 3:
+	case 4:
+		pitch_mask = align_large ? 63 : 15;
+		break;
+	}
+
+	aligned += pitch_mask;
+	aligned &= ~pitch_mask;
+	return aligned * cpp;
+}
+
 int radeon_mode_dumb_create(struct drm_file *file_priv,
 			    struct drm_device *dev,
 			    struct drm_mode_create_dumb *args)
@@ -826,7 +850,7 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
 	int r;
 
 	args->pitch = radeon_align_pitch(rdev, args->width,
-					 DIV_ROUND_UP(args->bpp, 8), 0);
+					 DIV_ROUND_UP(args->bpp, 8));
 	args->size = args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
 
diff --git a/drivers/gpu/drm/radeon/radeon_mode.h b/drivers/gpu/drm/radeon/radeon_mode.h
index 197423ad42c1..50ba2b5edb93 100644
--- a/drivers/gpu/drm/radeon/radeon_mode.h
+++ b/drivers/gpu/drm/radeon/radeon_mode.h
@@ -982,8 +982,6 @@ void radeon_crtc_handle_vblank(struct radeon_device *rdev, int crtc_id);
 
 void radeon_crtc_handle_flip(struct radeon_device *rdev, int crtc_id);
 
-int radeon_align_pitch(struct radeon_device *rdev, int width, int bpp, bool tiled);
-
 /* mst */
 int radeon_dp_mst_init(struct radeon_connector *radeon_connector);
 int radeon_dp_mst_probe(struct radeon_connector *radeon_connector);
-- 
2.29.2

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 7/7] drm/radeon: Move radeon_align_pitch() next to its only caller

[Thomas Zimmermann]

Allows to declare the function as static. The tiled parameter is always
false, so it is being removed.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/gpu/drm/radeon/radeon_fb.c   | 24 ------------------------
 drivers/gpu/drm/radeon/radeon_gem.c  | 26 +++++++++++++++++++++++++-
 drivers/gpu/drm/radeon/radeon_mode.h |  2 --
 3 files changed, 25 insertions(+), 27 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_fb.c b/drivers/gpu/drm/radeon/radeon_fb.c
index c687d14a1b93..0b89dd4f4af6 100644
--- a/drivers/gpu/drm/radeon/radeon_fb.c
+++ b/drivers/gpu/drm/radeon/radeon_fb.c
@@ -30,30 +30,6 @@
 
 #include "radeon.h"
 
-int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tiled)
-{
-	int aligned = width;
-	int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
-	int pitch_mask = 0;
-
-	switch (cpp) {
-	case 1:
-		pitch_mask = align_large ? 255 : 127;
-		break;
-	case 2:
-		pitch_mask = align_large ? 127 : 31;
-		break;
-	case 3:
-	case 4:
-		pitch_mask = align_large ? 63 : 15;
-		break;
-	}
-
-	aligned += pitch_mask;
-	aligned &= ~pitch_mask;
-	return aligned * cpp;
-}
-
 void radeon_fbdev_init(struct radeon_device *rdev)
 {
 	struct drm_device *ddev = rdev->ddev;
diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
index eaf7fc9a7b07..e19956c3309f 100644
--- a/drivers/gpu/drm/radeon/radeon_gem.c
+++ b/drivers/gpu/drm/radeon/radeon_gem.c
@@ -816,6 +816,30 @@ int radeon_gem_op_ioctl(struct drm_device *dev, void *data,
 	return r;
 }
 
+static int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp)
+{
+	int aligned = width;
+	int align_large = ASIC_IS_AVIVO(rdev);
+	int pitch_mask = 0;
+
+	switch (cpp) {
+	case 1:
+		pitch_mask = align_large ? 255 : 127;
+		break;
+	case 2:
+		pitch_mask = align_large ? 127 : 31;
+		break;
+	case 3:
+	case 4:
+		pitch_mask = align_large ? 63 : 15;
+		break;
+	}
+
+	aligned += pitch_mask;
+	aligned &= ~pitch_mask;
+	return aligned * cpp;
+}
+
 int radeon_mode_dumb_create(struct drm_file *file_priv,
 			    struct drm_device *dev,
 			    struct drm_mode_create_dumb *args)
@@ -826,7 +850,7 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
 	int r;
 
 	args->pitch = radeon_align_pitch(rdev, args->width,
-					 DIV_ROUND_UP(args->bpp, 8), 0);
+					 DIV_ROUND_UP(args->bpp, 8));
 	args->size = args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
 
diff --git a/drivers/gpu/drm/radeon/radeon_mode.h b/drivers/gpu/drm/radeon/radeon_mode.h
index 197423ad42c1..50ba2b5edb93 100644
--- a/drivers/gpu/drm/radeon/radeon_mode.h
+++ b/drivers/gpu/drm/radeon/radeon_mode.h
@@ -982,8 +982,6 @@ void radeon_crtc_handle_vblank(struct radeon_device *rdev, int crtc_id);
 
 void radeon_crtc_handle_flip(struct radeon_device *rdev, int crtc_id);
 
-int radeon_align_pitch(struct radeon_device *rdev, int width, int bpp, bool tiled);
-
 /* mst */
 int radeon_dp_mst_init(struct radeon_connector *radeon_connector);
 int radeon_dp_mst_probe(struct radeon_connector *radeon_connector);
-- 
2.29.2

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
> object's vmap implementation. Unpin them in the vunmap implementation.
> This is needed to make generic fbdev support work reliably. Without,
> the buffer object could be evicted while fbdev flushed its shadow buffer.
>
> In difference to the PRIME pin/unpin functions, the vmap code does not
> modify the BOs prime_shared_count, so a vmap-pinned BO does not count as
> shared.
>
> The actual pin location is not important as the vmap call returns
> information on how to access the buffer. Callers that require a
> specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

If yes then the pin location is actually rather important since the 
hardware can only scan out from VRAM.

Regards,
Christian.

>
> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
> ---
>   drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
>   1 file changed, 49 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
> index d2876ce3bc9e..eaf7fc9a7b07 100644
> --- a/drivers/gpu/drm/radeon/radeon_gem.c
> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct radeon_device *rdev, int r)
>   	return r;
>   }
>   
> +static int radeon_gem_object_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
> +{
> +	static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
> +					   RADEON_GEM_DOMAIN_GTT |
> +					   RADEON_GEM_DOMAIN_CPU;
> +
> +	struct radeon_bo *bo = gem_to_radeon_bo(obj);
> +	int ret;
> +
> +	ret = radeon_bo_reserve(bo, false);
> +	if (ret)
> +		return ret;
> +
> +	/* pin buffer at its current location */
> +	ret = radeon_bo_pin(bo, any_domain, NULL);
> +	if (ret)
> +		goto err_radeon_bo_unreserve;
> +
> +	ret = drm_gem_ttm_vmap(obj, map);
> +	if (ret)
> +		goto err_radeon_bo_unpin;
> +
> +	radeon_bo_unreserve(bo);
> +
> +	return 0;
> +
> +err_radeon_bo_unpin:
> +	radeon_bo_unpin(bo);
> +err_radeon_bo_unreserve:
> +	radeon_bo_unreserve(bo);
> +	return ret;
> +}
> +
> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj, struct dma_buf_map *map)
> +{
> +	struct radeon_bo *bo = gem_to_radeon_bo(obj);
> +	int ret;
> +
> +	ret = radeon_bo_reserve(bo, false);
> +	if (ret)
> +		return;
> +
> +	drm_gem_ttm_vunmap(obj, map);
> +	radeon_bo_unpin(bo);
> +	radeon_bo_unreserve(bo);
> +}
> +
>   static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>   	.free = radeon_gem_object_free,
>   	.open = radeon_gem_object_open,
> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>   	.pin = radeon_gem_prime_pin,
>   	.unpin = radeon_gem_prime_unpin,
>   	.get_sg_table = radeon_gem_prime_get_sg_table,
> -	.vmap = drm_gem_ttm_vmap,
> -	.vunmap = drm_gem_ttm_vunmap,
> +	.vmap = radeon_gem_object_vmap,
> +	.vunmap = radeon_gem_object_vunmap,
>   };
>   
>   /*

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
> object's vmap implementation. Unpin them in the vunmap implementation.
> This is needed to make generic fbdev support work reliably. Without,
> the buffer object could be evicted while fbdev flushed its shadow buffer.
>
> In difference to the PRIME pin/unpin functions, the vmap code does not
> modify the BOs prime_shared_count, so a vmap-pinned BO does not count as
> shared.
>
> The actual pin location is not important as the vmap call returns
> information on how to access the buffer. Callers that require a
> specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

If yes then the pin location is actually rather important since the 
hardware can only scan out from VRAM.

Regards,
Christian.

>
> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
> ---
>   drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
>   1 file changed, 49 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
> index d2876ce3bc9e..eaf7fc9a7b07 100644
> --- a/drivers/gpu/drm/radeon/radeon_gem.c
> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct radeon_device *rdev, int r)
>   	return r;
>   }
>   
> +static int radeon_gem_object_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
> +{
> +	static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
> +					   RADEON_GEM_DOMAIN_GTT |
> +					   RADEON_GEM_DOMAIN_CPU;
> +
> +	struct radeon_bo *bo = gem_to_radeon_bo(obj);
> +	int ret;
> +
> +	ret = radeon_bo_reserve(bo, false);
> +	if (ret)
> +		return ret;
> +
> +	/* pin buffer at its current location */
> +	ret = radeon_bo_pin(bo, any_domain, NULL);
> +	if (ret)
> +		goto err_radeon_bo_unreserve;
> +
> +	ret = drm_gem_ttm_vmap(obj, map);
> +	if (ret)
> +		goto err_radeon_bo_unpin;
> +
> +	radeon_bo_unreserve(bo);
> +
> +	return 0;
> +
> +err_radeon_bo_unpin:
> +	radeon_bo_unpin(bo);
> +err_radeon_bo_unreserve:
> +	radeon_bo_unreserve(bo);
> +	return ret;
> +}
> +
> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj, struct dma_buf_map *map)
> +{
> +	struct radeon_bo *bo = gem_to_radeon_bo(obj);
> +	int ret;
> +
> +	ret = radeon_bo_reserve(bo, false);
> +	if (ret)
> +		return;
> +
> +	drm_gem_ttm_vunmap(obj, map);
> +	radeon_bo_unpin(bo);
> +	radeon_bo_unreserve(bo);
> +}
> +
>   static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>   	.free = radeon_gem_object_free,
>   	.open = radeon_gem_object_open,
> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>   	.pin = radeon_gem_prime_pin,
>   	.unpin = radeon_gem_prime_unpin,
>   	.get_sg_table = radeon_gem_prime_get_sg_table,
> -	.vmap = drm_gem_ttm_vmap,
> -	.vunmap = drm_gem_ttm_vunmap,
> +	.vmap = radeon_gem_object_vmap,
> +	.vunmap = radeon_gem_object_vunmap,
>   };
>   
>   /*

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:
> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>> object's vmap implementation. Unpin them in the vunmap implementation.
>> This is needed to make generic fbdev support work reliably. Without,
>> the buffer object could be evicted while fbdev flushed its shadow buffer.
>>
>> In difference to the PRIME pin/unpin functions, the vmap code does not
>> modify the BOs prime_shared_count, so a vmap-pinned BO does not count as
>> shared.
>>
>> The actual pin location is not important as the vmap call returns
>> information on how to access the buffer. Callers that require a
>> specific location should explicitly pin the BO before vmapping it.
> 
> Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

> 
> If yes then the pin location is actually rather important since the
> hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any
relocation transparent to userspace. It flushes the shadow fb into the
BO's memory if there are updates. The code is in
drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
call now pins the BO to wherever it is. The actual location does not
matter. It's vunmap'ed immediately afterwards.

For dma-buf sharing, the regular procedure of pin + vmap still apply.
This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/drm_fb_helper.c#n432

> 
> Regards,
> Christian.
> 
>>
>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>> ---
>>   drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
>>   1 file changed, 49 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>> b/drivers/gpu/drm/radeon/radeon_gem.c
>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>> radeon_device *rdev, int r)
>>       return r;
>>   }
>>   +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>> struct dma_buf_map *map)
>> +{
>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>> +                       RADEON_GEM_DOMAIN_GTT |
>> +                       RADEON_GEM_DOMAIN_CPU;
>> +
>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>> +    int ret;
>> +
>> +    ret = radeon_bo_reserve(bo, false);
>> +    if (ret)
>> +        return ret;
>> +
>> +    /* pin buffer at its current location */
>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>> +    if (ret)
>> +        goto err_radeon_bo_unreserve;
>> +
>> +    ret = drm_gem_ttm_vmap(obj, map);
>> +    if (ret)
>> +        goto err_radeon_bo_unpin;
>> +
>> +    radeon_bo_unreserve(bo);
>> +
>> +    return 0;
>> +
>> +err_radeon_bo_unpin:
>> +    radeon_bo_unpin(bo);
>> +err_radeon_bo_unreserve:
>> +    radeon_bo_unreserve(bo);
>> +    return ret;
>> +}
>> +
>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>> struct dma_buf_map *map)
>> +{
>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>> +    int ret;
>> +
>> +    ret = radeon_bo_reserve(bo, false);
>> +    if (ret)
>> +        return;
>> +
>> +    drm_gem_ttm_vunmap(obj, map);
>> +    radeon_bo_unpin(bo);
>> +    radeon_bo_unreserve(bo);
>> +}
>> +
>>   static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>>       .free = radeon_gem_object_free,
>>       .open = radeon_gem_object_open,
>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>> radeon_gem_object_funcs = {
>>       .pin = radeon_gem_prime_pin,
>>       .unpin = radeon_gem_prime_unpin,
>>       .get_sg_table = radeon_gem_prime_get_sg_table,
>> -    .vmap = drm_gem_ttm_vmap,
>> -    .vunmap = drm_gem_ttm_vunmap,
>> +    .vmap = radeon_gem_object_vmap,
>> +    .vunmap = radeon_gem_object_vunmap,
>>   };
>>     /*
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:
> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>> object's vmap implementation. Unpin them in the vunmap implementation.
>> This is needed to make generic fbdev support work reliably. Without,
>> the buffer object could be evicted while fbdev flushed its shadow buffer.
>>
>> In difference to the PRIME pin/unpin functions, the vmap code does not
>> modify the BOs prime_shared_count, so a vmap-pinned BO does not count as
>> shared.
>>
>> The actual pin location is not important as the vmap call returns
>> information on how to access the buffer. Callers that require a
>> specific location should explicitly pin the BO before vmapping it.
> 
> Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

> 
> If yes then the pin location is actually rather important since the
> hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any
relocation transparent to userspace. It flushes the shadow fb into the
BO's memory if there are updates. The code is in
drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
call now pins the BO to wherever it is. The actual location does not
matter. It's vunmap'ed immediately afterwards.

For dma-buf sharing, the regular procedure of pin + vmap still apply.
This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/drm_fb_helper.c#n432

> 
> Regards,
> Christian.
> 
>>
>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>> ---
>>   drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
>>   1 file changed, 49 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>> b/drivers/gpu/drm/radeon/radeon_gem.c
>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>> radeon_device *rdev, int r)
>>       return r;
>>   }
>>   +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>> struct dma_buf_map *map)
>> +{
>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>> +                       RADEON_GEM_DOMAIN_GTT |
>> +                       RADEON_GEM_DOMAIN_CPU;
>> +
>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>> +    int ret;
>> +
>> +    ret = radeon_bo_reserve(bo, false);
>> +    if (ret)
>> +        return ret;
>> +
>> +    /* pin buffer at its current location */
>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>> +    if (ret)
>> +        goto err_radeon_bo_unreserve;
>> +
>> +    ret = drm_gem_ttm_vmap(obj, map);
>> +    if (ret)
>> +        goto err_radeon_bo_unpin;
>> +
>> +    radeon_bo_unreserve(bo);
>> +
>> +    return 0;
>> +
>> +err_radeon_bo_unpin:
>> +    radeon_bo_unpin(bo);
>> +err_radeon_bo_unreserve:
>> +    radeon_bo_unreserve(bo);
>> +    return ret;
>> +}
>> +
>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>> struct dma_buf_map *map)
>> +{
>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>> +    int ret;
>> +
>> +    ret = radeon_bo_reserve(bo, false);
>> +    if (ret)
>> +        return;
>> +
>> +    drm_gem_ttm_vunmap(obj, map);
>> +    radeon_bo_unpin(bo);
>> +    radeon_bo_unreserve(bo);
>> +}
>> +
>>   static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>>       .free = radeon_gem_object_free,
>>       .open = radeon_gem_object_open,
>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>> radeon_gem_object_funcs = {
>>       .pin = radeon_gem_prime_pin,
>>       .unpin = radeon_gem_prime_unpin,
>>       .get_sg_table = radeon_gem_prime_get_sg_table,
>> -    .vmap = drm_gem_ttm_vmap,
>> -    .vunmap = drm_gem_ttm_vunmap,
>> +    .vmap = radeon_gem_object_vmap,
>> +    .vunmap = radeon_gem_object_vunmap,
>>   };
>>     /*
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

Hi

Am 16.11.20 um 12:28 schrieb Christian König:
> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>> Hi Christian
>>
>> Am 12.11.20 um 18:16 schrieb Christian König:
>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>>> object's vmap implementation. Unpin them in the vunmap implementation.
>>>> This is needed to make generic fbdev support work reliably. Without,
>>>> the buffer object could be evicted while fbdev flushed its shadow
>>>> buffer.
>>>>
>>>> In difference to the PRIME pin/unpin functions, the vmap code does not
>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not
>>>> count as
>>>> shared.
>>>>
>>>> The actual pin location is not important as the vmap call returns
>>>> information on how to access the buffer. Callers that require a
>>>> specific location should explicitly pin the BO before vmapping it.
>>> Well is the buffer supposed to be scanned out?
>> No, not by the fbdev helper.
> 
> Ok in this case that should work.
> 
>>> If yes then the pin location is actually rather important since the
>>> hardware can only scan out from VRAM.
>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>> relocation transparent to userspace. It flushes the shadow fb into the
>> BO's memory if there are updates. The code is in
>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>> call now pins the BO to wherever it is. The actual location does not
>> matter. It's vunmap'ed immediately afterwards.
> 
> The problem is what happens when it is prepared for scanout, but can't
> be moved to VRAM because it is vmapped?
> 
> When the shadow is never scanned out that isn't a problem, but we need
> to keep that in mind.

If this is a problem is practice, it has never shown up with the drivers
that use it already.

I think here's a modeset lock somewhere that could serialize these
operations. The fbdev console is not double buffered, so there's no
frequent pageflipping; hence interference should be small.

Best regards
Thomas

> 
> Regards,
> Christian.
> 
>>
>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>> This should always move the BO into GTT-managed memory.
>>
>> Best regards
>> Thomas
>>
>> [1]
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0
>>
>>
>>> Regards,
>>> Christian.
>>>
>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>> ---
>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51
>>>> +++++++++++++++++++++++++++--
>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>> radeon_device *rdev, int r)
>>>>        return r;
>>>>    }
>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>> +
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return ret;
>>>> +
>>>> +    /* pin buffer at its current location */
>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unreserve;
>>>> +
>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unpin;
>>>> +
>>>> +    radeon_bo_unreserve(bo);
>>>> +
>>>> +    return 0;
>>>> +
>>>> +err_radeon_bo_unpin:
>>>> +    radeon_bo_unpin(bo);
>>>> +err_radeon_bo_unreserve:
>>>> +    radeon_bo_unreserve(bo);
>>>> +    return ret;
>>>> +}
>>>> +
>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return;
>>>> +
>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>> +    radeon_bo_unpin(bo);
>>>> +    radeon_bo_unreserve(bo);
>>>> +}
>>>> +
>>>>    static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>>>>        .free = radeon_gem_object_free,
>>>>        .open = radeon_gem_object_open,
>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>> radeon_gem_object_funcs = {
>>>>        .pin = radeon_gem_prime_pin,
>>>>        .unpin = radeon_gem_prime_unpin,
>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>> -    .vmap = drm_gem_ttm_vmap,
>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>> +    .vmap = radeon_gem_object_vmap,
>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>    };
>>>>      /*
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0
>>>
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

Hi

Am 16.11.20 um 12:28 schrieb Christian König:
> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>> Hi Christian
>>
>> Am 12.11.20 um 18:16 schrieb Christian König:
>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>>> object's vmap implementation. Unpin them in the vunmap implementation.
>>>> This is needed to make generic fbdev support work reliably. Without,
>>>> the buffer object could be evicted while fbdev flushed its shadow
>>>> buffer.
>>>>
>>>> In difference to the PRIME pin/unpin functions, the vmap code does not
>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not
>>>> count as
>>>> shared.
>>>>
>>>> The actual pin location is not important as the vmap call returns
>>>> information on how to access the buffer. Callers that require a
>>>> specific location should explicitly pin the BO before vmapping it.
>>> Well is the buffer supposed to be scanned out?
>> No, not by the fbdev helper.
> 
> Ok in this case that should work.
> 
>>> If yes then the pin location is actually rather important since the
>>> hardware can only scan out from VRAM.
>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>> relocation transparent to userspace. It flushes the shadow fb into the
>> BO's memory if there are updates. The code is in
>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>> call now pins the BO to wherever it is. The actual location does not
>> matter. It's vunmap'ed immediately afterwards.
> 
> The problem is what happens when it is prepared for scanout, but can't
> be moved to VRAM because it is vmapped?
> 
> When the shadow is never scanned out that isn't a problem, but we need
> to keep that in mind.

If this is a problem is practice, it has never shown up with the drivers
that use it already.

I think here's a modeset lock somewhere that could serialize these
operations. The fbdev console is not double buffered, so there's no
frequent pageflipping; hence interference should be small.

Best regards
Thomas

> 
> Regards,
> Christian.
> 
>>
>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>> This should always move the BO into GTT-managed memory.
>>
>> Best regards
>> Thomas
>>
>> [1]
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0
>>
>>
>>> Regards,
>>> Christian.
>>>
>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>> ---
>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51
>>>> +++++++++++++++++++++++++++--
>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>> radeon_device *rdev, int r)
>>>>        return r;
>>>>    }
>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>> +
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return ret;
>>>> +
>>>> +    /* pin buffer at its current location */
>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unreserve;
>>>> +
>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unpin;
>>>> +
>>>> +    radeon_bo_unreserve(bo);
>>>> +
>>>> +    return 0;
>>>> +
>>>> +err_radeon_bo_unpin:
>>>> +    radeon_bo_unpin(bo);
>>>> +err_radeon_bo_unreserve:
>>>> +    radeon_bo_unreserve(bo);
>>>> +    return ret;
>>>> +}
>>>> +
>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return;
>>>> +
>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>> +    radeon_bo_unpin(bo);
>>>> +    radeon_bo_unreserve(bo);
>>>> +}
>>>> +
>>>>    static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>>>>        .free = radeon_gem_object_free,
>>>>        .open = radeon_gem_object_open,
>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>> radeon_gem_object_funcs = {
>>>>        .pin = radeon_gem_prime_pin,
>>>>        .unpin = radeon_gem_prime_unpin,
>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>> -    .vmap = drm_gem_ttm_vmap,
>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>> +    .vmap = radeon_gem_object_vmap,
>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>    };
>>>>      /*
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0
>>>
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
> Hi Christian
>
> Am 12.11.20 um 18:16 schrieb Christian König:
>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>> object's vmap implementation. Unpin them in the vunmap implementation.
>>> This is needed to make generic fbdev support work reliably. Without,
>>> the buffer object could be evicted while fbdev flushed its shadow buffer.
>>>
>>> In difference to the PRIME pin/unpin functions, the vmap code does not
>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not count as
>>> shared.
>>>
>>> The actual pin location is not important as the vmap call returns
>>> information on how to access the buffer. Callers that require a
>>> specific location should explicitly pin the BO before vmapping it.
>> Well is the buffer supposed to be scanned out?
> No, not by the fbdev helper.

Ok in this case that should work.

>> If yes then the pin location is actually rather important since the
>> hardware can only scan out from VRAM.
> For relocatable BOs, fbdev uses a shadow buffer that makes all any
> relocation transparent to userspace. It flushes the shadow fb into the
> BO's memory if there are updates. The code is in
> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
> call now pins the BO to wherever it is. The actual location does not
> matter. It's vunmap'ed immediately afterwards.

The problem is what happens when it is prepared for scanout, but can't 
be moved to VRAM because it is vmapped?

When the shadow is never scanned out that isn't a problem, but we need 
to keep that in mind.

Regards,
Christian.

>
> For dma-buf sharing, the regular procedure of pin + vmap still apply.
> This should always move the BO into GTT-managed memory.
>
> Best regards
> Thomas
>
> [1]
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0
>
>> Regards,
>> Christian.
>>
>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>> ---
>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>> radeon_device *rdev, int r)
>>>        return r;
>>>    }
>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>> struct dma_buf_map *map)
>>> +{
>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>> +                       RADEON_GEM_DOMAIN_GTT |
>>> +                       RADEON_GEM_DOMAIN_CPU;
>>> +
>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>> +    int ret;
>>> +
>>> +    ret = radeon_bo_reserve(bo, false);
>>> +    if (ret)
>>> +        return ret;
>>> +
>>> +    /* pin buffer at its current location */
>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>> +    if (ret)
>>> +        goto err_radeon_bo_unreserve;
>>> +
>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>> +    if (ret)
>>> +        goto err_radeon_bo_unpin;
>>> +
>>> +    radeon_bo_unreserve(bo);
>>> +
>>> +    return 0;
>>> +
>>> +err_radeon_bo_unpin:
>>> +    radeon_bo_unpin(bo);
>>> +err_radeon_bo_unreserve:
>>> +    radeon_bo_unreserve(bo);
>>> +    return ret;
>>> +}
>>> +
>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>> struct dma_buf_map *map)
>>> +{
>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>> +    int ret;
>>> +
>>> +    ret = radeon_bo_reserve(bo, false);
>>> +    if (ret)
>>> +        return;
>>> +
>>> +    drm_gem_ttm_vunmap(obj, map);
>>> +    radeon_bo_unpin(bo);
>>> +    radeon_bo_unreserve(bo);
>>> +}
>>> +
>>>    static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>>>        .free = radeon_gem_object_free,
>>>        .open = radeon_gem_object_open,
>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>> radeon_gem_object_funcs = {
>>>        .pin = radeon_gem_prime_pin,
>>>        .unpin = radeon_gem_prime_unpin,
>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>> -    .vmap = drm_gem_ttm_vmap,
>>> -    .vunmap = drm_gem_ttm_vunmap,
>>> +    .vmap = radeon_gem_object_vmap,
>>> +    .vunmap = radeon_gem_object_vunmap,
>>>    };
>>>      /*
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
> Hi Christian
>
> Am 12.11.20 um 18:16 schrieb Christian König:
>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>> object's vmap implementation. Unpin them in the vunmap implementation.
>>> This is needed to make generic fbdev support work reliably. Without,
>>> the buffer object could be evicted while fbdev flushed its shadow buffer.
>>>
>>> In difference to the PRIME pin/unpin functions, the vmap code does not
>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not count as
>>> shared.
>>>
>>> The actual pin location is not important as the vmap call returns
>>> information on how to access the buffer. Callers that require a
>>> specific location should explicitly pin the BO before vmapping it.
>> Well is the buffer supposed to be scanned out?
> No, not by the fbdev helper.

Ok in this case that should work.

>> If yes then the pin location is actually rather important since the
>> hardware can only scan out from VRAM.
> For relocatable BOs, fbdev uses a shadow buffer that makes all any
> relocation transparent to userspace. It flushes the shadow fb into the
> BO's memory if there are updates. The code is in
> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
> call now pins the BO to wherever it is. The actual location does not
> matter. It's vunmap'ed immediately afterwards.

The problem is what happens when it is prepared for scanout, but can't 
be moved to VRAM because it is vmapped?

When the shadow is never scanned out that isn't a problem, but we need 
to keep that in mind.

Regards,
Christian.

>
> For dma-buf sharing, the regular procedure of pin + vmap still apply.
> This should always move the BO into GTT-managed memory.
>
> Best regards
> Thomas
>
> [1]
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0
>
>> Regards,
>> Christian.
>>
>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>> ---
>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>> radeon_device *rdev, int r)
>>>        return r;
>>>    }
>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>> struct dma_buf_map *map)
>>> +{
>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>> +                       RADEON_GEM_DOMAIN_GTT |
>>> +                       RADEON_GEM_DOMAIN_CPU;
>>> +
>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>> +    int ret;
>>> +
>>> +    ret = radeon_bo_reserve(bo, false);
>>> +    if (ret)
>>> +        return ret;
>>> +
>>> +    /* pin buffer at its current location */
>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>> +    if (ret)
>>> +        goto err_radeon_bo_unreserve;
>>> +
>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>> +    if (ret)
>>> +        goto err_radeon_bo_unpin;
>>> +
>>> +    radeon_bo_unreserve(bo);
>>> +
>>> +    return 0;
>>> +
>>> +err_radeon_bo_unpin:
>>> +    radeon_bo_unpin(bo);
>>> +err_radeon_bo_unreserve:
>>> +    radeon_bo_unreserve(bo);
>>> +    return ret;
>>> +}
>>> +
>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>> struct dma_buf_map *map)
>>> +{
>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>> +    int ret;
>>> +
>>> +    ret = radeon_bo_reserve(bo, false);
>>> +    if (ret)
>>> +        return;
>>> +
>>> +    drm_gem_ttm_vunmap(obj, map);
>>> +    radeon_bo_unpin(bo);
>>> +    radeon_bo_unreserve(bo);
>>> +}
>>> +
>>>    static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>>>        .free = radeon_gem_object_free,
>>>        .open = radeon_gem_object_open,
>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>> radeon_gem_object_funcs = {
>>>        .pin = radeon_gem_prime_pin,
>>>        .unpin = radeon_gem_prime_unpin,
>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>> -    .vmap = drm_gem_ttm_vmap,
>>> -    .vunmap = drm_gem_ttm_vunmap,
>>> +    .vmap = radeon_gem_object_vmap,
>>> +    .vunmap = radeon_gem_object_vunmap,
>>>    };
>>>      /*
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

Hi

Am 16.11.20 um 12:28 schrieb Christian König:
> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>> Hi Christian
>>
>> Am 12.11.20 um 18:16 schrieb Christian König:
>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>>> object's vmap implementation. Unpin them in the vunmap implementation.
>>>> This is needed to make generic fbdev support work reliably. Without,
>>>> the buffer object could be evicted while fbdev flushed its shadow
>>>> buffer.
>>>>
>>>> In difference to the PRIME pin/unpin functions, the vmap code does not
>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not
>>>> count as
>>>> shared.
>>>>
>>>> The actual pin location is not important as the vmap call returns
>>>> information on how to access the buffer. Callers that require a
>>>> specific location should explicitly pin the BO before vmapping it.
>>> Well is the buffer supposed to be scanned out?
>> No, not by the fbdev helper.
> 
> Ok in this case that should work.
> 
>>> If yes then the pin location is actually rather important since the
>>> hardware can only scan out from VRAM.
>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>> relocation transparent to userspace. It flushes the shadow fb into the
>> BO's memory if there are updates. The code is in
>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>> call now pins the BO to wherever it is. The actual location does not
>> matter. It's vunmap'ed immediately afterwards.
> 
> The problem is what happens when it is prepared for scanout, but can't
> be moved to VRAM because it is vmapped?
> 
> When the shadow is never scanned out that isn't a problem, but we need
> to keep that in mind.

I sent out a patchset that addresses the issue in it's final patch. [1]
I'd appreciate your feedback. It also tested the patches with the
converted radeon driver.

Best regards
Thomas

[1] https://patchwork.freedesktop.org/series/83918/

> 
> Regards,
> Christian.
> 
>>
>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>> This should always move the BO into GTT-managed memory.
>>
>> Best regards
>> Thomas
>>
>> [1]
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0
>>
>>
>>> Regards,
>>> Christian.
>>>
>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>> ---
>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51
>>>> +++++++++++++++++++++++++++--
>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>> radeon_device *rdev, int r)
>>>>        return r;
>>>>    }
>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>> +
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return ret;
>>>> +
>>>> +    /* pin buffer at its current location */
>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unreserve;
>>>> +
>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unpin;
>>>> +
>>>> +    radeon_bo_unreserve(bo);
>>>> +
>>>> +    return 0;
>>>> +
>>>> +err_radeon_bo_unpin:
>>>> +    radeon_bo_unpin(bo);
>>>> +err_radeon_bo_unreserve:
>>>> +    radeon_bo_unreserve(bo);
>>>> +    return ret;
>>>> +}
>>>> +
>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return;
>>>> +
>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>> +    radeon_bo_unpin(bo);
>>>> +    radeon_bo_unreserve(bo);
>>>> +}
>>>> +
>>>>    static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>>>>        .free = radeon_gem_object_free,
>>>>        .open = radeon_gem_object_open,
>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>> radeon_gem_object_funcs = {
>>>>        .pin = radeon_gem_prime_pin,
>>>>        .unpin = radeon_gem_prime_unpin,
>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>> -    .vmap = drm_gem_ttm_vmap,
>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>> +    .vmap = radeon_gem_object_vmap,
>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>    };
>>>>      /*
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0
>>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

Hi

Am 16.11.20 um 12:28 schrieb Christian König:
> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>> Hi Christian
>>
>> Am 12.11.20 um 18:16 schrieb Christian König:
>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>>> object's vmap implementation. Unpin them in the vunmap implementation.
>>>> This is needed to make generic fbdev support work reliably. Without,
>>>> the buffer object could be evicted while fbdev flushed its shadow
>>>> buffer.
>>>>
>>>> In difference to the PRIME pin/unpin functions, the vmap code does not
>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not
>>>> count as
>>>> shared.
>>>>
>>>> The actual pin location is not important as the vmap call returns
>>>> information on how to access the buffer. Callers that require a
>>>> specific location should explicitly pin the BO before vmapping it.
>>> Well is the buffer supposed to be scanned out?
>> No, not by the fbdev helper.
> 
> Ok in this case that should work.
> 
>>> If yes then the pin location is actually rather important since the
>>> hardware can only scan out from VRAM.
>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>> relocation transparent to userspace. It flushes the shadow fb into the
>> BO's memory if there are updates. The code is in
>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>> call now pins the BO to wherever it is. The actual location does not
>> matter. It's vunmap'ed immediately afterwards.
> 
> The problem is what happens when it is prepared for scanout, but can't
> be moved to VRAM because it is vmapped?
> 
> When the shadow is never scanned out that isn't a problem, but we need
> to keep that in mind.

I sent out a patchset that addresses the issue in it's final patch. [1]
I'd appreciate your feedback. It also tested the patches with the
converted radeon driver.

Best regards
Thomas

[1] https://patchwork.freedesktop.org/series/83918/

> 
> Regards,
> Christian.
> 
>>
>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>> This should always move the BO into GTT-managed memory.
>>
>> Best regards
>> Thomas
>>
>> [1]
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0
>>
>>
>>> Regards,
>>> Christian.
>>>
>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>> ---
>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51
>>>> +++++++++++++++++++++++++++--
>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>> radeon_device *rdev, int r)
>>>>        return r;
>>>>    }
>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>> +
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return ret;
>>>> +
>>>> +    /* pin buffer at its current location */
>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unreserve;
>>>> +
>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unpin;
>>>> +
>>>> +    radeon_bo_unreserve(bo);
>>>> +
>>>> +    return 0;
>>>> +
>>>> +err_radeon_bo_unpin:
>>>> +    radeon_bo_unpin(bo);
>>>> +err_radeon_bo_unreserve:
>>>> +    radeon_bo_unreserve(bo);
>>>> +    return ret;
>>>> +}
>>>> +
>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return;
>>>> +
>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>> +    radeon_bo_unpin(bo);
>>>> +    radeon_bo_unreserve(bo);
>>>> +}
>>>> +
>>>>    static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>>>>        .free = radeon_gem_object_free,
>>>>        .open = radeon_gem_object_open,
>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>> radeon_gem_object_funcs = {
>>>>        .pin = radeon_gem_prime_pin,
>>>>        .unpin = radeon_gem_prime_unpin,
>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>> -    .vmap = drm_gem_ttm_vmap,
>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>> +    .vmap = radeon_gem_object_vmap,
>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>    };
>>>>      /*
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0
>>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 7043 bytes --]

Hi Christian

Am 16.11.20 um 12:28 schrieb Christian König:
> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>> Hi Christian
>>
>> Am 12.11.20 um 18:16 schrieb Christian König:
>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>>> object's vmap implementation. Unpin them in the vunmap implementation.
>>>> This is needed to make generic fbdev support work reliably. Without,
>>>> the buffer object could be evicted while fbdev flushed its shadow 
>>>> buffer.
>>>>
>>>> In difference to the PRIME pin/unpin functions, the vmap code does not
>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not 
>>>> count as
>>>> shared.
>>>>
>>>> The actual pin location is not important as the vmap call returns
>>>> information on how to access the buffer. Callers that require a
>>>> specific location should explicitly pin the BO before vmapping it.
>>> Well is the buffer supposed to be scanned out?
>> No, not by the fbdev helper.
> 
> Ok in this case that should work.
> 
>>> If yes then the pin location is actually rather important since the
>>> hardware can only scan out from VRAM.
>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>> relocation transparent to userspace. It flushes the shadow fb into the
>> BO's memory if there are updates. The code is in
>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>> call now pins the BO to wherever it is. The actual location does not
>> matter. It's vunmap'ed immediately afterwards.
> 
> The problem is what happens when it is prepared for scanout, but can't 
> be moved to VRAM because it is vmapped?
> 
> When the shadow is never scanned out that isn't a problem, but we need 
> to keep that in mind.
> 

I'd like ask for your suggestions before sending an update for this patch.

After the discussion about locking in fbdev, [1] I intended to replace 
the pin call with code that acquires the reservation lock.

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then 
wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd 
expect that vmap/vunmap are close together and do not overlap for the 
same BO. Otherwise, acquiring the reservation lock would require another 
ref-counting variable or per-driver code.

Best regards
Thomas

[1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1

> Regards,
> Christian.
> 
>>
>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>> This should always move the BO into GTT-managed memory.
>>
>> Best regards
>> Thomas
>>
>> [1]
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0 
>>
>>
>>> Regards,
>>> Christian.
>>>
>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>> ---
>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 
>>>> +++++++++++++++++++++++++++--
>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>> radeon_device *rdev, int r)
>>>>        return r;
>>>>    }
>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>> +
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return ret;
>>>> +
>>>> +    /* pin buffer at its current location */
>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unreserve;
>>>> +
>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unpin;
>>>> +
>>>> +    radeon_bo_unreserve(bo);
>>>> +
>>>> +    return 0;
>>>> +
>>>> +err_radeon_bo_unpin:
>>>> +    radeon_bo_unpin(bo);
>>>> +err_radeon_bo_unreserve:
>>>> +    radeon_bo_unreserve(bo);
>>>> +    return ret;
>>>> +}
>>>> +
>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return;
>>>> +
>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>> +    radeon_bo_unpin(bo);
>>>> +    radeon_bo_unreserve(bo);
>>>> +}
>>>> +
>>>>    static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>>>>        .free = radeon_gem_object_free,
>>>>        .open = radeon_gem_object_open,
>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>> radeon_gem_object_funcs = {
>>>>        .pin = radeon_gem_prime_pin,
>>>>        .unpin = radeon_gem_prime_unpin,
>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>> -    .vmap = drm_gem_ttm_vmap,
>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>> +    .vmap = radeon_gem_object_vmap,
>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>    };
>>>>      /*
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0 
>>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 7043 bytes --]

Hi Christian

Am 16.11.20 um 12:28 schrieb Christian König:
> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>> Hi Christian
>>
>> Am 12.11.20 um 18:16 schrieb Christian König:
>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>>> object's vmap implementation. Unpin them in the vunmap implementation.
>>>> This is needed to make generic fbdev support work reliably. Without,
>>>> the buffer object could be evicted while fbdev flushed its shadow 
>>>> buffer.
>>>>
>>>> In difference to the PRIME pin/unpin functions, the vmap code does not
>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not 
>>>> count as
>>>> shared.
>>>>
>>>> The actual pin location is not important as the vmap call returns
>>>> information on how to access the buffer. Callers that require a
>>>> specific location should explicitly pin the BO before vmapping it.
>>> Well is the buffer supposed to be scanned out?
>> No, not by the fbdev helper.
> 
> Ok in this case that should work.
> 
>>> If yes then the pin location is actually rather important since the
>>> hardware can only scan out from VRAM.
>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>> relocation transparent to userspace. It flushes the shadow fb into the
>> BO's memory if there are updates. The code is in
>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>> call now pins the BO to wherever it is. The actual location does not
>> matter. It's vunmap'ed immediately afterwards.
> 
> The problem is what happens when it is prepared for scanout, but can't 
> be moved to VRAM because it is vmapped?
> 
> When the shadow is never scanned out that isn't a problem, but we need 
> to keep that in mind.
> 

I'd like ask for your suggestions before sending an update for this patch.

After the discussion about locking in fbdev, [1] I intended to replace 
the pin call with code that acquires the reservation lock.

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then 
wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd 
expect that vmap/vunmap are close together and do not overlap for the 
same BO. Otherwise, acquiring the reservation lock would require another 
ref-counting variable or per-driver code.

Best regards
Thomas

[1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1

> Regards,
> Christian.
> 
>>
>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>> This should always move the BO into GTT-managed memory.
>>
>> Best regards
>> Thomas
>>
>> [1]
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0 
>>
>>
>>> Regards,
>>> Christian.
>>>
>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>> ---
>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 
>>>> +++++++++++++++++++++++++++--
>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>> radeon_device *rdev, int r)
>>>>        return r;
>>>>    }
>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>> +
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return ret;
>>>> +
>>>> +    /* pin buffer at its current location */
>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unreserve;
>>>> +
>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>> +    if (ret)
>>>> +        goto err_radeon_bo_unpin;
>>>> +
>>>> +    radeon_bo_unreserve(bo);
>>>> +
>>>> +    return 0;
>>>> +
>>>> +err_radeon_bo_unpin:
>>>> +    radeon_bo_unpin(bo);
>>>> +err_radeon_bo_unreserve:
>>>> +    radeon_bo_unreserve(bo);
>>>> +    return ret;
>>>> +}
>>>> +
>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>> struct dma_buf_map *map)
>>>> +{
>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>> +    int ret;
>>>> +
>>>> +    ret = radeon_bo_reserve(bo, false);
>>>> +    if (ret)
>>>> +        return;
>>>> +
>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>> +    radeon_bo_unpin(bo);
>>>> +    radeon_bo_unreserve(bo);
>>>> +}
>>>> +
>>>>    static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
>>>>        .free = radeon_gem_object_free,
>>>>        .open = radeon_gem_object_open,
>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>> radeon_gem_object_funcs = {
>>>>        .pin = radeon_gem_prime_pin,
>>>>        .unpin = radeon_gem_prime_unpin,
>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>> -    .vmap = drm_gem_ttm_vmap,
>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>> +    .vmap = radeon_gem_object_vmap,
>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>    };
>>>>      /*
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0 
>>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:
> Hi Christian
>
> Am 16.11.20 um 12:28 schrieb Christian König:
>> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>>> Hi Christian
>>>
>>> Am 12.11.20 um 18:16 schrieb Christian König:
>>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>>>> object's vmap implementation. Unpin them in the vunmap 
>>>>> implementation.
>>>>> This is needed to make generic fbdev support work reliably. Without,
>>>>> the buffer object could be evicted while fbdev flushed its shadow 
>>>>> buffer.
>>>>>
>>>>> In difference to the PRIME pin/unpin functions, the vmap code does 
>>>>> not
>>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not 
>>>>> count as
>>>>> shared.
>>>>>
>>>>> The actual pin location is not important as the vmap call returns
>>>>> information on how to access the buffer. Callers that require a
>>>>> specific location should explicitly pin the BO before vmapping it.
>>>> Well is the buffer supposed to be scanned out?
>>> No, not by the fbdev helper.
>>
>> Ok in this case that should work.
>>
>>>> If yes then the pin location is actually rather important since the
>>>> hardware can only scan out from VRAM.
>>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>>> relocation transparent to userspace. It flushes the shadow fb into the
>>> BO's memory if there are updates. The code is in
>>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>>> call now pins the BO to wherever it is. The actual location does not
>>> matter. It's vunmap'ed immediately afterwards.
>>
>> The problem is what happens when it is prepared for scanout, but 
>> can't be moved to VRAM because it is vmapped?
>>
>> When the shadow is never scanned out that isn't a problem, but we 
>> need to keep that in mind.
>>
>
> I'd like ask for your suggestions before sending an update for this 
> patch.
>
> After the discussion about locking in fbdev, [1] I intended to replace 
> the pin call with code that acquires the reservation lock.

Yeah, that sounds like a good idea to me as well.

> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then 
> wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd 
> expect that vmap/vunmap are close together and do not overlap for the 
> same BO. 

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with 
additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the 
lock/unlock into it and I hope to replace the kmap/kunmap functions with 
them in the near term.

> Otherwise, acquiring the reservation lock would require another 
> ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as 
you initially planned.

Regards,
Christian.

>
> Best regards
> Thomas
>
> [1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1
>
>> Regards,
>> Christian.
>>
>>>
>>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>>> This should always move the BO into GTT-managed memory.
>>>
>>> Best regards
>>> Thomas
>>>
>>> [1]
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0 
>>>
>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>>> ---
>>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 
>>>>> +++++++++++++++++++++++++++--
>>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>>> radeon_device *rdev, int r)
>>>>>        return r;
>>>>>    }
>>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>>> struct dma_buf_map *map)
>>>>> +{
>>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>>> +
>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>> +    int ret;
>>>>> +
>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>> +    if (ret)
>>>>> +        return ret;
>>>>> +
>>>>> +    /* pin buffer at its current location */
>>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>>> +    if (ret)
>>>>> +        goto err_radeon_bo_unreserve;
>>>>> +
>>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>>> +    if (ret)
>>>>> +        goto err_radeon_bo_unpin;
>>>>> +
>>>>> +    radeon_bo_unreserve(bo);
>>>>> +
>>>>> +    return 0;
>>>>> +
>>>>> +err_radeon_bo_unpin:
>>>>> +    radeon_bo_unpin(bo);
>>>>> +err_radeon_bo_unreserve:
>>>>> +    radeon_bo_unreserve(bo);
>>>>> +    return ret;
>>>>> +}
>>>>> +
>>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>>> struct dma_buf_map *map)
>>>>> +{
>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>> +    int ret;
>>>>> +
>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>> +    if (ret)
>>>>> +        return;
>>>>> +
>>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>>> +    radeon_bo_unpin(bo);
>>>>> +    radeon_bo_unreserve(bo);
>>>>> +}
>>>>> +
>>>>>    static const struct drm_gem_object_funcs 
>>>>> radeon_gem_object_funcs = {
>>>>>        .free = radeon_gem_object_free,
>>>>>        .open = radeon_gem_object_open,
>>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>>> radeon_gem_object_funcs = {
>>>>>        .pin = radeon_gem_prime_pin,
>>>>>        .unpin = radeon_gem_prime_unpin,
>>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>>> -    .vmap = drm_gem_ttm_vmap,
>>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>>> +    .vmap = radeon_gem_object_vmap,
>>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>>    };
>>>>>      /*
>>>> _______________________________________________
>>>> dri-devel mailing list
>>>> dri-devel@lists.freedesktop.org
>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0 
>>>>
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:
> Hi Christian
>
> Am 16.11.20 um 12:28 schrieb Christian König:
>> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>>> Hi Christian
>>>
>>> Am 12.11.20 um 18:16 schrieb Christian König:
>>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>>>> object's vmap implementation. Unpin them in the vunmap 
>>>>> implementation.
>>>>> This is needed to make generic fbdev support work reliably. Without,
>>>>> the buffer object could be evicted while fbdev flushed its shadow 
>>>>> buffer.
>>>>>
>>>>> In difference to the PRIME pin/unpin functions, the vmap code does 
>>>>> not
>>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not 
>>>>> count as
>>>>> shared.
>>>>>
>>>>> The actual pin location is not important as the vmap call returns
>>>>> information on how to access the buffer. Callers that require a
>>>>> specific location should explicitly pin the BO before vmapping it.
>>>> Well is the buffer supposed to be scanned out?
>>> No, not by the fbdev helper.
>>
>> Ok in this case that should work.
>>
>>>> If yes then the pin location is actually rather important since the
>>>> hardware can only scan out from VRAM.
>>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>>> relocation transparent to userspace. It flushes the shadow fb into the
>>> BO's memory if there are updates. The code is in
>>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>>> call now pins the BO to wherever it is. The actual location does not
>>> matter. It's vunmap'ed immediately afterwards.
>>
>> The problem is what happens when it is prepared for scanout, but 
>> can't be moved to VRAM because it is vmapped?
>>
>> When the shadow is never scanned out that isn't a problem, but we 
>> need to keep that in mind.
>>
>
> I'd like ask for your suggestions before sending an update for this 
> patch.
>
> After the discussion about locking in fbdev, [1] I intended to replace 
> the pin call with code that acquires the reservation lock.

Yeah, that sounds like a good idea to me as well.

> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then 
> wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd 
> expect that vmap/vunmap are close together and do not overlap for the 
> same BO. 

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with 
additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the 
lock/unlock into it and I hope to replace the kmap/kunmap functions with 
them in the near term.

> Otherwise, acquiring the reservation lock would require another 
> ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as 
you initially planned.

Regards,
Christian.

>
> Best regards
> Thomas
>
> [1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1
>
>> Regards,
>> Christian.
>>
>>>
>>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>>> This should always move the BO into GTT-managed memory.
>>>
>>> Best regards
>>> Thomas
>>>
>>> [1]
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0 
>>>
>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>>> ---
>>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 
>>>>> +++++++++++++++++++++++++++--
>>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>>> radeon_device *rdev, int r)
>>>>>        return r;
>>>>>    }
>>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>>> struct dma_buf_map *map)
>>>>> +{
>>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>>> +
>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>> +    int ret;
>>>>> +
>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>> +    if (ret)
>>>>> +        return ret;
>>>>> +
>>>>> +    /* pin buffer at its current location */
>>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>>> +    if (ret)
>>>>> +        goto err_radeon_bo_unreserve;
>>>>> +
>>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>>> +    if (ret)
>>>>> +        goto err_radeon_bo_unpin;
>>>>> +
>>>>> +    radeon_bo_unreserve(bo);
>>>>> +
>>>>> +    return 0;
>>>>> +
>>>>> +err_radeon_bo_unpin:
>>>>> +    radeon_bo_unpin(bo);
>>>>> +err_radeon_bo_unreserve:
>>>>> +    radeon_bo_unreserve(bo);
>>>>> +    return ret;
>>>>> +}
>>>>> +
>>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>>> struct dma_buf_map *map)
>>>>> +{
>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>> +    int ret;
>>>>> +
>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>> +    if (ret)
>>>>> +        return;
>>>>> +
>>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>>> +    radeon_bo_unpin(bo);
>>>>> +    radeon_bo_unreserve(bo);
>>>>> +}
>>>>> +
>>>>>    static const struct drm_gem_object_funcs 
>>>>> radeon_gem_object_funcs = {
>>>>>        .free = radeon_gem_object_free,
>>>>>        .open = radeon_gem_object_open,
>>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>>> radeon_gem_object_funcs = {
>>>>>        .pin = radeon_gem_prime_pin,
>>>>>        .unpin = radeon_gem_prime_unpin,
>>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>>> -    .vmap = drm_gem_ttm_vmap,
>>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>>> +    .vmap = radeon_gem_object_vmap,
>>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>>    };
>>>>>      /*
>>>> _______________________________________________
>>>> dri-devel mailing list
>>>> dri-devel@lists.freedesktop.org
>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0 
>>>>
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 8442 bytes --]

Hi

Am 24.11.20 um 12:30 schrieb Christian König:
> Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:
>> Hi Christian
>>
>> Am 16.11.20 um 12:28 schrieb Christian König:
>>> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>>>> Hi Christian
>>>>
>>>> Am 12.11.20 um 18:16 schrieb Christian König:
>>>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>>>>> object's vmap implementation. Unpin them in the vunmap 
>>>>>> implementation.
>>>>>> This is needed to make generic fbdev support work reliably. Without,
>>>>>> the buffer object could be evicted while fbdev flushed its shadow 
>>>>>> buffer.
>>>>>>
>>>>>> In difference to the PRIME pin/unpin functions, the vmap code does 
>>>>>> not
>>>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not 
>>>>>> count as
>>>>>> shared.
>>>>>>
>>>>>> The actual pin location is not important as the vmap call returns
>>>>>> information on how to access the buffer. Callers that require a
>>>>>> specific location should explicitly pin the BO before vmapping it.
>>>>> Well is the buffer supposed to be scanned out?
>>>> No, not by the fbdev helper.
>>>
>>> Ok in this case that should work.
>>>
>>>>> If yes then the pin location is actually rather important since the
>>>>> hardware can only scan out from VRAM.
>>>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>>>> relocation transparent to userspace. It flushes the shadow fb into the
>>>> BO's memory if there are updates. The code is in
>>>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>>>> call now pins the BO to wherever it is. The actual location does not
>>>> matter. It's vunmap'ed immediately afterwards.
>>>
>>> The problem is what happens when it is prepared for scanout, but 
>>> can't be moved to VRAM because it is vmapped?
>>>
>>> When the shadow is never scanned out that isn't a problem, but we 
>>> need to keep that in mind.
>>>
>>
>> I'd like ask for your suggestions before sending an update for this 
>> patch.
>>
>> After the discussion about locking in fbdev, [1] I intended to replace 
>> the pin call with code that acquires the reservation lock.
> 
> Yeah, that sounds like a good idea to me as well.
> 
>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then 
>> wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd 
>> expect that vmap/vunmap are close together and do not overlap for the 
>> same BO. 
> 
> We have use cases like the following during command submission:
> 
> 1. lock
> 2. map
> 3. copy parts of the BO content somewhere else or patch it with 
> additional information
> 4. unmap
> 5. submit BO to the hardware
> 6. add hardware fence to the BO to make sure it doesn't move
> 7. unlock
> 
> That use case won't be possible with vmap/vunmap if we move the 
> lock/unlock into it and I hope to replace the kmap/kunmap functions with 
> them in the near term.
> 
>> Otherwise, acquiring the reservation lock would require another 
>> ref-counting variable or per-driver code.
> 
> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as 
> you initially planned.

Given your example above, step one would acquire the lock, and step two 
would also acquire the lock as part of the vmap implementation. Wouldn't 
this fail (At least during unmap or unlock steps) ?

Best regards
Thomas

> 
> Regards,
> Christian.
> 
>>
>> Best regards
>> Thomas
>>
>> [1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1
>>
>>> Regards,
>>> Christian.
>>>
>>>>
>>>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>>>> This should always move the BO into GTT-managed memory.
>>>>
>>>> Best regards
>>>> Thomas
>>>>
>>>> [1]
>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0 
>>>>
>>>>
>>>>> Regards,
>>>>> Christian.
>>>>>
>>>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>>>> ---
>>>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 
>>>>>> +++++++++++++++++++++++++++--
>>>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>>>> radeon_device *rdev, int r)
>>>>>>        return r;
>>>>>>    }
>>>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>>>> struct dma_buf_map *map)
>>>>>> +{
>>>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>>>> +
>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>> +    int ret;
>>>>>> +
>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>> +    if (ret)
>>>>>> +        return ret;
>>>>>> +
>>>>>> +    /* pin buffer at its current location */
>>>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>>>> +    if (ret)
>>>>>> +        goto err_radeon_bo_unreserve;
>>>>>> +
>>>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>>>> +    if (ret)
>>>>>> +        goto err_radeon_bo_unpin;
>>>>>> +
>>>>>> +    radeon_bo_unreserve(bo);
>>>>>> +
>>>>>> +    return 0;
>>>>>> +
>>>>>> +err_radeon_bo_unpin:
>>>>>> +    radeon_bo_unpin(bo);
>>>>>> +err_radeon_bo_unreserve:
>>>>>> +    radeon_bo_unreserve(bo);
>>>>>> +    return ret;
>>>>>> +}
>>>>>> +
>>>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>>>> struct dma_buf_map *map)
>>>>>> +{
>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>> +    int ret;
>>>>>> +
>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>> +    if (ret)
>>>>>> +        return;
>>>>>> +
>>>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>>>> +    radeon_bo_unpin(bo);
>>>>>> +    radeon_bo_unreserve(bo);
>>>>>> +}
>>>>>> +
>>>>>>    static const struct drm_gem_object_funcs 
>>>>>> radeon_gem_object_funcs = {
>>>>>>        .free = radeon_gem_object_free,
>>>>>>        .open = radeon_gem_object_open,
>>>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>>>> radeon_gem_object_funcs = {
>>>>>>        .pin = radeon_gem_prime_pin,
>>>>>>        .unpin = radeon_gem_prime_unpin,
>>>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>>>> -    .vmap = drm_gem_ttm_vmap,
>>>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>>>> +    .vmap = radeon_gem_object_vmap,
>>>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>>>    };
>>>>>>      /*
>>>>> _______________________________________________
>>>>> dri-devel mailing list
>>>>> dri-devel@lists.freedesktop.org
>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0 
>>>>>
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 8442 bytes --]

Hi

Am 24.11.20 um 12:30 schrieb Christian König:
> Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:
>> Hi Christian
>>
>> Am 16.11.20 um 12:28 schrieb Christian König:
>>> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>>>> Hi Christian
>>>>
>>>> Am 12.11.20 um 18:16 schrieb Christian König:
>>>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>>>> In order to avoid eviction of vmap'ed buffers, pin them in their GEM
>>>>>> object's vmap implementation. Unpin them in the vunmap 
>>>>>> implementation.
>>>>>> This is needed to make generic fbdev support work reliably. Without,
>>>>>> the buffer object could be evicted while fbdev flushed its shadow 
>>>>>> buffer.
>>>>>>
>>>>>> In difference to the PRIME pin/unpin functions, the vmap code does 
>>>>>> not
>>>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not 
>>>>>> count as
>>>>>> shared.
>>>>>>
>>>>>> The actual pin location is not important as the vmap call returns
>>>>>> information on how to access the buffer. Callers that require a
>>>>>> specific location should explicitly pin the BO before vmapping it.
>>>>> Well is the buffer supposed to be scanned out?
>>>> No, not by the fbdev helper.
>>>
>>> Ok in this case that should work.
>>>
>>>>> If yes then the pin location is actually rather important since the
>>>>> hardware can only scan out from VRAM.
>>>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>>>> relocation transparent to userspace. It flushes the shadow fb into the
>>>> BO's memory if there are updates. The code is in
>>>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>>>> call now pins the BO to wherever it is. The actual location does not
>>>> matter. It's vunmap'ed immediately afterwards.
>>>
>>> The problem is what happens when it is prepared for scanout, but 
>>> can't be moved to VRAM because it is vmapped?
>>>
>>> When the shadow is never scanned out that isn't a problem, but we 
>>> need to keep that in mind.
>>>
>>
>> I'd like ask for your suggestions before sending an update for this 
>> patch.
>>
>> After the discussion about locking in fbdev, [1] I intended to replace 
>> the pin call with code that acquires the reservation lock.
> 
> Yeah, that sounds like a good idea to me as well.
> 
>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then 
>> wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd 
>> expect that vmap/vunmap are close together and do not overlap for the 
>> same BO. 
> 
> We have use cases like the following during command submission:
> 
> 1. lock
> 2. map
> 3. copy parts of the BO content somewhere else or patch it with 
> additional information
> 4. unmap
> 5. submit BO to the hardware
> 6. add hardware fence to the BO to make sure it doesn't move
> 7. unlock
> 
> That use case won't be possible with vmap/vunmap if we move the 
> lock/unlock into it and I hope to replace the kmap/kunmap functions with 
> them in the near term.
> 
>> Otherwise, acquiring the reservation lock would require another 
>> ref-counting variable or per-driver code.
> 
> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as 
> you initially planned.

Given your example above, step one would acquire the lock, and step two 
would also acquire the lock as part of the vmap implementation. Wouldn't 
this fail (At least during unmap or unlock steps) ?

Best regards
Thomas

> 
> Regards,
> Christian.
> 
>>
>> Best regards
>> Thomas
>>
>> [1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1
>>
>>> Regards,
>>> Christian.
>>>
>>>>
>>>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>>>> This should always move the BO into GTT-managed memory.
>>>>
>>>> Best regards
>>>> Thomas
>>>>
>>>> [1]
>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0 
>>>>
>>>>
>>>>> Regards,
>>>>> Christian.
>>>>>
>>>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>>>> ---
>>>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 
>>>>>> +++++++++++++++++++++++++++--
>>>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>>>> radeon_device *rdev, int r)
>>>>>>        return r;
>>>>>>    }
>>>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>>>> struct dma_buf_map *map)
>>>>>> +{
>>>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>>>> +
>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>> +    int ret;
>>>>>> +
>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>> +    if (ret)
>>>>>> +        return ret;
>>>>>> +
>>>>>> +    /* pin buffer at its current location */
>>>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>>>> +    if (ret)
>>>>>> +        goto err_radeon_bo_unreserve;
>>>>>> +
>>>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>>>> +    if (ret)
>>>>>> +        goto err_radeon_bo_unpin;
>>>>>> +
>>>>>> +    radeon_bo_unreserve(bo);
>>>>>> +
>>>>>> +    return 0;
>>>>>> +
>>>>>> +err_radeon_bo_unpin:
>>>>>> +    radeon_bo_unpin(bo);
>>>>>> +err_radeon_bo_unreserve:
>>>>>> +    radeon_bo_unreserve(bo);
>>>>>> +    return ret;
>>>>>> +}
>>>>>> +
>>>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>>>> struct dma_buf_map *map)
>>>>>> +{
>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>> +    int ret;
>>>>>> +
>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>> +    if (ret)
>>>>>> +        return;
>>>>>> +
>>>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>>>> +    radeon_bo_unpin(bo);
>>>>>> +    radeon_bo_unreserve(bo);
>>>>>> +}
>>>>>> +
>>>>>>    static const struct drm_gem_object_funcs 
>>>>>> radeon_gem_object_funcs = {
>>>>>>        .free = radeon_gem_object_free,
>>>>>>        .open = radeon_gem_object_open,
>>>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>>>> radeon_gem_object_funcs = {
>>>>>>        .pin = radeon_gem_prime_pin,
>>>>>>        .unpin = radeon_gem_prime_unpin,
>>>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>>>> -    .vmap = drm_gem_ttm_vmap,
>>>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>>>> +    .vmap = radeon_gem_object_vmap,
>>>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>>>    };
>>>>>>      /*
>>>>> _______________________________________________
>>>>> dri-devel mailing list
>>>>> dri-devel@lists.freedesktop.org
>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0 
>>>>>
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 24.11.20 um 12:44 schrieb Thomas Zimmermann:
> Hi
>
> Am 24.11.20 um 12:30 schrieb Christian König:
>> Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:
>>> Hi Christian
>>>
>>> Am 16.11.20 um 12:28 schrieb Christian König:
>>>> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>>>>> Hi Christian
>>>>>
>>>>> Am 12.11.20 um 18:16 schrieb Christian König:
>>>>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>>>>> In order to avoid eviction of vmap'ed buffers, pin them in their 
>>>>>>> GEM
>>>>>>> object's vmap implementation. Unpin them in the vunmap 
>>>>>>> implementation.
>>>>>>> This is needed to make generic fbdev support work reliably. 
>>>>>>> Without,
>>>>>>> the buffer object could be evicted while fbdev flushed its 
>>>>>>> shadow buffer.
>>>>>>>
>>>>>>> In difference to the PRIME pin/unpin functions, the vmap code 
>>>>>>> does not
>>>>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not 
>>>>>>> count as
>>>>>>> shared.
>>>>>>>
>>>>>>> The actual pin location is not important as the vmap call returns
>>>>>>> information on how to access the buffer. Callers that require a
>>>>>>> specific location should explicitly pin the BO before vmapping it.
>>>>>> Well is the buffer supposed to be scanned out?
>>>>> No, not by the fbdev helper.
>>>>
>>>> Ok in this case that should work.
>>>>
>>>>>> If yes then the pin location is actually rather important since the
>>>>>> hardware can only scan out from VRAM.
>>>>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>>>>> relocation transparent to userspace. It flushes the shadow fb into 
>>>>> the
>>>>> BO's memory if there are updates. The code is in
>>>>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>>>>> call now pins the BO to wherever it is. The actual location does not
>>>>> matter. It's vunmap'ed immediately afterwards.
>>>>
>>>> The problem is what happens when it is prepared for scanout, but 
>>>> can't be moved to VRAM because it is vmapped?
>>>>
>>>> When the shadow is never scanned out that isn't a problem, but we 
>>>> need to keep that in mind.
>>>>
>>>
>>> I'd like ask for your suggestions before sending an update for this 
>>> patch.
>>>
>>> After the discussion about locking in fbdev, [1] I intended to 
>>> replace the pin call with code that acquires the reservation lock.
>>
>> Yeah, that sounds like a good idea to me as well.
>>
>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then 
>>> wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd 
>>> expect that vmap/vunmap are close together and do not overlap for 
>>> the same BO. 
>>
>> We have use cases like the following during command submission:
>>
>> 1. lock
>> 2. map
>> 3. copy parts of the BO content somewhere else or patch it with 
>> additional information
>> 4. unmap
>> 5. submit BO to the hardware
>> 6. add hardware fence to the BO to make sure it doesn't move
>> 7. unlock
>>
>> That use case won't be possible with vmap/vunmap if we move the 
>> lock/unlock into it and I hope to replace the kmap/kunmap functions 
>> with them in the near term.
>>
>>> Otherwise, acquiring the reservation lock would require another 
>>> ref-counting variable or per-driver code.
>>
>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as 
>> you initially planned.
>
> Given your example above, step one would acquire the lock, and step 
> two would also acquire the lock as part of the vmap implementation. 
> Wouldn't this fail (At least during unmap or unlock steps) ?

Oh, so you want to nest them? No, that is a rather bad no-go.

You need to make sure that the lock is only taken from the FB path which 
wants to vmap the object.

Why don't you lock the GEM object from the caller in the generic FB 
implementation?

Regards,
Christian.

>
> Best regards
> Thomas
>
>>
>> Regards,
>> Christian.
>>
>>>
>>> Best regards
>>> Thomas
>>>
>>> [1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1
>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>>
>>>>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>>>>> This should always move the BO into GTT-managed memory.
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>> [1]
>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0 
>>>>>
>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>>>>> ---
>>>>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 
>>>>>>> +++++++++++++++++++++++++++--
>>>>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>>>>> radeon_device *rdev, int r)
>>>>>>>        return r;
>>>>>>>    }
>>>>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>>>>> struct dma_buf_map *map)
>>>>>>> +{
>>>>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>>>>> +
>>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>>> +    int ret;
>>>>>>> +
>>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>>> +    if (ret)
>>>>>>> +        return ret;
>>>>>>> +
>>>>>>> +    /* pin buffer at its current location */
>>>>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>>>>> +    if (ret)
>>>>>>> +        goto err_radeon_bo_unreserve;
>>>>>>> +
>>>>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>>>>> +    if (ret)
>>>>>>> +        goto err_radeon_bo_unpin;
>>>>>>> +
>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>> +
>>>>>>> +    return 0;
>>>>>>> +
>>>>>>> +err_radeon_bo_unpin:
>>>>>>> +    radeon_bo_unpin(bo);
>>>>>>> +err_radeon_bo_unreserve:
>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>> +    return ret;
>>>>>>> +}
>>>>>>> +
>>>>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>>>>> struct dma_buf_map *map)
>>>>>>> +{
>>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>>> +    int ret;
>>>>>>> +
>>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>>> +    if (ret)
>>>>>>> +        return;
>>>>>>> +
>>>>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>>>>> +    radeon_bo_unpin(bo);
>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>> +}
>>>>>>> +
>>>>>>>    static const struct drm_gem_object_funcs 
>>>>>>> radeon_gem_object_funcs = {
>>>>>>>        .free = radeon_gem_object_free,
>>>>>>>        .open = radeon_gem_object_open,
>>>>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>>>>> radeon_gem_object_funcs = {
>>>>>>>        .pin = radeon_gem_prime_pin,
>>>>>>>        .unpin = radeon_gem_prime_unpin,
>>>>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>>>>> -    .vmap = drm_gem_ttm_vmap,
>>>>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>>>>> +    .vmap = radeon_gem_object_vmap,
>>>>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>>>>    };
>>>>>>>      /*
>>>>>> _______________________________________________
>>>>>> dri-devel mailing list
>>>>>> dri-devel@lists.freedesktop.org
>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0 
>>>>>>
>>>>
>>>> _______________________________________________
>>>> dri-devel mailing list
>>>> dri-devel@lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>
>>
>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 24.11.20 um 12:44 schrieb Thomas Zimmermann:
> Hi
>
> Am 24.11.20 um 12:30 schrieb Christian König:
>> Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:
>>> Hi Christian
>>>
>>> Am 16.11.20 um 12:28 schrieb Christian König:
>>>> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>>>>> Hi Christian
>>>>>
>>>>> Am 12.11.20 um 18:16 schrieb Christian König:
>>>>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>>>>> In order to avoid eviction of vmap'ed buffers, pin them in their 
>>>>>>> GEM
>>>>>>> object's vmap implementation. Unpin them in the vunmap 
>>>>>>> implementation.
>>>>>>> This is needed to make generic fbdev support work reliably. 
>>>>>>> Without,
>>>>>>> the buffer object could be evicted while fbdev flushed its 
>>>>>>> shadow buffer.
>>>>>>>
>>>>>>> In difference to the PRIME pin/unpin functions, the vmap code 
>>>>>>> does not
>>>>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not 
>>>>>>> count as
>>>>>>> shared.
>>>>>>>
>>>>>>> The actual pin location is not important as the vmap call returns
>>>>>>> information on how to access the buffer. Callers that require a
>>>>>>> specific location should explicitly pin the BO before vmapping it.
>>>>>> Well is the buffer supposed to be scanned out?
>>>>> No, not by the fbdev helper.
>>>>
>>>> Ok in this case that should work.
>>>>
>>>>>> If yes then the pin location is actually rather important since the
>>>>>> hardware can only scan out from VRAM.
>>>>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>>>>> relocation transparent to userspace. It flushes the shadow fb into 
>>>>> the
>>>>> BO's memory if there are updates. The code is in
>>>>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>>>>> call now pins the BO to wherever it is. The actual location does not
>>>>> matter. It's vunmap'ed immediately afterwards.
>>>>
>>>> The problem is what happens when it is prepared for scanout, but 
>>>> can't be moved to VRAM because it is vmapped?
>>>>
>>>> When the shadow is never scanned out that isn't a problem, but we 
>>>> need to keep that in mind.
>>>>
>>>
>>> I'd like ask for your suggestions before sending an update for this 
>>> patch.
>>>
>>> After the discussion about locking in fbdev, [1] I intended to 
>>> replace the pin call with code that acquires the reservation lock.
>>
>> Yeah, that sounds like a good idea to me as well.
>>
>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then 
>>> wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd 
>>> expect that vmap/vunmap are close together and do not overlap for 
>>> the same BO. 
>>
>> We have use cases like the following during command submission:
>>
>> 1. lock
>> 2. map
>> 3. copy parts of the BO content somewhere else or patch it with 
>> additional information
>> 4. unmap
>> 5. submit BO to the hardware
>> 6. add hardware fence to the BO to make sure it doesn't move
>> 7. unlock
>>
>> That use case won't be possible with vmap/vunmap if we move the 
>> lock/unlock into it and I hope to replace the kmap/kunmap functions 
>> with them in the near term.
>>
>>> Otherwise, acquiring the reservation lock would require another 
>>> ref-counting variable or per-driver code.
>>
>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as 
>> you initially planned.
>
> Given your example above, step one would acquire the lock, and step 
> two would also acquire the lock as part of the vmap implementation. 
> Wouldn't this fail (At least during unmap or unlock steps) ?

Oh, so you want to nest them? No, that is a rather bad no-go.

You need to make sure that the lock is only taken from the FB path which 
wants to vmap the object.

Why don't you lock the GEM object from the caller in the generic FB 
implementation?

Regards,
Christian.

>
> Best regards
> Thomas
>
>>
>> Regards,
>> Christian.
>>
>>>
>>> Best regards
>>> Thomas
>>>
>>> [1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1
>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>>
>>>>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>>>>> This should always move the BO into GTT-managed memory.
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>> [1]
>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0 
>>>>>
>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>>>>> ---
>>>>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 
>>>>>>> +++++++++++++++++++++++++++--
>>>>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>>>>> radeon_device *rdev, int r)
>>>>>>>        return r;
>>>>>>>    }
>>>>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>>>>> struct dma_buf_map *map)
>>>>>>> +{
>>>>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>>>>> +
>>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>>> +    int ret;
>>>>>>> +
>>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>>> +    if (ret)
>>>>>>> +        return ret;
>>>>>>> +
>>>>>>> +    /* pin buffer at its current location */
>>>>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>>>>> +    if (ret)
>>>>>>> +        goto err_radeon_bo_unreserve;
>>>>>>> +
>>>>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>>>>> +    if (ret)
>>>>>>> +        goto err_radeon_bo_unpin;
>>>>>>> +
>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>> +
>>>>>>> +    return 0;
>>>>>>> +
>>>>>>> +err_radeon_bo_unpin:
>>>>>>> +    radeon_bo_unpin(bo);
>>>>>>> +err_radeon_bo_unreserve:
>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>> +    return ret;
>>>>>>> +}
>>>>>>> +
>>>>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>>>>> struct dma_buf_map *map)
>>>>>>> +{
>>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>>> +    int ret;
>>>>>>> +
>>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>>> +    if (ret)
>>>>>>> +        return;
>>>>>>> +
>>>>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>>>>> +    radeon_bo_unpin(bo);
>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>> +}
>>>>>>> +
>>>>>>>    static const struct drm_gem_object_funcs 
>>>>>>> radeon_gem_object_funcs = {
>>>>>>>        .free = radeon_gem_object_free,
>>>>>>>        .open = radeon_gem_object_open,
>>>>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>>>>> radeon_gem_object_funcs = {
>>>>>>>        .pin = radeon_gem_prime_pin,
>>>>>>>        .unpin = radeon_gem_prime_unpin,
>>>>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>>>>> -    .vmap = drm_gem_ttm_vmap,
>>>>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>>>>> +    .vmap = radeon_gem_object_vmap,
>>>>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>>>>    };
>>>>>>>      /*
>>>>>> _______________________________________________
>>>>>> dri-devel mailing list
>>>>>> dri-devel@lists.freedesktop.org
>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0 
>>>>>>
>>>>
>>>> _______________________________________________
>>>> dri-devel mailing list
>>>> dri-devel@lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>
>>
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 10137 bytes --]

Hi

Am 24.11.20 um 12:54 schrieb Christian König:
> Am 24.11.20 um 12:44 schrieb Thomas Zimmermann:
>> Hi
>>
>> Am 24.11.20 um 12:30 schrieb Christian König:
>>> Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:
>>>> Hi Christian
>>>>
>>>> Am 16.11.20 um 12:28 schrieb Christian König:
>>>>> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>>>>>> Hi Christian
>>>>>>
>>>>>> Am 12.11.20 um 18:16 schrieb Christian König:
>>>>>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>>>>>> In order to avoid eviction of vmap'ed buffers, pin them in their 
>>>>>>>> GEM
>>>>>>>> object's vmap implementation. Unpin them in the vunmap 
>>>>>>>> implementation.
>>>>>>>> This is needed to make generic fbdev support work reliably. 
>>>>>>>> Without,
>>>>>>>> the buffer object could be evicted while fbdev flushed its 
>>>>>>>> shadow buffer.
>>>>>>>>
>>>>>>>> In difference to the PRIME pin/unpin functions, the vmap code 
>>>>>>>> does not
>>>>>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not 
>>>>>>>> count as
>>>>>>>> shared.
>>>>>>>>
>>>>>>>> The actual pin location is not important as the vmap call returns
>>>>>>>> information on how to access the buffer. Callers that require a
>>>>>>>> specific location should explicitly pin the BO before vmapping it.
>>>>>>> Well is the buffer supposed to be scanned out?
>>>>>> No, not by the fbdev helper.
>>>>>
>>>>> Ok in this case that should work.
>>>>>
>>>>>>> If yes then the pin location is actually rather important since the
>>>>>>> hardware can only scan out from VRAM.
>>>>>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>>>>>> relocation transparent to userspace. It flushes the shadow fb into 
>>>>>> the
>>>>>> BO's memory if there are updates. The code is in
>>>>>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>>>>>> call now pins the BO to wherever it is. The actual location does not
>>>>>> matter. It's vunmap'ed immediately afterwards.
>>>>>
>>>>> The problem is what happens when it is prepared for scanout, but 
>>>>> can't be moved to VRAM because it is vmapped?
>>>>>
>>>>> When the shadow is never scanned out that isn't a problem, but we 
>>>>> need to keep that in mind.
>>>>>
>>>>
>>>> I'd like ask for your suggestions before sending an update for this 
>>>> patch.
>>>>
>>>> After the discussion about locking in fbdev, [1] I intended to 
>>>> replace the pin call with code that acquires the reservation lock.
>>>
>>> Yeah, that sounds like a good idea to me as well.
>>>
>>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then 
>>>> wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd 
>>>> expect that vmap/vunmap are close together and do not overlap for 
>>>> the same BO. 
>>>
>>> We have use cases like the following during command submission:
>>>
>>> 1. lock
>>> 2. map
>>> 3. copy parts of the BO content somewhere else or patch it with 
>>> additional information
>>> 4. unmap
>>> 5. submit BO to the hardware
>>> 6. add hardware fence to the BO to make sure it doesn't move
>>> 7. unlock
>>>
>>> That use case won't be possible with vmap/vunmap if we move the 
>>> lock/unlock into it and I hope to replace the kmap/kunmap functions 
>>> with them in the near term.
>>>
>>>> Otherwise, acquiring the reservation lock would require another 
>>>> ref-counting variable or per-driver code.
>>>
>>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as 
>>> you initially planned.
>>
>> Given your example above, step one would acquire the lock, and step 
>> two would also acquire the lock as part of the vmap implementation. 
>> Wouldn't this fail (At least during unmap or unlock steps) ?
> 
> Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be 
required. Apparently not.

While the console's BO is being set for scanout, it's protected from 
movement via the pin/unpin implementation, right? The driver does not 
acquire the resv lock for longer periods. I'm asking because this would 
prevent any console-buffer updates while the console is being displayed.

> 
> You need to make sure that the lock is only taken from the FB path which 
> wants to vmap the object.
> 
> Why don't you lock the GEM object from the caller in the generic FB 
> implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap 
hold the lock implicitly, things would be easier.

Sorry for the noob questions. I'm still trying to understand the 
implications of acquiring these locks.

Best regards
Thomas

> 
> Regards,
> Christian.
> 
>>
>> Best regards
>> Thomas
>>
>>>
>>> Regards,
>>> Christian.
>>>
>>>>
>>>> Best regards
>>>> Thomas
>>>>
>>>> [1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1
>>>>
>>>>> Regards,
>>>>> Christian.
>>>>>
>>>>>>
>>>>>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>>>>>> This should always move the BO into GTT-managed memory.
>>>>>>
>>>>>> Best regards
>>>>>> Thomas
>>>>>>
>>>>>> [1]
>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0 
>>>>>>
>>>>>>
>>>>>>> Regards,
>>>>>>> Christian.
>>>>>>>
>>>>>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>>>>>> ---
>>>>>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 
>>>>>>>> +++++++++++++++++++++++++++--
>>>>>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>>>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>>>>>> radeon_device *rdev, int r)
>>>>>>>>        return r;
>>>>>>>>    }
>>>>>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>>>>>> struct dma_buf_map *map)
>>>>>>>> +{
>>>>>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>>>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>>>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>>>>>> +
>>>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>>>> +    int ret;
>>>>>>>> +
>>>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>>>> +    if (ret)
>>>>>>>> +        return ret;
>>>>>>>> +
>>>>>>>> +    /* pin buffer at its current location */
>>>>>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>>>>>> +    if (ret)
>>>>>>>> +        goto err_radeon_bo_unreserve;
>>>>>>>> +
>>>>>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>>>>>> +    if (ret)
>>>>>>>> +        goto err_radeon_bo_unpin;
>>>>>>>> +
>>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>>> +
>>>>>>>> +    return 0;
>>>>>>>> +
>>>>>>>> +err_radeon_bo_unpin:
>>>>>>>> +    radeon_bo_unpin(bo);
>>>>>>>> +err_radeon_bo_unreserve:
>>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>>> +    return ret;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>>>>>> struct dma_buf_map *map)
>>>>>>>> +{
>>>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>>>> +    int ret;
>>>>>>>> +
>>>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>>>> +    if (ret)
>>>>>>>> +        return;
>>>>>>>> +
>>>>>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>>>>>> +    radeon_bo_unpin(bo);
>>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>>    static const struct drm_gem_object_funcs 
>>>>>>>> radeon_gem_object_funcs = {
>>>>>>>>        .free = radeon_gem_object_free,
>>>>>>>>        .open = radeon_gem_object_open,
>>>>>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>>>>>> radeon_gem_object_funcs = {
>>>>>>>>        .pin = radeon_gem_prime_pin,
>>>>>>>>        .unpin = radeon_gem_prime_unpin,
>>>>>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>>>>>> -    .vmap = drm_gem_ttm_vmap,
>>>>>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>>>>>> +    .vmap = radeon_gem_object_vmap,
>>>>>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>>>>>    };
>>>>>>>>      /*
>>>>>>> _______________________________________________
>>>>>>> dri-devel mailing list
>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0 
>>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> dri-devel mailing list
>>>>> dri-devel@lists.freedesktop.org
>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>
>>>
>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 10137 bytes --]

Hi

Am 24.11.20 um 12:54 schrieb Christian König:
> Am 24.11.20 um 12:44 schrieb Thomas Zimmermann:
>> Hi
>>
>> Am 24.11.20 um 12:30 schrieb Christian König:
>>> Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:
>>>> Hi Christian
>>>>
>>>> Am 16.11.20 um 12:28 schrieb Christian König:
>>>>> Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:
>>>>>> Hi Christian
>>>>>>
>>>>>> Am 12.11.20 um 18:16 schrieb Christian König:
>>>>>>> Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:
>>>>>>>> In order to avoid eviction of vmap'ed buffers, pin them in their 
>>>>>>>> GEM
>>>>>>>> object's vmap implementation. Unpin them in the vunmap 
>>>>>>>> implementation.
>>>>>>>> This is needed to make generic fbdev support work reliably. 
>>>>>>>> Without,
>>>>>>>> the buffer object could be evicted while fbdev flushed its 
>>>>>>>> shadow buffer.
>>>>>>>>
>>>>>>>> In difference to the PRIME pin/unpin functions, the vmap code 
>>>>>>>> does not
>>>>>>>> modify the BOs prime_shared_count, so a vmap-pinned BO does not 
>>>>>>>> count as
>>>>>>>> shared.
>>>>>>>>
>>>>>>>> The actual pin location is not important as the vmap call returns
>>>>>>>> information on how to access the buffer. Callers that require a
>>>>>>>> specific location should explicitly pin the BO before vmapping it.
>>>>>>> Well is the buffer supposed to be scanned out?
>>>>>> No, not by the fbdev helper.
>>>>>
>>>>> Ok in this case that should work.
>>>>>
>>>>>>> If yes then the pin location is actually rather important since the
>>>>>>> hardware can only scan out from VRAM.
>>>>>> For relocatable BOs, fbdev uses a shadow buffer that makes all any
>>>>>> relocation transparent to userspace. It flushes the shadow fb into 
>>>>>> the
>>>>>> BO's memory if there are updates. The code is in
>>>>>> drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap
>>>>>> call now pins the BO to wherever it is. The actual location does not
>>>>>> matter. It's vunmap'ed immediately afterwards.
>>>>>
>>>>> The problem is what happens when it is prepared for scanout, but 
>>>>> can't be moved to VRAM because it is vmapped?
>>>>>
>>>>> When the shadow is never scanned out that isn't a problem, but we 
>>>>> need to keep that in mind.
>>>>>
>>>>
>>>> I'd like ask for your suggestions before sending an update for this 
>>>> patch.
>>>>
>>>> After the discussion about locking in fbdev, [1] I intended to 
>>>> replace the pin call with code that acquires the reservation lock.
>>>
>>> Yeah, that sounds like a good idea to me as well.
>>>
>>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then 
>>>> wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd 
>>>> expect that vmap/vunmap are close together and do not overlap for 
>>>> the same BO. 
>>>
>>> We have use cases like the following during command submission:
>>>
>>> 1. lock
>>> 2. map
>>> 3. copy parts of the BO content somewhere else or patch it with 
>>> additional information
>>> 4. unmap
>>> 5. submit BO to the hardware
>>> 6. add hardware fence to the BO to make sure it doesn't move
>>> 7. unlock
>>>
>>> That use case won't be possible with vmap/vunmap if we move the 
>>> lock/unlock into it and I hope to replace the kmap/kunmap functions 
>>> with them in the near term.
>>>
>>>> Otherwise, acquiring the reservation lock would require another 
>>>> ref-counting variable or per-driver code.
>>>
>>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as 
>>> you initially planned.
>>
>> Given your example above, step one would acquire the lock, and step 
>> two would also acquire the lock as part of the vmap implementation. 
>> Wouldn't this fail (At least during unmap or unlock steps) ?
> 
> Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be 
required. Apparently not.

While the console's BO is being set for scanout, it's protected from 
movement via the pin/unpin implementation, right? The driver does not 
acquire the resv lock for longer periods. I'm asking because this would 
prevent any console-buffer updates while the console is being displayed.

> 
> You need to make sure that the lock is only taken from the FB path which 
> wants to vmap the object.
> 
> Why don't you lock the GEM object from the caller in the generic FB 
> implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap 
hold the lock implicitly, things would be easier.

Sorry for the noob questions. I'm still trying to understand the 
implications of acquiring these locks.

Best regards
Thomas

> 
> Regards,
> Christian.
> 
>>
>> Best regards
>> Thomas
>>
>>>
>>> Regards,
>>> Christian.
>>>
>>>>
>>>> Best regards
>>>> Thomas
>>>>
>>>> [1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1
>>>>
>>>>> Regards,
>>>>> Christian.
>>>>>
>>>>>>
>>>>>> For dma-buf sharing, the regular procedure of pin + vmap still apply.
>>>>>> This should always move the BO into GTT-managed memory.
>>>>>>
>>>>>> Best regards
>>>>>> Thomas
>>>>>>
>>>>>> [1]
>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&reserved=0 
>>>>>>
>>>>>>
>>>>>>> Regards,
>>>>>>> Christian.
>>>>>>>
>>>>>>>> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
>>>>>>>> ---
>>>>>>>>    drivers/gpu/drm/radeon/radeon_gem.c | 51 
>>>>>>>> +++++++++++++++++++++++++++--
>>>>>>>>    1 file changed, 49 insertions(+), 2 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>>> b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>>> index d2876ce3bc9e..eaf7fc9a7b07 100644
>>>>>>>> --- a/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>>> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
>>>>>>>> @@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
>>>>>>>> radeon_device *rdev, int r)
>>>>>>>>        return r;
>>>>>>>>    }
>>>>>>>>    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
>>>>>>>> struct dma_buf_map *map)
>>>>>>>> +{
>>>>>>>> +    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
>>>>>>>> +                       RADEON_GEM_DOMAIN_GTT |
>>>>>>>> +                       RADEON_GEM_DOMAIN_CPU;
>>>>>>>> +
>>>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>>>> +    int ret;
>>>>>>>> +
>>>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>>>> +    if (ret)
>>>>>>>> +        return ret;
>>>>>>>> +
>>>>>>>> +    /* pin buffer at its current location */
>>>>>>>> +    ret = radeon_bo_pin(bo, any_domain, NULL);
>>>>>>>> +    if (ret)
>>>>>>>> +        goto err_radeon_bo_unreserve;
>>>>>>>> +
>>>>>>>> +    ret = drm_gem_ttm_vmap(obj, map);
>>>>>>>> +    if (ret)
>>>>>>>> +        goto err_radeon_bo_unpin;
>>>>>>>> +
>>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>>> +
>>>>>>>> +    return 0;
>>>>>>>> +
>>>>>>>> +err_radeon_bo_unpin:
>>>>>>>> +    radeon_bo_unpin(bo);
>>>>>>>> +err_radeon_bo_unreserve:
>>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>>> +    return ret;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> +static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
>>>>>>>> struct dma_buf_map *map)
>>>>>>>> +{
>>>>>>>> +    struct radeon_bo *bo = gem_to_radeon_bo(obj);
>>>>>>>> +    int ret;
>>>>>>>> +
>>>>>>>> +    ret = radeon_bo_reserve(bo, false);
>>>>>>>> +    if (ret)
>>>>>>>> +        return;
>>>>>>>> +
>>>>>>>> +    drm_gem_ttm_vunmap(obj, map);
>>>>>>>> +    radeon_bo_unpin(bo);
>>>>>>>> +    radeon_bo_unreserve(bo);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>>    static const struct drm_gem_object_funcs 
>>>>>>>> radeon_gem_object_funcs = {
>>>>>>>>        .free = radeon_gem_object_free,
>>>>>>>>        .open = radeon_gem_object_open,
>>>>>>>> @@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
>>>>>>>> radeon_gem_object_funcs = {
>>>>>>>>        .pin = radeon_gem_prime_pin,
>>>>>>>>        .unpin = radeon_gem_prime_unpin,
>>>>>>>>        .get_sg_table = radeon_gem_prime_get_sg_table,
>>>>>>>> -    .vmap = drm_gem_ttm_vmap,
>>>>>>>> -    .vunmap = drm_gem_ttm_vunmap,
>>>>>>>> +    .vmap = radeon_gem_object_vmap,
>>>>>>>> +    .vunmap = radeon_gem_object_vunmap,
>>>>>>>>    };
>>>>>>>>      /*
>>>>>>> _______________________________________________
>>>>>>> dri-devel mailing list
>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0 
>>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> dri-devel mailing list
>>>>> dri-devel@lists.freedesktop.org
>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>
>>>
>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
> [SNIP]
>>>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but 
>>>>> then wondered why ttm_bo_vmap() doe not acquire the lock 
>>>>> internally? I'd expect that vmap/vunmap are close together and do 
>>>>> not overlap for the same BO. 
>>>>
>>>> We have use cases like the following during command submission:
>>>>
>>>> 1. lock
>>>> 2. map
>>>> 3. copy parts of the BO content somewhere else or patch it with 
>>>> additional information
>>>> 4. unmap
>>>> 5. submit BO to the hardware
>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>> 7. unlock
>>>>
>>>> That use case won't be possible with vmap/vunmap if we move the 
>>>> lock/unlock into it and I hope to replace the kmap/kunmap functions 
>>>> with them in the near term.
>>>>
>>>>> Otherwise, acquiring the reservation lock would require another 
>>>>> ref-counting variable or per-driver code.
>>>>
>>>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper 
>>>> as you initially planned.
>>>
>>> Given your example above, step one would acquire the lock, and step 
>>> two would also acquire the lock as part of the vmap implementation. 
>>> Wouldn't this fail (At least during unmap or unlock steps) ?
>>
>> Oh, so you want to nest them? No, that is a rather bad no-go.
>
> I don't want to nest/overlap them. My question was whether that would 
> be required. Apparently not.
>
> While the console's BO is being set for scanout, it's protected from 
> movement via the pin/unpin implementation, right?

Yes, correct.

> The driver does not acquire the resv lock for longer periods. I'm 
> asking because this would prevent any console-buffer updates while the 
> console is being displayed.

Correct as well, we only hold the lock for things like command 
submission, pinning, unpinning etc etc....

>
>>
>> You need to make sure that the lock is only taken from the FB path 
>> which wants to vmap the object.
>>
>> Why don't you lock the GEM object from the caller in the generic FB 
>> implementation?
>
> With the current blitter code, it breaks abstraction. if vmap/vunmap 
> hold the lock implicitly, things would be easier.

Do you have a link to the code?

Please note that the reservation lock you need to take here is part of 
the GEM object.

Usually we design things in the way that the code needs to take a lock 
which protects an object, then do some operations with the object and 
then release the lock again.

Having in the lock inside the operation can be done as well, but 
returning with it is kind of unusual design.

> Sorry for the noob questions. I'm still trying to understand the 
> implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about 
here. We need to take that for a couple of different operations, 
vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

>
> Best regards
> Thomas

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
> [SNIP]
>>>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but 
>>>>> then wondered why ttm_bo_vmap() doe not acquire the lock 
>>>>> internally? I'd expect that vmap/vunmap are close together and do 
>>>>> not overlap for the same BO. 
>>>>
>>>> We have use cases like the following during command submission:
>>>>
>>>> 1. lock
>>>> 2. map
>>>> 3. copy parts of the BO content somewhere else or patch it with 
>>>> additional information
>>>> 4. unmap
>>>> 5. submit BO to the hardware
>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>> 7. unlock
>>>>
>>>> That use case won't be possible with vmap/vunmap if we move the 
>>>> lock/unlock into it and I hope to replace the kmap/kunmap functions 
>>>> with them in the near term.
>>>>
>>>>> Otherwise, acquiring the reservation lock would require another 
>>>>> ref-counting variable or per-driver code.
>>>>
>>>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper 
>>>> as you initially planned.
>>>
>>> Given your example above, step one would acquire the lock, and step 
>>> two would also acquire the lock as part of the vmap implementation. 
>>> Wouldn't this fail (At least during unmap or unlock steps) ?
>>
>> Oh, so you want to nest them? No, that is a rather bad no-go.
>
> I don't want to nest/overlap them. My question was whether that would 
> be required. Apparently not.
>
> While the console's BO is being set for scanout, it's protected from 
> movement via the pin/unpin implementation, right?

Yes, correct.

> The driver does not acquire the resv lock for longer periods. I'm 
> asking because this would prevent any console-buffer updates while the 
> console is being displayed.

Correct as well, we only hold the lock for things like command 
submission, pinning, unpinning etc etc....

>
>>
>> You need to make sure that the lock is only taken from the FB path 
>> which wants to vmap the object.
>>
>> Why don't you lock the GEM object from the caller in the generic FB 
>> implementation?
>
> With the current blitter code, it breaks abstraction. if vmap/vunmap 
> hold the lock implicitly, things would be easier.

Do you have a link to the code?

Please note that the reservation lock you need to take here is part of 
the GEM object.

Usually we design things in the way that the code needs to take a lock 
which protects an object, then do some operations with the object and 
then release the lock again.

Having in the lock inside the operation can be done as well, but 
returning with it is kind of unusual design.

> Sorry for the noob questions. I'm still trying to understand the 
> implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about 
here. We need to take that for a couple of different operations, 
vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

>
> Best regards
> Thomas

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 4248 bytes --]

Hi

Am 24.11.20 um 14:36 schrieb Christian König:
> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>> [SNIP]
>>>>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but 
>>>>>> then wondered why ttm_bo_vmap() doe not acquire the lock 
>>>>>> internally? I'd expect that vmap/vunmap are close together and do 
>>>>>> not overlap for the same BO. 
>>>>>
>>>>> We have use cases like the following during command submission:
>>>>>
>>>>> 1. lock
>>>>> 2. map
>>>>> 3. copy parts of the BO content somewhere else or patch it with 
>>>>> additional information
>>>>> 4. unmap
>>>>> 5. submit BO to the hardware
>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>> 7. unlock
>>>>>
>>>>> That use case won't be possible with vmap/vunmap if we move the 
>>>>> lock/unlock into it and I hope to replace the kmap/kunmap functions 
>>>>> with them in the near term.
>>>>>
>>>>>> Otherwise, acquiring the reservation lock would require another 
>>>>>> ref-counting variable or per-driver code.
>>>>>
>>>>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper 
>>>>> as you initially planned.
>>>>
>>>> Given your example above, step one would acquire the lock, and step 
>>>> two would also acquire the lock as part of the vmap implementation. 
>>>> Wouldn't this fail (At least during unmap or unlock steps) ?
>>>
>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>
>> I don't want to nest/overlap them. My question was whether that would 
>> be required. Apparently not.
>>
>> While the console's BO is being set for scanout, it's protected from 
>> movement via the pin/unpin implementation, right?
> 
> Yes, correct.
> 
>> The driver does not acquire the resv lock for longer periods. I'm 
>> asking because this would prevent any console-buffer updates while the 
>> console is being displayed.
> 
> Correct as well, we only hold the lock for things like command 
> submission, pinning, unpinning etc etc....
> 

Thanks for answering my questions.

>>
>>>
>>> You need to make sure that the lock is only taken from the FB path 
>>> which wants to vmap the object.
>>>
>>> Why don't you lock the GEM object from the caller in the generic FB 
>>> implementation?
>>
>> With the current blitter code, it breaks abstraction. if vmap/vunmap 
>> hold the lock implicitly, things would be easier.
> 
> Do you have a link to the code?

It's the damage blitter in the fbdev code. [1] While it flushes the 
shadow buffer into the BO, the BO has to be kept in place. I already 
changed it to lock struct drm_fb_helper.lock, but I don't think this is 
enough. TTM could still evict the BO concurrently.

There's no recursion taking place, so I guess the reservation lock could 
be acquired/release in drm_client_buffer_vmap/vunmap(), or a separate 
pair of DRM client functions could do the locking.

Best regards
Thomas

[1] 
https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

> 
> Please note that the reservation lock you need to take here is part of 
> the GEM object.
> 
> Usually we design things in the way that the code needs to take a lock 
> which protects an object, then do some operations with the object and 
> then release the lock again.
> 
> Having in the lock inside the operation can be done as well, but 
> returning with it is kind of unusual design.
> 
>> Sorry for the noob questions. I'm still trying to understand the 
>> implications of acquiring these locks.
> 
> Well this is the reservation lock of the GEM object we are talking about 
> here. We need to take that for a couple of different operations, 
> vmap/vunmap doesn't sound like a special case to me.
> 
> Regards,
> Christian.
> 
>>
>> Best regards
>> Thomas
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 4248 bytes --]

Hi

Am 24.11.20 um 14:36 schrieb Christian König:
> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>> [SNIP]
>>>>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but 
>>>>>> then wondered why ttm_bo_vmap() doe not acquire the lock 
>>>>>> internally? I'd expect that vmap/vunmap are close together and do 
>>>>>> not overlap for the same BO. 
>>>>>
>>>>> We have use cases like the following during command submission:
>>>>>
>>>>> 1. lock
>>>>> 2. map
>>>>> 3. copy parts of the BO content somewhere else or patch it with 
>>>>> additional information
>>>>> 4. unmap
>>>>> 5. submit BO to the hardware
>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>> 7. unlock
>>>>>
>>>>> That use case won't be possible with vmap/vunmap if we move the 
>>>>> lock/unlock into it and I hope to replace the kmap/kunmap functions 
>>>>> with them in the near term.
>>>>>
>>>>>> Otherwise, acquiring the reservation lock would require another 
>>>>>> ref-counting variable or per-driver code.
>>>>>
>>>>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper 
>>>>> as you initially planned.
>>>>
>>>> Given your example above, step one would acquire the lock, and step 
>>>> two would also acquire the lock as part of the vmap implementation. 
>>>> Wouldn't this fail (At least during unmap or unlock steps) ?
>>>
>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>
>> I don't want to nest/overlap them. My question was whether that would 
>> be required. Apparently not.
>>
>> While the console's BO is being set for scanout, it's protected from 
>> movement via the pin/unpin implementation, right?
> 
> Yes, correct.
> 
>> The driver does not acquire the resv lock for longer periods. I'm 
>> asking because this would prevent any console-buffer updates while the 
>> console is being displayed.
> 
> Correct as well, we only hold the lock for things like command 
> submission, pinning, unpinning etc etc....
> 

Thanks for answering my questions.

>>
>>>
>>> You need to make sure that the lock is only taken from the FB path 
>>> which wants to vmap the object.
>>>
>>> Why don't you lock the GEM object from the caller in the generic FB 
>>> implementation?
>>
>> With the current blitter code, it breaks abstraction. if vmap/vunmap 
>> hold the lock implicitly, things would be easier.
> 
> Do you have a link to the code?

It's the damage blitter in the fbdev code. [1] While it flushes the 
shadow buffer into the BO, the BO has to be kept in place. I already 
changed it to lock struct drm_fb_helper.lock, but I don't think this is 
enough. TTM could still evict the BO concurrently.

There's no recursion taking place, so I guess the reservation lock could 
be acquired/release in drm_client_buffer_vmap/vunmap(), or a separate 
pair of DRM client functions could do the locking.

Best regards
Thomas

[1] 
https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

> 
> Please note that the reservation lock you need to take here is part of 
> the GEM object.
> 
> Usually we design things in the way that the code needs to take a lock 
> which protects an object, then do some operations with the object and 
> then release the lock again.
> 
> Having in the lock inside the operation can be done as well, but 
> returning with it is kind of unusual design.
> 
>> Sorry for the noob questions. I'm still trying to understand the 
>> implications of acquiring these locks.
> 
> Well this is the reservation lock of the GEM object we are talking about 
> here. We need to take that for a couple of different operations, 
> vmap/vunmap doesn't sound like a special case to me.
> 
> Regards,
> Christian.
> 
>>
>> Best regards
>> Thomas
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 24.11.20 um 14:56 schrieb Thomas Zimmermann:
> Hi
>
> Am 24.11.20 um 14:36 schrieb Christian König:
>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>> [SNIP]
>>>>>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but 
>>>>>>> then wondered why ttm_bo_vmap() doe not acquire the lock 
>>>>>>> internally? I'd expect that vmap/vunmap are close together and 
>>>>>>> do not overlap for the same BO. 
>>>>>>
>>>>>> We have use cases like the following during command submission:
>>>>>>
>>>>>> 1. lock
>>>>>> 2. map
>>>>>> 3. copy parts of the BO content somewhere else or patch it with 
>>>>>> additional information
>>>>>> 4. unmap
>>>>>> 5. submit BO to the hardware
>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>> 7. unlock
>>>>>>
>>>>>> That use case won't be possible with vmap/vunmap if we move the 
>>>>>> lock/unlock into it and I hope to replace the kmap/kunmap 
>>>>>> functions with them in the near term.
>>>>>>
>>>>>>> Otherwise, acquiring the reservation lock would require another 
>>>>>>> ref-counting variable or per-driver code.
>>>>>>
>>>>>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() 
>>>>>> helper as you initially planned.
>>>>>
>>>>> Given your example above, step one would acquire the lock, and 
>>>>> step two would also acquire the lock as part of the vmap 
>>>>> implementation. Wouldn't this fail (At least during unmap or 
>>>>> unlock steps) ?
>>>>
>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>
>>> I don't want to nest/overlap them. My question was whether that 
>>> would be required. Apparently not.
>>>
>>> While the console's BO is being set for scanout, it's protected from 
>>> movement via the pin/unpin implementation, right?
>>
>> Yes, correct.
>>
>>> The driver does not acquire the resv lock for longer periods. I'm 
>>> asking because this would prevent any console-buffer updates while 
>>> the console is being displayed.
>>
>> Correct as well, we only hold the lock for things like command 
>> submission, pinning, unpinning etc etc....
>>
>
> Thanks for answering my questions.
>
>>>
>>>>
>>>> You need to make sure that the lock is only taken from the FB path 
>>>> which wants to vmap the object.
>>>>
>>>> Why don't you lock the GEM object from the caller in the generic FB 
>>>> implementation?
>>>
>>> With the current blitter code, it breaks abstraction. if vmap/vunmap 
>>> hold the lock implicitly, things would be easier.
>>
>> Do you have a link to the code?
>
> It's the damage blitter in the fbdev code. [1] While it flushes the 
> shadow buffer into the BO, the BO has to be kept in place. I already 
> changed it to lock struct drm_fb_helper.lock, but I don't think this 
> is enough. TTM could still evict the BO concurrently.

Yeah, that's correct.

But I still don't fully understand the problem. You just need to change 
the code like this:

     mutex_lock(&fb_helper->lock);
     dma_resv_lock(buffer->gem->resv, NULL);

     ret = drm_client_buffer_vmap(buffer, &map);
     if (ret)
         goto out;

     dst = map;
     drm_fb_helper_damage_blit_real(fb_helper, clip, &dst);

     drm_client_buffer_vunmap(buffer);

out:
     dma_resv_unlock(buffer->gem->resv);
     mutex_unlock(&fb_helper->lock);


You could abstract that in drm_client functions as well, but I don't 
really see the value in that.

Regards,
Christian.

> There's no recursion taking place, so I guess the reservation lock 
> could be acquired/release in drm_client_buffer_vmap/vunmap(), or a 
> separate pair of DRM client functions could do the locking.
>
> Best regards
> Thomas
>
> [1] 
> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>
>>
>> Please note that the reservation lock you need to take here is part 
>> of the GEM object.
>>
>> Usually we design things in the way that the code needs to take a 
>> lock which protects an object, then do some operations with the 
>> object and then release the lock again.
>>
>> Having in the lock inside the operation can be done as well, but 
>> returning with it is kind of unusual design.
>>
>>> Sorry for the noob questions. I'm still trying to understand the 
>>> implications of acquiring these locks.
>>
>> Well this is the reservation lock of the GEM object we are talking 
>> about here. We need to take that for a couple of different 
>> operations, vmap/vunmap doesn't sound like a special case to me.
>>
>> Regards,
>> Christian.
>>
>>>
>>> Best regards
>>> Thomas
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 24.11.20 um 14:56 schrieb Thomas Zimmermann:
> Hi
>
> Am 24.11.20 um 14:36 schrieb Christian König:
>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>> [SNIP]
>>>>>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but 
>>>>>>> then wondered why ttm_bo_vmap() doe not acquire the lock 
>>>>>>> internally? I'd expect that vmap/vunmap are close together and 
>>>>>>> do not overlap for the same BO. 
>>>>>>
>>>>>> We have use cases like the following during command submission:
>>>>>>
>>>>>> 1. lock
>>>>>> 2. map
>>>>>> 3. copy parts of the BO content somewhere else or patch it with 
>>>>>> additional information
>>>>>> 4. unmap
>>>>>> 5. submit BO to the hardware
>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>> 7. unlock
>>>>>>
>>>>>> That use case won't be possible with vmap/vunmap if we move the 
>>>>>> lock/unlock into it and I hope to replace the kmap/kunmap 
>>>>>> functions with them in the near term.
>>>>>>
>>>>>>> Otherwise, acquiring the reservation lock would require another 
>>>>>>> ref-counting variable or per-driver code.
>>>>>>
>>>>>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() 
>>>>>> helper as you initially planned.
>>>>>
>>>>> Given your example above, step one would acquire the lock, and 
>>>>> step two would also acquire the lock as part of the vmap 
>>>>> implementation. Wouldn't this fail (At least during unmap or 
>>>>> unlock steps) ?
>>>>
>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>
>>> I don't want to nest/overlap them. My question was whether that 
>>> would be required. Apparently not.
>>>
>>> While the console's BO is being set for scanout, it's protected from 
>>> movement via the pin/unpin implementation, right?
>>
>> Yes, correct.
>>
>>> The driver does not acquire the resv lock for longer periods. I'm 
>>> asking because this would prevent any console-buffer updates while 
>>> the console is being displayed.
>>
>> Correct as well, we only hold the lock for things like command 
>> submission, pinning, unpinning etc etc....
>>
>
> Thanks for answering my questions.
>
>>>
>>>>
>>>> You need to make sure that the lock is only taken from the FB path 
>>>> which wants to vmap the object.
>>>>
>>>> Why don't you lock the GEM object from the caller in the generic FB 
>>>> implementation?
>>>
>>> With the current blitter code, it breaks abstraction. if vmap/vunmap 
>>> hold the lock implicitly, things would be easier.
>>
>> Do you have a link to the code?
>
> It's the damage blitter in the fbdev code. [1] While it flushes the 
> shadow buffer into the BO, the BO has to be kept in place. I already 
> changed it to lock struct drm_fb_helper.lock, but I don't think this 
> is enough. TTM could still evict the BO concurrently.

Yeah, that's correct.

But I still don't fully understand the problem. You just need to change 
the code like this:

     mutex_lock(&fb_helper->lock);
     dma_resv_lock(buffer->gem->resv, NULL);

     ret = drm_client_buffer_vmap(buffer, &map);
     if (ret)
         goto out;

     dst = map;
     drm_fb_helper_damage_blit_real(fb_helper, clip, &dst);

     drm_client_buffer_vunmap(buffer);

out:
     dma_resv_unlock(buffer->gem->resv);
     mutex_unlock(&fb_helper->lock);


You could abstract that in drm_client functions as well, but I don't 
really see the value in that.

Regards,
Christian.

> There's no recursion taking place, so I guess the reservation lock 
> could be acquired/release in drm_client_buffer_vmap/vunmap(), or a 
> separate pair of DRM client functions could do the locking.
>
> Best regards
> Thomas
>
> [1] 
> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>
>>
>> Please note that the reservation lock you need to take here is part 
>> of the GEM object.
>>
>> Usually we design things in the way that the code needs to take a 
>> lock which protects an object, then do some operations with the 
>> object and then release the lock again.
>>
>> Having in the lock inside the operation can be done as well, but 
>> returning with it is kind of unusual design.
>>
>>> Sorry for the noob questions. I'm still trying to understand the 
>>> implications of acquiring these locks.
>>
>> Well this is the reservation lock of the GEM object we are talking 
>> about here. We need to take that for a couple of different 
>> operations, vmap/vunmap doesn't sound like a special case to me.
>>
>> Regards,
>> Christian.
>>
>>>
>>> Best regards
>>> Thomas
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Daniel Vetter]

On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
> Hi
> 
> Am 24.11.20 um 14:36 schrieb Christian König:
> > Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
> > > [SNIP]
> > > > > > > First I wanted to put this into
> > > > > > > drm_gem_ttm_vmap/vunmap(), but then wondered why
> > > > > > > ttm_bo_vmap() doe not acquire the lock internally?
> > > > > > > I'd expect that vmap/vunmap are close together and
> > > > > > > do not overlap for the same BO.
> > > > > > 
> > > > > > We have use cases like the following during command submission:
> > > > > > 
> > > > > > 1. lock
> > > > > > 2. map
> > > > > > 3. copy parts of the BO content somewhere else or patch
> > > > > > it with additional information
> > > > > > 4. unmap
> > > > > > 5. submit BO to the hardware
> > > > > > 6. add hardware fence to the BO to make sure it doesn't move
> > > > > > 7. unlock
> > > > > > 
> > > > > > That use case won't be possible with vmap/vunmap if we
> > > > > > move the lock/unlock into it and I hope to replace the
> > > > > > kmap/kunmap functions with them in the near term.
> > > > > > 
> > > > > > > Otherwise, acquiring the reservation lock would
> > > > > > > require another ref-counting variable or per-driver
> > > > > > > code.
> > > > > > 
> > > > > > Hui, why that? Just put this into
> > > > > > drm_gem_ttm_vmap/vunmap() helper as you initially
> > > > > > planned.
> > > > > 
> > > > > Given your example above, step one would acquire the lock,
> > > > > and step two would also acquire the lock as part of the vmap
> > > > > implementation. Wouldn't this fail (At least during unmap or
> > > > > unlock steps) ?
> > > > 
> > > > Oh, so you want to nest them? No, that is a rather bad no-go.
> > > 
> > > I don't want to nest/overlap them. My question was whether that
> > > would be required. Apparently not.
> > > 
> > > While the console's BO is being set for scanout, it's protected from
> > > movement via the pin/unpin implementation, right?
> > 
> > Yes, correct.
> > 
> > > The driver does not acquire the resv lock for longer periods. I'm
> > > asking because this would prevent any console-buffer updates while
> > > the console is being displayed.
> > 
> > Correct as well, we only hold the lock for things like command
> > submission, pinning, unpinning etc etc....
> > 
> 
> Thanks for answering my questions.
> 
> > > 
> > > > 
> > > > You need to make sure that the lock is only taken from the FB
> > > > path which wants to vmap the object.
> > > > 
> > > > Why don't you lock the GEM object from the caller in the generic
> > > > FB implementation?
> > > 
> > > With the current blitter code, it breaks abstraction. if vmap/vunmap
> > > hold the lock implicitly, things would be easier.
> > 
> > Do you have a link to the code?
> 
> It's the damage blitter in the fbdev code. [1] While it flushes the shadow
> buffer into the BO, the BO has to be kept in place. I already changed it to
> lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could
> still evict the BO concurrently.

So I'm not sure this is actually a problem: ttm could try to concurrently
evict the buffer we pinned into vram, and then just skip to the next one.

Plus atm generic fbdev isn't used on any chip where we really care about
that last few mb of vram being useable for command submission (well atm
there's no driver using it).

Having the buffer pinned into system memory and trying to do a concurrent
modeset that tries to pull it in is the hard failure mode. And holding
fb_helper.lock fully prevents that.

So not really clear on what failure mode you're seeing here?

> There's no recursion taking place, so I guess the reservation lock could be
> acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
> DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is
perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic
code, while many helpers/drivers still have their hand-rolled locking,
will make conversion over to dma_resv needlessly more complicated.
-Daniel

> 
> Best regards
> Thomas
> 
> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
> 
> > 
> > Please note that the reservation lock you need to take here is part of
> > the GEM object.
> > 
> > Usually we design things in the way that the code needs to take a lock
> > which protects an object, then do some operations with the object and
> > then release the lock again.
> > 
> > Having in the lock inside the operation can be done as well, but
> > returning with it is kind of unusual design.
> > 
> > > Sorry for the noob questions. I'm still trying to understand the
> > > implications of acquiring these locks.
> > 
> > Well this is the reservation lock of the GEM object we are talking about
> > here. We need to take that for a couple of different operations,
> > vmap/vunmap doesn't sound like a special case to me.
> > 
> > Regards,
> > Christian.
> > 
> > > 
> > > Best regards
> > > Thomas
> > 
> > _______________________________________________
> > dri-devel mailing list
> > dri-devel@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> 
> -- 
> Thomas Zimmermann
> Graphics Driver Developer
> SUSE Software Solutions Germany GmbH
> Maxfeldstr. 5, 90409 Nürnberg, Germany
> (HRB 36809, AG Nürnberg)
> Geschäftsführer: Felix Imendörffer






-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Daniel Vetter]

On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
> Hi
> 
> Am 24.11.20 um 14:36 schrieb Christian König:
> > Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
> > > [SNIP]
> > > > > > > First I wanted to put this into
> > > > > > > drm_gem_ttm_vmap/vunmap(), but then wondered why
> > > > > > > ttm_bo_vmap() doe not acquire the lock internally?
> > > > > > > I'd expect that vmap/vunmap are close together and
> > > > > > > do not overlap for the same BO.
> > > > > > 
> > > > > > We have use cases like the following during command submission:
> > > > > > 
> > > > > > 1. lock
> > > > > > 2. map
> > > > > > 3. copy parts of the BO content somewhere else or patch
> > > > > > it with additional information
> > > > > > 4. unmap
> > > > > > 5. submit BO to the hardware
> > > > > > 6. add hardware fence to the BO to make sure it doesn't move
> > > > > > 7. unlock
> > > > > > 
> > > > > > That use case won't be possible with vmap/vunmap if we
> > > > > > move the lock/unlock into it and I hope to replace the
> > > > > > kmap/kunmap functions with them in the near term.
> > > > > > 
> > > > > > > Otherwise, acquiring the reservation lock would
> > > > > > > require another ref-counting variable or per-driver
> > > > > > > code.
> > > > > > 
> > > > > > Hui, why that? Just put this into
> > > > > > drm_gem_ttm_vmap/vunmap() helper as you initially
> > > > > > planned.
> > > > > 
> > > > > Given your example above, step one would acquire the lock,
> > > > > and step two would also acquire the lock as part of the vmap
> > > > > implementation. Wouldn't this fail (At least during unmap or
> > > > > unlock steps) ?
> > > > 
> > > > Oh, so you want to nest them? No, that is a rather bad no-go.
> > > 
> > > I don't want to nest/overlap them. My question was whether that
> > > would be required. Apparently not.
> > > 
> > > While the console's BO is being set for scanout, it's protected from
> > > movement via the pin/unpin implementation, right?
> > 
> > Yes, correct.
> > 
> > > The driver does not acquire the resv lock for longer periods. I'm
> > > asking because this would prevent any console-buffer updates while
> > > the console is being displayed.
> > 
> > Correct as well, we only hold the lock for things like command
> > submission, pinning, unpinning etc etc....
> > 
> 
> Thanks for answering my questions.
> 
> > > 
> > > > 
> > > > You need to make sure that the lock is only taken from the FB
> > > > path which wants to vmap the object.
> > > > 
> > > > Why don't you lock the GEM object from the caller in the generic
> > > > FB implementation?
> > > 
> > > With the current blitter code, it breaks abstraction. if vmap/vunmap
> > > hold the lock implicitly, things would be easier.
> > 
> > Do you have a link to the code?
> 
> It's the damage blitter in the fbdev code. [1] While it flushes the shadow
> buffer into the BO, the BO has to be kept in place. I already changed it to
> lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could
> still evict the BO concurrently.

So I'm not sure this is actually a problem: ttm could try to concurrently
evict the buffer we pinned into vram, and then just skip to the next one.

Plus atm generic fbdev isn't used on any chip where we really care about
that last few mb of vram being useable for command submission (well atm
there's no driver using it).

Having the buffer pinned into system memory and trying to do a concurrent
modeset that tries to pull it in is the hard failure mode. And holding
fb_helper.lock fully prevents that.

So not really clear on what failure mode you're seeing here?

> There's no recursion taking place, so I guess the reservation lock could be
> acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
> DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is
perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic
code, while many helpers/drivers still have their hand-rolled locking,
will make conversion over to dma_resv needlessly more complicated.
-Daniel

> 
> Best regards
> Thomas
> 
> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
> 
> > 
> > Please note that the reservation lock you need to take here is part of
> > the GEM object.
> > 
> > Usually we design things in the way that the code needs to take a lock
> > which protects an object, then do some operations with the object and
> > then release the lock again.
> > 
> > Having in the lock inside the operation can be done as well, but
> > returning with it is kind of unusual design.
> > 
> > > Sorry for the noob questions. I'm still trying to understand the
> > > implications of acquiring these locks.
> > 
> > Well this is the reservation lock of the GEM object we are talking about
> > here. We need to take that for a couple of different operations,
> > vmap/vunmap doesn't sound like a special case to me.
> > 
> > Regards,
> > Christian.
> > 
> > > 
> > > Best regards
> > > Thomas
> > 
> > _______________________________________________
> > dri-devel mailing list
> > dri-devel@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> 
> -- 
> Thomas Zimmermann
> Graphics Driver Developer
> SUSE Software Solutions Germany GmbH
> Maxfeldstr. 5, 90409 Nürnberg, Germany
> (HRB 36809, AG Nürnberg)
> Geschäftsführer: Felix Imendörffer






-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 5736 bytes --]

Hi

Am 24.11.20 um 15:06 schrieb Christian König:
> Am 24.11.20 um 14:56 schrieb Thomas Zimmermann:
>> Hi
>>
>> Am 24.11.20 um 14:36 schrieb Christian König:
>>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>>> [SNIP]
>>>>>>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but 
>>>>>>>> then wondered why ttm_bo_vmap() doe not acquire the lock 
>>>>>>>> internally? I'd expect that vmap/vunmap are close together and 
>>>>>>>> do not overlap for the same BO. 
>>>>>>>
>>>>>>> We have use cases like the following during command submission:
>>>>>>>
>>>>>>> 1. lock
>>>>>>> 2. map
>>>>>>> 3. copy parts of the BO content somewhere else or patch it with 
>>>>>>> additional information
>>>>>>> 4. unmap
>>>>>>> 5. submit BO to the hardware
>>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>>> 7. unlock
>>>>>>>
>>>>>>> That use case won't be possible with vmap/vunmap if we move the 
>>>>>>> lock/unlock into it and I hope to replace the kmap/kunmap 
>>>>>>> functions with them in the near term.
>>>>>>>
>>>>>>>> Otherwise, acquiring the reservation lock would require another 
>>>>>>>> ref-counting variable or per-driver code.
>>>>>>>
>>>>>>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() 
>>>>>>> helper as you initially planned.
>>>>>>
>>>>>> Given your example above, step one would acquire the lock, and 
>>>>>> step two would also acquire the lock as part of the vmap 
>>>>>> implementation. Wouldn't this fail (At least during unmap or 
>>>>>> unlock steps) ?
>>>>>
>>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>>
>>>> I don't want to nest/overlap them. My question was whether that 
>>>> would be required. Apparently not.
>>>>
>>>> While the console's BO is being set for scanout, it's protected from 
>>>> movement via the pin/unpin implementation, right?
>>>
>>> Yes, correct.
>>>
>>>> The driver does not acquire the resv lock for longer periods. I'm 
>>>> asking because this would prevent any console-buffer updates while 
>>>> the console is being displayed.
>>>
>>> Correct as well, we only hold the lock for things like command 
>>> submission, pinning, unpinning etc etc....
>>>
>>
>> Thanks for answering my questions.
>>
>>>>
>>>>>
>>>>> You need to make sure that the lock is only taken from the FB path 
>>>>> which wants to vmap the object.
>>>>>
>>>>> Why don't you lock the GEM object from the caller in the generic FB 
>>>>> implementation?
>>>>
>>>> With the current blitter code, it breaks abstraction. if vmap/vunmap 
>>>> hold the lock implicitly, things would be easier.
>>>
>>> Do you have a link to the code?
>>
>> It's the damage blitter in the fbdev code. [1] While it flushes the 
>> shadow buffer into the BO, the BO has to be kept in place. I already 
>> changed it to lock struct drm_fb_helper.lock, but I don't think this 
>> is enough. TTM could still evict the BO concurrently.
> 
> Yeah, that's correct.
> 
> But I still don't fully understand the problem. You just need to change 
> the code like this:
> 
>      mutex_lock(&fb_helper->lock);
>      dma_resv_lock(buffer->gem->resv, NULL);
> 
>      ret = drm_client_buffer_vmap(buffer, &map);
>      if (ret)
>          goto out;
> 
>      dst = map;
>      drm_fb_helper_damage_blit_real(fb_helper, clip, &dst);
> 
>      drm_client_buffer_vunmap(buffer);
> 
> out:
>      dma_resv_unlock(buffer->gem->resv);
>      mutex_unlock(&fb_helper->lock);
> 

Yes, that's the code I had in mind.

> 
> You could abstract that in drm_client functions as well, but I don't 
> really see the value in that.

The fbdev code tries hard to not use GEM directly, but to wrap 
everything behind client interfaces. I'm not sure if I like that, but 
for now I'd stick to this design.

Best regards
Thomas

> 
> Regards,
> Christian.
> 
>> There's no recursion taking place, so I guess the reservation lock 
>> could be acquired/release in drm_client_buffer_vmap/vunmap(), or a 
>> separate pair of DRM client functions could do the locking.
>>
>> Best regards
>> Thomas
>>
>> [1] 
>> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394 
>>
>>
>>>
>>> Please note that the reservation lock you need to take here is part 
>>> of the GEM object.
>>>
>>> Usually we design things in the way that the code needs to take a 
>>> lock which protects an object, then do some operations with the 
>>> object and then release the lock again.
>>>
>>> Having in the lock inside the operation can be done as well, but 
>>> returning with it is kind of unusual design.
>>>
>>>> Sorry for the noob questions. I'm still trying to understand the 
>>>> implications of acquiring these locks.
>>>
>>> Well this is the reservation lock of the GEM object we are talking 
>>> about here. We need to take that for a couple of different 
>>> operations, vmap/vunmap doesn't sound like a special case to me.
>>>
>>> Regards,
>>> Christian.
>>>
>>>>
>>>> Best regards
>>>> Thomas
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 5736 bytes --]

Hi

Am 24.11.20 um 15:06 schrieb Christian König:
> Am 24.11.20 um 14:56 schrieb Thomas Zimmermann:
>> Hi
>>
>> Am 24.11.20 um 14:36 schrieb Christian König:
>>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>>> [SNIP]
>>>>>>>> First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but 
>>>>>>>> then wondered why ttm_bo_vmap() doe not acquire the lock 
>>>>>>>> internally? I'd expect that vmap/vunmap are close together and 
>>>>>>>> do not overlap for the same BO. 
>>>>>>>
>>>>>>> We have use cases like the following during command submission:
>>>>>>>
>>>>>>> 1. lock
>>>>>>> 2. map
>>>>>>> 3. copy parts of the BO content somewhere else or patch it with 
>>>>>>> additional information
>>>>>>> 4. unmap
>>>>>>> 5. submit BO to the hardware
>>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>>> 7. unlock
>>>>>>>
>>>>>>> That use case won't be possible with vmap/vunmap if we move the 
>>>>>>> lock/unlock into it and I hope to replace the kmap/kunmap 
>>>>>>> functions with them in the near term.
>>>>>>>
>>>>>>>> Otherwise, acquiring the reservation lock would require another 
>>>>>>>> ref-counting variable or per-driver code.
>>>>>>>
>>>>>>> Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() 
>>>>>>> helper as you initially planned.
>>>>>>
>>>>>> Given your example above, step one would acquire the lock, and 
>>>>>> step two would also acquire the lock as part of the vmap 
>>>>>> implementation. Wouldn't this fail (At least during unmap or 
>>>>>> unlock steps) ?
>>>>>
>>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>>
>>>> I don't want to nest/overlap them. My question was whether that 
>>>> would be required. Apparently not.
>>>>
>>>> While the console's BO is being set for scanout, it's protected from 
>>>> movement via the pin/unpin implementation, right?
>>>
>>> Yes, correct.
>>>
>>>> The driver does not acquire the resv lock for longer periods. I'm 
>>>> asking because this would prevent any console-buffer updates while 
>>>> the console is being displayed.
>>>
>>> Correct as well, we only hold the lock for things like command 
>>> submission, pinning, unpinning etc etc....
>>>
>>
>> Thanks for answering my questions.
>>
>>>>
>>>>>
>>>>> You need to make sure that the lock is only taken from the FB path 
>>>>> which wants to vmap the object.
>>>>>
>>>>> Why don't you lock the GEM object from the caller in the generic FB 
>>>>> implementation?
>>>>
>>>> With the current blitter code, it breaks abstraction. if vmap/vunmap 
>>>> hold the lock implicitly, things would be easier.
>>>
>>> Do you have a link to the code?
>>
>> It's the damage blitter in the fbdev code. [1] While it flushes the 
>> shadow buffer into the BO, the BO has to be kept in place. I already 
>> changed it to lock struct drm_fb_helper.lock, but I don't think this 
>> is enough. TTM could still evict the BO concurrently.
> 
> Yeah, that's correct.
> 
> But I still don't fully understand the problem. You just need to change 
> the code like this:
> 
>      mutex_lock(&fb_helper->lock);
>      dma_resv_lock(buffer->gem->resv, NULL);
> 
>      ret = drm_client_buffer_vmap(buffer, &map);
>      if (ret)
>          goto out;
> 
>      dst = map;
>      drm_fb_helper_damage_blit_real(fb_helper, clip, &dst);
> 
>      drm_client_buffer_vunmap(buffer);
> 
> out:
>      dma_resv_unlock(buffer->gem->resv);
>      mutex_unlock(&fb_helper->lock);
> 

Yes, that's the code I had in mind.

> 
> You could abstract that in drm_client functions as well, but I don't 
> really see the value in that.

The fbdev code tries hard to not use GEM directly, but to wrap 
everything behind client interfaces. I'm not sure if I like that, but 
for now I'd stick to this design.

Best regards
Thomas

> 
> Regards,
> Christian.
> 
>> There's no recursion taking place, so I guess the reservation lock 
>> could be acquired/release in drm_client_buffer_vmap/vunmap(), or a 
>> separate pair of DRM client functions could do the locking.
>>
>> Best regards
>> Thomas
>>
>> [1] 
>> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394 
>>
>>
>>>
>>> Please note that the reservation lock you need to take here is part 
>>> of the GEM object.
>>>
>>> Usually we design things in the way that the code needs to take a 
>>> lock which protects an object, then do some operations with the 
>>> object and then release the lock again.
>>>
>>> Having in the lock inside the operation can be done as well, but 
>>> returning with it is kind of unusual design.
>>>
>>>> Sorry for the noob questions. I'm still trying to understand the 
>>>> implications of acquiring these locks.
>>>
>>> Well this is the reservation lock of the GEM object we are talking 
>>> about here. We need to take that for a couple of different 
>>> operations, vmap/vunmap doesn't sound like a special case to me.
>>>
>>> Regards,
>>> Christian.
>>>
>>>>
>>>> Best regards
>>>> Thomas
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 6541 bytes --]

Hi

Am 24.11.20 um 15:09 schrieb Daniel Vetter:
> On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
>> Hi
>>
>> Am 24.11.20 um 14:36 schrieb Christian König:
>>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>>> [SNIP]
>>>>>>>> First I wanted to put this into
>>>>>>>> drm_gem_ttm_vmap/vunmap(), but then wondered why
>>>>>>>> ttm_bo_vmap() doe not acquire the lock internally?
>>>>>>>> I'd expect that vmap/vunmap are close together and
>>>>>>>> do not overlap for the same BO.
>>>>>>>
>>>>>>> We have use cases like the following during command submission:
>>>>>>>
>>>>>>> 1. lock
>>>>>>> 2. map
>>>>>>> 3. copy parts of the BO content somewhere else or patch
>>>>>>> it with additional information
>>>>>>> 4. unmap
>>>>>>> 5. submit BO to the hardware
>>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>>> 7. unlock
>>>>>>>
>>>>>>> That use case won't be possible with vmap/vunmap if we
>>>>>>> move the lock/unlock into it and I hope to replace the
>>>>>>> kmap/kunmap functions with them in the near term.
>>>>>>>
>>>>>>>> Otherwise, acquiring the reservation lock would
>>>>>>>> require another ref-counting variable or per-driver
>>>>>>>> code.
>>>>>>>
>>>>>>> Hui, why that? Just put this into
>>>>>>> drm_gem_ttm_vmap/vunmap() helper as you initially
>>>>>>> planned.
>>>>>>
>>>>>> Given your example above, step one would acquire the lock,
>>>>>> and step two would also acquire the lock as part of the vmap
>>>>>> implementation. Wouldn't this fail (At least during unmap or
>>>>>> unlock steps) ?
>>>>>
>>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>>
>>>> I don't want to nest/overlap them. My question was whether that
>>>> would be required. Apparently not.
>>>>
>>>> While the console's BO is being set for scanout, it's protected from
>>>> movement via the pin/unpin implementation, right?
>>>
>>> Yes, correct.
>>>
>>>> The driver does not acquire the resv lock for longer periods. I'm
>>>> asking because this would prevent any console-buffer updates while
>>>> the console is being displayed.
>>>
>>> Correct as well, we only hold the lock for things like command
>>> submission, pinning, unpinning etc etc....
>>>
>>
>> Thanks for answering my questions.
>>
>>>>
>>>>>
>>>>> You need to make sure that the lock is only taken from the FB
>>>>> path which wants to vmap the object.
>>>>>
>>>>> Why don't you lock the GEM object from the caller in the generic
>>>>> FB implementation?
>>>>
>>>> With the current blitter code, it breaks abstraction. if vmap/vunmap
>>>> hold the lock implicitly, things would be easier.
>>>
>>> Do you have a link to the code?
>>
>> It's the damage blitter in the fbdev code. [1] While it flushes the shadow
>> buffer into the BO, the BO has to be kept in place. I already changed it to
>> lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could
>> still evict the BO concurrently.
> 
> So I'm not sure this is actually a problem: ttm could try to concurrently
> evict the buffer we pinned into vram, and then just skip to the next one.
> 
> Plus atm generic fbdev isn't used on any chip where we really care about
> that last few mb of vram being useable for command submission (well atm
> there's no driver using it).

Well, this is the patchset for radeon. If it works out, amdgpu and 
nouveau are natural next choices. Especially radeon and nouveau support 
cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly 
matter.

> 
> Having the buffer pinned into system memory and trying to do a concurrent
> modeset that tries to pull it in is the hard failure mode. And holding
> fb_helper.lock fully prevents that.
> 
> So not really clear on what failure mode you're seeing here?

Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland 
is running.) The fbdev BO is a few MiBs and not in use, so TTM would 
want to evict it if memory gets tight.

What I have in mind is a concurrent modeset that requires the memory. 
If we do a concurrent damage blit without protecting against eviction, 
things go boom. Same for concurrent 3d graphics with textures, model 
data, etc.

Best regards
Thomas

> 
>> There's no recursion taking place, so I guess the reservation lock could be
>> acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
>> DRM client functions could do the locking.
> 
> Given how this "do the right locking" is a can of worms (and I think it's
> worse than what you dug out already) I think the fb_helper.lock hack is
> perfectly good enough.
> 
> I'm also somewhat worried that starting to use dma_resv lock in generic
> code, while many helpers/drivers still have their hand-rolled locking,
> will make conversion over to dma_resv needlessly more complicated.
> -Daniel
> 
>>
>> Best regards
>> Thomas
>>
>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>
>>>
>>> Please note that the reservation lock you need to take here is part of
>>> the GEM object.
>>>
>>> Usually we design things in the way that the code needs to take a lock
>>> which protects an object, then do some operations with the object and
>>> then release the lock again.
>>>
>>> Having in the lock inside the operation can be done as well, but
>>> returning with it is kind of unusual design.
>>>
>>>> Sorry for the noob questions. I'm still trying to understand the
>>>> implications of acquiring these locks.
>>>
>>> Well this is the reservation lock of the GEM object we are talking about
>>> here. We need to take that for a couple of different operations,
>>> vmap/vunmap doesn't sound like a special case to me.
>>>
>>> Regards,
>>> Christian.
>>>
>>>>
>>>> Best regards
>>>> Thomas
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>
>> -- 
>> Thomas Zimmermann
>> Graphics Driver Developer
>> SUSE Software Solutions Germany GmbH
>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>> (HRB 36809, AG Nürnberg)
>> Geschäftsführer: Felix Imendörffer
> 
> 
> 
> 
> 
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 6541 bytes --]

Hi

Am 24.11.20 um 15:09 schrieb Daniel Vetter:
> On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
>> Hi
>>
>> Am 24.11.20 um 14:36 schrieb Christian König:
>>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>>> [SNIP]
>>>>>>>> First I wanted to put this into
>>>>>>>> drm_gem_ttm_vmap/vunmap(), but then wondered why
>>>>>>>> ttm_bo_vmap() doe not acquire the lock internally?
>>>>>>>> I'd expect that vmap/vunmap are close together and
>>>>>>>> do not overlap for the same BO.
>>>>>>>
>>>>>>> We have use cases like the following during command submission:
>>>>>>>
>>>>>>> 1. lock
>>>>>>> 2. map
>>>>>>> 3. copy parts of the BO content somewhere else or patch
>>>>>>> it with additional information
>>>>>>> 4. unmap
>>>>>>> 5. submit BO to the hardware
>>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>>> 7. unlock
>>>>>>>
>>>>>>> That use case won't be possible with vmap/vunmap if we
>>>>>>> move the lock/unlock into it and I hope to replace the
>>>>>>> kmap/kunmap functions with them in the near term.
>>>>>>>
>>>>>>>> Otherwise, acquiring the reservation lock would
>>>>>>>> require another ref-counting variable or per-driver
>>>>>>>> code.
>>>>>>>
>>>>>>> Hui, why that? Just put this into
>>>>>>> drm_gem_ttm_vmap/vunmap() helper as you initially
>>>>>>> planned.
>>>>>>
>>>>>> Given your example above, step one would acquire the lock,
>>>>>> and step two would also acquire the lock as part of the vmap
>>>>>> implementation. Wouldn't this fail (At least during unmap or
>>>>>> unlock steps) ?
>>>>>
>>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>>
>>>> I don't want to nest/overlap them. My question was whether that
>>>> would be required. Apparently not.
>>>>
>>>> While the console's BO is being set for scanout, it's protected from
>>>> movement via the pin/unpin implementation, right?
>>>
>>> Yes, correct.
>>>
>>>> The driver does not acquire the resv lock for longer periods. I'm
>>>> asking because this would prevent any console-buffer updates while
>>>> the console is being displayed.
>>>
>>> Correct as well, we only hold the lock for things like command
>>> submission, pinning, unpinning etc etc....
>>>
>>
>> Thanks for answering my questions.
>>
>>>>
>>>>>
>>>>> You need to make sure that the lock is only taken from the FB
>>>>> path which wants to vmap the object.
>>>>>
>>>>> Why don't you lock the GEM object from the caller in the generic
>>>>> FB implementation?
>>>>
>>>> With the current blitter code, it breaks abstraction. if vmap/vunmap
>>>> hold the lock implicitly, things would be easier.
>>>
>>> Do you have a link to the code?
>>
>> It's the damage blitter in the fbdev code. [1] While it flushes the shadow
>> buffer into the BO, the BO has to be kept in place. I already changed it to
>> lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could
>> still evict the BO concurrently.
> 
> So I'm not sure this is actually a problem: ttm could try to concurrently
> evict the buffer we pinned into vram, and then just skip to the next one.
> 
> Plus atm generic fbdev isn't used on any chip where we really care about
> that last few mb of vram being useable for command submission (well atm
> there's no driver using it).

Well, this is the patchset for radeon. If it works out, amdgpu and 
nouveau are natural next choices. Especially radeon and nouveau support 
cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly 
matter.

> 
> Having the buffer pinned into system memory and trying to do a concurrent
> modeset that tries to pull it in is the hard failure mode. And holding
> fb_helper.lock fully prevents that.
> 
> So not really clear on what failure mode you're seeing here?

Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland 
is running.) The fbdev BO is a few MiBs and not in use, so TTM would 
want to evict it if memory gets tight.

What I have in mind is a concurrent modeset that requires the memory. 
If we do a concurrent damage blit without protecting against eviction, 
things go boom. Same for concurrent 3d graphics with textures, model 
data, etc.

Best regards
Thomas

> 
>> There's no recursion taking place, so I guess the reservation lock could be
>> acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
>> DRM client functions could do the locking.
> 
> Given how this "do the right locking" is a can of worms (and I think it's
> worse than what you dug out already) I think the fb_helper.lock hack is
> perfectly good enough.
> 
> I'm also somewhat worried that starting to use dma_resv lock in generic
> code, while many helpers/drivers still have their hand-rolled locking,
> will make conversion over to dma_resv needlessly more complicated.
> -Daniel
> 
>>
>> Best regards
>> Thomas
>>
>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>
>>>
>>> Please note that the reservation lock you need to take here is part of
>>> the GEM object.
>>>
>>> Usually we design things in the way that the code needs to take a lock
>>> which protects an object, then do some operations with the object and
>>> then release the lock again.
>>>
>>> Having in the lock inside the operation can be done as well, but
>>> returning with it is kind of unusual design.
>>>
>>>> Sorry for the noob questions. I'm still trying to understand the
>>>> implications of acquiring these locks.
>>>
>>> Well this is the reservation lock of the GEM object we are talking about
>>> here. We need to take that for a couple of different operations,
>>> vmap/vunmap doesn't sound like a special case to me.
>>>
>>> Regards,
>>> Christian.
>>>
>>>>
>>>> Best regards
>>>> Thomas
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>
>> -- 
>> Thomas Zimmermann
>> Graphics Driver Developer
>> SUSE Software Solutions Germany GmbH
>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>> (HRB 36809, AG Nürnberg)
>> Geschäftsführer: Felix Imendörffer
> 
> 
> 
> 
> 
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 7535 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:
> Hi
>
> Am 24.11.20 um 15:09 schrieb Daniel Vetter:
>> On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
>>> Hi
>>>
>>> Am 24.11.20 um 14:36 schrieb Christian König:
>>>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>>>> [SNIP]
>>>>>>>>> First I wanted to put this into
>>>>>>>>> drm_gem_ttm_vmap/vunmap(), but then wondered why
>>>>>>>>> ttm_bo_vmap() doe not acquire the lock internally?
>>>>>>>>> I'd expect that vmap/vunmap are close together and
>>>>>>>>> do not overlap for the same BO.
>>>>>>>>
>>>>>>>> We have use cases like the following during command submission:
>>>>>>>>
>>>>>>>> 1. lock
>>>>>>>> 2. map
>>>>>>>> 3. copy parts of the BO content somewhere else or patch
>>>>>>>> it with additional information
>>>>>>>> 4. unmap
>>>>>>>> 5. submit BO to the hardware
>>>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>>>> 7. unlock
>>>>>>>>
>>>>>>>> That use case won't be possible with vmap/vunmap if we
>>>>>>>> move the lock/unlock into it and I hope to replace the
>>>>>>>> kmap/kunmap functions with them in the near term.
>>>>>>>>
>>>>>>>>> Otherwise, acquiring the reservation lock would
>>>>>>>>> require another ref-counting variable or per-driver
>>>>>>>>> code.
>>>>>>>>
>>>>>>>> Hui, why that? Just put this into
>>>>>>>> drm_gem_ttm_vmap/vunmap() helper as you initially
>>>>>>>> planned.
>>>>>>>
>>>>>>> Given your example above, step one would acquire the lock,
>>>>>>> and step two would also acquire the lock as part of the vmap
>>>>>>> implementation. Wouldn't this fail (At least during unmap or
>>>>>>> unlock steps) ?
>>>>>>
>>>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>>>
>>>>> I don't want to nest/overlap them. My question was whether that
>>>>> would be required. Apparently not.
>>>>>
>>>>> While the console's BO is being set for scanout, it's protected from
>>>>> movement via the pin/unpin implementation, right?
>>>>
>>>> Yes, correct.
>>>>
>>>>> The driver does not acquire the resv lock for longer periods. I'm
>>>>> asking because this would prevent any console-buffer updates while
>>>>> the console is being displayed.
>>>>
>>>> Correct as well, we only hold the lock for things like command
>>>> submission, pinning, unpinning etc etc....
>>>>
>>>
>>> Thanks for answering my questions.
>>>
>>>>>
>>>>>>
>>>>>> You need to make sure that the lock is only taken from the FB
>>>>>> path which wants to vmap the object.
>>>>>>
>>>>>> Why don't you lock the GEM object from the caller in the generic
>>>>>> FB implementation?
>>>>>
>>>>> With the current blitter code, it breaks abstraction. if vmap/vunmap
>>>>> hold the lock implicitly, things would be easier.
>>>>
>>>> Do you have a link to the code?
>>>
>>> It's the damage blitter in the fbdev code. [1] While it flushes the 
>>> shadow
>>> buffer into the BO, the BO has to be kept in place. I already 
>>> changed it to
>>> lock struct drm_fb_helper.lock, but I don't think this is enough. 
>>> TTM could
>>> still evict the BO concurrently.
>>
>> So I'm not sure this is actually a problem: ttm could try to 
>> concurrently
>> evict the buffer we pinned into vram, and then just skip to the next 
>> one.
>>
>> Plus atm generic fbdev isn't used on any chip where we really care about
>> that last few mb of vram being useable for command submission (well atm
>> there's no driver using it).
>
> Well, this is the patchset for radeon. If it works out, amdgpu and 
> nouveau are natural next choices. Especially radeon and nouveau 
> support cards with low- to medium-sized VRAM. The MiBs wasted on fbdev 
> certainly matter.
>
>>
>> Having the buffer pinned into system memory and trying to do a 
>> concurrent
>> modeset that tries to pull it in is the hard failure mode. And holding
>> fb_helper.lock fully prevents that.
>>
>> So not really clear on what failure mode you're seeing here?
>
> Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or 
> Wayland is running.) The fbdev BO is a few MiBs and not in use, so TTM 
> would want to evict it if memory gets tight.
>
> What I have in mind is a concurrent modeset that requires the memory. 
> If we do a concurrent damage blit without protecting against eviction, 
> things go boom. Same for concurrent 3d graphics with textures, model 
> data, etc.

Completely agree.

This needs proper lock protection of the memory mapped buffer. Relying 
on that some other code isn't run because we have some third part locks 
taken is not sufficient here.

Regards,
Christian.

>
> Best regards
> Thomas
>
>>
>>> There's no recursion taking place, so I guess the reservation lock 
>>> could be
>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a separate 
>>> pair of
>>> DRM client functions could do the locking.
>>
>> Given how this "do the right locking" is a can of worms (and I think 
>> it's
>> worse than what you dug out already) I think the fb_helper.lock hack is
>> perfectly good enough.
>>
>> I'm also somewhat worried that starting to use dma_resv lock in generic
>> code, while many helpers/drivers still have their hand-rolled locking,
>> will make conversion over to dma_resv needlessly more complicated.
>> -Daniel
>>
>>>
>>> Best regards
>>> Thomas
>>>
>>> [1] 
>>> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>
>>>>
>>>> Please note that the reservation lock you need to take here is part of
>>>> the GEM object.
>>>>
>>>> Usually we design things in the way that the code needs to take a lock
>>>> which protects an object, then do some operations with the object and
>>>> then release the lock again.
>>>>
>>>> Having in the lock inside the operation can be done as well, but
>>>> returning with it is kind of unusual design.
>>>>
>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>> implications of acquiring these locks.
>>>>
>>>> Well this is the reservation lock of the GEM object we are talking 
>>>> about
>>>> here. We need to take that for a couple of different operations,
>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>
>>>> _______________________________________________
>>>> dri-devel mailing list
>>>> dri-devel@lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>
>>> -- 
>>> Thomas Zimmermann
>>> Graphics Driver Developer
>>> SUSE Software Solutions Germany GmbH
>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>> (HRB 36809, AG Nürnberg)
>>> Geschäftsführer: Felix Imendörffer
>>
>>
>>
>>
>>
>>
>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:
> Hi
>
> Am 24.11.20 um 15:09 schrieb Daniel Vetter:
>> On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
>>> Hi
>>>
>>> Am 24.11.20 um 14:36 schrieb Christian König:
>>>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>>>> [SNIP]
>>>>>>>>> First I wanted to put this into
>>>>>>>>> drm_gem_ttm_vmap/vunmap(), but then wondered why
>>>>>>>>> ttm_bo_vmap() doe not acquire the lock internally?
>>>>>>>>> I'd expect that vmap/vunmap are close together and
>>>>>>>>> do not overlap for the same BO.
>>>>>>>>
>>>>>>>> We have use cases like the following during command submission:
>>>>>>>>
>>>>>>>> 1. lock
>>>>>>>> 2. map
>>>>>>>> 3. copy parts of the BO content somewhere else or patch
>>>>>>>> it with additional information
>>>>>>>> 4. unmap
>>>>>>>> 5. submit BO to the hardware
>>>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>>>> 7. unlock
>>>>>>>>
>>>>>>>> That use case won't be possible with vmap/vunmap if we
>>>>>>>> move the lock/unlock into it and I hope to replace the
>>>>>>>> kmap/kunmap functions with them in the near term.
>>>>>>>>
>>>>>>>>> Otherwise, acquiring the reservation lock would
>>>>>>>>> require another ref-counting variable or per-driver
>>>>>>>>> code.
>>>>>>>>
>>>>>>>> Hui, why that? Just put this into
>>>>>>>> drm_gem_ttm_vmap/vunmap() helper as you initially
>>>>>>>> planned.
>>>>>>>
>>>>>>> Given your example above, step one would acquire the lock,
>>>>>>> and step two would also acquire the lock as part of the vmap
>>>>>>> implementation. Wouldn't this fail (At least during unmap or
>>>>>>> unlock steps) ?
>>>>>>
>>>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>>>
>>>>> I don't want to nest/overlap them. My question was whether that
>>>>> would be required. Apparently not.
>>>>>
>>>>> While the console's BO is being set for scanout, it's protected from
>>>>> movement via the pin/unpin implementation, right?
>>>>
>>>> Yes, correct.
>>>>
>>>>> The driver does not acquire the resv lock for longer periods. I'm
>>>>> asking because this would prevent any console-buffer updates while
>>>>> the console is being displayed.
>>>>
>>>> Correct as well, we only hold the lock for things like command
>>>> submission, pinning, unpinning etc etc....
>>>>
>>>
>>> Thanks for answering my questions.
>>>
>>>>>
>>>>>>
>>>>>> You need to make sure that the lock is only taken from the FB
>>>>>> path which wants to vmap the object.
>>>>>>
>>>>>> Why don't you lock the GEM object from the caller in the generic
>>>>>> FB implementation?
>>>>>
>>>>> With the current blitter code, it breaks abstraction. if vmap/vunmap
>>>>> hold the lock implicitly, things would be easier.
>>>>
>>>> Do you have a link to the code?
>>>
>>> It's the damage blitter in the fbdev code. [1] While it flushes the 
>>> shadow
>>> buffer into the BO, the BO has to be kept in place. I already 
>>> changed it to
>>> lock struct drm_fb_helper.lock, but I don't think this is enough. 
>>> TTM could
>>> still evict the BO concurrently.
>>
>> So I'm not sure this is actually a problem: ttm could try to 
>> concurrently
>> evict the buffer we pinned into vram, and then just skip to the next 
>> one.
>>
>> Plus atm generic fbdev isn't used on any chip where we really care about
>> that last few mb of vram being useable for command submission (well atm
>> there's no driver using it).
>
> Well, this is the patchset for radeon. If it works out, amdgpu and 
> nouveau are natural next choices. Especially radeon and nouveau 
> support cards with low- to medium-sized VRAM. The MiBs wasted on fbdev 
> certainly matter.
>
>>
>> Having the buffer pinned into system memory and trying to do a 
>> concurrent
>> modeset that tries to pull it in is the hard failure mode. And holding
>> fb_helper.lock fully prevents that.
>>
>> So not really clear on what failure mode you're seeing here?
>
> Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or 
> Wayland is running.) The fbdev BO is a few MiBs and not in use, so TTM 
> would want to evict it if memory gets tight.
>
> What I have in mind is a concurrent modeset that requires the memory. 
> If we do a concurrent damage blit without protecting against eviction, 
> things go boom. Same for concurrent 3d graphics with textures, model 
> data, etc.

Completely agree.

This needs proper lock protection of the memory mapped buffer. Relying 
on that some other code isn't run because we have some third part locks 
taken is not sufficient here.

Regards,
Christian.

>
> Best regards
> Thomas
>
>>
>>> There's no recursion taking place, so I guess the reservation lock 
>>> could be
>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a separate 
>>> pair of
>>> DRM client functions could do the locking.
>>
>> Given how this "do the right locking" is a can of worms (and I think 
>> it's
>> worse than what you dug out already) I think the fb_helper.lock hack is
>> perfectly good enough.
>>
>> I'm also somewhat worried that starting to use dma_resv lock in generic
>> code, while many helpers/drivers still have their hand-rolled locking,
>> will make conversion over to dma_resv needlessly more complicated.
>> -Daniel
>>
>>>
>>> Best regards
>>> Thomas
>>>
>>> [1] 
>>> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>
>>>>
>>>> Please note that the reservation lock you need to take here is part of
>>>> the GEM object.
>>>>
>>>> Usually we design things in the way that the code needs to take a lock
>>>> which protects an object, then do some operations with the object and
>>>> then release the lock again.
>>>>
>>>> Having in the lock inside the operation can be done as well, but
>>>> returning with it is kind of unusual design.
>>>>
>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>> implications of acquiring these locks.
>>>>
>>>> Well this is the reservation lock of the GEM object we are talking 
>>>> about
>>>> here. We need to take that for a couple of different operations,
>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>
>>>> _______________________________________________
>>>> dri-devel mailing list
>>>> dri-devel@lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>
>>> -- 
>>> Thomas Zimmermann
>>> Graphics Driver Developer
>>> SUSE Software Solutions Germany GmbH
>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>> (HRB 36809, AG Nürnberg)
>>> Geschäftsführer: Felix Imendörffer
>>
>>
>>
>>
>>
>>
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Daniel Vetter]

On Wed, Nov 25, 2020 at 11:13:13AM +0100, Christian König wrote:
> Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:
> > Hi
> > 
> > Am 24.11.20 um 15:09 schrieb Daniel Vetter:
> > > On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
> > > > Hi
> > > > 
> > > > Am 24.11.20 um 14:36 schrieb Christian König:
> > > > > Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
> > > > > > [SNIP]
> > > > > > > > > > First I wanted to put this into
> > > > > > > > > > drm_gem_ttm_vmap/vunmap(), but then wondered why
> > > > > > > > > > ttm_bo_vmap() doe not acquire the lock internally?
> > > > > > > > > > I'd expect that vmap/vunmap are close together and
> > > > > > > > > > do not overlap for the same BO.
> > > > > > > > > 
> > > > > > > > > We have use cases like the following during command submission:
> > > > > > > > > 
> > > > > > > > > 1. lock
> > > > > > > > > 2. map
> > > > > > > > > 3. copy parts of the BO content somewhere else or patch
> > > > > > > > > it with additional information
> > > > > > > > > 4. unmap
> > > > > > > > > 5. submit BO to the hardware
> > > > > > > > > 6. add hardware fence to the BO to make sure it doesn't move
> > > > > > > > > 7. unlock
> > > > > > > > > 
> > > > > > > > > That use case won't be possible with vmap/vunmap if we
> > > > > > > > > move the lock/unlock into it and I hope to replace the
> > > > > > > > > kmap/kunmap functions with them in the near term.
> > > > > > > > > 
> > > > > > > > > > Otherwise, acquiring the reservation lock would
> > > > > > > > > > require another ref-counting variable or per-driver
> > > > > > > > > > code.
> > > > > > > > > 
> > > > > > > > > Hui, why that? Just put this into
> > > > > > > > > drm_gem_ttm_vmap/vunmap() helper as you initially
> > > > > > > > > planned.
> > > > > > > > 
> > > > > > > > Given your example above, step one would acquire the lock,
> > > > > > > > and step two would also acquire the lock as part of the vmap
> > > > > > > > implementation. Wouldn't this fail (At least during unmap or
> > > > > > > > unlock steps) ?
> > > > > > > 
> > > > > > > Oh, so you want to nest them? No, that is a rather bad no-go.
> > > > > > 
> > > > > > I don't want to nest/overlap them. My question was whether that
> > > > > > would be required. Apparently not.
> > > > > > 
> > > > > > While the console's BO is being set for scanout, it's protected from
> > > > > > movement via the pin/unpin implementation, right?
> > > > > 
> > > > > Yes, correct.
> > > > > 
> > > > > > The driver does not acquire the resv lock for longer periods. I'm
> > > > > > asking because this would prevent any console-buffer updates while
> > > > > > the console is being displayed.
> > > > > 
> > > > > Correct as well, we only hold the lock for things like command
> > > > > submission, pinning, unpinning etc etc....
> > > > > 
> > > > 
> > > > Thanks for answering my questions.
> > > > 
> > > > > > 
> > > > > > > 
> > > > > > > You need to make sure that the lock is only taken from the FB
> > > > > > > path which wants to vmap the object.
> > > > > > > 
> > > > > > > Why don't you lock the GEM object from the caller in the generic
> > > > > > > FB implementation?
> > > > > > 
> > > > > > With the current blitter code, it breaks abstraction. if vmap/vunmap
> > > > > > hold the lock implicitly, things would be easier.
> > > > > 
> > > > > Do you have a link to the code?
> > > > 
> > > > It's the damage blitter in the fbdev code. [1] While it flushes
> > > > the shadow
> > > > buffer into the BO, the BO has to be kept in place. I already
> > > > changed it to
> > > > lock struct drm_fb_helper.lock, but I don't think this is
> > > > enough. TTM could
> > > > still evict the BO concurrently.
> > > 
> > > So I'm not sure this is actually a problem: ttm could try to
> > > concurrently
> > > evict the buffer we pinned into vram, and then just skip to the next
> > > one.
> > > 
> > > Plus atm generic fbdev isn't used on any chip where we really care about
> > > that last few mb of vram being useable for command submission (well atm
> > > there's no driver using it).
> > 
> > Well, this is the patchset for radeon. If it works out, amdgpu and
> > nouveau are natural next choices. Especially radeon and nouveau support
> > cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly
> > matter.
> > 
> > > 
> > > Having the buffer pinned into system memory and trying to do a
> > > concurrent
> > > modeset that tries to pull it in is the hard failure mode. And holding
> > > fb_helper.lock fully prevents that.
> > > 
> > > So not really clear on what failure mode you're seeing here?
> > 
> > Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland
> > is running.) The fbdev BO is a few MiBs and not in use, so TTM would
> > want to evict it if memory gets tight.
> > 
> > What I have in mind is a concurrent modeset that requires the memory. If
> > we do a concurrent damage blit without protecting against eviction,
> > things go boom. Same for concurrent 3d graphics with textures, model
> > data, etc.
> 
> Completely agree.
> 
> This needs proper lock protection of the memory mapped buffer. Relying on
> that some other code isn't run because we have some third part locks taken
> is not sufficient here.

We are still protected by the pin count in this scenario. Plus, with
current drivers we always pin the fbdev buffer into vram, so occasionally
failing to move it out isn't a regression.

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but
the issue there is that beyond ttm, none of the helpers (and few of the
drivers) use dma_resv. So this is a fairly big uphill battle. Quick
interim fix seems like the right solution to me.
-Daniel

> 
> Regards,
> Christian.
> 
> > 
> > Best regards
> > Thomas
> > 
> > > 
> > > > There's no recursion taking place, so I guess the reservation
> > > > lock could be
> > > > acquired/release in drm_client_buffer_vmap/vunmap(), or a
> > > > separate pair of
> > > > DRM client functions could do the locking.
> > > 
> > > Given how this "do the right locking" is a can of worms (and I think
> > > it's
> > > worse than what you dug out already) I think the fb_helper.lock hack is
> > > perfectly good enough.
> > > 
> > > I'm also somewhat worried that starting to use dma_resv lock in generic
> > > code, while many helpers/drivers still have their hand-rolled locking,
> > > will make conversion over to dma_resv needlessly more complicated.
> > > -Daniel
> > > 
> > > > 
> > > > Best regards
> > > > Thomas
> > > > 
> > > > [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
> > > > 
> > > > > 
> > > > > Please note that the reservation lock you need to take here is part of
> > > > > the GEM object.
> > > > > 
> > > > > Usually we design things in the way that the code needs to take a lock
> > > > > which protects an object, then do some operations with the object and
> > > > > then release the lock again.
> > > > > 
> > > > > Having in the lock inside the operation can be done as well, but
> > > > > returning with it is kind of unusual design.
> > > > > 
> > > > > > Sorry for the noob questions. I'm still trying to understand the
> > > > > > implications of acquiring these locks.
> > > > > 
> > > > > Well this is the reservation lock of the GEM object we are
> > > > > talking about
> > > > > here. We need to take that for a couple of different operations,
> > > > > vmap/vunmap doesn't sound like a special case to me.
> > > > > 
> > > > > Regards,
> > > > > Christian.
> > > > > 
> > > > > > 
> > > > > > Best regards
> > > > > > Thomas
> > > > > 
> > > > > _______________________________________________
> > > > > dri-devel mailing list
> > > > > dri-devel@lists.freedesktop.org
> > > > > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > > > 
> > > > -- 
> > > > Thomas Zimmermann
> > > > Graphics Driver Developer
> > > > SUSE Software Solutions Germany GmbH
> > > > Maxfeldstr. 5, 90409 Nürnberg, Germany
> > > > (HRB 36809, AG Nürnberg)
> > > > Geschäftsführer: Felix Imendörffer
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > 
> 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Daniel Vetter]

On Wed, Nov 25, 2020 at 11:13:13AM +0100, Christian König wrote:
> Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:
> > Hi
> > 
> > Am 24.11.20 um 15:09 schrieb Daniel Vetter:
> > > On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
> > > > Hi
> > > > 
> > > > Am 24.11.20 um 14:36 schrieb Christian König:
> > > > > Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
> > > > > > [SNIP]
> > > > > > > > > > First I wanted to put this into
> > > > > > > > > > drm_gem_ttm_vmap/vunmap(), but then wondered why
> > > > > > > > > > ttm_bo_vmap() doe not acquire the lock internally?
> > > > > > > > > > I'd expect that vmap/vunmap are close together and
> > > > > > > > > > do not overlap for the same BO.
> > > > > > > > > 
> > > > > > > > > We have use cases like the following during command submission:
> > > > > > > > > 
> > > > > > > > > 1. lock
> > > > > > > > > 2. map
> > > > > > > > > 3. copy parts of the BO content somewhere else or patch
> > > > > > > > > it with additional information
> > > > > > > > > 4. unmap
> > > > > > > > > 5. submit BO to the hardware
> > > > > > > > > 6. add hardware fence to the BO to make sure it doesn't move
> > > > > > > > > 7. unlock
> > > > > > > > > 
> > > > > > > > > That use case won't be possible with vmap/vunmap if we
> > > > > > > > > move the lock/unlock into it and I hope to replace the
> > > > > > > > > kmap/kunmap functions with them in the near term.
> > > > > > > > > 
> > > > > > > > > > Otherwise, acquiring the reservation lock would
> > > > > > > > > > require another ref-counting variable or per-driver
> > > > > > > > > > code.
> > > > > > > > > 
> > > > > > > > > Hui, why that? Just put this into
> > > > > > > > > drm_gem_ttm_vmap/vunmap() helper as you initially
> > > > > > > > > planned.
> > > > > > > > 
> > > > > > > > Given your example above, step one would acquire the lock,
> > > > > > > > and step two would also acquire the lock as part of the vmap
> > > > > > > > implementation. Wouldn't this fail (At least during unmap or
> > > > > > > > unlock steps) ?
> > > > > > > 
> > > > > > > Oh, so you want to nest them? No, that is a rather bad no-go.
> > > > > > 
> > > > > > I don't want to nest/overlap them. My question was whether that
> > > > > > would be required. Apparently not.
> > > > > > 
> > > > > > While the console's BO is being set for scanout, it's protected from
> > > > > > movement via the pin/unpin implementation, right?
> > > > > 
> > > > > Yes, correct.
> > > > > 
> > > > > > The driver does not acquire the resv lock for longer periods. I'm
> > > > > > asking because this would prevent any console-buffer updates while
> > > > > > the console is being displayed.
> > > > > 
> > > > > Correct as well, we only hold the lock for things like command
> > > > > submission, pinning, unpinning etc etc....
> > > > > 
> > > > 
> > > > Thanks for answering my questions.
> > > > 
> > > > > > 
> > > > > > > 
> > > > > > > You need to make sure that the lock is only taken from the FB
> > > > > > > path which wants to vmap the object.
> > > > > > > 
> > > > > > > Why don't you lock the GEM object from the caller in the generic
> > > > > > > FB implementation?
> > > > > > 
> > > > > > With the current blitter code, it breaks abstraction. if vmap/vunmap
> > > > > > hold the lock implicitly, things would be easier.
> > > > > 
> > > > > Do you have a link to the code?
> > > > 
> > > > It's the damage blitter in the fbdev code. [1] While it flushes
> > > > the shadow
> > > > buffer into the BO, the BO has to be kept in place. I already
> > > > changed it to
> > > > lock struct drm_fb_helper.lock, but I don't think this is
> > > > enough. TTM could
> > > > still evict the BO concurrently.
> > > 
> > > So I'm not sure this is actually a problem: ttm could try to
> > > concurrently
> > > evict the buffer we pinned into vram, and then just skip to the next
> > > one.
> > > 
> > > Plus atm generic fbdev isn't used on any chip where we really care about
> > > that last few mb of vram being useable for command submission (well atm
> > > there's no driver using it).
> > 
> > Well, this is the patchset for radeon. If it works out, amdgpu and
> > nouveau are natural next choices. Especially radeon and nouveau support
> > cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly
> > matter.
> > 
> > > 
> > > Having the buffer pinned into system memory and trying to do a
> > > concurrent
> > > modeset that tries to pull it in is the hard failure mode. And holding
> > > fb_helper.lock fully prevents that.
> > > 
> > > So not really clear on what failure mode you're seeing here?
> > 
> > Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland
> > is running.) The fbdev BO is a few MiBs and not in use, so TTM would
> > want to evict it if memory gets tight.
> > 
> > What I have in mind is a concurrent modeset that requires the memory. If
> > we do a concurrent damage blit without protecting against eviction,
> > things go boom. Same for concurrent 3d graphics with textures, model
> > data, etc.
> 
> Completely agree.
> 
> This needs proper lock protection of the memory mapped buffer. Relying on
> that some other code isn't run because we have some third part locks taken
> is not sufficient here.

We are still protected by the pin count in this scenario. Plus, with
current drivers we always pin the fbdev buffer into vram, so occasionally
failing to move it out isn't a regression.

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but
the issue there is that beyond ttm, none of the helpers (and few of the
drivers) use dma_resv. So this is a fairly big uphill battle. Quick
interim fix seems like the right solution to me.
-Daniel

> 
> Regards,
> Christian.
> 
> > 
> > Best regards
> > Thomas
> > 
> > > 
> > > > There's no recursion taking place, so I guess the reservation
> > > > lock could be
> > > > acquired/release in drm_client_buffer_vmap/vunmap(), or a
> > > > separate pair of
> > > > DRM client functions could do the locking.
> > > 
> > > Given how this "do the right locking" is a can of worms (and I think
> > > it's
> > > worse than what you dug out already) I think the fb_helper.lock hack is
> > > perfectly good enough.
> > > 
> > > I'm also somewhat worried that starting to use dma_resv lock in generic
> > > code, while many helpers/drivers still have their hand-rolled locking,
> > > will make conversion over to dma_resv needlessly more complicated.
> > > -Daniel
> > > 
> > > > 
> > > > Best regards
> > > > Thomas
> > > > 
> > > > [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
> > > > 
> > > > > 
> > > > > Please note that the reservation lock you need to take here is part of
> > > > > the GEM object.
> > > > > 
> > > > > Usually we design things in the way that the code needs to take a lock
> > > > > which protects an object, then do some operations with the object and
> > > > > then release the lock again.
> > > > > 
> > > > > Having in the lock inside the operation can be done as well, but
> > > > > returning with it is kind of unusual design.
> > > > > 
> > > > > > Sorry for the noob questions. I'm still trying to understand the
> > > > > > implications of acquiring these locks.
> > > > > 
> > > > > Well this is the reservation lock of the GEM object we are
> > > > > talking about
> > > > > here. We need to take that for a couple of different operations,
> > > > > vmap/vunmap doesn't sound like a special case to me.
> > > > > 
> > > > > Regards,
> > > > > Christian.
> > > > > 
> > > > > > 
> > > > > > Best regards
> > > > > > Thomas
> > > > > 
> > > > > _______________________________________________
> > > > > dri-devel mailing list
> > > > > dri-devel@lists.freedesktop.org
> > > > > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > > > 
> > > > -- 
> > > > Thomas Zimmermann
> > > > Graphics Driver Developer
> > > > SUSE Software Solutions Germany GmbH
> > > > Maxfeldstr. 5, 90409 Nürnberg, Germany
> > > > (HRB 36809, AG Nürnberg)
> > > > Geschäftsführer: Felix Imendörffer
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > 
> 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 25.11.20 um 11:36 schrieb Daniel Vetter:
> On Wed, Nov 25, 2020 at 11:13:13AM +0100, Christian König wrote:
>> Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:
>>> Hi
>>>
>>> Am 24.11.20 um 15:09 schrieb Daniel Vetter:
>>>> On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
>>>>> Hi
>>>>>
>>>>> Am 24.11.20 um 14:36 schrieb Christian König:
>>>>>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>>>>>> [SNIP]
>>>>>>>>>>> First I wanted to put this into
>>>>>>>>>>> drm_gem_ttm_vmap/vunmap(), but then wondered why
>>>>>>>>>>> ttm_bo_vmap() doe not acquire the lock internally?
>>>>>>>>>>> I'd expect that vmap/vunmap are close together and
>>>>>>>>>>> do not overlap for the same BO.
>>>>>>>>>> We have use cases like the following during command submission:
>>>>>>>>>>
>>>>>>>>>> 1. lock
>>>>>>>>>> 2. map
>>>>>>>>>> 3. copy parts of the BO content somewhere else or patch
>>>>>>>>>> it with additional information
>>>>>>>>>> 4. unmap
>>>>>>>>>> 5. submit BO to the hardware
>>>>>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>>>>>> 7. unlock
>>>>>>>>>>
>>>>>>>>>> That use case won't be possible with vmap/vunmap if we
>>>>>>>>>> move the lock/unlock into it and I hope to replace the
>>>>>>>>>> kmap/kunmap functions with them in the near term.
>>>>>>>>>>
>>>>>>>>>>> Otherwise, acquiring the reservation lock would
>>>>>>>>>>> require another ref-counting variable or per-driver
>>>>>>>>>>> code.
>>>>>>>>>> Hui, why that? Just put this into
>>>>>>>>>> drm_gem_ttm_vmap/vunmap() helper as you initially
>>>>>>>>>> planned.
>>>>>>>>> Given your example above, step one would acquire the lock,
>>>>>>>>> and step two would also acquire the lock as part of the vmap
>>>>>>>>> implementation. Wouldn't this fail (At least during unmap or
>>>>>>>>> unlock steps) ?
>>>>>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>>>>> I don't want to nest/overlap them. My question was whether that
>>>>>>> would be required. Apparently not.
>>>>>>>
>>>>>>> While the console's BO is being set for scanout, it's protected from
>>>>>>> movement via the pin/unpin implementation, right?
>>>>>> Yes, correct.
>>>>>>
>>>>>>> The driver does not acquire the resv lock for longer periods. I'm
>>>>>>> asking because this would prevent any console-buffer updates while
>>>>>>> the console is being displayed.
>>>>>> Correct as well, we only hold the lock for things like command
>>>>>> submission, pinning, unpinning etc etc....
>>>>>>
>>>>> Thanks for answering my questions.
>>>>>
>>>>>>>> You need to make sure that the lock is only taken from the FB
>>>>>>>> path which wants to vmap the object.
>>>>>>>>
>>>>>>>> Why don't you lock the GEM object from the caller in the generic
>>>>>>>> FB implementation?
>>>>>>> With the current blitter code, it breaks abstraction. if vmap/vunmap
>>>>>>> hold the lock implicitly, things would be easier.
>>>>>> Do you have a link to the code?
>>>>> It's the damage blitter in the fbdev code. [1] While it flushes
>>>>> the shadow
>>>>> buffer into the BO, the BO has to be kept in place. I already
>>>>> changed it to
>>>>> lock struct drm_fb_helper.lock, but I don't think this is
>>>>> enough. TTM could
>>>>> still evict the BO concurrently.
>>>> So I'm not sure this is actually a problem: ttm could try to
>>>> concurrently
>>>> evict the buffer we pinned into vram, and then just skip to the next
>>>> one.
>>>>
>>>> Plus atm generic fbdev isn't used on any chip where we really care about
>>>> that last few mb of vram being useable for command submission (well atm
>>>> there's no driver using it).
>>> Well, this is the patchset for radeon. If it works out, amdgpu and
>>> nouveau are natural next choices. Especially radeon and nouveau support
>>> cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly
>>> matter.
>>>
>>>> Having the buffer pinned into system memory and trying to do a
>>>> concurrent
>>>> modeset that tries to pull it in is the hard failure mode. And holding
>>>> fb_helper.lock fully prevents that.
>>>>
>>>> So not really clear on what failure mode you're seeing here?
>>> Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland
>>> is running.) The fbdev BO is a few MiBs and not in use, so TTM would
>>> want to evict it if memory gets tight.
>>>
>>> What I have in mind is a concurrent modeset that requires the memory. If
>>> we do a concurrent damage blit without protecting against eviction,
>>> things go boom. Same for concurrent 3d graphics with textures, model
>>> data, etc.
>> Completely agree.
>>
>> This needs proper lock protection of the memory mapped buffer. Relying on
>> that some other code isn't run because we have some third part locks taken
>> is not sufficient here.
> We are still protected by the pin count in this scenario. Plus, with
> current drivers we always pin the fbdev buffer into vram, so occasionally
> failing to move it out isn't a regression.
>
> So I'm still not seeing how this can go boom.

Well as far as I understand it the pin count is zero for this buffer in 
this case here :)

I might be wrong on this because I don't know the FB code at all, but 
Thomas seems to be pretty clear that this is the shadow buffer which is 
not scanned out from.

Regards,
Christian.

>
> Now long term it'd be nice to cut everything over to dma_resv locking, but
> the issue there is that beyond ttm, none of the helpers (and few of the
> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
> interim fix seems like the right solution to me.
> -Daniel
>
>> Regards,
>> Christian.
>>
>>> Best regards
>>> Thomas
>>>
>>>>> There's no recursion taking place, so I guess the reservation
>>>>> lock could be
>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>> separate pair of
>>>>> DRM client functions could do the locking.
>>>> Given how this "do the right locking" is a can of worms (and I think
>>>> it's
>>>> worse than what you dug out already) I think the fb_helper.lock hack is
>>>> perfectly good enough.
>>>>
>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
>>>> code, while many helpers/drivers still have their hand-rolled locking,
>>>> will make conversion over to dma_resv needlessly more complicated.
>>>> -Daniel
>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>>>
>>>>>> Please note that the reservation lock you need to take here is part of
>>>>>> the GEM object.
>>>>>>
>>>>>> Usually we design things in the way that the code needs to take a lock
>>>>>> which protects an object, then do some operations with the object and
>>>>>> then release the lock again.
>>>>>>
>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>> returning with it is kind of unusual design.
>>>>>>
>>>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>>>> implications of acquiring these locks.
>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>> talking about
>>>>>> here. We need to take that for a couple of different operations,
>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>> _______________________________________________
>>>>>> dri-devel mailing list
>>>>>> dri-devel@lists.freedesktop.org
>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>> -- 
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH
>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>> (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Felix Imendörffer
>>>>
>>>>
>>>>
>>>>
>>>>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 25.11.20 um 11:36 schrieb Daniel Vetter:
> On Wed, Nov 25, 2020 at 11:13:13AM +0100, Christian König wrote:
>> Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:
>>> Hi
>>>
>>> Am 24.11.20 um 15:09 schrieb Daniel Vetter:
>>>> On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
>>>>> Hi
>>>>>
>>>>> Am 24.11.20 um 14:36 schrieb Christian König:
>>>>>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>>>>>> [SNIP]
>>>>>>>>>>> First I wanted to put this into
>>>>>>>>>>> drm_gem_ttm_vmap/vunmap(), but then wondered why
>>>>>>>>>>> ttm_bo_vmap() doe not acquire the lock internally?
>>>>>>>>>>> I'd expect that vmap/vunmap are close together and
>>>>>>>>>>> do not overlap for the same BO.
>>>>>>>>>> We have use cases like the following during command submission:
>>>>>>>>>>
>>>>>>>>>> 1. lock
>>>>>>>>>> 2. map
>>>>>>>>>> 3. copy parts of the BO content somewhere else or patch
>>>>>>>>>> it with additional information
>>>>>>>>>> 4. unmap
>>>>>>>>>> 5. submit BO to the hardware
>>>>>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>>>>>> 7. unlock
>>>>>>>>>>
>>>>>>>>>> That use case won't be possible with vmap/vunmap if we
>>>>>>>>>> move the lock/unlock into it and I hope to replace the
>>>>>>>>>> kmap/kunmap functions with them in the near term.
>>>>>>>>>>
>>>>>>>>>>> Otherwise, acquiring the reservation lock would
>>>>>>>>>>> require another ref-counting variable or per-driver
>>>>>>>>>>> code.
>>>>>>>>>> Hui, why that? Just put this into
>>>>>>>>>> drm_gem_ttm_vmap/vunmap() helper as you initially
>>>>>>>>>> planned.
>>>>>>>>> Given your example above, step one would acquire the lock,
>>>>>>>>> and step two would also acquire the lock as part of the vmap
>>>>>>>>> implementation. Wouldn't this fail (At least during unmap or
>>>>>>>>> unlock steps) ?
>>>>>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>>>>> I don't want to nest/overlap them. My question was whether that
>>>>>>> would be required. Apparently not.
>>>>>>>
>>>>>>> While the console's BO is being set for scanout, it's protected from
>>>>>>> movement via the pin/unpin implementation, right?
>>>>>> Yes, correct.
>>>>>>
>>>>>>> The driver does not acquire the resv lock for longer periods. I'm
>>>>>>> asking because this would prevent any console-buffer updates while
>>>>>>> the console is being displayed.
>>>>>> Correct as well, we only hold the lock for things like command
>>>>>> submission, pinning, unpinning etc etc....
>>>>>>
>>>>> Thanks for answering my questions.
>>>>>
>>>>>>>> You need to make sure that the lock is only taken from the FB
>>>>>>>> path which wants to vmap the object.
>>>>>>>>
>>>>>>>> Why don't you lock the GEM object from the caller in the generic
>>>>>>>> FB implementation?
>>>>>>> With the current blitter code, it breaks abstraction. if vmap/vunmap
>>>>>>> hold the lock implicitly, things would be easier.
>>>>>> Do you have a link to the code?
>>>>> It's the damage blitter in the fbdev code. [1] While it flushes
>>>>> the shadow
>>>>> buffer into the BO, the BO has to be kept in place. I already
>>>>> changed it to
>>>>> lock struct drm_fb_helper.lock, but I don't think this is
>>>>> enough. TTM could
>>>>> still evict the BO concurrently.
>>>> So I'm not sure this is actually a problem: ttm could try to
>>>> concurrently
>>>> evict the buffer we pinned into vram, and then just skip to the next
>>>> one.
>>>>
>>>> Plus atm generic fbdev isn't used on any chip where we really care about
>>>> that last few mb of vram being useable for command submission (well atm
>>>> there's no driver using it).
>>> Well, this is the patchset for radeon. If it works out, amdgpu and
>>> nouveau are natural next choices. Especially radeon and nouveau support
>>> cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly
>>> matter.
>>>
>>>> Having the buffer pinned into system memory and trying to do a
>>>> concurrent
>>>> modeset that tries to pull it in is the hard failure mode. And holding
>>>> fb_helper.lock fully prevents that.
>>>>
>>>> So not really clear on what failure mode you're seeing here?
>>> Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland
>>> is running.) The fbdev BO is a few MiBs and not in use, so TTM would
>>> want to evict it if memory gets tight.
>>>
>>> What I have in mind is a concurrent modeset that requires the memory. If
>>> we do a concurrent damage blit without protecting against eviction,
>>> things go boom. Same for concurrent 3d graphics with textures, model
>>> data, etc.
>> Completely agree.
>>
>> This needs proper lock protection of the memory mapped buffer. Relying on
>> that some other code isn't run because we have some third part locks taken
>> is not sufficient here.
> We are still protected by the pin count in this scenario. Plus, with
> current drivers we always pin the fbdev buffer into vram, so occasionally
> failing to move it out isn't a regression.
>
> So I'm still not seeing how this can go boom.

Well as far as I understand it the pin count is zero for this buffer in 
this case here :)

I might be wrong on this because I don't know the FB code at all, but 
Thomas seems to be pretty clear that this is the shadow buffer which is 
not scanned out from.

Regards,
Christian.

>
> Now long term it'd be nice to cut everything over to dma_resv locking, but
> the issue there is that beyond ttm, none of the helpers (and few of the
> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
> interim fix seems like the right solution to me.
> -Daniel
>
>> Regards,
>> Christian.
>>
>>> Best regards
>>> Thomas
>>>
>>>>> There's no recursion taking place, so I guess the reservation
>>>>> lock could be
>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>> separate pair of
>>>>> DRM client functions could do the locking.
>>>> Given how this "do the right locking" is a can of worms (and I think
>>>> it's
>>>> worse than what you dug out already) I think the fb_helper.lock hack is
>>>> perfectly good enough.
>>>>
>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
>>>> code, while many helpers/drivers still have their hand-rolled locking,
>>>> will make conversion over to dma_resv needlessly more complicated.
>>>> -Daniel
>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>>>
>>>>>> Please note that the reservation lock you need to take here is part of
>>>>>> the GEM object.
>>>>>>
>>>>>> Usually we design things in the way that the code needs to take a lock
>>>>>> which protects an object, then do some operations with the object and
>>>>>> then release the lock again.
>>>>>>
>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>> returning with it is kind of unusual design.
>>>>>>
>>>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>>>> implications of acquiring these locks.
>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>> talking about
>>>>>> here. We need to take that for a couple of different operations,
>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>> _______________________________________________
>>>>>> dri-devel mailing list
>>>>>> dri-devel@lists.freedesktop.org
>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>> -- 
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH
>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>> (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Felix Imendörffer
>>>>
>>>>
>>>>
>>>>
>>>>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 9026 bytes --]

Hi

Am 25.11.20 um 11:36 schrieb Daniel Vetter:
> On Wed, Nov 25, 2020 at 11:13:13AM +0100, Christian König wrote:
>> Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:
>>> Hi
>>>
>>> Am 24.11.20 um 15:09 schrieb Daniel Vetter:
>>>> On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
>>>>> Hi
>>>>>
>>>>> Am 24.11.20 um 14:36 schrieb Christian König:
>>>>>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>>>>>> [SNIP]
>>>>>>>>>>> First I wanted to put this into
>>>>>>>>>>> drm_gem_ttm_vmap/vunmap(), but then wondered why
>>>>>>>>>>> ttm_bo_vmap() doe not acquire the lock internally?
>>>>>>>>>>> I'd expect that vmap/vunmap are close together and
>>>>>>>>>>> do not overlap for the same BO.
>>>>>>>>>>
>>>>>>>>>> We have use cases like the following during command submission:
>>>>>>>>>>
>>>>>>>>>> 1. lock
>>>>>>>>>> 2. map
>>>>>>>>>> 3. copy parts of the BO content somewhere else or patch
>>>>>>>>>> it with additional information
>>>>>>>>>> 4. unmap
>>>>>>>>>> 5. submit BO to the hardware
>>>>>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>>>>>> 7. unlock
>>>>>>>>>>
>>>>>>>>>> That use case won't be possible with vmap/vunmap if we
>>>>>>>>>> move the lock/unlock into it and I hope to replace the
>>>>>>>>>> kmap/kunmap functions with them in the near term.
>>>>>>>>>>
>>>>>>>>>>> Otherwise, acquiring the reservation lock would
>>>>>>>>>>> require another ref-counting variable or per-driver
>>>>>>>>>>> code.
>>>>>>>>>>
>>>>>>>>>> Hui, why that? Just put this into
>>>>>>>>>> drm_gem_ttm_vmap/vunmap() helper as you initially
>>>>>>>>>> planned.
>>>>>>>>>
>>>>>>>>> Given your example above, step one would acquire the lock,
>>>>>>>>> and step two would also acquire the lock as part of the vmap
>>>>>>>>> implementation. Wouldn't this fail (At least during unmap or
>>>>>>>>> unlock steps) ?
>>>>>>>>
>>>>>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>>>>>
>>>>>>> I don't want to nest/overlap them. My question was whether that
>>>>>>> would be required. Apparently not.
>>>>>>>
>>>>>>> While the console's BO is being set for scanout, it's protected from
>>>>>>> movement via the pin/unpin implementation, right?
>>>>>>
>>>>>> Yes, correct.
>>>>>>
>>>>>>> The driver does not acquire the resv lock for longer periods. I'm
>>>>>>> asking because this would prevent any console-buffer updates while
>>>>>>> the console is being displayed.
>>>>>>
>>>>>> Correct as well, we only hold the lock for things like command
>>>>>> submission, pinning, unpinning etc etc....
>>>>>>
>>>>>
>>>>> Thanks for answering my questions.
>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> You need to make sure that the lock is only taken from the FB
>>>>>>>> path which wants to vmap the object.
>>>>>>>>
>>>>>>>> Why don't you lock the GEM object from the caller in the generic
>>>>>>>> FB implementation?
>>>>>>>
>>>>>>> With the current blitter code, it breaks abstraction. if vmap/vunmap
>>>>>>> hold the lock implicitly, things would be easier.
>>>>>>
>>>>>> Do you have a link to the code?
>>>>>
>>>>> It's the damage blitter in the fbdev code. [1] While it flushes
>>>>> the shadow
>>>>> buffer into the BO, the BO has to be kept in place. I already
>>>>> changed it to
>>>>> lock struct drm_fb_helper.lock, but I don't think this is
>>>>> enough. TTM could
>>>>> still evict the BO concurrently.
>>>>
>>>> So I'm not sure this is actually a problem: ttm could try to
>>>> concurrently
>>>> evict the buffer we pinned into vram, and then just skip to the next
>>>> one.
>>>>
>>>> Plus atm generic fbdev isn't used on any chip where we really care about
>>>> that last few mb of vram being useable for command submission (well atm
>>>> there's no driver using it).
>>>
>>> Well, this is the patchset for radeon. If it works out, amdgpu and
>>> nouveau are natural next choices. Especially radeon and nouveau support
>>> cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly
>>> matter.
>>>
>>>>
>>>> Having the buffer pinned into system memory and trying to do a
>>>> concurrent
>>>> modeset that tries to pull it in is the hard failure mode. And holding
>>>> fb_helper.lock fully prevents that.
>>>>
>>>> So not really clear on what failure mode you're seeing here?
>>>
>>> Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland
>>> is running.) The fbdev BO is a few MiBs and not in use, so TTM would
>>> want to evict it if memory gets tight.
>>>
>>> What I have in mind is a concurrent modeset that requires the memory. If
>>> we do a concurrent damage blit without protecting against eviction,
>>> things go boom. Same for concurrent 3d graphics with textures, model
>>> data, etc.
>>
>> Completely agree.
>>
>> This needs proper lock protection of the memory mapped buffer. Relying on
>> that some other code isn't run because we have some third part locks taken
>> is not sufficient here.
> 
> We are still protected by the pin count in this scenario. Plus, with
> current drivers we always pin the fbdev buffer into vram, so occasionally
> failing to move it out isn't a regression.

Why is this protected by the pin count? The counter should be zero in 
this scenario. Otherwise, we could not evict the fbdev BO on all those 
systems where that's a hard requirement (e.g., ast).

The pin count is currently maintained by the vmap implementation in vram 
helpers. Calling vmap is an implicit pin; calling vunmap is an implicit 
unpin. This prevents eviction in the damage worker. But now I was told 
than pinning is only for BOs that are controlled by userspace and 
internal users should acquire the resv lock. So vram helpers have to be 
fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation 
only marks the BO as unmappable. The real unmap happens when the 
eviction takes place. This avoids many modifications to the page tables. 
IOW an unpinned, unmapped BO will remain in VRAM until the memory is 
actually needed.

Best regards
Thomas

> 
> So I'm still not seeing how this can go boom.
> 
> Now long term it'd be nice to cut everything over to dma_resv locking, but
> the issue there is that beyond ttm, none of the helpers (and few of the
> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
> interim fix seems like the right solution to me.
> -Daniel
> 
>>
>> Regards,
>> Christian.
>>
>>>
>>> Best regards
>>> Thomas
>>>
>>>>
>>>>> There's no recursion taking place, so I guess the reservation
>>>>> lock could be
>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>> separate pair of
>>>>> DRM client functions could do the locking.
>>>>
>>>> Given how this "do the right locking" is a can of worms (and I think
>>>> it's
>>>> worse than what you dug out already) I think the fb_helper.lock hack is
>>>> perfectly good enough.
>>>>
>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
>>>> code, while many helpers/drivers still have their hand-rolled locking,
>>>> will make conversion over to dma_resv needlessly more complicated.
>>>> -Daniel
>>>>
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>>>
>>>>>>
>>>>>> Please note that the reservation lock you need to take here is part of
>>>>>> the GEM object.
>>>>>>
>>>>>> Usually we design things in the way that the code needs to take a lock
>>>>>> which protects an object, then do some operations with the object and
>>>>>> then release the lock again.
>>>>>>
>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>> returning with it is kind of unusual design.
>>>>>>
>>>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>>>> implications of acquiring these locks.
>>>>>>
>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>> talking about
>>>>>> here. We need to take that for a couple of different operations,
>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>>
>>>>>> _______________________________________________
>>>>>> dri-devel mailing list
>>>>>> dri-devel@lists.freedesktop.org
>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>
>>>>> -- 
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH
>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>> (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Felix Imendörffer
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 8003 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 9026 bytes --]

Hi

Am 25.11.20 um 11:36 schrieb Daniel Vetter:
> On Wed, Nov 25, 2020 at 11:13:13AM +0100, Christian König wrote:
>> Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:
>>> Hi
>>>
>>> Am 24.11.20 um 15:09 schrieb Daniel Vetter:
>>>> On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
>>>>> Hi
>>>>>
>>>>> Am 24.11.20 um 14:36 schrieb Christian König:
>>>>>> Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
>>>>>>> [SNIP]
>>>>>>>>>>> First I wanted to put this into
>>>>>>>>>>> drm_gem_ttm_vmap/vunmap(), but then wondered why
>>>>>>>>>>> ttm_bo_vmap() doe not acquire the lock internally?
>>>>>>>>>>> I'd expect that vmap/vunmap are close together and
>>>>>>>>>>> do not overlap for the same BO.
>>>>>>>>>>
>>>>>>>>>> We have use cases like the following during command submission:
>>>>>>>>>>
>>>>>>>>>> 1. lock
>>>>>>>>>> 2. map
>>>>>>>>>> 3. copy parts of the BO content somewhere else or patch
>>>>>>>>>> it with additional information
>>>>>>>>>> 4. unmap
>>>>>>>>>> 5. submit BO to the hardware
>>>>>>>>>> 6. add hardware fence to the BO to make sure it doesn't move
>>>>>>>>>> 7. unlock
>>>>>>>>>>
>>>>>>>>>> That use case won't be possible with vmap/vunmap if we
>>>>>>>>>> move the lock/unlock into it and I hope to replace the
>>>>>>>>>> kmap/kunmap functions with them in the near term.
>>>>>>>>>>
>>>>>>>>>>> Otherwise, acquiring the reservation lock would
>>>>>>>>>>> require another ref-counting variable or per-driver
>>>>>>>>>>> code.
>>>>>>>>>>
>>>>>>>>>> Hui, why that? Just put this into
>>>>>>>>>> drm_gem_ttm_vmap/vunmap() helper as you initially
>>>>>>>>>> planned.
>>>>>>>>>
>>>>>>>>> Given your example above, step one would acquire the lock,
>>>>>>>>> and step two would also acquire the lock as part of the vmap
>>>>>>>>> implementation. Wouldn't this fail (At least during unmap or
>>>>>>>>> unlock steps) ?
>>>>>>>>
>>>>>>>> Oh, so you want to nest them? No, that is a rather bad no-go.
>>>>>>>
>>>>>>> I don't want to nest/overlap them. My question was whether that
>>>>>>> would be required. Apparently not.
>>>>>>>
>>>>>>> While the console's BO is being set for scanout, it's protected from
>>>>>>> movement via the pin/unpin implementation, right?
>>>>>>
>>>>>> Yes, correct.
>>>>>>
>>>>>>> The driver does not acquire the resv lock for longer periods. I'm
>>>>>>> asking because this would prevent any console-buffer updates while
>>>>>>> the console is being displayed.
>>>>>>
>>>>>> Correct as well, we only hold the lock for things like command
>>>>>> submission, pinning, unpinning etc etc....
>>>>>>
>>>>>
>>>>> Thanks for answering my questions.
>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> You need to make sure that the lock is only taken from the FB
>>>>>>>> path which wants to vmap the object.
>>>>>>>>
>>>>>>>> Why don't you lock the GEM object from the caller in the generic
>>>>>>>> FB implementation?
>>>>>>>
>>>>>>> With the current blitter code, it breaks abstraction. if vmap/vunmap
>>>>>>> hold the lock implicitly, things would be easier.
>>>>>>
>>>>>> Do you have a link to the code?
>>>>>
>>>>> It's the damage blitter in the fbdev code. [1] While it flushes
>>>>> the shadow
>>>>> buffer into the BO, the BO has to be kept in place. I already
>>>>> changed it to
>>>>> lock struct drm_fb_helper.lock, but I don't think this is
>>>>> enough. TTM could
>>>>> still evict the BO concurrently.
>>>>
>>>> So I'm not sure this is actually a problem: ttm could try to
>>>> concurrently
>>>> evict the buffer we pinned into vram, and then just skip to the next
>>>> one.
>>>>
>>>> Plus atm generic fbdev isn't used on any chip where we really care about
>>>> that last few mb of vram being useable for command submission (well atm
>>>> there's no driver using it).
>>>
>>> Well, this is the patchset for radeon. If it works out, amdgpu and
>>> nouveau are natural next choices. Especially radeon and nouveau support
>>> cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly
>>> matter.
>>>
>>>>
>>>> Having the buffer pinned into system memory and trying to do a
>>>> concurrent
>>>> modeset that tries to pull it in is the hard failure mode. And holding
>>>> fb_helper.lock fully prevents that.
>>>>
>>>> So not really clear on what failure mode you're seeing here?
>>>
>>> Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland
>>> is running.) The fbdev BO is a few MiBs and not in use, so TTM would
>>> want to evict it if memory gets tight.
>>>
>>> What I have in mind is a concurrent modeset that requires the memory. If
>>> we do a concurrent damage blit without protecting against eviction,
>>> things go boom. Same for concurrent 3d graphics with textures, model
>>> data, etc.
>>
>> Completely agree.
>>
>> This needs proper lock protection of the memory mapped buffer. Relying on
>> that some other code isn't run because we have some third part locks taken
>> is not sufficient here.
> 
> We are still protected by the pin count in this scenario. Plus, with
> current drivers we always pin the fbdev buffer into vram, so occasionally
> failing to move it out isn't a regression.

Why is this protected by the pin count? The counter should be zero in 
this scenario. Otherwise, we could not evict the fbdev BO on all those 
systems where that's a hard requirement (e.g., ast).

The pin count is currently maintained by the vmap implementation in vram 
helpers. Calling vmap is an implicit pin; calling vunmap is an implicit 
unpin. This prevents eviction in the damage worker. But now I was told 
than pinning is only for BOs that are controlled by userspace and 
internal users should acquire the resv lock. So vram helpers have to be 
fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation 
only marks the BO as unmappable. The real unmap happens when the 
eviction takes place. This avoids many modifications to the page tables. 
IOW an unpinned, unmapped BO will remain in VRAM until the memory is 
actually needed.

Best regards
Thomas

> 
> So I'm still not seeing how this can go boom.
> 
> Now long term it'd be nice to cut everything over to dma_resv locking, but
> the issue there is that beyond ttm, none of the helpers (and few of the
> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
> interim fix seems like the right solution to me.
> -Daniel
> 
>>
>> Regards,
>> Christian.
>>
>>>
>>> Best regards
>>> Thomas
>>>
>>>>
>>>>> There's no recursion taking place, so I guess the reservation
>>>>> lock could be
>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>> separate pair of
>>>>> DRM client functions could do the locking.
>>>>
>>>> Given how this "do the right locking" is a can of worms (and I think
>>>> it's
>>>> worse than what you dug out already) I think the fb_helper.lock hack is
>>>> perfectly good enough.
>>>>
>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
>>>> code, while many helpers/drivers still have their hand-rolled locking,
>>>> will make conversion over to dma_resv needlessly more complicated.
>>>> -Daniel
>>>>
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>>>
>>>>>>
>>>>>> Please note that the reservation lock you need to take here is part of
>>>>>> the GEM object.
>>>>>>
>>>>>> Usually we design things in the way that the code needs to take a lock
>>>>>> which protects an object, then do some operations with the object and
>>>>>> then release the lock again.
>>>>>>
>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>> returning with it is kind of unusual design.
>>>>>>
>>>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>>>> implications of acquiring these locks.
>>>>>>
>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>> talking about
>>>>>> here. We need to take that for a couple of different operations,
>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>>
>>>>>> _______________________________________________
>>>>>> dri-devel mailing list
>>>>>> dri-devel@lists.freedesktop.org
>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>
>>>>> -- 
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH
>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>> (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Felix Imendörffer
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 8003 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Daniel Vetter]

On Wed, Nov 25, 2020 at 12:38:01PM +0100, Thomas Zimmermann wrote:
> Hi
> 
> Am 25.11.20 um 11:36 schrieb Daniel Vetter:
> > On Wed, Nov 25, 2020 at 11:13:13AM +0100, Christian König wrote:
> > > Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:
> > > > Hi
> > > > 
> > > > Am 24.11.20 um 15:09 schrieb Daniel Vetter:
> > > > > On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
> > > > > > Hi
> > > > > > 
> > > > > > Am 24.11.20 um 14:36 schrieb Christian König:
> > > > > > > Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
> > > > > > > > [SNIP]
> > > > > > > > > > > > First I wanted to put this into
> > > > > > > > > > > > drm_gem_ttm_vmap/vunmap(), but then wondered why
> > > > > > > > > > > > ttm_bo_vmap() doe not acquire the lock internally?
> > > > > > > > > > > > I'd expect that vmap/vunmap are close together and
> > > > > > > > > > > > do not overlap for the same BO.
> > > > > > > > > > > 
> > > > > > > > > > > We have use cases like the following during command submission:
> > > > > > > > > > > 
> > > > > > > > > > > 1. lock
> > > > > > > > > > > 2. map
> > > > > > > > > > > 3. copy parts of the BO content somewhere else or patch
> > > > > > > > > > > it with additional information
> > > > > > > > > > > 4. unmap
> > > > > > > > > > > 5. submit BO to the hardware
> > > > > > > > > > > 6. add hardware fence to the BO to make sure it doesn't move
> > > > > > > > > > > 7. unlock
> > > > > > > > > > > 
> > > > > > > > > > > That use case won't be possible with vmap/vunmap if we
> > > > > > > > > > > move the lock/unlock into it and I hope to replace the
> > > > > > > > > > > kmap/kunmap functions with them in the near term.
> > > > > > > > > > > 
> > > > > > > > > > > > Otherwise, acquiring the reservation lock would
> > > > > > > > > > > > require another ref-counting variable or per-driver
> > > > > > > > > > > > code.
> > > > > > > > > > > 
> > > > > > > > > > > Hui, why that? Just put this into
> > > > > > > > > > > drm_gem_ttm_vmap/vunmap() helper as you initially
> > > > > > > > > > > planned.
> > > > > > > > > > 
> > > > > > > > > > Given your example above, step one would acquire the lock,
> > > > > > > > > > and step two would also acquire the lock as part of the vmap
> > > > > > > > > > implementation. Wouldn't this fail (At least during unmap or
> > > > > > > > > > unlock steps) ?
> > > > > > > > > 
> > > > > > > > > Oh, so you want to nest them? No, that is a rather bad no-go.
> > > > > > > > 
> > > > > > > > I don't want to nest/overlap them. My question was whether that
> > > > > > > > would be required. Apparently not.
> > > > > > > > 
> > > > > > > > While the console's BO is being set for scanout, it's protected from
> > > > > > > > movement via the pin/unpin implementation, right?
> > > > > > > 
> > > > > > > Yes, correct.
> > > > > > > 
> > > > > > > > The driver does not acquire the resv lock for longer periods. I'm
> > > > > > > > asking because this would prevent any console-buffer updates while
> > > > > > > > the console is being displayed.
> > > > > > > 
> > > > > > > Correct as well, we only hold the lock for things like command
> > > > > > > submission, pinning, unpinning etc etc....
> > > > > > > 
> > > > > > 
> > > > > > Thanks for answering my questions.
> > > > > > 
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > You need to make sure that the lock is only taken from the FB
> > > > > > > > > path which wants to vmap the object.
> > > > > > > > > 
> > > > > > > > > Why don't you lock the GEM object from the caller in the generic
> > > > > > > > > FB implementation?
> > > > > > > > 
> > > > > > > > With the current blitter code, it breaks abstraction. if vmap/vunmap
> > > > > > > > hold the lock implicitly, things would be easier.
> > > > > > > 
> > > > > > > Do you have a link to the code?
> > > > > > 
> > > > > > It's the damage blitter in the fbdev code. [1] While it flushes
> > > > > > the shadow
> > > > > > buffer into the BO, the BO has to be kept in place. I already
> > > > > > changed it to
> > > > > > lock struct drm_fb_helper.lock, but I don't think this is
> > > > > > enough. TTM could
> > > > > > still evict the BO concurrently.
> > > > > 
> > > > > So I'm not sure this is actually a problem: ttm could try to
> > > > > concurrently
> > > > > evict the buffer we pinned into vram, and then just skip to the next
> > > > > one.
> > > > > 
> > > > > Plus atm generic fbdev isn't used on any chip where we really care about
> > > > > that last few mb of vram being useable for command submission (well atm
> > > > > there's no driver using it).
> > > > 
> > > > Well, this is the patchset for radeon. If it works out, amdgpu and
> > > > nouveau are natural next choices. Especially radeon and nouveau support
> > > > cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly
> > > > matter.
> > > > 
> > > > > 
> > > > > Having the buffer pinned into system memory and trying to do a
> > > > > concurrent
> > > > > modeset that tries to pull it in is the hard failure mode. And holding
> > > > > fb_helper.lock fully prevents that.
> > > > > 
> > > > > So not really clear on what failure mode you're seeing here?
> > > > 
> > > > Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland
> > > > is running.) The fbdev BO is a few MiBs and not in use, so TTM would
> > > > want to evict it if memory gets tight.
> > > > 
> > > > What I have in mind is a concurrent modeset that requires the memory. If
> > > > we do a concurrent damage blit without protecting against eviction,
> > > > things go boom. Same for concurrent 3d graphics with textures, model
> > > > data, etc.
> > > 
> > > Completely agree.
> > > 
> > > This needs proper lock protection of the memory mapped buffer. Relying on
> > > that some other code isn't run because we have some third part locks taken
> > > is not sufficient here.
> > 
> > We are still protected by the pin count in this scenario. Plus, with
> > current drivers we always pin the fbdev buffer into vram, so occasionally
> > failing to move it out isn't a regression.
> 
> Why is this protected by the pin count? The counter should be zero in this
> scenario. Otherwise, we could not evict the fbdev BO on all those systems
> where that's a hard requirement (e.g., ast).

Ah yes, I mixed that up.

I guess full locking is required :-/ I'm not exactly sure how to make this
happen with the current plethora of helpers ... I think we need an
_locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

And then document that if the callers of the _locked version wants a
permanent mapping, it also needs to pin it. Plus I guess ideally implement
the unlocked/permanent versions in terms of that, so that drivers only
have to implement one or the other.

That should give us at least some way forward to gradually conver all the
drivers and helpers over to dma_resv locking.
-Daniel

> The pin count is currently maintained by the vmap implementation in vram
> helpers. Calling vmap is an implicit pin; calling vunmap is an implicit
> unpin. This prevents eviction in the damage worker. But now I was told than
> pinning is only for BOs that are controlled by userspace and internal users
> should acquire the resv lock. So vram helpers have to be fixed, actually.
> 
> In vram helpers, unmapping does not mean eviction. The unmap operation only
> marks the BO as unmappable. The real unmap happens when the eviction takes
> place. This avoids many modifications to the page tables. IOW an unpinned,
> unmapped BO will remain in VRAM until the memory is actually needed.
> 
> Best regards
> Thomas
> 
> > 
> > So I'm still not seeing how this can go boom.
> > 
> > Now long term it'd be nice to cut everything over to dma_resv locking, but
> > the issue there is that beyond ttm, none of the helpers (and few of the
> > drivers) use dma_resv. So this is a fairly big uphill battle. Quick
> > interim fix seems like the right solution to me.
> > -Daniel
> > 
> > > 
> > > Regards,
> > > Christian.
> > > 
> > > > 
> > > > Best regards
> > > > Thomas
> > > > 
> > > > > 
> > > > > > There's no recursion taking place, so I guess the reservation
> > > > > > lock could be
> > > > > > acquired/release in drm_client_buffer_vmap/vunmap(), or a
> > > > > > separate pair of
> > > > > > DRM client functions could do the locking.
> > > > > 
> > > > > Given how this "do the right locking" is a can of worms (and I think
> > > > > it's
> > > > > worse than what you dug out already) I think the fb_helper.lock hack is
> > > > > perfectly good enough.
> > > > > 
> > > > > I'm also somewhat worried that starting to use dma_resv lock in generic
> > > > > code, while many helpers/drivers still have their hand-rolled locking,
> > > > > will make conversion over to dma_resv needlessly more complicated.
> > > > > -Daniel
> > > > > 
> > > > > > 
> > > > > > Best regards
> > > > > > Thomas
> > > > > > 
> > > > > > [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
> > > > > > 
> > > > > > > 
> > > > > > > Please note that the reservation lock you need to take here is part of
> > > > > > > the GEM object.
> > > > > > > 
> > > > > > > Usually we design things in the way that the code needs to take a lock
> > > > > > > which protects an object, then do some operations with the object and
> > > > > > > then release the lock again.
> > > > > > > 
> > > > > > > Having in the lock inside the operation can be done as well, but
> > > > > > > returning with it is kind of unusual design.
> > > > > > > 
> > > > > > > > Sorry for the noob questions. I'm still trying to understand the
> > > > > > > > implications of acquiring these locks.
> > > > > > > 
> > > > > > > Well this is the reservation lock of the GEM object we are
> > > > > > > talking about
> > > > > > > here. We need to take that for a couple of different operations,
> > > > > > > vmap/vunmap doesn't sound like a special case to me.
> > > > > > > 
> > > > > > > Regards,
> > > > > > > Christian.
> > > > > > > 
> > > > > > > > 
> > > > > > > > Best regards
> > > > > > > > Thomas
> > > > > > > 
> > > > > > > _______________________________________________
> > > > > > > dri-devel mailing list
> > > > > > > dri-devel@lists.freedesktop.org
> > > > > > > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > > > > > 
> > > > > > -- 
> > > > > > Thomas Zimmermann
> > > > > > Graphics Driver Developer
> > > > > > SUSE Software Solutions Germany GmbH
> > > > > > Maxfeldstr. 5, 90409 Nürnberg, Germany
> > > > > > (HRB 36809, AG Nürnberg)
> > > > > > Geschäftsführer: Felix Imendörffer
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> -- 
> Thomas Zimmermann
> Graphics Driver Developer
> SUSE Software Solutions Germany GmbH
> Maxfeldstr. 5, 90409 Nürnberg, Germany
> (HRB 36809, AG Nürnberg)
> Geschäftsführer: Felix Imendörffer






-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Daniel Vetter]

On Wed, Nov 25, 2020 at 12:38:01PM +0100, Thomas Zimmermann wrote:
> Hi
> 
> Am 25.11.20 um 11:36 schrieb Daniel Vetter:
> > On Wed, Nov 25, 2020 at 11:13:13AM +0100, Christian König wrote:
> > > Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:
> > > > Hi
> > > > 
> > > > Am 24.11.20 um 15:09 schrieb Daniel Vetter:
> > > > > On Tue, Nov 24, 2020 at 02:56:51PM +0100, Thomas Zimmermann wrote:
> > > > > > Hi
> > > > > > 
> > > > > > Am 24.11.20 um 14:36 schrieb Christian König:
> > > > > > > Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:
> > > > > > > > [SNIP]
> > > > > > > > > > > > First I wanted to put this into
> > > > > > > > > > > > drm_gem_ttm_vmap/vunmap(), but then wondered why
> > > > > > > > > > > > ttm_bo_vmap() doe not acquire the lock internally?
> > > > > > > > > > > > I'd expect that vmap/vunmap are close together and
> > > > > > > > > > > > do not overlap for the same BO.
> > > > > > > > > > > 
> > > > > > > > > > > We have use cases like the following during command submission:
> > > > > > > > > > > 
> > > > > > > > > > > 1. lock
> > > > > > > > > > > 2. map
> > > > > > > > > > > 3. copy parts of the BO content somewhere else or patch
> > > > > > > > > > > it with additional information
> > > > > > > > > > > 4. unmap
> > > > > > > > > > > 5. submit BO to the hardware
> > > > > > > > > > > 6. add hardware fence to the BO to make sure it doesn't move
> > > > > > > > > > > 7. unlock
> > > > > > > > > > > 
> > > > > > > > > > > That use case won't be possible with vmap/vunmap if we
> > > > > > > > > > > move the lock/unlock into it and I hope to replace the
> > > > > > > > > > > kmap/kunmap functions with them in the near term.
> > > > > > > > > > > 
> > > > > > > > > > > > Otherwise, acquiring the reservation lock would
> > > > > > > > > > > > require another ref-counting variable or per-driver
> > > > > > > > > > > > code.
> > > > > > > > > > > 
> > > > > > > > > > > Hui, why that? Just put this into
> > > > > > > > > > > drm_gem_ttm_vmap/vunmap() helper as you initially
> > > > > > > > > > > planned.
> > > > > > > > > > 
> > > > > > > > > > Given your example above, step one would acquire the lock,
> > > > > > > > > > and step two would also acquire the lock as part of the vmap
> > > > > > > > > > implementation. Wouldn't this fail (At least during unmap or
> > > > > > > > > > unlock steps) ?
> > > > > > > > > 
> > > > > > > > > Oh, so you want to nest them? No, that is a rather bad no-go.
> > > > > > > > 
> > > > > > > > I don't want to nest/overlap them. My question was whether that
> > > > > > > > would be required. Apparently not.
> > > > > > > > 
> > > > > > > > While the console's BO is being set for scanout, it's protected from
> > > > > > > > movement via the pin/unpin implementation, right?
> > > > > > > 
> > > > > > > Yes, correct.
> > > > > > > 
> > > > > > > > The driver does not acquire the resv lock for longer periods. I'm
> > > > > > > > asking because this would prevent any console-buffer updates while
> > > > > > > > the console is being displayed.
> > > > > > > 
> > > > > > > Correct as well, we only hold the lock for things like command
> > > > > > > submission, pinning, unpinning etc etc....
> > > > > > > 
> > > > > > 
> > > > > > Thanks for answering my questions.
> > > > > > 
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > You need to make sure that the lock is only taken from the FB
> > > > > > > > > path which wants to vmap the object.
> > > > > > > > > 
> > > > > > > > > Why don't you lock the GEM object from the caller in the generic
> > > > > > > > > FB implementation?
> > > > > > > > 
> > > > > > > > With the current blitter code, it breaks abstraction. if vmap/vunmap
> > > > > > > > hold the lock implicitly, things would be easier.
> > > > > > > 
> > > > > > > Do you have a link to the code?
> > > > > > 
> > > > > > It's the damage blitter in the fbdev code. [1] While it flushes
> > > > > > the shadow
> > > > > > buffer into the BO, the BO has to be kept in place. I already
> > > > > > changed it to
> > > > > > lock struct drm_fb_helper.lock, but I don't think this is
> > > > > > enough. TTM could
> > > > > > still evict the BO concurrently.
> > > > > 
> > > > > So I'm not sure this is actually a problem: ttm could try to
> > > > > concurrently
> > > > > evict the buffer we pinned into vram, and then just skip to the next
> > > > > one.
> > > > > 
> > > > > Plus atm generic fbdev isn't used on any chip where we really care about
> > > > > that last few mb of vram being useable for command submission (well atm
> > > > > there's no driver using it).
> > > > 
> > > > Well, this is the patchset for radeon. If it works out, amdgpu and
> > > > nouveau are natural next choices. Especially radeon and nouveau support
> > > > cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly
> > > > matter.
> > > > 
> > > > > 
> > > > > Having the buffer pinned into system memory and trying to do a
> > > > > concurrent
> > > > > modeset that tries to pull it in is the hard failure mode. And holding
> > > > > fb_helper.lock fully prevents that.
> > > > > 
> > > > > So not really clear on what failure mode you're seeing here?
> > > > 
> > > > Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland
> > > > is running.) The fbdev BO is a few MiBs and not in use, so TTM would
> > > > want to evict it if memory gets tight.
> > > > 
> > > > What I have in mind is a concurrent modeset that requires the memory. If
> > > > we do a concurrent damage blit without protecting against eviction,
> > > > things go boom. Same for concurrent 3d graphics with textures, model
> > > > data, etc.
> > > 
> > > Completely agree.
> > > 
> > > This needs proper lock protection of the memory mapped buffer. Relying on
> > > that some other code isn't run because we have some third part locks taken
> > > is not sufficient here.
> > 
> > We are still protected by the pin count in this scenario. Plus, with
> > current drivers we always pin the fbdev buffer into vram, so occasionally
> > failing to move it out isn't a regression.
> 
> Why is this protected by the pin count? The counter should be zero in this
> scenario. Otherwise, we could not evict the fbdev BO on all those systems
> where that's a hard requirement (e.g., ast).

Ah yes, I mixed that up.

I guess full locking is required :-/ I'm not exactly sure how to make this
happen with the current plethora of helpers ... I think we need an
_locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

And then document that if the callers of the _locked version wants a
permanent mapping, it also needs to pin it. Plus I guess ideally implement
the unlocked/permanent versions in terms of that, so that drivers only
have to implement one or the other.

That should give us at least some way forward to gradually conver all the
drivers and helpers over to dma_resv locking.
-Daniel

> The pin count is currently maintained by the vmap implementation in vram
> helpers. Calling vmap is an implicit pin; calling vunmap is an implicit
> unpin. This prevents eviction in the damage worker. But now I was told than
> pinning is only for BOs that are controlled by userspace and internal users
> should acquire the resv lock. So vram helpers have to be fixed, actually.
> 
> In vram helpers, unmapping does not mean eviction. The unmap operation only
> marks the BO as unmappable. The real unmap happens when the eviction takes
> place. This avoids many modifications to the page tables. IOW an unpinned,
> unmapped BO will remain in VRAM until the memory is actually needed.
> 
> Best regards
> Thomas
> 
> > 
> > So I'm still not seeing how this can go boom.
> > 
> > Now long term it'd be nice to cut everything over to dma_resv locking, but
> > the issue there is that beyond ttm, none of the helpers (and few of the
> > drivers) use dma_resv. So this is a fairly big uphill battle. Quick
> > interim fix seems like the right solution to me.
> > -Daniel
> > 
> > > 
> > > Regards,
> > > Christian.
> > > 
> > > > 
> > > > Best regards
> > > > Thomas
> > > > 
> > > > > 
> > > > > > There's no recursion taking place, so I guess the reservation
> > > > > > lock could be
> > > > > > acquired/release in drm_client_buffer_vmap/vunmap(), or a
> > > > > > separate pair of
> > > > > > DRM client functions could do the locking.
> > > > > 
> > > > > Given how this "do the right locking" is a can of worms (and I think
> > > > > it's
> > > > > worse than what you dug out already) I think the fb_helper.lock hack is
> > > > > perfectly good enough.
> > > > > 
> > > > > I'm also somewhat worried that starting to use dma_resv lock in generic
> > > > > code, while many helpers/drivers still have their hand-rolled locking,
> > > > > will make conversion over to dma_resv needlessly more complicated.
> > > > > -Daniel
> > > > > 
> > > > > > 
> > > > > > Best regards
> > > > > > Thomas
> > > > > > 
> > > > > > [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
> > > > > > 
> > > > > > > 
> > > > > > > Please note that the reservation lock you need to take here is part of
> > > > > > > the GEM object.
> > > > > > > 
> > > > > > > Usually we design things in the way that the code needs to take a lock
> > > > > > > which protects an object, then do some operations with the object and
> > > > > > > then release the lock again.
> > > > > > > 
> > > > > > > Having in the lock inside the operation can be done as well, but
> > > > > > > returning with it is kind of unusual design.
> > > > > > > 
> > > > > > > > Sorry for the noob questions. I'm still trying to understand the
> > > > > > > > implications of acquiring these locks.
> > > > > > > 
> > > > > > > Well this is the reservation lock of the GEM object we are
> > > > > > > talking about
> > > > > > > here. We need to take that for a couple of different operations,
> > > > > > > vmap/vunmap doesn't sound like a special case to me.
> > > > > > > 
> > > > > > > Regards,
> > > > > > > Christian.
> > > > > > > 
> > > > > > > > 
> > > > > > > > Best regards
> > > > > > > > Thomas
> > > > > > > 
> > > > > > > _______________________________________________
> > > > > > > dri-devel mailing list
> > > > > > > dri-devel@lists.freedesktop.org
> > > > > > > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > > > > > 
> > > > > > -- 
> > > > > > Thomas Zimmermann
> > > > > > Graphics Driver Developer
> > > > > > SUSE Software Solutions Germany GmbH
> > > > > > Maxfeldstr. 5, 90409 Nürnberg, Germany
> > > > > > (HRB 36809, AG Nürnberg)
> > > > > > Geschäftsführer: Felix Imendörffer
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> -- 
> Thomas Zimmermann
> Graphics Driver Developer
> SUSE Software Solutions Germany GmbH
> Maxfeldstr. 5, 90409 Nürnberg, Germany
> (HRB 36809, AG Nürnberg)
> Geschäftsführer: Felix Imendörffer






-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 6730 bytes --]

Hi

Am 25.11.20 um 17:32 schrieb Daniel Vetter:
> [...]
> I guess full locking is required :-/ I'm not exactly sure how to make this
> happen with the current plethora of helpers ... I think we need an
> _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

I think we might be able to get away without new callbacks.

I looked through the sources that implement and use vmap. All the 
implementations are without taking resv locks. For locking, we can wrap 
them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked() 
that locks and vmaps should make this easy.

In terms of implementation, only vram helpers do a pin+map in their vmap 
code. And as I mentioned before, this is actually wrong. The pattern 
dates back to when the code was still in individual drivers. It's time 
to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.

Finally, there aren't that many users of vmap. A few drivers use it 
while blitting framebuffers into HW buffers and ast does some permanent 
mapping of the cursor BO. All this is trivial to turn into small pairs
of lock+vmap and vunmap+unlock.

That leaves generic fbdev. The shadow buffer is also trivial to fix, as 
outlined during this discussion.

The code for fbdev in hardware buffers is a special case. It vmaps the 
buffer during initialization and only vunmaps it during shutdown. As 
this has worked so far without locking, I'd leave it as it is and put a 
big comment next to is.

Hardware fbdev buffers is only required by few drivers; namely those 
that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work. 
We should consider to make the fbdev shadow buffer the default and have 
drivers opt-in for the hardware buffer, if they need it.

> 
> And then document that if the callers of the _locked version wants a
> permanent mapping, it also needs to pin it. Plus I guess ideally implement
> the unlocked/permanent versions in terms of that, so that drivers only
> have to implement one or the other.

For my understanding, pinning is only done in prepare_fb code. And ast 
pins its cursor BOs into vram. We should document to hols vmap/vunmap 
only for time and cover them with resv locks. Pinning is for cases where 
the hardware requires buffers in a special location, but does not 
protect against concurrent threat. I think those are the implicit rules 
already.

I updated the radeon patchset, where all this appears to be working well.

Best regards
Thomas

> 
> That should give us at least some way forward to gradually conver all the
> drivers and helpers over to dma_resv locking.
> -Daniel
> 
>> The pin count is currently maintained by the vmap implementation in vram
>> helpers. Calling vmap is an implicit pin; calling vunmap is an implicit
>> unpin. This prevents eviction in the damage worker. But now I was told than
>> pinning is only for BOs that are controlled by userspace and internal users
>> should acquire the resv lock. So vram helpers have to be fixed, actually.
>>
>> In vram helpers, unmapping does not mean eviction. The unmap operation only
>> marks the BO as unmappable. The real unmap happens when the eviction takes
>> place. This avoids many modifications to the page tables. IOW an unpinned,
>> unmapped BO will remain in VRAM until the memory is actually needed.
>>
>> Best regards
>> Thomas
>>
>>>
>>> So I'm still not seeing how this can go boom.
>>>
>>> Now long term it'd be nice to cut everything over to dma_resv locking, but
>>> the issue there is that beyond ttm, none of the helpers (and few of the
>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>> interim fix seems like the right solution to me.
>>> -Daniel
>>>
>>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>>>
>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>> lock could be
>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>> separate pair of
>>>>>>> DRM client functions could do the locking.
>>>>>>
>>>>>> Given how this "do the right locking" is a can of worms (and I think
>>>>>> it's
>>>>>> worse than what you dug out already) I think the fb_helper.lock hack is
>>>>>> perfectly good enough.
>>>>>>
>>>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
>>>>>> code, while many helpers/drivers still have their hand-rolled locking,
>>>>>> will make conversion over to dma_resv needlessly more complicated.
>>>>>> -Daniel
>>>>>>
>>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>>>
>>>>>>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>>>>>
>>>>>>>>
>>>>>>>> Please note that the reservation lock you need to take here is part of
>>>>>>>> the GEM object.
>>>>>>>>
>>>>>>>> Usually we design things in the way that the code needs to take a lock
>>>>>>>> which protects an object, then do some operations with the object and
>>>>>>>> then release the lock again.
>>>>>>>>
>>>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>
>>>>>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>>>>>> implications of acquiring these locks.
>>>>>>>>
>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>> talking about
>>>>>>>> here. We need to take that for a couple of different operations,
>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Christian.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>> Thomas
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> dri-devel mailing list
>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>>>
>>>>>>> -- 
>>>>>>> Thomas Zimmermann
>>>>>>> Graphics Driver Developer
>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>> -- 
>> Thomas Zimmermann
>> Graphics Driver Developer
>> SUSE Software Solutions Germany GmbH
>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>> (HRB 36809, AG Nürnberg)
>> Geschäftsführer: Felix Imendörffer
> 
> 
> 
> 
> 
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 8003 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 6730 bytes --]

Hi

Am 25.11.20 um 17:32 schrieb Daniel Vetter:
> [...]
> I guess full locking is required :-/ I'm not exactly sure how to make this
> happen with the current plethora of helpers ... I think we need an
> _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

I think we might be able to get away without new callbacks.

I looked through the sources that implement and use vmap. All the 
implementations are without taking resv locks. For locking, we can wrap 
them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked() 
that locks and vmaps should make this easy.

In terms of implementation, only vram helpers do a pin+map in their vmap 
code. And as I mentioned before, this is actually wrong. The pattern 
dates back to when the code was still in individual drivers. It's time 
to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.

Finally, there aren't that many users of vmap. A few drivers use it 
while blitting framebuffers into HW buffers and ast does some permanent 
mapping of the cursor BO. All this is trivial to turn into small pairs
of lock+vmap and vunmap+unlock.

That leaves generic fbdev. The shadow buffer is also trivial to fix, as 
outlined during this discussion.

The code for fbdev in hardware buffers is a special case. It vmaps the 
buffer during initialization and only vunmaps it during shutdown. As 
this has worked so far without locking, I'd leave it as it is and put a 
big comment next to is.

Hardware fbdev buffers is only required by few drivers; namely those 
that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work. 
We should consider to make the fbdev shadow buffer the default and have 
drivers opt-in for the hardware buffer, if they need it.

> 
> And then document that if the callers of the _locked version wants a
> permanent mapping, it also needs to pin it. Plus I guess ideally implement
> the unlocked/permanent versions in terms of that, so that drivers only
> have to implement one or the other.

For my understanding, pinning is only done in prepare_fb code. And ast 
pins its cursor BOs into vram. We should document to hols vmap/vunmap 
only for time and cover them with resv locks. Pinning is for cases where 
the hardware requires buffers in a special location, but does not 
protect against concurrent threat. I think those are the implicit rules 
already.

I updated the radeon patchset, where all this appears to be working well.

Best regards
Thomas

> 
> That should give us at least some way forward to gradually conver all the
> drivers and helpers over to dma_resv locking.
> -Daniel
> 
>> The pin count is currently maintained by the vmap implementation in vram
>> helpers. Calling vmap is an implicit pin; calling vunmap is an implicit
>> unpin. This prevents eviction in the damage worker. But now I was told than
>> pinning is only for BOs that are controlled by userspace and internal users
>> should acquire the resv lock. So vram helpers have to be fixed, actually.
>>
>> In vram helpers, unmapping does not mean eviction. The unmap operation only
>> marks the BO as unmappable. The real unmap happens when the eviction takes
>> place. This avoids many modifications to the page tables. IOW an unpinned,
>> unmapped BO will remain in VRAM until the memory is actually needed.
>>
>> Best regards
>> Thomas
>>
>>>
>>> So I'm still not seeing how this can go boom.
>>>
>>> Now long term it'd be nice to cut everything over to dma_resv locking, but
>>> the issue there is that beyond ttm, none of the helpers (and few of the
>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>> interim fix seems like the right solution to me.
>>> -Daniel
>>>
>>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>>>
>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>> lock could be
>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>> separate pair of
>>>>>>> DRM client functions could do the locking.
>>>>>>
>>>>>> Given how this "do the right locking" is a can of worms (and I think
>>>>>> it's
>>>>>> worse than what you dug out already) I think the fb_helper.lock hack is
>>>>>> perfectly good enough.
>>>>>>
>>>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
>>>>>> code, while many helpers/drivers still have their hand-rolled locking,
>>>>>> will make conversion over to dma_resv needlessly more complicated.
>>>>>> -Daniel
>>>>>>
>>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>>>
>>>>>>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>>>>>
>>>>>>>>
>>>>>>>> Please note that the reservation lock you need to take here is part of
>>>>>>>> the GEM object.
>>>>>>>>
>>>>>>>> Usually we design things in the way that the code needs to take a lock
>>>>>>>> which protects an object, then do some operations with the object and
>>>>>>>> then release the lock again.
>>>>>>>>
>>>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>
>>>>>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>>>>>> implications of acquiring these locks.
>>>>>>>>
>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>> talking about
>>>>>>>> here. We need to take that for a couple of different operations,
>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Christian.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>> Thomas
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> dri-devel mailing list
>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>>>
>>>>>>> -- 
>>>>>>> Thomas Zimmermann
>>>>>>> Graphics Driver Developer
>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>> -- 
>> Thomas Zimmermann
>> Graphics Driver Developer
>> SUSE Software Solutions Germany GmbH
>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>> (HRB 36809, AG Nürnberg)
>> Geschäftsführer: Felix Imendörffer
> 
> 
> 
> 
> 
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 8003 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Daniel Vetter]

On Thu, Nov 26, 2020 at 11:15 AM Thomas Zimmermann <tzimmermann@suse.de> wrote:
>
> Hi
>
> Am 25.11.20 um 17:32 schrieb Daniel Vetter:
> > [...]
> > I guess full locking is required :-/ I'm not exactly sure how to make this
> > happen with the current plethora of helpers ... I think we need an
> > _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.
>
> I think we might be able to get away without new callbacks.
>
> I looked through the sources that implement and use vmap. All the
> implementations are without taking resv locks. For locking, we can wrap
> them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked()
> that locks and vmaps should make this easy.
>
> In terms of implementation, only vram helpers do a pin+map in their vmap
> code. And as I mentioned before, this is actually wrong. The pattern
> dates back to when the code was still in individual drivers. It's time
> to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.
>
> Finally, there aren't that many users of vmap. A few drivers use it
> while blitting framebuffers into HW buffers and ast does some permanent
> mapping of the cursor BO. All this is trivial to turn into small pairs
> of lock+vmap and vunmap+unlock.
>
> That leaves generic fbdev. The shadow buffer is also trivial to fix, as
> outlined during this discussion.
>
> The code for fbdev in hardware buffers is a special case. It vmaps the
> buffer during initialization and only vunmaps it during shutdown. As
> this has worked so far without locking, I'd leave it as it is and put a
> big comment next to is.
>
> Hardware fbdev buffers is only required by few drivers; namely those
> that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work.
> We should consider to make the fbdev shadow buffer the default and have
> drivers opt-in for the hardware buffer, if they need it.
>
> >
> > And then document that if the callers of the _locked version wants a
> > permanent mapping, it also needs to pin it. Plus I guess ideally implement
> > the unlocked/permanent versions in terms of that, so that drivers only
> > have to implement one or the other.
>
> For my understanding, pinning is only done in prepare_fb code. And ast
> pins its cursor BOs into vram. We should document to hols vmap/vunmap
> only for time and cover them with resv locks. Pinning is for cases where
> the hardware requires buffers in a special location, but does not
> protect against concurrent threat. I think those are the implicit rules
> already.
>
> I updated the radeon patchset, where all this appears to be working well.

Hm yeah if you want to do the full change, then that works out too.
It's just a pile of work.

But if we can finish off with an dma_resv_assert_locked in
dma_buf_vmap/vunmap, then I think that's ok. It does mean that
exporters must implement vmap caching, or performance will be
terrible. So quite some update for the dma-buf docs.

But if you're willing to do all that conversion of callers, then of
course I'm not stopping you. Not at all, it's great to see that kind
of maze untangled.
-Daniel

>
> Best regards
> Thomas
>
> >
> > That should give us at least some way forward to gradually conver all the
> > drivers and helpers over to dma_resv locking.
> > -Daniel
> >
> >> The pin count is currently maintained by the vmap implementation in vram
> >> helpers. Calling vmap is an implicit pin; calling vunmap is an implicit
> >> unpin. This prevents eviction in the damage worker. But now I was told than
> >> pinning is only for BOs that are controlled by userspace and internal users
> >> should acquire the resv lock. So vram helpers have to be fixed, actually.
> >>
> >> In vram helpers, unmapping does not mean eviction. The unmap operation only
> >> marks the BO as unmappable. The real unmap happens when the eviction takes
> >> place. This avoids many modifications to the page tables. IOW an unpinned,
> >> unmapped BO will remain in VRAM until the memory is actually needed.
> >>
> >> Best regards
> >> Thomas
> >>
> >>>
> >>> So I'm still not seeing how this can go boom.
> >>>
> >>> Now long term it'd be nice to cut everything over to dma_resv locking, but
> >>> the issue there is that beyond ttm, none of the helpers (and few of the
> >>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
> >>> interim fix seems like the right solution to me.
> >>> -Daniel
> >>>
> >>>>
> >>>> Regards,
> >>>> Christian.
> >>>>
> >>>>>
> >>>>> Best regards
> >>>>> Thomas
> >>>>>
> >>>>>>
> >>>>>>> There's no recursion taking place, so I guess the reservation
> >>>>>>> lock could be
> >>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
> >>>>>>> separate pair of
> >>>>>>> DRM client functions could do the locking.
> >>>>>>
> >>>>>> Given how this "do the right locking" is a can of worms (and I think
> >>>>>> it's
> >>>>>> worse than what you dug out already) I think the fb_helper.lock hack is
> >>>>>> perfectly good enough.
> >>>>>>
> >>>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
> >>>>>> code, while many helpers/drivers still have their hand-rolled locking,
> >>>>>> will make conversion over to dma_resv needlessly more complicated.
> >>>>>> -Daniel
> >>>>>>
> >>>>>>>
> >>>>>>> Best regards
> >>>>>>> Thomas
> >>>>>>>
> >>>>>>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Please note that the reservation lock you need to take here is part of
> >>>>>>>> the GEM object.
> >>>>>>>>
> >>>>>>>> Usually we design things in the way that the code needs to take a lock
> >>>>>>>> which protects an object, then do some operations with the object and
> >>>>>>>> then release the lock again.
> >>>>>>>>
> >>>>>>>> Having in the lock inside the operation can be done as well, but
> >>>>>>>> returning with it is kind of unusual design.
> >>>>>>>>
> >>>>>>>>> Sorry for the noob questions. I'm still trying to understand the
> >>>>>>>>> implications of acquiring these locks.
> >>>>>>>>
> >>>>>>>> Well this is the reservation lock of the GEM object we are
> >>>>>>>> talking about
> >>>>>>>> here. We need to take that for a couple of different operations,
> >>>>>>>> vmap/vunmap doesn't sound like a special case to me.
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Christian.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Best regards
> >>>>>>>>> Thomas
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> dri-devel mailing list
> >>>>>>>> dri-devel@lists.freedesktop.org
> >>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >>>>>>>
> >>>>>>> --
> >>>>>>> Thomas Zimmermann
> >>>>>>> Graphics Driver Developer
> >>>>>>> SUSE Software Solutions Germany GmbH
> >>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
> >>>>>>> (HRB 36809, AG Nürnberg)
> >>>>>>> Geschäftsführer: Felix Imendörffer
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >> --
> >> Thomas Zimmermann
> >> Graphics Driver Developer
> >> SUSE Software Solutions Germany GmbH
> >> Maxfeldstr. 5, 90409 Nürnberg, Germany
> >> (HRB 36809, AG Nürnberg)
> >> Geschäftsführer: Felix Imendörffer
> >
> >
> >
> >
> >
> >
>
> --
> Thomas Zimmermann
> Graphics Driver Developer
> SUSE Software Solutions Germany GmbH
> Maxfeldstr. 5, 90409 Nürnberg, Germany
> (HRB 36809, AG Nürnberg)
> Geschäftsführer: Felix Imendörffer



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Daniel Vetter]

On Thu, Nov 26, 2020 at 11:15 AM Thomas Zimmermann <tzimmermann@suse.de> wrote:
>
> Hi
>
> Am 25.11.20 um 17:32 schrieb Daniel Vetter:
> > [...]
> > I guess full locking is required :-/ I'm not exactly sure how to make this
> > happen with the current plethora of helpers ... I think we need an
> > _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.
>
> I think we might be able to get away without new callbacks.
>
> I looked through the sources that implement and use vmap. All the
> implementations are without taking resv locks. For locking, we can wrap
> them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked()
> that locks and vmaps should make this easy.
>
> In terms of implementation, only vram helpers do a pin+map in their vmap
> code. And as I mentioned before, this is actually wrong. The pattern
> dates back to when the code was still in individual drivers. It's time
> to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.
>
> Finally, there aren't that many users of vmap. A few drivers use it
> while blitting framebuffers into HW buffers and ast does some permanent
> mapping of the cursor BO. All this is trivial to turn into small pairs
> of lock+vmap and vunmap+unlock.
>
> That leaves generic fbdev. The shadow buffer is also trivial to fix, as
> outlined during this discussion.
>
> The code for fbdev in hardware buffers is a special case. It vmaps the
> buffer during initialization and only vunmaps it during shutdown. As
> this has worked so far without locking, I'd leave it as it is and put a
> big comment next to is.
>
> Hardware fbdev buffers is only required by few drivers; namely those
> that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work.
> We should consider to make the fbdev shadow buffer the default and have
> drivers opt-in for the hardware buffer, if they need it.
>
> >
> > And then document that if the callers of the _locked version wants a
> > permanent mapping, it also needs to pin it. Plus I guess ideally implement
> > the unlocked/permanent versions in terms of that, so that drivers only
> > have to implement one or the other.
>
> For my understanding, pinning is only done in prepare_fb code. And ast
> pins its cursor BOs into vram. We should document to hols vmap/vunmap
> only for time and cover them with resv locks. Pinning is for cases where
> the hardware requires buffers in a special location, but does not
> protect against concurrent threat. I think those are the implicit rules
> already.
>
> I updated the radeon patchset, where all this appears to be working well.

Hm yeah if you want to do the full change, then that works out too.
It's just a pile of work.

But if we can finish off with an dma_resv_assert_locked in
dma_buf_vmap/vunmap, then I think that's ok. It does mean that
exporters must implement vmap caching, or performance will be
terrible. So quite some update for the dma-buf docs.

But if you're willing to do all that conversion of callers, then of
course I'm not stopping you. Not at all, it's great to see that kind
of maze untangled.
-Daniel

>
> Best regards
> Thomas
>
> >
> > That should give us at least some way forward to gradually conver all the
> > drivers and helpers over to dma_resv locking.
> > -Daniel
> >
> >> The pin count is currently maintained by the vmap implementation in vram
> >> helpers. Calling vmap is an implicit pin; calling vunmap is an implicit
> >> unpin. This prevents eviction in the damage worker. But now I was told than
> >> pinning is only for BOs that are controlled by userspace and internal users
> >> should acquire the resv lock. So vram helpers have to be fixed, actually.
> >>
> >> In vram helpers, unmapping does not mean eviction. The unmap operation only
> >> marks the BO as unmappable. The real unmap happens when the eviction takes
> >> place. This avoids many modifications to the page tables. IOW an unpinned,
> >> unmapped BO will remain in VRAM until the memory is actually needed.
> >>
> >> Best regards
> >> Thomas
> >>
> >>>
> >>> So I'm still not seeing how this can go boom.
> >>>
> >>> Now long term it'd be nice to cut everything over to dma_resv locking, but
> >>> the issue there is that beyond ttm, none of the helpers (and few of the
> >>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
> >>> interim fix seems like the right solution to me.
> >>> -Daniel
> >>>
> >>>>
> >>>> Regards,
> >>>> Christian.
> >>>>
> >>>>>
> >>>>> Best regards
> >>>>> Thomas
> >>>>>
> >>>>>>
> >>>>>>> There's no recursion taking place, so I guess the reservation
> >>>>>>> lock could be
> >>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
> >>>>>>> separate pair of
> >>>>>>> DRM client functions could do the locking.
> >>>>>>
> >>>>>> Given how this "do the right locking" is a can of worms (and I think
> >>>>>> it's
> >>>>>> worse than what you dug out already) I think the fb_helper.lock hack is
> >>>>>> perfectly good enough.
> >>>>>>
> >>>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
> >>>>>> code, while many helpers/drivers still have their hand-rolled locking,
> >>>>>> will make conversion over to dma_resv needlessly more complicated.
> >>>>>> -Daniel
> >>>>>>
> >>>>>>>
> >>>>>>> Best regards
> >>>>>>> Thomas
> >>>>>>>
> >>>>>>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Please note that the reservation lock you need to take here is part of
> >>>>>>>> the GEM object.
> >>>>>>>>
> >>>>>>>> Usually we design things in the way that the code needs to take a lock
> >>>>>>>> which protects an object, then do some operations with the object and
> >>>>>>>> then release the lock again.
> >>>>>>>>
> >>>>>>>> Having in the lock inside the operation can be done as well, but
> >>>>>>>> returning with it is kind of unusual design.
> >>>>>>>>
> >>>>>>>>> Sorry for the noob questions. I'm still trying to understand the
> >>>>>>>>> implications of acquiring these locks.
> >>>>>>>>
> >>>>>>>> Well this is the reservation lock of the GEM object we are
> >>>>>>>> talking about
> >>>>>>>> here. We need to take that for a couple of different operations,
> >>>>>>>> vmap/vunmap doesn't sound like a special case to me.
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Christian.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Best regards
> >>>>>>>>> Thomas
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> dri-devel mailing list
> >>>>>>>> dri-devel@lists.freedesktop.org
> >>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >>>>>>>
> >>>>>>> --
> >>>>>>> Thomas Zimmermann
> >>>>>>> Graphics Driver Developer
> >>>>>>> SUSE Software Solutions Germany GmbH
> >>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
> >>>>>>> (HRB 36809, AG Nürnberg)
> >>>>>>> Geschäftsführer: Felix Imendörffer
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >> --
> >> Thomas Zimmermann
> >> Graphics Driver Developer
> >> SUSE Software Solutions Germany GmbH
> >> Maxfeldstr. 5, 90409 Nürnberg, Germany
> >> (HRB 36809, AG Nürnberg)
> >> Geschäftsführer: Felix Imendörffer
> >
> >
> >
> >
> >
> >
>
> --
> Thomas Zimmermann
> Graphics Driver Developer
> SUSE Software Solutions Germany GmbH
> Maxfeldstr. 5, 90409 Nürnberg, Germany
> (HRB 36809, AG Nürnberg)
> Geschäftsführer: Felix Imendörffer



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 26.11.20 um 12:04 schrieb Daniel Vetter:
> On Thu, Nov 26, 2020 at 11:15 AM Thomas Zimmermann <tzimmermann@suse.de> wrote:
>> Hi
>>
>> Am 25.11.20 um 17:32 schrieb Daniel Vetter:
>>> [...]
>>> I guess full locking is required :-/ I'm not exactly sure how to make this
>>> happen with the current plethora of helpers ... I think we need an
>>> _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.
>> I think we might be able to get away without new callbacks.
>>
>> I looked through the sources that implement and use vmap. All the
>> implementations are without taking resv locks. For locking, we can wrap
>> them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked()
>> that locks and vmaps should make this easy.
>>
>> In terms of implementation, only vram helpers do a pin+map in their vmap
>> code. And as I mentioned before, this is actually wrong. The pattern
>> dates back to when the code was still in individual drivers. It's time
>> to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.
>>
>> Finally, there aren't that many users of vmap. A few drivers use it
>> while blitting framebuffers into HW buffers and ast does some permanent
>> mapping of the cursor BO. All this is trivial to turn into small pairs
>> of lock+vmap and vunmap+unlock.
>>
>> That leaves generic fbdev. The shadow buffer is also trivial to fix, as
>> outlined during this discussion.
>>
>> The code for fbdev in hardware buffers is a special case. It vmaps the
>> buffer during initialization and only vunmaps it during shutdown. As
>> this has worked so far without locking, I'd leave it as it is and put a
>> big comment next to is.

Please keep in mind that you only need to grab the lock if the buffer is 
not pinned otherwise.

In other words when we are scanning out from the BO it is guaranteed 
that it can't move around.

Maybe this makes the case here easier to handle.

>>
>> Hardware fbdev buffers is only required by few drivers; namely those
>> that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work.
>> We should consider to make the fbdev shadow buffer the default and have
>> drivers opt-in for the hardware buffer, if they need it.
>>
>>> And then document that if the callers of the _locked version wants a
>>> permanent mapping, it also needs to pin it. Plus I guess ideally implement
>>> the unlocked/permanent versions in terms of that, so that drivers only
>>> have to implement one or the other.
>> For my understanding, pinning is only done in prepare_fb code. And ast
>> pins its cursor BOs into vram. We should document to hols vmap/vunmap
>> only for time and cover them with resv locks. Pinning is for cases where
>> the hardware requires buffers in a special location, but does not
>> protect against concurrent threat. I think those are the implicit rules
>> already.
>>
>> I updated the radeon patchset, where all this appears to be working well.
> Hm yeah if you want to do the full change, then that works out too.
> It's just a pile of work.
>
> But if we can finish off with an dma_resv_assert_locked in
> dma_buf_vmap/vunmap, then I think that's ok. It does mean that
> exporters must implement vmap caching, or performance will be
> terrible. So quite some update for the dma-buf docs.

That's one possibility, but I think we should keep the ability to use 
pin+vmap instead of lock+vmap.

Regards,
Christian.

>
> But if you're willing to do all that conversion of callers, then of
> course I'm not stopping you. Not at all, it's great to see that kind
> of maze untangled.
> -Daniel
>
>> Best regards
>> Thomas
>>
>>> That should give us at least some way forward to gradually conver all the
>>> drivers and helpers over to dma_resv locking.
>>> -Daniel
>>>
>>>> The pin count is currently maintained by the vmap implementation in vram
>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an implicit
>>>> unpin. This prevents eviction in the damage worker. But now I was told than
>>>> pinning is only for BOs that are controlled by userspace and internal users
>>>> should acquire the resv lock. So vram helpers have to be fixed, actually.
>>>>
>>>> In vram helpers, unmapping does not mean eviction. The unmap operation only
>>>> marks the BO as unmappable. The real unmap happens when the eviction takes
>>>> place. This avoids many modifications to the page tables. IOW an unpinned,
>>>> unmapped BO will remain in VRAM until the memory is actually needed.
>>>>
>>>> Best regards
>>>> Thomas
>>>>
>>>>> So I'm still not seeing how this can go boom.
>>>>>
>>>>> Now long term it'd be nice to cut everything over to dma_resv locking, but
>>>>> the issue there is that beyond ttm, none of the helpers (and few of the
>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>>>> interim fix seems like the right solution to me.
>>>>> -Daniel
>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>>>
>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>> lock could be
>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>> separate pair of
>>>>>>>>> DRM client functions could do the locking.
>>>>>>>> Given how this "do the right locking" is a can of worms (and I think
>>>>>>>> it's
>>>>>>>> worse than what you dug out already) I think the fb_helper.lock hack is
>>>>>>>> perfectly good enough.
>>>>>>>>
>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
>>>>>>>> code, while many helpers/drivers still have their hand-rolled locking,
>>>>>>>> will make conversion over to dma_resv needlessly more complicated.
>>>>>>>> -Daniel
>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>> Thomas
>>>>>>>>>
>>>>>>>>> [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcgit.freedesktop.org%2Fdrm%2Fdrm-tip%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%3Fid%3Dac60f3f3090115d21f028bffa2dcfb67f695c4f2%23n394&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682660550%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Cky%2BozENU1nsd4hlfAdsvA6wC0RXsex7gpFuvHlCROM%3D&amp;reserved=0
>>>>>>>>>
>>>>>>>>>> Please note that the reservation lock you need to take here is part of
>>>>>>>>>> the GEM object.
>>>>>>>>>>
>>>>>>>>>> Usually we design things in the way that the code needs to take a lock
>>>>>>>>>> which protects an object, then do some operations with the object and
>>>>>>>>>> then release the lock again.
>>>>>>>>>>
>>>>>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>
>>>>>>>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>> talking about
>>>>>>>>>> here. We need to take that for a couple of different operations,
>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Christian.
>>>>>>>>>>
>>>>>>>>>>> Best regards
>>>>>>>>>>> Thomas
>>>>>>>>>> _______________________________________________
>>>>>>>>>> dri-devel mailing list
>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682670543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Sa5ao1X5JGFgcnhNiDbCjI4SlMMWzHITBylAZsG%2BVzs%3D&amp;reserved=0
>>>>>>>>> --
>>>>>>>>> Thomas Zimmermann
>>>>>>>>> Graphics Driver Developer
>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>> --
>>>> Thomas Zimmermann
>>>> Graphics Driver Developer
>>>> SUSE Software Solutions Germany GmbH
>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>> (HRB 36809, AG Nürnberg)
>>>> Geschäftsführer: Felix Imendörffer
>>>
>>>
>>>
>>>
>>>
>> --
>> Thomas Zimmermann
>> Graphics Driver Developer
>> SUSE Software Solutions Germany GmbH
>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>> (HRB 36809, AG Nürnberg)
>> Geschäftsführer: Felix Imendörffer
>
>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 26.11.20 um 12:04 schrieb Daniel Vetter:
> On Thu, Nov 26, 2020 at 11:15 AM Thomas Zimmermann <tzimmermann@suse.de> wrote:
>> Hi
>>
>> Am 25.11.20 um 17:32 schrieb Daniel Vetter:
>>> [...]
>>> I guess full locking is required :-/ I'm not exactly sure how to make this
>>> happen with the current plethora of helpers ... I think we need an
>>> _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.
>> I think we might be able to get away without new callbacks.
>>
>> I looked through the sources that implement and use vmap. All the
>> implementations are without taking resv locks. For locking, we can wrap
>> them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked()
>> that locks and vmaps should make this easy.
>>
>> In terms of implementation, only vram helpers do a pin+map in their vmap
>> code. And as I mentioned before, this is actually wrong. The pattern
>> dates back to when the code was still in individual drivers. It's time
>> to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.
>>
>> Finally, there aren't that many users of vmap. A few drivers use it
>> while blitting framebuffers into HW buffers and ast does some permanent
>> mapping of the cursor BO. All this is trivial to turn into small pairs
>> of lock+vmap and vunmap+unlock.
>>
>> That leaves generic fbdev. The shadow buffer is also trivial to fix, as
>> outlined during this discussion.
>>
>> The code for fbdev in hardware buffers is a special case. It vmaps the
>> buffer during initialization and only vunmaps it during shutdown. As
>> this has worked so far without locking, I'd leave it as it is and put a
>> big comment next to is.

Please keep in mind that you only need to grab the lock if the buffer is 
not pinned otherwise.

In other words when we are scanning out from the BO it is guaranteed 
that it can't move around.

Maybe this makes the case here easier to handle.

>>
>> Hardware fbdev buffers is only required by few drivers; namely those
>> that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work.
>> We should consider to make the fbdev shadow buffer the default and have
>> drivers opt-in for the hardware buffer, if they need it.
>>
>>> And then document that if the callers of the _locked version wants a
>>> permanent mapping, it also needs to pin it. Plus I guess ideally implement
>>> the unlocked/permanent versions in terms of that, so that drivers only
>>> have to implement one or the other.
>> For my understanding, pinning is only done in prepare_fb code. And ast
>> pins its cursor BOs into vram. We should document to hols vmap/vunmap
>> only for time and cover them with resv locks. Pinning is for cases where
>> the hardware requires buffers in a special location, but does not
>> protect against concurrent threat. I think those are the implicit rules
>> already.
>>
>> I updated the radeon patchset, where all this appears to be working well.
> Hm yeah if you want to do the full change, then that works out too.
> It's just a pile of work.
>
> But if we can finish off with an dma_resv_assert_locked in
> dma_buf_vmap/vunmap, then I think that's ok. It does mean that
> exporters must implement vmap caching, or performance will be
> terrible. So quite some update for the dma-buf docs.

That's one possibility, but I think we should keep the ability to use 
pin+vmap instead of lock+vmap.

Regards,
Christian.

>
> But if you're willing to do all that conversion of callers, then of
> course I'm not stopping you. Not at all, it's great to see that kind
> of maze untangled.
> -Daniel
>
>> Best regards
>> Thomas
>>
>>> That should give us at least some way forward to gradually conver all the
>>> drivers and helpers over to dma_resv locking.
>>> -Daniel
>>>
>>>> The pin count is currently maintained by the vmap implementation in vram
>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an implicit
>>>> unpin. This prevents eviction in the damage worker. But now I was told than
>>>> pinning is only for BOs that are controlled by userspace and internal users
>>>> should acquire the resv lock. So vram helpers have to be fixed, actually.
>>>>
>>>> In vram helpers, unmapping does not mean eviction. The unmap operation only
>>>> marks the BO as unmappable. The real unmap happens when the eviction takes
>>>> place. This avoids many modifications to the page tables. IOW an unpinned,
>>>> unmapped BO will remain in VRAM until the memory is actually needed.
>>>>
>>>> Best regards
>>>> Thomas
>>>>
>>>>> So I'm still not seeing how this can go boom.
>>>>>
>>>>> Now long term it'd be nice to cut everything over to dma_resv locking, but
>>>>> the issue there is that beyond ttm, none of the helpers (and few of the
>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>>>> interim fix seems like the right solution to me.
>>>>> -Daniel
>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>>>
>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>> lock could be
>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>> separate pair of
>>>>>>>>> DRM client functions could do the locking.
>>>>>>>> Given how this "do the right locking" is a can of worms (and I think
>>>>>>>> it's
>>>>>>>> worse than what you dug out already) I think the fb_helper.lock hack is
>>>>>>>> perfectly good enough.
>>>>>>>>
>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
>>>>>>>> code, while many helpers/drivers still have their hand-rolled locking,
>>>>>>>> will make conversion over to dma_resv needlessly more complicated.
>>>>>>>> -Daniel
>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>> Thomas
>>>>>>>>>
>>>>>>>>> [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcgit.freedesktop.org%2Fdrm%2Fdrm-tip%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%3Fid%3Dac60f3f3090115d21f028bffa2dcfb67f695c4f2%23n394&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682660550%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Cky%2BozENU1nsd4hlfAdsvA6wC0RXsex7gpFuvHlCROM%3D&amp;reserved=0
>>>>>>>>>
>>>>>>>>>> Please note that the reservation lock you need to take here is part of
>>>>>>>>>> the GEM object.
>>>>>>>>>>
>>>>>>>>>> Usually we design things in the way that the code needs to take a lock
>>>>>>>>>> which protects an object, then do some operations with the object and
>>>>>>>>>> then release the lock again.
>>>>>>>>>>
>>>>>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>
>>>>>>>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>> talking about
>>>>>>>>>> here. We need to take that for a couple of different operations,
>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Christian.
>>>>>>>>>>
>>>>>>>>>>> Best regards
>>>>>>>>>>> Thomas
>>>>>>>>>> _______________________________________________
>>>>>>>>>> dri-devel mailing list
>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682670543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Sa5ao1X5JGFgcnhNiDbCjI4SlMMWzHITBylAZsG%2BVzs%3D&amp;reserved=0
>>>>>>>>> --
>>>>>>>>> Thomas Zimmermann
>>>>>>>>> Graphics Driver Developer
>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>> --
>>>> Thomas Zimmermann
>>>> Graphics Driver Developer
>>>> SUSE Software Solutions Germany GmbH
>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>> (HRB 36809, AG Nürnberg)
>>>> Geschäftsführer: Felix Imendörffer
>>>
>>>
>>>
>>>
>>>
>> --
>> Thomas Zimmermann
>> Graphics Driver Developer
>> SUSE Software Solutions Germany GmbH
>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>> (HRB 36809, AG Nürnberg)
>> Geschäftsführer: Felix Imendörffer
>
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 9860 bytes --]

Hi

Am 26.11.20 um 12:28 schrieb Christian König:
> Am 26.11.20 um 12:04 schrieb Daniel Vetter:
>> On Thu, Nov 26, 2020 at 11:15 AM Thomas Zimmermann 
>> <tzimmermann@suse.de> wrote:
>>> Hi
>>>
>>> Am 25.11.20 um 17:32 schrieb Daniel Vetter:
>>>> [...]
>>>> I guess full locking is required :-/ I'm not exactly sure how to 
>>>> make this
>>>> happen with the current plethora of helpers ... I think we need an
>>>> _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.
>>> I think we might be able to get away without new callbacks.
>>>
>>> I looked through the sources that implement and use vmap. All the
>>> implementations are without taking resv locks. For locking, we can wrap
>>> them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked()
>>> that locks and vmaps should make this easy.
>>>
>>> In terms of implementation, only vram helpers do a pin+map in their vmap
>>> code. And as I mentioned before, this is actually wrong. The pattern
>>> dates back to when the code was still in individual drivers. It's time
>>> to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.
>>>
>>> Finally, there aren't that many users of vmap. A few drivers use it
>>> while blitting framebuffers into HW buffers and ast does some permanent
>>> mapping of the cursor BO. All this is trivial to turn into small pairs
>>> of lock+vmap and vunmap+unlock.
>>>
>>> That leaves generic fbdev. The shadow buffer is also trivial to fix, as
>>> outlined during this discussion.
>>>
>>> The code for fbdev in hardware buffers is a special case. It vmaps the
>>> buffer during initialization and only vunmaps it during shutdown. As
>>> this has worked so far without locking, I'd leave it as it is and put a
>>> big comment next to is.
> 
> Please keep in mind that you only need to grab the lock if the buffer is 
> not pinned otherwise.
> 
> In other words when we are scanning out from the BO it is guaranteed 
> that it can't move around.
> 
> Maybe this makes the case here easier to handle.

The fbdev code is already fragile. If no shadow FB is selected, the 
hardware BO is vmapped, but never pinned; if only for the reason that 
there's no useful generic interface to do this. So we cannot lock for 
longer periods, but it's also not pinned either. This really only work 
with a few drivers that use CMA helpers, where BOs don't move.

Best regards
Thomas

> 
>>>
>>> Hardware fbdev buffers is only required by few drivers; namely those
>>> that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work.
>>> We should consider to make the fbdev shadow buffer the default and have
>>> drivers opt-in for the hardware buffer, if they need it.
>>>
>>>> And then document that if the callers of the _locked version wants a
>>>> permanent mapping, it also needs to pin it. Plus I guess ideally 
>>>> implement
>>>> the unlocked/permanent versions in terms of that, so that drivers only
>>>> have to implement one or the other.
>>> For my understanding, pinning is only done in prepare_fb code. And ast
>>> pins its cursor BOs into vram. We should document to hols vmap/vunmap
>>> only for time and cover them with resv locks. Pinning is for cases where
>>> the hardware requires buffers in a special location, but does not
>>> protect against concurrent threat. I think those are the implicit rules
>>> already.
>>>
>>> I updated the radeon patchset, where all this appears to be working 
>>> well.
>> Hm yeah if you want to do the full change, then that works out too.
>> It's just a pile of work.
>>
>> But if we can finish off with an dma_resv_assert_locked in
>> dma_buf_vmap/vunmap, then I think that's ok. It does mean that
>> exporters must implement vmap caching, or performance will be
>> terrible. So quite some update for the dma-buf docs.
> 
> That's one possibility, but I think we should keep the ability to use 
> pin+vmap instead of lock+vmap.
> 
> Regards,
> Christian.
> 
>>
>> But if you're willing to do all that conversion of callers, then of
>> course I'm not stopping you. Not at all, it's great to see that kind
>> of maze untangled.
>> -Daniel
>>
>>> Best regards
>>> Thomas
>>>
>>>> That should give us at least some way forward to gradually conver 
>>>> all the
>>>> drivers and helpers over to dma_resv locking.
>>>> -Daniel
>>>>
>>>>> The pin count is currently maintained by the vmap implementation in 
>>>>> vram
>>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an 
>>>>> implicit
>>>>> unpin. This prevents eviction in the damage worker. But now I was 
>>>>> told than
>>>>> pinning is only for BOs that are controlled by userspace and 
>>>>> internal users
>>>>> should acquire the resv lock. So vram helpers have to be fixed, 
>>>>> actually.
>>>>>
>>>>> In vram helpers, unmapping does not mean eviction. The unmap 
>>>>> operation only
>>>>> marks the BO as unmappable. The real unmap happens when the 
>>>>> eviction takes
>>>>> place. This avoids many modifications to the page tables. IOW an 
>>>>> unpinned,
>>>>> unmapped BO will remain in VRAM until the memory is actually needed.
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>>> So I'm still not seeing how this can go boom.
>>>>>>
>>>>>> Now long term it'd be nice to cut everything over to dma_resv 
>>>>>> locking, but
>>>>>> the issue there is that beyond ttm, none of the helpers (and few 
>>>>>> of the
>>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>>>>> interim fix seems like the right solution to me.
>>>>>> -Daniel
>>>>>>
>>>>>>> Regards,
>>>>>>> Christian.
>>>>>>>
>>>>>>>> Best regards
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>>> lock could be
>>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>>> separate pair of
>>>>>>>>>> DRM client functions could do the locking.
>>>>>>>>> Given how this "do the right locking" is a can of worms (and I 
>>>>>>>>> think
>>>>>>>>> it's
>>>>>>>>> worse than what you dug out already) I think the fb_helper.lock 
>>>>>>>>> hack is
>>>>>>>>> perfectly good enough.
>>>>>>>>>
>>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock in 
>>>>>>>>> generic
>>>>>>>>> code, while many helpers/drivers still have their hand-rolled 
>>>>>>>>> locking,
>>>>>>>>> will make conversion over to dma_resv needlessly more complicated.
>>>>>>>>> -Daniel
>>>>>>>>>
>>>>>>>>>> Best regards
>>>>>>>>>> Thomas
>>>>>>>>>>
>>>>>>>>>> [1] 
>>>>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcgit.freedesktop.org%2Fdrm%2Fdrm-tip%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%3Fid%3Dac60f3f3090115d21f028bffa2dcfb67f695c4f2%23n394&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682660550%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Cky%2BozENU1nsd4hlfAdsvA6wC0RXsex7gpFuvHlCROM%3D&amp;reserved=0 
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Please note that the reservation lock you need to take here 
>>>>>>>>>>> is part of
>>>>>>>>>>> the GEM object.
>>>>>>>>>>>
>>>>>>>>>>> Usually we design things in the way that the code needs to 
>>>>>>>>>>> take a lock
>>>>>>>>>>> which protects an object, then do some operations with the 
>>>>>>>>>>> object and
>>>>>>>>>>> then release the lock again.
>>>>>>>>>>>
>>>>>>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>>
>>>>>>>>>>>> Sorry for the noob questions. I'm still trying to understand 
>>>>>>>>>>>> the
>>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>>> talking about
>>>>>>>>>>> here. We need to take that for a couple of different operations,
>>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Christian.
>>>>>>>>>>>
>>>>>>>>>>>> Best regards
>>>>>>>>>>>> Thomas
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> dri-devel mailing list
>>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682670543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Sa5ao1X5JGFgcnhNiDbCjI4SlMMWzHITBylAZsG%2BVzs%3D&amp;reserved=0 
>>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> Thomas Zimmermann
>>>>>>>>>> Graphics Driver Developer
>>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> -- 
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH
>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>> (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Felix Imendörffer
>>>>
>>>>
>>>>
>>>>
>>>>
>>> -- 
>>> Thomas Zimmermann
>>> Graphics Driver Developer
>>> SUSE Software Solutions Germany GmbH
>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>> (HRB 36809, AG Nürnberg)
>>> Geschäftsführer: Felix Imendörffer
>>
>>
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 8003 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 9860 bytes --]

Hi

Am 26.11.20 um 12:28 schrieb Christian König:
> Am 26.11.20 um 12:04 schrieb Daniel Vetter:
>> On Thu, Nov 26, 2020 at 11:15 AM Thomas Zimmermann 
>> <tzimmermann@suse.de> wrote:
>>> Hi
>>>
>>> Am 25.11.20 um 17:32 schrieb Daniel Vetter:
>>>> [...]
>>>> I guess full locking is required :-/ I'm not exactly sure how to 
>>>> make this
>>>> happen with the current plethora of helpers ... I think we need an
>>>> _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.
>>> I think we might be able to get away without new callbacks.
>>>
>>> I looked through the sources that implement and use vmap. All the
>>> implementations are without taking resv locks. For locking, we can wrap
>>> them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked()
>>> that locks and vmaps should make this easy.
>>>
>>> In terms of implementation, only vram helpers do a pin+map in their vmap
>>> code. And as I mentioned before, this is actually wrong. The pattern
>>> dates back to when the code was still in individual drivers. It's time
>>> to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.
>>>
>>> Finally, there aren't that many users of vmap. A few drivers use it
>>> while blitting framebuffers into HW buffers and ast does some permanent
>>> mapping of the cursor BO. All this is trivial to turn into small pairs
>>> of lock+vmap and vunmap+unlock.
>>>
>>> That leaves generic fbdev. The shadow buffer is also trivial to fix, as
>>> outlined during this discussion.
>>>
>>> The code for fbdev in hardware buffers is a special case. It vmaps the
>>> buffer during initialization and only vunmaps it during shutdown. As
>>> this has worked so far without locking, I'd leave it as it is and put a
>>> big comment next to is.
> 
> Please keep in mind that you only need to grab the lock if the buffer is 
> not pinned otherwise.
> 
> In other words when we are scanning out from the BO it is guaranteed 
> that it can't move around.
> 
> Maybe this makes the case here easier to handle.

The fbdev code is already fragile. If no shadow FB is selected, the 
hardware BO is vmapped, but never pinned; if only for the reason that 
there's no useful generic interface to do this. So we cannot lock for 
longer periods, but it's also not pinned either. This really only work 
with a few drivers that use CMA helpers, where BOs don't move.

Best regards
Thomas

> 
>>>
>>> Hardware fbdev buffers is only required by few drivers; namely those
>>> that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work.
>>> We should consider to make the fbdev shadow buffer the default and have
>>> drivers opt-in for the hardware buffer, if they need it.
>>>
>>>> And then document that if the callers of the _locked version wants a
>>>> permanent mapping, it also needs to pin it. Plus I guess ideally 
>>>> implement
>>>> the unlocked/permanent versions in terms of that, so that drivers only
>>>> have to implement one or the other.
>>> For my understanding, pinning is only done in prepare_fb code. And ast
>>> pins its cursor BOs into vram. We should document to hols vmap/vunmap
>>> only for time and cover them with resv locks. Pinning is for cases where
>>> the hardware requires buffers in a special location, but does not
>>> protect against concurrent threat. I think those are the implicit rules
>>> already.
>>>
>>> I updated the radeon patchset, where all this appears to be working 
>>> well.
>> Hm yeah if you want to do the full change, then that works out too.
>> It's just a pile of work.
>>
>> But if we can finish off with an dma_resv_assert_locked in
>> dma_buf_vmap/vunmap, then I think that's ok. It does mean that
>> exporters must implement vmap caching, or performance will be
>> terrible. So quite some update for the dma-buf docs.
> 
> That's one possibility, but I think we should keep the ability to use 
> pin+vmap instead of lock+vmap.
> 
> Regards,
> Christian.
> 
>>
>> But if you're willing to do all that conversion of callers, then of
>> course I'm not stopping you. Not at all, it's great to see that kind
>> of maze untangled.
>> -Daniel
>>
>>> Best regards
>>> Thomas
>>>
>>>> That should give us at least some way forward to gradually conver 
>>>> all the
>>>> drivers and helpers over to dma_resv locking.
>>>> -Daniel
>>>>
>>>>> The pin count is currently maintained by the vmap implementation in 
>>>>> vram
>>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an 
>>>>> implicit
>>>>> unpin. This prevents eviction in the damage worker. But now I was 
>>>>> told than
>>>>> pinning is only for BOs that are controlled by userspace and 
>>>>> internal users
>>>>> should acquire the resv lock. So vram helpers have to be fixed, 
>>>>> actually.
>>>>>
>>>>> In vram helpers, unmapping does not mean eviction. The unmap 
>>>>> operation only
>>>>> marks the BO as unmappable. The real unmap happens when the 
>>>>> eviction takes
>>>>> place. This avoids many modifications to the page tables. IOW an 
>>>>> unpinned,
>>>>> unmapped BO will remain in VRAM until the memory is actually needed.
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>>> So I'm still not seeing how this can go boom.
>>>>>>
>>>>>> Now long term it'd be nice to cut everything over to dma_resv 
>>>>>> locking, but
>>>>>> the issue there is that beyond ttm, none of the helpers (and few 
>>>>>> of the
>>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>>>>> interim fix seems like the right solution to me.
>>>>>> -Daniel
>>>>>>
>>>>>>> Regards,
>>>>>>> Christian.
>>>>>>>
>>>>>>>> Best regards
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>>> lock could be
>>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>>> separate pair of
>>>>>>>>>> DRM client functions could do the locking.
>>>>>>>>> Given how this "do the right locking" is a can of worms (and I 
>>>>>>>>> think
>>>>>>>>> it's
>>>>>>>>> worse than what you dug out already) I think the fb_helper.lock 
>>>>>>>>> hack is
>>>>>>>>> perfectly good enough.
>>>>>>>>>
>>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock in 
>>>>>>>>> generic
>>>>>>>>> code, while many helpers/drivers still have their hand-rolled 
>>>>>>>>> locking,
>>>>>>>>> will make conversion over to dma_resv needlessly more complicated.
>>>>>>>>> -Daniel
>>>>>>>>>
>>>>>>>>>> Best regards
>>>>>>>>>> Thomas
>>>>>>>>>>
>>>>>>>>>> [1] 
>>>>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcgit.freedesktop.org%2Fdrm%2Fdrm-tip%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%3Fid%3Dac60f3f3090115d21f028bffa2dcfb67f695c4f2%23n394&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682660550%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Cky%2BozENU1nsd4hlfAdsvA6wC0RXsex7gpFuvHlCROM%3D&amp;reserved=0 
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Please note that the reservation lock you need to take here 
>>>>>>>>>>> is part of
>>>>>>>>>>> the GEM object.
>>>>>>>>>>>
>>>>>>>>>>> Usually we design things in the way that the code needs to 
>>>>>>>>>>> take a lock
>>>>>>>>>>> which protects an object, then do some operations with the 
>>>>>>>>>>> object and
>>>>>>>>>>> then release the lock again.
>>>>>>>>>>>
>>>>>>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>>
>>>>>>>>>>>> Sorry for the noob questions. I'm still trying to understand 
>>>>>>>>>>>> the
>>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>>> talking about
>>>>>>>>>>> here. We need to take that for a couple of different operations,
>>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Christian.
>>>>>>>>>>>
>>>>>>>>>>>> Best regards
>>>>>>>>>>>> Thomas
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> dri-devel mailing list
>>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682670543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Sa5ao1X5JGFgcnhNiDbCjI4SlMMWzHITBylAZsG%2BVzs%3D&amp;reserved=0 
>>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> Thomas Zimmermann
>>>>>>>>>> Graphics Driver Developer
>>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>> -- 
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH
>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>> (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Felix Imendörffer
>>>>
>>>>
>>>>
>>>>
>>>>
>>> -- 
>>> Thomas Zimmermann
>>> Graphics Driver Developer
>>> SUSE Software Solutions Germany GmbH
>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>> (HRB 36809, AG Nürnberg)
>>> Geschäftsführer: Felix Imendörffer
>>
>>
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 8003 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 8582 bytes --]

Hi

Am 26.11.20 um 12:04 schrieb Daniel Vetter:
> On Thu, Nov 26, 2020 at 11:15 AM Thomas Zimmermann <tzimmermann@suse.de> wrote:
>>
>> Hi
>>
>> Am 25.11.20 um 17:32 schrieb Daniel Vetter:
>>> [...]
>>> I guess full locking is required :-/ I'm not exactly sure how to make this
>>> happen with the current plethora of helpers ... I think we need an
>>> _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.
>>
>> I think we might be able to get away without new callbacks.
>>
>> I looked through the sources that implement and use vmap. All the
>> implementations are without taking resv locks. For locking, we can wrap
>> them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked()
>> that locks and vmaps should make this easy.
>>
>> In terms of implementation, only vram helpers do a pin+map in their vmap
>> code. And as I mentioned before, this is actually wrong. The pattern
>> dates back to when the code was still in individual drivers. It's time
>> to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.
>>
>> Finally, there aren't that many users of vmap. A few drivers use it
>> while blitting framebuffers into HW buffers and ast does some permanent
>> mapping of the cursor BO. All this is trivial to turn into small pairs
>> of lock+vmap and vunmap+unlock.
>>
>> That leaves generic fbdev. The shadow buffer is also trivial to fix, as
>> outlined during this discussion.
>>
>> The code for fbdev in hardware buffers is a special case. It vmaps the
>> buffer during initialization and only vunmaps it during shutdown. As
>> this has worked so far without locking, I'd leave it as it is and put a
>> big comment next to is.
>>
>> Hardware fbdev buffers is only required by few drivers; namely those
>> that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work.
>> We should consider to make the fbdev shadow buffer the default and have
>> drivers opt-in for the hardware buffer, if they need it.
>>
>>>
>>> And then document that if the callers of the _locked version wants a
>>> permanent mapping, it also needs to pin it. Plus I guess ideally implement
>>> the unlocked/permanent versions in terms of that, so that drivers only
>>> have to implement one or the other.
>>
>> For my understanding, pinning is only done in prepare_fb code. And ast
>> pins its cursor BOs into vram. We should document to hols vmap/vunmap
>> only for time and cover them with resv locks. Pinning is for cases where
>> the hardware requires buffers in a special location, but does not
>> protect against concurrent threat. I think those are the implicit rules
>> already.
>>
>> I updated the radeon patchset, where all this appears to be working well.
> 
> Hm yeah if you want to do the full change, then that works out too.
> It's just a pile of work.
> 
> But if we can finish off with an dma_resv_assert_locked in
> dma_buf_vmap/vunmap, then I think that's ok. It does mean that
> exporters must implement vmap caching, or performance will be
> terrible. So quite some update for the dma-buf docs.

Yeah, I remember a bug report about frequent page-table modifications 
wrt to vram helpers. So we implemented the lazy unmapping / vmap 
caching, as suggested by Christian back them. My guess is that anything 
TTM-based can use a similar pattern. Christian probably knows the corner 
cases.

CMA seems obviously working correctly already.

For SHMEM, I'd have to figure out the reference counting of the pages 
involved. My guess is that the vunmap in drm_gem_shmem_vunmap() could be 
moved into the free callback, plus a few other modifications.

Best regards
Thomas

> 
> But if you're willing to do all that conversion of callers, then of
> course I'm not stopping you. Not at all, it's great to see that kind
> of maze untangled.
> -Daniel
> 
>>
>> Best regards
>> Thomas
>>
>>>
>>> That should give us at least some way forward to gradually conver all the
>>> drivers and helpers over to dma_resv locking.
>>> -Daniel
>>>
>>>> The pin count is currently maintained by the vmap implementation in vram
>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an implicit
>>>> unpin. This prevents eviction in the damage worker. But now I was told than
>>>> pinning is only for BOs that are controlled by userspace and internal users
>>>> should acquire the resv lock. So vram helpers have to be fixed, actually.
>>>>
>>>> In vram helpers, unmapping does not mean eviction. The unmap operation only
>>>> marks the BO as unmappable. The real unmap happens when the eviction takes
>>>> place. This avoids many modifications to the page tables. IOW an unpinned,
>>>> unmapped BO will remain in VRAM until the memory is actually needed.
>>>>
>>>> Best regards
>>>> Thomas
>>>>
>>>>>
>>>>> So I'm still not seeing how this can go boom.
>>>>>
>>>>> Now long term it'd be nice to cut everything over to dma_resv locking, but
>>>>> the issue there is that beyond ttm, none of the helpers (and few of the
>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>>>> interim fix seems like the right solution to me.
>>>>> -Daniel
>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>>>
>>>>>>>>
>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>> lock could be
>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>> separate pair of
>>>>>>>>> DRM client functions could do the locking.
>>>>>>>>
>>>>>>>> Given how this "do the right locking" is a can of worms (and I think
>>>>>>>> it's
>>>>>>>> worse than what you dug out already) I think the fb_helper.lock hack is
>>>>>>>> perfectly good enough.
>>>>>>>>
>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
>>>>>>>> code, while many helpers/drivers still have their hand-rolled locking,
>>>>>>>> will make conversion over to dma_resv needlessly more complicated.
>>>>>>>> -Daniel
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>> Thomas
>>>>>>>>>
>>>>>>>>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please note that the reservation lock you need to take here is part of
>>>>>>>>>> the GEM object.
>>>>>>>>>>
>>>>>>>>>> Usually we design things in the way that the code needs to take a lock
>>>>>>>>>> which protects an object, then do some operations with the object and
>>>>>>>>>> then release the lock again.
>>>>>>>>>>
>>>>>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>
>>>>>>>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>>
>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>> talking about
>>>>>>>>>> here. We need to take that for a couple of different operations,
>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Christian.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Best regards
>>>>>>>>>>> Thomas
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> dri-devel mailing list
>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thomas Zimmermann
>>>>>>>>> Graphics Driver Developer
>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Thomas Zimmermann
>>>> Graphics Driver Developer
>>>> SUSE Software Solutions Germany GmbH
>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>> (HRB 36809, AG Nürnberg)
>>>> Geschäftsführer: Felix Imendörffer
>>>
>>>
>>>
>>>
>>>
>>>
>>
>> --
>> Thomas Zimmermann
>> Graphics Driver Developer
>> SUSE Software Solutions Germany GmbH
>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>> (HRB 36809, AG Nürnberg)
>> Geschäftsführer: Felix Imendörffer
> 
> 
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 8003 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 8582 bytes --]

Hi

Am 26.11.20 um 12:04 schrieb Daniel Vetter:
> On Thu, Nov 26, 2020 at 11:15 AM Thomas Zimmermann <tzimmermann@suse.de> wrote:
>>
>> Hi
>>
>> Am 25.11.20 um 17:32 schrieb Daniel Vetter:
>>> [...]
>>> I guess full locking is required :-/ I'm not exactly sure how to make this
>>> happen with the current plethora of helpers ... I think we need an
>>> _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.
>>
>> I think we might be able to get away without new callbacks.
>>
>> I looked through the sources that implement and use vmap. All the
>> implementations are without taking resv locks. For locking, we can wrap
>> them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked()
>> that locks and vmaps should make this easy.
>>
>> In terms of implementation, only vram helpers do a pin+map in their vmap
>> code. And as I mentioned before, this is actually wrong. The pattern
>> dates back to when the code was still in individual drivers. It's time
>> to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.
>>
>> Finally, there aren't that many users of vmap. A few drivers use it
>> while blitting framebuffers into HW buffers and ast does some permanent
>> mapping of the cursor BO. All this is trivial to turn into small pairs
>> of lock+vmap and vunmap+unlock.
>>
>> That leaves generic fbdev. The shadow buffer is also trivial to fix, as
>> outlined during this discussion.
>>
>> The code for fbdev in hardware buffers is a special case. It vmaps the
>> buffer during initialization and only vunmaps it during shutdown. As
>> this has worked so far without locking, I'd leave it as it is and put a
>> big comment next to is.
>>
>> Hardware fbdev buffers is only required by few drivers; namely those
>> that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work.
>> We should consider to make the fbdev shadow buffer the default and have
>> drivers opt-in for the hardware buffer, if they need it.
>>
>>>
>>> And then document that if the callers of the _locked version wants a
>>> permanent mapping, it also needs to pin it. Plus I guess ideally implement
>>> the unlocked/permanent versions in terms of that, so that drivers only
>>> have to implement one or the other.
>>
>> For my understanding, pinning is only done in prepare_fb code. And ast
>> pins its cursor BOs into vram. We should document to hols vmap/vunmap
>> only for time and cover them with resv locks. Pinning is for cases where
>> the hardware requires buffers in a special location, but does not
>> protect against concurrent threat. I think those are the implicit rules
>> already.
>>
>> I updated the radeon patchset, where all this appears to be working well.
> 
> Hm yeah if you want to do the full change, then that works out too.
> It's just a pile of work.
> 
> But if we can finish off with an dma_resv_assert_locked in
> dma_buf_vmap/vunmap, then I think that's ok. It does mean that
> exporters must implement vmap caching, or performance will be
> terrible. So quite some update for the dma-buf docs.

Yeah, I remember a bug report about frequent page-table modifications 
wrt to vram helpers. So we implemented the lazy unmapping / vmap 
caching, as suggested by Christian back them. My guess is that anything 
TTM-based can use a similar pattern. Christian probably knows the corner 
cases.

CMA seems obviously working correctly already.

For SHMEM, I'd have to figure out the reference counting of the pages 
involved. My guess is that the vunmap in drm_gem_shmem_vunmap() could be 
moved into the free callback, plus a few other modifications.

Best regards
Thomas

> 
> But if you're willing to do all that conversion of callers, then of
> course I'm not stopping you. Not at all, it's great to see that kind
> of maze untangled.
> -Daniel
> 
>>
>> Best regards
>> Thomas
>>
>>>
>>> That should give us at least some way forward to gradually conver all the
>>> drivers and helpers over to dma_resv locking.
>>> -Daniel
>>>
>>>> The pin count is currently maintained by the vmap implementation in vram
>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an implicit
>>>> unpin. This prevents eviction in the damage worker. But now I was told than
>>>> pinning is only for BOs that are controlled by userspace and internal users
>>>> should acquire the resv lock. So vram helpers have to be fixed, actually.
>>>>
>>>> In vram helpers, unmapping does not mean eviction. The unmap operation only
>>>> marks the BO as unmappable. The real unmap happens when the eviction takes
>>>> place. This avoids many modifications to the page tables. IOW an unpinned,
>>>> unmapped BO will remain in VRAM until the memory is actually needed.
>>>>
>>>> Best regards
>>>> Thomas
>>>>
>>>>>
>>>>> So I'm still not seeing how this can go boom.
>>>>>
>>>>> Now long term it'd be nice to cut everything over to dma_resv locking, but
>>>>> the issue there is that beyond ttm, none of the helpers (and few of the
>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>>>> interim fix seems like the right solution to me.
>>>>> -Daniel
>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>>>
>>>>>>>>
>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>> lock could be
>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>> separate pair of
>>>>>>>>> DRM client functions could do the locking.
>>>>>>>>
>>>>>>>> Given how this "do the right locking" is a can of worms (and I think
>>>>>>>> it's
>>>>>>>> worse than what you dug out already) I think the fb_helper.lock hack is
>>>>>>>> perfectly good enough.
>>>>>>>>
>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock in generic
>>>>>>>> code, while many helpers/drivers still have their hand-rolled locking,
>>>>>>>> will make conversion over to dma_resv needlessly more complicated.
>>>>>>>> -Daniel
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>> Thomas
>>>>>>>>>
>>>>>>>>> [1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please note that the reservation lock you need to take here is part of
>>>>>>>>>> the GEM object.
>>>>>>>>>>
>>>>>>>>>> Usually we design things in the way that the code needs to take a lock
>>>>>>>>>> which protects an object, then do some operations with the object and
>>>>>>>>>> then release the lock again.
>>>>>>>>>>
>>>>>>>>>> Having in the lock inside the operation can be done as well, but
>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>
>>>>>>>>>>> Sorry for the noob questions. I'm still trying to understand the
>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>>
>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>> talking about
>>>>>>>>>> here. We need to take that for a couple of different operations,
>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Christian.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Best regards
>>>>>>>>>>> Thomas
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> dri-devel mailing list
>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thomas Zimmermann
>>>>>>>>> Graphics Driver Developer
>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Thomas Zimmermann
>>>> Graphics Driver Developer
>>>> SUSE Software Solutions Germany GmbH
>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>> (HRB 36809, AG Nürnberg)
>>>> Geschäftsführer: Felix Imendörffer
>>>
>>>
>>>
>>>
>>>
>>>
>>
>> --
>> Thomas Zimmermann
>> Graphics Driver Developer
>> SUSE Software Solutions Germany GmbH
>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>> (HRB 36809, AG Nürnberg)
>> Geschäftsführer: Felix Imendörffer
> 
> 
> 

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 8003 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 26.11.20 um 12:59 schrieb Thomas Zimmermann:
> Hi
>
> Am 26.11.20 um 12:04 schrieb Daniel Vetter:
>> On Thu, Nov 26, 2020 at 11:15 AM Thomas Zimmermann 
>> <tzimmermann@suse.de> wrote:
>>>
>>> Hi
>>>
>>> Am 25.11.20 um 17:32 schrieb Daniel Vetter:
>>>> [...]
>>>> I guess full locking is required :-/ I'm not exactly sure how to 
>>>> make this
>>>> happen with the current plethora of helpers ... I think we need an
>>>> _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.
>>>
>>> I think we might be able to get away without new callbacks.
>>>
>>> I looked through the sources that implement and use vmap. All the
>>> implementations are without taking resv locks. For locking, we can wrap
>>> them in lock/unlock pairs. Having something like 
>>> drm_gem_vmap_unlocked()
>>> that locks and vmaps should make this easy.
>>>
>>> In terms of implementation, only vram helpers do a pin+map in their 
>>> vmap
>>> code. And as I mentioned before, this is actually wrong. The pattern
>>> dates back to when the code was still in individual drivers. It's time
>>> to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.
>>>
>>> Finally, there aren't that many users of vmap. A few drivers use it
>>> while blitting framebuffers into HW buffers and ast does some permanent
>>> mapping of the cursor BO. All this is trivial to turn into small pairs
>>> of lock+vmap and vunmap+unlock.
>>>
>>> That leaves generic fbdev. The shadow buffer is also trivial to fix, as
>>> outlined during this discussion.
>>>
>>> The code for fbdev in hardware buffers is a special case. It vmaps the
>>> buffer during initialization and only vunmaps it during shutdown. As
>>> this has worked so far without locking, I'd leave it as it is and put a
>>> big comment next to is.
>>>
>>> Hardware fbdev buffers is only required by few drivers; namely those
>>> that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work.
>>> We should consider to make the fbdev shadow buffer the default and have
>>> drivers opt-in for the hardware buffer, if they need it.
>>>
>>>>
>>>> And then document that if the callers of the _locked version wants a
>>>> permanent mapping, it also needs to pin it. Plus I guess ideally 
>>>> implement
>>>> the unlocked/permanent versions in terms of that, so that drivers only
>>>> have to implement one or the other.
>>>
>>> For my understanding, pinning is only done in prepare_fb code. And ast
>>> pins its cursor BOs into vram. We should document to hols vmap/vunmap
>>> only for time and cover them with resv locks. Pinning is for cases 
>>> where
>>> the hardware requires buffers in a special location, but does not
>>> protect against concurrent threat. I think those are the implicit rules
>>> already.
>>>
>>> I updated the radeon patchset, where all this appears to be working 
>>> well.
>>
>> Hm yeah if you want to do the full change, then that works out too.
>> It's just a pile of work.
>>
>> But if we can finish off with an dma_resv_assert_locked in
>> dma_buf_vmap/vunmap, then I think that's ok. It does mean that
>> exporters must implement vmap caching, or performance will be
>> terrible. So quite some update for the dma-buf docs.
>
> Yeah, I remember a bug report about frequent page-table modifications 
> wrt to vram helpers. So we implemented the lazy unmapping / vmap 
> caching, as suggested by Christian back them. My guess is that 
> anything TTM-based can use a similar pattern. Christian probably knows 
> the corner cases.

My memory is failing me, what corner case are you talking about?

Christian.

>
> CMA seems obviously working correctly already.
>
> For SHMEM, I'd have to figure out the reference counting of the pages 
> involved. My guess is that the vunmap in drm_gem_shmem_vunmap() could 
> be moved into the free callback, plus a few other modifications.
>
> Best regards
> Thomas
>
>>
>> But if you're willing to do all that conversion of callers, then of
>> course I'm not stopping you. Not at all, it's great to see that kind
>> of maze untangled.
>> -Daniel
>>
>>>
>>> Best regards
>>> Thomas
>>>
>>>>
>>>> That should give us at least some way forward to gradually conver 
>>>> all the
>>>> drivers and helpers over to dma_resv locking.
>>>> -Daniel
>>>>
>>>>> The pin count is currently maintained by the vmap implementation 
>>>>> in vram
>>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an 
>>>>> implicit
>>>>> unpin. This prevents eviction in the damage worker. But now I was 
>>>>> told than
>>>>> pinning is only for BOs that are controlled by userspace and 
>>>>> internal users
>>>>> should acquire the resv lock. So vram helpers have to be fixed, 
>>>>> actually.
>>>>>
>>>>> In vram helpers, unmapping does not mean eviction. The unmap 
>>>>> operation only
>>>>> marks the BO as unmappable. The real unmap happens when the 
>>>>> eviction takes
>>>>> place. This avoids many modifications to the page tables. IOW an 
>>>>> unpinned,
>>>>> unmapped BO will remain in VRAM until the memory is actually needed.
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>>>
>>>>>> So I'm still not seeing how this can go boom.
>>>>>>
>>>>>> Now long term it'd be nice to cut everything over to dma_resv 
>>>>>> locking, but
>>>>>> the issue there is that beyond ttm, none of the helpers (and few 
>>>>>> of the
>>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>>>>> interim fix seems like the right solution to me.
>>>>>> -Daniel
>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Christian.
>>>>>>>
>>>>>>>>
>>>>>>>> Best regards
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>>> lock could be
>>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>>> separate pair of
>>>>>>>>>> DRM client functions could do the locking.
>>>>>>>>>
>>>>>>>>> Given how this "do the right locking" is a can of worms (and I 
>>>>>>>>> think
>>>>>>>>> it's
>>>>>>>>> worse than what you dug out already) I think the 
>>>>>>>>> fb_helper.lock hack is
>>>>>>>>> perfectly good enough.
>>>>>>>>>
>>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock 
>>>>>>>>> in generic
>>>>>>>>> code, while many helpers/drivers still have their hand-rolled 
>>>>>>>>> locking,
>>>>>>>>> will make conversion over to dma_resv needlessly more 
>>>>>>>>> complicated.
>>>>>>>>> -Daniel
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Best regards
>>>>>>>>>> Thomas
>>>>>>>>>>
>>>>>>>>>> [1] 
>>>>>>>>>> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Please note that the reservation lock you need to take here 
>>>>>>>>>>> is part of
>>>>>>>>>>> the GEM object.
>>>>>>>>>>>
>>>>>>>>>>> Usually we design things in the way that the code needs to 
>>>>>>>>>>> take a lock
>>>>>>>>>>> which protects an object, then do some operations with the 
>>>>>>>>>>> object and
>>>>>>>>>>> then release the lock again.
>>>>>>>>>>>
>>>>>>>>>>> Having in the lock inside the operation can be done as well, 
>>>>>>>>>>> but
>>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>>
>>>>>>>>>>>> Sorry for the noob questions. I'm still trying to 
>>>>>>>>>>>> understand the
>>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>>>
>>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>>> talking about
>>>>>>>>>>> here. We need to take that for a couple of different 
>>>>>>>>>>> operations,
>>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Christian.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards
>>>>>>>>>>>> Thomas
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> dri-devel mailing list
>>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> Thomas Zimmermann
>>>>>>>>>> Graphics Driver Developer
>>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> -- 
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH
>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>> (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Felix Imendörffer
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> -- 
>>> Thomas Zimmermann
>>> Graphics Driver Developer
>>> SUSE Software Solutions Germany GmbH
>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>> (HRB 36809, AG Nürnberg)
>>> Geschäftsführer: Felix Imendörffer
>>
>>
>>
>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 26.11.20 um 12:59 schrieb Thomas Zimmermann:
> Hi
>
> Am 26.11.20 um 12:04 schrieb Daniel Vetter:
>> On Thu, Nov 26, 2020 at 11:15 AM Thomas Zimmermann 
>> <tzimmermann@suse.de> wrote:
>>>
>>> Hi
>>>
>>> Am 25.11.20 um 17:32 schrieb Daniel Vetter:
>>>> [...]
>>>> I guess full locking is required :-/ I'm not exactly sure how to 
>>>> make this
>>>> happen with the current plethora of helpers ... I think we need an
>>>> _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.
>>>
>>> I think we might be able to get away without new callbacks.
>>>
>>> I looked through the sources that implement and use vmap. All the
>>> implementations are without taking resv locks. For locking, we can wrap
>>> them in lock/unlock pairs. Having something like 
>>> drm_gem_vmap_unlocked()
>>> that locks and vmaps should make this easy.
>>>
>>> In terms of implementation, only vram helpers do a pin+map in their 
>>> vmap
>>> code. And as I mentioned before, this is actually wrong. The pattern
>>> dates back to when the code was still in individual drivers. It's time
>>> to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.
>>>
>>> Finally, there aren't that many users of vmap. A few drivers use it
>>> while blitting framebuffers into HW buffers and ast does some permanent
>>> mapping of the cursor BO. All this is trivial to turn into small pairs
>>> of lock+vmap and vunmap+unlock.
>>>
>>> That leaves generic fbdev. The shadow buffer is also trivial to fix, as
>>> outlined during this discussion.
>>>
>>> The code for fbdev in hardware buffers is a special case. It vmaps the
>>> buffer during initialization and only vunmaps it during shutdown. As
>>> this has worked so far without locking, I'd leave it as it is and put a
>>> big comment next to is.
>>>
>>> Hardware fbdev buffers is only required by few drivers; namely those
>>> that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work.
>>> We should consider to make the fbdev shadow buffer the default and have
>>> drivers opt-in for the hardware buffer, if they need it.
>>>
>>>>
>>>> And then document that if the callers of the _locked version wants a
>>>> permanent mapping, it also needs to pin it. Plus I guess ideally 
>>>> implement
>>>> the unlocked/permanent versions in terms of that, so that drivers only
>>>> have to implement one or the other.
>>>
>>> For my understanding, pinning is only done in prepare_fb code. And ast
>>> pins its cursor BOs into vram. We should document to hols vmap/vunmap
>>> only for time and cover them with resv locks. Pinning is for cases 
>>> where
>>> the hardware requires buffers in a special location, but does not
>>> protect against concurrent threat. I think those are the implicit rules
>>> already.
>>>
>>> I updated the radeon patchset, where all this appears to be working 
>>> well.
>>
>> Hm yeah if you want to do the full change, then that works out too.
>> It's just a pile of work.
>>
>> But if we can finish off with an dma_resv_assert_locked in
>> dma_buf_vmap/vunmap, then I think that's ok. It does mean that
>> exporters must implement vmap caching, or performance will be
>> terrible. So quite some update for the dma-buf docs.
>
> Yeah, I remember a bug report about frequent page-table modifications 
> wrt to vram helpers. So we implemented the lazy unmapping / vmap 
> caching, as suggested by Christian back them. My guess is that 
> anything TTM-based can use a similar pattern. Christian probably knows 
> the corner cases.

My memory is failing me, what corner case are you talking about?

Christian.

>
> CMA seems obviously working correctly already.
>
> For SHMEM, I'd have to figure out the reference counting of the pages 
> involved. My guess is that the vunmap in drm_gem_shmem_vunmap() could 
> be moved into the free callback, plus a few other modifications.
>
> Best regards
> Thomas
>
>>
>> But if you're willing to do all that conversion of callers, then of
>> course I'm not stopping you. Not at all, it's great to see that kind
>> of maze untangled.
>> -Daniel
>>
>>>
>>> Best regards
>>> Thomas
>>>
>>>>
>>>> That should give us at least some way forward to gradually conver 
>>>> all the
>>>> drivers and helpers over to dma_resv locking.
>>>> -Daniel
>>>>
>>>>> The pin count is currently maintained by the vmap implementation 
>>>>> in vram
>>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an 
>>>>> implicit
>>>>> unpin. This prevents eviction in the damage worker. But now I was 
>>>>> told than
>>>>> pinning is only for BOs that are controlled by userspace and 
>>>>> internal users
>>>>> should acquire the resv lock. So vram helpers have to be fixed, 
>>>>> actually.
>>>>>
>>>>> In vram helpers, unmapping does not mean eviction. The unmap 
>>>>> operation only
>>>>> marks the BO as unmappable. The real unmap happens when the 
>>>>> eviction takes
>>>>> place. This avoids many modifications to the page tables. IOW an 
>>>>> unpinned,
>>>>> unmapped BO will remain in VRAM until the memory is actually needed.
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>>>
>>>>>> So I'm still not seeing how this can go boom.
>>>>>>
>>>>>> Now long term it'd be nice to cut everything over to dma_resv 
>>>>>> locking, but
>>>>>> the issue there is that beyond ttm, none of the helpers (and few 
>>>>>> of the
>>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>>>>> interim fix seems like the right solution to me.
>>>>>> -Daniel
>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Christian.
>>>>>>>
>>>>>>>>
>>>>>>>> Best regards
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>>> lock could be
>>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>>> separate pair of
>>>>>>>>>> DRM client functions could do the locking.
>>>>>>>>>
>>>>>>>>> Given how this "do the right locking" is a can of worms (and I 
>>>>>>>>> think
>>>>>>>>> it's
>>>>>>>>> worse than what you dug out already) I think the 
>>>>>>>>> fb_helper.lock hack is
>>>>>>>>> perfectly good enough.
>>>>>>>>>
>>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock 
>>>>>>>>> in generic
>>>>>>>>> code, while many helpers/drivers still have their hand-rolled 
>>>>>>>>> locking,
>>>>>>>>> will make conversion over to dma_resv needlessly more 
>>>>>>>>> complicated.
>>>>>>>>> -Daniel
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Best regards
>>>>>>>>>> Thomas
>>>>>>>>>>
>>>>>>>>>> [1] 
>>>>>>>>>> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Please note that the reservation lock you need to take here 
>>>>>>>>>>> is part of
>>>>>>>>>>> the GEM object.
>>>>>>>>>>>
>>>>>>>>>>> Usually we design things in the way that the code needs to 
>>>>>>>>>>> take a lock
>>>>>>>>>>> which protects an object, then do some operations with the 
>>>>>>>>>>> object and
>>>>>>>>>>> then release the lock again.
>>>>>>>>>>>
>>>>>>>>>>> Having in the lock inside the operation can be done as well, 
>>>>>>>>>>> but
>>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>>
>>>>>>>>>>>> Sorry for the noob questions. I'm still trying to 
>>>>>>>>>>>> understand the
>>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>>>
>>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>>> talking about
>>>>>>>>>>> here. We need to take that for a couple of different 
>>>>>>>>>>> operations,
>>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Christian.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards
>>>>>>>>>>>> Thomas
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> dri-devel mailing list
>>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>>>>>>
>>>>>>>>>> -- 
>>>>>>>>>> Thomas Zimmermann
>>>>>>>>>> Graphics Driver Developer
>>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> -- 
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH
>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>> (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Felix Imendörffer
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> -- 
>>> Thomas Zimmermann
>>> Graphics Driver Developer
>>> SUSE Software Solutions Germany GmbH
>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>> (HRB 36809, AG Nürnberg)
>>> Geschäftsführer: Felix Imendörffer
>>
>>
>>
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 6571 bytes --]

Hi

Am 26.11.20 um 13:08 schrieb Christian König:
>> [...]
>> Yeah, I remember a bug report about frequent page-table modifications 
>> wrt to vram helpers. So we implemented the lazy unmapping / vmap 
>> caching, as suggested by Christian back them. My guess is that 
>> anything TTM-based can use a similar pattern. Christian probably knows 
>> the corner cases.
> 
> My memory is failing me, what corner case are you talking about?

Haha! :D What I meant was that if there were corner cases, you'd know 
about them. From your comment, I guess there are none.

Best regards
Thomas

> 
> Christian.
> 
>>
>> CMA seems obviously working correctly already.
>>
>> For SHMEM, I'd have to figure out the reference counting of the pages 
>> involved. My guess is that the vunmap in drm_gem_shmem_vunmap() could 
>> be moved into the free callback, plus a few other modifications.
>>
>> Best regards
>> Thomas
>>
>>>
>>> But if you're willing to do all that conversion of callers, then of
>>> course I'm not stopping you. Not at all, it's great to see that kind
>>> of maze untangled.
>>> -Daniel
>>>
>>>>
>>>> Best regards
>>>> Thomas
>>>>
>>>>>
>>>>> That should give us at least some way forward to gradually conver 
>>>>> all the
>>>>> drivers and helpers over to dma_resv locking.
>>>>> -Daniel
>>>>>
>>>>>> The pin count is currently maintained by the vmap implementation 
>>>>>> in vram
>>>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an 
>>>>>> implicit
>>>>>> unpin. This prevents eviction in the damage worker. But now I was 
>>>>>> told than
>>>>>> pinning is only for BOs that are controlled by userspace and 
>>>>>> internal users
>>>>>> should acquire the resv lock. So vram helpers have to be fixed, 
>>>>>> actually.
>>>>>>
>>>>>> In vram helpers, unmapping does not mean eviction. The unmap 
>>>>>> operation only
>>>>>> marks the BO as unmappable. The real unmap happens when the 
>>>>>> eviction takes
>>>>>> place. This avoids many modifications to the page tables. IOW an 
>>>>>> unpinned,
>>>>>> unmapped BO will remain in VRAM until the memory is actually needed.
>>>>>>
>>>>>> Best regards
>>>>>> Thomas
>>>>>>
>>>>>>>
>>>>>>> So I'm still not seeing how this can go boom.
>>>>>>>
>>>>>>> Now long term it'd be nice to cut everything over to dma_resv 
>>>>>>> locking, but
>>>>>>> the issue there is that beyond ttm, none of the helpers (and few 
>>>>>>> of the
>>>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>>>>>> interim fix seems like the right solution to me.
>>>>>>> -Daniel
>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Christian.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>> Thomas
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>>>> lock could be
>>>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>>>> separate pair of
>>>>>>>>>>> DRM client functions could do the locking.
>>>>>>>>>>
>>>>>>>>>> Given how this "do the right locking" is a can of worms (and I 
>>>>>>>>>> think
>>>>>>>>>> it's
>>>>>>>>>> worse than what you dug out already) I think the 
>>>>>>>>>> fb_helper.lock hack is
>>>>>>>>>> perfectly good enough.
>>>>>>>>>>
>>>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock 
>>>>>>>>>> in generic
>>>>>>>>>> code, while many helpers/drivers still have their hand-rolled 
>>>>>>>>>> locking,
>>>>>>>>>> will make conversion over to dma_resv needlessly more 
>>>>>>>>>> complicated.
>>>>>>>>>> -Daniel
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Best regards
>>>>>>>>>>> Thomas
>>>>>>>>>>>
>>>>>>>>>>> [1] 
>>>>>>>>>>> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394 
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Please note that the reservation lock you need to take here 
>>>>>>>>>>>> is part of
>>>>>>>>>>>> the GEM object.
>>>>>>>>>>>>
>>>>>>>>>>>> Usually we design things in the way that the code needs to 
>>>>>>>>>>>> take a lock
>>>>>>>>>>>> which protects an object, then do some operations with the 
>>>>>>>>>>>> object and
>>>>>>>>>>>> then release the lock again.
>>>>>>>>>>>>
>>>>>>>>>>>> Having in the lock inside the operation can be done as well, 
>>>>>>>>>>>> but
>>>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>>>
>>>>>>>>>>>>> Sorry for the noob questions. I'm still trying to 
>>>>>>>>>>>>> understand the
>>>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>>>>
>>>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>>>> talking about
>>>>>>>>>>>> here. We need to take that for a couple of different 
>>>>>>>>>>>> operations,
>>>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Christian.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>> Thomas
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> dri-devel mailing list
>>>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>>>>>>>
>>>>>>>>>>> -- 
>>>>>>>>>>> Thomas Zimmermann
>>>>>>>>>>> Graphics Driver Developer
>>>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> -- 
>>>>>> Thomas Zimmermann
>>>>>> Graphics Driver Developer
>>>>>> SUSE Software Solutions Germany GmbH
>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>> (HRB 36809, AG Nürnberg)
>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> -- 
>>>> Thomas Zimmermann
>>>> Graphics Driver Developer
>>>> SUSE Software Solutions Germany GmbH
>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>> (HRB 36809, AG Nürnberg)
>>>> Geschäftsführer: Felix Imendörffer
>>>
>>>
>>>
>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 8003 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Thomas Zimmermann]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 6571 bytes --]

Hi

Am 26.11.20 um 13:08 schrieb Christian König:
>> [...]
>> Yeah, I remember a bug report about frequent page-table modifications 
>> wrt to vram helpers. So we implemented the lazy unmapping / vmap 
>> caching, as suggested by Christian back them. My guess is that 
>> anything TTM-based can use a similar pattern. Christian probably knows 
>> the corner cases.
> 
> My memory is failing me, what corner case are you talking about?

Haha! :D What I meant was that if there were corner cases, you'd know 
about them. From your comment, I guess there are none.

Best regards
Thomas

> 
> Christian.
> 
>>
>> CMA seems obviously working correctly already.
>>
>> For SHMEM, I'd have to figure out the reference counting of the pages 
>> involved. My guess is that the vunmap in drm_gem_shmem_vunmap() could 
>> be moved into the free callback, plus a few other modifications.
>>
>> Best regards
>> Thomas
>>
>>>
>>> But if you're willing to do all that conversion of callers, then of
>>> course I'm not stopping you. Not at all, it's great to see that kind
>>> of maze untangled.
>>> -Daniel
>>>
>>>>
>>>> Best regards
>>>> Thomas
>>>>
>>>>>
>>>>> That should give us at least some way forward to gradually conver 
>>>>> all the
>>>>> drivers and helpers over to dma_resv locking.
>>>>> -Daniel
>>>>>
>>>>>> The pin count is currently maintained by the vmap implementation 
>>>>>> in vram
>>>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an 
>>>>>> implicit
>>>>>> unpin. This prevents eviction in the damage worker. But now I was 
>>>>>> told than
>>>>>> pinning is only for BOs that are controlled by userspace and 
>>>>>> internal users
>>>>>> should acquire the resv lock. So vram helpers have to be fixed, 
>>>>>> actually.
>>>>>>
>>>>>> In vram helpers, unmapping does not mean eviction. The unmap 
>>>>>> operation only
>>>>>> marks the BO as unmappable. The real unmap happens when the 
>>>>>> eviction takes
>>>>>> place. This avoids many modifications to the page tables. IOW an 
>>>>>> unpinned,
>>>>>> unmapped BO will remain in VRAM until the memory is actually needed.
>>>>>>
>>>>>> Best regards
>>>>>> Thomas
>>>>>>
>>>>>>>
>>>>>>> So I'm still not seeing how this can go boom.
>>>>>>>
>>>>>>> Now long term it'd be nice to cut everything over to dma_resv 
>>>>>>> locking, but
>>>>>>> the issue there is that beyond ttm, none of the helpers (and few 
>>>>>>> of the
>>>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. Quick
>>>>>>> interim fix seems like the right solution to me.
>>>>>>> -Daniel
>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Christian.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>> Thomas
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>>>> lock could be
>>>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>>>> separate pair of
>>>>>>>>>>> DRM client functions could do the locking.
>>>>>>>>>>
>>>>>>>>>> Given how this "do the right locking" is a can of worms (and I 
>>>>>>>>>> think
>>>>>>>>>> it's
>>>>>>>>>> worse than what you dug out already) I think the 
>>>>>>>>>> fb_helper.lock hack is
>>>>>>>>>> perfectly good enough.
>>>>>>>>>>
>>>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock 
>>>>>>>>>> in generic
>>>>>>>>>> code, while many helpers/drivers still have their hand-rolled 
>>>>>>>>>> locking,
>>>>>>>>>> will make conversion over to dma_resv needlessly more 
>>>>>>>>>> complicated.
>>>>>>>>>> -Daniel
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Best regards
>>>>>>>>>>> Thomas
>>>>>>>>>>>
>>>>>>>>>>> [1] 
>>>>>>>>>>> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394 
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Please note that the reservation lock you need to take here 
>>>>>>>>>>>> is part of
>>>>>>>>>>>> the GEM object.
>>>>>>>>>>>>
>>>>>>>>>>>> Usually we design things in the way that the code needs to 
>>>>>>>>>>>> take a lock
>>>>>>>>>>>> which protects an object, then do some operations with the 
>>>>>>>>>>>> object and
>>>>>>>>>>>> then release the lock again.
>>>>>>>>>>>>
>>>>>>>>>>>> Having in the lock inside the operation can be done as well, 
>>>>>>>>>>>> but
>>>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>>>
>>>>>>>>>>>>> Sorry for the noob questions. I'm still trying to 
>>>>>>>>>>>>> understand the
>>>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>>>>
>>>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>>>> talking about
>>>>>>>>>>>> here. We need to take that for a couple of different 
>>>>>>>>>>>> operations,
>>>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Christian.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>> Thomas
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> dri-devel mailing list
>>>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>>>>>>>
>>>>>>>>>>> -- 
>>>>>>>>>>> Thomas Zimmermann
>>>>>>>>>>> Graphics Driver Developer
>>>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> -- 
>>>>>> Thomas Zimmermann
>>>>>> Graphics Driver Developer
>>>>>> SUSE Software Solutions Germany GmbH
>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>> (HRB 36809, AG Nürnberg)
>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> -- 
>>>> Thomas Zimmermann
>>>> Graphics Driver Developer
>>>> SUSE Software Solutions Germany GmbH
>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>> (HRB 36809, AG Nürnberg)
>>>> Geschäftsführer: Felix Imendörffer
>>>
>>>
>>>
>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

[-- Attachment #1.1.1.2: OpenPGP_0x680DC11D530B7A23.asc --]
[-- Type: application/pgp-keys, Size: 8003 bytes --]

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[-- Attachment #2: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 26.11.20 um 13:14 schrieb Thomas Zimmermann:
> Hi
>
> Am 26.11.20 um 13:08 schrieb Christian König:
>>> [...]
>>> Yeah, I remember a bug report about frequent page-table 
>>> modifications wrt to vram helpers. So we implemented the lazy 
>>> unmapping / vmap caching, as suggested by Christian back them. My 
>>> guess is that anything TTM-based can use a similar pattern. 
>>> Christian probably knows the corner cases.
>>
>> My memory is failing me, what corner case are you talking about?
>
> Haha! :D What I meant was that if there were corner cases, you'd know 
> about them. From your comment, I guess there are none.

Ah, ok :) I was wondering for a moment if I have missed something.

Cheers,
Christian.

>
> Best regards
> Thomas
>
>>
>> Christian.
>>
>>>
>>> CMA seems obviously working correctly already.
>>>
>>> For SHMEM, I'd have to figure out the reference counting of the 
>>> pages involved. My guess is that the vunmap in 
>>> drm_gem_shmem_vunmap() could be moved into the free callback, plus a 
>>> few other modifications.
>>>
>>> Best regards
>>> Thomas
>>>
>>>>
>>>> But if you're willing to do all that conversion of callers, then of
>>>> course I'm not stopping you. Not at all, it's great to see that kind
>>>> of maze untangled.
>>>> -Daniel
>>>>
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>>>
>>>>>> That should give us at least some way forward to gradually conver 
>>>>>> all the
>>>>>> drivers and helpers over to dma_resv locking.
>>>>>> -Daniel
>>>>>>
>>>>>>> The pin count is currently maintained by the vmap implementation 
>>>>>>> in vram
>>>>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an 
>>>>>>> implicit
>>>>>>> unpin. This prevents eviction in the damage worker. But now I 
>>>>>>> was told than
>>>>>>> pinning is only for BOs that are controlled by userspace and 
>>>>>>> internal users
>>>>>>> should acquire the resv lock. So vram helpers have to be fixed, 
>>>>>>> actually.
>>>>>>>
>>>>>>> In vram helpers, unmapping does not mean eviction. The unmap 
>>>>>>> operation only
>>>>>>> marks the BO as unmappable. The real unmap happens when the 
>>>>>>> eviction takes
>>>>>>> place. This avoids many modifications to the page tables. IOW an 
>>>>>>> unpinned,
>>>>>>> unmapped BO will remain in VRAM until the memory is actually 
>>>>>>> needed.
>>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>>>
>>>>>>>>
>>>>>>>> So I'm still not seeing how this can go boom.
>>>>>>>>
>>>>>>>> Now long term it'd be nice to cut everything over to dma_resv 
>>>>>>>> locking, but
>>>>>>>> the issue there is that beyond ttm, none of the helpers (and 
>>>>>>>> few of the
>>>>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. 
>>>>>>>> Quick
>>>>>>>> interim fix seems like the right solution to me.
>>>>>>>> -Daniel
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Christian.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Best regards
>>>>>>>>>> Thomas
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>>>>> lock could be
>>>>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>>>>> separate pair of
>>>>>>>>>>>> DRM client functions could do the locking.
>>>>>>>>>>>
>>>>>>>>>>> Given how this "do the right locking" is a can of worms (and 
>>>>>>>>>>> I think
>>>>>>>>>>> it's
>>>>>>>>>>> worse than what you dug out already) I think the 
>>>>>>>>>>> fb_helper.lock hack is
>>>>>>>>>>> perfectly good enough.
>>>>>>>>>>>
>>>>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock 
>>>>>>>>>>> in generic
>>>>>>>>>>> code, while many helpers/drivers still have their 
>>>>>>>>>>> hand-rolled locking,
>>>>>>>>>>> will make conversion over to dma_resv needlessly more 
>>>>>>>>>>> complicated.
>>>>>>>>>>> -Daniel
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards
>>>>>>>>>>>> Thomas
>>>>>>>>>>>>
>>>>>>>>>>>> [1] 
>>>>>>>>>>>> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394 
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please note that the reservation lock you need to take 
>>>>>>>>>>>>> here is part of
>>>>>>>>>>>>> the GEM object.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Usually we design things in the way that the code needs to 
>>>>>>>>>>>>> take a lock
>>>>>>>>>>>>> which protects an object, then do some operations with the 
>>>>>>>>>>>>> object and
>>>>>>>>>>>>> then release the lock again.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Having in the lock inside the operation can be done as 
>>>>>>>>>>>>> well, but
>>>>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry for the noob questions. I'm still trying to 
>>>>>>>>>>>>>> understand the
>>>>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>>>>> talking about
>>>>>>>>>>>>> here. We need to take that for a couple of different 
>>>>>>>>>>>>> operations,
>>>>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Christian.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>>> Thomas
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> dri-devel mailing list
>>>>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>>>>>>>>
>>>>>>>>>>>> -- 
>>>>>>>>>>>> Thomas Zimmermann
>>>>>>>>>>>> Graphics Driver Developer
>>>>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>>>> Thomas Zimmermann
>>>>>>> Graphics Driver Developer
>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> -- 
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH
>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>> (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Felix Imendörffer
>>>>
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

[Christian König]

Am 26.11.20 um 13:14 schrieb Thomas Zimmermann:
> Hi
>
> Am 26.11.20 um 13:08 schrieb Christian König:
>>> [...]
>>> Yeah, I remember a bug report about frequent page-table 
>>> modifications wrt to vram helpers. So we implemented the lazy 
>>> unmapping / vmap caching, as suggested by Christian back them. My 
>>> guess is that anything TTM-based can use a similar pattern. 
>>> Christian probably knows the corner cases.
>>
>> My memory is failing me, what corner case are you talking about?
>
> Haha! :D What I meant was that if there were corner cases, you'd know 
> about them. From your comment, I guess there are none.

Ah, ok :) I was wondering for a moment if I have missed something.

Cheers,
Christian.

>
> Best regards
> Thomas
>
>>
>> Christian.
>>
>>>
>>> CMA seems obviously working correctly already.
>>>
>>> For SHMEM, I'd have to figure out the reference counting of the 
>>> pages involved. My guess is that the vunmap in 
>>> drm_gem_shmem_vunmap() could be moved into the free callback, plus a 
>>> few other modifications.
>>>
>>> Best regards
>>> Thomas
>>>
>>>>
>>>> But if you're willing to do all that conversion of callers, then of
>>>> course I'm not stopping you. Not at all, it's great to see that kind
>>>> of maze untangled.
>>>> -Daniel
>>>>
>>>>>
>>>>> Best regards
>>>>> Thomas
>>>>>
>>>>>>
>>>>>> That should give us at least some way forward to gradually conver 
>>>>>> all the
>>>>>> drivers and helpers over to dma_resv locking.
>>>>>> -Daniel
>>>>>>
>>>>>>> The pin count is currently maintained by the vmap implementation 
>>>>>>> in vram
>>>>>>> helpers. Calling vmap is an implicit pin; calling vunmap is an 
>>>>>>> implicit
>>>>>>> unpin. This prevents eviction in the damage worker. But now I 
>>>>>>> was told than
>>>>>>> pinning is only for BOs that are controlled by userspace and 
>>>>>>> internal users
>>>>>>> should acquire the resv lock. So vram helpers have to be fixed, 
>>>>>>> actually.
>>>>>>>
>>>>>>> In vram helpers, unmapping does not mean eviction. The unmap 
>>>>>>> operation only
>>>>>>> marks the BO as unmappable. The real unmap happens when the 
>>>>>>> eviction takes
>>>>>>> place. This avoids many modifications to the page tables. IOW an 
>>>>>>> unpinned,
>>>>>>> unmapped BO will remain in VRAM until the memory is actually 
>>>>>>> needed.
>>>>>>>
>>>>>>> Best regards
>>>>>>> Thomas
>>>>>>>
>>>>>>>>
>>>>>>>> So I'm still not seeing how this can go boom.
>>>>>>>>
>>>>>>>> Now long term it'd be nice to cut everything over to dma_resv 
>>>>>>>> locking, but
>>>>>>>> the issue there is that beyond ttm, none of the helpers (and 
>>>>>>>> few of the
>>>>>>>> drivers) use dma_resv. So this is a fairly big uphill battle. 
>>>>>>>> Quick
>>>>>>>> interim fix seems like the right solution to me.
>>>>>>>> -Daniel
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Christian.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Best regards
>>>>>>>>>> Thomas
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> There's no recursion taking place, so I guess the reservation
>>>>>>>>>>>> lock could be
>>>>>>>>>>>> acquired/release in drm_client_buffer_vmap/vunmap(), or a
>>>>>>>>>>>> separate pair of
>>>>>>>>>>>> DRM client functions could do the locking.
>>>>>>>>>>>
>>>>>>>>>>> Given how this "do the right locking" is a can of worms (and 
>>>>>>>>>>> I think
>>>>>>>>>>> it's
>>>>>>>>>>> worse than what you dug out already) I think the 
>>>>>>>>>>> fb_helper.lock hack is
>>>>>>>>>>> perfectly good enough.
>>>>>>>>>>>
>>>>>>>>>>> I'm also somewhat worried that starting to use dma_resv lock 
>>>>>>>>>>> in generic
>>>>>>>>>>> code, while many helpers/drivers still have their 
>>>>>>>>>>> hand-rolled locking,
>>>>>>>>>>> will make conversion over to dma_resv needlessly more 
>>>>>>>>>>> complicated.
>>>>>>>>>>> -Daniel
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards
>>>>>>>>>>>> Thomas
>>>>>>>>>>>>
>>>>>>>>>>>> [1] 
>>>>>>>>>>>> https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394 
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please note that the reservation lock you need to take 
>>>>>>>>>>>>> here is part of
>>>>>>>>>>>>> the GEM object.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Usually we design things in the way that the code needs to 
>>>>>>>>>>>>> take a lock
>>>>>>>>>>>>> which protects an object, then do some operations with the 
>>>>>>>>>>>>> object and
>>>>>>>>>>>>> then release the lock again.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Having in the lock inside the operation can be done as 
>>>>>>>>>>>>> well, but
>>>>>>>>>>>>> returning with it is kind of unusual design.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry for the noob questions. I'm still trying to 
>>>>>>>>>>>>>> understand the
>>>>>>>>>>>>>> implications of acquiring these locks.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Well this is the reservation lock of the GEM object we are
>>>>>>>>>>>>> talking about
>>>>>>>>>>>>> here. We need to take that for a couple of different 
>>>>>>>>>>>>> operations,
>>>>>>>>>>>>> vmap/vunmap doesn't sound like a special case to me.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Christian.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>>> Thomas
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> dri-devel mailing list
>>>>>>>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>>>>>>>>>
>>>>>>>>>>>> -- 
>>>>>>>>>>>> Thomas Zimmermann
>>>>>>>>>>>> Graphics Driver Developer
>>>>>>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>>>> Thomas Zimmermann
>>>>>>> Graphics Driver Developer
>>>>>>> SUSE Software Solutions Germany GmbH
>>>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>>>> (HRB 36809, AG Nürnberg)
>>>>>>> Geschäftsführer: Felix Imendörffer
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> -- 
>>>>> Thomas Zimmermann
>>>>> Graphics Driver Developer
>>>>> SUSE Software Solutions Germany GmbH
>>>>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>>> (HRB 36809, AG Nürnberg)
>>>>> Geschäftsführer: Felix Imendörffer
>>>>
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx