[PATCH v10 0/2] drm/i915: Allow user to set cache at BO creation

[PATCH v10 0/2] drm/i915: Allow user to set cache at BO creation

[fei.yang]

From: Fei Yang <fei.yang@intel.com>

This series introduce a new extension for GEM_CREATE,
1. end support for set caching ioctl [PATCH 1/2]
2. add set_pat extension for gem_create [PATCH 2/2]

v2: drop one patch that was merged separately
    commit 341ad0e8e254 ("drm/i915/mtl: Add PTE encode function")
v3: rebased on https://patchwork.freedesktop.org/series/117082/
v4: fix missing unlock introduced in v3, and
    solve a rebase conflict
v5: replace obj->cache_level with pat_set_by_user,
    fix i915_cache_level_str() for legacy platforms.
v6: rebased on https://patchwork.freedesktop.org/series/117480/
v7: rebased on https://patchwork.freedesktop.org/series/117528/
v8: dropped the two dependent patches that has been merged
    separately. Add IGT link and Tested-by (MESA).
v9: addressing comments (Andi)
v10: acked-by and tested-by MESA

Fei Yang (2):
  drm/i915/mtl: end support for set caching ioctl
  drm/i915: Allow user to set cache at BO creation

 drivers/gpu/drm/i915/gem/i915_gem_create.c | 36 +++++++++++++++++++
 drivers/gpu/drm/i915/gem/i915_gem_domain.c |  3 ++
 drivers/gpu/drm/i915/gem/i915_gem_object.c |  6 ++++
 drivers/gpu/drm/i915/gem/i915_gem_shmem.c  |  9 ++++-
 include/uapi/drm/i915_drm.h                | 42 ++++++++++++++++++++++
 tools/include/uapi/drm/i915_drm.h          | 42 ++++++++++++++++++++++
 6 files changed, 137 insertions(+), 1 deletion(-)

-- 
2.25.1

[PATCH v10 1/2] drm/i915/mtl: end support for set caching ioctl

[fei.yang]

From: Fei Yang <fei.yang@intel.com>

The design is to keep Buffer Object's caching policy immutable through
out its life cycle. This patch ends the support for set caching ioctl
from MTL onward. While doing that we also set BO's to be 1-way coherent
at creation time because GPU is no longer automatically snooping CPU
cache. For userspace components needing to fine tune the caching policy
for BO's, a follow up patch will extend the GEM_CREATE uAPI to allow
them specify caching mode at BO creation time.

Signed-off-by: Fei Yang <fei.yang@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com>
---
 drivers/gpu/drm/i915/gem/i915_gem_domain.c | 3 +++
 drivers/gpu/drm/i915/gem/i915_gem_shmem.c  | 9 ++++++++-
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/gem/i915_gem_domain.c b/drivers/gpu/drm/i915/gem/i915_gem_domain.c
index 05107a6efe45..dfaaa8b66ac3 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_domain.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_domain.c
@@ -350,6 +350,9 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data,
 	if (IS_DGFX(i915))
 		return -ENODEV;
 
+	if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70))
+		return -EOPNOTSUPP;
+
 	switch (args->caching) {
 	case I915_CACHING_NONE:
 		level = I915_CACHE_NONE;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
index 37d1efcd3ca6..cad4a6017f4b 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c
@@ -601,7 +601,14 @@ static int shmem_object_init(struct intel_memory_region *mem,
 	obj->write_domain = I915_GEM_DOMAIN_CPU;
 	obj->read_domains = I915_GEM_DOMAIN_CPU;
 
-	if (HAS_LLC(i915))
+	/*
+	 * MTL doesn't snoop CPU cache by default for GPU access (namely
+	 * 1-way coherency). However some UMD's are currently depending on
+	 * that. Make 1-way coherent the default setting for MTL. A follow
+	 * up patch will extend the GEM_CREATE uAPI to allow UMD's specify
+	 * caching mode at BO creation time
+	 */
+	if (HAS_LLC(i915) || (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 70)))
 		/* On some devices, we can have the GPU use the LLC (the CPU
 		 * cache) for about a 10% performance improvement
 		 * compared to uncached.  Graphics requests other than
-- 
2.25.1

[PATCH v10 2/2] drm/i915: Allow user to set cache at BO creation

[fei.yang]

From: Fei Yang <fei.yang@intel.com>

To comply with the design that buffer objects shall have immutable
cache setting through out their life cycle, {set, get}_caching ioctl's
are no longer supported from MTL onward. With that change caching
policy can only be set at object creation time. The current code
applies a default (platform dependent) cache setting for all objects.
However this is not optimal for performance tuning. The patch extends
the existing gem_create uAPI to let user set PAT index for the object
at creation time.
The new extension is platform independent, so UMD's can switch to using
this extension for older platforms as well, while {set, get}_caching are
still supported on these legacy paltforms for compatibility reason.

Test igt@gem_create@create_ext_set_pat posted at
https://patchwork.freedesktop.org/series/117695/

Tested with https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22878

Signed-off-by: Fei Yang <fei.yang@intel.com>
Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Acked-by: Jordan Justen <jordan.l.justen@intel.com>
Tested-by: Jordan Justen <jordan.l.justen@intel.com>
---
 drivers/gpu/drm/i915/gem/i915_gem_create.c | 36 +++++++++++++++++++
 drivers/gpu/drm/i915/gem/i915_gem_object.c |  6 ++++
 include/uapi/drm/i915_drm.h                | 42 ++++++++++++++++++++++
 tools/include/uapi/drm/i915_drm.h          | 42 ++++++++++++++++++++++
 4 files changed, 126 insertions(+)

diff --git a/drivers/gpu/drm/i915/gem/i915_gem_create.c b/drivers/gpu/drm/i915/gem/i915_gem_create.c
index bfe1dbda4cb7..644a936248ad 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_create.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_create.c
@@ -245,6 +245,7 @@ struct create_ext {
 	unsigned int n_placements;
 	unsigned int placement_mask;
 	unsigned long flags;
+	unsigned int pat_index;
 };
 
 static void repr_placements(char *buf, size_t size,
@@ -394,11 +395,39 @@ static int ext_set_protected(struct i915_user_extension __user *base, void *data
 	return 0;
 }
 
+static int ext_set_pat(struct i915_user_extension __user *base, void *data)
+{
+	struct create_ext *ext_data = data;
+	struct drm_i915_private *i915 = ext_data->i915;
+	struct drm_i915_gem_create_ext_set_pat ext;
+	unsigned int max_pat_index;
+
+	BUILD_BUG_ON(sizeof(struct drm_i915_gem_create_ext_set_pat) !=
+		     offsetofend(struct drm_i915_gem_create_ext_set_pat, rsvd));
+
+	if (copy_from_user(&ext, base, sizeof(ext)))
+		return -EFAULT;
+
+	max_pat_index = INTEL_INFO(i915)->max_pat_index;
+
+	if (ext.pat_index > max_pat_index) {
+		drm_dbg(&i915->drm, "PAT index is invalid: %u\n",
+			ext.pat_index);
+		return -EINVAL;
+	}
+
+	ext_data->pat_index = ext.pat_index;
+
+	return 0;
+}
+
 static const i915_user_extension_fn create_extensions[] = {
 	[I915_GEM_CREATE_EXT_MEMORY_REGIONS] = ext_set_placements,
 	[I915_GEM_CREATE_EXT_PROTECTED_CONTENT] = ext_set_protected,
+	[I915_GEM_CREATE_EXT_SET_PAT] = ext_set_pat,
 };
 
+#define PAT_INDEX_NOT_SET	0xffff
 /**
  * i915_gem_create_ext_ioctl - Creates a new mm object and returns a handle to it.
  * @dev: drm device pointer
@@ -418,6 +447,7 @@ i915_gem_create_ext_ioctl(struct drm_device *dev, void *data,
 	if (args->flags & ~I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS)
 		return -EINVAL;
 
+	ext_data.pat_index = PAT_INDEX_NOT_SET;
 	ret = i915_user_extensions(u64_to_user_ptr(args->extensions),
 				   create_extensions,
 				   ARRAY_SIZE(create_extensions),
@@ -454,5 +484,11 @@ i915_gem_create_ext_ioctl(struct drm_device *dev, void *data,
 	if (IS_ERR(obj))
 		return PTR_ERR(obj);
 
+	if (ext_data.pat_index != PAT_INDEX_NOT_SET) {
+		i915_gem_object_set_pat_index(obj, ext_data.pat_index);
+		/* Mark pat_index is set by UMD */
+		obj->pat_set_by_user = true;
+	}
+
 	return i915_gem_publish(obj, file, &args->size, &args->handle);
 }
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c
index 46a19b099ec8..97ac6fb37958 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c
@@ -208,6 +208,12 @@ bool i915_gem_object_can_bypass_llc(struct drm_i915_gem_object *obj)
 	if (!(obj->flags & I915_BO_ALLOC_USER))
 		return false;
 
+	/*
+	 * Always flush cache for UMD objects at creation time.
+	 */
+	if (obj->pat_set_by_user)
+		return true;
+
 	/*
 	 * EHL and JSL add the 'Bypass LLC' MOCS entry, which should make it
 	 * possible for userspace to bypass the GTT caching bits set by the
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index ba40855dbc93..4f3fd650e5e1 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -3664,9 +3664,13 @@ struct drm_i915_gem_create_ext {
 	 *
 	 * For I915_GEM_CREATE_EXT_PROTECTED_CONTENT usage see
 	 * struct drm_i915_gem_create_ext_protected_content.
+	 *
+	 * For I915_GEM_CREATE_EXT_SET_PAT usage see
+	 * struct drm_i915_gem_create_ext_set_pat.
 	 */
 #define I915_GEM_CREATE_EXT_MEMORY_REGIONS 0
 #define I915_GEM_CREATE_EXT_PROTECTED_CONTENT 1
+#define I915_GEM_CREATE_EXT_SET_PAT 2
 	__u64 extensions;
 };
 
@@ -3781,6 +3785,44 @@ struct drm_i915_gem_create_ext_protected_content {
 	__u32 flags;
 };
 
+/**
+ * struct drm_i915_gem_create_ext_set_pat - The
+ * I915_GEM_CREATE_EXT_SET_PAT extension.
+ *
+ * If this extension is provided, the specified caching policy (PAT index) is
+ * applied to the buffer object.
+ *
+ * Below is an example on how to create an object with specific caching policy:
+ *
+ * .. code-block:: C
+ *
+ *      struct drm_i915_gem_create_ext_set_pat set_pat_ext = {
+ *              .base = { .name = I915_GEM_CREATE_EXT_SET_PAT },
+ *              .pat_index = 0,
+ *      };
+ *      struct drm_i915_gem_create_ext create_ext = {
+ *              .size = PAGE_SIZE,
+ *              .extensions = (uintptr_t)&set_pat_ext,
+ *      };
+ *
+ *      int err = ioctl(fd, DRM_IOCTL_I915_GEM_CREATE_EXT, &create_ext);
+ *      if (err) ...
+ */
+struct drm_i915_gem_create_ext_set_pat {
+	/** @base: Extension link. See struct i915_user_extension. */
+	struct i915_user_extension base;
+	/**
+	 * @pat_index: PAT index to be set
+	 * PAT index is a bit field in Page Table Entry to control caching
+	 * behaviors for GPU accesses. The definition of PAT index is
+	 * platform dependent and can be found in hardware specifications,
+	 * e.g. BSpec 45101.
+	 */
+	__u32 pat_index;
+	/** @rsvd: reserved for future use */
+	__u32 rsvd;
+};
+
 /* ID of the protected content session managed by i915 when PXP is active */
 #define I915_PROTECTED_CONTENT_DEFAULT_SESSION 0xf
 
diff --git a/tools/include/uapi/drm/i915_drm.h b/tools/include/uapi/drm/i915_drm.h
index 8df261c5ab9b..4fbfa472b9fc 100644
--- a/tools/include/uapi/drm/i915_drm.h
+++ b/tools/include/uapi/drm/i915_drm.h
@@ -3607,9 +3607,13 @@ struct drm_i915_gem_create_ext {
 	 *
 	 * For I915_GEM_CREATE_EXT_PROTECTED_CONTENT usage see
 	 * struct drm_i915_gem_create_ext_protected_content.
+	 *
+	 * For I915_GEM_CREATE_EXT_SET_PAT usage see
+	 * struct drm_i915_gem_create_ext_set_pat.
 	 */
 #define I915_GEM_CREATE_EXT_MEMORY_REGIONS 0
 #define I915_GEM_CREATE_EXT_PROTECTED_CONTENT 1
+#define I915_GEM_CREATE_EXT_SET_PAT 2
 	__u64 extensions;
 };
 
@@ -3724,6 +3728,44 @@ struct drm_i915_gem_create_ext_protected_content {
 	__u32 flags;
 };
 
+/**
+ * struct drm_i915_gem_create_ext_set_pat - The
+ * I915_GEM_CREATE_EXT_SET_PAT extension.
+ *
+ * If this extension is provided, the specified caching policy (PAT index) is
+ * applied to the buffer object.
+ *
+ * Below is an example on how to create an object with specific caching policy:
+ *
+ * .. code-block:: C
+ *
+ *      struct drm_i915_gem_create_ext_set_pat set_pat_ext = {
+ *              .base = { .name = I915_GEM_CREATE_EXT_SET_PAT },
+ *              .pat_index = 0,
+ *      };
+ *      struct drm_i915_gem_create_ext create_ext = {
+ *              .size = PAGE_SIZE,
+ *              .extensions = (uintptr_t)&set_pat_ext,
+ *      };
+ *
+ *      int err = ioctl(fd, DRM_IOCTL_I915_GEM_CREATE_EXT, &create_ext);
+ *      if (err) ...
+ */
+struct drm_i915_gem_create_ext_set_pat {
+	/** @base: Extension link. See struct i915_user_extension. */
+	struct i915_user_extension base;
+	/**
+	 * @pat_index: PAT index to be set
+	 * PAT index is a bit field in Page Table Entry to control caching
+	 * behaviors for GPU accesses. The definition of PAT index is
+	 * platform dependent and can be found in hardware specifications,
+	 * e.g. BSpec 45101.
+	 */
+	__u32 pat_index;
+	/** @rsvd: reserved for future use */
+	__u32 rsvd;
+};
+
 /* ID of the protected content session managed by i915 when PXP is active */
 #define I915_PROTECTED_CONTENT_DEFAULT_SESSION 0xf
 
-- 
2.25.1

Re: [PATCH v10 2/2] drm/i915: Allow user to set cache at BO creation

[Jordan Justen]

On 2023-05-18 22:11:03,  wrote:
> From: Fei Yang <fei.yang@intel.com>
> 
> To comply with the design that buffer objects shall have immutable
> cache setting through out their life cycle, {set, get}_caching ioctl's
> are no longer supported from MTL onward. With that change caching
> policy can only be set at object creation time. The current code
> applies a default (platform dependent) cache setting for all objects.
> However this is not optimal for performance tuning. The patch extends
> the existing gem_create uAPI to let user set PAT index for the object
> at creation time.
> The new extension is platform independent, so UMD's can switch to using
> this extension for older platforms as well, while {set, get}_caching are
> still supported on these legacy paltforms for compatibility reason.
> 
> Test igt@gem_create@create_ext_set_pat posted at
> https://patchwork.freedesktop.org/series/117695/
> 
> Tested with https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22878
> 
> Signed-off-by: Fei Yang <fei.yang@intel.com>
> Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
> Cc: Matt Roper <matthew.d.roper@intel.com>
> Cc: Andi Shyti <andi.shyti@linux.intel.com>
> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
> Acked-by: Jordan Justen <jordan.l.justen@intel.com>

Nevertheless, I'm still disappointed my suggestion was so quickly shot
down.

I tried to look over our usage Mesa of i915 extensions, and found
this:

I915_GEM_CREATE_EXT_MEMORY_REGIONS:

 * If DRM_I915_QUERY_MEMORY_REGIONS is found

I915_GEM_CREATE_EXT_PROTECTED_CONTENT:

 * Probed via the current "robust" method. Resulted in 8s driver
   startup delay in some bad scenarios.

 * Will be guarded by I915_PARAM_PXP_STATUS when available in future

I915_CONTEXT_CREATE_EXT_SETPARAM (I915_CONTEXT_PARAM_ENGINES):

 * If DRM_I915_QUERY_ENGINE_INFO is found

I915_GEM_CREATE_EXT_SET_PAT:

 * When platform is mtl or newer

I think we will continue to try to find workarounds that imply the
extension's existence, but it could be nice to have a generic way to
find out what extensions the kernel knows about.

-Jordan

Re: [PATCH v10 2/2] drm/i915: Allow user to set cache at BO creation

[Andi Shyti]

Hi,

On Thu, May 18, 2023 at 10:11:03PM -0700, fei.yang@intel.com wrote:
> From: Fei Yang <fei.yang@intel.com>
> 
> To comply with the design that buffer objects shall have immutable
> cache setting through out their life cycle, {set, get}_caching ioctl's
> are no longer supported from MTL onward. With that change caching
> policy can only be set at object creation time. The current code
> applies a default (platform dependent) cache setting for all objects.
> However this is not optimal for performance tuning. The patch extends
> the existing gem_create uAPI to let user set PAT index for the object
> at creation time.
> The new extension is platform independent, so UMD's can switch to using
> this extension for older platforms as well, while {set, get}_caching are
> still supported on these legacy paltforms for compatibility reason.
> 
> Test igt@gem_create@create_ext_set_pat posted at
> https://patchwork.freedesktop.org/series/117695/
> 
> Tested with https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22878
> 
> Signed-off-by: Fei Yang <fei.yang@intel.com>
> Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
> Cc: Matt Roper <matthew.d.roper@intel.com>
> Cc: Andi Shyti <andi.shyti@linux.intel.com>
> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
> Acked-by: Jordan Justen <jordan.l.justen@intel.com>
> Tested-by: Jordan Justen <jordan.l.justen@intel.com>

last call for comments and reviews on this patch. If nothing
comes I am going to push it tomorrow morning (Europe).

There is also a merge request from Mesa pending on this so that I
don't want to keep it hanging any longer.

Thanks,
Andi

Re: [PATCH v10 2/2] drm/i915: Allow user to set cache at BO creation

[Jordan Justen]

On 2023-05-22 04:52:56, Andi Shyti wrote:
> Hi,
> 
> On Thu, May 18, 2023 at 10:11:03PM -0700, fei.yang@intel.com wrote:
> > From: Fei Yang <fei.yang@intel.com>
> > 
> > To comply with the design that buffer objects shall have immutable
> > cache setting through out their life cycle, {set, get}_caching ioctl's
> > are no longer supported from MTL onward. With that change caching
> > policy can only be set at object creation time. The current code
> > applies a default (platform dependent) cache setting for all objects.
> > However this is not optimal for performance tuning. The patch extends
> > the existing gem_create uAPI to let user set PAT index for the object
> > at creation time.
> > The new extension is platform independent, so UMD's can switch to using
> > this extension for older platforms as well, while {set, get}_caching are
> > still supported on these legacy paltforms for compatibility reason.
> > 
> > Test igt@gem_create@create_ext_set_pat posted at
> > https://patchwork.freedesktop.org/series/117695/
> > 
> > Tested with https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22878
> > 
> > Signed-off-by: Fei Yang <fei.yang@intel.com>
> > Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
> > Cc: Matt Roper <matthew.d.roper@intel.com>
> > Cc: Andi Shyti <andi.shyti@linux.intel.com>
> > Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
> > Acked-by: Jordan Justen <jordan.l.justen@intel.com>
> > Tested-by: Jordan Justen <jordan.l.justen@intel.com>
> 
> last call for comments and reviews on this patch. If nothing
> comes I am going to push it tomorrow morning (Europe).
> 
> There is also a merge request from Mesa pending on this so that I
> don't want to keep it hanging any longer.

No need to wait any longer with regard to feedback from Mesa.

I definitely was hoping for more consideration of the userspace
request, but it's been decisively rejected. My ack was not readily
given, but it stands.

-Jordan

Re: [PATCH v10 2/2] drm/i915: Allow user to set cache at BO creation

[Andi Shyti]

> > > To comply with the design that buffer objects shall have immutable
> > > cache setting through out their life cycle, {set, get}_caching ioctl's
> > > are no longer supported from MTL onward. With that change caching
> > > policy can only be set at object creation time. The current code
> > > applies a default (platform dependent) cache setting for all objects.
> > > However this is not optimal for performance tuning. The patch extends
> > > the existing gem_create uAPI to let user set PAT index for the object
> > > at creation time.
> > > The new extension is platform independent, so UMD's can switch to using
> > > this extension for older platforms as well, while {set, get}_caching are
> > > still supported on these legacy paltforms for compatibility reason.
> > > 
> > > Test igt@gem_create@create_ext_set_pat posted at
> > > https://patchwork.freedesktop.org/series/117695/
> > > 
> > > Tested with https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22878
> > > 
> > > Signed-off-by: Fei Yang <fei.yang@intel.com>
> > > Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
> > > Cc: Matt Roper <matthew.d.roper@intel.com>
> > > Cc: Andi Shyti <andi.shyti@linux.intel.com>
> > > Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
> > > Acked-by: Jordan Justen <jordan.l.justen@intel.com>
> > > Tested-by: Jordan Justen <jordan.l.justen@intel.com>
> > 
> > last call for comments and reviews on this patch. If nothing
> > comes I am going to push it tomorrow morning (Europe).
> > 
> > There is also a merge request from Mesa pending on this so that I
> > don't want to keep it hanging any longer.
> 
> No need to wait any longer with regard to feedback from Mesa.
> 
> I definitely was hoping for more consideration of the userspace
> request, but it's been decisively rejected. My ack was not readily
> given, but it stands.

Thanks, Jordan!

> -Jordan

Re: [Intel-gfx] [PATCH v10 0/2] drm/i915: Allow user to set cache at BO creation

[Andi Shyti]

Hi Fei,

finally... pushed in drm-intel-gt-next! :)

Andi

On Thu, May 18, 2023 at 10:11:01PM -0700, fei.yang@intel.com wrote:
> From: Fei Yang <fei.yang@intel.com>
> 
> This series introduce a new extension for GEM_CREATE,
> 1. end support for set caching ioctl [PATCH 1/2]
> 2. add set_pat extension for gem_create [PATCH 2/2]
> 
> v2: drop one patch that was merged separately
>     commit 341ad0e8e254 ("drm/i915/mtl: Add PTE encode function")
> v3: rebased on https://patchwork.freedesktop.org/series/117082/
> v4: fix missing unlock introduced in v3, and
>     solve a rebase conflict
> v5: replace obj->cache_level with pat_set_by_user,
>     fix i915_cache_level_str() for legacy platforms.
> v6: rebased on https://patchwork.freedesktop.org/series/117480/
> v7: rebased on https://patchwork.freedesktop.org/series/117528/
> v8: dropped the two dependent patches that has been merged
>     separately. Add IGT link and Tested-by (MESA).
> v9: addressing comments (Andi)
> v10: acked-by and tested-by MESA
> 
> Fei Yang (2):
>   drm/i915/mtl: end support for set caching ioctl
>   drm/i915: Allow user to set cache at BO creation
> 
>  drivers/gpu/drm/i915/gem/i915_gem_create.c | 36 +++++++++++++++++++
>  drivers/gpu/drm/i915/gem/i915_gem_domain.c |  3 ++
>  drivers/gpu/drm/i915/gem/i915_gem_object.c |  6 ++++
>  drivers/gpu/drm/i915/gem/i915_gem_shmem.c  |  9 ++++-
>  include/uapi/drm/i915_drm.h                | 42 ++++++++++++++++++++++
>  tools/include/uapi/drm/i915_drm.h          | 42 ++++++++++++++++++++++
>  6 files changed, 137 insertions(+), 1 deletion(-)
> 
> -- 
> 2.25.1

Re: [Intel-gfx] [PATCH v10 0/2] drm/i915: Allow user to set cache at BO creation

[Tvrtko Ursulin]

On 23/05/2023 09:37, Andi Shyti wrote:
> Hi Fei,
> 
> finally... pushed in drm-intel-gt-next! :)

I had to revert this (uapi commit only) by force pushing, luckily it was 
the top commit.

1)
IGT is not merged yet.

2)
The tools/include/uapi/drm/i915_drm.h part of the patch was not removed.

Please fix both issues before re-sending and re-merging.

Regards,

Tvrtko

Re: [Intel-gfx] [PATCH v10 0/2] drm/i915: Allow user to set cache at BO creation

[Tvrtko Ursulin]

On 24/05/2023 12:56, Tvrtko Ursulin wrote:
> 
> On 23/05/2023 09:37, Andi Shyti wrote:
>> Hi Fei,
>>
>> finally... pushed in drm-intel-gt-next! :)
> 
> I had to revert this (uapi commit only) by force pushing, luckily it was 
> the top commit.
> 
> 1)
> IGT is not merged yet.
> 
> 2)
> The tools/include/uapi/drm/i915_drm.h part of the patch was not removed.
> 
> Please fix both issues before re-sending and re-merging.

3)
Please remove the BSpec 45101 reference too and replace it with a link 
to PRMs. I understand updated docs will land there soon which will 
include the necessary info.

Regards,

Tvrtko

Re: [Intel-gfx] [PATCH v10 0/2] drm/i915: Allow user to set cache at BO creation

[Andi Shyti]

Hi Tvrtko,

> > finally... pushed in drm-intel-gt-next! :)
> 
> I had to revert this (uapi commit only) by force pushing, luckily it was the
> top commit.

OK, sorry!

> 1)
> IGT is not merged yet.

if igt is merged without the kernel it would fail, though.

> 2)
> The tools/include/uapi/drm/i915_drm.h part of the patch was not removed.

Will follow up on these two points.

Andi

Re: [Intel-gfx] [PATCH v10 0/2] drm/i915: Allow user to set cache at BO creation

[Andi Shyti]

Hi again,

> > > finally... pushed in drm-intel-gt-next! :)
> > 
> > I had to revert this (uapi commit only) by force pushing, luckily it was the
> > top commit.
> 
> OK, sorry!
> 
> > 1)
> > IGT is not merged yet.
> 
> if igt is merged without the kernel it would fail, though.

can we at least agree on having igt patches reviewed and merge
them right after?

> > 2)
> > The tools/include/uapi/drm/i915_drm.h part of the patch was not removed.
> 
> Will follow up on these two points.

three...

Andi

Re: [Intel-gfx] [PATCH v10 0/2] drm/i915: Allow user to set cache at BO creation

[Tvrtko Ursulin]

On 24/05/2023 13:19, Andi Shyti wrote:
> Hi Tvrtko,
> 
>>> finally... pushed in drm-intel-gt-next! :)
>>
>> I had to revert this (uapi commit only) by force pushing, luckily it was the
>> top commit.
> 
> OK, sorry!
> 
>> 1)
>> IGT is not merged yet.
> 
> if igt is merged without the kernel it would fail, though.

Ideally it should skip, as with any new ABI testing. That way we can 
nicely test the older kernels with the same IGT code base (like 
drm-intel-fixes tree). So something like 
igt_require(has_pat_ext_something) should be doable. Or just igt_skip if 
most basic gem_create fails with the correct error code.

> 
>> 2)
>> The tools/include/uapi/drm/i915_drm.h part of the patch was not removed.
> 
> Will follow up on these two points.

Thank you!

Regards,

Tvrtko

Re: [Intel-gfx] [PATCH v10 0/2] drm/i915: Allow user to set cache at BO creation

[Tvrtko Ursulin]

On 24/05/2023 13:30, Andi Shyti wrote:
> Hi again,
> 
>>>> finally... pushed in drm-intel-gt-next! :)
>>>
>>> I had to revert this (uapi commit only) by force pushing, luckily it was the
>>> top commit.
>>
>> OK, sorry!
>>
>>> 1)
>>> IGT is not merged yet.
>>
>> if igt is merged without the kernel it would fail, though.
> 
> can we at least agree on having igt patches reviewed and merge
> them right after?

Yeah that okay, just make it _right_ after. :)

Regards,

Tvrtko

Extension detection by enumeration vs attempt to use extension (Was: Re: [Intel-gfx] [PATCH v10 2/2] drm/i915: Allow user to set cache at BO creation)

[Joonas Lahtinen]

Quoting Jordan Justen (2023-05-21 07:30:52)
> On 2023-05-18 22:11:03,  wrote:
> > From: Fei Yang <fei.yang@intel.com>
> > 
> > To comply with the design that buffer objects shall have immutable
> > cache setting through out their life cycle, {set, get}_caching ioctl's
> > are no longer supported from MTL onward. With that change caching
> > policy can only be set at object creation time. The current code
> > applies a default (platform dependent) cache setting for all objects.
> > However this is not optimal for performance tuning. The patch extends
> > the existing gem_create uAPI to let user set PAT index for the object
> > at creation time.
> > The new extension is platform independent, so UMD's can switch to using
> > this extension for older platforms as well, while {set, get}_caching are
> > still supported on these legacy paltforms for compatibility reason.
> > 
> > Test igt@gem_create@create_ext_set_pat posted at
> > https://patchwork.freedesktop.org/series/117695/
> > 
> > Tested with https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22878
> > 
> > Signed-off-by: Fei Yang <fei.yang@intel.com>
> > Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
> > Cc: Matt Roper <matthew.d.roper@intel.com>
> > Cc: Andi Shyti <andi.shyti@linux.intel.com>
> > Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
> > Acked-by: Jordan Justen <jordan.l.justen@intel.com>
> 
> Nevertheless, I'm still disappointed my suggestion was so quickly shot
> down.

Sorry to hear that you feel that your suggestion was shot down quickly.
The intent was not to be rude. Attempt was just to make sure we're not
blocking an important patch due to repeat of an orthogonal discussion
which has been discussed to exhaustion in past.

There are pros and cons to both solutions, some of which were recapped
in the thread quickly. We can surely continue the discussion on the
details now that the patch is not blocked.

> I tried to look over our usage Mesa of i915 extensions, and found
> this:
> 
> I915_GEM_CREATE_EXT_MEMORY_REGIONS:
> 
>  * If DRM_I915_QUERY_MEMORY_REGIONS is found
> 
> I915_GEM_CREATE_EXT_PROTECTED_CONTENT:
> 
>  * Probed via the current "robust" method. Resulted in 8s driver
>    startup delay in some bad scenarios.

I think this is an another orthogonal topic. Just listing the existence
of that extension in the kernel codebase would not really help.

The fact that an uAPI is known at compile time by kernel doesn't mean it
would not be defunctional / disabled / fused out on the specific system.

>  * Will be guarded by I915_PARAM_PXP_STATUS when available in future
> 
> I915_CONTEXT_CREATE_EXT_SETPARAM (I915_CONTEXT_PARAM_ENGINES):
> 
>  * If DRM_I915_QUERY_ENGINE_INFO is found
> 
> I915_GEM_CREATE_EXT_SET_PAT:
> 
>  * When platform is mtl or newer
> 
> I think we will continue to try to find workarounds that imply the
> extension's existence,

You're not supposed to just probe for existence of an extension in the
kernel codebase, but also check that the extension works on that system.

So probing for the extension array is just half the work. If the KMD
started to block out extensions from the array dynamically, then we
would be doing that based on adding heuristics that are better added in
the userspace. Ultimately you need to have the error handling for the
initialization added anyway to userspace, so there should not be that
much new code that needs adding.

Regards, Joonas

> but it could be nice to have a generic way to
> find out what extensions the kernel knows about.
> 
> -Jordan