[PATCH v2 0/3] Disable dynamic load balancing and support fixed balancing

[PATCH v2 0/3] Disable dynamic load balancing and support fixed balancing

[Andi Shyti]

Hi,

This series aims to disable the CCS hardware load balancing, as recommended by
hardware directives in Wa_16016805146.

In the meantime, we need to define and support a fixed CCS mode of balancing
that can be configured by the user.

Thanks,
Andi

Changelog:
==========
v1 -> v2:
 - update comment about how the user sets the ccs mode.

Andi Shyti (3):
  drm/i915/gt: Support fixed CCS mode
  drm/i915/gt: Allow user to set up the CSS mode
  drm/i915/gt: Disable HW load balancing for CCS

 drivers/gpu/drm/i915/Makefile               |   1 +
 drivers/gpu/drm/i915/gt/intel_gt.c          |   6 +
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c | 149 ++++++++++++++++++++
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h |  17 +++
 drivers/gpu/drm/i915/gt/intel_gt_regs.h     |  14 ++
 drivers/gpu/drm/i915/gt/intel_gt_sysfs.c    |   2 +
 drivers/gpu/drm/i915/gt/intel_gt_types.h    |  19 +++
 drivers/gpu/drm/i915/gt/intel_workarounds.c |   6 +
 drivers/gpu/drm/i915/i915_drv.h             |   2 +
 9 files changed, 216 insertions(+)
 create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
 create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h

-- 
2.43.0

[PATCH v2 0/3] Disable dynamic load balancing and support fixed balancing

[Andi Shyti]

Hi,

This series aims to disable the CCS hardware load balancing, as recommended by
hardware directives in Wa_16016805146.

In the meantime, we need to define and support a fixed CCS mode of balancing
that can be configured by the user.

Thanks,
Andi

Changelog:
==========
v1 -> v2:
 - update comment about how the user sets the ccs mode.

Andi Shyti (3):
  drm/i915/gt: Support fixed CCS mode
  drm/i915/gt: Allow user to set up the CSS mode
  drm/i915/gt: Disable HW load balancing for CCS

 drivers/gpu/drm/i915/Makefile               |   1 +
 drivers/gpu/drm/i915/gt/intel_gt.c          |   6 +
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c | 149 ++++++++++++++++++++
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h |  17 +++
 drivers/gpu/drm/i915/gt/intel_gt_regs.h     |  14 ++
 drivers/gpu/drm/i915/gt/intel_gt_sysfs.c    |   2 +
 drivers/gpu/drm/i915/gt/intel_gt_types.h    |  19 +++
 drivers/gpu/drm/i915/gt/intel_workarounds.c |   6 +
 drivers/gpu/drm/i915/i915_drv.h             |   2 +
 9 files changed, 216 insertions(+)
 create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
 create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h

-- 
2.43.0

[PATCH v2 1/3] drm/i915/gt: Support fixed CCS mode

[Andi Shyti]

The CCS mode involves assigning CCS engines to slices depending
on the number of slices and the number of engines the user wishes
to set.

In this patch, the default CCS setting is established during the
initial GT settings. It involves assigning only one CCS to all
the slices.

Based on a patch by Chris Wilson <chris.p.wilson@linux.intel.com>
and Tejas Upadhyay <tejas.upadhyay@intel.com>.

Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
---
 drivers/gpu/drm/i915/Makefile               |  1 +
 drivers/gpu/drm/i915/gt/intel_gt.c          |  6 ++
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c | 81 +++++++++++++++++++++
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h | 16 ++++
 drivers/gpu/drm/i915/gt/intel_gt_regs.h     | 13 ++++
 drivers/gpu/drm/i915/gt/intel_gt_types.h    | 19 +++++
 drivers/gpu/drm/i915/i915_drv.h             |  2 +
 7 files changed, 138 insertions(+)
 create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
 create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h

diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index e777686190ca..1dce15d6306b 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -119,6 +119,7 @@ gt-y += \
 	gt/intel_ggtt_fencing.o \
 	gt/intel_gt.o \
 	gt/intel_gt_buffer_pool.o \
+	gt/intel_gt_ccs_mode.o \
 	gt/intel_gt_clock_utils.o \
 	gt/intel_gt_debugfs.o \
 	gt/intel_gt_engines_debugfs.o \
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c
index a425db5ed3a2..e83c7b80c07a 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt.c
@@ -17,6 +17,7 @@
 #include "intel_engine_regs.h"
 #include "intel_ggtt_gmch.h"
 #include "intel_gt.h"
+#include "intel_gt_ccs_mode.h"
 #include "intel_gt_buffer_pool.h"
 #include "intel_gt_clock_utils.h"
 #include "intel_gt_debugfs.h"
@@ -47,6 +48,7 @@ void intel_gt_common_init_early(struct intel_gt *gt)
 	init_llist_head(&gt->watchdog.list);
 	INIT_WORK(&gt->watchdog.work, intel_gt_watchdog_work);
 
+	intel_gt_init_ccs_mode(gt);
 	intel_gt_init_buffer_pool(gt);
 	intel_gt_init_reset(gt);
 	intel_gt_init_requests(gt);
@@ -195,6 +197,9 @@ int intel_gt_init_hw(struct intel_gt *gt)
 
 	intel_gt_init_swizzling(gt);
 
+	/* Configure CCS mode */
+	intel_gt_apply_ccs_mode(gt);
+
 	/*
 	 * At least 830 can leave some of the unused rings
 	 * "active" (ie. head != tail) after resume which
@@ -860,6 +865,7 @@ void intel_gt_driver_late_release_all(struct drm_i915_private *i915)
 
 	for_each_gt(gt, i915, id) {
 		intel_uc_driver_late_release(&gt->uc);
+		intel_gt_fini_ccs_mode(gt);
 		intel_gt_fini_requests(gt);
 		intel_gt_fini_reset(gt);
 		intel_gt_fini_timelines(gt);
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
new file mode 100644
index 000000000000..fab8a77bded2
--- /dev/null
+++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
@@ -0,0 +1,81 @@
+//SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#include "i915_drv.h"
+
+#include "intel_gt.h"
+#include "intel_gt_ccs_mode.h"
+#include "intel_gt_regs.h"
+#include "intel_gt_types.h"
+
+static void __intel_gt_apply_ccs_mode(struct intel_gt *gt)
+{
+	u32 mode = XEHP_CCS_MODE_CSLICE_0_3_MASK; /* disable all by default */
+	int num_slices = hweight32(CCS_MASK(gt));
+	int num_engines = gt->ccs.mode;
+	int slice = 0;
+	int i;
+
+	if (!num_engines)
+		return;
+
+	/*
+	 * Loop over all available slices and assign each a user engine.
+	 *
+	 * With 1 engine (ccs0):
+	 *   slice 0, 1, 2, 3: ccs0
+	 *
+	 * With 2 engines (ccs0, ccs1):
+	 *   slice 0, 2: ccs0
+	 *   slice 1, 3: ccs1
+	 *
+	 * With 4 engines (ccs0, ccs1, ccs2, ccs3):
+	 *   slice 0: ccs0
+	 *   slice 1: ccs1
+	 *   slice 2: ccs2
+	 *   slice 3: ccs3
+	 *
+	 * Since the number of slices and the number of engines is
+	 * known, and we ensure that there is an exact multiple of
+	 * engines for slices, the double loop becomes a loop over each
+	 * slice.
+	 */
+	for (i = num_slices / num_engines; i < num_slices; i++) {
+		struct intel_engine_cs *engine;
+		intel_engine_mask_t tmp;
+
+		for_each_engine_masked(engine, gt, ALL_CCS(gt), tmp) {
+			/* If a slice is fused off, leave disabled */
+			while (!(CCS_MASK(gt) & BIT(slice)))
+				slice++;
+
+			mode &= ~XEHP_CCS_MODE_CSLICE(slice, XEHP_CCS_MODE_CSLICE_MASK);
+			mode |= XEHP_CCS_MODE_CSLICE(slice, engine->instance);
+
+			/* assign the next slice */
+			slice++;
+		}
+	}
+
+	intel_uncore_write(gt->uncore, XEHP_CCS_MODE, mode);
+}
+
+void intel_gt_apply_ccs_mode(struct intel_gt *gt)
+{
+	mutex_lock(&gt->ccs.mutex);
+	__intel_gt_apply_ccs_mode(gt);
+	mutex_unlock(&gt->ccs.mutex);
+}
+
+void intel_gt_init_ccs_mode(struct intel_gt *gt)
+{
+	mutex_init(&gt->ccs.mutex);
+	gt->ccs.mode = 1;
+}
+
+void intel_gt_fini_ccs_mode(struct intel_gt *gt)
+{
+	mutex_destroy(&gt->ccs.mutex);
+}
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
new file mode 100644
index 000000000000..751c5700944b
--- /dev/null
+++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef INTEL_GT_CCS_MODE_H
+#define INTEL_GT_CCS_MODE_H
+
+struct intel_gt;
+
+void intel_gt_init_ccs_mode(struct intel_gt *gt);
+void intel_gt_fini_ccs_mode(struct intel_gt *gt);
+
+void intel_gt_apply_ccs_mode(struct intel_gt *gt);
+
+#endif /* INTEL_GT_CCS_MODE_H */
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_regs.h b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
index 50962cfd1353..3e558d6d5e89 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_regs.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
@@ -1604,6 +1604,19 @@
 #define   GEN12_VOLTAGE_MASK			REG_GENMASK(10, 0)
 #define   GEN12_CAGF_MASK			REG_GENMASK(19, 11)
 
+/*
+ * Total of 4 cslices, where each cslice is in the form:
+ *   [0-3]     CCS ID
+ *   [4-6]     RSVD
+ *   [7]       Disabled
+ */
+#define XEHP_CCS_MODE				_MMIO(0x14804)
+#define   XEHP_CCS_MODE_CSLICE_0_3_MASK		REG_GENMASK(11, 0)
+#define   XEHP_CCS_MODE_CSLICE_MASK		0x7 /* CCS0-3 + rsvd */
+#define   XEHP_CCS_MODE_CSLICE_WIDTH		ilog2(XEHP_CCS_MODE_CSLICE_MASK + 1)
+#define   XEHP_CCS_MODE_CSLICE(cslice, ccs) \
+	(ccs << (cslice * XEHP_CCS_MODE_CSLICE_WIDTH))
+
 #define GEN11_GT_INTR_DW(x)			_MMIO(0x190018 + ((x) * 4))
 #define   GEN11_CSME				(31)
 #define   GEN12_HECI_2				(30)
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_types.h b/drivers/gpu/drm/i915/gt/intel_gt_types.h
index def7dd0eb6f1..45db98a5ce97 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_types.h
@@ -207,6 +207,25 @@ struct intel_gt {
 					    [MAX_ENGINE_INSTANCE + 1];
 	enum intel_submission_method submission_method;
 
+	/*
+	 * Track fixed mapping between CCS engines and compute slices.
+	 *
+	 * In order to w/a HW that has the inability to dynamically load
+	 * balance between CCS engines and EU in the compute slices, we have to
+	 * reconfigure a static mapping on the fly. We track the current CCS
+	 * configuration (set by thr user through a sysfs interface) and compare
+	 * it against the current CCS_MODE (which maps CCS engines to compute
+	 * slices). If there is only a single engine selected, we can map it to
+	 * all available compute slices for maximal single task performance
+	 * (fast/narrow). If there are more then one engine selected, we have to
+	 * reduce the number of slices allocated to each engine (wide/slow),
+	 * fairly distributing the EU between the equivalent engines.
+	 */
+	struct {
+		struct mutex mutex;
+		u32 mode;
+	} ccs;
+
 	/*
 	 * Default address space (either GGTT or ppGTT depending on arch).
 	 *
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index e81b3b2858ac..cd85889ecfe4 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -659,6 +659,8 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
 #define CCS_MASK(gt) \
 	ENGINE_INSTANCES_MASK(gt, CCS0, I915_MAX_CCS)
 
+#define ALL_CCS(gt) (CCS_MASK(gt) << CCS0)
+
 #define HAS_MEDIA_RATIO_MODE(i915) (INTEL_INFO(i915)->has_media_ratio_mode)
 
 /*
-- 
2.43.0

[PATCH v2 1/3] drm/i915/gt: Support fixed CCS mode

[Andi Shyti]

The CCS mode involves assigning CCS engines to slices depending
on the number of slices and the number of engines the user wishes
to set.

In this patch, the default CCS setting is established during the
initial GT settings. It involves assigning only one CCS to all
the slices.

Based on a patch by Chris Wilson <chris.p.wilson@linux.intel.com>
and Tejas Upadhyay <tejas.upadhyay@intel.com>.

Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
---
 drivers/gpu/drm/i915/Makefile               |  1 +
 drivers/gpu/drm/i915/gt/intel_gt.c          |  6 ++
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c | 81 +++++++++++++++++++++
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h | 16 ++++
 drivers/gpu/drm/i915/gt/intel_gt_regs.h     | 13 ++++
 drivers/gpu/drm/i915/gt/intel_gt_types.h    | 19 +++++
 drivers/gpu/drm/i915/i915_drv.h             |  2 +
 7 files changed, 138 insertions(+)
 create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
 create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h

diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index e777686190ca..1dce15d6306b 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -119,6 +119,7 @@ gt-y += \
 	gt/intel_ggtt_fencing.o \
 	gt/intel_gt.o \
 	gt/intel_gt_buffer_pool.o \
+	gt/intel_gt_ccs_mode.o \
 	gt/intel_gt_clock_utils.o \
 	gt/intel_gt_debugfs.o \
 	gt/intel_gt_engines_debugfs.o \
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c
index a425db5ed3a2..e83c7b80c07a 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt.c
@@ -17,6 +17,7 @@
 #include "intel_engine_regs.h"
 #include "intel_ggtt_gmch.h"
 #include "intel_gt.h"
+#include "intel_gt_ccs_mode.h"
 #include "intel_gt_buffer_pool.h"
 #include "intel_gt_clock_utils.h"
 #include "intel_gt_debugfs.h"
@@ -47,6 +48,7 @@ void intel_gt_common_init_early(struct intel_gt *gt)
 	init_llist_head(&gt->watchdog.list);
 	INIT_WORK(&gt->watchdog.work, intel_gt_watchdog_work);
 
+	intel_gt_init_ccs_mode(gt);
 	intel_gt_init_buffer_pool(gt);
 	intel_gt_init_reset(gt);
 	intel_gt_init_requests(gt);
@@ -195,6 +197,9 @@ int intel_gt_init_hw(struct intel_gt *gt)
 
 	intel_gt_init_swizzling(gt);
 
+	/* Configure CCS mode */
+	intel_gt_apply_ccs_mode(gt);
+
 	/*
 	 * At least 830 can leave some of the unused rings
 	 * "active" (ie. head != tail) after resume which
@@ -860,6 +865,7 @@ void intel_gt_driver_late_release_all(struct drm_i915_private *i915)
 
 	for_each_gt(gt, i915, id) {
 		intel_uc_driver_late_release(&gt->uc);
+		intel_gt_fini_ccs_mode(gt);
 		intel_gt_fini_requests(gt);
 		intel_gt_fini_reset(gt);
 		intel_gt_fini_timelines(gt);
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
new file mode 100644
index 000000000000..fab8a77bded2
--- /dev/null
+++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
@@ -0,0 +1,81 @@
+//SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#include "i915_drv.h"
+
+#include "intel_gt.h"
+#include "intel_gt_ccs_mode.h"
+#include "intel_gt_regs.h"
+#include "intel_gt_types.h"
+
+static void __intel_gt_apply_ccs_mode(struct intel_gt *gt)
+{
+	u32 mode = XEHP_CCS_MODE_CSLICE_0_3_MASK; /* disable all by default */
+	int num_slices = hweight32(CCS_MASK(gt));
+	int num_engines = gt->ccs.mode;
+	int slice = 0;
+	int i;
+
+	if (!num_engines)
+		return;
+
+	/*
+	 * Loop over all available slices and assign each a user engine.
+	 *
+	 * With 1 engine (ccs0):
+	 *   slice 0, 1, 2, 3: ccs0
+	 *
+	 * With 2 engines (ccs0, ccs1):
+	 *   slice 0, 2: ccs0
+	 *   slice 1, 3: ccs1
+	 *
+	 * With 4 engines (ccs0, ccs1, ccs2, ccs3):
+	 *   slice 0: ccs0
+	 *   slice 1: ccs1
+	 *   slice 2: ccs2
+	 *   slice 3: ccs3
+	 *
+	 * Since the number of slices and the number of engines is
+	 * known, and we ensure that there is an exact multiple of
+	 * engines for slices, the double loop becomes a loop over each
+	 * slice.
+	 */
+	for (i = num_slices / num_engines; i < num_slices; i++) {
+		struct intel_engine_cs *engine;
+		intel_engine_mask_t tmp;
+
+		for_each_engine_masked(engine, gt, ALL_CCS(gt), tmp) {
+			/* If a slice is fused off, leave disabled */
+			while (!(CCS_MASK(gt) & BIT(slice)))
+				slice++;
+
+			mode &= ~XEHP_CCS_MODE_CSLICE(slice, XEHP_CCS_MODE_CSLICE_MASK);
+			mode |= XEHP_CCS_MODE_CSLICE(slice, engine->instance);
+
+			/* assign the next slice */
+			slice++;
+		}
+	}
+
+	intel_uncore_write(gt->uncore, XEHP_CCS_MODE, mode);
+}
+
+void intel_gt_apply_ccs_mode(struct intel_gt *gt)
+{
+	mutex_lock(&gt->ccs.mutex);
+	__intel_gt_apply_ccs_mode(gt);
+	mutex_unlock(&gt->ccs.mutex);
+}
+
+void intel_gt_init_ccs_mode(struct intel_gt *gt)
+{
+	mutex_init(&gt->ccs.mutex);
+	gt->ccs.mode = 1;
+}
+
+void intel_gt_fini_ccs_mode(struct intel_gt *gt)
+{
+	mutex_destroy(&gt->ccs.mutex);
+}
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
new file mode 100644
index 000000000000..751c5700944b
--- /dev/null
+++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef INTEL_GT_CCS_MODE_H
+#define INTEL_GT_CCS_MODE_H
+
+struct intel_gt;
+
+void intel_gt_init_ccs_mode(struct intel_gt *gt);
+void intel_gt_fini_ccs_mode(struct intel_gt *gt);
+
+void intel_gt_apply_ccs_mode(struct intel_gt *gt);
+
+#endif /* INTEL_GT_CCS_MODE_H */
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_regs.h b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
index 50962cfd1353..3e558d6d5e89 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_regs.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
@@ -1604,6 +1604,19 @@
 #define   GEN12_VOLTAGE_MASK			REG_GENMASK(10, 0)
 #define   GEN12_CAGF_MASK			REG_GENMASK(19, 11)
 
+/*
+ * Total of 4 cslices, where each cslice is in the form:
+ *   [0-3]     CCS ID
+ *   [4-6]     RSVD
+ *   [7]       Disabled
+ */
+#define XEHP_CCS_MODE				_MMIO(0x14804)
+#define   XEHP_CCS_MODE_CSLICE_0_3_MASK		REG_GENMASK(11, 0)
+#define   XEHP_CCS_MODE_CSLICE_MASK		0x7 /* CCS0-3 + rsvd */
+#define   XEHP_CCS_MODE_CSLICE_WIDTH		ilog2(XEHP_CCS_MODE_CSLICE_MASK + 1)
+#define   XEHP_CCS_MODE_CSLICE(cslice, ccs) \
+	(ccs << (cslice * XEHP_CCS_MODE_CSLICE_WIDTH))
+
 #define GEN11_GT_INTR_DW(x)			_MMIO(0x190018 + ((x) * 4))
 #define   GEN11_CSME				(31)
 #define   GEN12_HECI_2				(30)
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_types.h b/drivers/gpu/drm/i915/gt/intel_gt_types.h
index def7dd0eb6f1..45db98a5ce97 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_types.h
@@ -207,6 +207,25 @@ struct intel_gt {
 					    [MAX_ENGINE_INSTANCE + 1];
 	enum intel_submission_method submission_method;
 
+	/*
+	 * Track fixed mapping between CCS engines and compute slices.
+	 *
+	 * In order to w/a HW that has the inability to dynamically load
+	 * balance between CCS engines and EU in the compute slices, we have to
+	 * reconfigure a static mapping on the fly. We track the current CCS
+	 * configuration (set by thr user through a sysfs interface) and compare
+	 * it against the current CCS_MODE (which maps CCS engines to compute
+	 * slices). If there is only a single engine selected, we can map it to
+	 * all available compute slices for maximal single task performance
+	 * (fast/narrow). If there are more then one engine selected, we have to
+	 * reduce the number of slices allocated to each engine (wide/slow),
+	 * fairly distributing the EU between the equivalent engines.
+	 */
+	struct {
+		struct mutex mutex;
+		u32 mode;
+	} ccs;
+
 	/*
 	 * Default address space (either GGTT or ppGTT depending on arch).
 	 *
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index e81b3b2858ac..cd85889ecfe4 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -659,6 +659,8 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
 #define CCS_MASK(gt) \
 	ENGINE_INSTANCES_MASK(gt, CCS0, I915_MAX_CCS)
 
+#define ALL_CCS(gt) (CCS_MASK(gt) << CCS0)
+
 #define HAS_MEDIA_RATIO_MODE(i915) (INTEL_INFO(i915)->has_media_ratio_mode)
 
 /*
-- 
2.43.0

[PATCH v2 2/3] drm/i915/gt: Allow user to set up the CSS mode

[Andi Shyti]

Now that the CCS mode is configurable, an interface has been
exposed in the GT's sysfs set of files, allowing users to set the
mode.

Additionally, another interface has been added to display the
number of available slices, named 'num_slices.'

Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
---
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c | 68 +++++++++++++++++++++
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h |  1 +
 drivers/gpu/drm/i915/gt/intel_gt_sysfs.c    |  2 +
 3 files changed, 71 insertions(+)

diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
index fab8a77bded2..88663698eb1f 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
@@ -7,6 +7,7 @@
 
 #include "intel_gt.h"
 #include "intel_gt_ccs_mode.h"
+#include "intel_gt_print.h"
 #include "intel_gt_regs.h"
 #include "intel_gt_types.h"
 
@@ -79,3 +80,70 @@ void intel_gt_fini_ccs_mode(struct intel_gt *gt)
 {
 	mutex_destroy(&gt->ccs.mutex);
 }
+
+static ssize_t
+ccs_mode_show(struct kobject *kobj, struct kobj_attribute *attr, char *buff)
+{
+	struct intel_gt *gt = container_of(kobj, struct intel_gt, sysfs_gt);
+
+	return sysfs_emit(buff, "%u\n", gt->ccs.mode);
+}
+
+static ssize_t
+ccs_mode_store(struct kobject *kobj, struct kobj_attribute *attr,
+	       const char *buff, size_t count)
+{
+	struct intel_gt *gt = container_of(kobj, struct intel_gt, sysfs_gt);
+	int num_slices = hweight32(CCS_MASK(gt));
+	int err;
+	u32 val;
+
+	err = kstrtou32(buff, 0, &val);
+	if (err)
+		return err;
+
+	if ((!val) || (val > num_slices) || (val % num_slices))
+		return -EINVAL;
+
+	mutex_lock(&gt->ccs.mutex);
+
+	if (val == gt->ccs.mode)
+		goto out;
+
+	gt->ccs.mode = val;
+	intel_gt_apply_ccs_mode(gt);
+
+out:
+	mutex_unlock(&gt->ccs.mutex);
+
+	return count;
+}
+
+static ssize_t
+num_slices_show(struct kobject *kobj, struct kobj_attribute *attr, char *buff)
+{
+	struct intel_gt *gt = container_of(kobj, struct intel_gt, sysfs_gt);
+	u32 num_slices;
+
+	num_slices = hweight32(CCS_MASK(gt));
+
+	return sysfs_emit(buff, "%u\n", num_slices);
+}
+
+static struct kobj_attribute ccs_mode = __ATTR_RW(ccs_mode);
+static struct kobj_attribute num_slices = __ATTR_RO(num_slices);
+
+static const struct attribute * const ccs_mode_attrs[] = {
+	&ccs_mode.attr,
+	&num_slices.attr,
+	NULL
+};
+
+void intel_gt_sysfs_ccs_mode(struct intel_gt *gt)
+{
+	int ret;
+
+	ret = sysfs_create_files(&gt->sysfs_gt, ccs_mode_attrs);
+	if (ret)
+		gt_warn(gt, "Failed to create ccs mode sysfs files");
+}
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
index 751c5700944b..ae96de1b36c5 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
@@ -12,5 +12,6 @@ void intel_gt_init_ccs_mode(struct intel_gt *gt);
 void intel_gt_fini_ccs_mode(struct intel_gt *gt);
 
 void intel_gt_apply_ccs_mode(struct intel_gt *gt);
+void intel_gt_sysfs_ccs_mode(struct intel_gt *gt);
 
 #endif /* INTEL_GT_CCS_MODE_H */
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c b/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c
index 33cba406b569..a0290347938d 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c
@@ -12,6 +12,7 @@
 #include "i915_drv.h"
 #include "i915_sysfs.h"
 #include "intel_gt.h"
+#include "intel_gt_ccs_mode.h"
 #include "intel_gt_print.h"
 #include "intel_gt_sysfs.h"
 #include "intel_gt_sysfs_pm.h"
@@ -101,6 +102,7 @@ void intel_gt_sysfs_register(struct intel_gt *gt)
 		goto exit_fail;
 
 	intel_gt_sysfs_pm_init(gt, &gt->sysfs_gt);
+	intel_gt_sysfs_ccs_mode(gt);
 
 	return;
 
-- 
2.43.0

[PATCH v2 2/3] drm/i915/gt: Allow user to set up the CSS mode

[Andi Shyti]

Now that the CCS mode is configurable, an interface has been
exposed in the GT's sysfs set of files, allowing users to set the
mode.

Additionally, another interface has been added to display the
number of available slices, named 'num_slices.'

Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
---
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c | 68 +++++++++++++++++++++
 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h |  1 +
 drivers/gpu/drm/i915/gt/intel_gt_sysfs.c    |  2 +
 3 files changed, 71 insertions(+)

diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
index fab8a77bded2..88663698eb1f 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
@@ -7,6 +7,7 @@
 
 #include "intel_gt.h"
 #include "intel_gt_ccs_mode.h"
+#include "intel_gt_print.h"
 #include "intel_gt_regs.h"
 #include "intel_gt_types.h"
 
@@ -79,3 +80,70 @@ void intel_gt_fini_ccs_mode(struct intel_gt *gt)
 {
 	mutex_destroy(&gt->ccs.mutex);
 }
+
+static ssize_t
+ccs_mode_show(struct kobject *kobj, struct kobj_attribute *attr, char *buff)
+{
+	struct intel_gt *gt = container_of(kobj, struct intel_gt, sysfs_gt);
+
+	return sysfs_emit(buff, "%u\n", gt->ccs.mode);
+}
+
+static ssize_t
+ccs_mode_store(struct kobject *kobj, struct kobj_attribute *attr,
+	       const char *buff, size_t count)
+{
+	struct intel_gt *gt = container_of(kobj, struct intel_gt, sysfs_gt);
+	int num_slices = hweight32(CCS_MASK(gt));
+	int err;
+	u32 val;
+
+	err = kstrtou32(buff, 0, &val);
+	if (err)
+		return err;
+
+	if ((!val) || (val > num_slices) || (val % num_slices))
+		return -EINVAL;
+
+	mutex_lock(&gt->ccs.mutex);
+
+	if (val == gt->ccs.mode)
+		goto out;
+
+	gt->ccs.mode = val;
+	intel_gt_apply_ccs_mode(gt);
+
+out:
+	mutex_unlock(&gt->ccs.mutex);
+
+	return count;
+}
+
+static ssize_t
+num_slices_show(struct kobject *kobj, struct kobj_attribute *attr, char *buff)
+{
+	struct intel_gt *gt = container_of(kobj, struct intel_gt, sysfs_gt);
+	u32 num_slices;
+
+	num_slices = hweight32(CCS_MASK(gt));
+
+	return sysfs_emit(buff, "%u\n", num_slices);
+}
+
+static struct kobj_attribute ccs_mode = __ATTR_RW(ccs_mode);
+static struct kobj_attribute num_slices = __ATTR_RO(num_slices);
+
+static const struct attribute * const ccs_mode_attrs[] = {
+	&ccs_mode.attr,
+	&num_slices.attr,
+	NULL
+};
+
+void intel_gt_sysfs_ccs_mode(struct intel_gt *gt)
+{
+	int ret;
+
+	ret = sysfs_create_files(&gt->sysfs_gt, ccs_mode_attrs);
+	if (ret)
+		gt_warn(gt, "Failed to create ccs mode sysfs files");
+}
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
index 751c5700944b..ae96de1b36c5 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
@@ -12,5 +12,6 @@ void intel_gt_init_ccs_mode(struct intel_gt *gt);
 void intel_gt_fini_ccs_mode(struct intel_gt *gt);
 
 void intel_gt_apply_ccs_mode(struct intel_gt *gt);
+void intel_gt_sysfs_ccs_mode(struct intel_gt *gt);
 
 #endif /* INTEL_GT_CCS_MODE_H */
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c b/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c
index 33cba406b569..a0290347938d 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt_sysfs.c
@@ -12,6 +12,7 @@
 #include "i915_drv.h"
 #include "i915_sysfs.h"
 #include "intel_gt.h"
+#include "intel_gt_ccs_mode.h"
 #include "intel_gt_print.h"
 #include "intel_gt_sysfs.h"
 #include "intel_gt_sysfs_pm.h"
@@ -101,6 +102,7 @@ void intel_gt_sysfs_register(struct intel_gt *gt)
 		goto exit_fail;
 
 	intel_gt_sysfs_pm_init(gt, &gt->sysfs_gt);
+	intel_gt_sysfs_ccs_mode(gt);
 
 	return;
 
-- 
2.43.0

[PATCH v2 3/3] drm/i915/gt: Disable HW load balancing for CCS

[Andi Shyti]

The hardware is not able to dynamically balance the load between
CCS engines. Wa_16016805146 suggests disabling it for all
platforms.

Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
---
 drivers/gpu/drm/i915/gt/intel_gt_regs.h     | 1 +
 drivers/gpu/drm/i915/gt/intel_workarounds.c | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/drivers/gpu/drm/i915/gt/intel_gt_regs.h b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
index 3e558d6d5e89..edaa446abd91 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_regs.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
@@ -1478,6 +1478,7 @@
 
 #define GEN12_RCU_MODE				_MMIO(0x14800)
 #define   GEN12_RCU_MODE_CCS_ENABLE		REG_BIT(0)
+#define   XEHP_RCU_MODE_FIXED_SLICE_CCS_MODE	REG_BIT(1)
 
 #define CHV_FUSE_GT				_MMIO(VLV_GUNIT_BASE + 0x2168)
 #define   CHV_FGT_DISABLE_SS0			(1 << 10)
diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c
index 3eacbc50caf8..a7718f7d2925 100644
--- a/drivers/gpu/drm/i915/gt/intel_workarounds.c
+++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c
@@ -2978,6 +2978,12 @@ general_render_compute_wa_init(struct intel_engine_cs *engine, struct i915_wa_li
 		wa_mcr_masked_en(wal, GEN8_HALF_SLICE_CHICKEN1,
 				 GEN7_PSD_SINGLE_PORT_DISPATCH_ENABLE);
 	}
+
+	/*
+	 * Wa_16016805146: disable the CCS load balancing
+	 * indiscriminately for all the platforms
+	 */
+	wa_masked_en(wal, GEN12_RCU_MODE, XEHP_RCU_MODE_FIXED_SLICE_CCS_MODE);
 }
 
 static void
-- 
2.43.0

[PATCH v2 3/3] drm/i915/gt: Disable HW load balancing for CCS

[Andi Shyti]

The hardware is not able to dynamically balance the load between
CCS engines. Wa_16016805146 suggests disabling it for all
platforms.

Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
---
 drivers/gpu/drm/i915/gt/intel_gt_regs.h     | 1 +
 drivers/gpu/drm/i915/gt/intel_workarounds.c | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/drivers/gpu/drm/i915/gt/intel_gt_regs.h b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
index 3e558d6d5e89..edaa446abd91 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_regs.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
@@ -1478,6 +1478,7 @@
 
 #define GEN12_RCU_MODE				_MMIO(0x14800)
 #define   GEN12_RCU_MODE_CCS_ENABLE		REG_BIT(0)
+#define   XEHP_RCU_MODE_FIXED_SLICE_CCS_MODE	REG_BIT(1)
 
 #define CHV_FUSE_GT				_MMIO(VLV_GUNIT_BASE + 0x2168)
 #define   CHV_FGT_DISABLE_SS0			(1 << 10)
diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c
index 3eacbc50caf8..a7718f7d2925 100644
--- a/drivers/gpu/drm/i915/gt/intel_workarounds.c
+++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c
@@ -2978,6 +2978,12 @@ general_render_compute_wa_init(struct intel_engine_cs *engine, struct i915_wa_li
 		wa_mcr_masked_en(wal, GEN8_HALF_SLICE_CHICKEN1,
 				 GEN7_PSD_SINGLE_PORT_DISPATCH_ENABLE);
 	}
+
+	/*
+	 * Wa_16016805146: disable the CCS load balancing
+	 * indiscriminately for all the platforms
+	 */
+	wa_masked_en(wal, GEN12_RCU_MODE, XEHP_RCU_MODE_FIXED_SLICE_CCS_MODE);
 }
 
 static void
-- 
2.43.0

✗ Fi.CI.CHECKPATCH: warning for Disable dynamic load balancing and support fixed balancing

[Patchwork]

== Series Details ==

Series: Disable dynamic load balancing and support fixed balancing
URL   : https://patchwork.freedesktop.org/series/128202/
State : warning

== Summary ==

Error: dim checkpatch failed
478beb8ee33d drm/i915/gt: Support fixed CCS mode
Traceback (most recent call last):
  File "scripts/spdxcheck.py", line 6, in <module>
    from ply import lex, yacc
ModuleNotFoundError: No module named 'ply'
-:74: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating?
#74: 
new file mode 100644

-:79: WARNING:SPDX_LICENSE_TAG: Missing or malformed SPDX-License-Identifier tag in line 1
#79: FILE: drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c:1:
+//SPDX-License-Identifier: MIT

-:200: CHECK:MACRO_ARG_PRECEDENCE: Macro argument 'cslice' may be better as '(cslice)' to avoid precedence issues
#200: FILE: drivers/gpu/drm/i915/gt/intel_gt_regs.h:1617:
+#define   XEHP_CCS_MODE_CSLICE(cslice, ccs) \
+	(ccs << (cslice * XEHP_CCS_MODE_CSLICE_WIDTH))

-:200: CHECK:MACRO_ARG_PRECEDENCE: Macro argument 'ccs' may be better as '(ccs)' to avoid precedence issues
#200: FILE: drivers/gpu/drm/i915/gt/intel_gt_regs.h:1617:
+#define   XEHP_CCS_MODE_CSLICE(cslice, ccs) \
+	(ccs << (cslice * XEHP_CCS_MODE_CSLICE_WIDTH))

-:229: CHECK:UNCOMMENTED_DEFINITION: struct mutex definition without comment
#229: FILE: drivers/gpu/drm/i915/gt/intel_gt_types.h:225:
+		struct mutex mutex;

total: 0 errors, 2 warnings, 3 checks, 186 lines checked
592cf5d3fe96 drm/i915/gt: Allow user to set up the CSS mode
-:57: CHECK:UNNECESSARY_PARENTHESES: Unnecessary parentheses around '!val'
#57: FILE: drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c:105:
+	if ((!val) || (val > num_slices) || (val % num_slices))

-:57: CHECK:UNNECESSARY_PARENTHESES: Unnecessary parentheses around 'val > num_slices'
#57: FILE: drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c:105:
+	if ((!val) || (val > num_slices) || (val % num_slices))

total: 0 errors, 0 warnings, 2 checks, 97 lines checked
0a1daf5f7606 drm/i915/gt: Disable HW load balancing for CCS

✗ Fi.CI.SPARSE: warning for Disable dynamic load balancing and support fixed balancing

[Patchwork]

== Series Details ==

Series: Disable dynamic load balancing and support fixed balancing
URL   : https://patchwork.freedesktop.org/series/128202/
State : warning

== Summary ==

Error: dim sparse failed
Sparse version: v0.6.2
Fast mode used, each commit won't be checked separately.

Re: [PATCH v2 1/3] drm/i915/gt: Support fixed CCS mode

[Tvrtko Ursulin]

On 04/01/2024 14:35, Andi Shyti wrote:
> The CCS mode involves assigning CCS engines to slices depending
> on the number of slices and the number of engines the user wishes
> to set.
> 
> In this patch, the default CCS setting is established during the
> initial GT settings. It involves assigning only one CCS to all
> the slices.
> 
> Based on a patch by Chris Wilson <chris.p.wilson@linux.intel.com>
> and Tejas Upadhyay <tejas.upadhyay@intel.com>.
> 
> Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
> Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Cc: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
> Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
> ---
>   drivers/gpu/drm/i915/Makefile               |  1 +
>   drivers/gpu/drm/i915/gt/intel_gt.c          |  6 ++
>   drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c | 81 +++++++++++++++++++++
>   drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h | 16 ++++
>   drivers/gpu/drm/i915/gt/intel_gt_regs.h     | 13 ++++
>   drivers/gpu/drm/i915/gt/intel_gt_types.h    | 19 +++++
>   drivers/gpu/drm/i915/i915_drv.h             |  2 +
>   7 files changed, 138 insertions(+)
>   create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
>   create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
> 
> diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
> index e777686190ca..1dce15d6306b 100644
> --- a/drivers/gpu/drm/i915/Makefile
> +++ b/drivers/gpu/drm/i915/Makefile
> @@ -119,6 +119,7 @@ gt-y += \
>   	gt/intel_ggtt_fencing.o \
>   	gt/intel_gt.o \
>   	gt/intel_gt_buffer_pool.o \
> +	gt/intel_gt_ccs_mode.o \
>   	gt/intel_gt_clock_utils.o \
>   	gt/intel_gt_debugfs.o \
>   	gt/intel_gt_engines_debugfs.o \
> diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c
> index a425db5ed3a2..e83c7b80c07a 100644
> --- a/drivers/gpu/drm/i915/gt/intel_gt.c
> +++ b/drivers/gpu/drm/i915/gt/intel_gt.c
> @@ -17,6 +17,7 @@
>   #include "intel_engine_regs.h"
>   #include "intel_ggtt_gmch.h"
>   #include "intel_gt.h"
> +#include "intel_gt_ccs_mode.h"
>   #include "intel_gt_buffer_pool.h"
>   #include "intel_gt_clock_utils.h"
>   #include "intel_gt_debugfs.h"
> @@ -47,6 +48,7 @@ void intel_gt_common_init_early(struct intel_gt *gt)
>   	init_llist_head(&gt->watchdog.list);
>   	INIT_WORK(&gt->watchdog.work, intel_gt_watchdog_work);
>   
> +	intel_gt_init_ccs_mode(gt);
>   	intel_gt_init_buffer_pool(gt);
>   	intel_gt_init_reset(gt);
>   	intel_gt_init_requests(gt);
> @@ -195,6 +197,9 @@ int intel_gt_init_hw(struct intel_gt *gt)
>   
>   	intel_gt_init_swizzling(gt);
>   
> +	/* Configure CCS mode */
> +	intel_gt_apply_ccs_mode(gt);
> +
>   	/*
>   	 * At least 830 can leave some of the unused rings
>   	 * "active" (ie. head != tail) after resume which
> @@ -860,6 +865,7 @@ void intel_gt_driver_late_release_all(struct drm_i915_private *i915)
>   
>   	for_each_gt(gt, i915, id) {
>   		intel_uc_driver_late_release(&gt->uc);
> +		intel_gt_fini_ccs_mode(gt);
>   		intel_gt_fini_requests(gt);
>   		intel_gt_fini_reset(gt);
>   		intel_gt_fini_timelines(gt);
> diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
> new file mode 100644
> index 000000000000..fab8a77bded2
> --- /dev/null
> +++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c
> @@ -0,0 +1,81 @@
> +//SPDX-License-Identifier: MIT
> +/*
> + * Copyright © 2023 Intel Corporation
> + */
> +
> +#include "i915_drv.h"
> +
> +#include "intel_gt.h"
> +#include "intel_gt_ccs_mode.h"
> +#include "intel_gt_regs.h"
> +#include "intel_gt_types.h"
> +
> +static void __intel_gt_apply_ccs_mode(struct intel_gt *gt)
> +{
> +	u32 mode = XEHP_CCS_MODE_CSLICE_0_3_MASK; /* disable all by default */
> +	int num_slices = hweight32(CCS_MASK(gt));
> +	int num_engines = gt->ccs.mode;
> +	int slice = 0;
> +	int i;
> +
> +	if (!num_engines)
> +		return;
> +
> +	/*
> +	 * Loop over all available slices and assign each a user engine.
> +	 *
> +	 * With 1 engine (ccs0):
> +	 *   slice 0, 1, 2, 3: ccs0
> +	 *
> +	 * With 2 engines (ccs0, ccs1):
> +	 *   slice 0, 2: ccs0
> +	 *   slice 1, 3: ccs1
> +	 *
> +	 * With 4 engines (ccs0, ccs1, ccs2, ccs3):
> +	 *   slice 0: ccs0
> +	 *   slice 1: ccs1
> +	 *   slice 2: ccs2
> +	 *   slice 3: ccs3
> +	 *
> +	 * Since the number of slices and the number of engines is
> +	 * known, and we ensure that there is an exact multiple of
> +	 * engines for slices, the double loop becomes a loop over each
> +	 * slice.
> +	 */
> +	for (i = num_slices / num_engines; i < num_slices; i++) {
> +		struct intel_engine_cs *engine;
> +		intel_engine_mask_t tmp;
> +
> +		for_each_engine_masked(engine, gt, ALL_CCS(gt), tmp) {
> +			/* If a slice is fused off, leave disabled */
> +			while (!(CCS_MASK(gt) & BIT(slice)))
> +				slice++;
> +
> +			mode &= ~XEHP_CCS_MODE_CSLICE(slice, XEHP_CCS_MODE_CSLICE_MASK);
> +			mode |= XEHP_CCS_MODE_CSLICE(slice, engine->instance);
> +
> +			/* assign the next slice */
> +			slice++;
> +		}
> +	}
> +
> +	intel_uncore_write(gt->uncore, XEHP_CCS_MODE, mode);
> +}
> +
> +void intel_gt_apply_ccs_mode(struct intel_gt *gt)
> +{
> +	mutex_lock(&gt->ccs.mutex);
> +	__intel_gt_apply_ccs_mode(gt);
> +	mutex_unlock(&gt->ccs.mutex);
> +}
> +
> +void intel_gt_init_ccs_mode(struct intel_gt *gt)
> +{
> +	mutex_init(&gt->ccs.mutex);
> +	gt->ccs.mode = 1;

What is '1'? And this question carries over to the sysfs interface in 
the following patch - who will use it and where it is documented how to 
use it?

Also, should this setting somehow be gated by an applicable platform? Or 
if not on setting then when acting on it in __intel_gt_apply_ccs_mode?

Creation of sysfs files as well should be gated by platform too in the 
following patch?

Regards,

Tvrtko

> +}
> +
> +void intel_gt_fini_ccs_mode(struct intel_gt *gt)
> +{
> +	mutex_destroy(&gt->ccs.mutex);
> +}
> diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
> new file mode 100644
> index 000000000000..751c5700944b
> --- /dev/null
> +++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
> @@ -0,0 +1,16 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * Copyright © 2023 Intel Corporation
> + */
> +
> +#ifndef INTEL_GT_CCS_MODE_H
> +#define INTEL_GT_CCS_MODE_H
> +
> +struct intel_gt;
> +
> +void intel_gt_init_ccs_mode(struct intel_gt *gt);
> +void intel_gt_fini_ccs_mode(struct intel_gt *gt);
> +
> +void intel_gt_apply_ccs_mode(struct intel_gt *gt);
> +
> +#endif /* INTEL_GT_CCS_MODE_H */
> diff --git a/drivers/gpu/drm/i915/gt/intel_gt_regs.h b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
> index 50962cfd1353..3e558d6d5e89 100644
> --- a/drivers/gpu/drm/i915/gt/intel_gt_regs.h
> +++ b/drivers/gpu/drm/i915/gt/intel_gt_regs.h
> @@ -1604,6 +1604,19 @@
>   #define   GEN12_VOLTAGE_MASK			REG_GENMASK(10, 0)
>   #define   GEN12_CAGF_MASK			REG_GENMASK(19, 11)
>   
> +/*
> + * Total of 4 cslices, where each cslice is in the form:
> + *   [0-3]     CCS ID
> + *   [4-6]     RSVD
> + *   [7]       Disabled
> + */
> +#define XEHP_CCS_MODE				_MMIO(0x14804)
> +#define   XEHP_CCS_MODE_CSLICE_0_3_MASK		REG_GENMASK(11, 0)
> +#define   XEHP_CCS_MODE_CSLICE_MASK		0x7 /* CCS0-3 + rsvd */
> +#define   XEHP_CCS_MODE_CSLICE_WIDTH		ilog2(XEHP_CCS_MODE_CSLICE_MASK + 1)
> +#define   XEHP_CCS_MODE_CSLICE(cslice, ccs) \
> +	(ccs << (cslice * XEHP_CCS_MODE_CSLICE_WIDTH))
> +
>   #define GEN11_GT_INTR_DW(x)			_MMIO(0x190018 + ((x) * 4))
>   #define   GEN11_CSME				(31)
>   #define   GEN12_HECI_2				(30)
> diff --git a/drivers/gpu/drm/i915/gt/intel_gt_types.h b/drivers/gpu/drm/i915/gt/intel_gt_types.h
> index def7dd0eb6f1..45db98a5ce97 100644
> --- a/drivers/gpu/drm/i915/gt/intel_gt_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_gt_types.h
> @@ -207,6 +207,25 @@ struct intel_gt {
>   					    [MAX_ENGINE_INSTANCE + 1];
>   	enum intel_submission_method submission_method;
>   
> +	/*
> +	 * Track fixed mapping between CCS engines and compute slices.
> +	 *
> +	 * In order to w/a HW that has the inability to dynamically load
> +	 * balance between CCS engines and EU in the compute slices, we have to
> +	 * reconfigure a static mapping on the fly. We track the current CCS
> +	 * configuration (set by thr user through a sysfs interface) and compare
> +	 * it against the current CCS_MODE (which maps CCS engines to compute
> +	 * slices). If there is only a single engine selected, we can map it to
> +	 * all available compute slices for maximal single task performance
> +	 * (fast/narrow). If there are more then one engine selected, we have to
> +	 * reduce the number of slices allocated to each engine (wide/slow),
> +	 * fairly distributing the EU between the equivalent engines.
> +	 */
> +	struct {
> +		struct mutex mutex;
> +		u32 mode;
> +	} ccs;
> +
>   	/*
>   	 * Default address space (either GGTT or ppGTT depending on arch).
>   	 *
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index e81b3b2858ac..cd85889ecfe4 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -659,6 +659,8 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
>   #define CCS_MASK(gt) \
>   	ENGINE_INSTANCES_MASK(gt, CCS0, I915_MAX_CCS)
>   
> +#define ALL_CCS(gt) (CCS_MASK(gt) << CCS0)
> +
>   #define HAS_MEDIA_RATIO_MODE(i915) (INTEL_INFO(i915)->has_media_ratio_mode)
>   
>   /*

✗ Fi.CI.BAT: failure for Disable dynamic load balancing and support fixed balancing

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 11439 bytes --]

== Series Details ==

Series: Disable dynamic load balancing and support fixed balancing
URL   : https://patchwork.freedesktop.org/series/128202/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_14078 -> Patchwork_128202v1
====================================================

Summary
-------

  **FAILURE**

  Serious unknown changes coming with Patchwork_128202v1 absolutely need to be
  verified manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in Patchwork_128202v1, please notify your bug team (I915-ci-infra@lists.freedesktop.org) to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/index.html

Participating hosts (38 -> 35)
------------------------------

  Additional (2): bat-rpls-2 fi-pnv-d510 
  Missing    (5): fi-bsw-n3050 fi-apl-guc fi-snb-2520m fi-glk-j4005 bat-mtlp-8 

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in Patchwork_128202v1:

### IGT changes ###

#### Possible regressions ####

  * igt@i915_module_load@load:
    - fi-skl-6600u:       [PASS][1] -> [ABORT][2]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-skl-6600u/igt@i915_module_load@load.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-skl-6600u/igt@i915_module_load@load.html
    - fi-skl-guc:         [PASS][3] -> [ABORT][4]
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-skl-guc/igt@i915_module_load@load.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-skl-guc/igt@i915_module_load@load.html
    - fi-kbl-7567u:       [PASS][5] -> [ABORT][6]
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-kbl-7567u/igt@i915_module_load@load.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-kbl-7567u/igt@i915_module_load@load.html
    - fi-cfl-8700k:       [PASS][7] -> [ABORT][8]
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-cfl-8700k/igt@i915_module_load@load.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-cfl-8700k/igt@i915_module_load@load.html
    - fi-bsw-nick:        [PASS][9] -> [ABORT][10]
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-bsw-nick/igt@i915_module_load@load.html
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-bsw-nick/igt@i915_module_load@load.html
    - bat-kbl-2:          [PASS][11] -> [ABORT][12]
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/bat-kbl-2/igt@i915_module_load@load.html
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-kbl-2/igt@i915_module_load@load.html
    - fi-cfl-guc:         [PASS][13] -> [ABORT][14]
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-cfl-guc/igt@i915_module_load@load.html
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-cfl-guc/igt@i915_module_load@load.html
    - fi-kbl-x1275:       [PASS][15] -> [ABORT][16]
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-kbl-x1275/igt@i915_module_load@load.html
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-kbl-x1275/igt@i915_module_load@load.html
    - fi-cfl-8109u:       [PASS][17] -> [ABORT][18]
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-cfl-8109u/igt@i915_module_load@load.html
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-cfl-8109u/igt@i915_module_load@load.html
    - fi-ivb-3770:        [PASS][19] -> [ABORT][20]
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-ivb-3770/igt@i915_module_load@load.html
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-ivb-3770/igt@i915_module_load@load.html
    - fi-kbl-guc:         [PASS][21] -> [ABORT][22]
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-kbl-guc/igt@i915_module_load@load.html
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-kbl-guc/igt@i915_module_load@load.html

  * igt@i915_selftest@live@client:
    - fi-elk-e7500:       [PASS][23] -> [DMESG-WARN][24] +48 other tests dmesg-warn
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-elk-e7500/igt@i915_selftest@live@client.html
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-elk-e7500/igt@i915_selftest@live@client.html

  * igt@i915_selftest@live@coherency:
    - bat-jsl-3:          [PASS][25] -> [DMESG-WARN][26] +47 other tests dmesg-warn
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/bat-jsl-3/igt@i915_selftest@live@coherency.html
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-jsl-3/igt@i915_selftest@live@coherency.html

  * igt@i915_selftest@live@gt_contexts:
    - fi-ilk-650:         [PASS][27] -> [DMESG-WARN][28] +48 other tests dmesg-warn
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/fi-ilk-650/igt@i915_selftest@live@gt_contexts.html
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-ilk-650/igt@i915_selftest@live@gt_contexts.html

  * igt@i915_selftest@live@gt_pm:
    - bat-jsl-1:          [PASS][29] -> [DMESG-WARN][30] +47 other tests dmesg-warn
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/bat-jsl-1/igt@i915_selftest@live@gt_pm.html
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-jsl-1/igt@i915_selftest@live@gt_pm.html

  
Known issues
------------

  Here are the changes found in Patchwork_128202v1 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@debugfs_test@basic-hwmon:
    - bat-rpls-2:         NOTRUN -> [SKIP][31] ([i915#9318])
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-rpls-2/igt@debugfs_test@basic-hwmon.html

  * igt@gem_lmem_swapping@basic:
    - fi-pnv-d510:        NOTRUN -> [SKIP][32] ([fdo#109271]) +28 other tests skip
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/fi-pnv-d510/igt@gem_lmem_swapping@basic.html

  * igt@gem_lmem_swapping@parallel-random-engines:
    - bat-adlm-1:         NOTRUN -> [SKIP][33] ([i915#4613]) +3 other tests skip
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-adlm-1/igt@gem_lmem_swapping@parallel-random-engines.html

  * igt@gem_tiled_pread_basic:
    - bat-rpls-2:         NOTRUN -> [SKIP][34] ([i915#3282])
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-rpls-2/igt@gem_tiled_pread_basic.html

  * igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy:
    - bat-rpls-2:         NOTRUN -> [SKIP][35] ([i915#4103]) +1 other test skip
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-rpls-2/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy.html

  * igt@kms_dsc@dsc-basic:
    - bat-rpls-2:         NOTRUN -> [SKIP][36] ([i915#3555] / [i915#3840] / [i915#9886])
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-rpls-2/igt@kms_dsc@dsc-basic.html

  * igt@kms_force_connector_basic@force-load-detect:
    - bat-rpls-2:         NOTRUN -> [SKIP][37] ([fdo#109285])
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-rpls-2/igt@kms_force_connector_basic@force-load-detect.html

  * igt@kms_pipe_crc_basic@suspend-read-crc:
    - bat-adlm-1:         NOTRUN -> [SKIP][38] ([i915#9875] / [i915#9900])
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-adlm-1/igt@kms_pipe_crc_basic@suspend-read-crc.html

  * igt@kms_pipe_crc_basic@suspend-read-crc@pipe-a-edp-1:
    - bat-jsl-3:          [PASS][39] -> [DMESG-WARN][40] ([i915#1982])
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/bat-jsl-3/igt@kms_pipe_crc_basic@suspend-read-crc@pipe-a-edp-1.html
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-jsl-3/igt@kms_pipe_crc_basic@suspend-read-crc@pipe-a-edp-1.html
    - bat-jsl-1:          [PASS][41] -> [DMESG-WARN][42] ([i915#1982])
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/bat-jsl-1/igt@kms_pipe_crc_basic@suspend-read-crc@pipe-a-edp-1.html
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-jsl-1/igt@kms_pipe_crc_basic@suspend-read-crc@pipe-a-edp-1.html

  * igt@kms_pm_backlight@basic-brightness:
    - bat-rpls-2:         NOTRUN -> [SKIP][43] ([i915#5354])
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-rpls-2/igt@kms_pm_backlight@basic-brightness.html

  * igt@kms_pm_backlight@basic-brightness@edp-1:
    - bat-rplp-1:         NOTRUN -> [ABORT][44] ([i915#8668])
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-rplp-1/igt@kms_pm_backlight@basic-brightness@edp-1.html

  * igt@kms_pm_rpm@basic-rte:
    - bat-rpls-2:         NOTRUN -> [ABORT][45] ([i915#8668] / [i915#9368] / [i915#9897])
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-rpls-2/igt@kms_pm_rpm@basic-rte.html

  
#### Possible fixes ####

  * igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1:
    - bat-rplp-1:         [ABORT][46] ([i915#8668]) -> [PASS][47]
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_14078/bat-rplp-1/igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1.html
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/bat-rplp-1/igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
  [i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
  [i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3840]: https://gitlab.freedesktop.org/drm/intel/issues/3840
  [i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103
  [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
  [i915#5354]: https://gitlab.freedesktop.org/drm/intel/issues/5354
  [i915#8668]: https://gitlab.freedesktop.org/drm/intel/issues/8668
  [i915#9318]: https://gitlab.freedesktop.org/drm/intel/issues/9318
  [i915#9368]: https://gitlab.freedesktop.org/drm/intel/issues/9368
  [i915#9875]: https://gitlab.freedesktop.org/drm/intel/issues/9875
  [i915#9886]: https://gitlab.freedesktop.org/drm/intel/issues/9886
  [i915#9897]: https://gitlab.freedesktop.org/drm/intel/issues/9897
  [i915#9900]: https://gitlab.freedesktop.org/drm/intel/issues/9900


Build changes
-------------

  * Linux: CI_DRM_14078 -> Patchwork_128202v1

  CI-20190529: 20190529
  CI_DRM_14078: 1baf990bc673f31d9eba7dfcb597ac0cb7420b14 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGT_7655: ddf7cf40a00caa7d02f3729e1e50f78f102463d9 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  Patchwork_128202v1: 1baf990bc673f31d9eba7dfcb597ac0cb7420b14 @ git://anongit.freedesktop.org/gfx-ci/linux


### Linux commits

0794bcb36b44 drm/i915/gt: Disable HW load balancing for CCS
1770c3f0b128 drm/i915/gt: Allow user to set up the CSS mode
219c4039bc14 drm/i915/gt: Support fixed CCS mode

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_128202v1/index.html

[-- Attachment #2: Type: text/html, Size: 12779 bytes --]

Re: [PATCH v2 1/3] drm/i915/gt: Support fixed CCS mode

[Andi Shyti]

Hi Tvrtko,

[1]

> > +	/*
> > +	 * Loop over all available slices and assign each a user engine.
> > +	 *
> > +	 * With 1 engine (ccs0):
> > +	 *   slice 0, 1, 2, 3: ccs0
> > +	 *
> > +	 * With 2 engines (ccs0, ccs1):
> > +	 *   slice 0, 2: ccs0
> > +	 *   slice 1, 3: ccs1
> > +	 *
> > +	 * With 4 engines (ccs0, ccs1, ccs2, ccs3):
> > +	 *   slice 0: ccs0
> > +	 *   slice 1: ccs1
> > +	 *   slice 2: ccs2
> > +	 *   slice 3: ccs3
> > +	 *
> > +	 * Since the number of slices and the number of engines is
> > +	 * known, and we ensure that there is an exact multiple of
> > +	 * engines for slices, the double loop becomes a loop over each
> > +	 * slice.
> > +	 */
> > +	for (i = num_slices / num_engines; i < num_slices; i++) {
> > +		struct intel_engine_cs *engine;
> > +		intel_engine_mask_t tmp;
> > +
> > +		for_each_engine_masked(engine, gt, ALL_CCS(gt), tmp) {
> > +			/* If a slice is fused off, leave disabled */
> > +			while (!(CCS_MASK(gt) & BIT(slice)))
> > +				slice++;
> > +
> > +			mode &= ~XEHP_CCS_MODE_CSLICE(slice, XEHP_CCS_MODE_CSLICE_MASK);
> > +			mode |= XEHP_CCS_MODE_CSLICE(slice, engine->instance);
> > +
> > +			/* assign the next slice */
> > +			slice++;
> > +		}
> > +	}
> > +
> > +	intel_uncore_write(gt->uncore, XEHP_CCS_MODE, mode);
> > +}
> > +
> > +void intel_gt_apply_ccs_mode(struct intel_gt *gt)
> > +{
> > +	mutex_lock(&gt->ccs.mutex);
> > +	__intel_gt_apply_ccs_mode(gt);
> > +	mutex_unlock(&gt->ccs.mutex);
> > +}
> > +
> > +void intel_gt_init_ccs_mode(struct intel_gt *gt)
> > +{
> > +	mutex_init(&gt->ccs.mutex);
> > +	gt->ccs.mode = 1;
> 
> What is '1'? And this question carries over to the sysfs interface in the
> following patch - who will use it and where it is documented how to use it?

The value '1' is explained in the comment above[1] and in the
comment below[2]. Maybe we should give it an enum meaning? But
that would be something like CCS_MODE_1/2/4, I thinks
ccs.mode = 1/2/4 is more understandable.

> Also, should this setting somehow be gated by an applicable platform? Or if
> not on setting then when acting on it in __intel_gt_apply_ccs_mode?
> 
> Creation of sysfs files as well should be gated by platform too in the
> following patch?

The idea of this series is to disable the CCS load balancing
(which automatically chooses between mode 1/2/4) and used the
a fixed scheme chosen by the user.

(I'm preparing v3 as Chris was so kind to recommend some changes
offline)

Thanks,
Andi

[2]

> > +	/*
> > +	 * Track fixed mapping between CCS engines and compute slices.
> > +	 *
> > +	 * In order to w/a HW that has the inability to dynamically load
> > +	 * balance between CCS engines and EU in the compute slices, we have to
> > +	 * reconfigure a static mapping on the fly. We track the current CCS
> > +	 * configuration (set by thr user through a sysfs interface) and compare
> > +	 * it against the current CCS_MODE (which maps CCS engines to compute
> > +	 * slices). If there is only a single engine selected, we can map it to
> > +	 * all available compute slices for maximal single task performance
> > +	 * (fast/narrow). If there are more then one engine selected, we have to
> > +	 * reduce the number of slices allocated to each engine (wide/slow),
> > +	 * fairly distributing the EU between the equivalent engines.
> > +	 */
> > +	struct {
> > +		struct mutex mutex;
> > +		u32 mode;
> > +	} ccs;

Re: [PATCH v2 1/3] drm/i915/gt: Support fixed CCS mode

[Andi Shyti]

Hi Tvrtko,

[1]

> > +	/*
> > +	 * Loop over all available slices and assign each a user engine.
> > +	 *
> > +	 * With 1 engine (ccs0):
> > +	 *   slice 0, 1, 2, 3: ccs0
> > +	 *
> > +	 * With 2 engines (ccs0, ccs1):
> > +	 *   slice 0, 2: ccs0
> > +	 *   slice 1, 3: ccs1
> > +	 *
> > +	 * With 4 engines (ccs0, ccs1, ccs2, ccs3):
> > +	 *   slice 0: ccs0
> > +	 *   slice 1: ccs1
> > +	 *   slice 2: ccs2
> > +	 *   slice 3: ccs3
> > +	 *
> > +	 * Since the number of slices and the number of engines is
> > +	 * known, and we ensure that there is an exact multiple of
> > +	 * engines for slices, the double loop becomes a loop over each
> > +	 * slice.
> > +	 */
> > +	for (i = num_slices / num_engines; i < num_slices; i++) {
> > +		struct intel_engine_cs *engine;
> > +		intel_engine_mask_t tmp;
> > +
> > +		for_each_engine_masked(engine, gt, ALL_CCS(gt), tmp) {
> > +			/* If a slice is fused off, leave disabled */
> > +			while (!(CCS_MASK(gt) & BIT(slice)))
> > +				slice++;
> > +
> > +			mode &= ~XEHP_CCS_MODE_CSLICE(slice, XEHP_CCS_MODE_CSLICE_MASK);
> > +			mode |= XEHP_CCS_MODE_CSLICE(slice, engine->instance);
> > +
> > +			/* assign the next slice */
> > +			slice++;
> > +		}
> > +	}
> > +
> > +	intel_uncore_write(gt->uncore, XEHP_CCS_MODE, mode);
> > +}
> > +
> > +void intel_gt_apply_ccs_mode(struct intel_gt *gt)
> > +{
> > +	mutex_lock(&gt->ccs.mutex);
> > +	__intel_gt_apply_ccs_mode(gt);
> > +	mutex_unlock(&gt->ccs.mutex);
> > +}
> > +
> > +void intel_gt_init_ccs_mode(struct intel_gt *gt)
> > +{
> > +	mutex_init(&gt->ccs.mutex);
> > +	gt->ccs.mode = 1;
> 
> What is '1'? And this question carries over to the sysfs interface in the
> following patch - who will use it and where it is documented how to use it?

The value '1' is explained in the comment above[1] and in the
comment below[2]. Maybe we should give it an enum meaning? But
that would be something like CCS_MODE_1/2/4, I thinks
ccs.mode = 1/2/4 is more understandable.

> Also, should this setting somehow be gated by an applicable platform? Or if
> not on setting then when acting on it in __intel_gt_apply_ccs_mode?
> 
> Creation of sysfs files as well should be gated by platform too in the
> following patch?

The idea of this series is to disable the CCS load balancing
(which automatically chooses between mode 1/2/4) and used the
a fixed scheme chosen by the user.

(I'm preparing v3 as Chris was so kind to recommend some changes
offline)

Thanks,
Andi

[2]

> > +	/*
> > +	 * Track fixed mapping between CCS engines and compute slices.
> > +	 *
> > +	 * In order to w/a HW that has the inability to dynamically load
> > +	 * balance between CCS engines and EU in the compute slices, we have to
> > +	 * reconfigure a static mapping on the fly. We track the current CCS
> > +	 * configuration (set by thr user through a sysfs interface) and compare
> > +	 * it against the current CCS_MODE (which maps CCS engines to compute
> > +	 * slices). If there is only a single engine selected, we can map it to
> > +	 * all available compute slices for maximal single task performance
> > +	 * (fast/narrow). If there are more then one engine selected, we have to
> > +	 * reduce the number of slices allocated to each engine (wide/slow),
> > +	 * fairly distributing the EU between the equivalent engines.
> > +	 */
> > +	struct {
> > +		struct mutex mutex;
> > +		u32 mode;
> > +	} ccs;

Re: [PATCH v2 1/3] drm/i915/gt: Support fixed CCS mode

[Tvrtko Ursulin]

On 04/01/2024 21:23, Andi Shyti wrote:
> Hi Tvrtko,
> 
> [1]
> 
>>> +	/*
>>> +	 * Loop over all available slices and assign each a user engine.
>>> +	 *
>>> +	 * With 1 engine (ccs0):
>>> +	 *   slice 0, 1, 2, 3: ccs0
>>> +	 *
>>> +	 * With 2 engines (ccs0, ccs1):
>>> +	 *   slice 0, 2: ccs0
>>> +	 *   slice 1, 3: ccs1
>>> +	 *
>>> +	 * With 4 engines (ccs0, ccs1, ccs2, ccs3):
>>> +	 *   slice 0: ccs0
>>> +	 *   slice 1: ccs1
>>> +	 *   slice 2: ccs2
>>> +	 *   slice 3: ccs3
>>> +	 *
>>> +	 * Since the number of slices and the number of engines is
>>> +	 * known, and we ensure that there is an exact multiple of
>>> +	 * engines for slices, the double loop becomes a loop over each
>>> +	 * slice.
>>> +	 */
>>> +	for (i = num_slices / num_engines; i < num_slices; i++) {
>>> +		struct intel_engine_cs *engine;
>>> +		intel_engine_mask_t tmp;
>>> +
>>> +		for_each_engine_masked(engine, gt, ALL_CCS(gt), tmp) {
>>> +			/* If a slice is fused off, leave disabled */
>>> +			while (!(CCS_MASK(gt) & BIT(slice)))
>>> +				slice++;
>>> +
>>> +			mode &= ~XEHP_CCS_MODE_CSLICE(slice, XEHP_CCS_MODE_CSLICE_MASK);
>>> +			mode |= XEHP_CCS_MODE_CSLICE(slice, engine->instance);
>>> +
>>> +			/* assign the next slice */
>>> +			slice++;
>>> +		}
>>> +	}
>>> +
>>> +	intel_uncore_write(gt->uncore, XEHP_CCS_MODE, mode);
>>> +}
>>> +
>>> +void intel_gt_apply_ccs_mode(struct intel_gt *gt)
>>> +{
>>> +	mutex_lock(&gt->ccs.mutex);
>>> +	__intel_gt_apply_ccs_mode(gt);
>>> +	mutex_unlock(&gt->ccs.mutex);
>>> +}
>>> +
>>> +void intel_gt_init_ccs_mode(struct intel_gt *gt)
>>> +{
>>> +	mutex_init(&gt->ccs.mutex);
>>> +	gt->ccs.mode = 1;
>>
>> What is '1'? And this question carries over to the sysfs interface in the
>> following patch - who will use it and where it is documented how to use it?
> 
> The value '1' is explained in the comment above[1] and in the

Do you mean this is mode '1':

  * With 1 engine (ccs0):
  *   slice 0, 1, 2, 3: ccs0

?

But I don't see where it says what do different modes mean on different 
SKU configurations.

It also does not say what should the num_slices sysfs file be used for.

Does "mode N" mean "assign each command streamer N compute slices"? Or 
"assign each compute slice N command streamers"?

I wonder if we should add something user friendly into 
Documentation/ABI/*/sysfs-... Joonas your thoughts?

> comment below[2]. Maybe we should give it an enum meaning? But
> that would be something like CCS_MODE_1/2/4, I thinks
> ccs.mode = 1/2/4 is more understandable.
> 
>> Also, should this setting somehow be gated by an applicable platform? Or if
>> not on setting then when acting on it in __intel_gt_apply_ccs_mode?
>>
>> Creation of sysfs files as well should be gated by platform too in the
>> following patch?
> 
> The idea of this series is to disable the CCS load balancing
> (which automatically chooses between mode 1/2/4) and used the
> a fixed scheme chosen by the user.
> 
> (I'm preparing v3 as Chris was so kind to recommend some changes
> offline)

Okay lets wait for v2 and I will then see if I will this that will make 
it clearer to casual observers.

Regards,

Tvrtko


> Thanks,
> Andi
> 
> [2]
> 
>>> +	/*
>>> +	 * Track fixed mapping between CCS engines and compute slices.
>>> +	 *
>>> +	 * In order to w/a HW that has the inability to dynamically load
>>> +	 * balance between CCS engines and EU in the compute slices, we have to
>>> +	 * reconfigure a static mapping on the fly. We track the current CCS
>>> +	 * configuration (set by thr user through a sysfs interface) and compare
>>> +	 * it against the current CCS_MODE (which maps CCS engines to compute
>>> +	 * slices). If there is only a single engine selected, we can map it to
>>> +	 * all available compute slices for maximal single task performance
>>> +	 * (fast/narrow). If there are more then one engine selected, we have to
>>> +	 * reduce the number of slices allocated to each engine (wide/slow),
>>> +	 * fairly distributing the EU between the equivalent engines.
>>> +	 */
>>> +	struct {
>>> +		struct mutex mutex;
>>> +		u32 mode;
>>> +	} ccs;

Re: [PATCH v2 1/3] drm/i915/gt: Support fixed CCS mode

[Joonas Lahtinen]

Quoting Tvrtko Ursulin (2024-01-05 12:39:31)
> 
> On 04/01/2024 21:23, Andi Shyti wrote:

<SNIP>

> >>> +void intel_gt_apply_ccs_mode(struct intel_gt *gt)
> >>> +{
> >>> +   mutex_lock(&gt->ccs.mutex);
> >>> +   __intel_gt_apply_ccs_mode(gt);
> >>> +   mutex_unlock(&gt->ccs.mutex);
> >>> +}
> >>> +
> >>> +void intel_gt_init_ccs_mode(struct intel_gt *gt)
> >>> +{
> >>> +   mutex_init(&gt->ccs.mutex);
> >>> +   gt->ccs.mode = 1;
> >>
> >> What is '1'? And this question carries over to the sysfs interface in the
> >> following patch - who will use it and where it is documented how to use it?
> > 
> > The value '1' is explained in the comment above[1] and in the
> 
> Do you mean this is mode '1':
> 
>   * With 1 engine (ccs0):
>   *   slice 0, 1, 2, 3: ccs0
> 
> ?
> 
> But I don't see where it says what do different modes mean on different 
> SKU configurations.
> 
> It also does not say what should the num_slices sysfs file be used for.
> 
> Does "mode N" mean "assign each command streamer N compute slices"? Or 
> "assign each compute slice N command streamers"?
> 
> I wonder if we should add something user friendly into 
> Documentation/ABI/*/sysfs-... Joonas your thoughts?

We definitely should always properly document all sysfs additions, just
seems like we less frequently remember to do so. So yeah, this should be
documented just like other uAPI.

I also like the idea of not exposing the the file at all if the value
can't be modified.

The ccs_mode is just supposed to allow user to select how many CCS
engines they want to expose, and always make an even split of slices
between them, nothing more nothing less.

Regards, Joonas

Re: [PATCH v2 1/3] drm/i915/gt: Support fixed CCS mode

[Tvrtko Ursulin]

On 08/01/2024 15:13, Joonas Lahtinen wrote:
> Quoting Tvrtko Ursulin (2024-01-05 12:39:31)
>>
>> On 04/01/2024 21:23, Andi Shyti wrote:
> 
> <SNIP>
> 
>>>>> +void intel_gt_apply_ccs_mode(struct intel_gt *gt)
>>>>> +{
>>>>> +   mutex_lock(&gt->ccs.mutex);
>>>>> +   __intel_gt_apply_ccs_mode(gt);
>>>>> +   mutex_unlock(&gt->ccs.mutex);
>>>>> +}
>>>>> +
>>>>> +void intel_gt_init_ccs_mode(struct intel_gt *gt)
>>>>> +{
>>>>> +   mutex_init(&gt->ccs.mutex);
>>>>> +   gt->ccs.mode = 1;
>>>>
>>>> What is '1'? And this question carries over to the sysfs interface in the
>>>> following patch - who will use it and where it is documented how to use it?
>>>
>>> The value '1' is explained in the comment above[1] and in the
>>
>> Do you mean this is mode '1':
>>
>>    * With 1 engine (ccs0):
>>    *   slice 0, 1, 2, 3: ccs0
>>
>> ?
>>
>> But I don't see where it says what do different modes mean on different
>> SKU configurations.
>>
>> It also does not say what should the num_slices sysfs file be used for.
>>
>> Does "mode N" mean "assign each command streamer N compute slices"? Or
>> "assign each compute slice N command streamers"?
>>
>> I wonder if we should add something user friendly into
>> Documentation/ABI/*/sysfs-... Joonas your thoughts?
> 
> We definitely should always properly document all sysfs additions, just
> seems like we less frequently remember to do so. So yeah, this should be
> documented just like other uAPI.
> 
> I also like the idea of not exposing the the file at all if the value
> can't be modified.
> 
> The ccs_mode is just supposed to allow user to select how many CCS
> engines they want to expose, and always make an even split of slices
> between them, nothing more nothing less.

Hmm I can't see that the series changes anywhere what command streamers 
will get reported as available.

Regards,

Tvrtko