[RFC PATCH 00/15] drm/i915: initial support for huge gtt pages

[RFC PATCH 00/15] drm/i915: initial support for huge gtt pages

[Matthew Auld]

This series adds support for huge-pages for the gtt, where "huge"
is 64K, 2M and 1G. This isn't everything I have and there are still some
things which I have yet to implement, like handling evict-for-node with the
64K/4K trickiness, but the hope here is to get some early feedback if possible.

One open question I still have is how the page-size should be handled at the
gem object level, should the page-size be an implementation detail of whichever
backend the gem object uses, where the selected page-size would solely depend
on the size of the object and the availability of huge pages, or do we intend
to expose some kind of hinting, both within our driver and possibly to
userspace?

Patches 1-2 should be ready to go and 3 just needs an r-b from Chris, but the
rest should be considered RFC.

Thanks,
Matt

Matthew Auld (15):
  drm/i915/selftests: don't leak the gem object
  drm/i915: use correct node for handling cache domain eviction
  drm/i915/selftests: exercise cache domain eviction
  drm/i915: add page_size_mask to dev_info
  drm/i915: introduce drm_i915_gem_object page_size member
  drm/i915: pass page_size to insert_entries
  drm/i915: s/i915_gtt_color_adjust/i915_cache_color_adjust
  drm/i915: clean up cache coloring
  drm/i915: export color_differs
  drm/i915: introduce ppgtt page coloring
  drm/i915: support inserting 64K pages in the ppgtt
  drm/i915: support inserting 2M pages in the ppgtt
  drm/i915: support inserting 1G pages in the ppgtt
  drm/i915/selftests: exercise 4K and 64K mm insertion
  drm/i915/selftests: modify the gtt tests to also exercise huge pages

 drivers/gpu/drm/i915/i915_drv.h                 |   3 +-
 drivers/gpu/drm/i915/i915_gem.c                 |   5 +-
 drivers/gpu/drm/i915/i915_gem_evict.c           |  20 +-
 drivers/gpu/drm/i915/i915_gem_gtt.c             | 235 ++++++++++++++++++++++--
 drivers/gpu/drm/i915/i915_gem_gtt.h             |  27 +++
 drivers/gpu/drm/i915/i915_gem_object.h          |   2 +
 drivers/gpu/drm/i915/i915_pci.c                 |  23 ++-
 drivers/gpu/drm/i915/i915_vma.c                 |  21 ++-
 drivers/gpu/drm/i915/i915_vma.h                 |   6 +
 drivers/gpu/drm/i915/selftests/i915_gem_evict.c |  89 +++++++++
 drivers/gpu/drm/i915/selftests/i915_gem_gtt.c   | 203 ++++++++++++++++----
 drivers/gpu/drm/i915/selftests/mock_gtt.c       |   4 +
 12 files changed, 563 insertions(+), 75 deletions(-)

-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 01/15] drm/i915/selftests: don't leak the gem object

[Matthew Auld]

For our fake dma objects we can leak the underlying gem object if we
fail to pin our "backing storage".

[   39.952618] =============================================================================
[   39.952625] BUG mock_object (Tainted: G     U         ): Objects remaining in mock_object on __kmem_cache_shutdown()
[   39.952629] -----------------------------------------------------------------------------

[   39.952633] Disabling lock debugging due to kernel taint
[   39.952635] INFO: Slab 0xffffea00086c6a00 objects=21 used=1 fp=0xffff88021b1abc00 flags=0x5fff8000008100
[   39.952640] CPU: 1 PID: 1258 Comm: drv_selftest Tainted: G    BU          4.10.0+ #46
[   39.952641] Hardware name: Apple Inc. MacBookPro11,1/Mac-189A3D4F975D5FFC, BIOS MBP111.88Z.0138.B17.1602221718 02/22/2016
[   39.952642] Call Trace:
[   39.952648]  dump_stack+0x4d/0x6f
[   39.952651]  slab_err+0x9d/0xb0
[   39.952654]  ? ksm_migrate_page+0xe0/0xe0
[   39.952657]  ? on_each_cpu_cond+0x9a/0xc0
[   39.952658]  ? __kmalloc+0x1af/0x1c0
[   39.952660]  ? __kmem_cache_shutdown+0x173/0x3e0
[   39.952661]  __kmem_cache_shutdown+0x196/0x3e0
[   39.952664]  kmem_cache_destroy+0xa0/0x150
[   39.952708]  mock_device_release+0x113/0x140 [i915]
[   39.952726]  drm_dev_release+0x20/0x40 [drm]
[   39.952735]  drm_dev_unref+0x23/0x30 [drm]
[   39.952768]  i915_gem_gtt_mock_selftests+0x55/0x70 [i915]
[   39.952803]  __run_selftests+0x169/0x1c0 [i915]
[   39.952805]  ? 0xffffffffa0151000
[   39.952840]  i915_mock_selftests+0x30/0x60 [i915]
[   39.952869]  i915_init+0xc/0x78 [i915]
[   39.952870]  ? 0xffffffffa0151000
[   39.952872]  do_one_initcall+0x43/0x170
[   39.952874]  ? __vunmap+0x81/0xd0
[   39.952875]  ? kmem_cache_alloc_trace+0x37/0x170
[   39.952877]  ? do_init_module+0x27/0x1f8
[   39.952879]  do_init_module+0x5f/0x1f8
[   39.952881]  load_module+0x2423/0x29b0
[   39.952882]  ? __symbol_put+0x40/0x40
[   39.952885]  ? kernel_read_file+0x1a3/0x1c0
[   39.952887]  SYSC_finit_module+0xbc/0xf0
[   39.952889]  SyS_finit_module+0xe/0x10
[   39.952892]  entry_SYSCALL_64_fastpath+0x13/0x94

v2: use onion teardown and favour i915_gem_object_put

Fixes: 8d28ba4568f4 ("drm/i915: Exercise filling the top/bottom portions of the ppgtt")
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
---
 drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
index 0f3fa34377c6..c4a39b7d7dd0 100644
--- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
@@ -103,7 +103,7 @@ fake_dma_object(struct drm_i915_private *i915, u64 size)
 
 	obj = i915_gem_object_alloc(i915);
 	if (!obj)
-		return ERR_PTR(-ENOMEM);
+		goto err;
 
 	drm_gem_private_object_init(&i915->drm, &obj->base, size);
 	i915_gem_object_init(obj, &fake_ops);
@@ -114,10 +114,15 @@ fake_dma_object(struct drm_i915_private *i915, u64 size)
 
 	/* Preallocate the "backing storage" */
 	if (i915_gem_object_pin_pages(obj))
-		return ERR_PTR(-ENOMEM);
+		goto err_obj;
 
 	i915_gem_object_unpin_pages(obj);
 	return obj;
+
+err_obj:
+	i915_gem_object_put(obj);
+err:
+	return ERR_PTR(-ENOMEM);
 }
 
 static int igt_ppgtt_alloc(void *arg)
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 02/15] drm/i915: use correct node for handling cache domain eviction

[Matthew Auld]

It looks like we were incorrectly comparing vma->node against itself
instead of the target node, when evicting for a node on systems where we
need guard pages between regions with different cache domains. As a
consequence we can end up trying to needlessly evict neighbouring nodes,
even if they have the same cache domain, and if they were pinned we
would fail the eviction.

Fixes: 625d988acc28 ("drm/i915: Extract reserving space in the GTT to a helper")
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
---
 drivers/gpu/drm/i915/i915_gem_evict.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c
index a0de5734f7d0..2da3a94fc9f3 100644
--- a/drivers/gpu/drm/i915/i915_gem_evict.c
+++ b/drivers/gpu/drm/i915/i915_gem_evict.c
@@ -299,12 +299,12 @@ int i915_gem_evict_for_node(struct i915_address_space *vm,
 		 * those as well to make room for our guard pages.
 		 */
 		if (check_color) {
-			if (vma->node.start + vma->node.size == node->start) {
-				if (vma->node.color == node->color)
+			if (node->start + node->size == target->start) {
+				if (node->color == target->color)
 					continue;
 			}
-			if (vma->node.start == node->start + node->size) {
-				if (vma->node.color == node->color)
+			if (node->start == target->start + target->size) {
+				if (node->color == target->color)
 					continue;
 			}
 		}
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 03/15] drm/i915/selftests: exercise cache domain eviction

[Matthew Auld]

v2: add a peppering of comments

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
---
 drivers/gpu/drm/i915/selftests/i915_gem_evict.c | 89 +++++++++++++++++++++++++
 1 file changed, 89 insertions(+)

diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c
index 97af353db218..e2838fb428f3 100644
--- a/drivers/gpu/drm/i915/selftests/i915_gem_evict.c
+++ b/drivers/gpu/drm/i915/selftests/i915_gem_evict.c
@@ -202,6 +202,94 @@ static int igt_evict_for_vma(void *arg)
 	return err;
 }
 
+static void mock_color_adjust(const struct drm_mm_node *node,
+			      unsigned long color,
+			      u64 *start,
+			      u64 *end)
+{
+}
+
+static int igt_evict_for_cache_color(void *arg)
+{
+	struct drm_i915_private *i915 = arg;
+	struct i915_ggtt *ggtt = &i915->ggtt;
+	const unsigned long flags = PIN_OFFSET_FIXED;
+	struct drm_mm_node target = {
+		.start = I915_GTT_PAGE_SIZE * 2,
+		.size = I915_GTT_PAGE_SIZE,
+		.color = I915_CACHE_LLC,
+	};
+	struct drm_i915_gem_object *obj;
+	struct i915_vma *vma;
+	int err;
+
+	/* Currently the use of color_adjust is limited to cache domains within
+	 * the ggtt, and so the presence of mm.color_adjust is assumed to be
+	 * i915_gtt_color_adjust throughout our driver, so using a mock color
+	 * adjust will work just fine for our purposes.
+	 */
+	ggtt->base.mm.color_adjust = mock_color_adjust;
+
+	obj = i915_gem_object_create_internal(i915, I915_GTT_PAGE_SIZE);
+	if (IS_ERR(obj)) {
+		err = PTR_ERR(obj);
+		goto cleanup;
+	}
+	i915_gem_object_set_cache_level(obj, I915_CACHE_LLC);
+
+	vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0,
+				       I915_GTT_PAGE_SIZE | flags);
+	if (IS_ERR(vma)) {
+		pr_err("[0]i915_gem_object_ggtt_pin failed\n");
+		err = PTR_ERR(vma);
+		goto cleanup;
+	}
+
+	obj = i915_gem_object_create_internal(i915, I915_GTT_PAGE_SIZE);
+	if (IS_ERR(obj)) {
+		err = PTR_ERR(obj);
+		goto cleanup;
+	}
+	i915_gem_object_set_cache_level(obj, I915_CACHE_LLC);
+
+	/* Neighbouring; same colour - should fit */
+	vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0,
+				       (I915_GTT_PAGE_SIZE * 2) | flags);
+	if (IS_ERR(vma)) {
+		pr_err("[1]i915_gem_object_ggtt_pin failed\n");
+		err = PTR_ERR(vma);
+		goto cleanup;
+	}
+
+	i915_vma_unpin(vma);
+
+	/* Remove just the second vma */
+	err = i915_gem_evict_for_node(&ggtt->base, &target, 0);
+	if (err) {
+		pr_err("[0]i915_gem_evict_for_node returned err=%d\n", err);
+		goto cleanup;
+	}
+
+	/* Attempt to remove the first *pinned* vma, by removing the (empty)
+	 * neighbour */
+	target.color = I915_CACHE_L3_LLC;
+
+	err = i915_gem_evict_for_node(&ggtt->base, &target, 0);
+	if (!err) {
+		pr_err("[1]i915_gem_evict_for_node returned err=%d\n", err);
+		err = -EINVAL;
+		goto cleanup;
+	}
+
+	err = 0;
+
+cleanup:
+	unpin_ggtt(i915);
+	cleanup_objects(i915);
+	ggtt->base.mm.color_adjust = NULL;
+	return err;
+}
+
 static int igt_evict_vm(void *arg)
 {
 	struct drm_i915_private *i915 = arg;
@@ -241,6 +329,7 @@ int i915_gem_evict_mock_selftests(void)
 	static const struct i915_subtest tests[] = {
 		SUBTEST(igt_evict_something),
 		SUBTEST(igt_evict_for_vma),
+		SUBTEST(igt_evict_for_cache_color),
 		SUBTEST(igt_evict_vm),
 		SUBTEST(igt_overcommit),
 	};
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 04/15] drm/i915: add page_size_mask to dev_info

[Matthew Auld]

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/i915_drv.h     |  1 +
 drivers/gpu/drm/i915/i915_gem_gtt.h | 14 ++++++++++++++
 drivers/gpu/drm/i915/i915_pci.c     | 23 ++++++++++++++++++++++-
 3 files changed, 37 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 1fd4128a10b1..e45b8d74cebf 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -913,6 +913,7 @@ struct intel_device_info {
 	enum intel_platform platform;
 	u8 ring_mask; /* Rings supported by the HW */
 	u8 num_rings;
+	unsigned long page_size_mask; /* page sizes supported by the HW */
 #define DEFINE_FLAG(name) u8 name:1
 	DEV_INFO_FOR_EACH_FLAG(DEFINE_FLAG);
 #undef DEFINE_FLAG
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index fb15684c1d83..6c90a2ffd0e1 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -43,8 +43,18 @@
 #include "i915_selftest.h"
 
 #define I915_GTT_PAGE_SIZE 4096UL
+#define I915_GTT_PAGE_SIZE_64K 65536UL
+#define I915_GTT_PAGE_SIZE_2M 2097152UL
+#define I915_GTT_PAGE_SIZE_1G 1073741824UL
+
+#define I915_GTT_PAGE_SIZE_MASK (I915_GTT_PAGE_SIZE | \
+				 I915_GTT_PAGE_SIZE_64K | \
+				 I915_GTT_PAGE_SIZE_2M | \
+				 I915_GTT_PAGE_SIZE_1G)
+
 #define I915_GTT_MIN_ALIGNMENT I915_GTT_PAGE_SIZE
 
+
 #define I915_FENCE_REG_NONE -1
 #define I915_MAX_NUM_FENCES 32
 /* 32 fences + sign bit for FENCE_REG_NONE */
@@ -143,6 +153,10 @@ typedef u64 gen8_ppgtt_pml4e_t;
 #define GEN8_PPAT_ELLC_OVERRIDE		(0<<2)
 #define GEN8_PPAT(i, x)			((u64)(x) << ((i) * 8))
 
+#define GEN8_PDPE_PS_1G  BIT(7)
+#define GEN8_PDE_PS_2M   BIT(7)
+#define GEN8_PDE_IPS_64K BIT(11)
+
 struct sg_table;
 
 struct intel_rotation_info {
diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pci.c
index 732101ed57fb..5abb7d84b65a 100644
--- a/drivers/gpu/drm/i915/i915_pci.c
+++ b/drivers/gpu/drm/i915/i915_pci.c
@@ -56,6 +56,10 @@
 	.color = { .degamma_lut_size = 65, .gamma_lut_size = 257 }
 
 /* Keep in gen based order, and chronological order within a gen */
+
+#define GEN_DEFAULT_PAGE_SZ \
+	.page_size_mask = I915_GTT_PAGE_SIZE
+
 #define GEN2_FEATURES \
 	.gen = 2, .num_pipes = 1, \
 	.has_overlay = 1, .overlay_needs_physical = 1, \
@@ -63,6 +67,7 @@
 	.hws_needs_physical = 1, \
 	.ring_mask = RENDER_RING, \
 	GEN_DEFAULT_PIPEOFFSETS, \
+	GEN_DEFAULT_PAGE_SZ, \
 	CURSOR_OFFSETS
 
 static const struct intel_device_info intel_i830_info = {
@@ -95,6 +100,7 @@ static const struct intel_device_info intel_i865g_info = {
 	.has_gmch_display = 1, \
 	.ring_mask = RENDER_RING, \
 	GEN_DEFAULT_PIPEOFFSETS, \
+	GEN_DEFAULT_PAGE_SZ, \
 	CURSOR_OFFSETS
 
 static const struct intel_device_info intel_i915g_info = {
@@ -153,6 +159,7 @@ static const struct intel_device_info intel_pineview_info = {
 	.has_gmch_display = 1, \
 	.ring_mask = RENDER_RING, \
 	GEN_DEFAULT_PIPEOFFSETS, \
+	GEN_DEFAULT_PAGE_SZ, \
 	CURSOR_OFFSETS
 
 static const struct intel_device_info intel_i965g_info = {
@@ -193,6 +200,7 @@ static const struct intel_device_info intel_gm45_info = {
 	.has_gmbus_irq = 1, \
 	.ring_mask = RENDER_RING | BSD_RING, \
 	GEN_DEFAULT_PIPEOFFSETS, \
+	GEN_DEFAULT_PAGE_SZ, \
 	CURSOR_OFFSETS
 
 static const struct intel_device_info intel_ironlake_d_info = {
@@ -218,6 +226,7 @@ static const struct intel_device_info intel_ironlake_m_info = {
 	.has_hw_contexts = 1, \
 	.has_aliasing_ppgtt = 1, \
 	GEN_DEFAULT_PIPEOFFSETS, \
+	GEN_DEFAULT_PAGE_SZ, \
 	CURSOR_OFFSETS
 
 static const struct intel_device_info intel_sandybridge_d_info = {
@@ -244,6 +253,7 @@ static const struct intel_device_info intel_sandybridge_m_info = {
 	.has_aliasing_ppgtt = 1, \
 	.has_full_ppgtt = 1, \
 	GEN_DEFAULT_PIPEOFFSETS, \
+	GEN_DEFAULT_PAGE_SZ, \
 	IVB_CURSOR_OFFSETS
 
 static const struct intel_device_info intel_ivybridge_d_info = {
@@ -282,6 +292,7 @@ static const struct intel_device_info intel_valleyview_info = {
 	.has_full_ppgtt = 1,
 	.ring_mask = RENDER_RING | BSD_RING | BLT_RING,
 	.display_mmio_offset = VLV_DISPLAY_BASE,
+	GEN_DEFAULT_PAGE_SZ,
 	GEN_DEFAULT_PIPEOFFSETS,
 	CURSOR_OFFSETS
 };
@@ -308,7 +319,8 @@ static const struct intel_device_info intel_haswell_info = {
 	BDW_COLORS, \
 	.has_logical_ring_contexts = 1, \
 	.has_full_48bit_ppgtt = 1, \
-	.has_64bit_reloc = 1
+	.has_64bit_reloc = 1, \
+	.page_size_mask = I915_GTT_PAGE_SIZE | I915_GTT_PAGE_SIZE_2M | I915_GTT_PAGE_SIZE_1G
 
 static const struct intel_device_info intel_broadwell_info = {
 	BDW_FEATURES,
@@ -341,13 +353,18 @@ static const struct intel_device_info intel_cherryview_info = {
 	.has_aliasing_ppgtt = 1,
 	.has_full_ppgtt = 1,
 	.display_mmio_offset = VLV_DISPLAY_BASE,
+	.page_size_mask = I915_GTT_PAGE_SIZE | I915_GTT_PAGE_SIZE_64K | I915_GTT_PAGE_SIZE_2M | I915_GTT_PAGE_SIZE_1G,
 	GEN_CHV_PIPEOFFSETS,
 	CURSOR_OFFSETS,
 	CHV_COLORS,
 };
 
+#define GEN9_DEFAULT_PAGE_SZ \
+	.page_size_mask = I915_GTT_PAGE_SIZE | I915_GTT_PAGE_SIZE_64K | I915_GTT_PAGE_SIZE_2M | I915_GTT_PAGE_SIZE_1G
+
 static const struct intel_device_info intel_skylake_info = {
 	BDW_FEATURES,
+	GEN9_DEFAULT_PAGE_SZ,
 	.platform = INTEL_SKYLAKE,
 	.gen = 9,
 	.has_csr = 1,
@@ -357,6 +374,7 @@ static const struct intel_device_info intel_skylake_info = {
 
 static const struct intel_device_info intel_skylake_gt3_info = {
 	BDW_FEATURES,
+	GEN9_DEFAULT_PAGE_SZ,
 	.platform = INTEL_SKYLAKE,
 	.gen = 9,
 	.has_csr = 1,
@@ -389,6 +407,7 @@ static const struct intel_device_info intel_skylake_gt3_info = {
 	.has_aliasing_ppgtt = 1, \
 	.has_full_ppgtt = 1, \
 	.has_full_48bit_ppgtt = 1, \
+	GEN9_DEFAULT_PAGE_SZ, \
 	GEN_DEFAULT_PIPEOFFSETS, \
 	IVB_CURSOR_OFFSETS, \
 	BDW_COLORS
@@ -409,6 +428,7 @@ static const struct intel_device_info intel_geminilake_info = {
 
 static const struct intel_device_info intel_kabylake_info = {
 	BDW_FEATURES,
+	GEN9_DEFAULT_PAGE_SZ,
 	.platform = INTEL_KABYLAKE,
 	.gen = 9,
 	.has_csr = 1,
@@ -418,6 +438,7 @@ static const struct intel_device_info intel_kabylake_info = {
 
 static const struct intel_device_info intel_kabylake_gt3_info = {
 	BDW_FEATURES,
+	GEN9_DEFAULT_PAGE_SZ,
 	.platform = INTEL_KABYLAKE,
 	.gen = 9,
 	.has_csr = 1,
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 05/15] drm/i915: introduce drm_i915_gem_object page_size member

[Matthew Auld]

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/i915_gem.c        | 2 ++
 drivers/gpu/drm/i915/i915_gem_object.h | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 7c20601fe1de..0a6ed2c54629 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3975,6 +3975,8 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
 
 	obj->ops = ops;
 
+	obj->page_size = I915_GTT_PAGE_SIZE;
+
 	reservation_object_init(&obj->__builtin_resv);
 	obj->resv = &obj->__builtin_resv;
 
diff --git a/drivers/gpu/drm/i915/i915_gem_object.h b/drivers/gpu/drm/i915/i915_gem_object.h
index 33b0dc4782a9..31bcf87d3745 100644
--- a/drivers/gpu/drm/i915/i915_gem_object.h
+++ b/drivers/gpu/drm/i915/i915_gem_object.h
@@ -90,6 +90,8 @@ struct drm_i915_gem_object {
 
 	unsigned long flags;
 
+	unsigned long page_size; /* 4K(default), 64K, 2M, 1G */
+
 	/**
 	 * Have we taken a reference for the object for incomplete GPU
 	 * activity?
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 06/15] drm/i915: pass page_size to insert_entries

[Matthew Auld]

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/i915_gem_gtt.c           | 32 ++++++++++++++++++++++-----
 drivers/gpu/drm/i915/i915_gem_gtt.h           |  1 +
 drivers/gpu/drm/i915/selftests/i915_gem_gtt.c |  3 ++-
 drivers/gpu/drm/i915/selftests/mock_gtt.c     |  1 +
 4 files changed, 30 insertions(+), 7 deletions(-)

diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index cee9c4fec52a..9274974956b3 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -205,7 +205,7 @@ static int ppgtt_bind_vma(struct i915_vma *vma,
 		pte_flags |= PTE_READ_ONLY;
 
 	vma->vm->insert_entries(vma->vm, vma->pages, vma->node.start,
-				cache_level, pte_flags);
+				vma->obj->page_size, cache_level, pte_flags);
 
 	return 0;
 }
@@ -906,6 +906,7 @@ gen8_ppgtt_insert_pte_entries(struct i915_hw_ppgtt *ppgtt,
 static void gen8_ppgtt_insert_3lvl(struct i915_address_space *vm,
 				   struct sg_table *pages,
 				   u64 start,
+				   unsigned long page_size,
 				   enum i915_cache_level cache_level,
 				   u32 unused)
 {
@@ -924,6 +925,7 @@ static void gen8_ppgtt_insert_3lvl(struct i915_address_space *vm,
 static void gen8_ppgtt_insert_4lvl(struct i915_address_space *vm,
 				   struct sg_table *pages,
 				   u64 start,
+				   unsigned long page_size,
 				   enum i915_cache_level cache_level,
 				   u32 unused)
 {
@@ -935,9 +937,23 @@ static void gen8_ppgtt_insert_4lvl(struct i915_address_space *vm,
 	};
 	struct i915_page_directory_pointer **pdps = ppgtt->pml4.pdps;
 	struct gen8_insert_pte idx = gen8_insert_pte(start);
+	bool (*insert_entries)(struct i915_hw_ppgtt *ppgtt,
+			       struct i915_page_directory_pointer *pdp,
+			       struct sgt_dma *iter,
+			       struct gen8_insert_pte *idx,
+			       enum i915_cache_level cache_level);
+
+	switch (page_size) {
+	case I915_GTT_PAGE_SIZE:
+		insert_entries = gen8_ppgtt_insert_pte_entries;
+		break;
+	default:
+		MISSING_CASE(page_size);
+		return;
+	} 
 
-	while (gen8_ppgtt_insert_pte_entries(ppgtt, pdps[idx.pml4e++], &iter,
-					     &idx, cache_level))
+	while (insert_entries(ppgtt, pdps[idx.pml4e++], &iter, &idx,
+			      cache_level))
 		GEM_BUG_ON(idx.pml4e >= GEN8_PML4ES_PER_PML4);
 }
 
@@ -1620,6 +1636,7 @@ static void gen6_ppgtt_clear_range(struct i915_address_space *vm,
 static void gen6_ppgtt_insert_entries(struct i915_address_space *vm,
 				      struct sg_table *pages,
 				      u64 start,
+				      unsigned long page_size,
 				      enum i915_cache_level cache_level,
 				      u32 flags)
 {
@@ -2093,6 +2110,7 @@ static void gen8_ggtt_insert_page(struct i915_address_space *vm,
 static void gen8_ggtt_insert_entries(struct i915_address_space *vm,
 				     struct sg_table *st,
 				     u64 start,
+				     unsigned long page_size,
 				     enum i915_cache_level level,
 				     u32 unused)
 {
@@ -2140,6 +2158,7 @@ static void gen6_ggtt_insert_page(struct i915_address_space *vm,
 static void gen6_ggtt_insert_entries(struct i915_address_space *vm,
 				     struct sg_table *st,
 				     u64 start,
+				     unsigned long page_size,
 				     enum i915_cache_level level,
 				     u32 flags)
 {
@@ -2224,6 +2243,7 @@ static void i915_ggtt_insert_page(struct i915_address_space *vm,
 static void i915_ggtt_insert_entries(struct i915_address_space *vm,
 				     struct sg_table *pages,
 				     u64 start,
+				     unsigned long page_size,
 				     enum i915_cache_level cache_level,
 				     u32 unused)
 {
@@ -2260,7 +2280,7 @@ static int ggtt_bind_vma(struct i915_vma *vma,
 
 	intel_runtime_pm_get(i915);
 	vma->vm->insert_entries(vma->vm, vma->pages, vma->node.start,
-				cache_level, pte_flags);
+				I915_GTT_PAGE_SIZE, cache_level, pte_flags);
 	intel_runtime_pm_put(i915);
 
 	/*
@@ -2314,14 +2334,14 @@ static int aliasing_gtt_bind_vma(struct i915_vma *vma,
 
 		appgtt->base.insert_entries(&appgtt->base,
 					    vma->pages, vma->node.start,
-					    cache_level, pte_flags);
+					    I915_GTT_PAGE_SIZE, cache_level, pte_flags);
 	}
 
 	if (flags & I915_VMA_GLOBAL_BIND) {
 		intel_runtime_pm_get(i915);
 		vma->vm->insert_entries(vma->vm,
 					vma->pages, vma->node.start,
-					cache_level, pte_flags);
+					I915_GTT_PAGE_SIZE, cache_level, pte_flags);
 		intel_runtime_pm_put(i915);
 	}
 
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index 6c90a2ffd0e1..1f51402cf816 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -328,6 +328,7 @@ struct i915_address_space {
 	void (*insert_entries)(struct i915_address_space *vm,
 			       struct sg_table *st,
 			       u64 start,
+			       unsigned long page_size,
 			       enum i915_cache_level cache_level,
 			       u32 flags);
 	void (*cleanup)(struct i915_address_space *vm);
diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
index c4a39b7d7dd0..c7963efe46ba 100644
--- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
@@ -256,7 +256,8 @@ static int lowlevel_hole(struct drm_i915_private *i915,
 				break;
 
 			vm->insert_entries(vm, obj->mm.pages, addr,
-					   I915_CACHE_NONE, 0);
+					   I915_GTT_PAGE_SIZE, I915_CACHE_NONE,
+					   0);
 		}
 		count = n;
 
diff --git a/drivers/gpu/drm/i915/selftests/mock_gtt.c b/drivers/gpu/drm/i915/selftests/mock_gtt.c
index a61309c7cb3e..a2137100d2f5 100644
--- a/drivers/gpu/drm/i915/selftests/mock_gtt.c
+++ b/drivers/gpu/drm/i915/selftests/mock_gtt.c
@@ -35,6 +35,7 @@ static void mock_insert_page(struct i915_address_space *vm,
 static void mock_insert_entries(struct i915_address_space *vm,
 				struct sg_table *st,
 				u64 start,
+				unsigned long page_size,
 				enum i915_cache_level level, u32 flags)
 {
 }
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 07/15] drm/i915: s/i915_gtt_color_adjust/i915_cache_color_adjust

[Matthew Auld]

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/i915_gem_gtt.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 9274974956b3..6b70332cafac 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -2394,10 +2394,10 @@ void i915_gem_gtt_finish_pages(struct drm_i915_gem_object *obj,
 	dma_unmap_sg(kdev, pages->sgl, pages->nents, PCI_DMA_BIDIRECTIONAL);
 }
 
-static void i915_gtt_color_adjust(const struct drm_mm_node *node,
-				  unsigned long color,
-				  u64 *start,
-				  u64 *end)
+static void i915_cache_color_adjust(const struct drm_mm_node *node,
+				    unsigned long color,
+				    u64 *start,
+				    u64 *end)
 {
 	if (node->allocated && node->color != color)
 		*start += I915_GTT_PAGE_SIZE;
@@ -2969,7 +2969,7 @@ int i915_ggtt_init_hw(struct drm_i915_private *dev_priv)
 	mutex_lock(&dev_priv->drm.struct_mutex);
 	i915_address_space_init(&ggtt->base, dev_priv, "[global]");
 	if (!HAS_LLC(dev_priv) && !USES_PPGTT(dev_priv))
-		ggtt->base.mm.color_adjust = i915_gtt_color_adjust;
+		ggtt->base.mm.color_adjust = i915_cache_color_adjust;
 	mutex_unlock(&dev_priv->drm.struct_mutex);
 
 	if (!io_mapping_init_wc(&dev_priv->ggtt.mappable,
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 08/15] drm/i915: clean up cache coloring

[Matthew Auld]

To keep the next patch simple, rid the code of any mm.color_adjust
assumptions to allow adding another flavour of coloring.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/i915_drv.h       |  2 +-
 drivers/gpu/drm/i915/i915_gem.c       |  3 ++-
 drivers/gpu/drm/i915/i915_gem_evict.c | 12 +++++-------
 drivers/gpu/drm/i915/i915_gem_gtt.h   |  6 ++++++
 drivers/gpu/drm/i915/i915_vma.c       | 10 +++++++---
 5 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index e45b8d74cebf..aac764b5aad4 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3526,7 +3526,7 @@ int i915_perf_open_ioctl(struct drm_device *dev, void *data,
 /* i915_gem_evict.c */
 int __must_check i915_gem_evict_something(struct i915_address_space *vm,
 					  u64 min_size, u64 alignment,
-					  unsigned cache_level,
+					  unsigned long color,
 					  u64 start, u64 end,
 					  unsigned flags);
 int __must_check i915_gem_evict_for_node(struct i915_address_space *vm,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 0a6ed2c54629..9acf279e5f93 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3392,7 +3392,8 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
 		obj->cache_dirty = true;
 
 	list_for_each_entry(vma, &obj->vma_list, obj_link)
-		vma->node.color = cache_level;
+		if (i915_uses_cache_coloring(vma->vm))
+			vma->node.color = cache_level;
 	obj->cache_level = cache_level;
 
 	return 0;
diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c
index 2da3a94fc9f3..f9364f917b67 100644
--- a/drivers/gpu/drm/i915/i915_gem_evict.c
+++ b/drivers/gpu/drm/i915/i915_gem_evict.c
@@ -74,7 +74,7 @@ mark_free(struct drm_mm_scan *scan,
  * @vm: address space to evict from
  * @min_size: size of the desired free space
  * @alignment: alignment constraint of the desired free space
- * @cache_level: cache_level for the desired space
+ * @color: color for the desired space
  * @start: start (inclusive) of the range from which to evict objects
  * @end: end (exclusive) of the range from which to evict objects
  * @flags: additional flags to control the eviction algorithm
@@ -95,7 +95,7 @@ mark_free(struct drm_mm_scan *scan,
 int
 i915_gem_evict_something(struct i915_address_space *vm,
 			 u64 min_size, u64 alignment,
-			 unsigned cache_level,
+			 unsigned long color,
 			 u64 start, u64 end,
 			 unsigned flags)
 {
@@ -134,7 +134,7 @@ i915_gem_evict_something(struct i915_address_space *vm,
 	if (flags & PIN_MAPPABLE)
 		mode = DRM_MM_INSERT_LOW;
 	drm_mm_scan_init_with_range(&scan, &vm->mm,
-				    min_size, alignment, cache_level,
+				    min_size, alignment, color,
 				    start, end, mode);
 
 	/* Retire before we search the active list. Although we have
@@ -254,7 +254,6 @@ int i915_gem_evict_for_node(struct i915_address_space *vm,
 	u64 start = target->start;
 	u64 end = start + target->size;
 	struct i915_vma *vma, *next;
-	bool check_color;
 	int ret = 0;
 
 	lockdep_assert_held(&vm->i915->drm.struct_mutex);
@@ -271,8 +270,7 @@ int i915_gem_evict_for_node(struct i915_address_space *vm,
 	if (!(flags & PIN_NONBLOCK))
 		i915_gem_retire_requests(vm->i915);
 
-	check_color = vm->mm.color_adjust;
-	if (check_color) {
+	if (i915_uses_cache_coloring(vm)) {
 		/* Expand search to cover neighbouring guard pages (or lack!) */
 		if (start)
 			start -= I915_GTT_PAGE_SIZE;
@@ -298,7 +296,7 @@ int i915_gem_evict_for_node(struct i915_address_space *vm,
 		 * abutt and conflict. If they are in conflict, then we evict
 		 * those as well to make room for our guard pages.
 		 */
-		if (check_color) {
+		if (i915_uses_cache_coloring(vm)) {
 			if (node->start + node->size == target->start) {
 				if (node->color == target->color)
 					continue;
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index 1f51402cf816..8d7436105718 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -346,6 +346,12 @@ struct i915_address_space {
 #define i915_is_ggtt(V) (!(V)->file)
 
 static inline bool
+i915_uses_cache_coloring(const struct i915_address_space *vm)
+{
+	return vm->mm.color_adjust && i915_is_ggtt(vm);
+}
+
+static inline bool
 i915_vm_is_48bit(const struct i915_address_space *vm)
 {
 	return (vm->total - 1) >> 32;
diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
index 1aba47024656..31e2327492ba 100644
--- a/drivers/gpu/drm/i915/i915_vma.c
+++ b/drivers/gpu/drm/i915/i915_vma.c
@@ -390,7 +390,7 @@ bool i915_gem_valid_gtt_space(struct i915_vma *vma, unsigned long cache_level)
 	 * these constraints apply and set the drm_mm.color_adjust
 	 * appropriately.
 	 */
-	if (vma->vm->mm.color_adjust == NULL)
+	if (!i915_uses_cache_coloring(vma->vm))
 		return true;
 
 	/* Only valid to be called on an already inserted vma */
@@ -429,6 +429,7 @@ i915_vma_insert(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
 	struct drm_i915_gem_object *obj = vma->obj;
 	u64 start, end;
 	int ret;
+	unsigned long color = 0;
 
 	GEM_BUG_ON(vma->flags & (I915_VMA_GLOBAL_BIND | I915_VMA_LOCAL_BIND));
 	GEM_BUG_ON(drm_mm_node_allocated(&vma->node));
@@ -471,6 +472,9 @@ i915_vma_insert(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
 	if (ret)
 		return ret;
 
+	if (i915_uses_cache_coloring(vma->vm))
+		color = obj->cache_level;
+
 	if (flags & PIN_OFFSET_FIXED) {
 		u64 offset = flags & PIN_OFFSET_MASK;
 		if (!IS_ALIGNED(offset, alignment) ||
@@ -480,13 +484,13 @@ i915_vma_insert(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
 		}
 
 		ret = i915_gem_gtt_reserve(vma->vm, &vma->node,
-					   size, offset, obj->cache_level,
+					   size, offset, color,
 					   flags);
 		if (ret)
 			goto err_unpin;
 	} else {
 		ret = i915_gem_gtt_insert(vma->vm, &vma->node,
-					  size, alignment, obj->cache_level,
+					  size, alignment, color,
 					  start, end, flags);
 		if (ret)
 			goto err_unpin;
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 09/15] drm/i915: export color_differs

[Matthew Auld]

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/i915_vma.c | 9 ++-------
 drivers/gpu/drm/i915/i915_vma.h | 6 ++++++
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
index 31e2327492ba..0d289cbc0123 100644
--- a/drivers/gpu/drm/i915/i915_vma.c
+++ b/drivers/gpu/drm/i915/i915_vma.c
@@ -373,11 +373,6 @@ void __i915_vma_set_map_and_fenceable(struct i915_vma *vma)
 		vma->flags &= ~I915_VMA_CAN_FENCE;
 }
 
-static bool color_differs(struct drm_mm_node *node, unsigned long color)
-{
-	return node->allocated && node->color != color;
-}
-
 bool i915_gem_valid_gtt_space(struct i915_vma *vma, unsigned long cache_level)
 {
 	struct drm_mm_node *node = &vma->node;
@@ -398,11 +393,11 @@ bool i915_gem_valid_gtt_space(struct i915_vma *vma, unsigned long cache_level)
 	GEM_BUG_ON(list_empty(&node->node_list));
 
 	other = list_prev_entry(node, node_list);
-	if (color_differs(other, cache_level) && !drm_mm_hole_follows(other))
+	if (i915_color_differs(other, cache_level) && !drm_mm_hole_follows(other))
 		return false;
 
 	other = list_next_entry(node, node_list);
-	if (color_differs(other, cache_level) && !drm_mm_hole_follows(node))
+	if (i915_color_differs(other, cache_level) && !drm_mm_hole_follows(node))
 		return false;
 
 	return true;
diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h
index 2e03f81dddbe..5492b03d6a77 100644
--- a/drivers/gpu/drm/i915/i915_vma.h
+++ b/drivers/gpu/drm/i915/i915_vma.h
@@ -284,6 +284,12 @@ static inline void i915_vma_unpin(struct i915_vma *vma)
 	__i915_vma_unpin(vma);
 }
 
+static inline bool i915_color_differs(const struct drm_mm_node *node,
+				      unsigned long color)
+{
+	return node->allocated && node->color != color;
+}
+
 /**
  * i915_vma_pin_iomap - calls ioremap_wc to map the GGTT VMA via the aperture
  * @vma: VMA to iomap
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 10/15] drm/i915: introduce ppgtt page coloring

[Matthew Auld]

To enable 64K pages we need to set the intermediate-page-size(IPS) bit
of the pde, therefore a page table is said to be either operating in 64K
or 4K mode. To accommodate this vm placement restriction we introduce a
color for pages and corresponding color_adjust callback. Currently this
only considers the full 48bit ppgtt case.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/i915_gem_gtt.c | 25 +++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_gem_gtt.h |  6 ++++++
 drivers/gpu/drm/i915/i915_vma.c     |  2 ++
 3 files changed, 33 insertions(+)

diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 6b70332cafac..0fb67941ba6b 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -1331,6 +1331,28 @@ static int gen8_preallocate_top_level_pdp(struct i915_hw_ppgtt *ppgtt)
 	return -ENOMEM;
 }
 
+static void i915_page_color_adjust(const struct drm_mm_node *node,
+				   unsigned long color,
+				   u64 *start,
+				   u64 *end)
+{
+	GEM_BUG_ON(color & ~I915_GTT_PAGE_SIZE_MASK);
+
+	if (!(color & (I915_GTT_PAGE_SIZE_64K | I915_GTT_PAGE_SIZE)))
+		return;
+
+	GEM_BUG_ON(node->allocated && node->color & ~I915_GTT_PAGE_SIZE_MASK);
+
+	if (i915_color_differs(node, color))
+		*start = roundup(*start, 1 << GEN8_PDE_SHIFT);
+
+	node = list_next_entry(node, node_list);
+	if (i915_color_differs(node, color))
+		*end = rounddown(*end, 1 << GEN8_PDE_SHIFT);
+
+	GEM_BUG_ON(node->allocated && node->color & ~I915_GTT_PAGE_SIZE_MASK);
+}
+
 /*
  * GEN8 legacy ppgtt programming is accomplished through a max 4 PDP registers
  * with a net effect resembling a 2-level page table in normal x86 terms. Each
@@ -1371,6 +1393,9 @@ static int gen8_ppgtt_init(struct i915_hw_ppgtt *ppgtt)
 		ppgtt->base.allocate_va_range = gen8_ppgtt_alloc_4lvl;
 		ppgtt->base.insert_entries = gen8_ppgtt_insert_4lvl;
 		ppgtt->base.clear_range = gen8_ppgtt_clear_4lvl;
+
+		if (INTEL_INFO(dev_priv)->page_size_mask & I915_GTT_PAGE_SIZE_64K)
+			ppgtt->base.mm.color_adjust = i915_page_color_adjust;
 	} else {
 		ret = __pdp_init(&ppgtt->base, &ppgtt->pdp);
 		if (ret)
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index 8d7436105718..e1b8d1c4734b 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -352,6 +352,12 @@ i915_uses_cache_coloring(const struct i915_address_space *vm)
 }
 
 static inline bool
+i915_uses_page_coloring(const struct i915_address_space *vm)
+{
+	return vm->mm.color_adjust && !i915_is_ggtt(vm);
+}
+
+static inline bool
 i915_vm_is_48bit(const struct i915_address_space *vm)
 {
 	return (vm->total - 1) >> 32;
diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
index 0d289cbc0123..ee5395e744e1 100644
--- a/drivers/gpu/drm/i915/i915_vma.c
+++ b/drivers/gpu/drm/i915/i915_vma.c
@@ -469,6 +469,8 @@ i915_vma_insert(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
 
 	if (i915_uses_cache_coloring(vma->vm))
 		color = obj->cache_level;
+	else if (i915_uses_page_coloring(vma->vm))
+		color = obj->page_size;
 
 	if (flags & PIN_OFFSET_FIXED) {
 		u64 offset = flags & PIN_OFFSET_MASK;
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 11/15] drm/i915: support inserting 64K pages in the ppgtt

[Matthew Auld]

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/i915_gem_gtt.c | 70 +++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 0fb67941ba6b..4d863de6473d 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -848,6 +848,73 @@ static __always_inline struct gen8_insert_pte gen8_insert_pte(u64 start)
 }
 
 static __always_inline bool
+gen8_ppgtt_insert_64K_pte_entries(struct i915_hw_ppgtt *ppgtt,
+				  struct i915_page_directory_pointer *pdp,
+				  struct sgt_dma *iter,
+				  struct gen8_insert_pte *idx,
+				  enum i915_cache_level cache_level)
+{
+	struct i915_page_directory *pd;
+	const gen8_pte_t pte_encode = gen8_pte_encode(0, cache_level);
+	gen8_pte_t *vaddr;
+	bool ret;
+
+	GEM_BUG_ON(idx->pte % 16);
+	GEM_BUG_ON(idx->pdpe >= i915_pdpes_per_pdp(&ppgtt->base));
+	/* TODO: probably move this to the allocation phase.. */
+	pd = pdp->page_directory[idx->pdpe];
+	vaddr = kmap_atomic_px(pd);
+	vaddr[idx->pde] |= GEN8_PDE_IPS_64K;
+	kunmap_atomic(vaddr);
+
+	vaddr = kmap_atomic_px(pd->page_table[idx->pde]);
+	do {
+		vaddr[idx->pte] = pte_encode | iter->dma;
+		iter->dma += I915_GTT_PAGE_SIZE_64K;
+		if (iter->dma >= iter->max) {
+			iter->sg = __sg_next(iter->sg);
+			if (!iter->sg) {
+				ret = false;
+				break;
+			}
+
+			iter->dma = sg_dma_address(iter->sg);
+			iter->max = iter->dma + iter->sg->length;
+		}
+
+		idx->pte += 16;
+
+		if (idx->pte == GEN8_PTES) {
+			idx->pte = 0;
+
+			if (++idx->pde == I915_PDES) {
+				idx->pde = 0;
+
+				/* Limited by sg length for 3lvl */
+				if (++idx->pdpe == GEN8_PML4ES_PER_PML4) {
+					idx->pdpe = 0;
+					ret = true;
+					break;
+				}
+
+				GEM_BUG_ON(idx->pdpe >= i915_pdpes_per_pdp(&ppgtt->base));
+				pd = pdp->page_directory[idx->pdpe];
+			}
+
+			kunmap_atomic(vaddr);
+			vaddr = kmap_atomic_px(pd);
+			vaddr[idx->pde] |= GEN8_PDE_IPS_64K;
+			kunmap_atomic(vaddr);
+
+			vaddr = kmap_atomic_px(pd->page_table[idx->pde]);
+		}
+	} while (1);
+	kunmap_atomic(vaddr);
+
+	return ret;
+}
+
+static __always_inline bool
 gen8_ppgtt_insert_pte_entries(struct i915_hw_ppgtt *ppgtt,
 			      struct i915_page_directory_pointer *pdp,
 			      struct sgt_dma *iter,
@@ -947,6 +1014,9 @@ static void gen8_ppgtt_insert_4lvl(struct i915_address_space *vm,
 	case I915_GTT_PAGE_SIZE:
 		insert_entries = gen8_ppgtt_insert_pte_entries;
 		break;
+	case I915_GTT_PAGE_SIZE_64K:
+		insert_entries = gen8_ppgtt_insert_64K_pte_entries;
+		break;
 	default:
 		MISSING_CASE(page_size);
 		return;
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 12/15] drm/i915: support inserting 2M pages in the ppgtt

[Matthew Auld]

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/i915_gem_gtt.c | 53 +++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 4d863de6473d..f71a6320ae1b 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -848,6 +848,56 @@ static __always_inline struct gen8_insert_pte gen8_insert_pte(u64 start)
 }
 
 static __always_inline bool
+gen8_ppgtt_insert_2M_pde_entries(struct i915_hw_ppgtt *ppgtt,
+				 struct i915_page_directory_pointer *pdp,
+				 struct sgt_dma *iter,
+				 struct gen8_insert_pte *idx,
+				 enum i915_cache_level cache_level)
+{
+	const gen8_pte_t pde_encode = gen8_pte_encode(GEN8_PDE_PS_2M,
+						      cache_level);
+	gen8_pte_t *vaddr;
+	bool ret;
+
+	GEM_BUG_ON(idx->pte);
+	GEM_BUG_ON(idx->pdpe >= i915_pdpes_per_pdp(&ppgtt->base));
+	vaddr = kmap_atomic_px(pdp->page_directory[idx->pdpe]);
+	do {
+		vaddr[idx->pde] = pde_encode | iter->dma;
+		iter->dma += I915_GTT_PAGE_SIZE_2M;
+		if (iter->dma >= iter->max) {
+			iter->sg = __sg_next(iter->sg);
+			if (!iter->sg) {
+				ret = false;
+				break;
+			}
+
+			iter->dma = sg_dma_address(iter->sg);
+			iter->max = iter->dma + iter->sg->length;
+		}
+
+		if (++idx->pde == I915_PDES) {
+			idx->pde = 0;
+
+			if (++idx->pdpe == GEN8_PML4ES_PER_PML4) {
+				idx->pdpe = 0;
+				ret = true;
+				break;
+			}
+
+			kunmap_atomic(vaddr);
+			vaddr = kmap_atomic_px(pdp->page_directory[idx->pdpe]);
+		}
+
+	} while (1);
+	kunmap_atomic(vaddr);
+
+	mark_tlbs_dirty(ppgtt);
+
+	return ret;
+}
+
+static __always_inline bool
 gen8_ppgtt_insert_64K_pte_entries(struct i915_hw_ppgtt *ppgtt,
 				  struct i915_page_directory_pointer *pdp,
 				  struct sgt_dma *iter,
@@ -1017,6 +1067,9 @@ static void gen8_ppgtt_insert_4lvl(struct i915_address_space *vm,
 	case I915_GTT_PAGE_SIZE_64K:
 		insert_entries = gen8_ppgtt_insert_64K_pte_entries;
 		break;
+	case I915_GTT_PAGE_SIZE_2M:
+		insert_entries = gen8_ppgtt_insert_2M_pde_entries;
+		break;
 	default:
 		MISSING_CASE(page_size);
 		return;
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 13/15] drm/i915: support inserting 1G pages in the ppgtt

[Matthew Auld]

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/i915_gem_gtt.c | 45 +++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index f71a6320ae1b..ecc63bc1fcc1 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -848,6 +848,48 @@ static __always_inline struct gen8_insert_pte gen8_insert_pte(u64 start)
 }
 
 static __always_inline bool
+gen8_ppgtt_insert_1G_pdpe_entries(struct i915_hw_ppgtt *ppgtt,
+				  struct i915_page_directory_pointer *pdp,
+				  struct sgt_dma *iter,
+				  struct gen8_insert_pte *idx,
+				  enum i915_cache_level cache_level)
+{
+	const gen8_pte_t pdpe_encode = gen8_pte_encode(GEN8_PDPE_PS_1G,
+						       cache_level);
+	gen8_pte_t *vaddr;
+	bool ret;
+
+	GEM_BUG_ON(idx->pte);
+	GEM_BUG_ON(idx->pde);
+	GEM_BUG_ON(idx->pdpe >= i915_pdpes_per_pdp(&ppgtt->base));
+	vaddr = kmap_atomic_px(pdp);
+	do {
+		vaddr[idx->pdpe] = pdpe_encode | iter->dma;
+		iter->dma += I915_GTT_PAGE_SIZE_1G;
+		if (iter->dma >= iter->max) {
+			iter->sg = __sg_next(iter->sg);
+			if (!iter->sg) {
+				ret = false;
+				break;
+			}
+
+			iter->dma = sg_dma_address(iter->sg);
+			iter->max = iter->dma + iter->sg->length;
+		}
+
+		if (++idx->pdpe == GEN8_PML4ES_PER_PML4) {
+			idx->pdpe = 0;
+			ret = true;
+			break;
+		}
+
+	} while (1);
+	kunmap_atomic(vaddr);
+
+	return ret;
+}
+
+static __always_inline bool
 gen8_ppgtt_insert_2M_pde_entries(struct i915_hw_ppgtt *ppgtt,
 				 struct i915_page_directory_pointer *pdp,
 				 struct sgt_dma *iter,
@@ -1070,6 +1112,9 @@ static void gen8_ppgtt_insert_4lvl(struct i915_address_space *vm,
 	case I915_GTT_PAGE_SIZE_2M:
 		insert_entries = gen8_ppgtt_insert_2M_pde_entries;
 		break;
+	case I915_GTT_PAGE_SIZE_1G:
+		insert_entries = gen8_ppgtt_insert_1G_pdpe_entries;
+		break;
 	default:
 		MISSING_CASE(page_size);
 		return;
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 14/15] drm/i915/selftests: exercise 4K and 64K mm insertion

[Matthew Auld]

Mock test filling an address space with 4K and 64K objects, in the hope
of exercising the page color adjust fun.

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 68 +++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
index c7963efe46ba..9b2a7228a78f 100644
--- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
@@ -31,6 +31,7 @@
 #include "mock_context.h"
 #include "mock_drm.h"
 #include "mock_gem_device.h"
+#include "mock_gtt.h"
 
 static void fake_free_pages(struct drm_i915_gem_object *obj,
 			    struct sg_table *pages)
@@ -1306,6 +1307,72 @@ static int igt_gtt_reserve(void *arg)
 	return err;
 }
 
+static int igt_ppgtt_page_color(void *arg)
+{
+	struct drm_mm mm;
+	struct drm_mm_node *node, *prev, *next;
+	unsigned long page_colors[] = {
+		I915_GTT_PAGE_SIZE,
+		I915_GTT_PAGE_SIZE_64K,
+	};
+	int idx = 0;
+	u64 count = 0;
+	u64 size;
+
+	drm_mm_init(&mm, 0, U64_MAX);
+	mm.color_adjust = i915_page_color_adjust;
+
+	/* Running out of memory is okay. */
+
+	for_each_prime_number_from(size, 0, U64_MAX) {
+		node = kzalloc(sizeof(*node), GFP_KERNEL);
+		if (!node) {
+			pr_info("finished test early, unable to allocate node, count=%llu\n", count);
+			break;
+		}
+
+		size = roundup(size, page_colors[idx]);
+
+		if (drm_mm_insert_node_in_range(&mm, node, size,
+						page_colors[idx],
+						page_colors[idx],
+						0, U64_MAX,
+						DRM_MM_INSERT_BEST)) {
+			pr_info("test finished, unable to insert node: color=%lu, size=%llx, count=%llu\n",
+				page_colors[idx], size, count);
+			kfree(node);
+			break;
+		}
+
+		GEM_BUG_ON(!IS_ALIGNED(node->start, node->color));
+		GEM_BUG_ON(!IS_ALIGNED(node->size, node->color));
+
+		/* We can't mix 4K and 64K pte's in the same pt. */
+
+		prev = list_prev_entry(node, node_list);
+		if (i915_color_differs(prev, node->color))
+			GEM_BUG_ON(prev->start >> GEN8_PDE_SHIFT ==
+				   node->start >> GEN8_PDE_SHIFT);
+
+		next = list_next_entry(node, node_list);
+		if (i915_color_differs(next, node->color))
+			GEM_BUG_ON(((next->start + next->size) >> GEN8_PDE_SHIFT) ==
+				   ((node->start + node->size) >> GEN8_PDE_SHIFT));
+
+		idx ^= 1;
+		++count;
+	}
+
+	drm_mm_for_each_node_safe(node, next, &mm) {
+		drm_mm_remove_node(node);
+		kfree(node);
+	}
+
+	drm_mm_takedown(&mm);
+
+	return 0;
+}
+
 static int igt_gtt_insert(void *arg)
 {
 	struct drm_i915_private *i915 = arg;
@@ -1522,6 +1589,7 @@ int i915_gem_gtt_mock_selftests(void)
 		SUBTEST(igt_mock_fill),
 		SUBTEST(igt_gtt_reserve),
 		SUBTEST(igt_gtt_insert),
+		SUBTEST(igt_ppgtt_page_color),
 	};
 	struct drm_i915_private *i915;
 	int err;
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[PATCH 15/15] drm/i915/selftests: modify the gtt tests to also exercise huge pages

[Matthew Auld]

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 123 ++++++++++++++++++--------
 drivers/gpu/drm/i915/selftests/mock_gtt.c     |   3 +
 2 files changed, 89 insertions(+), 37 deletions(-)

diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
index 9b2a7228a78f..4625f93a3890 100644
--- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
@@ -92,12 +92,14 @@ static const struct drm_i915_gem_object_ops fake_ops = {
 };
 
 static struct drm_i915_gem_object *
-fake_dma_object(struct drm_i915_private *i915, u64 size)
+fake_dma_object(struct drm_i915_private *i915, u64 size, unsigned long page_size)
 {
 	struct drm_i915_gem_object *obj;
 
 	GEM_BUG_ON(!size);
-	GEM_BUG_ON(!IS_ALIGNED(size, I915_GTT_PAGE_SIZE));
+	GEM_BUG_ON(page_size & ~I915_GTT_PAGE_SIZE_MASK);
+
+	size = roundup(size, page_size);
 
 	if (overflows_type(size, obj->base.size))
 		return ERR_PTR(-E2BIG);
@@ -107,8 +109,13 @@ fake_dma_object(struct drm_i915_private *i915, u64 size)
 		goto err;
 
 	drm_gem_private_object_init(&i915->drm, &obj->base, size);
+
 	i915_gem_object_init(obj, &fake_ops);
 
+	obj->page_size = page_size;
+
+	GEM_BUG_ON(!IS_ALIGNED(obj->base.size, obj->page_size));
+
 	obj->base.write_domain = I915_GEM_DOMAIN_CPU;
 	obj->base.read_domains = I915_GEM_DOMAIN_CPU;
 	obj->cache_level = I915_CACHE_NONE;
@@ -194,13 +201,14 @@ static int igt_ppgtt_alloc(void *arg)
 static int lowlevel_hole(struct drm_i915_private *i915,
 			 struct i915_address_space *vm,
 			 u64 hole_start, u64 hole_end,
+			 unsigned long page_size,
 			 unsigned long end_time)
 {
 	I915_RND_STATE(seed_prng);
 	unsigned int size;
 
 	/* Keep creating larger objects until one cannot fit into the hole */
-	for (size = 12; (hole_end - hole_start) >> size; size++) {
+	for (size = ilog2(page_size); (hole_end - hole_start) >> size; size++) {
 		I915_RND_SUBSTATE(prng, seed_prng);
 		struct drm_i915_gem_object *obj;
 		unsigned int *order, count, n;
@@ -226,7 +234,7 @@ static int lowlevel_hole(struct drm_i915_private *i915,
 		 * memory. We expect to hit -ENOMEM.
 		 */
 
-		obj = fake_dma_object(i915, BIT_ULL(size));
+		obj = fake_dma_object(i915, BIT_ULL(size), page_size);
 		if (IS_ERR(obj)) {
 			kfree(order);
 			break;
@@ -303,18 +311,25 @@ static void close_object_list(struct list_head *objects,
 static int fill_hole(struct drm_i915_private *i915,
 		     struct i915_address_space *vm,
 		     u64 hole_start, u64 hole_end,
+		     unsigned long page_size,
 		     unsigned long end_time)
 {
 	const u64 hole_size = hole_end - hole_start;
 	struct drm_i915_gem_object *obj;
-	const unsigned long max_pages =
-		min_t(u64, ULONG_MAX - 1, hole_size/2 >> PAGE_SHIFT);
-	const unsigned long max_step = max(int_sqrt(max_pages), 2UL);
-	unsigned long npages, prime, flags;
+	const unsigned page_shift = ilog2(page_size);
+	unsigned long max_pages, max_step, npages, prime, flags;
 	struct i915_vma *vma;
 	LIST_HEAD(objects);
 	int err;
 
+	hole_start = roundup(hole_start, page_size);
+	hole_end = rounddown(hole_end, page_size);
+
+	GEM_BUG_ON(hole_start >= hole_end);
+
+	max_pages = min_t(u64, ULONG_MAX - 1, hole_size/2 >> page_shift);
+	max_step = max(int_sqrt(max_pages), 2UL);
+
 	/* Try binding many VMA working inwards from either edge */
 
 	flags = PIN_OFFSET_FIXED | PIN_USER;
@@ -323,7 +338,7 @@ static int fill_hole(struct drm_i915_private *i915,
 
 	for_each_prime_number_from(prime, 2, max_step) {
 		for (npages = 1; npages <= max_pages; npages *= prime) {
-			const u64 full_size = npages << PAGE_SHIFT;
+			const u64 full_size = npages << page_shift;
 			const struct {
 				const char *name;
 				u64 offset;
@@ -334,7 +349,7 @@ static int fill_hole(struct drm_i915_private *i915,
 				{ }
 			}, *p;
 
-			obj = fake_dma_object(i915, full_size);
+			obj = fake_dma_object(i915, full_size, page_size);
 			if (IS_ERR(obj))
 				break;
 
@@ -359,7 +374,7 @@ static int fill_hole(struct drm_i915_private *i915,
 						offset -= obj->base.size;
 					}
 
-					err = i915_vma_pin(vma, 0, 0, offset | flags);
+					err = i915_vma_pin(vma, 0, page_size, offset | flags);
 					if (err) {
 						pr_err("%s(%s) pin (forward) failed with err=%d on size=%lu pages (prime=%lu), offset=%llx\n",
 						       __func__, p->name, err, npages, prime, offset);
@@ -367,7 +382,7 @@ static int fill_hole(struct drm_i915_private *i915,
 					}
 
 					if (!drm_mm_node_allocated(&vma->node) ||
-					    i915_vma_misplaced(vma, 0, 0, offset | flags)) {
+					    i915_vma_misplaced(vma, 0, page_size, offset | flags)) {
 						pr_err("%s(%s) (forward) insert failed: vma.node=%llx + %llx [allocated? %d], expected offset %llx\n",
 						       __func__, p->name, vma->node.start, vma->node.size, drm_mm_node_allocated(&vma->node),
 						       offset);
@@ -397,7 +412,7 @@ static int fill_hole(struct drm_i915_private *i915,
 					}
 
 					if (!drm_mm_node_allocated(&vma->node) ||
-					    i915_vma_misplaced(vma, 0, 0, offset | flags)) {
+					    i915_vma_misplaced(vma, 0, page_size, offset | flags)) {
 						pr_err("%s(%s) (forward) moved vma.node=%llx + %llx, expected offset %llx\n",
 						       __func__, p->name, vma->node.start, vma->node.size,
 						       offset);
@@ -432,7 +447,7 @@ static int fill_hole(struct drm_i915_private *i915,
 						offset -= obj->base.size;
 					}
 
-					err = i915_vma_pin(vma, 0, 0, offset | flags);
+					err = i915_vma_pin(vma, 0, page_size, offset | flags);
 					if (err) {
 						pr_err("%s(%s) pin (backward) failed with err=%d on size=%lu pages (prime=%lu), offset=%llx\n",
 						       __func__, p->name, err, npages, prime, offset);
@@ -440,7 +455,7 @@ static int fill_hole(struct drm_i915_private *i915,
 					}
 
 					if (!drm_mm_node_allocated(&vma->node) ||
-					    i915_vma_misplaced(vma, 0, 0, offset | flags)) {
+					    i915_vma_misplaced(vma, 0, page_size, offset | flags)) {
 						pr_err("%s(%s) (backward) insert failed: vma.node=%llx + %llx [allocated? %d], expected offset %llx\n",
 						       __func__, p->name, vma->node.start, vma->node.size, drm_mm_node_allocated(&vma->node),
 						       offset);
@@ -470,7 +485,7 @@ static int fill_hole(struct drm_i915_private *i915,
 					}
 
 					if (!drm_mm_node_allocated(&vma->node) ||
-					    i915_vma_misplaced(vma, 0, 0, offset | flags)) {
+					    i915_vma_misplaced(vma, 0, page_size, offset | flags)) {
 						pr_err("%s(%s) (backward) moved vma.node=%llx + %llx [allocated? %d], expected offset %llx\n",
 						       __func__, p->name, vma->node.start, vma->node.size, drm_mm_node_allocated(&vma->node),
 						       offset);
@@ -514,11 +529,13 @@ static int fill_hole(struct drm_i915_private *i915,
 static int walk_hole(struct drm_i915_private *i915,
 		     struct i915_address_space *vm,
 		     u64 hole_start, u64 hole_end,
+		     unsigned long page_size,
 		     unsigned long end_time)
 {
 	const u64 hole_size = hole_end - hole_start;
+	const unsigned page_shift = ilog2(page_size);
 	const unsigned long max_pages =
-		min_t(u64, ULONG_MAX - 1, hole_size >> PAGE_SHIFT);
+		min_t(u64, ULONG_MAX - 1, hole_size >> page_shift);
 	unsigned long flags;
 	u64 size;
 
@@ -534,7 +551,7 @@ static int walk_hole(struct drm_i915_private *i915,
 		u64 addr;
 		int err = 0;
 
-		obj = fake_dma_object(i915, size << PAGE_SHIFT);
+		obj = fake_dma_object(i915, size << page_shift, page_size);
 		if (IS_ERR(obj))
 			break;
 
@@ -547,7 +564,7 @@ static int walk_hole(struct drm_i915_private *i915,
 		for (addr = hole_start;
 		     addr + obj->base.size < hole_end;
 		     addr += obj->base.size) {
-			err = i915_vma_pin(vma, 0, 0, addr | flags);
+			err = i915_vma_pin(vma, 0, page_size, addr | flags);
 			if (err) {
 				pr_err("%s bind failed at %llx + %llx [hole %llx- %llx] with err=%d\n",
 				       __func__, addr, vma->size,
@@ -557,7 +574,7 @@ static int walk_hole(struct drm_i915_private *i915,
 			i915_vma_unpin(vma);
 
 			if (!drm_mm_node_allocated(&vma->node) ||
-			    i915_vma_misplaced(vma, 0, 0, addr | flags)) {
+			    i915_vma_misplaced(vma, 0, page_size, addr | flags)) {
 				pr_err("%s incorrect at %llx + %llx\n",
 				       __func__, addr, vma->size);
 				err = -EINVAL;
@@ -595,6 +612,7 @@ static int walk_hole(struct drm_i915_private *i915,
 static int pot_hole(struct drm_i915_private *i915,
 		    struct i915_address_space *vm,
 		    u64 hole_start, u64 hole_end,
+		    unsigned long page_size,
 		    unsigned long end_time)
 {
 	struct drm_i915_gem_object *obj;
@@ -607,7 +625,7 @@ static int pot_hole(struct drm_i915_private *i915,
 	if (i915_is_ggtt(vm))
 		flags |= PIN_GLOBAL;
 
-	obj = i915_gem_object_create_internal(i915, 2 * I915_GTT_PAGE_SIZE);
+	obj = fake_dma_object(i915, 2 * page_size, page_size);
 	if (IS_ERR(obj))
 		return PTR_ERR(obj);
 
@@ -619,15 +637,15 @@ static int pot_hole(struct drm_i915_private *i915,
 
 	/* Insert a pair of pages across every pot boundary within the hole */
 	for (pot = fls64(hole_end - 1) - 1;
-	     pot > ilog2(2 * I915_GTT_PAGE_SIZE);
+	     pot > ilog2(2 * page_size);
 	     pot--) {
 		u64 step = BIT_ULL(pot);
 		u64 addr;
 
-		for (addr = round_up(hole_start + I915_GTT_PAGE_SIZE, step) - I915_GTT_PAGE_SIZE;
-		     addr <= round_down(hole_end - 2*I915_GTT_PAGE_SIZE, step) - I915_GTT_PAGE_SIZE;
+		for (addr = round_up(hole_start + page_size, step) - page_size;
+		     addr <= round_down(hole_end - 2*page_size, step) - page_size;
 		     addr += step) {
-			err = i915_vma_pin(vma, 0, 0, addr | flags);
+			err = i915_vma_pin(vma, 0, page_size, addr | flags);
 			if (err) {
 				pr_err("%s failed to pin object at %llx in hole [%llx - %llx], with err=%d\n",
 				       __func__,
@@ -671,6 +689,7 @@ static int pot_hole(struct drm_i915_private *i915,
 static int drunk_hole(struct drm_i915_private *i915,
 		      struct i915_address_space *vm,
 		      u64 hole_start, u64 hole_end,
+		      unsigned long page_size,
 		      unsigned long end_time)
 {
 	I915_RND_STATE(prng);
@@ -682,7 +701,7 @@ static int drunk_hole(struct drm_i915_private *i915,
 		flags |= PIN_GLOBAL;
 
 	/* Keep creating larger objects until one cannot fit into the hole */
-	for (size = 12; (hole_end - hole_start) >> size; size++) {
+	for (size = ilog2(page_size); (hole_end - hole_start) >> size; size++) {
 		struct drm_i915_gem_object *obj;
 		unsigned int *order, count, n;
 		struct i915_vma *vma;
@@ -706,7 +725,7 @@ static int drunk_hole(struct drm_i915_private *i915,
 		 * memory. We expect to hit -ENOMEM.
 		 */
 
-		obj = fake_dma_object(i915, BIT_ULL(size));
+		obj = fake_dma_object(i915, BIT_ULL(size), page_size);
 		if (IS_ERR(obj)) {
 			kfree(order);
 			break;
@@ -723,6 +742,8 @@ static int drunk_hole(struct drm_i915_private *i915,
 		for (n = 0; n < count; n++) {
 			u64 addr = hole_start + order[n] * BIT_ULL(size);
 
+			GEM_BUG_ON(!IS_ALIGNED(addr, page_size));
+
 			err = i915_vma_pin(vma, 0, 0, addr | flags);
 			if (err) {
 				pr_err("%s failed to pin object at %llx + %llx in hole [%llx - %llx], with err=%d\n",
@@ -734,7 +755,7 @@ static int drunk_hole(struct drm_i915_private *i915,
 			}
 
 			if (!drm_mm_node_allocated(&vma->node) ||
-			    i915_vma_misplaced(vma, 0, 0, addr | flags)) {
+			    i915_vma_misplaced(vma, 0, page_size, addr | flags)) {
 				pr_err("%s incorrect at %llx + %llx\n",
 				       __func__, addr, BIT_ULL(size));
 				i915_vma_unpin(vma);
@@ -771,11 +792,12 @@ static int drunk_hole(struct drm_i915_private *i915,
 static int __shrink_hole(struct drm_i915_private *i915,
 			 struct i915_address_space *vm,
 			 u64 hole_start, u64 hole_end,
+			 unsigned long page_size,
 			 unsigned long end_time)
 {
 	struct drm_i915_gem_object *obj;
 	unsigned long flags = PIN_OFFSET_FIXED | PIN_USER;
-	unsigned int order = 12;
+	unsigned int order = ilog2(page_size);
 	LIST_HEAD(objects);
 	int err = 0;
 	u64 addr;
@@ -786,7 +808,7 @@ static int __shrink_hole(struct drm_i915_private *i915,
 		u64 size = BIT_ULL(order++);
 
 		size = min(size, hole_end - addr);
-		obj = fake_dma_object(i915, size);
+		obj = fake_dma_object(i915, size, page_size);
 		if (IS_ERR(obj)) {
 			err = PTR_ERR(obj);
 			break;
@@ -802,7 +824,7 @@ static int __shrink_hole(struct drm_i915_private *i915,
 
 		GEM_BUG_ON(vma->size != size);
 
-		err = i915_vma_pin(vma, 0, 0, addr | flags);
+		err = i915_vma_pin(vma, 0, page_size, addr | flags);
 		if (err) {
 			pr_err("%s failed to pin object at %llx + %llx in hole [%llx - %llx], with err=%d\n",
 			       __func__, addr, size, hole_start, hole_end, err);
@@ -810,7 +832,7 @@ static int __shrink_hole(struct drm_i915_private *i915,
 		}
 
 		if (!drm_mm_node_allocated(&vma->node) ||
-		    i915_vma_misplaced(vma, 0, 0, addr | flags)) {
+		    i915_vma_misplaced(vma, 0, page_size, addr | flags)) {
 			pr_err("%s incorrect at %llx + %llx\n",
 			       __func__, addr, size);
 			i915_vma_unpin(vma);
@@ -837,6 +859,7 @@ static int __shrink_hole(struct drm_i915_private *i915,
 static int shrink_hole(struct drm_i915_private *i915,
 		       struct i915_address_space *vm,
 		       u64 hole_start, u64 hole_end,
+		       unsigned long page_size,
 		       unsigned long end_time)
 {
 	unsigned long prime;
@@ -847,7 +870,8 @@ static int shrink_hole(struct drm_i915_private *i915,
 
 	for_each_prime_number_from(prime, 0, ULONG_MAX - 1) {
 		vm->fault_attr.interval = prime;
-		err = __shrink_hole(i915, vm, hole_start, hole_end, end_time);
+		err = __shrink_hole(i915, vm, hole_start, hole_end, page_size,
+				    end_time);
 		if (err)
 			break;
 	}
@@ -861,12 +885,20 @@ static int exercise_ppgtt(struct drm_i915_private *dev_priv,
 			  int (*func)(struct drm_i915_private *i915,
 				      struct i915_address_space *vm,
 				      u64 hole_start, u64 hole_end,
+				      unsigned long page_size,
 				      unsigned long end_time))
 {
 	struct drm_file *file;
 	struct i915_hw_ppgtt *ppgtt;
 	IGT_TIMEOUT(end_time);
-	int err;
+	unsigned long page_sizes[] = {
+		I915_GTT_PAGE_SIZE,
+		I915_GTT_PAGE_SIZE_64K,
+		I915_GTT_PAGE_SIZE_2M,
+		I915_GTT_PAGE_SIZE_1G,
+	};
+	int err = 0;
+	int i;
 
 	if (!USES_FULL_PPGTT(dev_priv))
 		return 0;
@@ -884,7 +916,11 @@ static int exercise_ppgtt(struct drm_i915_private *dev_priv,
 	GEM_BUG_ON(offset_in_page(ppgtt->base.total));
 	GEM_BUG_ON(ppgtt->base.closed);
 
-	err = func(dev_priv, &ppgtt->base, 0, ppgtt->base.total, end_time);
+	for (i = 0; i < ARRAY_SIZE(page_sizes); ++i) {
+		if (INTEL_INFO(dev_priv)->page_size_mask & page_sizes[i])
+			err = func(dev_priv, &ppgtt->base, 0, ppgtt->base.total,
+				   page_sizes[i], end_time);
+	}
 
 	i915_ppgtt_close(&ppgtt->base);
 	i915_ppgtt_put(ppgtt);
@@ -940,6 +976,7 @@ static int exercise_ggtt(struct drm_i915_private *i915,
 			 int (*func)(struct drm_i915_private *i915,
 				     struct i915_address_space *vm,
 				     u64 hole_start, u64 hole_end,
+				     unsigned long page_size,
 				     unsigned long end_time))
 {
 	struct i915_ggtt *ggtt = &i915->ggtt;
@@ -961,7 +998,8 @@ static int exercise_ggtt(struct drm_i915_private *i915,
 		if (hole_start >= hole_end)
 			continue;
 
-		err = func(i915, &ggtt->base, hole_start, hole_end, end_time);
+		err = func(i915, &ggtt->base, hole_start, hole_end,
+			   I915_GTT_PAGE_SIZE, end_time);
 		if (err)
 			break;
 
@@ -1104,12 +1142,20 @@ static int exercise_mock(struct drm_i915_private *i915,
 			 int (*func)(struct drm_i915_private *i915,
 				     struct i915_address_space *vm,
 				     u64 hole_start, u64 hole_end,
+				     unsigned long page_size,
 				     unsigned long end_time))
 {
 	struct i915_gem_context *ctx;
 	struct i915_hw_ppgtt *ppgtt;
 	IGT_TIMEOUT(end_time);
+	unsigned long page_sizes[] = {
+		I915_GTT_PAGE_SIZE,
+		I915_GTT_PAGE_SIZE_64K,
+		I915_GTT_PAGE_SIZE_2M,
+		I915_GTT_PAGE_SIZE_1G,
+	};
 	int err;
+	int i;
 
 	ctx = mock_context(i915, "mock");
 	if (!ctx)
@@ -1118,7 +1164,10 @@ static int exercise_mock(struct drm_i915_private *i915,
 	ppgtt = ctx->ppgtt;
 	GEM_BUG_ON(!ppgtt);
 
-	err = func(i915, &ppgtt->base, 0, ppgtt->base.total, end_time);
+	for (i = 0; i < ARRAY_SIZE(page_sizes); ++i) {
+		err = func(i915, &ppgtt->base, 0, ppgtt->base.total,
+			   page_sizes[i], end_time);
+	}
 
 	mock_context_close(ctx);
 	return err;
diff --git a/drivers/gpu/drm/i915/selftests/mock_gtt.c b/drivers/gpu/drm/i915/selftests/mock_gtt.c
index a2137100d2f5..9428ea09d05d 100644
--- a/drivers/gpu/drm/i915/selftests/mock_gtt.c
+++ b/drivers/gpu/drm/i915/selftests/mock_gtt.c
@@ -88,6 +88,9 @@ mock_ppgtt(struct drm_i915_private *i915,
 	ppgtt->base.unbind_vma = mock_unbind_ppgtt;
 	ppgtt->base.cleanup = mock_cleanup;
 
+	/* For mock testing huge-page support */
+	ppgtt->base.mm.color_adjust = i915_page_color_adjust;
+
 	return ppgtt;
 }
 
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

✓ Fi.CI.BAT: success for drm/i915: initial support for huge gtt pages

[Patchwork]

== Series Details ==

Series: drm/i915: initial support for huge gtt pages
URL   : https://patchwork.freedesktop.org/series/20789/
State : success

== Summary ==

Series 20789v1 drm/i915: initial support for huge gtt pages
https://patchwork.freedesktop.org/api/1.0/series/20789/revisions/1/mbox/

Test drv_module_reload:
        Subgroup basic-no-display:
                incomplete -> PASS       (fi-hsw-4770r)
Test kms_cursor_legacy:
        Subgroup basic-flip-before-cursor-varying-size:
                dmesg-warn -> PASS       (fi-byt-n2820)

fi-bdw-5557u     total:278  pass:267  dwarn:0   dfail:0   fail:0   skip:11  time: 476s
fi-bsw-n3050     total:278  pass:239  dwarn:0   dfail:0   fail:0   skip:39  time: 621s
fi-bxt-j4205     total:278  pass:259  dwarn:0   dfail:0   fail:0   skip:19  time: 544s
fi-bxt-t5700     total:278  pass:258  dwarn:0   dfail:0   fail:0   skip:20  time: 613s
fi-byt-j1900     total:278  pass:251  dwarn:0   dfail:0   fail:0   skip:27  time: 504s
fi-byt-n2820     total:278  pass:247  dwarn:0   dfail:0   fail:0   skip:31  time: 498s
fi-hsw-4770      total:278  pass:262  dwarn:0   dfail:0   fail:0   skip:16  time: 440s
fi-hsw-4770r     total:278  pass:262  dwarn:0   dfail:0   fail:0   skip:16  time: 436s
fi-ilk-650       total:278  pass:228  dwarn:0   dfail:0   fail:0   skip:50  time: 444s
fi-ivb-3520m     total:278  pass:260  dwarn:0   dfail:0   fail:0   skip:18  time: 499s
fi-ivb-3770      total:278  pass:260  dwarn:0   dfail:0   fail:0   skip:18  time: 493s
fi-kbl-7500u     total:278  pass:259  dwarn:1   dfail:0   fail:0   skip:18  time: 474s
fi-skl-6260u     total:278  pass:268  dwarn:0   dfail:0   fail:0   skip:10  time: 516s
fi-skl-6700hq    total:278  pass:261  dwarn:0   dfail:0   fail:0   skip:17  time: 599s
fi-skl-6700k     total:278  pass:256  dwarn:4   dfail:0   fail:0   skip:18  time: 504s
fi-skl-6770hq    total:278  pass:268  dwarn:0   dfail:0   fail:0   skip:10  time: 548s
fi-snb-2520m     total:278  pass:250  dwarn:0   dfail:0   fail:0   skip:28  time: 559s
fi-snb-2600      total:278  pass:249  dwarn:0   dfail:0   fail:0   skip:29  time: 425s

1e8ad06b985178ccd08c175dee3e650d6734f3d1 drm-tip: 2017y-03m-06d-23h-42m-09s UTC integration manifest
9d6f3ce drm/i915/selftests: modify the gtt tests to also exercise huge pages
08353c4 drm/i915/selftests: exercise 4K and 64K mm insertion
17dfb48 drm/i915: support inserting 1G pages in the ppgtt
ed367a7 drm/i915: support inserting 2M pages in the ppgtt
f1d3620 drm/i915: support inserting 64K pages in the ppgtt
2b2ff61 drm/i915: introduce ppgtt page coloring
33ebb7c drm/i915: export color_differs
db95e23 drm/i915: clean up cache coloring
e31a73b drm/i915: s/i915_gtt_color_adjust/i915_cache_color_adjust
424ba66 drm/i915: pass page_size to insert_entries
08f8df8 drm/i915: introduce drm_i915_gem_object page_size member
5841480 drm/i915: add page_size_mask to dev_info
b510286 drm/i915/selftests: exercise cache domain eviction
bb3bd5a drm/i915: use correct node for handling cache domain eviction
0457be3 drm/i915/selftests: don't leak the gem object

== Logs ==

For more details see: https://intel-gfx-ci.01.org/CI/Patchwork_4077/
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH 04/15] drm/i915: add page_size_mask to dev_info

[Mika Kuoppala]

Matthew Auld <matthew.auld@intel.com> writes:

> Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> ---
>  drivers/gpu/drm/i915/i915_drv.h     |  1 +
>  drivers/gpu/drm/i915/i915_gem_gtt.h | 14 ++++++++++++++
>  drivers/gpu/drm/i915/i915_pci.c     | 23 ++++++++++++++++++++++-
>  3 files changed, 37 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index 1fd4128a10b1..e45b8d74cebf 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -913,6 +913,7 @@ struct intel_device_info {
>  	enum intel_platform platform;
>  	u8 ring_mask; /* Rings supported by the HW */
>  	u8 num_rings;
> +	unsigned long page_size_mask; /* page sizes supported by the HW */
>  #define DEFINE_FLAG(name) u8 name:1
>  	DEV_INFO_FOR_EACH_FLAG(DEFINE_FLAG);
>  #undef DEFINE_FLAG
> diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
> index fb15684c1d83..6c90a2ffd0e1 100644
> --- a/drivers/gpu/drm/i915/i915_gem_gtt.h
> +++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
> @@ -43,8 +43,18 @@
>  #include "i915_selftest.h"
>  
>  #define I915_GTT_PAGE_SIZE 4096UL
> +#define I915_GTT_PAGE_SIZE_64K 65536UL
> +#define I915_GTT_PAGE_SIZE_2M 2097152UL
> +#define I915_GTT_PAGE_SIZE_1G 1073741824UL
> +
> +#define I915_GTT_PAGE_SIZE_MASK (I915_GTT_PAGE_SIZE | \
> +				 I915_GTT_PAGE_SIZE_64K | \
> +				 I915_GTT_PAGE_SIZE_2M | \
> +				 I915_GTT_PAGE_SIZE_1G)
> +
>  #define I915_GTT_MIN_ALIGNMENT I915_GTT_PAGE_SIZE
>  
> +
>  #define I915_FENCE_REG_NONE -1
>  #define I915_MAX_NUM_FENCES 32
>  /* 32 fences + sign bit for FENCE_REG_NONE */
> @@ -143,6 +153,10 @@ typedef u64 gen8_ppgtt_pml4e_t;
>  #define GEN8_PPAT_ELLC_OVERRIDE		(0<<2)
>  #define GEN8_PPAT(i, x)			((u64)(x) << ((i) * 8))
>  
> +#define GEN8_PDPE_PS_1G  BIT(7)
> +#define GEN8_PDE_PS_2M   BIT(7)

Just a minor bikeshed here to help the reader:
Show the PDE entries first and from smallest first,
then have one empty line before the PDPE entry. Also
this patch doesn't seem to be needing these so
consider moving these to the patch that does.

-Mika

> +#define GEN8_PDE_IPS_64K BIT(11)
> +
>  struct sg_table;
>  
>  struct intel_rotation_info {
> diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pci.c
> index 732101ed57fb..5abb7d84b65a 100644
> --- a/drivers/gpu/drm/i915/i915_pci.c
> +++ b/drivers/gpu/drm/i915/i915_pci.c
> @@ -56,6 +56,10 @@
>  	.color = { .degamma_lut_size = 65, .gamma_lut_size = 257 }
>  
>  /* Keep in gen based order, and chronological order within a gen */
> +
> +#define GEN_DEFAULT_PAGE_SZ \
> +	.page_size_mask = I915_GTT_PAGE_SIZE
> +
>  #define GEN2_FEATURES \
>  	.gen = 2, .num_pipes = 1, \
>  	.has_overlay = 1, .overlay_needs_physical = 1, \
> @@ -63,6 +67,7 @@
>  	.hws_needs_physical = 1, \
>  	.ring_mask = RENDER_RING, \
>  	GEN_DEFAULT_PIPEOFFSETS, \
> +	GEN_DEFAULT_PAGE_SZ, \
>  	CURSOR_OFFSETS
>  
>  static const struct intel_device_info intel_i830_info = {
> @@ -95,6 +100,7 @@ static const struct intel_device_info intel_i865g_info = {
>  	.has_gmch_display = 1, \
>  	.ring_mask = RENDER_RING, \
>  	GEN_DEFAULT_PIPEOFFSETS, \
> +	GEN_DEFAULT_PAGE_SZ, \
>  	CURSOR_OFFSETS
>  
>  static const struct intel_device_info intel_i915g_info = {
> @@ -153,6 +159,7 @@ static const struct intel_device_info intel_pineview_info = {
>  	.has_gmch_display = 1, \
>  	.ring_mask = RENDER_RING, \
>  	GEN_DEFAULT_PIPEOFFSETS, \
> +	GEN_DEFAULT_PAGE_SZ, \
>  	CURSOR_OFFSETS
>  
>  static const struct intel_device_info intel_i965g_info = {
> @@ -193,6 +200,7 @@ static const struct intel_device_info intel_gm45_info = {
>  	.has_gmbus_irq = 1, \
>  	.ring_mask = RENDER_RING | BSD_RING, \
>  	GEN_DEFAULT_PIPEOFFSETS, \
> +	GEN_DEFAULT_PAGE_SZ, \
>  	CURSOR_OFFSETS
>  
>  static const struct intel_device_info intel_ironlake_d_info = {
> @@ -218,6 +226,7 @@ static const struct intel_device_info intel_ironlake_m_info = {
>  	.has_hw_contexts = 1, \
>  	.has_aliasing_ppgtt = 1, \
>  	GEN_DEFAULT_PIPEOFFSETS, \
> +	GEN_DEFAULT_PAGE_SZ, \
>  	CURSOR_OFFSETS
>  
>  static const struct intel_device_info intel_sandybridge_d_info = {
> @@ -244,6 +253,7 @@ static const struct intel_device_info intel_sandybridge_m_info = {
>  	.has_aliasing_ppgtt = 1, \
>  	.has_full_ppgtt = 1, \
>  	GEN_DEFAULT_PIPEOFFSETS, \
> +	GEN_DEFAULT_PAGE_SZ, \
>  	IVB_CURSOR_OFFSETS
>  
>  static const struct intel_device_info intel_ivybridge_d_info = {
> @@ -282,6 +292,7 @@ static const struct intel_device_info intel_valleyview_info = {
>  	.has_full_ppgtt = 1,
>  	.ring_mask = RENDER_RING | BSD_RING | BLT_RING,
>  	.display_mmio_offset = VLV_DISPLAY_BASE,
> +	GEN_DEFAULT_PAGE_SZ,
>  	GEN_DEFAULT_PIPEOFFSETS,
>  	CURSOR_OFFSETS
>  };
> @@ -308,7 +319,8 @@ static const struct intel_device_info intel_haswell_info = {
>  	BDW_COLORS, \
>  	.has_logical_ring_contexts = 1, \
>  	.has_full_48bit_ppgtt = 1, \
> -	.has_64bit_reloc = 1
> +	.has_64bit_reloc = 1, \
> +	.page_size_mask = I915_GTT_PAGE_SIZE | I915_GTT_PAGE_SIZE_2M | I915_GTT_PAGE_SIZE_1G
>  
>  static const struct intel_device_info intel_broadwell_info = {
>  	BDW_FEATURES,
> @@ -341,13 +353,18 @@ static const struct intel_device_info intel_cherryview_info = {
>  	.has_aliasing_ppgtt = 1,
>  	.has_full_ppgtt = 1,
>  	.display_mmio_offset = VLV_DISPLAY_BASE,
> +	.page_size_mask = I915_GTT_PAGE_SIZE | I915_GTT_PAGE_SIZE_64K | I915_GTT_PAGE_SIZE_2M | I915_GTT_PAGE_SIZE_1G,
>  	GEN_CHV_PIPEOFFSETS,
>  	CURSOR_OFFSETS,
>  	CHV_COLORS,
>  };
>  
> +#define GEN9_DEFAULT_PAGE_SZ \
> +	.page_size_mask = I915_GTT_PAGE_SIZE | I915_GTT_PAGE_SIZE_64K | I915_GTT_PAGE_SIZE_2M | I915_GTT_PAGE_SIZE_1G
> +
>  static const struct intel_device_info intel_skylake_info = {
>  	BDW_FEATURES,
> +	GEN9_DEFAULT_PAGE_SZ,
>  	.platform = INTEL_SKYLAKE,
>  	.gen = 9,
>  	.has_csr = 1,
> @@ -357,6 +374,7 @@ static const struct intel_device_info intel_skylake_info = {
>  
>  static const struct intel_device_info intel_skylake_gt3_info = {
>  	BDW_FEATURES,
> +	GEN9_DEFAULT_PAGE_SZ,
>  	.platform = INTEL_SKYLAKE,
>  	.gen = 9,
>  	.has_csr = 1,
> @@ -389,6 +407,7 @@ static const struct intel_device_info intel_skylake_gt3_info = {
>  	.has_aliasing_ppgtt = 1, \
>  	.has_full_ppgtt = 1, \
>  	.has_full_48bit_ppgtt = 1, \
> +	GEN9_DEFAULT_PAGE_SZ, \
>  	GEN_DEFAULT_PIPEOFFSETS, \
>  	IVB_CURSOR_OFFSETS, \
>  	BDW_COLORS
> @@ -409,6 +428,7 @@ static const struct intel_device_info intel_geminilake_info = {
>  
>  static const struct intel_device_info intel_kabylake_info = {
>  	BDW_FEATURES,
> +	GEN9_DEFAULT_PAGE_SZ,
>  	.platform = INTEL_KABYLAKE,
>  	.gen = 9,
>  	.has_csr = 1,
> @@ -418,6 +438,7 @@ static const struct intel_device_info intel_kabylake_info = {
>  
>  static const struct intel_device_info intel_kabylake_gt3_info = {
>  	BDW_FEATURES,
> +	GEN9_DEFAULT_PAGE_SZ,
>  	.platform = INTEL_KABYLAKE,
>  	.gen = 9,
>  	.has_csr = 1,
> -- 
> 2.9.3
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH 05/15] drm/i915: introduce drm_i915_gem_object page_size member

[Tvrtko Ursulin]

Hi,

On 06/03/2017 23:54, Matthew Auld wrote:
> Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> ---
>  drivers/gpu/drm/i915/i915_gem.c        | 2 ++
>  drivers/gpu/drm/i915/i915_gem_object.h | 2 ++
>  2 files changed, 4 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 7c20601fe1de..0a6ed2c54629 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -3975,6 +3975,8 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
>
>  	obj->ops = ops;
>
> +	obj->page_size = I915_GTT_PAGE_SIZE;
> +
>  	reservation_object_init(&obj->__builtin_resv);
>  	obj->resv = &obj->__builtin_resv;
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_object.h b/drivers/gpu/drm/i915/i915_gem_object.h
> index 33b0dc4782a9..31bcf87d3745 100644
> --- a/drivers/gpu/drm/i915/i915_gem_object.h
> +++ b/drivers/gpu/drm/i915/i915_gem_object.h
> @@ -90,6 +90,8 @@ struct drm_i915_gem_object {
>
>  	unsigned long flags;
>
> +	unsigned long page_size; /* 4K(default), 64K, 2M, 1G */
> +

Could we get away with an unsigned int for this field and the colouring, 
function arguments, etc following in subsequent patches? Or it is 
expected that >4G pages might happen in some reasonable time? Maybe a 
pointless saving, I don't know, just asking.

Regards,

Tvrtko

>  	/**
>  	 * Have we taken a reference for the object for incomplete GPU
>  	 * activity?
>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH 06/15] drm/i915: pass page_size to insert_entries

[Tvrtko Ursulin]

On 06/03/2017 23:54, Matthew Auld wrote:
> Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> ---
>  drivers/gpu/drm/i915/i915_gem_gtt.c           | 32 ++++++++++++++++++++++-----
>  drivers/gpu/drm/i915/i915_gem_gtt.h           |  1 +
>  drivers/gpu/drm/i915/selftests/i915_gem_gtt.c |  3 ++-
>  drivers/gpu/drm/i915/selftests/mock_gtt.c     |  1 +
>  4 files changed, 30 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
> index cee9c4fec52a..9274974956b3 100644
> --- a/drivers/gpu/drm/i915/i915_gem_gtt.c
> +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
> @@ -205,7 +205,7 @@ static int ppgtt_bind_vma(struct i915_vma *vma,
>  		pte_flags |= PTE_READ_ONLY;
>
>  	vma->vm->insert_entries(vma->vm, vma->pages, vma->node.start,
> -				cache_level, pte_flags);
> +				vma->obj->page_size, cache_level, pte_flags);
>
>  	return 0;
>  }
> @@ -906,6 +906,7 @@ gen8_ppgtt_insert_pte_entries(struct i915_hw_ppgtt *ppgtt,
>  static void gen8_ppgtt_insert_3lvl(struct i915_address_space *vm,
>  				   struct sg_table *pages,
>  				   u64 start,
> +				   unsigned long page_size,
>  				   enum i915_cache_level cache_level,
>  				   u32 unused)
>  {
> @@ -924,6 +925,7 @@ static void gen8_ppgtt_insert_3lvl(struct i915_address_space *vm,
>  static void gen8_ppgtt_insert_4lvl(struct i915_address_space *vm,
>  				   struct sg_table *pages,
>  				   u64 start,
> +				   unsigned long page_size,
>  				   enum i915_cache_level cache_level,
>  				   u32 unused)
>  {
> @@ -935,9 +937,23 @@ static void gen8_ppgtt_insert_4lvl(struct i915_address_space *vm,
>  	};
>  	struct i915_page_directory_pointer **pdps = ppgtt->pml4.pdps;
>  	struct gen8_insert_pte idx = gen8_insert_pte(start);
> +	bool (*insert_entries)(struct i915_hw_ppgtt *ppgtt,
> +			       struct i915_page_directory_pointer *pdp,
> +			       struct sgt_dma *iter,
> +			       struct gen8_insert_pte *idx,
> +			       enum i915_cache_level cache_level);
> +
> +	switch (page_size) {
> +	case I915_GTT_PAGE_SIZE:
> +		insert_entries = gen8_ppgtt_insert_pte_entries;
> +		break;
> +	default:
> +		MISSING_CASE(page_size);
> +		return;
> +	}

Hm, if you stored obj->page_size as an enum of known possible sizes, 
then perhaps we could have an array of vm->insert_entries indexed by 
that enum. You would then pass around this enum (could also use for 
colouring I guess?) and instead of the above, maybe in ppgtt_bind_vma 
just do something like 
vma->vm->insert_entries[vma->obj->page_size_id](...). Or something like 
that. Just thinking on how keep the vfunc approach as much as possible.

Regards,

Tvrtko

>
> -	while (gen8_ppgtt_insert_pte_entries(ppgtt, pdps[idx.pml4e++], &iter,
> -					     &idx, cache_level))
> +	while (insert_entries(ppgtt, pdps[idx.pml4e++], &iter, &idx,
> +			      cache_level))
>  		GEM_BUG_ON(idx.pml4e >= GEN8_PML4ES_PER_PML4);
>  }
>
> @@ -1620,6 +1636,7 @@ static void gen6_ppgtt_clear_range(struct i915_address_space *vm,
>  static void gen6_ppgtt_insert_entries(struct i915_address_space *vm,
>  				      struct sg_table *pages,
>  				      u64 start,
> +				      unsigned long page_size,
>  				      enum i915_cache_level cache_level,
>  				      u32 flags)
>  {
> @@ -2093,6 +2110,7 @@ static void gen8_ggtt_insert_page(struct i915_address_space *vm,
>  static void gen8_ggtt_insert_entries(struct i915_address_space *vm,
>  				     struct sg_table *st,
>  				     u64 start,
> +				     unsigned long page_size,
>  				     enum i915_cache_level level,
>  				     u32 unused)
>  {
> @@ -2140,6 +2158,7 @@ static void gen6_ggtt_insert_page(struct i915_address_space *vm,
>  static void gen6_ggtt_insert_entries(struct i915_address_space *vm,
>  				     struct sg_table *st,
>  				     u64 start,
> +				     unsigned long page_size,
>  				     enum i915_cache_level level,
>  				     u32 flags)
>  {
> @@ -2224,6 +2243,7 @@ static void i915_ggtt_insert_page(struct i915_address_space *vm,
>  static void i915_ggtt_insert_entries(struct i915_address_space *vm,
>  				     struct sg_table *pages,
>  				     u64 start,
> +				     unsigned long page_size,
>  				     enum i915_cache_level cache_level,
>  				     u32 unused)
>  {
> @@ -2260,7 +2280,7 @@ static int ggtt_bind_vma(struct i915_vma *vma,
>
>  	intel_runtime_pm_get(i915);
>  	vma->vm->insert_entries(vma->vm, vma->pages, vma->node.start,
> -				cache_level, pte_flags);
> +				I915_GTT_PAGE_SIZE, cache_level, pte_flags);
>  	intel_runtime_pm_put(i915);
>
>  	/*
> @@ -2314,14 +2334,14 @@ static int aliasing_gtt_bind_vma(struct i915_vma *vma,
>
>  		appgtt->base.insert_entries(&appgtt->base,
>  					    vma->pages, vma->node.start,
> -					    cache_level, pte_flags);
> +					    I915_GTT_PAGE_SIZE, cache_level, pte_flags);
>  	}
>
>  	if (flags & I915_VMA_GLOBAL_BIND) {
>  		intel_runtime_pm_get(i915);
>  		vma->vm->insert_entries(vma->vm,
>  					vma->pages, vma->node.start,
> -					cache_level, pte_flags);
> +					I915_GTT_PAGE_SIZE, cache_level, pte_flags);
>  		intel_runtime_pm_put(i915);
>  	}
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
> index 6c90a2ffd0e1..1f51402cf816 100644
> --- a/drivers/gpu/drm/i915/i915_gem_gtt.h
> +++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
> @@ -328,6 +328,7 @@ struct i915_address_space {
>  	void (*insert_entries)(struct i915_address_space *vm,
>  			       struct sg_table *st,
>  			       u64 start,
> +			       unsigned long page_size,
>  			       enum i915_cache_level cache_level,
>  			       u32 flags);
>  	void (*cleanup)(struct i915_address_space *vm);
> diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
> index c4a39b7d7dd0..c7963efe46ba 100644
> --- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
> +++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c
> @@ -256,7 +256,8 @@ static int lowlevel_hole(struct drm_i915_private *i915,
>  				break;
>
>  			vm->insert_entries(vm, obj->mm.pages, addr,
> -					   I915_CACHE_NONE, 0);
> +					   I915_GTT_PAGE_SIZE, I915_CACHE_NONE,
> +					   0);
>  		}
>  		count = n;
>
> diff --git a/drivers/gpu/drm/i915/selftests/mock_gtt.c b/drivers/gpu/drm/i915/selftests/mock_gtt.c
> index a61309c7cb3e..a2137100d2f5 100644
> --- a/drivers/gpu/drm/i915/selftests/mock_gtt.c
> +++ b/drivers/gpu/drm/i915/selftests/mock_gtt.c
> @@ -35,6 +35,7 @@ static void mock_insert_page(struct i915_address_space *vm,
>  static void mock_insert_entries(struct i915_address_space *vm,
>  				struct sg_table *st,
>  				u64 start,
> +				unsigned long page_size,
>  				enum i915_cache_level level, u32 flags)
>  {
>  }
>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH 10/15] drm/i915: introduce ppgtt page coloring

[Chris Wilson]

On Mon, Mar 06, 2017 at 11:54:09PM +0000, Matthew Auld wrote:
> To enable 64K pages we need to set the intermediate-page-size(IPS) bit
> of the pde, therefore a page table is said to be either operating in 64K
> or 4K mode. To accommodate this vm placement restriction we introduce a
> color for pages and corresponding color_adjust callback. Currently this
> only considers the full 48bit ppgtt case.

Why? That's just an alignment restriction.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH 08/15] drm/i915: clean up cache coloring

[Mika Kuoppala]

Matthew Auld <matthew.auld@intel.com> writes:

> To keep the next patch simple, rid the code of any mm.color_adjust
> assumptions to allow adding another flavour of coloring.
>
> Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> ---
>  drivers/gpu/drm/i915/i915_drv.h       |  2 +-
>  drivers/gpu/drm/i915/i915_gem.c       |  3 ++-
>  drivers/gpu/drm/i915/i915_gem_evict.c | 12 +++++-------
>  drivers/gpu/drm/i915/i915_gem_gtt.h   |  6 ++++++
>  drivers/gpu/drm/i915/i915_vma.c       | 10 +++++++---
>  5 files changed, 21 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index e45b8d74cebf..aac764b5aad4 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -3526,7 +3526,7 @@ int i915_perf_open_ioctl(struct drm_device *dev, void *data,
>  /* i915_gem_evict.c */
>  int __must_check i915_gem_evict_something(struct i915_address_space *vm,
>  					  u64 min_size, u64 alignment,
> -					  unsigned cache_level,
> +					  unsigned long color,
>  					  u64 start, u64 end,
>  					  unsigned flags);
>  int __must_check i915_gem_evict_for_node(struct i915_address_space *vm,
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 0a6ed2c54629..9acf279e5f93 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -3392,7 +3392,8 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
>  		obj->cache_dirty = true;
>  
>  	list_for_each_entry(vma, &obj->vma_list, obj_link)
> -		vma->node.color = cache_level;
> +		if (i915_uses_cache_coloring(vma->vm))
> +			vma->node.color = cache_level;
>  	obj->cache_level = cache_level;
>  
>  	return 0;
> diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c
> index 2da3a94fc9f3..f9364f917b67 100644
> --- a/drivers/gpu/drm/i915/i915_gem_evict.c
> +++ b/drivers/gpu/drm/i915/i915_gem_evict.c
> @@ -74,7 +74,7 @@ mark_free(struct drm_mm_scan *scan,
>   * @vm: address space to evict from
>   * @min_size: size of the desired free space
>   * @alignment: alignment constraint of the desired free space
> - * @cache_level: cache_level for the desired space
> + * @color: color for the desired space
>   * @start: start (inclusive) of the range from which to evict objects
>   * @end: end (exclusive) of the range from which to evict objects
>   * @flags: additional flags to control the eviction algorithm
> @@ -95,7 +95,7 @@ mark_free(struct drm_mm_scan *scan,
>  int
>  i915_gem_evict_something(struct i915_address_space *vm,
>  			 u64 min_size, u64 alignment,
> -			 unsigned cache_level,
> +			 unsigned long color,
>  			 u64 start, u64 end,
>  			 unsigned flags)
>  {
> @@ -134,7 +134,7 @@ i915_gem_evict_something(struct i915_address_space *vm,
>  	if (flags & PIN_MAPPABLE)
>  		mode = DRM_MM_INSERT_LOW;
>  	drm_mm_scan_init_with_range(&scan, &vm->mm,
> -				    min_size, alignment, cache_level,
> +				    min_size, alignment, color,
>  				    start, end, mode);
>  
>  	/* Retire before we search the active list. Although we have
> @@ -254,7 +254,6 @@ int i915_gem_evict_for_node(struct i915_address_space *vm,
>  	u64 start = target->start;
>  	u64 end = start + target->size;
>  	struct i915_vma *vma, *next;
> -	bool check_color;
>  	int ret = 0;
>  
>  	lockdep_assert_held(&vm->i915->drm.struct_mutex);
> @@ -271,8 +270,7 @@ int i915_gem_evict_for_node(struct i915_address_space *vm,
>  	if (!(flags & PIN_NONBLOCK))
>  		i915_gem_retire_requests(vm->i915);
>  
> -	check_color = vm->mm.color_adjust;
> -	if (check_color) {
> +	if (i915_uses_cache_coloring(vm)) {
>  		/* Expand search to cover neighbouring guard pages (or lack!) */
>  		if (start)
>  			start -= I915_GTT_PAGE_SIZE;
> @@ -298,7 +296,7 @@ int i915_gem_evict_for_node(struct i915_address_space *vm,
>  		 * abutt and conflict. If they are in conflict, then we evict
>  		 * those as well to make room for our guard pages.
>  		 */
> -		if (check_color) {
> +		if (i915_uses_cache_coloring(vm)) {
>  			if (node->start + node->size == target->start) {
>  				if (node->color == target->color)
>  					continue;
> diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
> index 1f51402cf816..8d7436105718 100644
> --- a/drivers/gpu/drm/i915/i915_gem_gtt.h
> +++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
> @@ -346,6 +346,12 @@ struct i915_address_space {
>  #define i915_is_ggtt(V) (!(V)->file)
>  
>  static inline bool
> +i915_uses_cache_coloring(const struct i915_address_space *vm)

i915_vm_uses_color()
i915_vm_has_coloring()

Just trying to think of better naming as this is
vm property.

-Mika

> +{
> +	return vm->mm.color_adjust && i915_is_ggtt(vm);
> +}
> +
> +static inline bool
>  i915_vm_is_48bit(const struct i915_address_space *vm)
>  {
>  	return (vm->total - 1) >> 32;
> diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
> index 1aba47024656..31e2327492ba 100644
> --- a/drivers/gpu/drm/i915/i915_vma.c
> +++ b/drivers/gpu/drm/i915/i915_vma.c
> @@ -390,7 +390,7 @@ bool i915_gem_valid_gtt_space(struct i915_vma *vma, unsigned long cache_level)
>  	 * these constraints apply and set the drm_mm.color_adjust
>  	 * appropriately.
>  	 */
> -	if (vma->vm->mm.color_adjust == NULL)
> +	if (!i915_uses_cache_coloring(vma->vm))
>  		return true;
>  
>  	/* Only valid to be called on an already inserted vma */
> @@ -429,6 +429,7 @@ i915_vma_insert(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
>  	struct drm_i915_gem_object *obj = vma->obj;
>  	u64 start, end;
>  	int ret;
> +	unsigned long color = 0;
>  
>  	GEM_BUG_ON(vma->flags & (I915_VMA_GLOBAL_BIND | I915_VMA_LOCAL_BIND));
>  	GEM_BUG_ON(drm_mm_node_allocated(&vma->node));
> @@ -471,6 +472,9 @@ i915_vma_insert(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
>  	if (ret)
>  		return ret;
>  
> +	if (i915_uses_cache_coloring(vma->vm))
> +		color = obj->cache_level;
> +
>  	if (flags & PIN_OFFSET_FIXED) {
>  		u64 offset = flags & PIN_OFFSET_MASK;
>  		if (!IS_ALIGNED(offset, alignment) ||
> @@ -480,13 +484,13 @@ i915_vma_insert(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
>  		}
>  
>  		ret = i915_gem_gtt_reserve(vma->vm, &vma->node,
> -					   size, offset, obj->cache_level,
> +					   size, offset, color,
>  					   flags);
>  		if (ret)
>  			goto err_unpin;
>  	} else {
>  		ret = i915_gem_gtt_insert(vma->vm, &vma->node,
> -					  size, alignment, obj->cache_level,
> +					  size, alignment, color,
>  					  start, end, flags);
>  		if (ret)
>  			goto err_unpin;
> -- 
> 2.9.3
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH 09/15] drm/i915: export color_differs

[Mika Kuoppala]

Matthew Auld <matthew.auld@intel.com> writes:

> Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> ---
>  drivers/gpu/drm/i915/i915_vma.c | 9 ++-------
>  drivers/gpu/drm/i915/i915_vma.h | 6 ++++++
>  2 files changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
> index 31e2327492ba..0d289cbc0123 100644
> --- a/drivers/gpu/drm/i915/i915_vma.c
> +++ b/drivers/gpu/drm/i915/i915_vma.c
> @@ -373,11 +373,6 @@ void __i915_vma_set_map_and_fenceable(struct i915_vma *vma)
>  		vma->flags &= ~I915_VMA_CAN_FENCE;
>  }
>  
> -static bool color_differs(struct drm_mm_node *node, unsigned long color)
> -{
> -	return node->allocated && node->color != color;
> -}
> -
>  bool i915_gem_valid_gtt_space(struct i915_vma *vma, unsigned long cache_level)
>  {
>  	struct drm_mm_node *node = &vma->node;
> @@ -398,11 +393,11 @@ bool i915_gem_valid_gtt_space(struct i915_vma *vma, unsigned long cache_level)
>  	GEM_BUG_ON(list_empty(&node->node_list));
>  
>  	other = list_prev_entry(node, node_list);
> -	if (color_differs(other, cache_level) && !drm_mm_hole_follows(other))
> +	if (i915_color_differs(other, cache_level) && !drm_mm_hole_follows(other))
>  		return false;
>  
>  	other = list_next_entry(node, node_list);
> -	if (color_differs(other, cache_level) && !drm_mm_hole_follows(node))
> +	if (i915_color_differs(other, cache_level) && !drm_mm_hole_follows(node))
>  		return false;
>  
>  	return true;
> diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h
> index 2e03f81dddbe..5492b03d6a77 100644
> --- a/drivers/gpu/drm/i915/i915_vma.h
> +++ b/drivers/gpu/drm/i915/i915_vma.h
> @@ -284,6 +284,12 @@ static inline void i915_vma_unpin(struct i915_vma *vma)
>  	__i915_vma_unpin(vma);
>  }
>  
> +static inline bool i915_color_differs(const struct drm_mm_node *node,
> +				      unsigned long color)

Consider i915_node_color_differs().
-Mika


> +{
> +	return node->allocated && node->color != color;
> +}
> +
>  /**
>   * i915_vma_pin_iomap - calls ioremap_wc to map the GGTT VMA via the aperture
>   * @vma: VMA to iomap
> -- 
> 2.9.3
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [RFC PATCH 00/15] drm/i915: initial support for huge gtt pages

[Chris Wilson]

On Mon, Mar 06, 2017 at 11:53:59PM +0000, Matthew Auld wrote:
> This series adds support for huge-pages for the gtt, where "huge"
> is 64K, 2M and 1G. This isn't everything I have and there are still some
> things which I have yet to implement, like handling evict-for-node with the
> 64K/4K trickiness, but the hope here is to get some early feedback if possible.
> 
> One open question I still have is how the page-size should be handled at the
> gem object level, should the page-size be an implementation detail of whichever
> backend the gem object uses, where the selected page-size would solely depend
> on the size of the object and the availability of huge pages, or do we intend
> to expose some kind of hinting, both within our driver and possibly to
> userspace?

I thought the intention was to use optimistic superpages in the GTT (and
so optimistic large allocations).

One question I have is how do you mix swiotlb limits? We currently have
a trial-and-error approach to whittle down the sg chunks, which prevents
having a predetermined pagesize (at least as far as the GTT is
concerned).
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH 02/15] drm/i915: use correct node for handling cache domain eviction

[Chris Wilson]

On Mon, Mar 06, 2017 at 11:54:01PM +0000, Matthew Auld wrote:
> It looks like we were incorrectly comparing vma->node against itself
> instead of the target node, when evicting for a node on systems where we
> need guard pages between regions with different cache domains. As a
> consequence we can end up trying to needlessly evict neighbouring nodes,
> even if they have the same cache domain, and if they were pinned we
> would fail the eviction.
> 
> Fixes: 625d988acc28 ("drm/i915: Extract reserving space in the GTT to a helper")
> Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> Cc: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>

Lost my /o\ :)
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH 03/15] drm/i915/selftests: exercise cache domain eviction

[Chris Wilson]

On Mon, Mar 06, 2017 at 11:54:02PM +0000, Matthew Auld wrote:

Add a selftest to exercise evicting neighbouring nodes that conflict due
to page colouring in the GTT.

> v2: add a peppering of comments
> 
> Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH 04/15] drm/i915: add page_size_mask to dev_info

[Chris Wilson]

On Mon, Mar 06, 2017 at 11:54:03PM +0000, Matthew Auld wrote:
> Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> ---
>  drivers/gpu/drm/i915/i915_drv.h     |  1 +
>  drivers/gpu/drm/i915/i915_gem_gtt.h | 14 ++++++++++++++
>  drivers/gpu/drm/i915/i915_pci.c     | 23 ++++++++++++++++++++++-
>  3 files changed, 37 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index 1fd4128a10b1..e45b8d74cebf 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -913,6 +913,7 @@ struct intel_device_info {
>  	enum intel_platform platform;
>  	u8 ring_mask; /* Rings supported by the HW */
>  	u8 num_rings;
> +	unsigned long page_size_mask; /* page sizes supported by the HW */
>  #define DEFINE_FLAG(name) u8 name:1
>  	DEV_INFO_FOR_EACH_FLAG(DEFINE_FLAG);
>  #undef DEFINE_FLAG
> diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
> index fb15684c1d83..6c90a2ffd0e1 100644
> --- a/drivers/gpu/drm/i915/i915_gem_gtt.h
> +++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
> @@ -43,8 +43,18 @@
>  #include "i915_selftest.h"
>  
>  #define I915_GTT_PAGE_SIZE 4096UL
> +#define I915_GTT_PAGE_SIZE_64K 65536UL
> +#define I915_GTT_PAGE_SIZE_2M 2097152UL
> +#define I915_GTT_PAGE_SIZE_1G 1073741824UL

I915_GTT_PAGE_SIZE_4K BIT(12)
I915_GTT_PAGE_SIZE_64K BIT(16)
I915_GTT_PAGE_SIZE_2M BIT(21)
I915_GTT_PAGE_SIZE_1G BIT(30)

#define I915_GTT_PAGE_SIZE I915_GTT_PAGE_SIZE_4K

Still debating the relative merits of a tight enum.

Note that you want to scatter

#define assert_valid_gtt_page_size(page_size) \
GEM_BUG_ON(!is_power_of_2(page_size) || \
	   page_size & ~I915_GTT_PAGE_SIZE_MASK);

around
or GEM_BUG_ON(is_valid_gtt_page_size(page_size))?
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH 03/15] drm/i915/selftests: exercise cache domain eviction

[Chris Wilson]

On Tue, Mar 07, 2017 at 10:06:51AM +0000, Chris Wilson wrote:
> On Mon, Mar 06, 2017 at 11:54:02PM +0000, Matthew Auld wrote:
> 
> Add a selftest to exercise evicting neighbouring nodes that conflict due
> to page colouring in the GTT.
> 
> > v2: add a peppering of comments
> > 
> > Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> > Cc: Chris Wilson <chris@chris-wilson.co.uk>
> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>

Pushed the first 3 fixes, thanks.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx