From: Matthew Auld <matthew.auld@intel.com> To: "Thomas Hellström" <thomas.hellstrom@linux.intel.com>, intel-gfx@lists.freedesktop.org, "Bloomfield, Jon" <jon.bloomfield@intel.com> Cc: dri-devel@lists.freedesktop.org Subject: Re: [PATCH 17/20] drm/i915/uapi: add NEEDS_CPU_ACCESS hint Date: Thu, 3 Feb 2022 11:38:17 +0000 [thread overview] Message-ID: <79378fce-e917-f5f7-d133-92b73a0e7b91@intel.com> (raw) In-Reply-To: <e9007965-a122-9f4d-e0fd-cfd2a8e1fb3d@linux.intel.com> On 03/02/2022 09:28, Thomas Hellström wrote: > > On 1/26/22 16:21, Matthew Auld wrote: >> If set, force the allocation to be placed in the mappable portion of >> LMEM. One big restriction here is that system memory must be given as a >> potential placement for the object, that way we can always spill the >> object into system memory if we can't make space. >> >> XXX: Still very much WIP and needs IGTs. Including now just for the sake >> of having more complete picture. >> >> Signed-off-by: Matthew Auld <matthew.auld@intel.com> >> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> >> --- >> drivers/gpu/drm/i915/gem/i915_gem_create.c | 28 ++++++++++++------- >> include/uapi/drm/i915_drm.h | 31 +++++++++++++++++++++- >> 2 files changed, 49 insertions(+), 10 deletions(-) >> >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_create.c >> b/drivers/gpu/drm/i915/gem/i915_gem_create.c >> index e7456443f163..98d63cb21e94 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_create.c >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_create.c >> @@ -238,6 +238,7 @@ struct create_ext { >> struct drm_i915_private *i915; >> struct intel_memory_region *placements[INTEL_REGION_UNKNOWN]; >> unsigned int n_placements; >> + unsigned int placement_mask; >> unsigned long flags; >> }; >> @@ -334,6 +335,7 @@ static int set_placements(struct >> drm_i915_gem_create_ext_memory_regions *args, >> for (i = 0; i < args->num_regions; i++) >> ext_data->placements[i] = placements[i]; >> + ext_data->placement_mask = mask; >> return 0; >> out_dump: >> @@ -408,7 +410,7 @@ i915_gem_create_ext_ioctl(struct drm_device *dev, >> void *data, >> struct drm_i915_gem_object *obj; >> int ret; >> - if (args->flags) >> + if (args->flags & ~I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS) >> return -EINVAL; >> ret = i915_user_extensions(u64_to_user_ptr(args->extensions), >> @@ -424,14 +426,22 @@ i915_gem_create_ext_ioctl(struct drm_device >> *dev, void *data, >> ext_data.n_placements = 1; >> } >> - /* >> - * TODO: add a userspace hint to force CPU_ACCESS for the object, >> which >> - * can override this. >> - */ >> - if (!IS_DG1(i915) && (ext_data.n_placements > 1 || >> - ext_data.placements[0]->type != >> - INTEL_MEMORY_SYSTEM)) >> - ext_data.flags |= I915_BO_ALLOC_TOPDOWN; >> + if (args->flags & I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS) { >> + if (ext_data.n_placements == 1) >> + return -EINVAL; >> + >> + /* >> + * We always need to be able to spill to system memory, if we >> + * can't place in the mappable part of LMEM. >> + */ >> + if (!(ext_data.placement_mask & BIT(INTEL_REGION_SMEM))) >> + return -EINVAL; >> + } else { >> + if (!IS_DG1(i915) && (ext_data.n_placements > 1 || >> + ext_data.placements[0]->type != >> + INTEL_MEMORY_SYSTEM)) >> + ext_data.flags |= I915_BO_ALLOC_TOPDOWN; >> + } >> obj = __i915_gem_object_create_user_ext(i915, args->size, >> ext_data.placements, >> diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h >> index 914ebd9290e5..ecfa805549a7 100644 >> --- a/include/uapi/drm/i915_drm.h >> +++ b/include/uapi/drm/i915_drm.h >> @@ -3157,7 +3157,36 @@ struct drm_i915_gem_create_ext { >> * Object handles are nonzero. >> */ >> __u32 handle; >> - /** @flags: MBZ */ >> + /** >> + * @flags: Optional flags. >> + * >> + * Supported values: >> + * >> + * I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS - Signal to the >> kernel that >> + * the object will need to be accessed via the CPU. >> + * >> + * Only valid when placing objects in I915_MEMORY_CLASS_DEVICE, and >> + * only strictly required on platforms where only some of the device >> + * memory is directly visible or mappable through the CPU, like >> on DG2+. >> + * >> + * One of the placements MUST also be I915_MEMORY_CLASS_SYSTEM, to >> + * ensure we can always spill the allocation to system memory, if we >> + * can't place the object in the mappable part of >> + * I915_MEMORY_CLASS_DEVICE. >> + * >> + * Note that buffers that need to be captured with >> EXEC_OBJECT_CAPTURE, >> + * will need to enable this hint, if the object can also be >> placed in >> + * I915_MEMORY_CLASS_DEVICE, starting from DG2+. The execbuf call >> will >> + * throw an error otherwise. This also means that such objects >> will need >> + * I915_MEMORY_CLASS_SYSTEM set as a possible placement. >> + * > > I wonder, should we try to migrate capture objects at execbuf time > instead on an on-demand basis? If migration fails, then we just skip > capturing that object, similar to how the capture code handles errors? So IIUC if the object has been marked for capture, unmark the TOPDOWN annotation, if it has been set, to force allocating in the mappable portion, or spill to system memory(if the placements allow it)? I think that should work. Jon any thoughts? > >> + * Without this hint, the kernel will assume that non-mappable >> + * I915_MEMORY_CLASS_DEVICE is preferred for this object. Note >> that the >> + * kernel can still migrate the object to the mappable part, as a >> last >> + * resort, if userspace ever CPU faults this object, but this >> might be >> + * expensive, and so ideally should be avoided. >> + */ >> +#define I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS (1<<0) >> __u32 flags; >> /** >> * @extensions: The chain of extensions to apply to this object.
WARNING: multiple messages have this Message-ID (diff)
From: Matthew Auld <matthew.auld@intel.com> To: "Thomas Hellström" <thomas.hellstrom@linux.intel.com>, intel-gfx@lists.freedesktop.org, "Bloomfield, Jon" <jon.bloomfield@intel.com> Cc: dri-devel@lists.freedesktop.org Subject: Re: [Intel-gfx] [PATCH 17/20] drm/i915/uapi: add NEEDS_CPU_ACCESS hint Date: Thu, 3 Feb 2022 11:38:17 +0000 [thread overview] Message-ID: <79378fce-e917-f5f7-d133-92b73a0e7b91@intel.com> (raw) In-Reply-To: <e9007965-a122-9f4d-e0fd-cfd2a8e1fb3d@linux.intel.com> On 03/02/2022 09:28, Thomas Hellström wrote: > > On 1/26/22 16:21, Matthew Auld wrote: >> If set, force the allocation to be placed in the mappable portion of >> LMEM. One big restriction here is that system memory must be given as a >> potential placement for the object, that way we can always spill the >> object into system memory if we can't make space. >> >> XXX: Still very much WIP and needs IGTs. Including now just for the sake >> of having more complete picture. >> >> Signed-off-by: Matthew Auld <matthew.auld@intel.com> >> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> >> --- >> drivers/gpu/drm/i915/gem/i915_gem_create.c | 28 ++++++++++++------- >> include/uapi/drm/i915_drm.h | 31 +++++++++++++++++++++- >> 2 files changed, 49 insertions(+), 10 deletions(-) >> >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_create.c >> b/drivers/gpu/drm/i915/gem/i915_gem_create.c >> index e7456443f163..98d63cb21e94 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_create.c >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_create.c >> @@ -238,6 +238,7 @@ struct create_ext { >> struct drm_i915_private *i915; >> struct intel_memory_region *placements[INTEL_REGION_UNKNOWN]; >> unsigned int n_placements; >> + unsigned int placement_mask; >> unsigned long flags; >> }; >> @@ -334,6 +335,7 @@ static int set_placements(struct >> drm_i915_gem_create_ext_memory_regions *args, >> for (i = 0; i < args->num_regions; i++) >> ext_data->placements[i] = placements[i]; >> + ext_data->placement_mask = mask; >> return 0; >> out_dump: >> @@ -408,7 +410,7 @@ i915_gem_create_ext_ioctl(struct drm_device *dev, >> void *data, >> struct drm_i915_gem_object *obj; >> int ret; >> - if (args->flags) >> + if (args->flags & ~I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS) >> return -EINVAL; >> ret = i915_user_extensions(u64_to_user_ptr(args->extensions), >> @@ -424,14 +426,22 @@ i915_gem_create_ext_ioctl(struct drm_device >> *dev, void *data, >> ext_data.n_placements = 1; >> } >> - /* >> - * TODO: add a userspace hint to force CPU_ACCESS for the object, >> which >> - * can override this. >> - */ >> - if (!IS_DG1(i915) && (ext_data.n_placements > 1 || >> - ext_data.placements[0]->type != >> - INTEL_MEMORY_SYSTEM)) >> - ext_data.flags |= I915_BO_ALLOC_TOPDOWN; >> + if (args->flags & I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS) { >> + if (ext_data.n_placements == 1) >> + return -EINVAL; >> + >> + /* >> + * We always need to be able to spill to system memory, if we >> + * can't place in the mappable part of LMEM. >> + */ >> + if (!(ext_data.placement_mask & BIT(INTEL_REGION_SMEM))) >> + return -EINVAL; >> + } else { >> + if (!IS_DG1(i915) && (ext_data.n_placements > 1 || >> + ext_data.placements[0]->type != >> + INTEL_MEMORY_SYSTEM)) >> + ext_data.flags |= I915_BO_ALLOC_TOPDOWN; >> + } >> obj = __i915_gem_object_create_user_ext(i915, args->size, >> ext_data.placements, >> diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h >> index 914ebd9290e5..ecfa805549a7 100644 >> --- a/include/uapi/drm/i915_drm.h >> +++ b/include/uapi/drm/i915_drm.h >> @@ -3157,7 +3157,36 @@ struct drm_i915_gem_create_ext { >> * Object handles are nonzero. >> */ >> __u32 handle; >> - /** @flags: MBZ */ >> + /** >> + * @flags: Optional flags. >> + * >> + * Supported values: >> + * >> + * I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS - Signal to the >> kernel that >> + * the object will need to be accessed via the CPU. >> + * >> + * Only valid when placing objects in I915_MEMORY_CLASS_DEVICE, and >> + * only strictly required on platforms where only some of the device >> + * memory is directly visible or mappable through the CPU, like >> on DG2+. >> + * >> + * One of the placements MUST also be I915_MEMORY_CLASS_SYSTEM, to >> + * ensure we can always spill the allocation to system memory, if we >> + * can't place the object in the mappable part of >> + * I915_MEMORY_CLASS_DEVICE. >> + * >> + * Note that buffers that need to be captured with >> EXEC_OBJECT_CAPTURE, >> + * will need to enable this hint, if the object can also be >> placed in >> + * I915_MEMORY_CLASS_DEVICE, starting from DG2+. The execbuf call >> will >> + * throw an error otherwise. This also means that such objects >> will need >> + * I915_MEMORY_CLASS_SYSTEM set as a possible placement. >> + * > > I wonder, should we try to migrate capture objects at execbuf time > instead on an on-demand basis? If migration fails, then we just skip > capturing that object, similar to how the capture code handles errors? So IIUC if the object has been marked for capture, unmark the TOPDOWN annotation, if it has been set, to force allocating in the mappable portion, or spill to system memory(if the placements allow it)? I think that should work. Jon any thoughts? > >> + * Without this hint, the kernel will assume that non-mappable >> + * I915_MEMORY_CLASS_DEVICE is preferred for this object. Note >> that the >> + * kernel can still migrate the object to the mappable part, as a >> last >> + * resort, if userspace ever CPU faults this object, but this >> might be >> + * expensive, and so ideally should be avoided. >> + */ >> +#define I915_GEM_CREATE_EXT_FLAG_NEEDS_CPU_ACCESS (1<<0) >> __u32 flags; >> /** >> * @extensions: The chain of extensions to apply to this object.
next prev parent reply other threads:[~2022-02-03 11:38 UTC|newest] Thread overview: 105+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-01-26 15:21 [PATCH 00/20] Initial support for small BAR recovery Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-01-26 15:21 ` [PATCH 01/20] drm: improve drm_buddy_alloc function Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-01-26 18:03 ` Jani Nikula 2022-01-26 15:21 ` [PATCH 02/20] drm: implement top-down allocation method Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-01-26 18:42 ` Robert Beckett 2022-01-26 15:21 ` [PATCH 03/20] drm: implement a method to free unused pages Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-01-26 15:21 ` [PATCH 04/20] drm/i915: add io_size plumbing Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-01-31 15:14 ` Thomas Hellström 2022-01-31 15:14 ` [Intel-gfx] " Thomas Hellström 2022-01-26 15:21 ` [PATCH 05/20] drm/i915/ttm: require mappable by default Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-01-26 15:21 ` [PATCH 06/20] drm/i915: add I915_BO_ALLOC_TOPDOWN Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-01-31 15:28 ` Thomas Hellström 2022-01-31 15:28 ` [Intel-gfx] " Thomas Hellström 2022-01-31 15:49 ` Matthew Auld 2022-01-31 15:49 ` [Intel-gfx] " Matthew Auld 2022-01-26 15:21 ` [PATCH 07/20] drm/i915/buddy: track available visible size Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-01-31 16:12 ` Thomas Hellström 2022-01-31 16:12 ` [Intel-gfx] " Thomas Hellström 2022-01-26 15:21 ` [PATCH 08/20] drm/i915/buddy: adjust res->start Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-02-01 10:38 ` Thomas Hellström 2022-02-01 10:38 ` [Intel-gfx] " Thomas Hellström 2022-01-26 15:21 ` [PATCH 09/20] drm/i915/buddy: tweak 2big check Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-02-01 10:39 ` Thomas Hellström 2022-02-01 10:39 ` [Intel-gfx] " Thomas Hellström 2022-01-26 15:21 ` [Intel-gfx] [PATCH 10/20] drm/i915/selftests: mock test io_size Matthew Auld 2022-01-26 15:21 ` Matthew Auld 2022-02-02 10:24 ` Thomas Hellström 2022-02-02 10:24 ` [Intel-gfx] " Thomas Hellström 2022-01-26 15:21 ` [PATCH 11/20] drm/i915/ttm: tweak priority hint selection Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-02-02 13:34 ` Thomas Hellström 2022-02-02 13:34 ` [Intel-gfx] " Thomas Hellström 2022-01-26 15:21 ` [Intel-gfx] [PATCH 12/20] drm/i915/ttm: make eviction mappable aware Matthew Auld 2022-01-26 15:21 ` Matthew Auld 2022-02-02 13:41 ` Thomas Hellström 2022-02-02 13:41 ` [Intel-gfx] " Thomas Hellström 2022-01-26 15:21 ` [PATCH 13/20] drm/i915/ttm: mappable migration on fault Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-02-03 7:59 ` Thomas Hellström 2022-02-03 7:59 ` [Intel-gfx] " Thomas Hellström 2022-01-26 15:21 ` [Intel-gfx] [PATCH 14/20] drm/i915/selftests: exercise mmap migration Matthew Auld 2022-01-26 15:21 ` Matthew Auld 2022-02-03 9:01 ` Thomas Hellström 2022-02-03 9:01 ` [Intel-gfx] " Thomas Hellström 2022-02-03 9:12 ` Matthew Auld 2022-02-03 9:12 ` [Intel-gfx] " Matthew Auld 2022-01-26 15:21 ` [PATCH 15/20] drm/i915/selftests: handle allocation failures Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-02-03 9:05 ` Thomas Hellström 2022-02-03 9:05 ` [Intel-gfx] " Thomas Hellström 2022-02-03 9:11 ` Matthew Auld 2022-02-03 9:11 ` [Intel-gfx] " Matthew Auld 2022-01-26 15:21 ` [PATCH 16/20] drm/i915/create: apply ALLOC_TOPDOWN by default Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-02-03 9:17 ` Thomas Hellström 2022-02-03 9:17 ` [Intel-gfx] " Thomas Hellström 2022-02-03 9:32 ` Matthew Auld 2022-02-03 9:32 ` [Intel-gfx] " Matthew Auld 2022-01-26 15:21 ` [PATCH 17/20] drm/i915/uapi: add NEEDS_CPU_ACCESS hint Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-02-03 9:28 ` Thomas Hellström 2022-02-03 9:28 ` [Intel-gfx] " Thomas Hellström 2022-02-03 11:38 ` Matthew Auld [this message] 2022-02-03 11:38 ` Matthew Auld 2022-02-03 13:29 ` Thomas Hellström 2022-02-03 13:29 ` [Intel-gfx] " Thomas Hellström 2022-01-26 15:21 ` [PATCH 18/20] drm/i915/uapi: forbid ALLOC_TOPDOWN for error capture Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-01-26 19:42 ` kernel test robot 2022-01-26 19:42 ` kernel test robot 2022-01-26 20:03 ` kernel test robot 2022-01-26 20:03 ` kernel test robot 2022-01-26 20:03 ` kernel test robot 2022-02-03 9:43 ` Thomas Hellström 2022-02-03 9:43 ` [Intel-gfx] " Thomas Hellström 2022-02-03 9:44 ` Matthew Auld 2022-02-03 9:44 ` [Intel-gfx] " Matthew Auld 2022-01-26 15:21 ` [PATCH 19/20] drm/i915/lmem: don't treat small BAR as an error Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-02-03 9:48 ` Thomas Hellström 2022-02-03 9:48 ` [Intel-gfx] " Thomas Hellström 2022-02-03 11:18 ` Matthew Auld 2022-02-03 11:18 ` [Intel-gfx] " Matthew Auld 2022-02-03 13:56 ` Thomas Hellström 2022-02-03 13:56 ` [Intel-gfx] " Thomas Hellström 2022-02-03 14:09 ` Matthew Auld 2022-02-03 14:09 ` [Intel-gfx] " Matthew Auld 2022-01-26 15:21 ` [PATCH 20/20] HAX: DG1 small BAR Matthew Auld 2022-01-26 15:21 ` [Intel-gfx] " Matthew Auld 2022-01-26 21:07 ` [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Initial support for small BAR recovery Patchwork 2022-01-26 21:08 ` [Intel-gfx] ✗ Fi.CI.SPARSE: " Patchwork 2022-01-26 21:41 ` [Intel-gfx] ✗ Fi.CI.BAT: failure " Patchwork 2022-01-27 16:27 ` [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Initial support for small BAR recovery (rev2) Patchwork 2022-01-27 16:28 ` [Intel-gfx] ✗ Fi.CI.SPARSE: " Patchwork 2022-01-27 16:57 ` [Intel-gfx] ✗ Fi.CI.BAT: failure " Patchwork
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=79378fce-e917-f5f7-d133-92b73a0e7b91@intel.com \ --to=matthew.auld@intel.com \ --cc=dri-devel@lists.freedesktop.org \ --cc=intel-gfx@lists.freedesktop.org \ --cc=jon.bloomfield@intel.com \ --cc=thomas.hellstrom@linux.intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.