RE: [PATCH v2 04/15] drm/i915: Bypass LMEMBAR/GTTMMADR for MTL stolen memory access

From: "Sripada, Radhakrishna" <radhakrishna.sripada@intel.com>
To: Ville Syrjala <ville.syrjala@linux.intel.com>,
	"intel-gfx@lists.freedesktop.org"
	<intel-gfx@lists.freedesktop.org>
Cc: "Das, Nirmoy" <nirmoy.das@intel.com>
Subject: RE: [PATCH v2 04/15] drm/i915: Bypass LMEMBAR/GTTMMADR for MTL stolen memory access
Date: Fri, 15 Dec 2023 21:58:41 +0000	[thread overview]
Message-ID: <DM4PR11MB5971E6B8BB644DE98E4764248793A@DM4PR11MB5971.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20231215105929.29568-5-ville.syrjala@linux.intel.com>


> -----Original Message-----
> From: Ville Syrjala <ville.syrjala@linux.intel.com>
> Sent: Friday, December 15, 2023 2:59 AM
> To: intel-gfx@lists.freedesktop.org
> Cc: Paz Zcharya <pazz@chromium.org>; Das, Nirmoy <nirmoy.das@intel.com>;
> Sripada, Radhakrishna <radhakrishna.sripada@intel.com>; Joonas Lahtinen
> <joonas.lahtinen@linux.intel.com>
> Subject: [PATCH v2 04/15] drm/i915: Bypass LMEMBAR/GTTMMADR for MTL
> stolen memory access
> 
> From: Ville Syrjälä <ville.syrjala@linux.intel.com>
> 
> On MTL accessing stolen memory via the BARs is somehow borked,
> and it can hang the machine. As a workaround let's bypass the
> BARs and just go straight to DSMBASE/GSMBASE instead.
> 
> Note that on every other platform this itself would hang the
> machine, but on MTL the system firmware is expected to relax
> the access permission guarding stolen memory to enable this
> workaround, and thus direct CPU accesses should be fine.
> 
> TODO: add w/a numbers and whatnot
Wa_22018444074 is more appropriate here.

With that,
Reviewed-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>

> 
> Cc: Paz Zcharya <pazz@chromium.org>
> Cc: Nirmoy Das <nirmoy.das@intel.com>
> Cc: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
> ---
>  drivers/gpu/drm/i915/gem/i915_gem_stolen.c | 11 ++++++++++-
>  drivers/gpu/drm/i915/gt/intel_ggtt.c       | 13 ++++++++++++-
>  2 files changed, 22 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c
> b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c
> index ee237043c302..252fe5cd6ede 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_stolen.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_stolen.c
> @@ -941,7 +941,16 @@ i915_gem_stolen_lmem_setup(struct drm_i915_private
> *i915, u16 type,
>  		dsm_size = ALIGN_DOWN(lmem_size - dsm_base, SZ_1M);
>  	}
> 
> -	if (pci_resource_len(pdev, GEN12_LMEM_BAR) < lmem_size) {
> +	if (IS_METEORLAKE(i915)) {
> +		/*
> +		 * Workaround: access via BAR can hang MTL, go directly to
> DSM.
> +		 *
> +		 * Normally this would not work but on MTL the system
> firmware
> +		 * should have relaxed the access permissions sufficiently.
> +		 */
> +		io_start = intel_uncore_read64(uncore, GEN12_DSMBASE) &
> GEN12_BDSM_MASK;
> +		io_size = dsm_size;
> +	} else if (pci_resource_len(pdev, GEN12_LMEM_BAR) < lmem_size) {
>  		io_start = 0;
>  		io_size = 0;
>  	} else {
> diff --git a/drivers/gpu/drm/i915/gt/intel_ggtt.c
> b/drivers/gpu/drm/i915/gt/intel_ggtt.c
> index 21a7e3191c18..ab71d74ec426 100644
> --- a/drivers/gpu/drm/i915/gt/intel_ggtt.c
> +++ b/drivers/gpu/drm/i915/gt/intel_ggtt.c
> @@ -24,6 +24,7 @@
>  #include "intel_ring.h"
>  #include "i915_drv.h"
>  #include "i915_pci.h"
> +#include "i915_reg.h"
>  #include "i915_request.h"
>  #include "i915_scatterlist.h"
>  #include "i915_utils.h"
> @@ -1152,13 +1153,23 @@ static unsigned int gen6_gttadr_offset(struct
> drm_i915_private *i915)
>  static int ggtt_probe_common(struct i915_ggtt *ggtt, u64 size)
>  {
>  	struct drm_i915_private *i915 = ggtt->vm.i915;
> +	struct intel_uncore *uncore = ggtt->vm.gt->uncore;
>  	struct pci_dev *pdev = to_pci_dev(i915->drm.dev);
>  	phys_addr_t phys_addr;
>  	u32 pte_flags;
>  	int ret;
> 
>  	GEM_WARN_ON(pci_resource_len(pdev, GEN4_GTTMMADR_BAR) !=
> gen6_gttmmadr_size(i915));
> -	phys_addr = pci_resource_start(pdev, GEN4_GTTMMADR_BAR) +
> gen6_gttadr_offset(i915);
> +	/*
> +	 * Workaround: access via BAR can hang MTL, go directly to GSM.
> +	 *
> +	 * Normally this would not work but on MTL the system firmware
> +	 * should have relaxed the access permissions sufficiently.
> +	 */
> +	if (IS_METEORLAKE(i915))
> +		phys_addr = intel_uncore_read64(uncore, GEN12_GSMBASE) &
> GEN12_BDSM_MASK;
> +	else
> +		phys_addr = pci_resource_start(pdev, GEN4_GTTMMADR_BAR)
> + gen6_gttadr_offset(i915);
> 
>  	if (needs_wc_ggtt_mapping(i915))
>  		ggtt->gsm = ioremap_wc(phys_addr, size);
> --
> 2.41.0