* [PATCH] drm/i915: Abandon the reset if we fail to stop the engines
@ 2017-10-26 12:12 Chris Wilson
2017-10-26 12:33 ` ✗ Fi.CI.BAT: failure for " Patchwork
2017-10-26 12:59 ` [PATCH] " Ville Syrjälä
0 siblings, 2 replies; 6+ messages in thread
From: Chris Wilson @ 2017-10-26 12:12 UTC (permalink / raw)
To: intel-gfx
Some machines, *cough* snb *cough*, fail catastrophically if asked to
reset the GPU under certain conditions. The initial guess is that this
is when the rings are still busy at the time of the reset request
(because that's a pattern we've seen elsewhere, hence why we do try
gen3_stop_engines() before reset) so abandon the reset and leave the
device wedged, if gen3_stop_engines() fails.
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=103240
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
Whee! Let's see how much breaks!
-Chris
---
drivers/gpu/drm/i915/intel_uncore.c | 33 ++++++++++++++++++++++++++-------
1 file changed, 26 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c
index 20e3c65c0999..c9a254b6125f 100644
--- a/drivers/gpu/drm/i915/intel_uncore.c
+++ b/drivers/gpu/drm/i915/intel_uncore.c
@@ -1372,20 +1372,23 @@ int i915_reg_read_ioctl(struct drm_device *dev,
return ret;
}
-static void gen3_stop_engine(struct intel_engine_cs *engine)
+static bool gen3_stop_engine(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
const u32 base = engine->mmio_base;
const i915_reg_t mode = RING_MI_MODE(base);
+
I915_WRITE_FW(mode, _MASKED_BIT_ENABLE(STOP_RING));
if (intel_wait_for_register_fw(dev_priv,
mode,
MODE_IDLE,
MODE_IDLE,
- 500))
+ 500)) {
DRM_DEBUG_DRIVER("%s: timed out on STOP_RING\n",
engine->name);
+ return false;
+ }
I915_WRITE_FW(RING_CTL(base), 0);
I915_WRITE_FW(RING_HEAD(base), 0);
@@ -1395,19 +1398,32 @@ static void gen3_stop_engine(struct intel_engine_cs *engine)
if (I915_READ_FW(RING_HEAD(base)) != 0)
DRM_DEBUG_DRIVER("%s: ring head not parked\n",
engine->name);
+
+ return true;
}
-static void i915_stop_engines(struct drm_i915_private *dev_priv,
- unsigned engine_mask)
+static int i915_stop_engines(struct drm_i915_private *dev_priv,
+ unsigned engine_mask)
{
struct intel_engine_cs *engine;
enum intel_engine_id id;
+ bool idle;
if (INTEL_GEN(dev_priv) < 3)
- return;
+ return true;
+ idle = true;
for_each_engine_masked(engine, dev_priv, engine_mask, id)
- gen3_stop_engine(engine);
+ idle &= gen3_stop_engine(engine);
+ if (idle)
+ return idle;
+
+ dev_err(dev_priv->drm.dev, "Failed to stop all engines\n");
+ for_each_engine_masked(engine, dev_priv, engine_mask, id) {
+ struct drm_printer p = drm_debug_printer(__func__);
+ intel_engine_dump(engine, &p);
+ }
+ return false;
}
static bool i915_reset_complete(struct pci_dev *pdev)
@@ -1768,7 +1784,10 @@ int intel_gpu_reset(struct drm_i915_private *dev_priv, unsigned engine_mask)
*
* FIXME: Wa for more modern gens needs to be validated
*/
- i915_stop_engines(dev_priv, engine_mask);
+ if (!i915_stop_engines(dev_priv, engine_mask)) {
+ ret = -EIO;
+ break;
+ }
ret = -ENODEV;
if (reset)
--
2.15.0.rc2
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 6+ messages in thread* ✗ Fi.CI.BAT: failure for drm/i915: Abandon the reset if we fail to stop the engines
2017-10-26 12:12 [PATCH] drm/i915: Abandon the reset if we fail to stop the engines Chris Wilson
@ 2017-10-26 12:33 ` Patchwork
2017-10-26 12:46 ` Chris Wilson
2017-10-26 12:59 ` [PATCH] " Ville Syrjälä
1 sibling, 1 reply; 6+ messages in thread
From: Patchwork @ 2017-10-26 12:33 UTC (permalink / raw)
To: Chris Wilson; +Cc: intel-gfx
== Series Details ==
Series: drm/i915: Abandon the reset if we fail to stop the engines
URL : https://patchwork.freedesktop.org/series/32692/
State : failure
== Summary ==
Series 32692v1 drm/i915: Abandon the reset if we fail to stop the engines
https://patchwork.freedesktop.org/api/1.0/series/32692/revisions/1/mbox/
Test core_auth:
Subgroup basic-auth:
pass -> SKIP (fi-ilk-650)
Test core_prop_blob:
Subgroup basic:
pass -> SKIP (fi-ilk-650)
Test debugfs_test:
Subgroup read_all_entries:
pass -> SKIP (fi-ilk-650)
Test drv_getparams_basic:
Subgroup basic-eu-total:
pass -> SKIP (fi-ilk-650)
Subgroup basic-subslice-total:
pass -> SKIP (fi-ilk-650)
Test drv_hangman:
Subgroup error-state-basic:
pass -> SKIP (fi-ilk-650)
Test gem_basic:
Subgroup bad-close:
pass -> SKIP (fi-ilk-650)
Subgroup create-close:
pass -> SKIP (fi-ilk-650)
Subgroup create-fd-close:
pass -> SKIP (fi-ilk-650)
Test gem_busy:
Subgroup basic-busy-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-hang-default:
pass -> SKIP (fi-ilk-650)
Test gem_close_race:
Subgroup basic-process:
pass -> SKIP (fi-ilk-650)
Subgroup basic-threads:
pass -> SKIP (fi-ilk-650)
Test gem_cpu_reloc:
Subgroup basic:
pass -> SKIP (fi-ilk-650)
Test gem_cs_tlb:
Subgroup basic-default:
pass -> SKIP (fi-ilk-650)
Test gem_exec_basic:
Subgroup basic-bsd:
pass -> SKIP (fi-ilk-650)
Subgroup basic-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-render:
pass -> SKIP (fi-ilk-650)
Subgroup gtt-bsd:
pass -> SKIP (fi-ilk-650)
Subgroup gtt-default:
pass -> SKIP (fi-ilk-650)
Subgroup gtt-render:
pass -> SKIP (fi-ilk-650)
Subgroup readonly-bsd:
pass -> SKIP (fi-ilk-650)
Subgroup readonly-default:
pass -> SKIP (fi-ilk-650)
Subgroup readonly-render:
pass -> SKIP (fi-ilk-650)
Test gem_exec_create:
Subgroup basic:
pass -> SKIP (fi-ilk-650)
Test gem_exec_fence:
Subgroup basic-busy-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-wait-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-await-default:
pass -> SKIP (fi-ilk-650)
Subgroup await-hang-default:
pass -> SKIP (fi-ilk-650)
Subgroup nb-await-default:
pass -> SKIP (fi-ilk-650)
Test gem_exec_flush:
Subgroup basic-batch-kernel-default-uc:
pass -> SKIP (fi-ilk-650)
Subgroup basic-batch-kernel-default-wb:
pass -> SKIP (fi-ilk-650)
Subgroup basic-uc-pro-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-uc-prw-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-uc-ro-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-uc-rw-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-uc-set-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-wb-pro-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-wb-prw-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-wb-ro-before-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-wb-ro-default:
pass -> SKIP (fi-ilk-650)
Subgroup basic-wb-rw-before-default:
WARNING: Long output truncated
fi-cnl-y failed to connect after reboot
df3033b174059a59aa0c890f81de8af037abd11f drm-tip: 2017y-10m-26d-11h-03m-59s UTC integration manifest
5d5082c1ab71 drm/i915: Abandon the reset if we fail to stop the engines
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_6202/
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: ✗ Fi.CI.BAT: failure for drm/i915: Abandon the reset if we fail to stop the engines
2017-10-26 12:33 ` ✗ Fi.CI.BAT: failure for " Patchwork
@ 2017-10-26 12:46 ` Chris Wilson
0 siblings, 0 replies; 6+ messages in thread
From: Chris Wilson @ 2017-10-26 12:46 UTC (permalink / raw)
To: Patchwork; +Cc: intel-gfx
Quoting Patchwork (2017-10-26 13:33:06)
> == Series Details ==
>
> Series: drm/i915: Abandon the reset if we fail to stop the engines
> URL : https://patchwork.freedesktop.org/series/32692/
> State : failure
>
> == Summary ==
>
> Series 32692v1 drm/i915: Abandon the reset if we fail to stop the engines
> https://patchwork.freedesktop.org/api/1.0/series/32692/revisions/1/mbox/
>
> Test core_auth:
> Subgroup basic-auth:
> pass -> SKIP (fi-ilk-650)
> Test core_prop_blob:
> Subgroup basic:
> pass -> SKIP (fi-ilk-650)
> Test debugfs_test:
> Subgroup read_all_entries:
> pass -> SKIP (fi-ilk-650)
> Test drv_getparams_basic:
> Subgroup basic-eu-total:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-subslice-total:
> pass -> SKIP (fi-ilk-650)
> Test drv_hangman:
> Subgroup error-state-basic:
> pass -> SKIP (fi-ilk-650)
> Test gem_basic:
> Subgroup bad-close:
> pass -> SKIP (fi-ilk-650)
> Subgroup create-close:
> pass -> SKIP (fi-ilk-650)
> Subgroup create-fd-close:
> pass -> SKIP (fi-ilk-650)
> Test gem_busy:
> Subgroup basic-busy-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-hang-default:
> pass -> SKIP (fi-ilk-650)
> Test gem_close_race:
> Subgroup basic-process:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-threads:
> pass -> SKIP (fi-ilk-650)
> Test gem_cpu_reloc:
> Subgroup basic:
> pass -> SKIP (fi-ilk-650)
> Test gem_cs_tlb:
> Subgroup basic-default:
> pass -> SKIP (fi-ilk-650)
> Test gem_exec_basic:
> Subgroup basic-bsd:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-render:
> pass -> SKIP (fi-ilk-650)
> Subgroup gtt-bsd:
> pass -> SKIP (fi-ilk-650)
> Subgroup gtt-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup gtt-render:
> pass -> SKIP (fi-ilk-650)
> Subgroup readonly-bsd:
> pass -> SKIP (fi-ilk-650)
> Subgroup readonly-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup readonly-render:
> pass -> SKIP (fi-ilk-650)
> Test gem_exec_create:
> Subgroup basic:
> pass -> SKIP (fi-ilk-650)
> Test gem_exec_fence:
> Subgroup basic-busy-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-wait-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-await-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup await-hang-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup nb-await-default:
> pass -> SKIP (fi-ilk-650)
> Test gem_exec_flush:
> Subgroup basic-batch-kernel-default-uc:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-batch-kernel-default-wb:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-uc-pro-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-uc-prw-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-uc-ro-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-uc-rw-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-uc-set-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-wb-pro-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-wb-prw-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-wb-ro-before-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-wb-ro-default:
> pass -> SKIP (fi-ilk-650)
> Subgroup basic-wb-rw-before-default:
> WARNING: Long output truncated
> fi-cnl-y failed to connect after reboot
That's suspiciously much less fallout than I expected. We have a severe
lack of GPU reset stress.
-Chris
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] drm/i915: Abandon the reset if we fail to stop the engines
2017-10-26 12:12 [PATCH] drm/i915: Abandon the reset if we fail to stop the engines Chris Wilson
2017-10-26 12:33 ` ✗ Fi.CI.BAT: failure for " Patchwork
@ 2017-10-26 12:59 ` Ville Syrjälä
2017-10-26 13:11 ` Chris Wilson
1 sibling, 1 reply; 6+ messages in thread
From: Ville Syrjälä @ 2017-10-26 12:59 UTC (permalink / raw)
To: Chris Wilson; +Cc: intel-gfx
On Thu, Oct 26, 2017 at 01:12:12PM +0100, Chris Wilson wrote:
> Some machines, *cough* snb *cough*, fail catastrophically if asked to
> reset the GPU under certain conditions.
Did we try skipping the gen6_rps_disable() already?
> The initial guess is that this
> is when the rings are still busy at the time of the reset request
> (because that's a pattern we've seen elsewhere, hence why we do try
> gen3_stop_engines() before reset) so abandon the reset and leave the
> device wedged, if gen3_stop_engines() fails.
>
> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=103240
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
> ---
> Whee! Let's see how much breaks!
> -Chris
> ---
> drivers/gpu/drm/i915/intel_uncore.c | 33 ++++++++++++++++++++++++++-------
> 1 file changed, 26 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c
> index 20e3c65c0999..c9a254b6125f 100644
> --- a/drivers/gpu/drm/i915/intel_uncore.c
> +++ b/drivers/gpu/drm/i915/intel_uncore.c
> @@ -1372,20 +1372,23 @@ int i915_reg_read_ioctl(struct drm_device *dev,
> return ret;
> }
>
> -static void gen3_stop_engine(struct intel_engine_cs *engine)
> +static bool gen3_stop_engine(struct intel_engine_cs *engine)
> {
> struct drm_i915_private *dev_priv = engine->i915;
> const u32 base = engine->mmio_base;
> const i915_reg_t mode = RING_MI_MODE(base);
>
> +
> I915_WRITE_FW(mode, _MASKED_BIT_ENABLE(STOP_RING));
> if (intel_wait_for_register_fw(dev_priv,
> mode,
> MODE_IDLE,
> MODE_IDLE,
> - 500))
> + 500)) {
> DRM_DEBUG_DRIVER("%s: timed out on STOP_RING\n",
> engine->name);
> + return false;
> + }
>
> I915_WRITE_FW(RING_CTL(base), 0);
> I915_WRITE_FW(RING_HEAD(base), 0);
> @@ -1395,19 +1398,32 @@ static void gen3_stop_engine(struct intel_engine_cs *engine)
> if (I915_READ_FW(RING_HEAD(base)) != 0)
> DRM_DEBUG_DRIVER("%s: ring head not parked\n",
> engine->name);
> +
> + return true;
> }
>
> -static void i915_stop_engines(struct drm_i915_private *dev_priv,
> - unsigned engine_mask)
> +static int i915_stop_engines(struct drm_i915_private *dev_priv,
> + unsigned engine_mask)
> {
> struct intel_engine_cs *engine;
> enum intel_engine_id id;
> + bool idle;
>
> if (INTEL_GEN(dev_priv) < 3)
> - return;
> + return true;
>
> + idle = true;
> for_each_engine_masked(engine, dev_priv, engine_mask, id)
> - gen3_stop_engine(engine);
> + idle &= gen3_stop_engine(engine);
> + if (idle)
> + return idle;
> +
> + dev_err(dev_priv->drm.dev, "Failed to stop all engines\n");
> + for_each_engine_masked(engine, dev_priv, engine_mask, id) {
> + struct drm_printer p = drm_debug_printer(__func__);
> + intel_engine_dump(engine, &p);
> + }
> + return false;
> }
>
> static bool i915_reset_complete(struct pci_dev *pdev)
> @@ -1768,7 +1784,10 @@ int intel_gpu_reset(struct drm_i915_private *dev_priv, unsigned engine_mask)
> *
> * FIXME: Wa for more modern gens needs to be validated
> */
> - i915_stop_engines(dev_priv, engine_mask);
> + if (!i915_stop_engines(dev_priv, engine_mask)) {
> + ret = -EIO;
> + break;
> + }
>
> ret = -ENODEV;
> if (reset)
> --
> 2.15.0.rc2
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
--
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] drm/i915: Abandon the reset if we fail to stop the engines
2017-10-26 12:59 ` [PATCH] " Ville Syrjälä
@ 2017-10-26 13:11 ` Chris Wilson
2017-10-26 13:30 ` Ville Syrjälä
0 siblings, 1 reply; 6+ messages in thread
From: Chris Wilson @ 2017-10-26 13:11 UTC (permalink / raw)
To: Ville Syrjälä; +Cc: intel-gfx
Quoting Ville Syrjälä (2017-10-26 13:59:05)
> On Thu, Oct 26, 2017 at 01:12:12PM +0100, Chris Wilson wrote:
> > Some machines, *cough* snb *cough*, fail catastrophically if asked to
> > reset the GPU under certain conditions.
>
> Did we try skipping the gen6_rps_disable() already?
I had thought we had taken that out a while ago...
commit f2a91d1a6f5960c08f1ca60bd076f4dc020c50c6
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date: Wed Sep 21 14:51:06 2016 +0100
drm/i915: Restore current RPS state after reset
removes the frobbing inside i915_reset() itself, but still talks about
RPS needing to be restored... Ok, that's the post-reset stuff to make
sure that the hw/sw tracking align.
We are not touching rc6/rps prior to hitting GDRST. Maybe we should?
(Unless I'm blind and overlooked something in the reset.)
-Chris
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] drm/i915: Abandon the reset if we fail to stop the engines
2017-10-26 13:11 ` Chris Wilson
@ 2017-10-26 13:30 ` Ville Syrjälä
0 siblings, 0 replies; 6+ messages in thread
From: Ville Syrjälä @ 2017-10-26 13:30 UTC (permalink / raw)
To: Chris Wilson; +Cc: intel-gfx
On Thu, Oct 26, 2017 at 02:11:22PM +0100, Chris Wilson wrote:
> Quoting Ville Syrjälä (2017-10-26 13:59:05)
> > On Thu, Oct 26, 2017 at 01:12:12PM +0100, Chris Wilson wrote:
> > > Some machines, *cough* snb *cough*, fail catastrophically if asked to
> > > reset the GPU under certain conditions.
> >
> > Did we try skipping the gen6_rps_disable() already?
>
> I had thought we had taken that out a while ago...
>
> commit f2a91d1a6f5960c08f1ca60bd076f4dc020c50c6
> Author: Chris Wilson <chris@chris-wilson.co.uk>
> Date: Wed Sep 21 14:51:06 2016 +0100
>
> drm/i915: Restore current RPS state after reset
>
> removes the frobbing inside i915_reset() itself, but still talks about
> RPS needing to be restored... Ok, that's the post-reset stuff to make
> sure that the hw/sw tracking align.
Hmm. Right. It looks like we do the disable+re-enable back to back after
the reset. I guess at that point it should be safe, assuming the reset
actually worked.
>
> We are not touching rc6/rps prior to hitting GDRST. Maybe we should?
Based on what I remember that would be more dangerous if the engined is
stuck in a bad way.
So I guess these reset problems are something else then.
--
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-10-26 13:30 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-10-26 12:12 [PATCH] drm/i915: Abandon the reset if we fail to stop the engines Chris Wilson
2017-10-26 12:33 ` ✗ Fi.CI.BAT: failure for " Patchwork
2017-10-26 12:46 ` Chris Wilson
2017-10-26 12:59 ` [PATCH] " Ville Syrjälä
2017-10-26 13:11 ` Chris Wilson
2017-10-26 13:30 ` Ville Syrjälä
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.