intel-gfx.lists.freedesktop.org archive mirror
 help / color / mirror / Atom feed
From: Matthew Brost <matthew.brost@intel.com>
To: <intel-gfx@lists.freedesktop.org>, <dri-devel@lists.freedesktop.org>
Subject: [Intel-gfx] [PATCH 18/33] drm/i915/guc: Capture error state on context reset
Date: Mon, 26 Jul 2021 17:23:33 -0700	[thread overview]
Message-ID: <20210727002348.97202-19-matthew.brost@intel.com> (raw)
In-Reply-To: <20210727002348.97202-1-matthew.brost@intel.com>

We receive notification of an engine reset from GuC at its
completion. Meaning GuC has potentially cleared any HW state
we may have been interested in capturing. GuC resumes scheduling
on the engine post-reset, as the resets are meant to be transparent,
further muddling our error state.

There is ongoing work to define an API for a GuC debug state dump. The
suggestion for now is to manually disable FW initiated resets in cases
where debug state is needed.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
 drivers/gpu/drm/i915/gt/intel_context.c       | 20 +++++++++++
 drivers/gpu/drm/i915/gt/intel_context.h       |  3 ++
 drivers/gpu/drm/i915/gt/intel_engine.h        | 21 ++++++++++-
 drivers/gpu/drm/i915/gt/intel_engine_cs.c     | 11 ++++--
 drivers/gpu/drm/i915/gt/intel_engine_types.h  |  2 ++
 .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 35 +++++++++----------
 drivers/gpu/drm/i915/i915_gpu_error.c         | 25 ++++++++++---
 7 files changed, 91 insertions(+), 26 deletions(-)

diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 0bf4a13e9759..237b70e98744 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -509,6 +509,26 @@ struct i915_request *intel_context_create_request(struct intel_context *ce)
 	return rq;
 }
 
+struct i915_request *intel_context_find_active_request(struct intel_context *ce)
+{
+	struct i915_request *rq, *active = NULL;
+	unsigned long flags;
+
+	GEM_BUG_ON(!intel_engine_uses_guc(ce->engine));
+
+	spin_lock_irqsave(&ce->guc_active.lock, flags);
+	list_for_each_entry_reverse(rq, &ce->guc_active.requests,
+				    sched.link) {
+		if (i915_request_completed(rq))
+			break;
+
+		active = rq;
+	}
+	spin_unlock_irqrestore(&ce->guc_active.lock, flags);
+
+	return active;
+}
+
 #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
 #include "selftest_context.c"
 #endif
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index 974ef85320c2..2ed9bf5f91a5 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -200,6 +200,9 @@ int intel_context_prepare_remote_request(struct intel_context *ce,
 
 struct i915_request *intel_context_create_request(struct intel_context *ce);
 
+struct i915_request *
+intel_context_find_active_request(struct intel_context *ce);
+
 static inline bool intel_context_is_barrier(const struct intel_context *ce)
 {
 	return test_bit(CONTEXT_BARRIER_BIT, &ce->flags);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
index 8fc76dc8bf98..1db2d3efc71f 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine.h
@@ -245,7 +245,7 @@ ktime_t intel_engine_get_busy_time(struct intel_engine_cs *engine,
 				   ktime_t *now);
 
 struct i915_request *
-intel_engine_find_active_request(struct intel_engine_cs *engine);
+intel_engine_execlist_find_hung_request(struct intel_engine_cs *engine);
 
 u32 intel_engine_context_size(struct intel_gt *gt, u8 class);
 struct intel_context *
@@ -313,4 +313,23 @@ intel_engine_get_sibling(struct intel_engine_cs *engine, unsigned int sibling)
 	return engine->cops->get_sibling(engine, sibling);
 }
 
+static inline void
+intel_engine_set_hung_context(struct intel_engine_cs *engine,
+			      struct intel_context *ce)
+{
+	engine->hung_ce = ce;
+}
+
+static inline void
+intel_engine_clear_hung_context(struct intel_engine_cs *engine)
+{
+	intel_engine_set_hung_context(engine, NULL);
+}
+
+static inline struct intel_context *
+intel_engine_get_hung_context(struct intel_engine_cs *engine)
+{
+	return engine->hung_ce;
+}
+
 #endif /* _INTEL_RINGBUFFER_H_ */
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 1eaa658507e1..0da7868c5a13 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -1731,7 +1731,7 @@ void intel_engine_dump(struct intel_engine_cs *engine,
 	drm_printf(m, "\tRequests:\n");
 
 	spin_lock_irqsave(&engine->sched_engine->lock, flags);
-	rq = intel_engine_find_active_request(engine);
+	rq = intel_engine_execlist_find_hung_request(engine);
 	if (rq) {
 		struct intel_timeline *tl = get_timeline(rq);
 
@@ -1842,10 +1842,17 @@ static bool match_ring(struct i915_request *rq)
 }
 
 struct i915_request *
-intel_engine_find_active_request(struct intel_engine_cs *engine)
+intel_engine_execlist_find_hung_request(struct intel_engine_cs *engine)
 {
 	struct i915_request *request, *active = NULL;
 
+	/*
+	 * This search does not work in GuC submission mode. However, the GuC
+	 * will report the hanging context directly to the driver itself. So
+	 * the driver should never get here when in GuC mode.
+	 */
+	GEM_BUG_ON(intel_uc_uses_guc_submission(&engine->gt->uc));
+
 	/*
 	 * We are called by the error capture, reset and to dump engine
 	 * state at random points in time. In particular, note that neither is
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index 1c7e2724cdae..260cce15cb62 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -298,6 +298,8 @@ struct intel_engine_cs {
 	/* keep a request in reserve for a [pm] barrier under oom */
 	struct i915_request *request_pool;
 
+	struct intel_context *hung_ce;
+
 	struct llist_head barrier_tasks;
 
 	struct intel_context *kernel_context; /* pinned */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index d76494e3e271..7d6ca0d54f9e 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -726,24 +726,6 @@ __unwind_incomplete_requests(struct intel_context *ce)
 	spin_unlock_irqrestore(&sched_engine->lock, flags);
 }
 
-static struct i915_request *context_find_active_request(struct intel_context *ce)
-{
-	struct i915_request *rq, *active = NULL;
-	unsigned long flags;
-
-	spin_lock_irqsave(&ce->guc_active.lock, flags);
-	list_for_each_entry_reverse(rq, &ce->guc_active.requests,
-				    sched.link) {
-		if (i915_request_completed(rq))
-			break;
-
-		active = rq;
-	}
-	spin_unlock_irqrestore(&ce->guc_active.lock, flags);
-
-	return active;
-}
-
 static void __guc_reset_context(struct intel_context *ce, bool stalled)
 {
 	struct i915_request *rq;
@@ -757,7 +739,7 @@ static void __guc_reset_context(struct intel_context *ce, bool stalled)
 	 */
 	clr_context_enabled(ce);
 
-	rq = context_find_active_request(ce);
+	rq = intel_context_find_active_request(ce);
 	if (!rq) {
 		head = ce->ring->tail;
 		stalled = false;
@@ -2201,6 +2183,20 @@ int intel_guc_sched_done_process_msg(struct intel_guc *guc,
 	return 0;
 }
 
+static void capture_error_state(struct intel_guc *guc,
+				struct intel_context *ce)
+{
+	struct intel_gt *gt = guc_to_gt(guc);
+	struct drm_i915_private *i915 = gt->i915;
+	struct intel_engine_cs *engine = __context_to_physical_engine(ce);
+	intel_wakeref_t wakeref;
+
+	intel_engine_set_hung_context(engine, ce);
+	with_intel_runtime_pm(&i915->runtime_pm, wakeref)
+		i915_capture_error_state(gt, engine->mask);
+	atomic_inc(&i915->gpu_error.reset_engine_count[engine->uabi_class]);
+}
+
 static void guc_context_replay(struct intel_context *ce)
 {
 	struct i915_sched_engine *sched_engine = ce->engine->sched_engine;
@@ -2213,6 +2209,7 @@ static void guc_handle_context_reset(struct intel_guc *guc,
 				     struct intel_context *ce)
 {
 	trace_intel_context_reset(ce);
+	capture_error_state(guc, ce);
 	guc_context_replay(ce);
 }
 
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index a2c58b54a592..0f08bcfbe964 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -1429,20 +1429,37 @@ capture_engine(struct intel_engine_cs *engine,
 {
 	struct intel_engine_capture_vma *capture = NULL;
 	struct intel_engine_coredump *ee;
-	struct i915_request *rq;
+	struct intel_context *ce;
+	struct i915_request *rq = NULL;
 	unsigned long flags;
 
 	ee = intel_engine_coredump_alloc(engine, GFP_KERNEL);
 	if (!ee)
 		return NULL;
 
-	spin_lock_irqsave(&engine->sched_engine->lock, flags);
-	rq = intel_engine_find_active_request(engine);
+	ce = intel_engine_get_hung_context(engine);
+	if (ce) {
+		intel_engine_clear_hung_context(engine);
+		rq = intel_context_find_active_request(ce);
+		if (!rq || !i915_request_started(rq))
+			goto no_request_capture;
+	} else {
+		/*
+		 * Getting here with GuC enabled means it is a forced error capture
+		 * with no actual hang. So, no need to attempt the execlist search.
+		 */
+		if (!intel_uc_uses_guc_submission(&engine->gt->uc)) {
+			spin_lock_irqsave(&engine->sched_engine->lock, flags);
+			rq = intel_engine_execlist_find_hung_request(engine);
+			spin_unlock_irqrestore(&engine->sched_engine->lock,
+					       flags);
+		}
+	}
 	if (rq)
 		capture = intel_engine_coredump_add_request(ee, rq,
 							    ATOMIC_MAYFAIL);
-	spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
 	if (!capture) {
+no_request_capture:
 		kfree(ee);
 		return NULL;
 	}
-- 
2.28.0

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

  parent reply	other threads:[~2021-07-27  0:07 UTC|newest]

Thread overview: 52+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-27  0:23 [Intel-gfx] [PATCH 00/33] Remaining patches for basic GuC submission Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 01/33] drm/i915/guc: GuC virtual engines Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 02/33] drm/i915/guc: Make hangcheck work with " Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 03/33] drm/i915: Hold reference to intel_context over life of i915_request Matthew Brost
2021-07-28  8:01   ` Daniel Vetter
2021-07-27  0:23 ` [Intel-gfx] [PATCH 04/33] drm/i915/guc: Disable bonding extension with GuC submission Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 05/33] drm/i915/guc: Direct all breadcrumbs for a class to single breadcrumbs Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 06/33] drm/i915: Add i915_sched_engine destroy vfunc Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 07/33] drm/i915: Move active request tracking to a vfunc Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 08/33] drm/i915/guc: Reset implementation for new GuC interface Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 09/33] drm/i915: Reset GPU immediately if submission is disabled Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 10/33] drm/i915/guc: Add disable interrupts to guc sanitize Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 11/33] drm/i915/guc: Suspend/resume implementation for new interface Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 12/33] drm/i915/guc: Handle context reset notification Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 13/33] drm/i915/guc: Handle engine reset failure notification Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 14/33] drm/i915/guc: Enable the timer expired interrupt for GuC Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 15/33] drm/i915/guc: Provide mmio list to be saved/restored on engine reset Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 16/33] drm/i915/guc: Don't complain about reset races Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 17/33] drm/i915/guc: Enable GuC engine reset Matthew Brost
2021-07-27  0:23 ` Matthew Brost [this message]
2021-07-27  0:23 ` [Intel-gfx] [PATCH 19/33] drm/i915/guc: Fix for error capture after full GPU reset with GuC Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 20/33] drm/i915/guc: Hook GuC scheduling policies up Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 21/33] drm/i915/guc: Connect reset modparam updates to GuC policy flags Matthew Brost
2021-08-26  8:55   ` Jani Nikula
2021-07-27  0:23 ` [Intel-gfx] [PATCH 22/33] drm/i915/guc: Include scheduling policies in the debugfs state dump Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 23/33] drm/i915/guc: Add golden context to GuC ADS Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 24/33] drm/i915/guc: Implement banned contexts for GuC submission Matthew Brost
2021-08-05 11:52   ` Tvrtko Ursulin
2021-08-25 10:39   ` Tvrtko Ursulin
2021-08-26  3:49     ` Matthew Brost
2021-08-26 11:27       ` Tvrtko Ursulin
2021-08-26 14:28         ` Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 25/33] drm/i915/guc: Support request cancellation Matthew Brost
2021-07-27 19:15   ` Daniele Ceraolo Spurio
2021-10-05  7:06     ` Sebastian Andrzej Siewior
2021-10-05 10:13       ` Tvrtko Ursulin
2021-10-05 10:58         ` Sebastian Andrzej Siewior
2021-07-27  0:23 ` [Intel-gfx] [PATCH 26/33] drm/i915/selftest: Better error reporting from hangcheck selftest Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 27/33] drm/i915/selftest: Fix workarounds selftest for GuC submission Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 28/33] drm/i915/selftest: Fix MOCS " Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 29/33] drm/i915/selftest: Increase some timeouts in live_requests Matthew Brost
2021-07-27 19:21   ` John Harrison
2021-07-27  0:23 ` [Intel-gfx] [PATCH 30/33] drm/i915/selftest: Fix hangcheck self test for GuC submission Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 31/33] drm/i915/selftest: Bump selftest timeouts for hangcheck Matthew Brost
2021-07-27  0:23 ` [Intel-gfx] [PATCH 32/33] drm/i915/guc: Implement GuC priority management Matthew Brost
2022-06-07 13:58   ` Tvrtko Ursulin
2022-06-07 22:40     ` John Harrison
2021-07-27  0:23 ` [Intel-gfx] [PATCH 33/33] drm/i915/guc: Unblock GuC submission on Gen11+ Matthew Brost
2021-07-27  0:34 ` [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Remaining patches for basic GuC submission (rev2) Patchwork
2021-07-27  1:04 ` [Intel-gfx] ✓ Fi.CI.BAT: success " Patchwork
2021-07-27  4:50 ` [Intel-gfx] ✓ Fi.CI.IGT: " Patchwork
  -- strict thread matches above, loose matches on Subject: below --
2021-07-22 23:53 [Intel-gfx] [PATCH 00/33] Remaining patches for basic GuC submission Matthew Brost
2021-07-22 23:54 ` [Intel-gfx] [PATCH 18/33] drm/i915/guc: Capture error state on context reset Matthew Brost

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210727002348.97202-19-matthew.brost@intel.com \
    --to=matthew.brost@intel.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=intel-gfx@lists.freedesktop.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).