From: Matthew Brost <matthew.brost@intel.com>
To: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org
Subject: Re: [Intel-gfx] [PATCH 05/47] drm/i915/guc: Add stall timer to non blocking CTB send function
Date: Thu, 24 Jun 2021 16:01:03 -0700 [thread overview]
Message-ID: <20210624230102.GB20966@sdutt-i7> (raw)
In-Reply-To: <d8454345-bba9-7173-4d03-83581c6ee583@intel.com>
On Thu, Jun 24, 2021 at 07:37:01PM +0200, Michal Wajdeczko wrote:
>
>
> On 24.06.2021 09:04, Matthew Brost wrote:
> > Implement a stall timer which fails H2G CTBs once a period of time
> > with no forward progress is reached to prevent deadlock.
> >
> > Also update to ct_write to return -EIO rather than -EPIPE on a
> > corrupted descriptor.
>
> by doing so you will have the same error code for two different problems:
>
> a) corrupted CTB descriptor (definitely unrecoverable)
> b) long stall in CTB processing (still recoverable)
>
Already discussed both are treated exactly the same by the rest of the
stack so we return a single error code.
> while caller is explicitly instructed to retry only on:
>
> c) temporary stall in CTB processing (likely recoverable)
>
> so why do we want to limit our diagnostics?
>
> >
> > Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> > Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 47 +++++++++++++++++++++--
> > drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h | 4 ++
> > 2 files changed, 48 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> > index c9a65d05911f..27ec30b5ef47 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
> > @@ -319,6 +319,7 @@ int intel_guc_ct_enable(struct intel_guc_ct *ct)
> > goto err_deregister;
> >
> > ct->enabled = true;
> > + ct->stall_time = KTIME_MAX;
> >
> > return 0;
> >
> > @@ -392,7 +393,7 @@ static int ct_write(struct intel_guc_ct *ct,
> > unsigned int i;
> >
> > if (unlikely(ctb->broken))
> > - return -EPIPE;
> > + return -EIO;
> >
> > if (unlikely(desc->status))
> > goto corrupted;
> > @@ -464,7 +465,7 @@ static int ct_write(struct intel_guc_ct *ct,
> > CT_ERROR(ct, "Corrupted descriptor head=%u tail=%u status=%#x\n",
> > desc->head, desc->tail, desc->status);
> > ctb->broken = true;
> > - return -EPIPE;
> > + return -EIO;
> > }
> >
> > /**
> > @@ -507,6 +508,18 @@ static int wait_for_ct_request_update(struct ct_request *req, u32 *status)
> > return err;
> > }
> >
> > +#define GUC_CTB_TIMEOUT_MS 1500
>
> it's 150% of core CTB timeout, maybe we should correlate them ?
>
Seems overkill.
> > +static inline bool ct_deadlocked(struct intel_guc_ct *ct)
> > +{
> > + long timeout = GUC_CTB_TIMEOUT_MS;
> > + bool ret = ktime_ms_delta(ktime_get(), ct->stall_time) > timeout;
> > +
> > + if (unlikely(ret))
> > + CT_ERROR(ct, "CT deadlocked\n");
>
> nit: in commit message you said all these changes are to "prevent
> deadlock" so maybe this message should rather be:
>
> int delta = ktime_ms_delta(ktime_get(), ct->stall_time);
>
> CT_ERROR(ct, "Communication stalled for %dms\n", delta);
>
Sure.
> (note that CT_ERROR already adds "CT" prefix)
>
> > +
> > + return ret;
> > +}
> > +
> > static inline bool h2g_has_room(struct intel_guc_ct_buffer *ctb, u32 len_dw)
> > {
> > struct guc_ct_buffer_desc *desc = ctb->desc;
> > @@ -518,6 +531,26 @@ static inline bool h2g_has_room(struct intel_guc_ct_buffer *ctb, u32 len_dw)
> > return space >= len_dw;
> > }
> >
> > +static int has_room_nb(struct intel_guc_ct *ct, u32 len_dw)
> > +{
> > + struct intel_guc_ct_buffer *ctb = &ct->ctbs.send;
> > +
> > + lockdep_assert_held(&ct->ctbs.send.lock);
> > +
> > + if (unlikely(!h2g_has_room(ctb, len_dw))) {
> > + if (ct->stall_time == KTIME_MAX)
> > + ct->stall_time = ktime_get();
> > +
> > + if (unlikely(ct_deadlocked(ct)))
>
> and maybe above message should be printed somewhere around here when we
> detect "deadlock" for the first time?
>
Not sure I follow. The error message is in the correct place if ask me.
Probably should set the broken flag though when the message is printed
though.
> > + return -EIO;
> > + else
> > + return -EBUSY;
> > + }
> > +
> > + ct->stall_time = KTIME_MAX;
> > + return 0;
> > +}
> > +
> > static int ct_send_nb(struct intel_guc_ct *ct,
> > const u32 *action,
> > u32 len,
> > @@ -530,7 +563,7 @@ static int ct_send_nb(struct intel_guc_ct *ct,
> >
> > spin_lock_irqsave(&ctb->lock, spin_flags);
> >
> > - ret = h2g_has_room(ctb, len + 1);
> > + ret = has_room_nb(ct, len + 1);
> > if (unlikely(ret))
> > goto out;
> >
> > @@ -574,11 +607,19 @@ static int ct_send(struct intel_guc_ct *ct,
> > retry:
> > spin_lock_irqsave(&ct->ctbs.send.lock, flags);
> > if (unlikely(!h2g_has_room(ctb, len + 1))) {
> > + if (ct->stall_time == KTIME_MAX)
> > + ct->stall_time = ktime_get();
>
> as this is a repeated pattern, maybe it should be moved to h2g_has_room
> or other wrapper ?
>
Once we check G2H credits the pattern changes, hence the reason for this
outside of the wrapper. Also IMO sometimes we go a little overboard with
wrappers and it makes the code harder to understand.
> > spin_unlock_irqrestore(&ct->ctbs.send.lock, flags);
> > +
> > + if (unlikely(ct_deadlocked(ct)))
> > + return -EIO;
> > +
> > cond_resched();
> > goto retry;
> > }
> >
> > + ct->stall_time = KTIME_MAX;
>
> this one too
>
Same as above.
Matt
> > +
> > fence = ct_get_next_fence(ct);
> > request.fence = fence;
> > request.status = 0;
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h
> > index eb69263324ba..55ef7c52472f 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h
> > @@ -9,6 +9,7 @@
> > #include <linux/interrupt.h>
> > #include <linux/spinlock.h>
> > #include <linux/workqueue.h>
> > +#include <linux/ktime.h>
> >
> > #include "intel_guc_fwif.h"
> >
> > @@ -68,6 +69,9 @@ struct intel_guc_ct {
> > struct list_head incoming; /* incoming requests */
> > struct work_struct worker; /* handler for incoming requests */
> > } requests;
> > +
> > + /** @stall_time: time of first time a CTB submission is stalled */
> > + ktime_t stall_time;
> > };
> >
> > void intel_guc_ct_init_early(struct intel_guc_ct *ct);
> >
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
next prev parent reply other threads:[~2021-06-24 23:07 UTC|newest]
Thread overview: 170+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-24 7:04 [Intel-gfx] [PATCH 00/47] GuC submission support Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 01/47] drm/i915/guc: Relax CTB response timeout Matthew Brost
2021-06-24 17:23 ` Michal Wajdeczko
2021-06-24 7:04 ` [Intel-gfx] [PATCH 02/47] drm/i915/guc: Improve error message for unsolicited CT response Matthew Brost
2021-06-25 11:58 ` Michal Wajdeczko
2021-06-24 7:04 ` [Intel-gfx] [PATCH 03/47] drm/i915/guc: Increase size of CTB buffers Matthew Brost
2021-06-24 13:49 ` Michal Wajdeczko
2021-06-24 15:41 ` Matthew Brost
2021-06-25 12:03 ` Michal Wajdeczko
2021-06-24 7:04 ` [Intel-gfx] [PATCH 04/47] drm/i915/guc: Add non blocking CTB send function Matthew Brost
2021-06-24 14:48 ` Michal Wajdeczko
2021-06-24 15:49 ` Matthew Brost
2021-06-24 17:02 ` Michal Wajdeczko
2021-06-24 22:41 ` Matthew Brost
2021-06-25 11:50 ` Michal Wajdeczko
2021-06-25 17:53 ` Matthew Brost
2021-06-24 22:47 ` Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 05/47] drm/i915/guc: Add stall timer to " Matthew Brost
2021-06-24 17:37 ` Michal Wajdeczko
2021-06-24 23:01 ` Matthew Brost [this message]
2021-06-24 7:04 ` [Intel-gfx] [PATCH 06/47] drm/i915/guc: Optimize CTB writes and reads Matthew Brost
2021-06-25 13:09 ` Michal Wajdeczko
2021-06-25 18:26 ` Matthew Brost
2021-06-25 20:28 ` Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 07/47] drm/i915/guc: Module load failure test for CT buffer creation Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 08/47] drm/i915/guc: Add new GuC interface defines and structures Matthew Brost
2021-06-29 21:11 ` John Harrison
2021-06-30 0:30 ` Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 09/47] drm/i915/guc: Remove GuC stage descriptor, add lrc descriptor Matthew Brost
2021-06-25 19:44 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 10/47] drm/i915/guc: Add lrc descriptor context lookup array Matthew Brost
2021-06-25 13:17 ` Michal Wajdeczko
2021-06-25 17:26 ` Matthew Brost
2021-06-29 21:20 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 11/47] drm/i915/guc: Implement GuC submission tasklet Matthew Brost
2021-06-29 22:04 ` John Harrison
2021-06-30 0:41 ` Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 12/47] drm/i915/guc: Add bypass tasklet submission path to GuC Matthew Brost
2021-06-29 22:09 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 13/47] drm/i915/guc: Implement GuC context operations for new inteface Matthew Brost
2021-06-25 13:25 ` Michal Wajdeczko
2021-06-25 17:46 ` Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 14/47] drm/i915/guc: Insert fence on context when deregistering Matthew Brost
2021-07-09 22:39 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 15/47] drm/i915/guc: Defer context unpin until scheduling is disabled Matthew Brost
2021-07-09 22:48 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 16/47] drm/i915/guc: Disable engine barriers with GuC during unpin Matthew Brost
2021-07-09 22:53 ` John Harrison
2021-07-10 3:00 ` Matthew Brost
2021-07-12 17:57 ` John Harrison
2021-07-12 18:11 ` Daniel Vetter
2021-06-24 7:04 ` [Intel-gfx] [PATCH 17/47] drm/i915/guc: Extend deregistration fence to schedule disable Matthew Brost
2021-07-09 22:59 ` John Harrison
2021-07-10 3:36 ` Matthew Brost
2021-07-12 17:54 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 18/47] drm/i915: Disable preempt busywait when using GuC scheduling Matthew Brost
2021-07-09 23:03 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 19/47] drm/i915/guc: Ensure request ordering via completion fences Matthew Brost
2021-07-15 1:51 ` Daniele Ceraolo Spurio
2021-06-24 7:04 ` [Intel-gfx] [PATCH 20/47] drm/i915/guc: Disable semaphores when using GuC scheduling Matthew Brost
2021-07-09 23:53 ` John Harrison
2021-07-15 0:07 ` Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 21/47] drm/i915/guc: Ensure G2H response has space in buffer Matthew Brost
2021-07-13 18:36 ` John Harrison
2021-07-15 0:06 ` Matthew Brost
2021-07-15 0:12 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 22/47] drm/i915/guc: Update intel_gt_wait_for_idle to work with GuC Matthew Brost
2021-07-10 0:16 ` John Harrison
2021-07-10 3:55 ` Matthew Brost
2021-07-17 4:09 ` Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 23/47] drm/i915/guc: Update GuC debugfs to support new GuC Matthew Brost
2021-07-12 18:05 ` John Harrison
2021-07-12 20:59 ` Matthew Brost
2021-07-12 21:37 ` John Harrison
2021-07-13 8:51 ` Michal Wajdeczko
2021-07-14 23:56 ` Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 24/47] drm/i915/guc: Add several request trace points Matthew Brost
2021-07-12 18:08 ` John Harrison
2021-07-13 9:06 ` Tvrtko Ursulin
2021-07-20 1:59 ` Matthew Brost
2021-07-22 13:55 ` Tvrtko Ursulin
2021-06-24 7:04 ` [Intel-gfx] [PATCH 25/47] drm/i915: Add intel_context tracing Matthew Brost
2021-07-12 18:10 ` John Harrison
2021-07-12 21:47 ` Matthew Brost
2021-07-12 21:51 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 26/47] drm/i915/guc: GuC virtual engines Matthew Brost
2021-07-15 1:21 ` Daniele Ceraolo Spurio
2021-06-24 7:04 ` [Intel-gfx] [PATCH 27/47] drm/i915: Track 'serial' counts for " Matthew Brost
2021-07-12 18:11 ` John Harrison
2021-07-12 20:06 ` Matthew Brost
2021-06-24 7:04 ` [Intel-gfx] [PATCH 28/47] drm/i915: Hold reference to intel_context over life of i915_request Matthew Brost
2021-07-12 18:23 ` John Harrison
2021-07-12 20:05 ` Matthew Brost
2021-07-12 21:36 ` Matthew Brost
2021-07-12 21:48 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 29/47] drm/i915/guc: Disable bonding extension with GuC submission Matthew Brost
2021-07-12 18:23 ` John Harrison
2021-06-24 7:04 ` [Intel-gfx] [PATCH 30/47] drm/i915/guc: Direct all breadcrumbs for a class to single breadcrumbs Matthew Brost
2021-07-12 19:19 ` John Harrison
2021-06-24 7:05 ` [Intel-gfx] [PATCH 31/47] drm/i915/guc: Reset implementation for new GuC interface Matthew Brost
2021-07-12 19:58 ` John Harrison
2021-07-15 0:53 ` Matthew Brost
2021-07-15 9:36 ` Tvrtko Ursulin
2021-07-26 22:48 ` Matthew Brost
2021-07-27 8:56 ` Tvrtko Ursulin
2021-07-27 18:30 ` Matthew Brost
2021-06-24 7:05 ` [Intel-gfx] [PATCH 32/47] drm/i915: Reset GPU immediately if submission is disabled Matthew Brost
2021-07-12 20:01 ` John Harrison
2021-06-24 7:05 ` [Intel-gfx] [PATCH 33/47] drm/i915/guc: Add disable interrupts to guc sanitize Matthew Brost
2021-07-12 20:11 ` John Harrison
2021-06-24 7:05 ` [Intel-gfx] [PATCH 34/47] drm/i915/guc: Suspend/resume implementation for new interface Matthew Brost
2021-07-12 22:56 ` John Harrison
2021-06-24 7:05 ` [Intel-gfx] [PATCH 35/47] drm/i915/guc: Handle context reset notification Matthew Brost
2021-07-12 22:58 ` John Harrison
2021-07-15 0:32 ` Matthew Brost
2021-06-24 7:05 ` [Intel-gfx] [PATCH 36/47] drm/i915/guc: Handle engine reset failure notification Matthew Brost
2021-07-12 22:59 ` John Harrison
2021-06-24 7:05 ` [Intel-gfx] [PATCH 37/47] drm/i915/guc: Enable the timer expired interrupt for GuC Matthew Brost
2021-07-12 23:00 ` John Harrison
2021-06-24 7:05 ` [Intel-gfx] [PATCH 38/47] drm/i915/guc: Provide mmio list to be saved/restored on engine reset Matthew Brost
2021-06-24 7:05 ` [Intel-gfx] [PATCH 39/47] drm/i915/guc: Don't complain about reset races Matthew Brost
2021-06-24 15:55 ` Matthew Brost
2021-06-24 7:05 ` [Intel-gfx] [PATCH 40/47] drm/i915/guc: Enable GuC engine reset Matthew Brost
2021-06-24 16:19 ` Matthew Brost
2021-06-24 7:05 ` [Intel-gfx] [PATCH 41/47] drm/i915/guc: Capture error state on context reset Matthew Brost
2021-07-12 23:05 ` John Harrison
2021-06-24 7:05 ` [Intel-gfx] [PATCH 42/47] drm/i915/guc: Fix for error capture after full GPU reset with GuC Matthew Brost
2021-07-15 0:43 ` Matthew Brost
2021-06-24 7:05 ` [Intel-gfx] [PATCH 43/47] drm/i915/guc: Hook GuC scheduling policies up Matthew Brost
2021-06-25 0:59 ` Matthew Brost
2021-06-25 19:10 ` John Harrison
2021-07-10 18:56 ` Matthew Brost
2021-06-24 7:05 ` [Intel-gfx] [PATCH 44/47] drm/i915/guc: Connect reset modparam updates to GuC policy flags Matthew Brost
2021-06-25 1:10 ` Matthew Brost
2021-06-24 7:05 ` [Intel-gfx] [PATCH 45/47] drm/i915/guc: Include scheduling policies in the debugfs state dump Matthew Brost
2021-06-24 16:34 ` Matthew Brost
2021-06-24 7:05 ` [Intel-gfx] [PATCH 46/47] drm/i915/guc: Add golden context to GuC ADS Matthew Brost
2021-06-24 7:05 ` [Intel-gfx] [PATCH 47/47] drm/i915/guc: Unblock GuC submission on Gen11+ Matthew Brost
2021-06-30 8:22 ` Martin Peres
2021-06-30 18:00 ` Matthew Brost
2021-07-01 18:24 ` Martin Peres
2021-07-02 8:13 ` Martin Peres
2021-07-02 13:06 ` Michal Wajdeczko
2021-07-02 13:12 ` Martin Peres
2021-07-02 14:08 ` Michal Wajdeczko
2021-06-30 18:58 ` John Harrison
2021-07-01 8:14 ` Pekka Paalanen
2021-07-01 18:27 ` Martin Peres
2021-07-01 19:28 ` Daniel Vetter
2021-07-02 7:29 ` Pekka Paalanen
2021-07-02 8:09 ` Martin Peres
2021-07-02 15:07 ` Michal Wajdeczko
2021-07-03 8:21 ` Martin Peres
2021-07-07 0:57 ` John Harrison
2021-07-07 7:47 ` Pekka Paalanen
2021-07-07 10:11 ` Michal Wajdeczko
2021-07-15 0:49 ` Matthew Brost
2021-06-24 7:17 ` [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for GuC submission support Patchwork
2021-06-24 7:19 ` [Intel-gfx] ✗ Fi.CI.SPARSE: " Patchwork
2021-06-24 7:47 ` [Intel-gfx] ✗ Fi.CI.BAT: failure " Patchwork
2021-07-12 19:23 ` [Intel-gfx] ✗ Fi.CI.BUILD: failure for GuC submission support (rev2) Patchwork
2021-10-22 9:35 ` [Intel-gfx] [PATCH 00/47] GuC submission support Joonas Lahtinen
2021-10-22 16:42 ` Matthew Brost
2021-10-25 9:37 ` Joonas Lahtinen
2021-10-25 15:15 ` Matthew Brost
2021-10-26 8:59 ` Joonas Lahtinen
2021-10-26 15:43 ` Matthew Brost
2021-10-26 15:51 ` Matthew Brost
2021-10-27 9:21 ` Joonas Lahtinen
2021-10-25 17:06 ` John Harrison
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210624230102.GB20966@sdutt-i7 \
--to=matthew.brost@intel.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=intel-gfx@lists.freedesktop.org \
--cc=michal.wajdeczko@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).