From: Daniel Vetter <daniel@ffwll.ch> To: Matthew Brost <matthew.brost@intel.com> Cc: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, daniel.vetter@ffwll.ch Subject: Re: [PATCH 16/22] drm/i915/guc: Release submit fence from an IRQ Date: Tue, 17 Aug 2021 12:08:09 +0200 [thread overview] Message-ID: <YRuKiZDoPeAMYvzj@phenom.ffwll.local> (raw) In-Reply-To: <20210816135139.10060-17-matthew.brost@intel.com> On Mon, Aug 16, 2021 at 06:51:33AM -0700, Matthew Brost wrote: > A subsequent patch will flip the locking hierarchy from > ce->guc_state.lock -> sched_engine->lock to sched_engine->lock -> > ce->guc_state.lock. As such we need to release the submit fence for a > request from an IRQ to break a lock inversion - i.e. the fence must be > release went holding ce->guc_state.lock and the releasing of the can > acquire sched_engine->lock. > > Signed-off-by: Matthew Brost <matthew.brost@intel.com> Title should be "irq work", otherwise it reads a bit strange. Also these kind of nestings would be good to document in the kerneldoc too (maybe as you go even). -Daniel > --- > drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 15 ++++++++++++++- > drivers/gpu/drm/i915/i915_request.h | 5 +++++ > 2 files changed, 19 insertions(+), 1 deletion(-) > > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > index 8c560ed14976..9ae4633aa7cb 100644 > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > @@ -2017,6 +2017,14 @@ static const struct intel_context_ops guc_context_ops = { > .create_virtual = guc_create_virtual, > }; > > +static void submit_work_cb(struct irq_work *wrk) > +{ > + struct i915_request *rq = container_of(wrk, typeof(*rq), submit_work); > + > + might_lock(&rq->engine->sched_engine->lock); > + i915_sw_fence_complete(&rq->submit); > +} > + > static void __guc_signal_context_fence(struct intel_context *ce) > { > struct i915_request *rq; > @@ -2026,8 +2034,12 @@ static void __guc_signal_context_fence(struct intel_context *ce) > if (!list_empty(&ce->guc_state.fences)) > trace_intel_context_fence_release(ce); > > + /* > + * Use an IRQ to ensure locking order of sched_engine->lock -> > + * ce->guc_state.lock is preserved. > + */ > list_for_each_entry(rq, &ce->guc_state.fences, guc_fence_link) > - i915_sw_fence_complete(&rq->submit); > + irq_work_queue(&rq->submit_work); > > INIT_LIST_HEAD(&ce->guc_state.fences); > } > @@ -2137,6 +2149,7 @@ static int guc_request_alloc(struct i915_request *rq) > spin_lock_irqsave(&ce->guc_state.lock, flags); > if (context_wait_for_deregister_to_register(ce) || > context_pending_disable(ce)) { > + init_irq_work(&rq->submit_work, submit_work_cb); > i915_sw_fence_await(&rq->submit); > > list_add_tail(&rq->guc_fence_link, &ce->guc_state.fences); > diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h > index 1bc1349ba3c2..d818cfbfc41d 100644 > --- a/drivers/gpu/drm/i915/i915_request.h > +++ b/drivers/gpu/drm/i915/i915_request.h > @@ -218,6 +218,11 @@ struct i915_request { > }; > struct llist_head execute_cb; > struct i915_sw_fence semaphore; > + /** > + * @submit_work: complete submit fence from an IRQ if needed for > + * locking hierarchy reasons. > + */ > + struct irq_work submit_work; > > /* > * A list of everyone we wait upon, and everyone who waits upon us. > -- > 2.32.0 > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
WARNING: multiple messages have this Message-ID (diff)
From: Daniel Vetter <daniel@ffwll.ch> To: Matthew Brost <matthew.brost@intel.com> Cc: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, daniel.vetter@ffwll.ch Subject: Re: [Intel-gfx] [PATCH 16/22] drm/i915/guc: Release submit fence from an IRQ Date: Tue, 17 Aug 2021 12:08:09 +0200 [thread overview] Message-ID: <YRuKiZDoPeAMYvzj@phenom.ffwll.local> (raw) In-Reply-To: <20210816135139.10060-17-matthew.brost@intel.com> On Mon, Aug 16, 2021 at 06:51:33AM -0700, Matthew Brost wrote: > A subsequent patch will flip the locking hierarchy from > ce->guc_state.lock -> sched_engine->lock to sched_engine->lock -> > ce->guc_state.lock. As such we need to release the submit fence for a > request from an IRQ to break a lock inversion - i.e. the fence must be > release went holding ce->guc_state.lock and the releasing of the can > acquire sched_engine->lock. > > Signed-off-by: Matthew Brost <matthew.brost@intel.com> Title should be "irq work", otherwise it reads a bit strange. Also these kind of nestings would be good to document in the kerneldoc too (maybe as you go even). -Daniel > --- > drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 15 ++++++++++++++- > drivers/gpu/drm/i915/i915_request.h | 5 +++++ > 2 files changed, 19 insertions(+), 1 deletion(-) > > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > index 8c560ed14976..9ae4633aa7cb 100644 > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > @@ -2017,6 +2017,14 @@ static const struct intel_context_ops guc_context_ops = { > .create_virtual = guc_create_virtual, > }; > > +static void submit_work_cb(struct irq_work *wrk) > +{ > + struct i915_request *rq = container_of(wrk, typeof(*rq), submit_work); > + > + might_lock(&rq->engine->sched_engine->lock); > + i915_sw_fence_complete(&rq->submit); > +} > + > static void __guc_signal_context_fence(struct intel_context *ce) > { > struct i915_request *rq; > @@ -2026,8 +2034,12 @@ static void __guc_signal_context_fence(struct intel_context *ce) > if (!list_empty(&ce->guc_state.fences)) > trace_intel_context_fence_release(ce); > > + /* > + * Use an IRQ to ensure locking order of sched_engine->lock -> > + * ce->guc_state.lock is preserved. > + */ > list_for_each_entry(rq, &ce->guc_state.fences, guc_fence_link) > - i915_sw_fence_complete(&rq->submit); > + irq_work_queue(&rq->submit_work); > > INIT_LIST_HEAD(&ce->guc_state.fences); > } > @@ -2137,6 +2149,7 @@ static int guc_request_alloc(struct i915_request *rq) > spin_lock_irqsave(&ce->guc_state.lock, flags); > if (context_wait_for_deregister_to_register(ce) || > context_pending_disable(ce)) { > + init_irq_work(&rq->submit_work, submit_work_cb); > i915_sw_fence_await(&rq->submit); > > list_add_tail(&rq->guc_fence_link, &ce->guc_state.fences); > diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h > index 1bc1349ba3c2..d818cfbfc41d 100644 > --- a/drivers/gpu/drm/i915/i915_request.h > +++ b/drivers/gpu/drm/i915/i915_request.h > @@ -218,6 +218,11 @@ struct i915_request { > }; > struct llist_head execute_cb; > struct i915_sw_fence semaphore; > + /** > + * @submit_work: complete submit fence from an IRQ if needed for > + * locking hierarchy reasons. > + */ > + struct irq_work submit_work; > > /* > * A list of everyone we wait upon, and everyone who waits upon us. > -- > 2.32.0 > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
next prev parent reply other threads:[~2021-08-17 10:08 UTC|newest] Thread overview: 104+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-08-16 13:51 [PATCH 00/22] Clean up GuC CI failures, simplify locking, and kernel DOC Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 01/22] drm/i915/guc: Fix blocked context accounting Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 02/22] drm/i915/guc: Fix outstanding G2H accounting Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 9:39 ` Daniel Vetter 2021-08-17 9:39 ` [Intel-gfx] " Daniel Vetter 2021-08-17 18:17 ` Matthew Brost 2021-08-17 18:17 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 03/22] drm/i915/guc: Unwind context requests in reverse order Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 04/22] drm/i915/guc: Don't drop ce->guc_active.lock when unwinding context Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 05/22] drm/i915/guc: Workaround reset G2H is received after schedule done G2H Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 9:32 ` Daniel Vetter 2021-08-17 9:32 ` [Intel-gfx] " Daniel Vetter 2021-08-17 15:03 ` Matthew Brost 2021-08-17 15:03 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 06/22] drm/i915/execlists: Do not propagate errors to dependent fences Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 9:21 ` Daniel Vetter 2021-08-17 9:21 ` [Intel-gfx] " Daniel Vetter 2021-08-17 15:08 ` Matthew Brost 2021-08-17 15:08 ` [Intel-gfx] " Matthew Brost 2021-08-17 15:49 ` Daniel Vetter 2021-08-17 15:49 ` [Intel-gfx] " Daniel Vetter 2021-08-16 13:51 ` [PATCH 07/22] drm/i915/selftests: Add a cancel request selftest that triggers a reset Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 08/22] drm/i915/guc: Don't enable scheduling on a banned context, guc_id invalid, not registered Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 9:47 ` Daniel Vetter 2021-08-17 9:47 ` [Intel-gfx] " Daniel Vetter 2021-08-17 9:57 ` Daniel Vetter 2021-08-17 9:57 ` [Intel-gfx] " Daniel Vetter 2021-08-17 16:44 ` Matthew Brost 2021-08-17 16:44 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 09/22] drm/i915/selftests: Fix memory corruption in live_lrc_isolation Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 10/22] drm/i915/selftests: Add initial GuC selftest for scrubbing lost G2H Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 11/22] drm/i915/guc: Take context ref when cancelling request Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 12/22] drm/i915/guc: Don't touch guc_state.sched_state without a lock Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 7:21 ` kernel test robot 2021-08-17 7:21 ` kernel test robot 2021-08-17 7:21 ` [Intel-gfx] " kernel test robot 2021-08-16 13:51 ` [PATCH 13/22] drm/i915/guc: Reset LRC descriptor if register returns -ENODEV Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 14/22] drm/i915: Allocate error capture in atomic context Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 10:06 ` Daniel Vetter 2021-08-17 10:06 ` [Intel-gfx] " Daniel Vetter 2021-08-17 16:12 ` Matthew Brost 2021-08-17 16:12 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 15/22] drm/i915/guc: Flush G2H work queue during reset Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 10:06 ` Daniel Vetter 2021-08-16 13:51 ` [PATCH 16/22] drm/i915/guc: Release submit fence from an IRQ Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 10:08 ` Daniel Vetter [this message] 2021-08-17 10:08 ` Daniel Vetter 2021-08-16 13:51 ` [PATCH 17/22] drm/i915/guc: Move guc_blocked fence to struct guc_state Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 10:10 ` Daniel Vetter 2021-08-17 10:10 ` [Intel-gfx] " Daniel Vetter 2021-08-16 13:51 ` [PATCH 18/22] drm/i915/guc: Rework and simplify locking Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 10:15 ` Daniel Vetter 2021-08-17 10:15 ` [Intel-gfx] " Daniel Vetter 2021-08-17 15:30 ` Matthew Brost 2021-08-17 15:30 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 19/22] drm/i915/guc: Proper xarray usage for contexts_lookup Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 10:27 ` Daniel Vetter 2021-08-17 15:26 ` Matthew Brost 2021-08-17 17:13 ` Daniel Vetter 2021-08-17 17:13 ` Matthew Brost 2021-08-16 13:51 ` [PATCH 20/22] drm/i915/guc: Drop pin count check trick between sched_disable and re-pin Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 21/22] drm/i915/guc: Move GuC priority fields in context under guc_active Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-16 13:51 ` [PATCH 22/22] drm/i915/guc: Add GuC kernel doc Matthew Brost 2021-08-16 13:51 ` [Intel-gfx] " Matthew Brost 2021-08-17 11:11 ` Daniel Vetter 2021-08-17 11:11 ` [Intel-gfx] " Daniel Vetter 2021-08-17 16:36 ` Matthew Brost 2021-08-17 16:36 ` [Intel-gfx] " Matthew Brost 2021-08-17 17:20 ` Daniel Vetter 2021-08-17 17:20 ` [Intel-gfx] " Daniel Vetter 2021-08-17 17:27 ` Michal Wajdeczko 2021-08-17 17:27 ` [Intel-gfx] " Michal Wajdeczko 2021-08-17 17:34 ` Daniel Vetter 2021-08-17 17:34 ` [Intel-gfx] " Daniel Vetter 2021-08-17 20:41 ` Michal Wajdeczko 2021-08-17 20:41 ` [Intel-gfx] " Michal Wajdeczko 2021-08-17 21:49 ` Daniel Vetter 2021-08-17 21:49 ` [Intel-gfx] " Daniel Vetter 2021-08-17 12:49 ` [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Clean up GuC CI failures, simplify locking, and kernel DOC (rev2) Patchwork 2021-08-17 12:51 ` [Intel-gfx] ✗ Fi.CI.SPARSE: " Patchwork 2021-08-17 13:22 ` [Intel-gfx] ✓ Fi.CI.BAT: success " Patchwork 2021-08-17 14:39 ` [Intel-gfx] ✗ Fi.CI.IGT: failure " Patchwork
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=YRuKiZDoPeAMYvzj@phenom.ffwll.local \ --to=daniel@ffwll.ch \ --cc=daniel.vetter@ffwll.ch \ --cc=dri-devel@lists.freedesktop.org \ --cc=intel-gfx@lists.freedesktop.org \ --cc=matthew.brost@intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.