From: Chris Wilson <chris@chris-wilson.co.uk>
To: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
intel-gfx@lists.freedesktop.org
Subject: Re: [Intel-gfx] [PATCH 2/2] drm/i915: Avoid using rq->engine after free during i915_fence_release
Date: Thu, 21 May 2020 10:44:16 +0100 [thread overview]
Message-ID: <159005425694.32320.3220801731152833885@build.alporthouse.com> (raw)
In-Reply-To: <159005356958.32320.5047736211612324801@build.alporthouse.com>
Quoting Chris Wilson (2020-05-21 10:32:49)
> Quoting Chris Wilson (2020-05-21 10:27:16)
> > Quoting Tvrtko Ursulin (2020-05-21 10:13:14)
> > >
> > > On 21/05/2020 09:53, Chris Wilson wrote:
> > > > In order to be valid to dereference during the i915_fence_release, after
> > > > retiring the fence and releasing its refererences, we assume that
> > > > rq->engine can only be a real engine (that stay intact until the device
> > > > is shutdown after all fences have been flushed). However, due to a quirk
> > > > of preempt-to-busy, we may retire a request that still belongs to a
> > > > virtual engine and so eventually free it with rq->engine being invalid.
> > > > To avoid dereferencing that invalid engine, we look at the
> > > > execution_mask which if it indicates it may be executed on more than one
> > > > engine, we know it originated on a virtual engine and may still be on
> > > > one.
> > > >
> > > > Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/1906
> > > > Fixes: 43acd6516ca9 ("drm/i915: Keep a per-engine request pool")
> > > > Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> > > > Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> > > > ---
> > > > drivers/gpu/drm/i915/i915_request.c | 25 +++++++++++++++++++++++--
> > > > 1 file changed, 23 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
> > > > index 526c1e9acbd5..6e357183bece 100644
> > > > --- a/drivers/gpu/drm/i915/i915_request.c
> > > > +++ b/drivers/gpu/drm/i915/i915_request.c
> > > > @@ -121,8 +121,29 @@ static void i915_fence_release(struct dma_fence *fence)
> > > > i915_sw_fence_fini(&rq->submit);
> > > > i915_sw_fence_fini(&rq->semaphore);
> > > >
> > > > - /* Keep one request on each engine for reserved use under mempressure */
> > > > - if (!cmpxchg(&rq->engine->request_pool, NULL, rq))
> > > > + /*
> > > > + * Keep one request on each engine for reserved use under mempressure
> > > > + *
> > > > + * We do not hold a reference to the engine here and so have to be
> > > > + * very careful in what rq->engine we poke. The virtual engine is
> > > > + * referenced via the rq->context and we released that ref during
> > > > + * i915_request_retire(), ergo we must not dereference a virtual
> > > > + * engine here. Not that we would want to, as the only consumer of
> > > > + * the reserved engine->request_pool is the powermanagent parking,
> > >
> > > power management
> > >
> > > > + * which must-not-fail, and that is only run on the physical engines.
> > > > + *
> > > > + * Since the request must have been executed to be have completed,
> > > > + * we know that it will have been processed by the HW and will
> > > > + * not be unsubmitted again, so rq->engine and rq->execution_mask
> > > > + * at this point is stable. rq->execution_mask will be a single
> > > > + * bit if the last and only engine it could execution on was a
> > > > + * physical engine, if it's multiple bits then it started on and
> > > > + * could still be on a virtual engine. Thus if the mask is not a
> > > > + * power-of-two we assume that rq->engine may still be a virtual
> > > > + * engien and so a dangling invalid pointer that we cannot
> > >
> > > engine
> > >
> > > But.. submit fence can mask out execution_mask bits and make it appear
> > > the request was on a physical engine. What then?
> >
> > Then we execute along a single engine and it is never returned to the
> > virtual engine (in __unwind_incomplete_requests).
>
> * For example, consider the flow of a bonded request through a virtual
> * engine. The request is created with a wide engine mask (all engines
> * that we might execute on). On processing the bond, the request mask
> * is reduced to one or more engines. If the request is subsequently
> * bound to a single engine, it will then be constrained to only
> * execute on that engine and never returned to the virtual engine
> * after timeslicing away, see __unwind_incomplete_requests(). Thus we
> * know that if the rq->execution_mask is a single bit, only rq->engine
rq->engine can only be a physical engine, with the exact corresponding mask.
> * can be the exact corresponding engine->mask.
>
> -Chris
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
next prev parent reply other threads:[~2020-05-21 9:44 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-21 8:53 [Intel-gfx] [PATCH 1/2] drm/i915: Disable semaphore inter-engine sync without timeslicing Chris Wilson
2020-05-21 8:53 ` [Intel-gfx] [PATCH 2/2] drm/i915: Avoid using rq->engine after free during i915_fence_release Chris Wilson
2020-05-21 9:13 ` Tvrtko Ursulin
2020-05-21 9:27 ` Chris Wilson
2020-05-21 9:32 ` Chris Wilson
2020-05-21 9:44 ` Chris Wilson [this message]
2020-05-21 13:57 ` Tvrtko Ursulin
2020-05-21 8:55 ` [Intel-gfx] [PATCH] drm/i915: Disable semaphore inter-engine sync without timeslicing Chris Wilson
2020-05-21 9:10 ` [Intel-gfx] [PATCH 1/2] " Tvrtko Ursulin
2020-05-21 9:42 ` Chris Wilson
2020-05-21 10:17 ` Chris Wilson
2020-05-21 13:58 ` Tvrtko Ursulin
2020-05-21 10:00 ` [Intel-gfx] ✗ Fi.CI.SPARSE: warning for series starting with drm/i915: Disable semaphore inter-engine sync without timeslicing (rev2) Patchwork
2020-05-21 10:21 ` [Intel-gfx] ✓ Fi.CI.BAT: success " Patchwork
2020-05-22 2:28 ` [Intel-gfx] ✓ Fi.CI.IGT: " Patchwork
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=159005425694.32320.3220801731152833885@build.alporthouse.com \
--to=chris@chris-wilson.co.uk \
--cc=intel-gfx@lists.freedesktop.org \
--cc=tvrtko.ursulin@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).