From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1476C07E99 for ; Mon, 12 Jul 2021 21:48:54 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7E56D61106 for ; Mon, 12 Jul 2021 21:48:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E56D61106 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 208D18982E; Mon, 12 Jul 2021 21:48:54 +0000 (UTC) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTPS id 399418982E; Mon, 12 Jul 2021 21:48:53 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10043"; a="210034052" X-IronPort-AV: E=Sophos;i="5.84,234,1620716400"; d="scan'208";a="210034052" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jul 2021 14:48:52 -0700 X-IronPort-AV: E=Sophos;i="5.84,234,1620716400"; d="scan'208";a="412721995" Received: from johnharr-mobl1.amr.corp.intel.com (HELO [10.209.125.18]) ([10.209.125.18]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jul 2021 14:48:52 -0700 To: Matthew Brost References: <20210624070516.21893-1-matthew.brost@intel.com> <20210624070516.21893-29-matthew.brost@intel.com> <324bd9ac-822a-e83a-4a04-f7eb61fd43f9@intel.com> <20210712200530.GA14920@DUT181-TGLU.fm.intel.com> <20210712213600.GA15656@DUT181-TGLU.fm.intel.com> From: John Harrison Message-ID: Date: Mon, 12 Jul 2021 14:48:51 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <20210712213600.GA15656@DUT181-TGLU.fm.intel.com> Content-Language: en-GB Subject: Re: [Intel-gfx] [PATCH 28/47] drm/i915: Hold reference to intel_context over life of i915_request X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" On 7/12/2021 14:36, Matthew Brost wrote: > On Mon, Jul 12, 2021 at 08:05:30PM +0000, Matthew Brost wrote: >> On Mon, Jul 12, 2021 at 11:23:14AM -0700, John Harrison wrote: >>> On 6/24/2021 00:04, Matthew Brost wrote: >>>> Hold a reference to the intel_context over life of an i915_request. >>>> Without this an i915_request can exist after the context has been >>>> destroyed (e.g. request retired, context closed, but user space holds a >>>> reference to the request from an out fence). In the case of GuC >>>> submission + virtual engine, the engine that the request references is >>>> also destroyed which can trigger bad pointer dref in fence ops (e.g. >>> Maybe quickly explain a why this is different for GuC submission vs >>> execlist? Presumably it is about only decomposing virtual engines to >>> physical ones in execlist mode? >>> >> Yes, it because in execlists we always end up pointing to a physical >> engine in the end while in GuC mode we can be pointing to dynamically >> allocated virtual engine. I can update the comment. >> >>>> i915_fence_get_driver_name). We could likely change >>>> i915_fence_get_driver_name to avoid touching the engine but let's just >>>> be safe and hold the intel_context reference. >>>> >>>> Signed-off-by: Matthew Brost >>>> --- >>>> drivers/gpu/drm/i915/i915_request.c | 54 ++++++++++++----------------- >>>> 1 file changed, 22 insertions(+), 32 deletions(-) >>>> >>>> diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c >>>> index de9deb95b8b1..dec5a35c9aa2 100644 >>>> --- a/drivers/gpu/drm/i915/i915_request.c >>>> +++ b/drivers/gpu/drm/i915/i915_request.c >>>> @@ -126,39 +126,17 @@ static void i915_fence_release(struct dma_fence *fence) >>>> i915_sw_fence_fini(&rq->semaphore); >>>> /* >>>> - * Keep one request on each engine for reserved use under mempressure >>>> - * >>>> - * We do not hold a reference to the engine here and so have to be >>>> - * very careful in what rq->engine we poke. The virtual engine is >>>> - * referenced via the rq->context and we released that ref during >>>> - * i915_request_retire(), ergo we must not dereference a virtual >>>> - * engine here. Not that we would want to, as the only consumer of >>>> - * the reserved engine->request_pool is the power management parking, >>>> - * which must-not-fail, and that is only run on the physical engines. >>>> - * >>>> - * Since the request must have been executed to be have completed, >>>> - * we know that it will have been processed by the HW and will >>>> - * not be unsubmitted again, so rq->engine and rq->execution_mask >>>> - * at this point is stable. rq->execution_mask will be a single >>>> - * bit if the last and _only_ engine it could execution on was a >>>> - * physical engine, if it's multiple bits then it started on and >>>> - * could still be on a virtual engine. Thus if the mask is not a >>>> - * power-of-two we assume that rq->engine may still be a virtual >>>> - * engine and so a dangling invalid pointer that we cannot dereference >>>> - * >>>> - * For example, consider the flow of a bonded request through a virtual >>>> - * engine. The request is created with a wide engine mask (all engines >>>> - * that we might execute on). On processing the bond, the request mask >>>> - * is reduced to one or more engines. If the request is subsequently >>>> - * bound to a single engine, it will then be constrained to only >>>> - * execute on that engine and never returned to the virtual engine >>>> - * after timeslicing away, see __unwind_incomplete_requests(). Thus we >>>> - * know that if the rq->execution_mask is a single bit, rq->engine >>>> - * can be a physical engine with the exact corresponding mask. >>>> + * Keep one request on each engine for reserved use under mempressure, >>>> + * do not use with virtual engines as this really is only needed for >>>> + * kernel contexts. >>>> */ >>>> - if (is_power_of_2(rq->execution_mask) && >>>> - !cmpxchg(&rq->engine->request_pool, NULL, rq)) >>>> + if (!intel_engine_is_virtual(rq->engine) && >>>> + !cmpxchg(&rq->engine->request_pool, NULL, rq)) { >>>> + intel_context_put(rq->context); >>>> return; >>>> + } >>>> + >>>> + intel_context_put(rq->context); >>> The put is actually unconditional? So it could be moved before the if? >>> >> Yep, I think so. >> > Wait nope. We reference rq->engine which could a virtual engine and the > intel_context_put could free that engine. So we need to do the put after > we reference it. > > Matt Doh! That's a pretty good reason. Okay, with a tweaked description to explain about virtual engines being different on GuC vs execlist... Reviewed-by: John Harrison > >> Matt >> >>> John. >>> >>>> kmem_cache_free(global.slab_requests, rq); >>>> } >>>> @@ -977,7 +955,18 @@ __i915_request_create(struct intel_context *ce, gfp_t gfp) >>>> } >>>> } >>>> - rq->context = ce; >>>> + /* >>>> + * Hold a reference to the intel_context over life of an i915_request. >>>> + * Without this an i915_request can exist after the context has been >>>> + * destroyed (e.g. request retired, context closed, but user space holds >>>> + * a reference to the request from an out fence). In the case of GuC >>>> + * submission + virtual engine, the engine that the request references >>>> + * is also destroyed which can trigger bad pointer dref in fence ops >>>> + * (e.g. i915_fence_get_driver_name). We could likely change these >>>> + * functions to avoid touching the engine but let's just be safe and >>>> + * hold the intel_context reference. >>>> + */ >>>> + rq->context = intel_context_get(ce); >>>> rq->engine = ce->engine; >>>> rq->ring = ce->ring; >>>> rq->execution_mask = ce->engine->mask; >>>> @@ -1054,6 +1043,7 @@ __i915_request_create(struct intel_context *ce, gfp_t gfp) >>>> GEM_BUG_ON(!list_empty(&rq->sched.waiters_list)); >>>> err_free: >>>> + intel_context_put(ce); >>>> kmem_cache_free(global.slab_requests, rq); >>>> err_unreserve: >>>> intel_context_unpin(ce); >> _______________________________________________ >> Intel-gfx mailing list >> Intel-gfx@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/intel-gfx _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx