From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E48E2C38147 for ; Wed, 18 Jan 2023 16:23:12 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 317D410E793; Wed, 18 Jan 2023 16:23:12 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 05DDF10E792; Wed, 18 Jan 2023 16:23:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1674058990; x=1705594990; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=9Fi480uzS5oBGcDSvrekY97oLJsbYkd3EZfuIFpLe2U=; b=IVJDYp3WmwYjaIbPaCbspXRiM0LXxSWK6rOoAqaXjq7cJDK7vkUbWN97 H0O5YzZo6ReP0oCZZhl5SuPa3evqOpLUuFga1xPXW4KHQ3uHp67gvIqR7 zPF7Dbltye4rfM/r9bJlcl734h/4ylbIUrBq+gjs3lxdMWqPR7FTuapc/ P+vbSxLbfcXTJiszD3h8suavSP5X+5pSVDYE+cREKeqSwbHRU98XonUeG V8CLTq4znMlcfricnjgGIiDwEIyKuvWNZfyns6oktnJeZlIquE7vjeo7A 9QONp8A+59R6y/25xHn8gGSeMceqh/ipGU7YJORCg+gJZgQizIEXK9fFO Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10594"; a="312899539" X-IronPort-AV: E=Sophos;i="5.97,226,1669104000"; d="scan'208";a="312899539" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jan 2023 08:22:36 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10594"; a="988612254" X-IronPort-AV: E=Sophos;i="5.97,226,1669104000"; d="scan'208";a="988612254" Received: from llvincen-mobl.ger.corp.intel.com (HELO [10.213.215.37]) ([10.213.215.37]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jan 2023 08:22:31 -0800 Message-ID: Date: Wed, 18 Jan 2023 16:22:29 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Subject: Re: [PATCH v2 1/5] drm/i915: Fix request locking during error capture & debugfs dump Content-Language: en-US To: John.C.Harrison@Intel.com, Intel-GFX@Lists.FreeDesktop.Org References: <20230117213630.2897570-1-John.C.Harrison@Intel.com> <20230117213630.2897570-2-John.C.Harrison@Intel.com> From: Tvrtko Ursulin Organization: Intel Corporation UK Plc In-Reply-To: <20230117213630.2897570-2-John.C.Harrison@Intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matthew Brost , Andy Shevchenko , Michael Cheng , Aravind Iddamsetty , Alan Previn , Umesh Nerlige Ramappa , Lucas De Marchi , Chris Wilson , Bruce Chang , Daniele Ceraolo Spurio , DRI-Devel@Lists.FreeDesktop.Org, Andrzej Hajda , Rodrigo Vivi , Tejas Upadhyay , Matthew Auld Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On 17/01/2023 21:36, John.C.Harrison@Intel.com wrote: > From: John Harrison > > When GuC support was added to error capture, the locking around the > request object was broken. Fix it up. > > The context based search manages the spinlocking around the search > internally. So it needs to grab the reference count internally as > well. The execlist only request based search relies on external > locking, so it needs an external reference count. So no change to that > code itself but the context version does change. > > The only other caller is the code for dumping engine state to debugfs. > That code wasn't previously getting an explicit reference at all as it > does everything while holding the execlist specific spinlock. So that > needs updaing as well as that spinlock doesn't help when using GuC > submission. Rather than trying to conditionally get/put depending on > submission model, just change it to always do the get/put. > > In addition, intel_guc_find_hung_context() was not acquiring the > correct spinlock before searching the request list. So fix that up too. > > Fixes: dc0dad365c5e ("drm/i915/guc: Fix for error capture after full GPU reset > with GuC") > Fixes: 573ba126aef3 ("drm/i915/guc: Capture error state on context reset") > Cc: Matthew Brost > Cc: John Harrison > Cc: Jani Nikula > Cc: Joonas Lahtinen > Cc: Rodrigo Vivi > Cc: Tvrtko Ursulin > Cc: Daniele Ceraolo Spurio > Cc: Andrzej Hajda > Cc: Chris Wilson > Cc: Matthew Auld > Cc: Matt Roper > Cc: Umesh Nerlige Ramappa > Cc: Michael Cheng > Cc: Lucas De Marchi > Cc: Tejas Upadhyay > Cc: Andy Shevchenko > Cc: Aravind Iddamsetty > Cc: Alan Previn > Cc: Bruce Chang > Cc: intel-gfx@lists.freedesktop.org > Signed-off-by: John Harrison > --- > drivers/gpu/drm/i915/gt/intel_context.c | 1 + > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 7 ++++++- > drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 11 +++++++++++ > drivers/gpu/drm/i915/i915_gpu_error.c | 5 ++--- > 4 files changed, 20 insertions(+), 4 deletions(-) > > diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c > index e94365b08f1ef..df64cf1954c1d 100644 > --- a/drivers/gpu/drm/i915/gt/intel_context.c > +++ b/drivers/gpu/drm/i915/gt/intel_context.c > @@ -552,6 +552,7 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce) > > active = rq; > } > + active = i915_request_get_rcu(active); > spin_unlock_irqrestore(&parent->guc_state.lock, flags); > > return active; > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c > index 922f1bb22dc68..517d1fb7ae333 100644 > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c > @@ -2236,10 +2236,13 @@ static void engine_dump_active_requests(struct intel_engine_cs *engine, struct d > guc = intel_uc_uses_guc_submission(&engine->gt->uc); > if (guc) { > ce = intel_engine_get_hung_context(engine); > - if (ce) > + if (ce) { > + /* This will reference count the request (if found) */ > hung_rq = intel_context_find_active_request(ce); > + } > } else { > hung_rq = intel_engine_execlist_find_hung_request(engine); > + hung_rq = i915_request_get_rcu(hung_rq); Looks like intel_engine_execlist_find_hung_request can return NULL which i915_request_get_rcu will not handle. Maybe it would come up simpler if intel_context_find_active_request wouldn't be getting the reference and then you can get one here at a single place for both branches? > } > > if (hung_rq) > @@ -2250,6 +2253,8 @@ static void engine_dump_active_requests(struct intel_engine_cs *engine, struct d > else > intel_engine_dump_active_requests(&engine->sched_engine->requests, > hung_rq, m); > + if (hung_rq) > + i915_request_put(hung_rq); > } > > void intel_engine_dump(struct intel_engine_cs *engine, > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > index b436dd7f12e42..3b34a82d692be 100644 > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > @@ -4820,6 +4820,8 @@ void intel_guc_find_hung_context(struct intel_engine_cs *engine) > > xa_lock_irqsave(&guc->context_lookup, flags); > xa_for_each(&guc->context_lookup, index, ce) { > + bool found; > + > if (!kref_get_unless_zero(&ce->ref)) > continue; > > @@ -4836,10 +4838,18 @@ void intel_guc_find_hung_context(struct intel_engine_cs *engine) > goto next; > } > > + found = false; > + spin_lock(&ce->guc_state.lock); > list_for_each_entry(rq, &ce->guc_state.requests, sched.link) { > if (i915_test_request_state(rq) != I915_REQUEST_ACTIVE) > continue; > > + found = true; > + break; > + } > + spin_unlock(&ce->guc_state.lock); > + > + if (found) { > intel_engine_set_hung_context(engine, ce); > > /* Can only cope with one hang at a time... */ > @@ -4847,6 +4857,7 @@ void intel_guc_find_hung_context(struct intel_engine_cs *engine) > xa_lock(&guc->context_lookup); > goto done; > } > + > next: > intel_context_put(ce); > xa_lock(&guc->context_lookup); This hunk I have to leave for someone who know the GuC backend well. Regards, Tvrtko > diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c > index 9d5d5a397b64e..4107a0dfcca7d 100644 > --- a/drivers/gpu/drm/i915/i915_gpu_error.c > +++ b/drivers/gpu/drm/i915/i915_gpu_error.c > @@ -1607,6 +1607,7 @@ capture_engine(struct intel_engine_cs *engine, > ce = intel_engine_get_hung_context(engine); > if (ce) { > intel_engine_clear_hung_context(engine); > + /* This will reference count the request (if found) */ > rq = intel_context_find_active_request(ce); > if (!rq || !i915_request_started(rq)) > goto no_request_capture; > @@ -1618,13 +1619,11 @@ capture_engine(struct intel_engine_cs *engine, > if (!intel_uc_uses_guc_submission(&engine->gt->uc)) { > spin_lock_irqsave(&engine->sched_engine->lock, flags); > rq = intel_engine_execlist_find_hung_request(engine); > + rq = i915_request_get_rcu(rq); > spin_unlock_irqrestore(&engine->sched_engine->lock, > flags); > } > } > - if (rq) > - rq = i915_request_get_rcu(rq); > - > if (!rq) > goto no_request_capture; > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D3492C38159 for ; Wed, 18 Jan 2023 16:23:17 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4D58310E792; Wed, 18 Jan 2023 16:23:13 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 05DDF10E792; Wed, 18 Jan 2023 16:23:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1674058990; x=1705594990; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=9Fi480uzS5oBGcDSvrekY97oLJsbYkd3EZfuIFpLe2U=; b=IVJDYp3WmwYjaIbPaCbspXRiM0LXxSWK6rOoAqaXjq7cJDK7vkUbWN97 H0O5YzZo6ReP0oCZZhl5SuPa3evqOpLUuFga1xPXW4KHQ3uHp67gvIqR7 zPF7Dbltye4rfM/r9bJlcl734h/4ylbIUrBq+gjs3lxdMWqPR7FTuapc/ P+vbSxLbfcXTJiszD3h8suavSP5X+5pSVDYE+cREKeqSwbHRU98XonUeG V8CLTq4znMlcfricnjgGIiDwEIyKuvWNZfyns6oktnJeZlIquE7vjeo7A 9QONp8A+59R6y/25xHn8gGSeMceqh/ipGU7YJORCg+gJZgQizIEXK9fFO Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10594"; a="312899539" X-IronPort-AV: E=Sophos;i="5.97,226,1669104000"; d="scan'208";a="312899539" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jan 2023 08:22:36 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10594"; a="988612254" X-IronPort-AV: E=Sophos;i="5.97,226,1669104000"; d="scan'208";a="988612254" Received: from llvincen-mobl.ger.corp.intel.com (HELO [10.213.215.37]) ([10.213.215.37]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jan 2023 08:22:31 -0800 Message-ID: Date: Wed, 18 Jan 2023 16:22:29 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Content-Language: en-US To: John.C.Harrison@Intel.com, Intel-GFX@Lists.FreeDesktop.Org References: <20230117213630.2897570-1-John.C.Harrison@Intel.com> <20230117213630.2897570-2-John.C.Harrison@Intel.com> From: Tvrtko Ursulin Organization: Intel Corporation UK Plc In-Reply-To: <20230117213630.2897570-2-John.C.Harrison@Intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Intel-gfx] [PATCH v2 1/5] drm/i915: Fix request locking during error capture & debugfs dump X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andy Shevchenko , Michael Cheng , Alan Previn , Lucas De Marchi , Chris Wilson , DRI-Devel@Lists.FreeDesktop.Org, Andrzej Hajda , Rodrigo Vivi , Tejas Upadhyay , Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" On 17/01/2023 21:36, John.C.Harrison@Intel.com wrote: > From: John Harrison > > When GuC support was added to error capture, the locking around the > request object was broken. Fix it up. > > The context based search manages the spinlocking around the search > internally. So it needs to grab the reference count internally as > well. The execlist only request based search relies on external > locking, so it needs an external reference count. So no change to that > code itself but the context version does change. > > The only other caller is the code for dumping engine state to debugfs. > That code wasn't previously getting an explicit reference at all as it > does everything while holding the execlist specific spinlock. So that > needs updaing as well as that spinlock doesn't help when using GuC > submission. Rather than trying to conditionally get/put depending on > submission model, just change it to always do the get/put. > > In addition, intel_guc_find_hung_context() was not acquiring the > correct spinlock before searching the request list. So fix that up too. > > Fixes: dc0dad365c5e ("drm/i915/guc: Fix for error capture after full GPU reset > with GuC") > Fixes: 573ba126aef3 ("drm/i915/guc: Capture error state on context reset") > Cc: Matthew Brost > Cc: John Harrison > Cc: Jani Nikula > Cc: Joonas Lahtinen > Cc: Rodrigo Vivi > Cc: Tvrtko Ursulin > Cc: Daniele Ceraolo Spurio > Cc: Andrzej Hajda > Cc: Chris Wilson > Cc: Matthew Auld > Cc: Matt Roper > Cc: Umesh Nerlige Ramappa > Cc: Michael Cheng > Cc: Lucas De Marchi > Cc: Tejas Upadhyay > Cc: Andy Shevchenko > Cc: Aravind Iddamsetty > Cc: Alan Previn > Cc: Bruce Chang > Cc: intel-gfx@lists.freedesktop.org > Signed-off-by: John Harrison > --- > drivers/gpu/drm/i915/gt/intel_context.c | 1 + > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 7 ++++++- > drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 11 +++++++++++ > drivers/gpu/drm/i915/i915_gpu_error.c | 5 ++--- > 4 files changed, 20 insertions(+), 4 deletions(-) > > diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c > index e94365b08f1ef..df64cf1954c1d 100644 > --- a/drivers/gpu/drm/i915/gt/intel_context.c > +++ b/drivers/gpu/drm/i915/gt/intel_context.c > @@ -552,6 +552,7 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce) > > active = rq; > } > + active = i915_request_get_rcu(active); > spin_unlock_irqrestore(&parent->guc_state.lock, flags); > > return active; > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c > index 922f1bb22dc68..517d1fb7ae333 100644 > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c > @@ -2236,10 +2236,13 @@ static void engine_dump_active_requests(struct intel_engine_cs *engine, struct d > guc = intel_uc_uses_guc_submission(&engine->gt->uc); > if (guc) { > ce = intel_engine_get_hung_context(engine); > - if (ce) > + if (ce) { > + /* This will reference count the request (if found) */ > hung_rq = intel_context_find_active_request(ce); > + } > } else { > hung_rq = intel_engine_execlist_find_hung_request(engine); > + hung_rq = i915_request_get_rcu(hung_rq); Looks like intel_engine_execlist_find_hung_request can return NULL which i915_request_get_rcu will not handle. Maybe it would come up simpler if intel_context_find_active_request wouldn't be getting the reference and then you can get one here at a single place for both branches? > } > > if (hung_rq) > @@ -2250,6 +2253,8 @@ static void engine_dump_active_requests(struct intel_engine_cs *engine, struct d > else > intel_engine_dump_active_requests(&engine->sched_engine->requests, > hung_rq, m); > + if (hung_rq) > + i915_request_put(hung_rq); > } > > void intel_engine_dump(struct intel_engine_cs *engine, > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > index b436dd7f12e42..3b34a82d692be 100644 > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c > @@ -4820,6 +4820,8 @@ void intel_guc_find_hung_context(struct intel_engine_cs *engine) > > xa_lock_irqsave(&guc->context_lookup, flags); > xa_for_each(&guc->context_lookup, index, ce) { > + bool found; > + > if (!kref_get_unless_zero(&ce->ref)) > continue; > > @@ -4836,10 +4838,18 @@ void intel_guc_find_hung_context(struct intel_engine_cs *engine) > goto next; > } > > + found = false; > + spin_lock(&ce->guc_state.lock); > list_for_each_entry(rq, &ce->guc_state.requests, sched.link) { > if (i915_test_request_state(rq) != I915_REQUEST_ACTIVE) > continue; > > + found = true; > + break; > + } > + spin_unlock(&ce->guc_state.lock); > + > + if (found) { > intel_engine_set_hung_context(engine, ce); > > /* Can only cope with one hang at a time... */ > @@ -4847,6 +4857,7 @@ void intel_guc_find_hung_context(struct intel_engine_cs *engine) > xa_lock(&guc->context_lookup); > goto done; > } > + > next: > intel_context_put(ce); > xa_lock(&guc->context_lookup); This hunk I have to leave for someone who know the GuC backend well. Regards, Tvrtko > diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c > index 9d5d5a397b64e..4107a0dfcca7d 100644 > --- a/drivers/gpu/drm/i915/i915_gpu_error.c > +++ b/drivers/gpu/drm/i915/i915_gpu_error.c > @@ -1607,6 +1607,7 @@ capture_engine(struct intel_engine_cs *engine, > ce = intel_engine_get_hung_context(engine); > if (ce) { > intel_engine_clear_hung_context(engine); > + /* This will reference count the request (if found) */ > rq = intel_context_find_active_request(ce); > if (!rq || !i915_request_started(rq)) > goto no_request_capture; > @@ -1618,13 +1619,11 @@ capture_engine(struct intel_engine_cs *engine, > if (!intel_uc_uses_guc_submission(&engine->gt->uc)) { > spin_lock_irqsave(&engine->sched_engine->lock, flags); > rq = intel_engine_execlist_find_hung_request(engine); > + rq = i915_request_get_rcu(rq); > spin_unlock_irqrestore(&engine->sched_engine->lock, > flags); > } > } > - if (rq) > - rq = i915_request_get_rcu(rq); > - > if (!rq) > goto no_request_capture; >