* [PATCH v2 1/2] drm/panfrost: Handle resetting on timeout better
@ 2019-10-09 9:44 Steven Price
2019-10-09 9:44 ` [PATCH v2 2/2] drm/panfrost: Remove commented out call to panfrost_core_dump Steven Price
2019-10-09 18:39 ` [PATCH v2 1/2] drm/panfrost: Handle resetting on timeout better Rob Herring
0 siblings, 2 replies; 3+ messages in thread
From: Steven Price @ 2019-10-09 9:44 UTC (permalink / raw)
To: Daniel Vetter, David Airlie, Rob Herring, Tomeu Vizoso
Cc: Alyssa Rosenzweig, Steven Price, dri-devel, linux-kernel, Neil Armstrong
Panfrost uses multiple schedulers (one for each slot, so 2 in reality),
and on a timeout has to stop all the schedulers to safely perform a
reset. However more than one scheduler can trigger a timeout at the same
time. This race condition results in jobs being freed while they are
still in use.
When stopping other slots use cancel_delayed_work_sync() to ensure that
any timeout started for that slot has completed. Also use
mutex_trylock() to obtain reset_lock. This means that only one thread
attempts the reset, the other threads will simply complete without doing
anything (the first thread will wait for this in the call to
cancel_delayed_work_sync()).
While we're here and since the function is already dependent on
sched_job not being NULL, let's remove the unnecessary checks.
Fixes: aa20236784ab ("drm/panfrost: Prevent concurrent resets")
Tested-by: Neil Armstrong <narmstrong@baylibre.com>
Signed-off-by: Steven Price <steven.price@arm.com>
---
v2:
* Added fixes and tested-by tags
* Moved cleanup of panfrost_core_dump() comment to separate patch
drivers/gpu/drm/panfrost/panfrost_job.c | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index a58551668d9a..21f34d44aac2 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -381,13 +381,19 @@ static void panfrost_job_timedout(struct drm_sched_job *sched_job)
job_read(pfdev, JS_TAIL_LO(js)),
sched_job);
- mutex_lock(&pfdev->reset_lock);
+ if (!mutex_trylock(&pfdev->reset_lock))
+ return;
- for (i = 0; i < NUM_JOB_SLOTS; i++)
- drm_sched_stop(&pfdev->js->queue[i].sched, sched_job);
+ for (i = 0; i < NUM_JOB_SLOTS; i++) {
+ struct drm_gpu_scheduler *sched = &pfdev->js->queue[i].sched;
+
+ drm_sched_stop(sched, sched_job);
+ if (js != i)
+ /* Ensure any timeouts on other slots have finished */
+ cancel_delayed_work_sync(&sched->work_tdr);
+ }
- if (sched_job)
- drm_sched_increase_karma(sched_job);
+ drm_sched_increase_karma(sched_job);
spin_lock_irqsave(&pfdev->js->job_lock, flags);
for (i = 0; i < NUM_JOB_SLOTS; i++) {
--
2.20.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [PATCH v2 2/2] drm/panfrost: Remove commented out call to panfrost_core_dump
2019-10-09 9:44 [PATCH v2 1/2] drm/panfrost: Handle resetting on timeout better Steven Price
@ 2019-10-09 9:44 ` Steven Price
2019-10-09 18:39 ` [PATCH v2 1/2] drm/panfrost: Handle resetting on timeout better Rob Herring
1 sibling, 0 replies; 3+ messages in thread
From: Steven Price @ 2019-10-09 9:44 UTC (permalink / raw)
To: Daniel Vetter, David Airlie, Rob Herring, Tomeu Vizoso
Cc: Alyssa Rosenzweig, Steven Price, dri-devel, linux-kernel
panfrost_core_dump() has never existed in mainline, so remove it and add
a TODO entry that core dump support is currently lacking.
Signed-off-by: Steven Price <steven.price@arm.com>
---
v2:
* New patch
drivers/gpu/drm/panfrost/TODO | 2 ++
drivers/gpu/drm/panfrost/panfrost_job.c | 2 --
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/panfrost/TODO b/drivers/gpu/drm/panfrost/TODO
index 536a0d4f8d29..8c811a9e683b 100644
--- a/drivers/gpu/drm/panfrost/TODO
+++ b/drivers/gpu/drm/panfrost/TODO
@@ -10,3 +10,5 @@
- Compute job support. So called 'compute only' jobs need to be plumbed up to
userspace.
+
+- Support core dump on job failure
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 21f34d44aac2..33bf25ba506e 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -404,8 +404,6 @@ static void panfrost_job_timedout(struct drm_sched_job *sched_job)
}
spin_unlock_irqrestore(&pfdev->js->job_lock, flags);
- /* panfrost_core_dump(pfdev); */
-
panfrost_devfreq_record_transition(pfdev, js);
panfrost_device_reset(pfdev);
--
2.20.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v2 1/2] drm/panfrost: Handle resetting on timeout better
2019-10-09 9:44 [PATCH v2 1/2] drm/panfrost: Handle resetting on timeout better Steven Price
2019-10-09 9:44 ` [PATCH v2 2/2] drm/panfrost: Remove commented out call to panfrost_core_dump Steven Price
@ 2019-10-09 18:39 ` Rob Herring
1 sibling, 0 replies; 3+ messages in thread
From: Rob Herring @ 2019-10-09 18:39 UTC (permalink / raw)
To: Steven Price
Cc: Daniel Vetter, David Airlie, Tomeu Vizoso, Alyssa Rosenzweig,
dri-devel, linux-kernel, Neil Armstrong
On Wed, Oct 9, 2019 at 4:45 AM Steven Price <steven.price@arm.com> wrote:
>
> Panfrost uses multiple schedulers (one for each slot, so 2 in reality),
> and on a timeout has to stop all the schedulers to safely perform a
> reset. However more than one scheduler can trigger a timeout at the same
> time. This race condition results in jobs being freed while they are
> still in use.
>
> When stopping other slots use cancel_delayed_work_sync() to ensure that
> any timeout started for that slot has completed. Also use
> mutex_trylock() to obtain reset_lock. This means that only one thread
> attempts the reset, the other threads will simply complete without doing
> anything (the first thread will wait for this in the call to
> cancel_delayed_work_sync()).
>
> While we're here and since the function is already dependent on
> sched_job not being NULL, let's remove the unnecessary checks.
>
> Fixes: aa20236784ab ("drm/panfrost: Prevent concurrent resets")
> Tested-by: Neil Armstrong <narmstrong@baylibre.com>
> Signed-off-by: Steven Price <steven.price@arm.com>
> ---
> v2:
> * Added fixes and tested-by tags
> * Moved cleanup of panfrost_core_dump() comment to separate patch
>
> drivers/gpu/drm/panfrost/panfrost_job.c | 16 +++++++++++-----
> 1 file changed, 11 insertions(+), 5 deletions(-)
Both patches applied.
Rob
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-10-09 18:40 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-09 9:44 [PATCH v2 1/2] drm/panfrost: Handle resetting on timeout better Steven Price
2019-10-09 9:44 ` [PATCH v2 2/2] drm/panfrost: Remove commented out call to panfrost_core_dump Steven Price
2019-10-09 18:39 ` [PATCH v2 1/2] drm/panfrost: Handle resetting on timeout better Rob Herring
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).