* [Intel-gfx] [PATCH v5 00/20] drm/sched dependency handling and implicit sync fixes
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development; +Cc: Intel Graphics Development, Daniel Vetter
Hi all,
Two big changes:
- bikeshed repainted in new paint, pls don't touch, it's all fresh! The
functions are now called _add_dependency and _add_implicit_dependencies.
- msm conversion, which includes a bugfix for the msm drm/sched
conversion. I think it would be really good if the first two patches
could land asap, but that means testing by some of the other drivers.
Etnaviv especially is pending some testing/reviewed-by.
In general please review and test.
Thanks, Daniel
Daniel Vetter (20):
drm/sched: Split drm_sched_job_init
drm/msm: Fix drm/sched point of no return rules
drm/sched: Barriers are needed for entity->last_scheduled
drm/sched: Add dependency tracking
drm/sched: drop entity parameter from drm_sched_push_job
drm/sched: improve docs around drm_sched_entity
drm/panfrost: use scheduler dependency tracking
drm/lima: use scheduler dependency tracking
drm/v3d: Move drm_sched_job_init to v3d_job_init
drm/v3d: Use scheduler dependency handling
drm/etnaviv: Use scheduler dependency handling
drm/msm: Use scheduler dependency handling
drm/gem: Delete gem array fencing helpers
drm/sched: Don't store self-dependencies
drm/sched: Check locking in drm_sched_job_await_implicit
drm/msm: Don't break exclusive fence ordering
drm/etnaviv: Don't break exclusive fence ordering
drm/i915: delete exclude argument from i915_sw_fence_await_reservation
drm/i915: Don't break exclusive fence ordering
dma-resv: Give the docs a do-over
Documentation/gpu/drm-mm.rst | 3 +
drivers/dma-buf/dma-resv.c | 24 ++-
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 4 +-
drivers/gpu/drm/drm_gem.c | 96 ---------
drivers/gpu/drm/etnaviv/etnaviv_gem.h | 5 +-
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 66 +++---
drivers/gpu/drm/etnaviv/etnaviv_sched.c | 65 +-----
drivers/gpu/drm/etnaviv/etnaviv_sched.h | 3 +-
drivers/gpu/drm/i915/display/intel_display.c | 4 +-
drivers/gpu/drm/i915/gem/i915_gem_clflush.c | 2 +-
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 6 +-
drivers/gpu/drm/i915/i915_sw_fence.c | 6 +-
drivers/gpu/drm/i915/i915_sw_fence.h | 1 -
drivers/gpu/drm/lima/lima_gem.c | 9 +-
drivers/gpu/drm/lima/lima_sched.c | 28 +--
drivers/gpu/drm/lima/lima_sched.h | 6 +-
drivers/gpu/drm/msm/msm_gem.h | 5 -
drivers/gpu/drm/msm/msm_gem_submit.c | 36 ++--
drivers/gpu/drm/msm/msm_ringbuffer.c | 12 --
drivers/gpu/drm/panfrost/panfrost_drv.c | 16 +-
drivers/gpu/drm/panfrost/panfrost_job.c | 40 +---
drivers/gpu/drm/panfrost/panfrost_job.h | 5 +-
drivers/gpu/drm/scheduler/sched_entity.c | 140 +++++++------
drivers/gpu/drm/scheduler/sched_fence.c | 19 +-
drivers/gpu/drm/scheduler/sched_main.c | 182 ++++++++++++++++-
drivers/gpu/drm/v3d/v3d_drv.h | 6 +-
drivers/gpu/drm/v3d/v3d_gem.c | 114 +++++------
drivers/gpu/drm/v3d/v3d_sched.c | 44 +---
include/drm/drm_gem.h | 5 -
include/drm/gpu_scheduler.h | 188 +++++++++++++++---
include/linux/dma-buf.h | 7 +
include/linux/dma-resv.h | 104 +++++++++-
33 files changed, 693 insertions(+), 562 deletions(-)
--
2.32.0
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 01/20] drm/sched: Split drm_sched_job_init
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Melissa Wen, Emma Anholt, Steven Price, Boris Brezillon,
Daniel Vetter, Lucas Stach, Russell King, Christian Gmeiner,
Qiang Yu, Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig,
David Airlie, Daniel Vetter, Sumit Semwal, Christian König,
Masahiro Yamada, Kees Cook, Adam Borowski, Nick Terrell,
Mauro Carvalho Chehab, Paul Menzel, Sami Tolvanen, Viresh Kumar,
Alex Deucher, Dave Airlie, Nirmoy Das, Deepak R Varma, Lee Jones,
Kevin Wang, Chen Li, Luben Tuikov, Marek Olšák,
Dennis Li, Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang,
Tian Tao, etnaviv, lima, linux-media, linaro-mm-sig, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
This is a very confusingly named function, because not just does it
init an object, it arms it and provides a point of no return for
pushing a job into the scheduler. It would be nice if that's a bit
clearer in the interface.
But the real reason is that I want to push the dependency tracking
helpers into the scheduler code, and that means drm_sched_job_init
must be called a lot earlier, without arming the job.
v2:
- don't change .gitignore (Steven)
- don't forget v3d (Emma)
v3: Emma noticed that I leak the memory allocated in
drm_sched_job_init if we bail out before the point of no return in
subsequent driver patches. To be able to fix this change
drm_sched_job_cleanup() so it can handle being called both before and
after drm_sched_job_arm().
Also improve the kerneldoc for this.
v4:
- Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
usual (Melissa)
- Christian pointed out that drm_sched_entity_select_rq() also needs
to be moved into drm_sched_job_arm, which made me realize that the
job->id definitely needs to be moved too.
Shuffle things to fit between job_init and job_arm.
v5:
Reshuffle the split between init/arm once more, amdgpu abuses
drm_sched.ready to signal gpu reset failures. Also document this
somewhat. (Christian)
v6:
Rebase on top of the msm drm/sched support. Note that the
drm_sched_job_init() call is completely misplaced, and hence also the
split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
patch will address.
Acked-by: Melissa Wen <mwen@igalia.com>
Cc: Melissa Wen <melissa.srw@gmail.com>
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Steven Price <steven.price@arm.com> (v2)
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: Qiang Yu <yuq825@gmail.com>
Cc: Rob Herring <robh@kernel.org>
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Adam Borowski <kilobyte@angband.pl>
Cc: Nick Terrell <terrelln@fb.com>
Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Cc: Paul Menzel <pmenzel@molgen.mpg.de>
Cc: Sami Tolvanen <samitolvanen@google.com>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Nirmoy Das <nirmoy.das@amd.com>
Cc: Deepak R Varma <mh12gx2825@gmail.com>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Kevin Wang <kevin1.wang@amd.com>
Cc: Chen Li <chenli@uniontech.com>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: "Marek Olšák" <marek.olsak@amd.com>
Cc: Dennis Li <Dennis.Li@amd.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Sonny Jiang <sonny.jiang@amd.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Tian Tao <tiantao6@hisilicon.com>
Cc: etnaviv@lists.freedesktop.org
Cc: lima@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: Emma Anholt <emma@anholt.net>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
---
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
drivers/gpu/drm/lima/lima_sched.c | 2 +
drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++
drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
drivers/gpu/drm/v3d/v3d_gem.c | 2 +
include/drm/gpu_scheduler.h | 7 ++-
11 files changed, 94 insertions(+), 22 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 139cd3bf1ad6..32e80bc6af22 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
if (r)
goto error_unlock;
+ drm_sched_job_arm(&job->base);
+
/* No memory allocation is allowed while holding the notifier lock.
* The lock is held until amdgpu_cs_submit is finished and fence is
* added to BOs.
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index d33e6d97cc89..5ddb955d2315 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
if (r)
return r;
+ drm_sched_job_arm(&job->base);
+
*f = dma_fence_get(&job->base.s_fence->finished);
amdgpu_job_free_resources(job);
drm_sched_entity_push_job(&job->base, entity);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index feb6da1b6ceb..05f412204118 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
if (ret)
goto out_unlock;
+ drm_sched_job_arm(&submit->sched_job);
+
submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
submit->out_fence, 0,
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index dba8329937a3..38f755580507 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
return err;
}
+ drm_sched_job_arm(&task->base);
+
task->num_bos = num_bos;
task->vm = lima_vm_get(vm);
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index fdc5367aecaa..6d6c44f0e1f3 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,6 +52,9 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
+ /* FIXME: this is way too early */
+ drm_sched_job_arm(&job->base);
+
xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
kref_init(&submit->ref);
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 71a72fb50e6b..2992dc85325f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
goto unlock;
}
+ drm_sched_job_arm(&job->base);
+
job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 79554aa4dbb1..f7347c284886 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
* @sched_job: job to submit
* @entity: scheduler entity
*
- * Note: To guarantee that the order of insertion to queue matches
- * the job's fence sequence number this function should be
- * called with drm_sched_job_init under common lock.
+ * Note: To guarantee that the order of insertion to queue matches the job's
+ * fence sequence number this function should be called with drm_sched_job_arm()
+ * under common lock.
*
* Returns 0 for success, negative error code otherwise.
*/
diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
index 69de2c76731f..bcea035cf4c6 100644
--- a/drivers/gpu/drm/scheduler/sched_fence.c
+++ b/drivers/gpu/drm/scheduler/sched_fence.c
@@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
*
* Free up the fence memory after the RCU grace period.
*/
-static void drm_sched_fence_free(struct rcu_head *rcu)
+void drm_sched_fence_free(struct rcu_head *rcu)
{
struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
struct drm_sched_fence *fence = to_drm_sched_fence(f);
@@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
}
EXPORT_SYMBOL(to_drm_sched_fence);
-struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
- void *owner)
+struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
+ void *owner)
{
struct drm_sched_fence *fence = NULL;
- unsigned seq;
fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
if (fence == NULL)
return NULL;
fence->owner = owner;
- fence->sched = entity->rq->sched;
spin_lock_init(&fence->lock);
+ return fence;
+}
+
+void drm_sched_fence_init(struct drm_sched_fence *fence,
+ struct drm_sched_entity *entity)
+{
+ unsigned seq;
+
+ fence->sched = entity->rq->sched;
seq = atomic_inc_return(&entity->fence_seq);
dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
&fence->lock, entity->fence_context, seq);
dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
&fence->lock, entity->fence_context + 1, seq);
-
- return fence;
}
module_init(drm_sched_fence_slab_init);
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index 33c414d55fab..454cb6164bdc 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -48,9 +48,11 @@
#include <linux/wait.h>
#include <linux/sched.h>
#include <linux/completion.h>
+#include <linux/dma-resv.h>
#include <uapi/linux/sched/types.h>
#include <drm/drm_print.h>
+#include <drm/drm_gem.h>
#include <drm/gpu_scheduler.h>
#include <drm/spsc_queue.h>
@@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
/**
* drm_sched_job_init - init a scheduler job
- *
* @job: scheduler job to init
* @entity: scheduler entity to use
* @owner: job owner for debugging
@@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
* Refer to drm_sched_entity_push_job() documentation
* for locking considerations.
*
+ * Drivers must make sure drm_sched_job_cleanup() if this function returns
+ * successfully, even when @job is aborted before drm_sched_job_arm() is called.
+ *
+ * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
+ * has died, which can mean that there's no valid runqueue for a @entity.
+ * This function returns -ENOENT in this case (which probably should be -EIO as
+ * a more meanigful return value).
+ *
* Returns 0 for success, negative error code otherwise.
*/
int drm_sched_job_init(struct drm_sched_job *job,
struct drm_sched_entity *entity,
void *owner)
{
- struct drm_gpu_scheduler *sched;
-
drm_sched_entity_select_rq(entity);
if (!entity->rq)
return -ENOENT;
- sched = entity->rq->sched;
-
- job->sched = sched;
job->entity = entity;
- job->s_priority = entity->rq - sched->sched_rq;
- job->s_fence = drm_sched_fence_create(entity, owner);
+ job->s_fence = drm_sched_fence_alloc(entity, owner);
if (!job->s_fence)
return -ENOMEM;
- job->id = atomic64_inc_return(&sched->job_id_count);
INIT_LIST_HEAD(&job->list);
@@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
EXPORT_SYMBOL(drm_sched_job_init);
/**
- * drm_sched_job_cleanup - clean up scheduler job resources
+ * drm_sched_job_arm - arm a scheduler job for execution
+ * @job: scheduler job to arm
+ *
+ * This arms a scheduler job for execution. Specifically it initializes the
+ * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
+ * or other places that need to track the completion of this job.
+ *
+ * Refer to drm_sched_entity_push_job() documentation for locking
+ * considerations.
*
+ * This can only be called if drm_sched_job_init() succeeded.
+ */
+void drm_sched_job_arm(struct drm_sched_job *job)
+{
+ struct drm_gpu_scheduler *sched;
+ struct drm_sched_entity *entity = job->entity;
+
+ BUG_ON(!entity);
+
+ sched = entity->rq->sched;
+
+ job->sched = sched;
+ job->s_priority = entity->rq - sched->sched_rq;
+ job->id = atomic64_inc_return(&sched->job_id_count);
+
+ drm_sched_fence_init(job->s_fence, job->entity);
+}
+EXPORT_SYMBOL(drm_sched_job_arm);
+
+/**
+ * drm_sched_job_cleanup - clean up scheduler job resources
* @job: scheduler job to clean up
+ *
+ * Cleans up the resources allocated with drm_sched_job_init().
+ *
+ * Drivers should call this from their error unwind code if @job is aborted
+ * before drm_sched_job_arm() is called.
+ *
+ * After that point of no return @job is committed to be executed by the
+ * scheduler, and this function should be called from the
+ * &drm_sched_backend_ops.free_job callback.
*/
void drm_sched_job_cleanup(struct drm_sched_job *job)
{
- dma_fence_put(&job->s_fence->finished);
+ if (kref_read(&job->s_fence->finished.refcount)) {
+ /* drm_sched_job_arm() has been called */
+ dma_fence_put(&job->s_fence->finished);
+ } else {
+ /* aborted job before committing to run it */
+ drm_sched_fence_free(&job->s_fence->finished.rcu);
+ }
+
job->s_fence = NULL;
}
EXPORT_SYMBOL(drm_sched_job_cleanup);
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 5689da118197..2e808097b4d1 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
if (ret)
return ret;
+ drm_sched_job_arm(&job->base);
+
job->done_fence = dma_fence_get(&job->base.s_fence->finished);
/* put by scheduler job completion */
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index 88ae7f331bb1..83afc3aa8e2f 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
int drm_sched_job_init(struct drm_sched_job *job,
struct drm_sched_entity *entity,
void *owner);
+void drm_sched_job_arm(struct drm_sched_job *job);
void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
struct drm_gpu_scheduler **sched_list,
unsigned int num_sched_list);
@@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
enum drm_sched_priority priority);
bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
-struct drm_sched_fence *drm_sched_fence_create(
+struct drm_sched_fence *drm_sched_fence_alloc(
struct drm_sched_entity *s_entity, void *owner);
+void drm_sched_fence_init(struct drm_sched_fence *fence,
+ struct drm_sched_entity *entity);
+void drm_sched_fence_free(struct rcu_head *rcu);
+
void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
void drm_sched_fence_finished(struct drm_sched_fence *fence);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 01/20] drm/sched: Split drm_sched_job_init
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Melissa Wen, Emma Anholt, Steven Price, Boris Brezillon,
Daniel Vetter, Lucas Stach, Russell King, Christian Gmeiner,
Qiang Yu, Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig,
David Airlie, Daniel Vetter, Sumit Semwal, Christian König,
Masahiro Yamada, Kees Cook, Adam Borowski, Nick Terrell,
Mauro Carvalho Chehab, Paul Menzel, Sami Tolvanen, Viresh Kumar,
Alex Deucher, Dave Airlie, Nirmoy Das, Deepak R Varma, Lee Jones,
Kevin Wang, Chen Li, Luben Tuikov, Marek Olšák,
Dennis Li, Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang,
Tian Tao, etnaviv, lima, linux-media, linaro-mm-sig, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
This is a very confusingly named function, because not just does it
init an object, it arms it and provides a point of no return for
pushing a job into the scheduler. It would be nice if that's a bit
clearer in the interface.
But the real reason is that I want to push the dependency tracking
helpers into the scheduler code, and that means drm_sched_job_init
must be called a lot earlier, without arming the job.
v2:
- don't change .gitignore (Steven)
- don't forget v3d (Emma)
v3: Emma noticed that I leak the memory allocated in
drm_sched_job_init if we bail out before the point of no return in
subsequent driver patches. To be able to fix this change
drm_sched_job_cleanup() so it can handle being called both before and
after drm_sched_job_arm().
Also improve the kerneldoc for this.
v4:
- Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
usual (Melissa)
- Christian pointed out that drm_sched_entity_select_rq() also needs
to be moved into drm_sched_job_arm, which made me realize that the
job->id definitely needs to be moved too.
Shuffle things to fit between job_init and job_arm.
v5:
Reshuffle the split between init/arm once more, amdgpu abuses
drm_sched.ready to signal gpu reset failures. Also document this
somewhat. (Christian)
v6:
Rebase on top of the msm drm/sched support. Note that the
drm_sched_job_init() call is completely misplaced, and hence also the
split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
patch will address.
Acked-by: Melissa Wen <mwen@igalia.com>
Cc: Melissa Wen <melissa.srw@gmail.com>
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Steven Price <steven.price@arm.com> (v2)
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: Qiang Yu <yuq825@gmail.com>
Cc: Rob Herring <robh@kernel.org>
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Adam Borowski <kilobyte@angband.pl>
Cc: Nick Terrell <terrelln@fb.com>
Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Cc: Paul Menzel <pmenzel@molgen.mpg.de>
Cc: Sami Tolvanen <samitolvanen@google.com>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Nirmoy Das <nirmoy.das@amd.com>
Cc: Deepak R Varma <mh12gx2825@gmail.com>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Kevin Wang <kevin1.wang@amd.com>
Cc: Chen Li <chenli@uniontech.com>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: "Marek Olšák" <marek.olsak@amd.com>
Cc: Dennis Li <Dennis.Li@amd.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Sonny Jiang <sonny.jiang@amd.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Tian Tao <tiantao6@hisilicon.com>
Cc: etnaviv@lists.freedesktop.org
Cc: lima@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: Emma Anholt <emma@anholt.net>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
---
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
drivers/gpu/drm/lima/lima_sched.c | 2 +
drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++
drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
drivers/gpu/drm/v3d/v3d_gem.c | 2 +
include/drm/gpu_scheduler.h | 7 ++-
11 files changed, 94 insertions(+), 22 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 139cd3bf1ad6..32e80bc6af22 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
if (r)
goto error_unlock;
+ drm_sched_job_arm(&job->base);
+
/* No memory allocation is allowed while holding the notifier lock.
* The lock is held until amdgpu_cs_submit is finished and fence is
* added to BOs.
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index d33e6d97cc89..5ddb955d2315 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
if (r)
return r;
+ drm_sched_job_arm(&job->base);
+
*f = dma_fence_get(&job->base.s_fence->finished);
amdgpu_job_free_resources(job);
drm_sched_entity_push_job(&job->base, entity);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index feb6da1b6ceb..05f412204118 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
if (ret)
goto out_unlock;
+ drm_sched_job_arm(&submit->sched_job);
+
submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
submit->out_fence, 0,
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index dba8329937a3..38f755580507 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
return err;
}
+ drm_sched_job_arm(&task->base);
+
task->num_bos = num_bos;
task->vm = lima_vm_get(vm);
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index fdc5367aecaa..6d6c44f0e1f3 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,6 +52,9 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
+ /* FIXME: this is way too early */
+ drm_sched_job_arm(&job->base);
+
xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
kref_init(&submit->ref);
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 71a72fb50e6b..2992dc85325f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
goto unlock;
}
+ drm_sched_job_arm(&job->base);
+
job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 79554aa4dbb1..f7347c284886 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
* @sched_job: job to submit
* @entity: scheduler entity
*
- * Note: To guarantee that the order of insertion to queue matches
- * the job's fence sequence number this function should be
- * called with drm_sched_job_init under common lock.
+ * Note: To guarantee that the order of insertion to queue matches the job's
+ * fence sequence number this function should be called with drm_sched_job_arm()
+ * under common lock.
*
* Returns 0 for success, negative error code otherwise.
*/
diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
index 69de2c76731f..bcea035cf4c6 100644
--- a/drivers/gpu/drm/scheduler/sched_fence.c
+++ b/drivers/gpu/drm/scheduler/sched_fence.c
@@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
*
* Free up the fence memory after the RCU grace period.
*/
-static void drm_sched_fence_free(struct rcu_head *rcu)
+void drm_sched_fence_free(struct rcu_head *rcu)
{
struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
struct drm_sched_fence *fence = to_drm_sched_fence(f);
@@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
}
EXPORT_SYMBOL(to_drm_sched_fence);
-struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
- void *owner)
+struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
+ void *owner)
{
struct drm_sched_fence *fence = NULL;
- unsigned seq;
fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
if (fence == NULL)
return NULL;
fence->owner = owner;
- fence->sched = entity->rq->sched;
spin_lock_init(&fence->lock);
+ return fence;
+}
+
+void drm_sched_fence_init(struct drm_sched_fence *fence,
+ struct drm_sched_entity *entity)
+{
+ unsigned seq;
+
+ fence->sched = entity->rq->sched;
seq = atomic_inc_return(&entity->fence_seq);
dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
&fence->lock, entity->fence_context, seq);
dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
&fence->lock, entity->fence_context + 1, seq);
-
- return fence;
}
module_init(drm_sched_fence_slab_init);
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index 33c414d55fab..454cb6164bdc 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -48,9 +48,11 @@
#include <linux/wait.h>
#include <linux/sched.h>
#include <linux/completion.h>
+#include <linux/dma-resv.h>
#include <uapi/linux/sched/types.h>
#include <drm/drm_print.h>
+#include <drm/drm_gem.h>
#include <drm/gpu_scheduler.h>
#include <drm/spsc_queue.h>
@@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
/**
* drm_sched_job_init - init a scheduler job
- *
* @job: scheduler job to init
* @entity: scheduler entity to use
* @owner: job owner for debugging
@@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
* Refer to drm_sched_entity_push_job() documentation
* for locking considerations.
*
+ * Drivers must make sure drm_sched_job_cleanup() if this function returns
+ * successfully, even when @job is aborted before drm_sched_job_arm() is called.
+ *
+ * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
+ * has died, which can mean that there's no valid runqueue for a @entity.
+ * This function returns -ENOENT in this case (which probably should be -EIO as
+ * a more meanigful return value).
+ *
* Returns 0 for success, negative error code otherwise.
*/
int drm_sched_job_init(struct drm_sched_job *job,
struct drm_sched_entity *entity,
void *owner)
{
- struct drm_gpu_scheduler *sched;
-
drm_sched_entity_select_rq(entity);
if (!entity->rq)
return -ENOENT;
- sched = entity->rq->sched;
-
- job->sched = sched;
job->entity = entity;
- job->s_priority = entity->rq - sched->sched_rq;
- job->s_fence = drm_sched_fence_create(entity, owner);
+ job->s_fence = drm_sched_fence_alloc(entity, owner);
if (!job->s_fence)
return -ENOMEM;
- job->id = atomic64_inc_return(&sched->job_id_count);
INIT_LIST_HEAD(&job->list);
@@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
EXPORT_SYMBOL(drm_sched_job_init);
/**
- * drm_sched_job_cleanup - clean up scheduler job resources
+ * drm_sched_job_arm - arm a scheduler job for execution
+ * @job: scheduler job to arm
+ *
+ * This arms a scheduler job for execution. Specifically it initializes the
+ * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
+ * or other places that need to track the completion of this job.
+ *
+ * Refer to drm_sched_entity_push_job() documentation for locking
+ * considerations.
*
+ * This can only be called if drm_sched_job_init() succeeded.
+ */
+void drm_sched_job_arm(struct drm_sched_job *job)
+{
+ struct drm_gpu_scheduler *sched;
+ struct drm_sched_entity *entity = job->entity;
+
+ BUG_ON(!entity);
+
+ sched = entity->rq->sched;
+
+ job->sched = sched;
+ job->s_priority = entity->rq - sched->sched_rq;
+ job->id = atomic64_inc_return(&sched->job_id_count);
+
+ drm_sched_fence_init(job->s_fence, job->entity);
+}
+EXPORT_SYMBOL(drm_sched_job_arm);
+
+/**
+ * drm_sched_job_cleanup - clean up scheduler job resources
* @job: scheduler job to clean up
+ *
+ * Cleans up the resources allocated with drm_sched_job_init().
+ *
+ * Drivers should call this from their error unwind code if @job is aborted
+ * before drm_sched_job_arm() is called.
+ *
+ * After that point of no return @job is committed to be executed by the
+ * scheduler, and this function should be called from the
+ * &drm_sched_backend_ops.free_job callback.
*/
void drm_sched_job_cleanup(struct drm_sched_job *job)
{
- dma_fence_put(&job->s_fence->finished);
+ if (kref_read(&job->s_fence->finished.refcount)) {
+ /* drm_sched_job_arm() has been called */
+ dma_fence_put(&job->s_fence->finished);
+ } else {
+ /* aborted job before committing to run it */
+ drm_sched_fence_free(&job->s_fence->finished.rcu);
+ }
+
job->s_fence = NULL;
}
EXPORT_SYMBOL(drm_sched_job_cleanup);
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 5689da118197..2e808097b4d1 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
if (ret)
return ret;
+ drm_sched_job_arm(&job->base);
+
job->done_fence = dma_fence_get(&job->base.s_fence->finished);
/* put by scheduler job completion */
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index 88ae7f331bb1..83afc3aa8e2f 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
int drm_sched_job_init(struct drm_sched_job *job,
struct drm_sched_entity *entity,
void *owner);
+void drm_sched_job_arm(struct drm_sched_job *job);
void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
struct drm_gpu_scheduler **sched_list,
unsigned int num_sched_list);
@@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
enum drm_sched_priority priority);
bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
-struct drm_sched_fence *drm_sched_fence_create(
+struct drm_sched_fence *drm_sched_fence_alloc(
struct drm_sched_entity *s_entity, void *owner);
+void drm_sched_fence_init(struct drm_sched_fence *fence,
+ struct drm_sched_entity *entity);
+void drm_sched_fence_free(struct rcu_head *rcu);
+
void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
void drm_sched_fence_finished(struct drm_sched_fence *fence);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 01/20] drm/sched: Split drm_sched_job_init
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 13:43 ` Christian König
-1 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:43 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Melissa Wen, Melissa Wen,
Emma Anholt, Steven Price, Boris Brezillon, Daniel Vetter,
Lucas Stach, Russell King, Christian Gmeiner, Qiang Yu,
Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig, David Airlie,
Daniel Vetter, Sumit Semwal, Masahiro Yamada, Kees Cook,
Adam Borowski, Nick Terrell, Mauro Carvalho Chehab, Paul Menzel,
Sami Tolvanen, Viresh Kumar, Alex Deucher, Dave Airlie,
Nirmoy Das, Deepak R Varma, Lee Jones, Kevin Wang, Chen Li,
Luben Tuikov, Marek Olšák, Dennis Li,
Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang, Tian Tao,
etnaviv, lima, linux-media, linaro-mm-sig, Rob Clark, Sean Paul,
linux-arm-msm, freedreno
Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> This is a very confusingly named function, because not just does it
> init an object, it arms it and provides a point of no return for
> pushing a job into the scheduler. It would be nice if that's a bit
> clearer in the interface.
>
> But the real reason is that I want to push the dependency tracking
> helpers into the scheduler code, and that means drm_sched_job_init
> must be called a lot earlier, without arming the job.
>
> v2:
> - don't change .gitignore (Steven)
> - don't forget v3d (Emma)
>
> v3: Emma noticed that I leak the memory allocated in
> drm_sched_job_init if we bail out before the point of no return in
> subsequent driver patches. To be able to fix this change
> drm_sched_job_cleanup() so it can handle being called both before and
> after drm_sched_job_arm().
>
> Also improve the kerneldoc for this.
>
> v4:
> - Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
> usual (Melissa)
>
> - Christian pointed out that drm_sched_entity_select_rq() also needs
> to be moved into drm_sched_job_arm, which made me realize that the
> job->id definitely needs to be moved too.
>
> Shuffle things to fit between job_init and job_arm.
>
> v5:
> Reshuffle the split between init/arm once more, amdgpu abuses
> drm_sched.ready to signal gpu reset failures. Also document this
> somewhat. (Christian)
>
> v6:
> Rebase on top of the msm drm/sched support. Note that the
> drm_sched_job_init() call is completely misplaced, and hence also the
> split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
> patch will address.
>
> Acked-by: Melissa Wen <mwen@igalia.com>
> Cc: Melissa Wen <melissa.srw@gmail.com>
> Acked-by: Emma Anholt <emma@anholt.net>
> Acked-by: Steven Price <steven.price@arm.com> (v2)
> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
At least the amdgpu parts look ok of hand, but I can't judge the rest I
think.
So only Acked-by: Christian König <christian.koenig@amd.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: David Airlie <airlied@linux.ie>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Cc: Kees Cook <keescook@chromium.org>
> Cc: Adam Borowski <kilobyte@angband.pl>
> Cc: Nick Terrell <terrelln@fb.com>
> Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> Cc: Paul Menzel <pmenzel@molgen.mpg.de>
> Cc: Sami Tolvanen <samitolvanen@google.com>
> Cc: Viresh Kumar <viresh.kumar@linaro.org>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Dave Airlie <airlied@redhat.com>
> Cc: Nirmoy Das <nirmoy.das@amd.com>
> Cc: Deepak R Varma <mh12gx2825@gmail.com>
> Cc: Lee Jones <lee.jones@linaro.org>
> Cc: Kevin Wang <kevin1.wang@amd.com>
> Cc: Chen Li <chenli@uniontech.com>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: "Marek Olšák" <marek.olsak@amd.com>
> Cc: Dennis Li <Dennis.Li@amd.com>
> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Sonny Jiang <sonny.jiang@amd.com>
> Cc: Boris Brezillon <boris.brezillon@collabora.com>
> Cc: Tian Tao <tiantao6@hisilicon.com>
> Cc: etnaviv@lists.freedesktop.org
> Cc: lima@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> Cc: Emma Anholt <emma@anholt.net>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
> drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
> drivers/gpu/drm/lima/lima_sched.c | 2 +
> drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++
> drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
> drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
> drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
> drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
> drivers/gpu/drm/v3d/v3d_gem.c | 2 +
> include/drm/gpu_scheduler.h | 7 ++-
> 11 files changed, 94 insertions(+), 22 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> index 139cd3bf1ad6..32e80bc6af22 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> @@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
> if (r)
> goto error_unlock;
>
> + drm_sched_job_arm(&job->base);
> +
> /* No memory allocation is allowed while holding the notifier lock.
> * The lock is held until amdgpu_cs_submit is finished and fence is
> * added to BOs.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> index d33e6d97cc89..5ddb955d2315 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> @@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
> if (r)
> return r;
>
> + drm_sched_job_arm(&job->base);
> +
> *f = dma_fence_get(&job->base.s_fence->finished);
> amdgpu_job_free_resources(job);
> drm_sched_entity_push_job(&job->base, entity);
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> index feb6da1b6ceb..05f412204118 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> @@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> if (ret)
> goto out_unlock;
>
> + drm_sched_job_arm(&submit->sched_job);
> +
> submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
> submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
> submit->out_fence, 0,
> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> index dba8329937a3..38f755580507 100644
> --- a/drivers/gpu/drm/lima/lima_sched.c
> +++ b/drivers/gpu/drm/lima/lima_sched.c
> @@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
> return err;
> }
>
> + drm_sched_job_arm(&task->base);
> +
> task->num_bos = num_bos;
> task->vm = lima_vm_get(vm);
>
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index fdc5367aecaa..6d6c44f0e1f3 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,6 +52,9 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> + /* FIXME: this is way too early */
> + drm_sched_job_arm(&job->base);
> +
> xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
>
> kref_init(&submit->ref);
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> index 71a72fb50e6b..2992dc85325f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> @@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
> goto unlock;
> }
>
> + drm_sched_job_arm(&job->base);
> +
> job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
>
> ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> index 79554aa4dbb1..f7347c284886 100644
> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> @@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
> * @sched_job: job to submit
> * @entity: scheduler entity
> *
> - * Note: To guarantee that the order of insertion to queue matches
> - * the job's fence sequence number this function should be
> - * called with drm_sched_job_init under common lock.
> + * Note: To guarantee that the order of insertion to queue matches the job's
> + * fence sequence number this function should be called with drm_sched_job_arm()
> + * under common lock.
> *
> * Returns 0 for success, negative error code otherwise.
> */
> diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
> index 69de2c76731f..bcea035cf4c6 100644
> --- a/drivers/gpu/drm/scheduler/sched_fence.c
> +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> @@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
> *
> * Free up the fence memory after the RCU grace period.
> */
> -static void drm_sched_fence_free(struct rcu_head *rcu)
> +void drm_sched_fence_free(struct rcu_head *rcu)
> {
> struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
> struct drm_sched_fence *fence = to_drm_sched_fence(f);
> @@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> }
> EXPORT_SYMBOL(to_drm_sched_fence);
>
> -struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
> - void *owner)
> +struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
> + void *owner)
> {
> struct drm_sched_fence *fence = NULL;
> - unsigned seq;
>
> fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
> if (fence == NULL)
> return NULL;
>
> fence->owner = owner;
> - fence->sched = entity->rq->sched;
> spin_lock_init(&fence->lock);
>
> + return fence;
> +}
> +
> +void drm_sched_fence_init(struct drm_sched_fence *fence,
> + struct drm_sched_entity *entity)
> +{
> + unsigned seq;
> +
> + fence->sched = entity->rq->sched;
> seq = atomic_inc_return(&entity->fence_seq);
> dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
> &fence->lock, entity->fence_context, seq);
> dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
> &fence->lock, entity->fence_context + 1, seq);
> -
> - return fence;
> }
>
> module_init(drm_sched_fence_slab_init);
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> index 33c414d55fab..454cb6164bdc 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -48,9 +48,11 @@
> #include <linux/wait.h>
> #include <linux/sched.h>
> #include <linux/completion.h>
> +#include <linux/dma-resv.h>
> #include <uapi/linux/sched/types.h>
>
> #include <drm/drm_print.h>
> +#include <drm/drm_gem.h>
> #include <drm/gpu_scheduler.h>
> #include <drm/spsc_queue.h>
>
> @@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
>
> /**
> * drm_sched_job_init - init a scheduler job
> - *
> * @job: scheduler job to init
> * @entity: scheduler entity to use
> * @owner: job owner for debugging
> @@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
> * Refer to drm_sched_entity_push_job() documentation
> * for locking considerations.
> *
> + * Drivers must make sure drm_sched_job_cleanup() if this function returns
> + * successfully, even when @job is aborted before drm_sched_job_arm() is called.
> + *
> + * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
> + * has died, which can mean that there's no valid runqueue for a @entity.
> + * This function returns -ENOENT in this case (which probably should be -EIO as
> + * a more meanigful return value).
> + *
> * Returns 0 for success, negative error code otherwise.
> */
> int drm_sched_job_init(struct drm_sched_job *job,
> struct drm_sched_entity *entity,
> void *owner)
> {
> - struct drm_gpu_scheduler *sched;
> -
> drm_sched_entity_select_rq(entity);
> if (!entity->rq)
> return -ENOENT;
>
> - sched = entity->rq->sched;
> -
> - job->sched = sched;
> job->entity = entity;
> - job->s_priority = entity->rq - sched->sched_rq;
> - job->s_fence = drm_sched_fence_create(entity, owner);
> + job->s_fence = drm_sched_fence_alloc(entity, owner);
> if (!job->s_fence)
> return -ENOMEM;
> - job->id = atomic64_inc_return(&sched->job_id_count);
>
> INIT_LIST_HEAD(&job->list);
>
> @@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
> EXPORT_SYMBOL(drm_sched_job_init);
>
> /**
> - * drm_sched_job_cleanup - clean up scheduler job resources
> + * drm_sched_job_arm - arm a scheduler job for execution
> + * @job: scheduler job to arm
> + *
> + * This arms a scheduler job for execution. Specifically it initializes the
> + * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
> + * or other places that need to track the completion of this job.
> + *
> + * Refer to drm_sched_entity_push_job() documentation for locking
> + * considerations.
> *
> + * This can only be called if drm_sched_job_init() succeeded.
> + */
> +void drm_sched_job_arm(struct drm_sched_job *job)
> +{
> + struct drm_gpu_scheduler *sched;
> + struct drm_sched_entity *entity = job->entity;
> +
> + BUG_ON(!entity);
> +
> + sched = entity->rq->sched;
> +
> + job->sched = sched;
> + job->s_priority = entity->rq - sched->sched_rq;
> + job->id = atomic64_inc_return(&sched->job_id_count);
> +
> + drm_sched_fence_init(job->s_fence, job->entity);
> +}
> +EXPORT_SYMBOL(drm_sched_job_arm);
> +
> +/**
> + * drm_sched_job_cleanup - clean up scheduler job resources
> * @job: scheduler job to clean up
> + *
> + * Cleans up the resources allocated with drm_sched_job_init().
> + *
> + * Drivers should call this from their error unwind code if @job is aborted
> + * before drm_sched_job_arm() is called.
> + *
> + * After that point of no return @job is committed to be executed by the
> + * scheduler, and this function should be called from the
> + * &drm_sched_backend_ops.free_job callback.
> */
> void drm_sched_job_cleanup(struct drm_sched_job *job)
> {
> - dma_fence_put(&job->s_fence->finished);
> + if (kref_read(&job->s_fence->finished.refcount)) {
> + /* drm_sched_job_arm() has been called */
> + dma_fence_put(&job->s_fence->finished);
> + } else {
> + /* aborted job before committing to run it */
> + drm_sched_fence_free(&job->s_fence->finished.rcu);
> + }
> +
> job->s_fence = NULL;
> }
> EXPORT_SYMBOL(drm_sched_job_cleanup);
> diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
> index 5689da118197..2e808097b4d1 100644
> --- a/drivers/gpu/drm/v3d/v3d_gem.c
> +++ b/drivers/gpu/drm/v3d/v3d_gem.c
> @@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
> if (ret)
> return ret;
>
> + drm_sched_job_arm(&job->base);
> +
> job->done_fence = dma_fence_get(&job->base.s_fence->finished);
>
> /* put by scheduler job completion */
> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> index 88ae7f331bb1..83afc3aa8e2f 100644
> --- a/include/drm/gpu_scheduler.h
> +++ b/include/drm/gpu_scheduler.h
> @@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
> int drm_sched_job_init(struct drm_sched_job *job,
> struct drm_sched_entity *entity,
> void *owner);
> +void drm_sched_job_arm(struct drm_sched_job *job);
> void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
> struct drm_gpu_scheduler **sched_list,
> unsigned int num_sched_list);
> @@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> enum drm_sched_priority priority);
> bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
>
> -struct drm_sched_fence *drm_sched_fence_create(
> +struct drm_sched_fence *drm_sched_fence_alloc(
> struct drm_sched_entity *s_entity, void *owner);
> +void drm_sched_fence_init(struct drm_sched_fence *fence,
> + struct drm_sched_entity *entity);
> +void drm_sched_fence_free(struct rcu_head *rcu);
> +
> void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
> void drm_sched_fence_finished(struct drm_sched_fence *fence);
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 01/20] drm/sched: Split drm_sched_job_init
@ 2021-08-05 13:43 ` Christian König
0 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:43 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Melissa Wen, Melissa Wen,
Emma Anholt, Steven Price, Boris Brezillon, Daniel Vetter,
Lucas Stach, Russell King, Christian Gmeiner, Qiang Yu,
Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig, David Airlie,
Daniel Vetter, Sumit Semwal, Masahiro Yamada, Kees Cook,
Adam Borowski, Nick Terrell, Mauro Carvalho Chehab, Paul Menzel,
Sami Tolvanen, Viresh Kumar, Alex Deucher, Dave Airlie,
Nirmoy Das, Deepak R Varma, Lee Jones, Kevin Wang, Chen Li,
Luben Tuikov, Marek Olšák, Dennis Li,
Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang, Tian Tao,
etnaviv, lima, linux-media, linaro-mm-sig, Rob Clark, Sean Paul,
linux-arm-msm, freedreno
Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> This is a very confusingly named function, because not just does it
> init an object, it arms it and provides a point of no return for
> pushing a job into the scheduler. It would be nice if that's a bit
> clearer in the interface.
>
> But the real reason is that I want to push the dependency tracking
> helpers into the scheduler code, and that means drm_sched_job_init
> must be called a lot earlier, without arming the job.
>
> v2:
> - don't change .gitignore (Steven)
> - don't forget v3d (Emma)
>
> v3: Emma noticed that I leak the memory allocated in
> drm_sched_job_init if we bail out before the point of no return in
> subsequent driver patches. To be able to fix this change
> drm_sched_job_cleanup() so it can handle being called both before and
> after drm_sched_job_arm().
>
> Also improve the kerneldoc for this.
>
> v4:
> - Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
> usual (Melissa)
>
> - Christian pointed out that drm_sched_entity_select_rq() also needs
> to be moved into drm_sched_job_arm, which made me realize that the
> job->id definitely needs to be moved too.
>
> Shuffle things to fit between job_init and job_arm.
>
> v5:
> Reshuffle the split between init/arm once more, amdgpu abuses
> drm_sched.ready to signal gpu reset failures. Also document this
> somewhat. (Christian)
>
> v6:
> Rebase on top of the msm drm/sched support. Note that the
> drm_sched_job_init() call is completely misplaced, and hence also the
> split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
> patch will address.
>
> Acked-by: Melissa Wen <mwen@igalia.com>
> Cc: Melissa Wen <melissa.srw@gmail.com>
> Acked-by: Emma Anholt <emma@anholt.net>
> Acked-by: Steven Price <steven.price@arm.com> (v2)
> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
At least the amdgpu parts look ok of hand, but I can't judge the rest I
think.
So only Acked-by: Christian König <christian.koenig@amd.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: David Airlie <airlied@linux.ie>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Cc: Kees Cook <keescook@chromium.org>
> Cc: Adam Borowski <kilobyte@angband.pl>
> Cc: Nick Terrell <terrelln@fb.com>
> Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> Cc: Paul Menzel <pmenzel@molgen.mpg.de>
> Cc: Sami Tolvanen <samitolvanen@google.com>
> Cc: Viresh Kumar <viresh.kumar@linaro.org>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Dave Airlie <airlied@redhat.com>
> Cc: Nirmoy Das <nirmoy.das@amd.com>
> Cc: Deepak R Varma <mh12gx2825@gmail.com>
> Cc: Lee Jones <lee.jones@linaro.org>
> Cc: Kevin Wang <kevin1.wang@amd.com>
> Cc: Chen Li <chenli@uniontech.com>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: "Marek Olšák" <marek.olsak@amd.com>
> Cc: Dennis Li <Dennis.Li@amd.com>
> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Sonny Jiang <sonny.jiang@amd.com>
> Cc: Boris Brezillon <boris.brezillon@collabora.com>
> Cc: Tian Tao <tiantao6@hisilicon.com>
> Cc: etnaviv@lists.freedesktop.org
> Cc: lima@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> Cc: Emma Anholt <emma@anholt.net>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
> drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
> drivers/gpu/drm/lima/lima_sched.c | 2 +
> drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++
> drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
> drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
> drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
> drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
> drivers/gpu/drm/v3d/v3d_gem.c | 2 +
> include/drm/gpu_scheduler.h | 7 ++-
> 11 files changed, 94 insertions(+), 22 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> index 139cd3bf1ad6..32e80bc6af22 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> @@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
> if (r)
> goto error_unlock;
>
> + drm_sched_job_arm(&job->base);
> +
> /* No memory allocation is allowed while holding the notifier lock.
> * The lock is held until amdgpu_cs_submit is finished and fence is
> * added to BOs.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> index d33e6d97cc89..5ddb955d2315 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> @@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
> if (r)
> return r;
>
> + drm_sched_job_arm(&job->base);
> +
> *f = dma_fence_get(&job->base.s_fence->finished);
> amdgpu_job_free_resources(job);
> drm_sched_entity_push_job(&job->base, entity);
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> index feb6da1b6ceb..05f412204118 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> @@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> if (ret)
> goto out_unlock;
>
> + drm_sched_job_arm(&submit->sched_job);
> +
> submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
> submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
> submit->out_fence, 0,
> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> index dba8329937a3..38f755580507 100644
> --- a/drivers/gpu/drm/lima/lima_sched.c
> +++ b/drivers/gpu/drm/lima/lima_sched.c
> @@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
> return err;
> }
>
> + drm_sched_job_arm(&task->base);
> +
> task->num_bos = num_bos;
> task->vm = lima_vm_get(vm);
>
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index fdc5367aecaa..6d6c44f0e1f3 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,6 +52,9 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> + /* FIXME: this is way too early */
> + drm_sched_job_arm(&job->base);
> +
> xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
>
> kref_init(&submit->ref);
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> index 71a72fb50e6b..2992dc85325f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> @@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
> goto unlock;
> }
>
> + drm_sched_job_arm(&job->base);
> +
> job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
>
> ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> index 79554aa4dbb1..f7347c284886 100644
> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> @@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
> * @sched_job: job to submit
> * @entity: scheduler entity
> *
> - * Note: To guarantee that the order of insertion to queue matches
> - * the job's fence sequence number this function should be
> - * called with drm_sched_job_init under common lock.
> + * Note: To guarantee that the order of insertion to queue matches the job's
> + * fence sequence number this function should be called with drm_sched_job_arm()
> + * under common lock.
> *
> * Returns 0 for success, negative error code otherwise.
> */
> diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
> index 69de2c76731f..bcea035cf4c6 100644
> --- a/drivers/gpu/drm/scheduler/sched_fence.c
> +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> @@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
> *
> * Free up the fence memory after the RCU grace period.
> */
> -static void drm_sched_fence_free(struct rcu_head *rcu)
> +void drm_sched_fence_free(struct rcu_head *rcu)
> {
> struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
> struct drm_sched_fence *fence = to_drm_sched_fence(f);
> @@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> }
> EXPORT_SYMBOL(to_drm_sched_fence);
>
> -struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
> - void *owner)
> +struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
> + void *owner)
> {
> struct drm_sched_fence *fence = NULL;
> - unsigned seq;
>
> fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
> if (fence == NULL)
> return NULL;
>
> fence->owner = owner;
> - fence->sched = entity->rq->sched;
> spin_lock_init(&fence->lock);
>
> + return fence;
> +}
> +
> +void drm_sched_fence_init(struct drm_sched_fence *fence,
> + struct drm_sched_entity *entity)
> +{
> + unsigned seq;
> +
> + fence->sched = entity->rq->sched;
> seq = atomic_inc_return(&entity->fence_seq);
> dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
> &fence->lock, entity->fence_context, seq);
> dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
> &fence->lock, entity->fence_context + 1, seq);
> -
> - return fence;
> }
>
> module_init(drm_sched_fence_slab_init);
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> index 33c414d55fab..454cb6164bdc 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -48,9 +48,11 @@
> #include <linux/wait.h>
> #include <linux/sched.h>
> #include <linux/completion.h>
> +#include <linux/dma-resv.h>
> #include <uapi/linux/sched/types.h>
>
> #include <drm/drm_print.h>
> +#include <drm/drm_gem.h>
> #include <drm/gpu_scheduler.h>
> #include <drm/spsc_queue.h>
>
> @@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
>
> /**
> * drm_sched_job_init - init a scheduler job
> - *
> * @job: scheduler job to init
> * @entity: scheduler entity to use
> * @owner: job owner for debugging
> @@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
> * Refer to drm_sched_entity_push_job() documentation
> * for locking considerations.
> *
> + * Drivers must make sure drm_sched_job_cleanup() if this function returns
> + * successfully, even when @job is aborted before drm_sched_job_arm() is called.
> + *
> + * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
> + * has died, which can mean that there's no valid runqueue for a @entity.
> + * This function returns -ENOENT in this case (which probably should be -EIO as
> + * a more meanigful return value).
> + *
> * Returns 0 for success, negative error code otherwise.
> */
> int drm_sched_job_init(struct drm_sched_job *job,
> struct drm_sched_entity *entity,
> void *owner)
> {
> - struct drm_gpu_scheduler *sched;
> -
> drm_sched_entity_select_rq(entity);
> if (!entity->rq)
> return -ENOENT;
>
> - sched = entity->rq->sched;
> -
> - job->sched = sched;
> job->entity = entity;
> - job->s_priority = entity->rq - sched->sched_rq;
> - job->s_fence = drm_sched_fence_create(entity, owner);
> + job->s_fence = drm_sched_fence_alloc(entity, owner);
> if (!job->s_fence)
> return -ENOMEM;
> - job->id = atomic64_inc_return(&sched->job_id_count);
>
> INIT_LIST_HEAD(&job->list);
>
> @@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
> EXPORT_SYMBOL(drm_sched_job_init);
>
> /**
> - * drm_sched_job_cleanup - clean up scheduler job resources
> + * drm_sched_job_arm - arm a scheduler job for execution
> + * @job: scheduler job to arm
> + *
> + * This arms a scheduler job for execution. Specifically it initializes the
> + * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
> + * or other places that need to track the completion of this job.
> + *
> + * Refer to drm_sched_entity_push_job() documentation for locking
> + * considerations.
> *
> + * This can only be called if drm_sched_job_init() succeeded.
> + */
> +void drm_sched_job_arm(struct drm_sched_job *job)
> +{
> + struct drm_gpu_scheduler *sched;
> + struct drm_sched_entity *entity = job->entity;
> +
> + BUG_ON(!entity);
> +
> + sched = entity->rq->sched;
> +
> + job->sched = sched;
> + job->s_priority = entity->rq - sched->sched_rq;
> + job->id = atomic64_inc_return(&sched->job_id_count);
> +
> + drm_sched_fence_init(job->s_fence, job->entity);
> +}
> +EXPORT_SYMBOL(drm_sched_job_arm);
> +
> +/**
> + * drm_sched_job_cleanup - clean up scheduler job resources
> * @job: scheduler job to clean up
> + *
> + * Cleans up the resources allocated with drm_sched_job_init().
> + *
> + * Drivers should call this from their error unwind code if @job is aborted
> + * before drm_sched_job_arm() is called.
> + *
> + * After that point of no return @job is committed to be executed by the
> + * scheduler, and this function should be called from the
> + * &drm_sched_backend_ops.free_job callback.
> */
> void drm_sched_job_cleanup(struct drm_sched_job *job)
> {
> - dma_fence_put(&job->s_fence->finished);
> + if (kref_read(&job->s_fence->finished.refcount)) {
> + /* drm_sched_job_arm() has been called */
> + dma_fence_put(&job->s_fence->finished);
> + } else {
> + /* aborted job before committing to run it */
> + drm_sched_fence_free(&job->s_fence->finished.rcu);
> + }
> +
> job->s_fence = NULL;
> }
> EXPORT_SYMBOL(drm_sched_job_cleanup);
> diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
> index 5689da118197..2e808097b4d1 100644
> --- a/drivers/gpu/drm/v3d/v3d_gem.c
> +++ b/drivers/gpu/drm/v3d/v3d_gem.c
> @@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
> if (ret)
> return ret;
>
> + drm_sched_job_arm(&job->base);
> +
> job->done_fence = dma_fence_get(&job->base.s_fence->finished);
>
> /* put by scheduler job completion */
> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> index 88ae7f331bb1..83afc3aa8e2f 100644
> --- a/include/drm/gpu_scheduler.h
> +++ b/include/drm/gpu_scheduler.h
> @@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
> int drm_sched_job_init(struct drm_sched_job *job,
> struct drm_sched_entity *entity,
> void *owner);
> +void drm_sched_job_arm(struct drm_sched_job *job);
> void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
> struct drm_gpu_scheduler **sched_list,
> unsigned int num_sched_list);
> @@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> enum drm_sched_priority priority);
> bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
>
> -struct drm_sched_fence *drm_sched_fence_create(
> +struct drm_sched_fence *drm_sched_fence_alloc(
> struct drm_sched_entity *s_entity, void *owner);
> +void drm_sched_fence_init(struct drm_sched_fence *fence,
> + struct drm_sched_entity *entity);
> +void drm_sched_fence_free(struct rcu_head *rcu);
> +
> void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
> void drm_sched_fence_finished(struct drm_sched_fence *fence);
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 01/20] drm/sched: Split drm_sched_job_init
2021-08-05 13:43 ` [Intel-gfx] " Christian König
@ 2021-08-05 14:07 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 14:07 UTC (permalink / raw)
To: Christian König
Cc: DRI Development, Intel Graphics Development, Melissa Wen,
Melissa Wen, Emma Anholt, Steven Price, Boris Brezillon,
Daniel Vetter, Lucas Stach, Russell King, Christian Gmeiner,
Qiang Yu, Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig,
David Airlie, Sumit Semwal, Masahiro Yamada, Kees Cook,
Adam Borowski, Nick Terrell, Mauro Carvalho Chehab, Paul Menzel,
Sami Tolvanen, Viresh Kumar, Alex Deucher, Dave Airlie,
Nirmoy Das, Deepak R Varma, Lee Jones, Kevin Wang, Chen Li,
Luben Tuikov, Marek Olšák, Dennis Li,
Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang, Tian Tao,
The etnaviv authors, lima,
open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
On Thu, Aug 5, 2021 at 3:44 PM Christian König <christian.koenig@amd.com> wrote:
> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> > This is a very confusingly named function, because not just does it
> > init an object, it arms it and provides a point of no return for
> > pushing a job into the scheduler. It would be nice if that's a bit
> > clearer in the interface.
> >
> > But the real reason is that I want to push the dependency tracking
> > helpers into the scheduler code, and that means drm_sched_job_init
> > must be called a lot earlier, without arming the job.
> >
> > v2:
> > - don't change .gitignore (Steven)
> > - don't forget v3d (Emma)
> >
> > v3: Emma noticed that I leak the memory allocated in
> > drm_sched_job_init if we bail out before the point of no return in
> > subsequent driver patches. To be able to fix this change
> > drm_sched_job_cleanup() so it can handle being called both before and
> > after drm_sched_job_arm().
> >
> > Also improve the kerneldoc for this.
> >
> > v4:
> > - Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
> > usual (Melissa)
> >
> > - Christian pointed out that drm_sched_entity_select_rq() also needs
> > to be moved into drm_sched_job_arm, which made me realize that the
> > job->id definitely needs to be moved too.
> >
> > Shuffle things to fit between job_init and job_arm.
> >
> > v5:
> > Reshuffle the split between init/arm once more, amdgpu abuses
> > drm_sched.ready to signal gpu reset failures. Also document this
> > somewhat. (Christian)
> >
> > v6:
> > Rebase on top of the msm drm/sched support. Note that the
> > drm_sched_job_init() call is completely misplaced, and hence also the
> > split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
> > patch will address.
> >
> > Acked-by: Melissa Wen <mwen@igalia.com>
> > Cc: Melissa Wen <melissa.srw@gmail.com>
> > Acked-by: Emma Anholt <emma@anholt.net>
> > Acked-by: Steven Price <steven.price@arm.com> (v2)
> > Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
>
> At least the amdgpu parts look ok of hand, but I can't judge the rest I
> think.
The thing that really scares me here and that I got wrong a few times
is the cleanup for drm_sched_job at the various points. Can you give
those parts in drm/scheduler/ a full review pls, just to make sure? I
can note that in the tag ofc, just like a bit more confidence here
that it's not busted :-)
> So only Acked-by: Christian König <christian.koenig@amd.com>
Thanks, Daniel
>
> > Cc: Lucas Stach <l.stach@pengutronix.de>
> > Cc: Russell King <linux+etnaviv@armlinux.org.uk>
> > Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> > Cc: Qiang Yu <yuq825@gmail.com>
> > Cc: Rob Herring <robh@kernel.org>
> > Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > Cc: Steven Price <steven.price@arm.com>
> > Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> > Cc: David Airlie <airlied@linux.ie>
> > Cc: Daniel Vetter <daniel@ffwll.ch>
> > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: Masahiro Yamada <masahiroy@kernel.org>
> > Cc: Kees Cook <keescook@chromium.org>
> > Cc: Adam Borowski <kilobyte@angband.pl>
> > Cc: Nick Terrell <terrelln@fb.com>
> > Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> > Cc: Paul Menzel <pmenzel@molgen.mpg.de>
> > Cc: Sami Tolvanen <samitolvanen@google.com>
> > Cc: Viresh Kumar <viresh.kumar@linaro.org>
> > Cc: Alex Deucher <alexander.deucher@amd.com>
> > Cc: Dave Airlie <airlied@redhat.com>
> > Cc: Nirmoy Das <nirmoy.das@amd.com>
> > Cc: Deepak R Varma <mh12gx2825@gmail.com>
> > Cc: Lee Jones <lee.jones@linaro.org>
> > Cc: Kevin Wang <kevin1.wang@amd.com>
> > Cc: Chen Li <chenli@uniontech.com>
> > Cc: Luben Tuikov <luben.tuikov@amd.com>
> > Cc: "Marek Olšák" <marek.olsak@amd.com>
> > Cc: Dennis Li <Dennis.Li@amd.com>
> > Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> > Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > Cc: Sonny Jiang <sonny.jiang@amd.com>
> > Cc: Boris Brezillon <boris.brezillon@collabora.com>
> > Cc: Tian Tao <tiantao6@hisilicon.com>
> > Cc: etnaviv@lists.freedesktop.org
> > Cc: lima@lists.freedesktop.org
> > Cc: linux-media@vger.kernel.org
> > Cc: linaro-mm-sig@lists.linaro.org
> > Cc: Emma Anholt <emma@anholt.net>
> > Cc: Rob Clark <robdclark@gmail.com>
> > Cc: Sean Paul <sean@poorly.run>
> > Cc: linux-arm-msm@vger.kernel.org
> > Cc: freedreno@lists.freedesktop.org
> > ---
> > drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
> > drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
> > drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
> > drivers/gpu/drm/lima/lima_sched.c | 2 +
> > drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++
> > drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
> > drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
> > drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
> > drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
> > drivers/gpu/drm/v3d/v3d_gem.c | 2 +
> > include/drm/gpu_scheduler.h | 7 ++-
> > 11 files changed, 94 insertions(+), 22 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> > index 139cd3bf1ad6..32e80bc6af22 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> > @@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
> > if (r)
> > goto error_unlock;
> >
> > + drm_sched_job_arm(&job->base);
> > +
> > /* No memory allocation is allowed while holding the notifier lock.
> > * The lock is held until amdgpu_cs_submit is finished and fence is
> > * added to BOs.
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> > index d33e6d97cc89..5ddb955d2315 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> > @@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
> > if (r)
> > return r;
> >
> > + drm_sched_job_arm(&job->base);
> > +
> > *f = dma_fence_get(&job->base.s_fence->finished);
> > amdgpu_job_free_resources(job);
> > drm_sched_entity_push_job(&job->base, entity);
> > diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> > index feb6da1b6ceb..05f412204118 100644
> > --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> > +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> > @@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> > if (ret)
> > goto out_unlock;
> >
> > + drm_sched_job_arm(&submit->sched_job);
> > +
> > submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
> > submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
> > submit->out_fence, 0,
> > diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> > index dba8329937a3..38f755580507 100644
> > --- a/drivers/gpu/drm/lima/lima_sched.c
> > +++ b/drivers/gpu/drm/lima/lima_sched.c
> > @@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
> > return err;
> > }
> >
> > + drm_sched_job_arm(&task->base);
> > +
> > task->num_bos = num_bos;
> > task->vm = lima_vm_get(vm);
> >
> > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > index fdc5367aecaa..6d6c44f0e1f3 100644
> > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > @@ -52,6 +52,9 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > return ERR_PTR(ret);
> > }
> >
> > + /* FIXME: this is way too early */
> > + drm_sched_job_arm(&job->base);
> > +
> > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> >
> > kref_init(&submit->ref);
> > diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> > index 71a72fb50e6b..2992dc85325f 100644
> > --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> > +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> > @@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
> > goto unlock;
> > }
> >
> > + drm_sched_job_arm(&job->base);
> > +
> > job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
> >
> > ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
> > diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> > index 79554aa4dbb1..f7347c284886 100644
> > --- a/drivers/gpu/drm/scheduler/sched_entity.c
> > +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> > @@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
> > * @sched_job: job to submit
> > * @entity: scheduler entity
> > *
> > - * Note: To guarantee that the order of insertion to queue matches
> > - * the job's fence sequence number this function should be
> > - * called with drm_sched_job_init under common lock.
> > + * Note: To guarantee that the order of insertion to queue matches the job's
> > + * fence sequence number this function should be called with drm_sched_job_arm()
> > + * under common lock.
> > *
> > * Returns 0 for success, negative error code otherwise.
> > */
> > diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
> > index 69de2c76731f..bcea035cf4c6 100644
> > --- a/drivers/gpu/drm/scheduler/sched_fence.c
> > +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> > @@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
> > *
> > * Free up the fence memory after the RCU grace period.
> > */
> > -static void drm_sched_fence_free(struct rcu_head *rcu)
> > +void drm_sched_fence_free(struct rcu_head *rcu)
> > {
> > struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
> > struct drm_sched_fence *fence = to_drm_sched_fence(f);
> > @@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> > }
> > EXPORT_SYMBOL(to_drm_sched_fence);
> >
> > -struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
> > - void *owner)
> > +struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
> > + void *owner)
> > {
> > struct drm_sched_fence *fence = NULL;
> > - unsigned seq;
> >
> > fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
> > if (fence == NULL)
> > return NULL;
> >
> > fence->owner = owner;
> > - fence->sched = entity->rq->sched;
> > spin_lock_init(&fence->lock);
> >
> > + return fence;
> > +}
> > +
> > +void drm_sched_fence_init(struct drm_sched_fence *fence,
> > + struct drm_sched_entity *entity)
> > +{
> > + unsigned seq;
> > +
> > + fence->sched = entity->rq->sched;
> > seq = atomic_inc_return(&entity->fence_seq);
> > dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
> > &fence->lock, entity->fence_context, seq);
> > dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
> > &fence->lock, entity->fence_context + 1, seq);
> > -
> > - return fence;
> > }
> >
> > module_init(drm_sched_fence_slab_init);
> > diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> > index 33c414d55fab..454cb6164bdc 100644
> > --- a/drivers/gpu/drm/scheduler/sched_main.c
> > +++ b/drivers/gpu/drm/scheduler/sched_main.c
> > @@ -48,9 +48,11 @@
> > #include <linux/wait.h>
> > #include <linux/sched.h>
> > #include <linux/completion.h>
> > +#include <linux/dma-resv.h>
> > #include <uapi/linux/sched/types.h>
> >
> > #include <drm/drm_print.h>
> > +#include <drm/drm_gem.h>
> > #include <drm/gpu_scheduler.h>
> > #include <drm/spsc_queue.h>
> >
> > @@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
> >
> > /**
> > * drm_sched_job_init - init a scheduler job
> > - *
> > * @job: scheduler job to init
> > * @entity: scheduler entity to use
> > * @owner: job owner for debugging
> > @@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
> > * Refer to drm_sched_entity_push_job() documentation
> > * for locking considerations.
> > *
> > + * Drivers must make sure drm_sched_job_cleanup() if this function returns
> > + * successfully, even when @job is aborted before drm_sched_job_arm() is called.
> > + *
> > + * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
> > + * has died, which can mean that there's no valid runqueue for a @entity.
> > + * This function returns -ENOENT in this case (which probably should be -EIO as
> > + * a more meanigful return value).
> > + *
> > * Returns 0 for success, negative error code otherwise.
> > */
> > int drm_sched_job_init(struct drm_sched_job *job,
> > struct drm_sched_entity *entity,
> > void *owner)
> > {
> > - struct drm_gpu_scheduler *sched;
> > -
> > drm_sched_entity_select_rq(entity);
> > if (!entity->rq)
> > return -ENOENT;
> >
> > - sched = entity->rq->sched;
> > -
> > - job->sched = sched;
> > job->entity = entity;
> > - job->s_priority = entity->rq - sched->sched_rq;
> > - job->s_fence = drm_sched_fence_create(entity, owner);
> > + job->s_fence = drm_sched_fence_alloc(entity, owner);
> > if (!job->s_fence)
> > return -ENOMEM;
> > - job->id = atomic64_inc_return(&sched->job_id_count);
> >
> > INIT_LIST_HEAD(&job->list);
> >
> > @@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
> > EXPORT_SYMBOL(drm_sched_job_init);
> >
> > /**
> > - * drm_sched_job_cleanup - clean up scheduler job resources
> > + * drm_sched_job_arm - arm a scheduler job for execution
> > + * @job: scheduler job to arm
> > + *
> > + * This arms a scheduler job for execution. Specifically it initializes the
> > + * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
> > + * or other places that need to track the completion of this job.
> > + *
> > + * Refer to drm_sched_entity_push_job() documentation for locking
> > + * considerations.
> > *
> > + * This can only be called if drm_sched_job_init() succeeded.
> > + */
> > +void drm_sched_job_arm(struct drm_sched_job *job)
> > +{
> > + struct drm_gpu_scheduler *sched;
> > + struct drm_sched_entity *entity = job->entity;
> > +
> > + BUG_ON(!entity);
> > +
> > + sched = entity->rq->sched;
> > +
> > + job->sched = sched;
> > + job->s_priority = entity->rq - sched->sched_rq;
> > + job->id = atomic64_inc_return(&sched->job_id_count);
> > +
> > + drm_sched_fence_init(job->s_fence, job->entity);
> > +}
> > +EXPORT_SYMBOL(drm_sched_job_arm);
> > +
> > +/**
> > + * drm_sched_job_cleanup - clean up scheduler job resources
> > * @job: scheduler job to clean up
> > + *
> > + * Cleans up the resources allocated with drm_sched_job_init().
> > + *
> > + * Drivers should call this from their error unwind code if @job is aborted
> > + * before drm_sched_job_arm() is called.
> > + *
> > + * After that point of no return @job is committed to be executed by the
> > + * scheduler, and this function should be called from the
> > + * &drm_sched_backend_ops.free_job callback.
> > */
> > void drm_sched_job_cleanup(struct drm_sched_job *job)
> > {
> > - dma_fence_put(&job->s_fence->finished);
> > + if (kref_read(&job->s_fence->finished.refcount)) {
> > + /* drm_sched_job_arm() has been called */
> > + dma_fence_put(&job->s_fence->finished);
> > + } else {
> > + /* aborted job before committing to run it */
> > + drm_sched_fence_free(&job->s_fence->finished.rcu);
> > + }
> > +
> > job->s_fence = NULL;
> > }
> > EXPORT_SYMBOL(drm_sched_job_cleanup);
> > diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
> > index 5689da118197..2e808097b4d1 100644
> > --- a/drivers/gpu/drm/v3d/v3d_gem.c
> > +++ b/drivers/gpu/drm/v3d/v3d_gem.c
> > @@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
> > if (ret)
> > return ret;
> >
> > + drm_sched_job_arm(&job->base);
> > +
> > job->done_fence = dma_fence_get(&job->base.s_fence->finished);
> >
> > /* put by scheduler job completion */
> > diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> > index 88ae7f331bb1..83afc3aa8e2f 100644
> > --- a/include/drm/gpu_scheduler.h
> > +++ b/include/drm/gpu_scheduler.h
> > @@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
> > int drm_sched_job_init(struct drm_sched_job *job,
> > struct drm_sched_entity *entity,
> > void *owner);
> > +void drm_sched_job_arm(struct drm_sched_job *job);
> > void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
> > struct drm_gpu_scheduler **sched_list,
> > unsigned int num_sched_list);
> > @@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> > enum drm_sched_priority priority);
> > bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
> >
> > -struct drm_sched_fence *drm_sched_fence_create(
> > +struct drm_sched_fence *drm_sched_fence_alloc(
> > struct drm_sched_entity *s_entity, void *owner);
> > +void drm_sched_fence_init(struct drm_sched_fence *fence,
> > + struct drm_sched_entity *entity);
> > +void drm_sched_fence_free(struct rcu_head *rcu);
> > +
> > void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
> > void drm_sched_fence_finished(struct drm_sched_fence *fence);
> >
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 01/20] drm/sched: Split drm_sched_job_init
@ 2021-08-05 14:07 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 14:07 UTC (permalink / raw)
To: Christian König
Cc: DRI Development, Intel Graphics Development, Melissa Wen,
Melissa Wen, Emma Anholt, Steven Price, Boris Brezillon,
Daniel Vetter, Lucas Stach, Russell King, Christian Gmeiner,
Qiang Yu, Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig,
David Airlie, Sumit Semwal, Masahiro Yamada, Kees Cook,
Adam Borowski, Nick Terrell, Mauro Carvalho Chehab, Paul Menzel,
Sami Tolvanen, Viresh Kumar, Alex Deucher, Dave Airlie,
Nirmoy Das, Deepak R Varma, Lee Jones, Kevin Wang, Chen Li,
Luben Tuikov, Marek Olšák, Dennis Li,
Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang, Tian Tao,
The etnaviv authors, lima,
open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
On Thu, Aug 5, 2021 at 3:44 PM Christian König <christian.koenig@amd.com> wrote:
> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> > This is a very confusingly named function, because not just does it
> > init an object, it arms it and provides a point of no return for
> > pushing a job into the scheduler. It would be nice if that's a bit
> > clearer in the interface.
> >
> > But the real reason is that I want to push the dependency tracking
> > helpers into the scheduler code, and that means drm_sched_job_init
> > must be called a lot earlier, without arming the job.
> >
> > v2:
> > - don't change .gitignore (Steven)
> > - don't forget v3d (Emma)
> >
> > v3: Emma noticed that I leak the memory allocated in
> > drm_sched_job_init if we bail out before the point of no return in
> > subsequent driver patches. To be able to fix this change
> > drm_sched_job_cleanup() so it can handle being called both before and
> > after drm_sched_job_arm().
> >
> > Also improve the kerneldoc for this.
> >
> > v4:
> > - Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
> > usual (Melissa)
> >
> > - Christian pointed out that drm_sched_entity_select_rq() also needs
> > to be moved into drm_sched_job_arm, which made me realize that the
> > job->id definitely needs to be moved too.
> >
> > Shuffle things to fit between job_init and job_arm.
> >
> > v5:
> > Reshuffle the split between init/arm once more, amdgpu abuses
> > drm_sched.ready to signal gpu reset failures. Also document this
> > somewhat. (Christian)
> >
> > v6:
> > Rebase on top of the msm drm/sched support. Note that the
> > drm_sched_job_init() call is completely misplaced, and hence also the
> > split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
> > patch will address.
> >
> > Acked-by: Melissa Wen <mwen@igalia.com>
> > Cc: Melissa Wen <melissa.srw@gmail.com>
> > Acked-by: Emma Anholt <emma@anholt.net>
> > Acked-by: Steven Price <steven.price@arm.com> (v2)
> > Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
>
> At least the amdgpu parts look ok of hand, but I can't judge the rest I
> think.
The thing that really scares me here and that I got wrong a few times
is the cleanup for drm_sched_job at the various points. Can you give
those parts in drm/scheduler/ a full review pls, just to make sure? I
can note that in the tag ofc, just like a bit more confidence here
that it's not busted :-)
> So only Acked-by: Christian König <christian.koenig@amd.com>
Thanks, Daniel
>
> > Cc: Lucas Stach <l.stach@pengutronix.de>
> > Cc: Russell King <linux+etnaviv@armlinux.org.uk>
> > Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> > Cc: Qiang Yu <yuq825@gmail.com>
> > Cc: Rob Herring <robh@kernel.org>
> > Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > Cc: Steven Price <steven.price@arm.com>
> > Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> > Cc: David Airlie <airlied@linux.ie>
> > Cc: Daniel Vetter <daniel@ffwll.ch>
> > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: Masahiro Yamada <masahiroy@kernel.org>
> > Cc: Kees Cook <keescook@chromium.org>
> > Cc: Adam Borowski <kilobyte@angband.pl>
> > Cc: Nick Terrell <terrelln@fb.com>
> > Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> > Cc: Paul Menzel <pmenzel@molgen.mpg.de>
> > Cc: Sami Tolvanen <samitolvanen@google.com>
> > Cc: Viresh Kumar <viresh.kumar@linaro.org>
> > Cc: Alex Deucher <alexander.deucher@amd.com>
> > Cc: Dave Airlie <airlied@redhat.com>
> > Cc: Nirmoy Das <nirmoy.das@amd.com>
> > Cc: Deepak R Varma <mh12gx2825@gmail.com>
> > Cc: Lee Jones <lee.jones@linaro.org>
> > Cc: Kevin Wang <kevin1.wang@amd.com>
> > Cc: Chen Li <chenli@uniontech.com>
> > Cc: Luben Tuikov <luben.tuikov@amd.com>
> > Cc: "Marek Olšák" <marek.olsak@amd.com>
> > Cc: Dennis Li <Dennis.Li@amd.com>
> > Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> > Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > Cc: Sonny Jiang <sonny.jiang@amd.com>
> > Cc: Boris Brezillon <boris.brezillon@collabora.com>
> > Cc: Tian Tao <tiantao6@hisilicon.com>
> > Cc: etnaviv@lists.freedesktop.org
> > Cc: lima@lists.freedesktop.org
> > Cc: linux-media@vger.kernel.org
> > Cc: linaro-mm-sig@lists.linaro.org
> > Cc: Emma Anholt <emma@anholt.net>
> > Cc: Rob Clark <robdclark@gmail.com>
> > Cc: Sean Paul <sean@poorly.run>
> > Cc: linux-arm-msm@vger.kernel.org
> > Cc: freedreno@lists.freedesktop.org
> > ---
> > drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
> > drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
> > drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
> > drivers/gpu/drm/lima/lima_sched.c | 2 +
> > drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++
> > drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
> > drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
> > drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
> > drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
> > drivers/gpu/drm/v3d/v3d_gem.c | 2 +
> > include/drm/gpu_scheduler.h | 7 ++-
> > 11 files changed, 94 insertions(+), 22 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> > index 139cd3bf1ad6..32e80bc6af22 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> > @@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
> > if (r)
> > goto error_unlock;
> >
> > + drm_sched_job_arm(&job->base);
> > +
> > /* No memory allocation is allowed while holding the notifier lock.
> > * The lock is held until amdgpu_cs_submit is finished and fence is
> > * added to BOs.
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> > index d33e6d97cc89..5ddb955d2315 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> > @@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
> > if (r)
> > return r;
> >
> > + drm_sched_job_arm(&job->base);
> > +
> > *f = dma_fence_get(&job->base.s_fence->finished);
> > amdgpu_job_free_resources(job);
> > drm_sched_entity_push_job(&job->base, entity);
> > diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> > index feb6da1b6ceb..05f412204118 100644
> > --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> > +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> > @@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> > if (ret)
> > goto out_unlock;
> >
> > + drm_sched_job_arm(&submit->sched_job);
> > +
> > submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
> > submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
> > submit->out_fence, 0,
> > diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> > index dba8329937a3..38f755580507 100644
> > --- a/drivers/gpu/drm/lima/lima_sched.c
> > +++ b/drivers/gpu/drm/lima/lima_sched.c
> > @@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
> > return err;
> > }
> >
> > + drm_sched_job_arm(&task->base);
> > +
> > task->num_bos = num_bos;
> > task->vm = lima_vm_get(vm);
> >
> > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > index fdc5367aecaa..6d6c44f0e1f3 100644
> > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > @@ -52,6 +52,9 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > return ERR_PTR(ret);
> > }
> >
> > + /* FIXME: this is way too early */
> > + drm_sched_job_arm(&job->base);
> > +
> > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> >
> > kref_init(&submit->ref);
> > diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> > index 71a72fb50e6b..2992dc85325f 100644
> > --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> > +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> > @@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
> > goto unlock;
> > }
> >
> > + drm_sched_job_arm(&job->base);
> > +
> > job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
> >
> > ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
> > diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> > index 79554aa4dbb1..f7347c284886 100644
> > --- a/drivers/gpu/drm/scheduler/sched_entity.c
> > +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> > @@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
> > * @sched_job: job to submit
> > * @entity: scheduler entity
> > *
> > - * Note: To guarantee that the order of insertion to queue matches
> > - * the job's fence sequence number this function should be
> > - * called with drm_sched_job_init under common lock.
> > + * Note: To guarantee that the order of insertion to queue matches the job's
> > + * fence sequence number this function should be called with drm_sched_job_arm()
> > + * under common lock.
> > *
> > * Returns 0 for success, negative error code otherwise.
> > */
> > diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
> > index 69de2c76731f..bcea035cf4c6 100644
> > --- a/drivers/gpu/drm/scheduler/sched_fence.c
> > +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> > @@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
> > *
> > * Free up the fence memory after the RCU grace period.
> > */
> > -static void drm_sched_fence_free(struct rcu_head *rcu)
> > +void drm_sched_fence_free(struct rcu_head *rcu)
> > {
> > struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
> > struct drm_sched_fence *fence = to_drm_sched_fence(f);
> > @@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> > }
> > EXPORT_SYMBOL(to_drm_sched_fence);
> >
> > -struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
> > - void *owner)
> > +struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
> > + void *owner)
> > {
> > struct drm_sched_fence *fence = NULL;
> > - unsigned seq;
> >
> > fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
> > if (fence == NULL)
> > return NULL;
> >
> > fence->owner = owner;
> > - fence->sched = entity->rq->sched;
> > spin_lock_init(&fence->lock);
> >
> > + return fence;
> > +}
> > +
> > +void drm_sched_fence_init(struct drm_sched_fence *fence,
> > + struct drm_sched_entity *entity)
> > +{
> > + unsigned seq;
> > +
> > + fence->sched = entity->rq->sched;
> > seq = atomic_inc_return(&entity->fence_seq);
> > dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
> > &fence->lock, entity->fence_context, seq);
> > dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
> > &fence->lock, entity->fence_context + 1, seq);
> > -
> > - return fence;
> > }
> >
> > module_init(drm_sched_fence_slab_init);
> > diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> > index 33c414d55fab..454cb6164bdc 100644
> > --- a/drivers/gpu/drm/scheduler/sched_main.c
> > +++ b/drivers/gpu/drm/scheduler/sched_main.c
> > @@ -48,9 +48,11 @@
> > #include <linux/wait.h>
> > #include <linux/sched.h>
> > #include <linux/completion.h>
> > +#include <linux/dma-resv.h>
> > #include <uapi/linux/sched/types.h>
> >
> > #include <drm/drm_print.h>
> > +#include <drm/drm_gem.h>
> > #include <drm/gpu_scheduler.h>
> > #include <drm/spsc_queue.h>
> >
> > @@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
> >
> > /**
> > * drm_sched_job_init - init a scheduler job
> > - *
> > * @job: scheduler job to init
> > * @entity: scheduler entity to use
> > * @owner: job owner for debugging
> > @@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
> > * Refer to drm_sched_entity_push_job() documentation
> > * for locking considerations.
> > *
> > + * Drivers must make sure drm_sched_job_cleanup() if this function returns
> > + * successfully, even when @job is aborted before drm_sched_job_arm() is called.
> > + *
> > + * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
> > + * has died, which can mean that there's no valid runqueue for a @entity.
> > + * This function returns -ENOENT in this case (which probably should be -EIO as
> > + * a more meanigful return value).
> > + *
> > * Returns 0 for success, negative error code otherwise.
> > */
> > int drm_sched_job_init(struct drm_sched_job *job,
> > struct drm_sched_entity *entity,
> > void *owner)
> > {
> > - struct drm_gpu_scheduler *sched;
> > -
> > drm_sched_entity_select_rq(entity);
> > if (!entity->rq)
> > return -ENOENT;
> >
> > - sched = entity->rq->sched;
> > -
> > - job->sched = sched;
> > job->entity = entity;
> > - job->s_priority = entity->rq - sched->sched_rq;
> > - job->s_fence = drm_sched_fence_create(entity, owner);
> > + job->s_fence = drm_sched_fence_alloc(entity, owner);
> > if (!job->s_fence)
> > return -ENOMEM;
> > - job->id = atomic64_inc_return(&sched->job_id_count);
> >
> > INIT_LIST_HEAD(&job->list);
> >
> > @@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
> > EXPORT_SYMBOL(drm_sched_job_init);
> >
> > /**
> > - * drm_sched_job_cleanup - clean up scheduler job resources
> > + * drm_sched_job_arm - arm a scheduler job for execution
> > + * @job: scheduler job to arm
> > + *
> > + * This arms a scheduler job for execution. Specifically it initializes the
> > + * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
> > + * or other places that need to track the completion of this job.
> > + *
> > + * Refer to drm_sched_entity_push_job() documentation for locking
> > + * considerations.
> > *
> > + * This can only be called if drm_sched_job_init() succeeded.
> > + */
> > +void drm_sched_job_arm(struct drm_sched_job *job)
> > +{
> > + struct drm_gpu_scheduler *sched;
> > + struct drm_sched_entity *entity = job->entity;
> > +
> > + BUG_ON(!entity);
> > +
> > + sched = entity->rq->sched;
> > +
> > + job->sched = sched;
> > + job->s_priority = entity->rq - sched->sched_rq;
> > + job->id = atomic64_inc_return(&sched->job_id_count);
> > +
> > + drm_sched_fence_init(job->s_fence, job->entity);
> > +}
> > +EXPORT_SYMBOL(drm_sched_job_arm);
> > +
> > +/**
> > + * drm_sched_job_cleanup - clean up scheduler job resources
> > * @job: scheduler job to clean up
> > + *
> > + * Cleans up the resources allocated with drm_sched_job_init().
> > + *
> > + * Drivers should call this from their error unwind code if @job is aborted
> > + * before drm_sched_job_arm() is called.
> > + *
> > + * After that point of no return @job is committed to be executed by the
> > + * scheduler, and this function should be called from the
> > + * &drm_sched_backend_ops.free_job callback.
> > */
> > void drm_sched_job_cleanup(struct drm_sched_job *job)
> > {
> > - dma_fence_put(&job->s_fence->finished);
> > + if (kref_read(&job->s_fence->finished.refcount)) {
> > + /* drm_sched_job_arm() has been called */
> > + dma_fence_put(&job->s_fence->finished);
> > + } else {
> > + /* aborted job before committing to run it */
> > + drm_sched_fence_free(&job->s_fence->finished.rcu);
> > + }
> > +
> > job->s_fence = NULL;
> > }
> > EXPORT_SYMBOL(drm_sched_job_cleanup);
> > diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
> > index 5689da118197..2e808097b4d1 100644
> > --- a/drivers/gpu/drm/v3d/v3d_gem.c
> > +++ b/drivers/gpu/drm/v3d/v3d_gem.c
> > @@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
> > if (ret)
> > return ret;
> >
> > + drm_sched_job_arm(&job->base);
> > +
> > job->done_fence = dma_fence_get(&job->base.s_fence->finished);
> >
> > /* put by scheduler job completion */
> > diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> > index 88ae7f331bb1..83afc3aa8e2f 100644
> > --- a/include/drm/gpu_scheduler.h
> > +++ b/include/drm/gpu_scheduler.h
> > @@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
> > int drm_sched_job_init(struct drm_sched_job *job,
> > struct drm_sched_entity *entity,
> > void *owner);
> > +void drm_sched_job_arm(struct drm_sched_job *job);
> > void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
> > struct drm_gpu_scheduler **sched_list,
> > unsigned int num_sched_list);
> > @@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> > enum drm_sched_priority priority);
> > bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
> >
> > -struct drm_sched_fence *drm_sched_fence_create(
> > +struct drm_sched_fence *drm_sched_fence_alloc(
> > struct drm_sched_entity *s_entity, void *owner);
> > +void drm_sched_fence_init(struct drm_sched_fence *fence,
> > + struct drm_sched_entity *entity);
> > +void drm_sched_fence_free(struct rcu_head *rcu);
> > +
> > void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
> > void drm_sched_fence_finished(struct drm_sched_fence *fence);
> >
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 01/20] drm/sched: Split drm_sched_job_init
2021-08-05 14:07 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 14:47 ` Christian König
-1 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 14:47 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Melissa Wen,
Melissa Wen, Emma Anholt, Steven Price, Boris Brezillon,
Daniel Vetter, Lucas Stach, Russell King, Christian Gmeiner,
Qiang Yu, Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig,
David Airlie, Sumit Semwal, Masahiro Yamada, Kees Cook,
Adam Borowski, Nick Terrell, Mauro Carvalho Chehab, Paul Menzel,
Sami Tolvanen, Viresh Kumar, Alex Deucher, Dave Airlie,
Nirmoy Das, Deepak R Varma, Lee Jones, Kevin Wang, Chen Li,
Luben Tuikov, Marek Olšák, Dennis Li,
Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang, Tian Tao,
The etnaviv authors, lima,
open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
Am 05.08.21 um 16:07 schrieb Daniel Vetter:
> On Thu, Aug 5, 2021 at 3:44 PM Christian König <christian.koenig@amd.com> wrote:
>> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
>>> This is a very confusingly named function, because not just does it
>>> init an object, it arms it and provides a point of no return for
>>> pushing a job into the scheduler. It would be nice if that's a bit
>>> clearer in the interface.
>>>
>>> But the real reason is that I want to push the dependency tracking
>>> helpers into the scheduler code, and that means drm_sched_job_init
>>> must be called a lot earlier, without arming the job.
>>>
>>> v2:
>>> - don't change .gitignore (Steven)
>>> - don't forget v3d (Emma)
>>>
>>> v3: Emma noticed that I leak the memory allocated in
>>> drm_sched_job_init if we bail out before the point of no return in
>>> subsequent driver patches. To be able to fix this change
>>> drm_sched_job_cleanup() so it can handle being called both before and
>>> after drm_sched_job_arm().
>>>
>>> Also improve the kerneldoc for this.
>>>
>>> v4:
>>> - Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
>>> usual (Melissa)
>>>
>>> - Christian pointed out that drm_sched_entity_select_rq() also needs
>>> to be moved into drm_sched_job_arm, which made me realize that the
>>> job->id definitely needs to be moved too.
>>>
>>> Shuffle things to fit between job_init and job_arm.
>>>
>>> v5:
>>> Reshuffle the split between init/arm once more, amdgpu abuses
>>> drm_sched.ready to signal gpu reset failures. Also document this
>>> somewhat. (Christian)
>>>
>>> v6:
>>> Rebase on top of the msm drm/sched support. Note that the
>>> drm_sched_job_init() call is completely misplaced, and hence also the
>>> split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
>>> patch will address.
>>>
>>> Acked-by: Melissa Wen <mwen@igalia.com>
>>> Cc: Melissa Wen <melissa.srw@gmail.com>
>>> Acked-by: Emma Anholt <emma@anholt.net>
>>> Acked-by: Steven Price <steven.price@arm.com> (v2)
>>> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
>>> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
>> At least the amdgpu parts look ok of hand, but I can't judge the rest I
>> think.
> The thing that really scares me here and that I got wrong a few times
> is the cleanup for drm_sched_job at the various points. Can you give
> those parts in drm/scheduler/ a full review pls, just to make sure? I
> can note that in the tag ofc, just like a bit more confidence here
> that it's not busted :-)
I can take another look, but I won't have time for that in the next two
weeks - vacation and kid starting school.
Christian.
>
>> So only Acked-by: Christian König <christian.koenig@amd.com>
> Thanks, Daniel
>
>>> Cc: Lucas Stach <l.stach@pengutronix.de>
>>> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
>>> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
>>> Cc: Qiang Yu <yuq825@gmail.com>
>>> Cc: Rob Herring <robh@kernel.org>
>>> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
>>> Cc: Steven Price <steven.price@arm.com>
>>> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
>>> Cc: David Airlie <airlied@linux.ie>
>>> Cc: Daniel Vetter <daniel@ffwll.ch>
>>> Cc: Sumit Semwal <sumit.semwal@linaro.org>
>>> Cc: "Christian König" <christian.koenig@amd.com>
>>> Cc: Masahiro Yamada <masahiroy@kernel.org>
>>> Cc: Kees Cook <keescook@chromium.org>
>>> Cc: Adam Borowski <kilobyte@angband.pl>
>>> Cc: Nick Terrell <terrelln@fb.com>
>>> Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
>>> Cc: Paul Menzel <pmenzel@molgen.mpg.de>
>>> Cc: Sami Tolvanen <samitolvanen@google.com>
>>> Cc: Viresh Kumar <viresh.kumar@linaro.org>
>>> Cc: Alex Deucher <alexander.deucher@amd.com>
>>> Cc: Dave Airlie <airlied@redhat.com>
>>> Cc: Nirmoy Das <nirmoy.das@amd.com>
>>> Cc: Deepak R Varma <mh12gx2825@gmail.com>
>>> Cc: Lee Jones <lee.jones@linaro.org>
>>> Cc: Kevin Wang <kevin1.wang@amd.com>
>>> Cc: Chen Li <chenli@uniontech.com>
>>> Cc: Luben Tuikov <luben.tuikov@amd.com>
>>> Cc: "Marek Olšák" <marek.olsak@amd.com>
>>> Cc: Dennis Li <Dennis.Li@amd.com>
>>> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
>>> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>> Cc: Sonny Jiang <sonny.jiang@amd.com>
>>> Cc: Boris Brezillon <boris.brezillon@collabora.com>
>>> Cc: Tian Tao <tiantao6@hisilicon.com>
>>> Cc: etnaviv@lists.freedesktop.org
>>> Cc: lima@lists.freedesktop.org
>>> Cc: linux-media@vger.kernel.org
>>> Cc: linaro-mm-sig@lists.linaro.org
>>> Cc: Emma Anholt <emma@anholt.net>
>>> Cc: Rob Clark <robdclark@gmail.com>
>>> Cc: Sean Paul <sean@poorly.run>
>>> Cc: linux-arm-msm@vger.kernel.org
>>> Cc: freedreno@lists.freedesktop.org
>>> ---
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
>>> drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
>>> drivers/gpu/drm/lima/lima_sched.c | 2 +
>>> drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++
>>> drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
>>> drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
>>> drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
>>> drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
>>> drivers/gpu/drm/v3d/v3d_gem.c | 2 +
>>> include/drm/gpu_scheduler.h | 7 ++-
>>> 11 files changed, 94 insertions(+), 22 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>>> index 139cd3bf1ad6..32e80bc6af22 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>>> @@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
>>> if (r)
>>> goto error_unlock;
>>>
>>> + drm_sched_job_arm(&job->base);
>>> +
>>> /* No memory allocation is allowed while holding the notifier lock.
>>> * The lock is held until amdgpu_cs_submit is finished and fence is
>>> * added to BOs.
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
>>> index d33e6d97cc89..5ddb955d2315 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
>>> @@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
>>> if (r)
>>> return r;
>>>
>>> + drm_sched_job_arm(&job->base);
>>> +
>>> *f = dma_fence_get(&job->base.s_fence->finished);
>>> amdgpu_job_free_resources(job);
>>> drm_sched_entity_push_job(&job->base, entity);
>>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
>>> index feb6da1b6ceb..05f412204118 100644
>>> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
>>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
>>> @@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
>>> if (ret)
>>> goto out_unlock;
>>>
>>> + drm_sched_job_arm(&submit->sched_job);
>>> +
>>> submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
>>> submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
>>> submit->out_fence, 0,
>>> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
>>> index dba8329937a3..38f755580507 100644
>>> --- a/drivers/gpu/drm/lima/lima_sched.c
>>> +++ b/drivers/gpu/drm/lima/lima_sched.c
>>> @@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
>>> return err;
>>> }
>>>
>>> + drm_sched_job_arm(&task->base);
>>> +
>>> task->num_bos = num_bos;
>>> task->vm = lima_vm_get(vm);
>>>
>>> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
>>> index fdc5367aecaa..6d6c44f0e1f3 100644
>>> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
>>> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
>>> @@ -52,6 +52,9 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
>>> return ERR_PTR(ret);
>>> }
>>>
>>> + /* FIXME: this is way too early */
>>> + drm_sched_job_arm(&job->base);
>>> +
>>> xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
>>>
>>> kref_init(&submit->ref);
>>> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
>>> index 71a72fb50e6b..2992dc85325f 100644
>>> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
>>> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
>>> @@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
>>> goto unlock;
>>> }
>>>
>>> + drm_sched_job_arm(&job->base);
>>> +
>>> job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
>>>
>>> ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
>>> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
>>> index 79554aa4dbb1..f7347c284886 100644
>>> --- a/drivers/gpu/drm/scheduler/sched_entity.c
>>> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
>>> @@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
>>> * @sched_job: job to submit
>>> * @entity: scheduler entity
>>> *
>>> - * Note: To guarantee that the order of insertion to queue matches
>>> - * the job's fence sequence number this function should be
>>> - * called with drm_sched_job_init under common lock.
>>> + * Note: To guarantee that the order of insertion to queue matches the job's
>>> + * fence sequence number this function should be called with drm_sched_job_arm()
>>> + * under common lock.
>>> *
>>> * Returns 0 for success, negative error code otherwise.
>>> */
>>> diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
>>> index 69de2c76731f..bcea035cf4c6 100644
>>> --- a/drivers/gpu/drm/scheduler/sched_fence.c
>>> +++ b/drivers/gpu/drm/scheduler/sched_fence.c
>>> @@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
>>> *
>>> * Free up the fence memory after the RCU grace period.
>>> */
>>> -static void drm_sched_fence_free(struct rcu_head *rcu)
>>> +void drm_sched_fence_free(struct rcu_head *rcu)
>>> {
>>> struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
>>> struct drm_sched_fence *fence = to_drm_sched_fence(f);
>>> @@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
>>> }
>>> EXPORT_SYMBOL(to_drm_sched_fence);
>>>
>>> -struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
>>> - void *owner)
>>> +struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
>>> + void *owner)
>>> {
>>> struct drm_sched_fence *fence = NULL;
>>> - unsigned seq;
>>>
>>> fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
>>> if (fence == NULL)
>>> return NULL;
>>>
>>> fence->owner = owner;
>>> - fence->sched = entity->rq->sched;
>>> spin_lock_init(&fence->lock);
>>>
>>> + return fence;
>>> +}
>>> +
>>> +void drm_sched_fence_init(struct drm_sched_fence *fence,
>>> + struct drm_sched_entity *entity)
>>> +{
>>> + unsigned seq;
>>> +
>>> + fence->sched = entity->rq->sched;
>>> seq = atomic_inc_return(&entity->fence_seq);
>>> dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
>>> &fence->lock, entity->fence_context, seq);
>>> dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
>>> &fence->lock, entity->fence_context + 1, seq);
>>> -
>>> - return fence;
>>> }
>>>
>>> module_init(drm_sched_fence_slab_init);
>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
>>> index 33c414d55fab..454cb6164bdc 100644
>>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>>> @@ -48,9 +48,11 @@
>>> #include <linux/wait.h>
>>> #include <linux/sched.h>
>>> #include <linux/completion.h>
>>> +#include <linux/dma-resv.h>
>>> #include <uapi/linux/sched/types.h>
>>>
>>> #include <drm/drm_print.h>
>>> +#include <drm/drm_gem.h>
>>> #include <drm/gpu_scheduler.h>
>>> #include <drm/spsc_queue.h>
>>>
>>> @@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
>>>
>>> /**
>>> * drm_sched_job_init - init a scheduler job
>>> - *
>>> * @job: scheduler job to init
>>> * @entity: scheduler entity to use
>>> * @owner: job owner for debugging
>>> @@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
>>> * Refer to drm_sched_entity_push_job() documentation
>>> * for locking considerations.
>>> *
>>> + * Drivers must make sure drm_sched_job_cleanup() if this function returns
>>> + * successfully, even when @job is aborted before drm_sched_job_arm() is called.
>>> + *
>>> + * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
>>> + * has died, which can mean that there's no valid runqueue for a @entity.
>>> + * This function returns -ENOENT in this case (which probably should be -EIO as
>>> + * a more meanigful return value).
>>> + *
>>> * Returns 0 for success, negative error code otherwise.
>>> */
>>> int drm_sched_job_init(struct drm_sched_job *job,
>>> struct drm_sched_entity *entity,
>>> void *owner)
>>> {
>>> - struct drm_gpu_scheduler *sched;
>>> -
>>> drm_sched_entity_select_rq(entity);
>>> if (!entity->rq)
>>> return -ENOENT;
>>>
>>> - sched = entity->rq->sched;
>>> -
>>> - job->sched = sched;
>>> job->entity = entity;
>>> - job->s_priority = entity->rq - sched->sched_rq;
>>> - job->s_fence = drm_sched_fence_create(entity, owner);
>>> + job->s_fence = drm_sched_fence_alloc(entity, owner);
>>> if (!job->s_fence)
>>> return -ENOMEM;
>>> - job->id = atomic64_inc_return(&sched->job_id_count);
>>>
>>> INIT_LIST_HEAD(&job->list);
>>>
>>> @@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
>>> EXPORT_SYMBOL(drm_sched_job_init);
>>>
>>> /**
>>> - * drm_sched_job_cleanup - clean up scheduler job resources
>>> + * drm_sched_job_arm - arm a scheduler job for execution
>>> + * @job: scheduler job to arm
>>> + *
>>> + * This arms a scheduler job for execution. Specifically it initializes the
>>> + * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
>>> + * or other places that need to track the completion of this job.
>>> + *
>>> + * Refer to drm_sched_entity_push_job() documentation for locking
>>> + * considerations.
>>> *
>>> + * This can only be called if drm_sched_job_init() succeeded.
>>> + */
>>> +void drm_sched_job_arm(struct drm_sched_job *job)
>>> +{
>>> + struct drm_gpu_scheduler *sched;
>>> + struct drm_sched_entity *entity = job->entity;
>>> +
>>> + BUG_ON(!entity);
>>> +
>>> + sched = entity->rq->sched;
>>> +
>>> + job->sched = sched;
>>> + job->s_priority = entity->rq - sched->sched_rq;
>>> + job->id = atomic64_inc_return(&sched->job_id_count);
>>> +
>>> + drm_sched_fence_init(job->s_fence, job->entity);
>>> +}
>>> +EXPORT_SYMBOL(drm_sched_job_arm);
>>> +
>>> +/**
>>> + * drm_sched_job_cleanup - clean up scheduler job resources
>>> * @job: scheduler job to clean up
>>> + *
>>> + * Cleans up the resources allocated with drm_sched_job_init().
>>> + *
>>> + * Drivers should call this from their error unwind code if @job is aborted
>>> + * before drm_sched_job_arm() is called.
>>> + *
>>> + * After that point of no return @job is committed to be executed by the
>>> + * scheduler, and this function should be called from the
>>> + * &drm_sched_backend_ops.free_job callback.
>>> */
>>> void drm_sched_job_cleanup(struct drm_sched_job *job)
>>> {
>>> - dma_fence_put(&job->s_fence->finished);
>>> + if (kref_read(&job->s_fence->finished.refcount)) {
>>> + /* drm_sched_job_arm() has been called */
>>> + dma_fence_put(&job->s_fence->finished);
>>> + } else {
>>> + /* aborted job before committing to run it */
>>> + drm_sched_fence_free(&job->s_fence->finished.rcu);
>>> + }
>>> +
>>> job->s_fence = NULL;
>>> }
>>> EXPORT_SYMBOL(drm_sched_job_cleanup);
>>> diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
>>> index 5689da118197..2e808097b4d1 100644
>>> --- a/drivers/gpu/drm/v3d/v3d_gem.c
>>> +++ b/drivers/gpu/drm/v3d/v3d_gem.c
>>> @@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
>>> if (ret)
>>> return ret;
>>>
>>> + drm_sched_job_arm(&job->base);
>>> +
>>> job->done_fence = dma_fence_get(&job->base.s_fence->finished);
>>>
>>> /* put by scheduler job completion */
>>> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
>>> index 88ae7f331bb1..83afc3aa8e2f 100644
>>> --- a/include/drm/gpu_scheduler.h
>>> +++ b/include/drm/gpu_scheduler.h
>>> @@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
>>> int drm_sched_job_init(struct drm_sched_job *job,
>>> struct drm_sched_entity *entity,
>>> void *owner);
>>> +void drm_sched_job_arm(struct drm_sched_job *job);
>>> void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
>>> struct drm_gpu_scheduler **sched_list,
>>> unsigned int num_sched_list);
>>> @@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
>>> enum drm_sched_priority priority);
>>> bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
>>>
>>> -struct drm_sched_fence *drm_sched_fence_create(
>>> +struct drm_sched_fence *drm_sched_fence_alloc(
>>> struct drm_sched_entity *s_entity, void *owner);
>>> +void drm_sched_fence_init(struct drm_sched_fence *fence,
>>> + struct drm_sched_entity *entity);
>>> +void drm_sched_fence_free(struct rcu_head *rcu);
>>> +
>>> void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
>>> void drm_sched_fence_finished(struct drm_sched_fence *fence);
>>>
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 01/20] drm/sched: Split drm_sched_job_init
@ 2021-08-05 14:47 ` Christian König
0 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 14:47 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Melissa Wen,
Melissa Wen, Emma Anholt, Steven Price, Boris Brezillon,
Daniel Vetter, Lucas Stach, Russell King, Christian Gmeiner,
Qiang Yu, Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig,
David Airlie, Sumit Semwal, Masahiro Yamada, Kees Cook,
Adam Borowski, Nick Terrell, Mauro Carvalho Chehab, Paul Menzel,
Sami Tolvanen, Viresh Kumar, Alex Deucher, Dave Airlie,
Nirmoy Das, Deepak R Varma, Lee Jones, Kevin Wang, Chen Li,
Luben Tuikov, Marek Olšák, Dennis Li,
Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang, Tian Tao,
The etnaviv authors, lima,
open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
Am 05.08.21 um 16:07 schrieb Daniel Vetter:
> On Thu, Aug 5, 2021 at 3:44 PM Christian König <christian.koenig@amd.com> wrote:
>> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
>>> This is a very confusingly named function, because not just does it
>>> init an object, it arms it and provides a point of no return for
>>> pushing a job into the scheduler. It would be nice if that's a bit
>>> clearer in the interface.
>>>
>>> But the real reason is that I want to push the dependency tracking
>>> helpers into the scheduler code, and that means drm_sched_job_init
>>> must be called a lot earlier, without arming the job.
>>>
>>> v2:
>>> - don't change .gitignore (Steven)
>>> - don't forget v3d (Emma)
>>>
>>> v3: Emma noticed that I leak the memory allocated in
>>> drm_sched_job_init if we bail out before the point of no return in
>>> subsequent driver patches. To be able to fix this change
>>> drm_sched_job_cleanup() so it can handle being called both before and
>>> after drm_sched_job_arm().
>>>
>>> Also improve the kerneldoc for this.
>>>
>>> v4:
>>> - Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
>>> usual (Melissa)
>>>
>>> - Christian pointed out that drm_sched_entity_select_rq() also needs
>>> to be moved into drm_sched_job_arm, which made me realize that the
>>> job->id definitely needs to be moved too.
>>>
>>> Shuffle things to fit between job_init and job_arm.
>>>
>>> v5:
>>> Reshuffle the split between init/arm once more, amdgpu abuses
>>> drm_sched.ready to signal gpu reset failures. Also document this
>>> somewhat. (Christian)
>>>
>>> v6:
>>> Rebase on top of the msm drm/sched support. Note that the
>>> drm_sched_job_init() call is completely misplaced, and hence also the
>>> split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
>>> patch will address.
>>>
>>> Acked-by: Melissa Wen <mwen@igalia.com>
>>> Cc: Melissa Wen <melissa.srw@gmail.com>
>>> Acked-by: Emma Anholt <emma@anholt.net>
>>> Acked-by: Steven Price <steven.price@arm.com> (v2)
>>> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
>>> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
>> At least the amdgpu parts look ok of hand, but I can't judge the rest I
>> think.
> The thing that really scares me here and that I got wrong a few times
> is the cleanup for drm_sched_job at the various points. Can you give
> those parts in drm/scheduler/ a full review pls, just to make sure? I
> can note that in the tag ofc, just like a bit more confidence here
> that it's not busted :-)
I can take another look, but I won't have time for that in the next two
weeks - vacation and kid starting school.
Christian.
>
>> So only Acked-by: Christian König <christian.koenig@amd.com>
> Thanks, Daniel
>
>>> Cc: Lucas Stach <l.stach@pengutronix.de>
>>> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
>>> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
>>> Cc: Qiang Yu <yuq825@gmail.com>
>>> Cc: Rob Herring <robh@kernel.org>
>>> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
>>> Cc: Steven Price <steven.price@arm.com>
>>> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
>>> Cc: David Airlie <airlied@linux.ie>
>>> Cc: Daniel Vetter <daniel@ffwll.ch>
>>> Cc: Sumit Semwal <sumit.semwal@linaro.org>
>>> Cc: "Christian König" <christian.koenig@amd.com>
>>> Cc: Masahiro Yamada <masahiroy@kernel.org>
>>> Cc: Kees Cook <keescook@chromium.org>
>>> Cc: Adam Borowski <kilobyte@angband.pl>
>>> Cc: Nick Terrell <terrelln@fb.com>
>>> Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
>>> Cc: Paul Menzel <pmenzel@molgen.mpg.de>
>>> Cc: Sami Tolvanen <samitolvanen@google.com>
>>> Cc: Viresh Kumar <viresh.kumar@linaro.org>
>>> Cc: Alex Deucher <alexander.deucher@amd.com>
>>> Cc: Dave Airlie <airlied@redhat.com>
>>> Cc: Nirmoy Das <nirmoy.das@amd.com>
>>> Cc: Deepak R Varma <mh12gx2825@gmail.com>
>>> Cc: Lee Jones <lee.jones@linaro.org>
>>> Cc: Kevin Wang <kevin1.wang@amd.com>
>>> Cc: Chen Li <chenli@uniontech.com>
>>> Cc: Luben Tuikov <luben.tuikov@amd.com>
>>> Cc: "Marek Olšák" <marek.olsak@amd.com>
>>> Cc: Dennis Li <Dennis.Li@amd.com>
>>> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
>>> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>> Cc: Sonny Jiang <sonny.jiang@amd.com>
>>> Cc: Boris Brezillon <boris.brezillon@collabora.com>
>>> Cc: Tian Tao <tiantao6@hisilicon.com>
>>> Cc: etnaviv@lists.freedesktop.org
>>> Cc: lima@lists.freedesktop.org
>>> Cc: linux-media@vger.kernel.org
>>> Cc: linaro-mm-sig@lists.linaro.org
>>> Cc: Emma Anholt <emma@anholt.net>
>>> Cc: Rob Clark <robdclark@gmail.com>
>>> Cc: Sean Paul <sean@poorly.run>
>>> Cc: linux-arm-msm@vger.kernel.org
>>> Cc: freedreno@lists.freedesktop.org
>>> ---
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
>>> drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
>>> drivers/gpu/drm/lima/lima_sched.c | 2 +
>>> drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++
>>> drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
>>> drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
>>> drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
>>> drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
>>> drivers/gpu/drm/v3d/v3d_gem.c | 2 +
>>> include/drm/gpu_scheduler.h | 7 ++-
>>> 11 files changed, 94 insertions(+), 22 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>>> index 139cd3bf1ad6..32e80bc6af22 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
>>> @@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
>>> if (r)
>>> goto error_unlock;
>>>
>>> + drm_sched_job_arm(&job->base);
>>> +
>>> /* No memory allocation is allowed while holding the notifier lock.
>>> * The lock is held until amdgpu_cs_submit is finished and fence is
>>> * added to BOs.
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
>>> index d33e6d97cc89..5ddb955d2315 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
>>> @@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
>>> if (r)
>>> return r;
>>>
>>> + drm_sched_job_arm(&job->base);
>>> +
>>> *f = dma_fence_get(&job->base.s_fence->finished);
>>> amdgpu_job_free_resources(job);
>>> drm_sched_entity_push_job(&job->base, entity);
>>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
>>> index feb6da1b6ceb..05f412204118 100644
>>> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
>>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
>>> @@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
>>> if (ret)
>>> goto out_unlock;
>>>
>>> + drm_sched_job_arm(&submit->sched_job);
>>> +
>>> submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
>>> submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
>>> submit->out_fence, 0,
>>> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
>>> index dba8329937a3..38f755580507 100644
>>> --- a/drivers/gpu/drm/lima/lima_sched.c
>>> +++ b/drivers/gpu/drm/lima/lima_sched.c
>>> @@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
>>> return err;
>>> }
>>>
>>> + drm_sched_job_arm(&task->base);
>>> +
>>> task->num_bos = num_bos;
>>> task->vm = lima_vm_get(vm);
>>>
>>> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
>>> index fdc5367aecaa..6d6c44f0e1f3 100644
>>> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
>>> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
>>> @@ -52,6 +52,9 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
>>> return ERR_PTR(ret);
>>> }
>>>
>>> + /* FIXME: this is way too early */
>>> + drm_sched_job_arm(&job->base);
>>> +
>>> xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
>>>
>>> kref_init(&submit->ref);
>>> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
>>> index 71a72fb50e6b..2992dc85325f 100644
>>> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
>>> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
>>> @@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
>>> goto unlock;
>>> }
>>>
>>> + drm_sched_job_arm(&job->base);
>>> +
>>> job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
>>>
>>> ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
>>> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
>>> index 79554aa4dbb1..f7347c284886 100644
>>> --- a/drivers/gpu/drm/scheduler/sched_entity.c
>>> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
>>> @@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
>>> * @sched_job: job to submit
>>> * @entity: scheduler entity
>>> *
>>> - * Note: To guarantee that the order of insertion to queue matches
>>> - * the job's fence sequence number this function should be
>>> - * called with drm_sched_job_init under common lock.
>>> + * Note: To guarantee that the order of insertion to queue matches the job's
>>> + * fence sequence number this function should be called with drm_sched_job_arm()
>>> + * under common lock.
>>> *
>>> * Returns 0 for success, negative error code otherwise.
>>> */
>>> diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
>>> index 69de2c76731f..bcea035cf4c6 100644
>>> --- a/drivers/gpu/drm/scheduler/sched_fence.c
>>> +++ b/drivers/gpu/drm/scheduler/sched_fence.c
>>> @@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
>>> *
>>> * Free up the fence memory after the RCU grace period.
>>> */
>>> -static void drm_sched_fence_free(struct rcu_head *rcu)
>>> +void drm_sched_fence_free(struct rcu_head *rcu)
>>> {
>>> struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
>>> struct drm_sched_fence *fence = to_drm_sched_fence(f);
>>> @@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
>>> }
>>> EXPORT_SYMBOL(to_drm_sched_fence);
>>>
>>> -struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
>>> - void *owner)
>>> +struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
>>> + void *owner)
>>> {
>>> struct drm_sched_fence *fence = NULL;
>>> - unsigned seq;
>>>
>>> fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
>>> if (fence == NULL)
>>> return NULL;
>>>
>>> fence->owner = owner;
>>> - fence->sched = entity->rq->sched;
>>> spin_lock_init(&fence->lock);
>>>
>>> + return fence;
>>> +}
>>> +
>>> +void drm_sched_fence_init(struct drm_sched_fence *fence,
>>> + struct drm_sched_entity *entity)
>>> +{
>>> + unsigned seq;
>>> +
>>> + fence->sched = entity->rq->sched;
>>> seq = atomic_inc_return(&entity->fence_seq);
>>> dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
>>> &fence->lock, entity->fence_context, seq);
>>> dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
>>> &fence->lock, entity->fence_context + 1, seq);
>>> -
>>> - return fence;
>>> }
>>>
>>> module_init(drm_sched_fence_slab_init);
>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
>>> index 33c414d55fab..454cb6164bdc 100644
>>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>>> @@ -48,9 +48,11 @@
>>> #include <linux/wait.h>
>>> #include <linux/sched.h>
>>> #include <linux/completion.h>
>>> +#include <linux/dma-resv.h>
>>> #include <uapi/linux/sched/types.h>
>>>
>>> #include <drm/drm_print.h>
>>> +#include <drm/drm_gem.h>
>>> #include <drm/gpu_scheduler.h>
>>> #include <drm/spsc_queue.h>
>>>
>>> @@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
>>>
>>> /**
>>> * drm_sched_job_init - init a scheduler job
>>> - *
>>> * @job: scheduler job to init
>>> * @entity: scheduler entity to use
>>> * @owner: job owner for debugging
>>> @@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
>>> * Refer to drm_sched_entity_push_job() documentation
>>> * for locking considerations.
>>> *
>>> + * Drivers must make sure drm_sched_job_cleanup() if this function returns
>>> + * successfully, even when @job is aborted before drm_sched_job_arm() is called.
>>> + *
>>> + * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
>>> + * has died, which can mean that there's no valid runqueue for a @entity.
>>> + * This function returns -ENOENT in this case (which probably should be -EIO as
>>> + * a more meanigful return value).
>>> + *
>>> * Returns 0 for success, negative error code otherwise.
>>> */
>>> int drm_sched_job_init(struct drm_sched_job *job,
>>> struct drm_sched_entity *entity,
>>> void *owner)
>>> {
>>> - struct drm_gpu_scheduler *sched;
>>> -
>>> drm_sched_entity_select_rq(entity);
>>> if (!entity->rq)
>>> return -ENOENT;
>>>
>>> - sched = entity->rq->sched;
>>> -
>>> - job->sched = sched;
>>> job->entity = entity;
>>> - job->s_priority = entity->rq - sched->sched_rq;
>>> - job->s_fence = drm_sched_fence_create(entity, owner);
>>> + job->s_fence = drm_sched_fence_alloc(entity, owner);
>>> if (!job->s_fence)
>>> return -ENOMEM;
>>> - job->id = atomic64_inc_return(&sched->job_id_count);
>>>
>>> INIT_LIST_HEAD(&job->list);
>>>
>>> @@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
>>> EXPORT_SYMBOL(drm_sched_job_init);
>>>
>>> /**
>>> - * drm_sched_job_cleanup - clean up scheduler job resources
>>> + * drm_sched_job_arm - arm a scheduler job for execution
>>> + * @job: scheduler job to arm
>>> + *
>>> + * This arms a scheduler job for execution. Specifically it initializes the
>>> + * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
>>> + * or other places that need to track the completion of this job.
>>> + *
>>> + * Refer to drm_sched_entity_push_job() documentation for locking
>>> + * considerations.
>>> *
>>> + * This can only be called if drm_sched_job_init() succeeded.
>>> + */
>>> +void drm_sched_job_arm(struct drm_sched_job *job)
>>> +{
>>> + struct drm_gpu_scheduler *sched;
>>> + struct drm_sched_entity *entity = job->entity;
>>> +
>>> + BUG_ON(!entity);
>>> +
>>> + sched = entity->rq->sched;
>>> +
>>> + job->sched = sched;
>>> + job->s_priority = entity->rq - sched->sched_rq;
>>> + job->id = atomic64_inc_return(&sched->job_id_count);
>>> +
>>> + drm_sched_fence_init(job->s_fence, job->entity);
>>> +}
>>> +EXPORT_SYMBOL(drm_sched_job_arm);
>>> +
>>> +/**
>>> + * drm_sched_job_cleanup - clean up scheduler job resources
>>> * @job: scheduler job to clean up
>>> + *
>>> + * Cleans up the resources allocated with drm_sched_job_init().
>>> + *
>>> + * Drivers should call this from their error unwind code if @job is aborted
>>> + * before drm_sched_job_arm() is called.
>>> + *
>>> + * After that point of no return @job is committed to be executed by the
>>> + * scheduler, and this function should be called from the
>>> + * &drm_sched_backend_ops.free_job callback.
>>> */
>>> void drm_sched_job_cleanup(struct drm_sched_job *job)
>>> {
>>> - dma_fence_put(&job->s_fence->finished);
>>> + if (kref_read(&job->s_fence->finished.refcount)) {
>>> + /* drm_sched_job_arm() has been called */
>>> + dma_fence_put(&job->s_fence->finished);
>>> + } else {
>>> + /* aborted job before committing to run it */
>>> + drm_sched_fence_free(&job->s_fence->finished.rcu);
>>> + }
>>> +
>>> job->s_fence = NULL;
>>> }
>>> EXPORT_SYMBOL(drm_sched_job_cleanup);
>>> diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
>>> index 5689da118197..2e808097b4d1 100644
>>> --- a/drivers/gpu/drm/v3d/v3d_gem.c
>>> +++ b/drivers/gpu/drm/v3d/v3d_gem.c
>>> @@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
>>> if (ret)
>>> return ret;
>>>
>>> + drm_sched_job_arm(&job->base);
>>> +
>>> job->done_fence = dma_fence_get(&job->base.s_fence->finished);
>>>
>>> /* put by scheduler job completion */
>>> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
>>> index 88ae7f331bb1..83afc3aa8e2f 100644
>>> --- a/include/drm/gpu_scheduler.h
>>> +++ b/include/drm/gpu_scheduler.h
>>> @@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
>>> int drm_sched_job_init(struct drm_sched_job *job,
>>> struct drm_sched_entity *entity,
>>> void *owner);
>>> +void drm_sched_job_arm(struct drm_sched_job *job);
>>> void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
>>> struct drm_gpu_scheduler **sched_list,
>>> unsigned int num_sched_list);
>>> @@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
>>> enum drm_sched_priority priority);
>>> bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
>>>
>>> -struct drm_sched_fence *drm_sched_fence_create(
>>> +struct drm_sched_fence *drm_sched_fence_alloc(
>>> struct drm_sched_entity *s_entity, void *owner);
>>> +void drm_sched_fence_init(struct drm_sched_fence *fence,
>>> + struct drm_sched_entity *entity);
>>> +void drm_sched_fence_free(struct rcu_head *rcu);
>>> +
>>> void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
>>> void drm_sched_fence_finished(struct drm_sched_fence *fence);
>>>
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 01/20] drm/sched: Split drm_sched_job_init
2021-08-05 14:47 ` [Intel-gfx] " Christian König
@ 2021-08-05 15:07 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 15:07 UTC (permalink / raw)
To: Christian König
Cc: DRI Development, Intel Graphics Development, Melissa Wen,
Melissa Wen, Emma Anholt, Steven Price, Boris Brezillon,
Daniel Vetter, Lucas Stach, Russell King, Christian Gmeiner,
Qiang Yu, Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig,
David Airlie, Sumit Semwal, Masahiro Yamada, Kees Cook,
Adam Borowski, Nick Terrell, Mauro Carvalho Chehab, Paul Menzel,
Sami Tolvanen, Viresh Kumar, Alex Deucher, Dave Airlie,
Nirmoy Das, Deepak R Varma, Lee Jones, Kevin Wang, Chen Li,
Luben Tuikov, Marek Olšák, Dennis Li,
Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang, Tian Tao,
The etnaviv authors, lima,
open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
On Thu, Aug 5, 2021 at 4:47 PM Christian König <christian.koenig@amd.com> wrote:
>
> Am 05.08.21 um 16:07 schrieb Daniel Vetter:
> > On Thu, Aug 5, 2021 at 3:44 PM Christian König <christian.koenig@amd.com> wrote:
> >> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> >>> This is a very confusingly named function, because not just does it
> >>> init an object, it arms it and provides a point of no return for
> >>> pushing a job into the scheduler. It would be nice if that's a bit
> >>> clearer in the interface.
> >>>
> >>> But the real reason is that I want to push the dependency tracking
> >>> helpers into the scheduler code, and that means drm_sched_job_init
> >>> must be called a lot earlier, without arming the job.
> >>>
> >>> v2:
> >>> - don't change .gitignore (Steven)
> >>> - don't forget v3d (Emma)
> >>>
> >>> v3: Emma noticed that I leak the memory allocated in
> >>> drm_sched_job_init if we bail out before the point of no return in
> >>> subsequent driver patches. To be able to fix this change
> >>> drm_sched_job_cleanup() so it can handle being called both before and
> >>> after drm_sched_job_arm().
> >>>
> >>> Also improve the kerneldoc for this.
> >>>
> >>> v4:
> >>> - Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
> >>> usual (Melissa)
> >>>
> >>> - Christian pointed out that drm_sched_entity_select_rq() also needs
> >>> to be moved into drm_sched_job_arm, which made me realize that the
> >>> job->id definitely needs to be moved too.
> >>>
> >>> Shuffle things to fit between job_init and job_arm.
> >>>
> >>> v5:
> >>> Reshuffle the split between init/arm once more, amdgpu abuses
> >>> drm_sched.ready to signal gpu reset failures. Also document this
> >>> somewhat. (Christian)
> >>>
> >>> v6:
> >>> Rebase on top of the msm drm/sched support. Note that the
> >>> drm_sched_job_init() call is completely misplaced, and hence also the
> >>> split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
> >>> patch will address.
> >>>
> >>> Acked-by: Melissa Wen <mwen@igalia.com>
> >>> Cc: Melissa Wen <melissa.srw@gmail.com>
> >>> Acked-by: Emma Anholt <emma@anholt.net>
> >>> Acked-by: Steven Price <steven.price@arm.com> (v2)
> >>> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
> >>> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> >> At least the amdgpu parts look ok of hand, but I can't judge the rest I
> >> think.
> > The thing that really scares me here and that I got wrong a few times
> > is the cleanup for drm_sched_job at the various points. Can you give
> > those parts in drm/scheduler/ a full review pls, just to make sure? I
> > can note that in the tag ofc, just like a bit more confidence here
> > that it's not busted :-)
>
> I can take another look, but I won't have time for that in the next two
> weeks - vacation and kid starting school.
Hm ok I'll ask others, since this is kinda needed for the msm fix. At
least the msm design relies on this split being present, so fixing it
without this split here would be a pile of rather pointless work.
-Daniel
> Christian.
>
> >
> >> So only Acked-by: Christian König <christian.koenig@amd.com>
> > Thanks, Daniel
> >
> >>> Cc: Lucas Stach <l.stach@pengutronix.de>
> >>> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
> >>> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> >>> Cc: Qiang Yu <yuq825@gmail.com>
> >>> Cc: Rob Herring <robh@kernel.org>
> >>> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> >>> Cc: Steven Price <steven.price@arm.com>
> >>> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> >>> Cc: David Airlie <airlied@linux.ie>
> >>> Cc: Daniel Vetter <daniel@ffwll.ch>
> >>> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> >>> Cc: "Christian König" <christian.koenig@amd.com>
> >>> Cc: Masahiro Yamada <masahiroy@kernel.org>
> >>> Cc: Kees Cook <keescook@chromium.org>
> >>> Cc: Adam Borowski <kilobyte@angband.pl>
> >>> Cc: Nick Terrell <terrelln@fb.com>
> >>> Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> >>> Cc: Paul Menzel <pmenzel@molgen.mpg.de>
> >>> Cc: Sami Tolvanen <samitolvanen@google.com>
> >>> Cc: Viresh Kumar <viresh.kumar@linaro.org>
> >>> Cc: Alex Deucher <alexander.deucher@amd.com>
> >>> Cc: Dave Airlie <airlied@redhat.com>
> >>> Cc: Nirmoy Das <nirmoy.das@amd.com>
> >>> Cc: Deepak R Varma <mh12gx2825@gmail.com>
> >>> Cc: Lee Jones <lee.jones@linaro.org>
> >>> Cc: Kevin Wang <kevin1.wang@amd.com>
> >>> Cc: Chen Li <chenli@uniontech.com>
> >>> Cc: Luben Tuikov <luben.tuikov@amd.com>
> >>> Cc: "Marek Olšák" <marek.olsak@amd.com>
> >>> Cc: Dennis Li <Dennis.Li@amd.com>
> >>> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> >>> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> >>> Cc: Sonny Jiang <sonny.jiang@amd.com>
> >>> Cc: Boris Brezillon <boris.brezillon@collabora.com>
> >>> Cc: Tian Tao <tiantao6@hisilicon.com>
> >>> Cc: etnaviv@lists.freedesktop.org
> >>> Cc: lima@lists.freedesktop.org
> >>> Cc: linux-media@vger.kernel.org
> >>> Cc: linaro-mm-sig@lists.linaro.org
> >>> Cc: Emma Anholt <emma@anholt.net>
> >>> Cc: Rob Clark <robdclark@gmail.com>
> >>> Cc: Sean Paul <sean@poorly.run>
> >>> Cc: linux-arm-msm@vger.kernel.org
> >>> Cc: freedreno@lists.freedesktop.org
> >>> ---
> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
> >>> drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
> >>> drivers/gpu/drm/lima/lima_sched.c | 2 +
> >>> drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++
> >>> drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
> >>> drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
> >>> drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
> >>> drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
> >>> drivers/gpu/drm/v3d/v3d_gem.c | 2 +
> >>> include/drm/gpu_scheduler.h | 7 ++-
> >>> 11 files changed, 94 insertions(+), 22 deletions(-)
> >>>
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> >>> index 139cd3bf1ad6..32e80bc6af22 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> >>> @@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
> >>> if (r)
> >>> goto error_unlock;
> >>>
> >>> + drm_sched_job_arm(&job->base);
> >>> +
> >>> /* No memory allocation is allowed while holding the notifier lock.
> >>> * The lock is held until amdgpu_cs_submit is finished and fence is
> >>> * added to BOs.
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> >>> index d33e6d97cc89..5ddb955d2315 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> >>> @@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
> >>> if (r)
> >>> return r;
> >>>
> >>> + drm_sched_job_arm(&job->base);
> >>> +
> >>> *f = dma_fence_get(&job->base.s_fence->finished);
> >>> amdgpu_job_free_resources(job);
> >>> drm_sched_entity_push_job(&job->base, entity);
> >>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> >>> index feb6da1b6ceb..05f412204118 100644
> >>> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> >>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> >>> @@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> >>> if (ret)
> >>> goto out_unlock;
> >>>
> >>> + drm_sched_job_arm(&submit->sched_job);
> >>> +
> >>> submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
> >>> submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
> >>> submit->out_fence, 0,
> >>> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> >>> index dba8329937a3..38f755580507 100644
> >>> --- a/drivers/gpu/drm/lima/lima_sched.c
> >>> +++ b/drivers/gpu/drm/lima/lima_sched.c
> >>> @@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
> >>> return err;
> >>> }
> >>>
> >>> + drm_sched_job_arm(&task->base);
> >>> +
> >>> task->num_bos = num_bos;
> >>> task->vm = lima_vm_get(vm);
> >>>
> >>> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> >>> index fdc5367aecaa..6d6c44f0e1f3 100644
> >>> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> >>> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> >>> @@ -52,6 +52,9 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> >>> return ERR_PTR(ret);
> >>> }
> >>>
> >>> + /* FIXME: this is way too early */
> >>> + drm_sched_job_arm(&job->base);
> >>> +
> >>> xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> >>>
> >>> kref_init(&submit->ref);
> >>> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> >>> index 71a72fb50e6b..2992dc85325f 100644
> >>> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> >>> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> >>> @@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
> >>> goto unlock;
> >>> }
> >>>
> >>> + drm_sched_job_arm(&job->base);
> >>> +
> >>> job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
> >>>
> >>> ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
> >>> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> >>> index 79554aa4dbb1..f7347c284886 100644
> >>> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> >>> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> >>> @@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
> >>> * @sched_job: job to submit
> >>> * @entity: scheduler entity
> >>> *
> >>> - * Note: To guarantee that the order of insertion to queue matches
> >>> - * the job's fence sequence number this function should be
> >>> - * called with drm_sched_job_init under common lock.
> >>> + * Note: To guarantee that the order of insertion to queue matches the job's
> >>> + * fence sequence number this function should be called with drm_sched_job_arm()
> >>> + * under common lock.
> >>> *
> >>> * Returns 0 for success, negative error code otherwise.
> >>> */
> >>> diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
> >>> index 69de2c76731f..bcea035cf4c6 100644
> >>> --- a/drivers/gpu/drm/scheduler/sched_fence.c
> >>> +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> >>> @@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
> >>> *
> >>> * Free up the fence memory after the RCU grace period.
> >>> */
> >>> -static void drm_sched_fence_free(struct rcu_head *rcu)
> >>> +void drm_sched_fence_free(struct rcu_head *rcu)
> >>> {
> >>> struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
> >>> struct drm_sched_fence *fence = to_drm_sched_fence(f);
> >>> @@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> >>> }
> >>> EXPORT_SYMBOL(to_drm_sched_fence);
> >>>
> >>> -struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
> >>> - void *owner)
> >>> +struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
> >>> + void *owner)
> >>> {
> >>> struct drm_sched_fence *fence = NULL;
> >>> - unsigned seq;
> >>>
> >>> fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
> >>> if (fence == NULL)
> >>> return NULL;
> >>>
> >>> fence->owner = owner;
> >>> - fence->sched = entity->rq->sched;
> >>> spin_lock_init(&fence->lock);
> >>>
> >>> + return fence;
> >>> +}
> >>> +
> >>> +void drm_sched_fence_init(struct drm_sched_fence *fence,
> >>> + struct drm_sched_entity *entity)
> >>> +{
> >>> + unsigned seq;
> >>> +
> >>> + fence->sched = entity->rq->sched;
> >>> seq = atomic_inc_return(&entity->fence_seq);
> >>> dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
> >>> &fence->lock, entity->fence_context, seq);
> >>> dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
> >>> &fence->lock, entity->fence_context + 1, seq);
> >>> -
> >>> - return fence;
> >>> }
> >>>
> >>> module_init(drm_sched_fence_slab_init);
> >>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> >>> index 33c414d55fab..454cb6164bdc 100644
> >>> --- a/drivers/gpu/drm/scheduler/sched_main.c
> >>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> >>> @@ -48,9 +48,11 @@
> >>> #include <linux/wait.h>
> >>> #include <linux/sched.h>
> >>> #include <linux/completion.h>
> >>> +#include <linux/dma-resv.h>
> >>> #include <uapi/linux/sched/types.h>
> >>>
> >>> #include <drm/drm_print.h>
> >>> +#include <drm/drm_gem.h>
> >>> #include <drm/gpu_scheduler.h>
> >>> #include <drm/spsc_queue.h>
> >>>
> >>> @@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
> >>>
> >>> /**
> >>> * drm_sched_job_init - init a scheduler job
> >>> - *
> >>> * @job: scheduler job to init
> >>> * @entity: scheduler entity to use
> >>> * @owner: job owner for debugging
> >>> @@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
> >>> * Refer to drm_sched_entity_push_job() documentation
> >>> * for locking considerations.
> >>> *
> >>> + * Drivers must make sure drm_sched_job_cleanup() if this function returns
> >>> + * successfully, even when @job is aborted before drm_sched_job_arm() is called.
> >>> + *
> >>> + * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
> >>> + * has died, which can mean that there's no valid runqueue for a @entity.
> >>> + * This function returns -ENOENT in this case (which probably should be -EIO as
> >>> + * a more meanigful return value).
> >>> + *
> >>> * Returns 0 for success, negative error code otherwise.
> >>> */
> >>> int drm_sched_job_init(struct drm_sched_job *job,
> >>> struct drm_sched_entity *entity,
> >>> void *owner)
> >>> {
> >>> - struct drm_gpu_scheduler *sched;
> >>> -
> >>> drm_sched_entity_select_rq(entity);
> >>> if (!entity->rq)
> >>> return -ENOENT;
> >>>
> >>> - sched = entity->rq->sched;
> >>> -
> >>> - job->sched = sched;
> >>> job->entity = entity;
> >>> - job->s_priority = entity->rq - sched->sched_rq;
> >>> - job->s_fence = drm_sched_fence_create(entity, owner);
> >>> + job->s_fence = drm_sched_fence_alloc(entity, owner);
> >>> if (!job->s_fence)
> >>> return -ENOMEM;
> >>> - job->id = atomic64_inc_return(&sched->job_id_count);
> >>>
> >>> INIT_LIST_HEAD(&job->list);
> >>>
> >>> @@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
> >>> EXPORT_SYMBOL(drm_sched_job_init);
> >>>
> >>> /**
> >>> - * drm_sched_job_cleanup - clean up scheduler job resources
> >>> + * drm_sched_job_arm - arm a scheduler job for execution
> >>> + * @job: scheduler job to arm
> >>> + *
> >>> + * This arms a scheduler job for execution. Specifically it initializes the
> >>> + * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
> >>> + * or other places that need to track the completion of this job.
> >>> + *
> >>> + * Refer to drm_sched_entity_push_job() documentation for locking
> >>> + * considerations.
> >>> *
> >>> + * This can only be called if drm_sched_job_init() succeeded.
> >>> + */
> >>> +void drm_sched_job_arm(struct drm_sched_job *job)
> >>> +{
> >>> + struct drm_gpu_scheduler *sched;
> >>> + struct drm_sched_entity *entity = job->entity;
> >>> +
> >>> + BUG_ON(!entity);
> >>> +
> >>> + sched = entity->rq->sched;
> >>> +
> >>> + job->sched = sched;
> >>> + job->s_priority = entity->rq - sched->sched_rq;
> >>> + job->id = atomic64_inc_return(&sched->job_id_count);
> >>> +
> >>> + drm_sched_fence_init(job->s_fence, job->entity);
> >>> +}
> >>> +EXPORT_SYMBOL(drm_sched_job_arm);
> >>> +
> >>> +/**
> >>> + * drm_sched_job_cleanup - clean up scheduler job resources
> >>> * @job: scheduler job to clean up
> >>> + *
> >>> + * Cleans up the resources allocated with drm_sched_job_init().
> >>> + *
> >>> + * Drivers should call this from their error unwind code if @job is aborted
> >>> + * before drm_sched_job_arm() is called.
> >>> + *
> >>> + * After that point of no return @job is committed to be executed by the
> >>> + * scheduler, and this function should be called from the
> >>> + * &drm_sched_backend_ops.free_job callback.
> >>> */
> >>> void drm_sched_job_cleanup(struct drm_sched_job *job)
> >>> {
> >>> - dma_fence_put(&job->s_fence->finished);
> >>> + if (kref_read(&job->s_fence->finished.refcount)) {
> >>> + /* drm_sched_job_arm() has been called */
> >>> + dma_fence_put(&job->s_fence->finished);
> >>> + } else {
> >>> + /* aborted job before committing to run it */
> >>> + drm_sched_fence_free(&job->s_fence->finished.rcu);
> >>> + }
> >>> +
> >>> job->s_fence = NULL;
> >>> }
> >>> EXPORT_SYMBOL(drm_sched_job_cleanup);
> >>> diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
> >>> index 5689da118197..2e808097b4d1 100644
> >>> --- a/drivers/gpu/drm/v3d/v3d_gem.c
> >>> +++ b/drivers/gpu/drm/v3d/v3d_gem.c
> >>> @@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
> >>> if (ret)
> >>> return ret;
> >>>
> >>> + drm_sched_job_arm(&job->base);
> >>> +
> >>> job->done_fence = dma_fence_get(&job->base.s_fence->finished);
> >>>
> >>> /* put by scheduler job completion */
> >>> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> >>> index 88ae7f331bb1..83afc3aa8e2f 100644
> >>> --- a/include/drm/gpu_scheduler.h
> >>> +++ b/include/drm/gpu_scheduler.h
> >>> @@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
> >>> int drm_sched_job_init(struct drm_sched_job *job,
> >>> struct drm_sched_entity *entity,
> >>> void *owner);
> >>> +void drm_sched_job_arm(struct drm_sched_job *job);
> >>> void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
> >>> struct drm_gpu_scheduler **sched_list,
> >>> unsigned int num_sched_list);
> >>> @@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> >>> enum drm_sched_priority priority);
> >>> bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
> >>>
> >>> -struct drm_sched_fence *drm_sched_fence_create(
> >>> +struct drm_sched_fence *drm_sched_fence_alloc(
> >>> struct drm_sched_entity *s_entity, void *owner);
> >>> +void drm_sched_fence_init(struct drm_sched_fence *fence,
> >>> + struct drm_sched_entity *entity);
> >>> +void drm_sched_fence_free(struct rcu_head *rcu);
> >>> +
> >>> void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
> >>> void drm_sched_fence_finished(struct drm_sched_fence *fence);
> >>>
> >
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 01/20] drm/sched: Split drm_sched_job_init
@ 2021-08-05 15:07 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 15:07 UTC (permalink / raw)
To: Christian König
Cc: DRI Development, Intel Graphics Development, Melissa Wen,
Melissa Wen, Emma Anholt, Steven Price, Boris Brezillon,
Daniel Vetter, Lucas Stach, Russell King, Christian Gmeiner,
Qiang Yu, Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig,
David Airlie, Sumit Semwal, Masahiro Yamada, Kees Cook,
Adam Borowski, Nick Terrell, Mauro Carvalho Chehab, Paul Menzel,
Sami Tolvanen, Viresh Kumar, Alex Deucher, Dave Airlie,
Nirmoy Das, Deepak R Varma, Lee Jones, Kevin Wang, Chen Li,
Luben Tuikov, Marek Olšák, Dennis Li,
Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang, Tian Tao,
The etnaviv authors, lima,
open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
On Thu, Aug 5, 2021 at 4:47 PM Christian König <christian.koenig@amd.com> wrote:
>
> Am 05.08.21 um 16:07 schrieb Daniel Vetter:
> > On Thu, Aug 5, 2021 at 3:44 PM Christian König <christian.koenig@amd.com> wrote:
> >> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> >>> This is a very confusingly named function, because not just does it
> >>> init an object, it arms it and provides a point of no return for
> >>> pushing a job into the scheduler. It would be nice if that's a bit
> >>> clearer in the interface.
> >>>
> >>> But the real reason is that I want to push the dependency tracking
> >>> helpers into the scheduler code, and that means drm_sched_job_init
> >>> must be called a lot earlier, without arming the job.
> >>>
> >>> v2:
> >>> - don't change .gitignore (Steven)
> >>> - don't forget v3d (Emma)
> >>>
> >>> v3: Emma noticed that I leak the memory allocated in
> >>> drm_sched_job_init if we bail out before the point of no return in
> >>> subsequent driver patches. To be able to fix this change
> >>> drm_sched_job_cleanup() so it can handle being called both before and
> >>> after drm_sched_job_arm().
> >>>
> >>> Also improve the kerneldoc for this.
> >>>
> >>> v4:
> >>> - Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
> >>> usual (Melissa)
> >>>
> >>> - Christian pointed out that drm_sched_entity_select_rq() also needs
> >>> to be moved into drm_sched_job_arm, which made me realize that the
> >>> job->id definitely needs to be moved too.
> >>>
> >>> Shuffle things to fit between job_init and job_arm.
> >>>
> >>> v5:
> >>> Reshuffle the split between init/arm once more, amdgpu abuses
> >>> drm_sched.ready to signal gpu reset failures. Also document this
> >>> somewhat. (Christian)
> >>>
> >>> v6:
> >>> Rebase on top of the msm drm/sched support. Note that the
> >>> drm_sched_job_init() call is completely misplaced, and hence also the
> >>> split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
> >>> patch will address.
> >>>
> >>> Acked-by: Melissa Wen <mwen@igalia.com>
> >>> Cc: Melissa Wen <melissa.srw@gmail.com>
> >>> Acked-by: Emma Anholt <emma@anholt.net>
> >>> Acked-by: Steven Price <steven.price@arm.com> (v2)
> >>> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
> >>> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> >> At least the amdgpu parts look ok of hand, but I can't judge the rest I
> >> think.
> > The thing that really scares me here and that I got wrong a few times
> > is the cleanup for drm_sched_job at the various points. Can you give
> > those parts in drm/scheduler/ a full review pls, just to make sure? I
> > can note that in the tag ofc, just like a bit more confidence here
> > that it's not busted :-)
>
> I can take another look, but I won't have time for that in the next two
> weeks - vacation and kid starting school.
Hm ok I'll ask others, since this is kinda needed for the msm fix. At
least the msm design relies on this split being present, so fixing it
without this split here would be a pile of rather pointless work.
-Daniel
> Christian.
>
> >
> >> So only Acked-by: Christian König <christian.koenig@amd.com>
> > Thanks, Daniel
> >
> >>> Cc: Lucas Stach <l.stach@pengutronix.de>
> >>> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
> >>> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> >>> Cc: Qiang Yu <yuq825@gmail.com>
> >>> Cc: Rob Herring <robh@kernel.org>
> >>> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> >>> Cc: Steven Price <steven.price@arm.com>
> >>> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> >>> Cc: David Airlie <airlied@linux.ie>
> >>> Cc: Daniel Vetter <daniel@ffwll.ch>
> >>> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> >>> Cc: "Christian König" <christian.koenig@amd.com>
> >>> Cc: Masahiro Yamada <masahiroy@kernel.org>
> >>> Cc: Kees Cook <keescook@chromium.org>
> >>> Cc: Adam Borowski <kilobyte@angband.pl>
> >>> Cc: Nick Terrell <terrelln@fb.com>
> >>> Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> >>> Cc: Paul Menzel <pmenzel@molgen.mpg.de>
> >>> Cc: Sami Tolvanen <samitolvanen@google.com>
> >>> Cc: Viresh Kumar <viresh.kumar@linaro.org>
> >>> Cc: Alex Deucher <alexander.deucher@amd.com>
> >>> Cc: Dave Airlie <airlied@redhat.com>
> >>> Cc: Nirmoy Das <nirmoy.das@amd.com>
> >>> Cc: Deepak R Varma <mh12gx2825@gmail.com>
> >>> Cc: Lee Jones <lee.jones@linaro.org>
> >>> Cc: Kevin Wang <kevin1.wang@amd.com>
> >>> Cc: Chen Li <chenli@uniontech.com>
> >>> Cc: Luben Tuikov <luben.tuikov@amd.com>
> >>> Cc: "Marek Olšák" <marek.olsak@amd.com>
> >>> Cc: Dennis Li <Dennis.Li@amd.com>
> >>> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> >>> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> >>> Cc: Sonny Jiang <sonny.jiang@amd.com>
> >>> Cc: Boris Brezillon <boris.brezillon@collabora.com>
> >>> Cc: Tian Tao <tiantao6@hisilicon.com>
> >>> Cc: etnaviv@lists.freedesktop.org
> >>> Cc: lima@lists.freedesktop.org
> >>> Cc: linux-media@vger.kernel.org
> >>> Cc: linaro-mm-sig@lists.linaro.org
> >>> Cc: Emma Anholt <emma@anholt.net>
> >>> Cc: Rob Clark <robdclark@gmail.com>
> >>> Cc: Sean Paul <sean@poorly.run>
> >>> Cc: linux-arm-msm@vger.kernel.org
> >>> Cc: freedreno@lists.freedesktop.org
> >>> ---
> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
> >>> drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
> >>> drivers/gpu/drm/lima/lima_sched.c | 2 +
> >>> drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++
> >>> drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
> >>> drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
> >>> drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
> >>> drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
> >>> drivers/gpu/drm/v3d/v3d_gem.c | 2 +
> >>> include/drm/gpu_scheduler.h | 7 ++-
> >>> 11 files changed, 94 insertions(+), 22 deletions(-)
> >>>
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> >>> index 139cd3bf1ad6..32e80bc6af22 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> >>> @@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
> >>> if (r)
> >>> goto error_unlock;
> >>>
> >>> + drm_sched_job_arm(&job->base);
> >>> +
> >>> /* No memory allocation is allowed while holding the notifier lock.
> >>> * The lock is held until amdgpu_cs_submit is finished and fence is
> >>> * added to BOs.
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> >>> index d33e6d97cc89..5ddb955d2315 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> >>> @@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
> >>> if (r)
> >>> return r;
> >>>
> >>> + drm_sched_job_arm(&job->base);
> >>> +
> >>> *f = dma_fence_get(&job->base.s_fence->finished);
> >>> amdgpu_job_free_resources(job);
> >>> drm_sched_entity_push_job(&job->base, entity);
> >>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> >>> index feb6da1b6ceb..05f412204118 100644
> >>> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> >>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> >>> @@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> >>> if (ret)
> >>> goto out_unlock;
> >>>
> >>> + drm_sched_job_arm(&submit->sched_job);
> >>> +
> >>> submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
> >>> submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
> >>> submit->out_fence, 0,
> >>> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> >>> index dba8329937a3..38f755580507 100644
> >>> --- a/drivers/gpu/drm/lima/lima_sched.c
> >>> +++ b/drivers/gpu/drm/lima/lima_sched.c
> >>> @@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
> >>> return err;
> >>> }
> >>>
> >>> + drm_sched_job_arm(&task->base);
> >>> +
> >>> task->num_bos = num_bos;
> >>> task->vm = lima_vm_get(vm);
> >>>
> >>> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> >>> index fdc5367aecaa..6d6c44f0e1f3 100644
> >>> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> >>> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> >>> @@ -52,6 +52,9 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> >>> return ERR_PTR(ret);
> >>> }
> >>>
> >>> + /* FIXME: this is way too early */
> >>> + drm_sched_job_arm(&job->base);
> >>> +
> >>> xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> >>>
> >>> kref_init(&submit->ref);
> >>> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> >>> index 71a72fb50e6b..2992dc85325f 100644
> >>> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> >>> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> >>> @@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
> >>> goto unlock;
> >>> }
> >>>
> >>> + drm_sched_job_arm(&job->base);
> >>> +
> >>> job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
> >>>
> >>> ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
> >>> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> >>> index 79554aa4dbb1..f7347c284886 100644
> >>> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> >>> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> >>> @@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
> >>> * @sched_job: job to submit
> >>> * @entity: scheduler entity
> >>> *
> >>> - * Note: To guarantee that the order of insertion to queue matches
> >>> - * the job's fence sequence number this function should be
> >>> - * called with drm_sched_job_init under common lock.
> >>> + * Note: To guarantee that the order of insertion to queue matches the job's
> >>> + * fence sequence number this function should be called with drm_sched_job_arm()
> >>> + * under common lock.
> >>> *
> >>> * Returns 0 for success, negative error code otherwise.
> >>> */
> >>> diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
> >>> index 69de2c76731f..bcea035cf4c6 100644
> >>> --- a/drivers/gpu/drm/scheduler/sched_fence.c
> >>> +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> >>> @@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
> >>> *
> >>> * Free up the fence memory after the RCU grace period.
> >>> */
> >>> -static void drm_sched_fence_free(struct rcu_head *rcu)
> >>> +void drm_sched_fence_free(struct rcu_head *rcu)
> >>> {
> >>> struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
> >>> struct drm_sched_fence *fence = to_drm_sched_fence(f);
> >>> @@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> >>> }
> >>> EXPORT_SYMBOL(to_drm_sched_fence);
> >>>
> >>> -struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
> >>> - void *owner)
> >>> +struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
> >>> + void *owner)
> >>> {
> >>> struct drm_sched_fence *fence = NULL;
> >>> - unsigned seq;
> >>>
> >>> fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
> >>> if (fence == NULL)
> >>> return NULL;
> >>>
> >>> fence->owner = owner;
> >>> - fence->sched = entity->rq->sched;
> >>> spin_lock_init(&fence->lock);
> >>>
> >>> + return fence;
> >>> +}
> >>> +
> >>> +void drm_sched_fence_init(struct drm_sched_fence *fence,
> >>> + struct drm_sched_entity *entity)
> >>> +{
> >>> + unsigned seq;
> >>> +
> >>> + fence->sched = entity->rq->sched;
> >>> seq = atomic_inc_return(&entity->fence_seq);
> >>> dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
> >>> &fence->lock, entity->fence_context, seq);
> >>> dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
> >>> &fence->lock, entity->fence_context + 1, seq);
> >>> -
> >>> - return fence;
> >>> }
> >>>
> >>> module_init(drm_sched_fence_slab_init);
> >>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> >>> index 33c414d55fab..454cb6164bdc 100644
> >>> --- a/drivers/gpu/drm/scheduler/sched_main.c
> >>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> >>> @@ -48,9 +48,11 @@
> >>> #include <linux/wait.h>
> >>> #include <linux/sched.h>
> >>> #include <linux/completion.h>
> >>> +#include <linux/dma-resv.h>
> >>> #include <uapi/linux/sched/types.h>
> >>>
> >>> #include <drm/drm_print.h>
> >>> +#include <drm/drm_gem.h>
> >>> #include <drm/gpu_scheduler.h>
> >>> #include <drm/spsc_queue.h>
> >>>
> >>> @@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
> >>>
> >>> /**
> >>> * drm_sched_job_init - init a scheduler job
> >>> - *
> >>> * @job: scheduler job to init
> >>> * @entity: scheduler entity to use
> >>> * @owner: job owner for debugging
> >>> @@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
> >>> * Refer to drm_sched_entity_push_job() documentation
> >>> * for locking considerations.
> >>> *
> >>> + * Drivers must make sure drm_sched_job_cleanup() if this function returns
> >>> + * successfully, even when @job is aborted before drm_sched_job_arm() is called.
> >>> + *
> >>> + * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
> >>> + * has died, which can mean that there's no valid runqueue for a @entity.
> >>> + * This function returns -ENOENT in this case (which probably should be -EIO as
> >>> + * a more meanigful return value).
> >>> + *
> >>> * Returns 0 for success, negative error code otherwise.
> >>> */
> >>> int drm_sched_job_init(struct drm_sched_job *job,
> >>> struct drm_sched_entity *entity,
> >>> void *owner)
> >>> {
> >>> - struct drm_gpu_scheduler *sched;
> >>> -
> >>> drm_sched_entity_select_rq(entity);
> >>> if (!entity->rq)
> >>> return -ENOENT;
> >>>
> >>> - sched = entity->rq->sched;
> >>> -
> >>> - job->sched = sched;
> >>> job->entity = entity;
> >>> - job->s_priority = entity->rq - sched->sched_rq;
> >>> - job->s_fence = drm_sched_fence_create(entity, owner);
> >>> + job->s_fence = drm_sched_fence_alloc(entity, owner);
> >>> if (!job->s_fence)
> >>> return -ENOMEM;
> >>> - job->id = atomic64_inc_return(&sched->job_id_count);
> >>>
> >>> INIT_LIST_HEAD(&job->list);
> >>>
> >>> @@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
> >>> EXPORT_SYMBOL(drm_sched_job_init);
> >>>
> >>> /**
> >>> - * drm_sched_job_cleanup - clean up scheduler job resources
> >>> + * drm_sched_job_arm - arm a scheduler job for execution
> >>> + * @job: scheduler job to arm
> >>> + *
> >>> + * This arms a scheduler job for execution. Specifically it initializes the
> >>> + * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
> >>> + * or other places that need to track the completion of this job.
> >>> + *
> >>> + * Refer to drm_sched_entity_push_job() documentation for locking
> >>> + * considerations.
> >>> *
> >>> + * This can only be called if drm_sched_job_init() succeeded.
> >>> + */
> >>> +void drm_sched_job_arm(struct drm_sched_job *job)
> >>> +{
> >>> + struct drm_gpu_scheduler *sched;
> >>> + struct drm_sched_entity *entity = job->entity;
> >>> +
> >>> + BUG_ON(!entity);
> >>> +
> >>> + sched = entity->rq->sched;
> >>> +
> >>> + job->sched = sched;
> >>> + job->s_priority = entity->rq - sched->sched_rq;
> >>> + job->id = atomic64_inc_return(&sched->job_id_count);
> >>> +
> >>> + drm_sched_fence_init(job->s_fence, job->entity);
> >>> +}
> >>> +EXPORT_SYMBOL(drm_sched_job_arm);
> >>> +
> >>> +/**
> >>> + * drm_sched_job_cleanup - clean up scheduler job resources
> >>> * @job: scheduler job to clean up
> >>> + *
> >>> + * Cleans up the resources allocated with drm_sched_job_init().
> >>> + *
> >>> + * Drivers should call this from their error unwind code if @job is aborted
> >>> + * before drm_sched_job_arm() is called.
> >>> + *
> >>> + * After that point of no return @job is committed to be executed by the
> >>> + * scheduler, and this function should be called from the
> >>> + * &drm_sched_backend_ops.free_job callback.
> >>> */
> >>> void drm_sched_job_cleanup(struct drm_sched_job *job)
> >>> {
> >>> - dma_fence_put(&job->s_fence->finished);
> >>> + if (kref_read(&job->s_fence->finished.refcount)) {
> >>> + /* drm_sched_job_arm() has been called */
> >>> + dma_fence_put(&job->s_fence->finished);
> >>> + } else {
> >>> + /* aborted job before committing to run it */
> >>> + drm_sched_fence_free(&job->s_fence->finished.rcu);
> >>> + }
> >>> +
> >>> job->s_fence = NULL;
> >>> }
> >>> EXPORT_SYMBOL(drm_sched_job_cleanup);
> >>> diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
> >>> index 5689da118197..2e808097b4d1 100644
> >>> --- a/drivers/gpu/drm/v3d/v3d_gem.c
> >>> +++ b/drivers/gpu/drm/v3d/v3d_gem.c
> >>> @@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
> >>> if (ret)
> >>> return ret;
> >>>
> >>> + drm_sched_job_arm(&job->base);
> >>> +
> >>> job->done_fence = dma_fence_get(&job->base.s_fence->finished);
> >>>
> >>> /* put by scheduler job completion */
> >>> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> >>> index 88ae7f331bb1..83afc3aa8e2f 100644
> >>> --- a/include/drm/gpu_scheduler.h
> >>> +++ b/include/drm/gpu_scheduler.h
> >>> @@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
> >>> int drm_sched_job_init(struct drm_sched_job *job,
> >>> struct drm_sched_entity *entity,
> >>> void *owner);
> >>> +void drm_sched_job_arm(struct drm_sched_job *job);
> >>> void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
> >>> struct drm_gpu_scheduler **sched_list,
> >>> unsigned int num_sched_list);
> >>> @@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> >>> enum drm_sched_priority priority);
> >>> bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
> >>>
> >>> -struct drm_sched_fence *drm_sched_fence_create(
> >>> +struct drm_sched_fence *drm_sched_fence_alloc(
> >>> struct drm_sched_entity *s_entity, void *owner);
> >>> +void drm_sched_fence_init(struct drm_sched_fence *fence,
> >>> + struct drm_sched_entity *entity);
> >>> +void drm_sched_fence_free(struct rcu_head *rcu);
> >>> +
> >>> void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
> >>> void drm_sched_fence_finished(struct drm_sched_fence *fence);
> >>>
> >
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH] drm/sched: Split drm_sched_job_init
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-17 8:49 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-17 8:49 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Christian König,
Melissa Wen, Melissa Wen, Emma Anholt, Steven Price,
Boris Brezillon, Daniel Vetter, Lucas Stach, Russell King,
Christian Gmeiner, Qiang Yu, Rob Herring, Tomeu Vizoso,
Alyssa Rosenzweig, David Airlie, Daniel Vetter, Sumit Semwal,
Masahiro Yamada, Kees Cook, Adam Borowski, Nick Terrell,
Mauro Carvalho Chehab, Paul Menzel, Sami Tolvanen, Viresh Kumar,
Alex Deucher, Dave Airlie, Nirmoy Das, Deepak R Varma, Lee Jones,
Kevin Wang, Chen Li, Luben Tuikov, Marek Olšák,
Dennis Li, Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang,
Tian Tao, etnaviv, lima, linux-media, linaro-mm-sig, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
This is a very confusingly named function, because not just does it
init an object, it arms it and provides a point of no return for
pushing a job into the scheduler. It would be nice if that's a bit
clearer in the interface.
But the real reason is that I want to push the dependency tracking
helpers into the scheduler code, and that means drm_sched_job_init
must be called a lot earlier, without arming the job.
v2:
- don't change .gitignore (Steven)
- don't forget v3d (Emma)
v3: Emma noticed that I leak the memory allocated in
drm_sched_job_init if we bail out before the point of no return in
subsequent driver patches. To be able to fix this change
drm_sched_job_cleanup() so it can handle being called both before and
after drm_sched_job_arm().
Also improve the kerneldoc for this.
v4:
- Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
usual (Melissa)
- Christian pointed out that drm_sched_entity_select_rq() also needs
to be moved into drm_sched_job_arm, which made me realize that the
job->id definitely needs to be moved too.
Shuffle things to fit between job_init and job_arm.
v5:
Reshuffle the split between init/arm once more, amdgpu abuses
drm_sched.ready to signal gpu reset failures. Also document this
somewhat. (Christian)
v6:
Rebase on top of the msm drm/sched support. Note that the
drm_sched_job_init() call is completely misplaced, and hence also the
split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
patch will address.
v7: Drop the FIXME in msm, after discussions with Rob I agree it shouldn't
be a problem where it is now.
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Melissa Wen <mwen@igalia.com>
Cc: Melissa Wen <melissa.srw@gmail.com>
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Steven Price <steven.price@arm.com> (v2)
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: Qiang Yu <yuq825@gmail.com>
Cc: Rob Herring <robh@kernel.org>
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Adam Borowski <kilobyte@angband.pl>
Cc: Nick Terrell <terrelln@fb.com>
Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Cc: Paul Menzel <pmenzel@molgen.mpg.de>
Cc: Sami Tolvanen <samitolvanen@google.com>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Nirmoy Das <nirmoy.das@amd.com>
Cc: Deepak R Varma <mh12gx2825@gmail.com>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Kevin Wang <kevin1.wang@amd.com>
Cc: Chen Li <chenli@uniontech.com>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: "Marek Olšák" <marek.olsak@amd.com>
Cc: Dennis Li <Dennis.Li@amd.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Sonny Jiang <sonny.jiang@amd.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Tian Tao <tiantao6@hisilicon.com>
Cc: etnaviv@lists.freedesktop.org
Cc: lima@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: Emma Anholt <emma@anholt.net>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
---
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
drivers/gpu/drm/lima/lima_sched.c | 2 +
drivers/gpu/drm/msm/msm_gem_submit.c | 2 +
drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
drivers/gpu/drm/v3d/v3d_gem.c | 2 +
include/drm/gpu_scheduler.h | 7 ++-
11 files changed, 93 insertions(+), 22 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index c5386d13eb4a..a4ec092af9a7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
if (r)
goto error_unlock;
+ drm_sched_job_arm(&job->base);
+
/* No memory allocation is allowed while holding the notifier lock.
* The lock is held until amdgpu_cs_submit is finished and fence is
* added to BOs.
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index d33e6d97cc89..5ddb955d2315 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
if (r)
return r;
+ drm_sched_job_arm(&job->base);
+
*f = dma_fence_get(&job->base.s_fence->finished);
amdgpu_job_free_resources(job);
drm_sched_entity_push_job(&job->base, entity);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index feb6da1b6ceb..05f412204118 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
if (ret)
goto out_unlock;
+ drm_sched_job_arm(&submit->sched_job);
+
submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
submit->out_fence, 0,
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index dba8329937a3..38f755580507 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
return err;
}
+ drm_sched_job_arm(&task->base);
+
task->num_bos = num_bos;
task->vm = lima_vm_get(vm);
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index fdc5367aecaa..4d1c4d5f6a2a 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,6 +52,8 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
+ drm_sched_job_arm(&job->base);
+
xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
kref_init(&submit->ref);
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 71a72fb50e6b..2992dc85325f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
goto unlock;
}
+ drm_sched_job_arm(&job->base);
+
job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 79554aa4dbb1..f7347c284886 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
* @sched_job: job to submit
* @entity: scheduler entity
*
- * Note: To guarantee that the order of insertion to queue matches
- * the job's fence sequence number this function should be
- * called with drm_sched_job_init under common lock.
+ * Note: To guarantee that the order of insertion to queue matches the job's
+ * fence sequence number this function should be called with drm_sched_job_arm()
+ * under common lock.
*
* Returns 0 for success, negative error code otherwise.
*/
diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
index 69de2c76731f..bcea035cf4c6 100644
--- a/drivers/gpu/drm/scheduler/sched_fence.c
+++ b/drivers/gpu/drm/scheduler/sched_fence.c
@@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
*
* Free up the fence memory after the RCU grace period.
*/
-static void drm_sched_fence_free(struct rcu_head *rcu)
+void drm_sched_fence_free(struct rcu_head *rcu)
{
struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
struct drm_sched_fence *fence = to_drm_sched_fence(f);
@@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
}
EXPORT_SYMBOL(to_drm_sched_fence);
-struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
- void *owner)
+struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
+ void *owner)
{
struct drm_sched_fence *fence = NULL;
- unsigned seq;
fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
if (fence == NULL)
return NULL;
fence->owner = owner;
- fence->sched = entity->rq->sched;
spin_lock_init(&fence->lock);
+ return fence;
+}
+
+void drm_sched_fence_init(struct drm_sched_fence *fence,
+ struct drm_sched_entity *entity)
+{
+ unsigned seq;
+
+ fence->sched = entity->rq->sched;
seq = atomic_inc_return(&entity->fence_seq);
dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
&fence->lock, entity->fence_context, seq);
dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
&fence->lock, entity->fence_context + 1, seq);
-
- return fence;
}
module_init(drm_sched_fence_slab_init);
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index 33c414d55fab..454cb6164bdc 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -48,9 +48,11 @@
#include <linux/wait.h>
#include <linux/sched.h>
#include <linux/completion.h>
+#include <linux/dma-resv.h>
#include <uapi/linux/sched/types.h>
#include <drm/drm_print.h>
+#include <drm/drm_gem.h>
#include <drm/gpu_scheduler.h>
#include <drm/spsc_queue.h>
@@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
/**
* drm_sched_job_init - init a scheduler job
- *
* @job: scheduler job to init
* @entity: scheduler entity to use
* @owner: job owner for debugging
@@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
* Refer to drm_sched_entity_push_job() documentation
* for locking considerations.
*
+ * Drivers must make sure drm_sched_job_cleanup() if this function returns
+ * successfully, even when @job is aborted before drm_sched_job_arm() is called.
+ *
+ * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
+ * has died, which can mean that there's no valid runqueue for a @entity.
+ * This function returns -ENOENT in this case (which probably should be -EIO as
+ * a more meanigful return value).
+ *
* Returns 0 for success, negative error code otherwise.
*/
int drm_sched_job_init(struct drm_sched_job *job,
struct drm_sched_entity *entity,
void *owner)
{
- struct drm_gpu_scheduler *sched;
-
drm_sched_entity_select_rq(entity);
if (!entity->rq)
return -ENOENT;
- sched = entity->rq->sched;
-
- job->sched = sched;
job->entity = entity;
- job->s_priority = entity->rq - sched->sched_rq;
- job->s_fence = drm_sched_fence_create(entity, owner);
+ job->s_fence = drm_sched_fence_alloc(entity, owner);
if (!job->s_fence)
return -ENOMEM;
- job->id = atomic64_inc_return(&sched->job_id_count);
INIT_LIST_HEAD(&job->list);
@@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
EXPORT_SYMBOL(drm_sched_job_init);
/**
- * drm_sched_job_cleanup - clean up scheduler job resources
+ * drm_sched_job_arm - arm a scheduler job for execution
+ * @job: scheduler job to arm
+ *
+ * This arms a scheduler job for execution. Specifically it initializes the
+ * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
+ * or other places that need to track the completion of this job.
+ *
+ * Refer to drm_sched_entity_push_job() documentation for locking
+ * considerations.
*
+ * This can only be called if drm_sched_job_init() succeeded.
+ */
+void drm_sched_job_arm(struct drm_sched_job *job)
+{
+ struct drm_gpu_scheduler *sched;
+ struct drm_sched_entity *entity = job->entity;
+
+ BUG_ON(!entity);
+
+ sched = entity->rq->sched;
+
+ job->sched = sched;
+ job->s_priority = entity->rq - sched->sched_rq;
+ job->id = atomic64_inc_return(&sched->job_id_count);
+
+ drm_sched_fence_init(job->s_fence, job->entity);
+}
+EXPORT_SYMBOL(drm_sched_job_arm);
+
+/**
+ * drm_sched_job_cleanup - clean up scheduler job resources
* @job: scheduler job to clean up
+ *
+ * Cleans up the resources allocated with drm_sched_job_init().
+ *
+ * Drivers should call this from their error unwind code if @job is aborted
+ * before drm_sched_job_arm() is called.
+ *
+ * After that point of no return @job is committed to be executed by the
+ * scheduler, and this function should be called from the
+ * &drm_sched_backend_ops.free_job callback.
*/
void drm_sched_job_cleanup(struct drm_sched_job *job)
{
- dma_fence_put(&job->s_fence->finished);
+ if (kref_read(&job->s_fence->finished.refcount)) {
+ /* drm_sched_job_arm() has been called */
+ dma_fence_put(&job->s_fence->finished);
+ } else {
+ /* aborted job before committing to run it */
+ drm_sched_fence_free(&job->s_fence->finished.rcu);
+ }
+
job->s_fence = NULL;
}
EXPORT_SYMBOL(drm_sched_job_cleanup);
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 5689da118197..2e808097b4d1 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
if (ret)
return ret;
+ drm_sched_job_arm(&job->base);
+
job->done_fence = dma_fence_get(&job->base.s_fence->finished);
/* put by scheduler job completion */
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index 88ae7f331bb1..83afc3aa8e2f 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
int drm_sched_job_init(struct drm_sched_job *job,
struct drm_sched_entity *entity,
void *owner);
+void drm_sched_job_arm(struct drm_sched_job *job);
void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
struct drm_gpu_scheduler **sched_list,
unsigned int num_sched_list);
@@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
enum drm_sched_priority priority);
bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
-struct drm_sched_fence *drm_sched_fence_create(
+struct drm_sched_fence *drm_sched_fence_alloc(
struct drm_sched_entity *s_entity, void *owner);
+void drm_sched_fence_init(struct drm_sched_fence *fence,
+ struct drm_sched_entity *entity);
+void drm_sched_fence_free(struct rcu_head *rcu);
+
void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
void drm_sched_fence_finished(struct drm_sched_fence *fence);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH] drm/sched: Split drm_sched_job_init
@ 2021-08-17 8:49 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-17 8:49 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Christian König,
Melissa Wen, Melissa Wen, Emma Anholt, Steven Price,
Boris Brezillon, Daniel Vetter, Lucas Stach, Russell King,
Christian Gmeiner, Qiang Yu, Rob Herring, Tomeu Vizoso,
Alyssa Rosenzweig, David Airlie, Daniel Vetter, Sumit Semwal,
Masahiro Yamada, Kees Cook, Adam Borowski, Nick Terrell,
Mauro Carvalho Chehab, Paul Menzel, Sami Tolvanen, Viresh Kumar,
Alex Deucher, Dave Airlie, Nirmoy Das, Deepak R Varma, Lee Jones,
Kevin Wang, Chen Li, Luben Tuikov, Marek Olšák,
Dennis Li, Maarten Lankhorst, Andrey Grodzovsky, Sonny Jiang,
Tian Tao, etnaviv, lima, linux-media, linaro-mm-sig, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
This is a very confusingly named function, because not just does it
init an object, it arms it and provides a point of no return for
pushing a job into the scheduler. It would be nice if that's a bit
clearer in the interface.
But the real reason is that I want to push the dependency tracking
helpers into the scheduler code, and that means drm_sched_job_init
must be called a lot earlier, without arming the job.
v2:
- don't change .gitignore (Steven)
- don't forget v3d (Emma)
v3: Emma noticed that I leak the memory allocated in
drm_sched_job_init if we bail out before the point of no return in
subsequent driver patches. To be able to fix this change
drm_sched_job_cleanup() so it can handle being called both before and
after drm_sched_job_arm().
Also improve the kerneldoc for this.
v4:
- Fix the drm_sched_job_cleanup logic, I inverted the booleans, as
usual (Melissa)
- Christian pointed out that drm_sched_entity_select_rq() also needs
to be moved into drm_sched_job_arm, which made me realize that the
job->id definitely needs to be moved too.
Shuffle things to fit between job_init and job_arm.
v5:
Reshuffle the split between init/arm once more, amdgpu abuses
drm_sched.ready to signal gpu reset failures. Also document this
somewhat. (Christian)
v6:
Rebase on top of the msm drm/sched support. Note that the
drm_sched_job_init() call is completely misplaced, and hence also the
split-out drm_sched_entity_push_job(). I've put in a FIXME which the next
patch will address.
v7: Drop the FIXME in msm, after discussions with Rob I agree it shouldn't
be a problem where it is now.
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Melissa Wen <mwen@igalia.com>
Cc: Melissa Wen <melissa.srw@gmail.com>
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Steven Price <steven.price@arm.com> (v2)
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v5)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: Qiang Yu <yuq825@gmail.com>
Cc: Rob Herring <robh@kernel.org>
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Adam Borowski <kilobyte@angband.pl>
Cc: Nick Terrell <terrelln@fb.com>
Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Cc: Paul Menzel <pmenzel@molgen.mpg.de>
Cc: Sami Tolvanen <samitolvanen@google.com>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Nirmoy Das <nirmoy.das@amd.com>
Cc: Deepak R Varma <mh12gx2825@gmail.com>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Kevin Wang <kevin1.wang@amd.com>
Cc: Chen Li <chenli@uniontech.com>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: "Marek Olšák" <marek.olsak@amd.com>
Cc: Dennis Li <Dennis.Li@amd.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Sonny Jiang <sonny.jiang@amd.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Tian Tao <tiantao6@hisilicon.com>
Cc: etnaviv@lists.freedesktop.org
Cc: lima@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: Emma Anholt <emma@anholt.net>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
---
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +
drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +
drivers/gpu/drm/lima/lima_sched.c | 2 +
drivers/gpu/drm/msm/msm_gem_submit.c | 2 +
drivers/gpu/drm/panfrost/panfrost_job.c | 2 +
drivers/gpu/drm/scheduler/sched_entity.c | 6 +--
drivers/gpu/drm/scheduler/sched_fence.c | 19 ++++---
drivers/gpu/drm/scheduler/sched_main.c | 69 ++++++++++++++++++++----
drivers/gpu/drm/v3d/v3d_gem.c | 2 +
include/drm/gpu_scheduler.h | 7 ++-
11 files changed, 93 insertions(+), 22 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index c5386d13eb4a..a4ec092af9a7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -1226,6 +1226,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
if (r)
goto error_unlock;
+ drm_sched_job_arm(&job->base);
+
/* No memory allocation is allowed while holding the notifier lock.
* The lock is held until amdgpu_cs_submit is finished and fence is
* added to BOs.
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index d33e6d97cc89..5ddb955d2315 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -170,6 +170,8 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
if (r)
return r;
+ drm_sched_job_arm(&job->base);
+
*f = dma_fence_get(&job->base.s_fence->finished);
amdgpu_job_free_resources(job);
drm_sched_entity_push_job(&job->base, entity);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index feb6da1b6ceb..05f412204118 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -163,6 +163,8 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
if (ret)
goto out_unlock;
+ drm_sched_job_arm(&submit->sched_job);
+
submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
submit->out_fence_id = idr_alloc_cyclic(&submit->gpu->fence_idr,
submit->out_fence, 0,
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index dba8329937a3..38f755580507 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -129,6 +129,8 @@ int lima_sched_task_init(struct lima_sched_task *task,
return err;
}
+ drm_sched_job_arm(&task->base);
+
task->num_bos = num_bos;
task->vm = lima_vm_get(vm);
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index fdc5367aecaa..4d1c4d5f6a2a 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,6 +52,8 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
+ drm_sched_job_arm(&job->base);
+
xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
kref_init(&submit->ref);
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 71a72fb50e6b..2992dc85325f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -288,6 +288,8 @@ int panfrost_job_push(struct panfrost_job *job)
goto unlock;
}
+ drm_sched_job_arm(&job->base);
+
job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 79554aa4dbb1..f7347c284886 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -485,9 +485,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
* @sched_job: job to submit
* @entity: scheduler entity
*
- * Note: To guarantee that the order of insertion to queue matches
- * the job's fence sequence number this function should be
- * called with drm_sched_job_init under common lock.
+ * Note: To guarantee that the order of insertion to queue matches the job's
+ * fence sequence number this function should be called with drm_sched_job_arm()
+ * under common lock.
*
* Returns 0 for success, negative error code otherwise.
*/
diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
index 69de2c76731f..bcea035cf4c6 100644
--- a/drivers/gpu/drm/scheduler/sched_fence.c
+++ b/drivers/gpu/drm/scheduler/sched_fence.c
@@ -90,7 +90,7 @@ static const char *drm_sched_fence_get_timeline_name(struct dma_fence *f)
*
* Free up the fence memory after the RCU grace period.
*/
-static void drm_sched_fence_free(struct rcu_head *rcu)
+void drm_sched_fence_free(struct rcu_head *rcu)
{
struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
struct drm_sched_fence *fence = to_drm_sched_fence(f);
@@ -152,27 +152,32 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
}
EXPORT_SYMBOL(to_drm_sched_fence);
-struct drm_sched_fence *drm_sched_fence_create(struct drm_sched_entity *entity,
- void *owner)
+struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
+ void *owner)
{
struct drm_sched_fence *fence = NULL;
- unsigned seq;
fence = kmem_cache_zalloc(sched_fence_slab, GFP_KERNEL);
if (fence == NULL)
return NULL;
fence->owner = owner;
- fence->sched = entity->rq->sched;
spin_lock_init(&fence->lock);
+ return fence;
+}
+
+void drm_sched_fence_init(struct drm_sched_fence *fence,
+ struct drm_sched_entity *entity)
+{
+ unsigned seq;
+
+ fence->sched = entity->rq->sched;
seq = atomic_inc_return(&entity->fence_seq);
dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
&fence->lock, entity->fence_context, seq);
dma_fence_init(&fence->finished, &drm_sched_fence_ops_finished,
&fence->lock, entity->fence_context + 1, seq);
-
- return fence;
}
module_init(drm_sched_fence_slab_init);
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index 33c414d55fab..454cb6164bdc 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -48,9 +48,11 @@
#include <linux/wait.h>
#include <linux/sched.h>
#include <linux/completion.h>
+#include <linux/dma-resv.h>
#include <uapi/linux/sched/types.h>
#include <drm/drm_print.h>
+#include <drm/drm_gem.h>
#include <drm/gpu_scheduler.h>
#include <drm/spsc_queue.h>
@@ -569,7 +571,6 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
/**
* drm_sched_job_init - init a scheduler job
- *
* @job: scheduler job to init
* @entity: scheduler entity to use
* @owner: job owner for debugging
@@ -577,27 +578,28 @@ EXPORT_SYMBOL(drm_sched_resubmit_jobs_ext);
* Refer to drm_sched_entity_push_job() documentation
* for locking considerations.
*
+ * Drivers must make sure drm_sched_job_cleanup() if this function returns
+ * successfully, even when @job is aborted before drm_sched_job_arm() is called.
+ *
+ * WARNING: amdgpu abuses &drm_sched.ready to signal when the hardware
+ * has died, which can mean that there's no valid runqueue for a @entity.
+ * This function returns -ENOENT in this case (which probably should be -EIO as
+ * a more meanigful return value).
+ *
* Returns 0 for success, negative error code otherwise.
*/
int drm_sched_job_init(struct drm_sched_job *job,
struct drm_sched_entity *entity,
void *owner)
{
- struct drm_gpu_scheduler *sched;
-
drm_sched_entity_select_rq(entity);
if (!entity->rq)
return -ENOENT;
- sched = entity->rq->sched;
-
- job->sched = sched;
job->entity = entity;
- job->s_priority = entity->rq - sched->sched_rq;
- job->s_fence = drm_sched_fence_create(entity, owner);
+ job->s_fence = drm_sched_fence_alloc(entity, owner);
if (!job->s_fence)
return -ENOMEM;
- job->id = atomic64_inc_return(&sched->job_id_count);
INIT_LIST_HEAD(&job->list);
@@ -606,13 +608,58 @@ int drm_sched_job_init(struct drm_sched_job *job,
EXPORT_SYMBOL(drm_sched_job_init);
/**
- * drm_sched_job_cleanup - clean up scheduler job resources
+ * drm_sched_job_arm - arm a scheduler job for execution
+ * @job: scheduler job to arm
+ *
+ * This arms a scheduler job for execution. Specifically it initializes the
+ * &drm_sched_job.s_fence of @job, so that it can be attached to struct dma_resv
+ * or other places that need to track the completion of this job.
+ *
+ * Refer to drm_sched_entity_push_job() documentation for locking
+ * considerations.
*
+ * This can only be called if drm_sched_job_init() succeeded.
+ */
+void drm_sched_job_arm(struct drm_sched_job *job)
+{
+ struct drm_gpu_scheduler *sched;
+ struct drm_sched_entity *entity = job->entity;
+
+ BUG_ON(!entity);
+
+ sched = entity->rq->sched;
+
+ job->sched = sched;
+ job->s_priority = entity->rq - sched->sched_rq;
+ job->id = atomic64_inc_return(&sched->job_id_count);
+
+ drm_sched_fence_init(job->s_fence, job->entity);
+}
+EXPORT_SYMBOL(drm_sched_job_arm);
+
+/**
+ * drm_sched_job_cleanup - clean up scheduler job resources
* @job: scheduler job to clean up
+ *
+ * Cleans up the resources allocated with drm_sched_job_init().
+ *
+ * Drivers should call this from their error unwind code if @job is aborted
+ * before drm_sched_job_arm() is called.
+ *
+ * After that point of no return @job is committed to be executed by the
+ * scheduler, and this function should be called from the
+ * &drm_sched_backend_ops.free_job callback.
*/
void drm_sched_job_cleanup(struct drm_sched_job *job)
{
- dma_fence_put(&job->s_fence->finished);
+ if (kref_read(&job->s_fence->finished.refcount)) {
+ /* drm_sched_job_arm() has been called */
+ dma_fence_put(&job->s_fence->finished);
+ } else {
+ /* aborted job before committing to run it */
+ drm_sched_fence_free(&job->s_fence->finished.rcu);
+ }
+
job->s_fence = NULL;
}
EXPORT_SYMBOL(drm_sched_job_cleanup);
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 5689da118197..2e808097b4d1 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -480,6 +480,8 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
if (ret)
return ret;
+ drm_sched_job_arm(&job->base);
+
job->done_fence = dma_fence_get(&job->base.s_fence->finished);
/* put by scheduler job completion */
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index 88ae7f331bb1..83afc3aa8e2f 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -348,6 +348,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched);
int drm_sched_job_init(struct drm_sched_job *job,
struct drm_sched_entity *entity,
void *owner);
+void drm_sched_job_arm(struct drm_sched_job *job);
void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
struct drm_gpu_scheduler **sched_list,
unsigned int num_sched_list);
@@ -387,8 +388,12 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
enum drm_sched_priority priority);
bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
-struct drm_sched_fence *drm_sched_fence_create(
+struct drm_sched_fence *drm_sched_fence_alloc(
struct drm_sched_entity *s_entity, void *owner);
+void drm_sched_fence_init(struct drm_sched_fence *fence,
+ struct drm_sched_entity *entity);
+void drm_sched_fence_free(struct rcu_head *rcu);
+
void drm_sched_fence_scheduled(struct drm_sched_fence *fence);
void drm_sched_fence_finished(struct drm_sched_fence *fence);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Rob Clark, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, linux-media, linaro-mm-sig, Daniel Vetter
Originally drm_sched_job_init was the point of no return, after which
drivers must submit a job. I've split that up, which allows us to fix
this issue pretty easily.
Only thing we have to take care of is to not skip to error paths after
that. Other drivers do this the same for out-fence and similar things.
Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
Cc: Rob Clark <robdclark@chromium.org>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: freedreno@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
1 file changed, 7 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 6d6c44f0e1f3..d0ed4ddc509e 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
- /* FIXME: this is way too early */
- drm_sched_job_arm(&job->base);
-
xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
kref_init(&submit->ref);
@@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
+ /* point of no return, we _have_ to submit no matter what */
+ drm_sched_job_arm(&submit->base);
+
/*
* Allocate an id which can be used by WAIT_FENCE ioctl to map back
* to the underlying fence.
@@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
if (submit->fence_id < 0) {
ret = submit->fence_id = 0;
submit->fence_id = 0;
- goto out;
}
- if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
+ if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
struct sync_file *sync_file = sync_file_create(submit->user_fence);
if (!sync_file) {
ret = -ENOMEM;
- goto out;
+ } else {
+ fd_install(out_fence_fd, sync_file->file);
+ args->fence_fd = out_fence_fd;
}
- fd_install(out_fence_fd, sync_file->file);
- args->fence_fd = out_fence_fd;
}
submit_attach_object_fences(submit);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Rob Clark, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, linux-media, linaro-mm-sig, Daniel Vetter
Originally drm_sched_job_init was the point of no return, after which
drivers must submit a job. I've split that up, which allows us to fix
this issue pretty easily.
Only thing we have to take care of is to not skip to error paths after
that. Other drivers do this the same for out-fence and similar things.
Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
Cc: Rob Clark <robdclark@chromium.org>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: freedreno@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
1 file changed, 7 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 6d6c44f0e1f3..d0ed4ddc509e 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
- /* FIXME: this is way too early */
- drm_sched_job_arm(&job->base);
-
xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
kref_init(&submit->ref);
@@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
+ /* point of no return, we _have_ to submit no matter what */
+ drm_sched_job_arm(&submit->base);
+
/*
* Allocate an id which can be used by WAIT_FENCE ioctl to map back
* to the underlying fence.
@@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
if (submit->fence_id < 0) {
ret = submit->fence_id = 0;
submit->fence_id = 0;
- goto out;
}
- if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
+ if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
struct sync_file *sync_file = sync_file_create(submit->user_fence);
if (!sync_file) {
ret = -ENOMEM;
- goto out;
+ } else {
+ fd_install(out_fence_fd, sync_file->file);
+ args->fence_fd = out_fence_fd;
}
- fd_install(out_fence_fd, sync_file->file);
- args->fence_fd = out_fence_fd;
}
submit_attach_object_fences(submit);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 23:02 ` Rob Clark
-1 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-05 23:02 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> Originally drm_sched_job_init was the point of no return, after which
> drivers must submit a job. I've split that up, which allows us to fix
> this issue pretty easily.
>
> Only thing we have to take care of is to not skip to error paths after
> that. Other drivers do this the same for out-fence and similar things.
>
> Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> Cc: Rob Clark <robdclark@chromium.org>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: dri-devel@lists.freedesktop.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> ---
> drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> 1 file changed, 7 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index 6d6c44f0e1f3..d0ed4ddc509e 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> - /* FIXME: this is way too early */
> - drm_sched_job_arm(&job->base);
> -
> xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
>
> kref_init(&submit->ref);
> @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
>
> submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
>
> + /* point of no return, we _have_ to submit no matter what */
> + drm_sched_job_arm(&submit->base);
> +
> /*
> * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> * to the underlying fence.
> @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> if (submit->fence_id < 0) {
> ret = submit->fence_id = 0;
> submit->fence_id = 0;
> - goto out;
> }
>
> - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> struct sync_file *sync_file = sync_file_create(submit->user_fence);
> if (!sync_file) {
> ret = -ENOMEM;
> - goto out;
> + } else {
> + fd_install(out_fence_fd, sync_file->file);
> + args->fence_fd = out_fence_fd;
> }
> - fd_install(out_fence_fd, sync_file->file);
> - args->fence_fd = out_fence_fd;
I wonder if instead we should (approximately) undo "drm/msm/submit:
Simplify out-fence-fd handling" so that the point that it could fail
is moved up ahead of the drm_sched_job_arm()?
Also, does the dma_fence_get() work before drm_sched_job_arm()? From
a quick look, it looks like it won't, but I'm still playing catchup
and haven't had a chance to look at your entire series. If it doesn't
work before drm_sched_job_arm(), then there is really no way to
prevent a error path between the fence-init and job-submit.
But, prior to your series, wouldn't a failure after
drm_sched_job_init() but before the job is submitted just burn a
fence-id, and otherwise carry on it's merry way?
BR,
-R
> }
>
> submit_attach_object_fences(submit);
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
@ 2021-08-05 23:02 ` Rob Clark
0 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-05 23:02 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> Originally drm_sched_job_init was the point of no return, after which
> drivers must submit a job. I've split that up, which allows us to fix
> this issue pretty easily.
>
> Only thing we have to take care of is to not skip to error paths after
> that. Other drivers do this the same for out-fence and similar things.
>
> Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> Cc: Rob Clark <robdclark@chromium.org>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: dri-devel@lists.freedesktop.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> ---
> drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> 1 file changed, 7 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index 6d6c44f0e1f3..d0ed4ddc509e 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> - /* FIXME: this is way too early */
> - drm_sched_job_arm(&job->base);
> -
> xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
>
> kref_init(&submit->ref);
> @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
>
> submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
>
> + /* point of no return, we _have_ to submit no matter what */
> + drm_sched_job_arm(&submit->base);
> +
> /*
> * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> * to the underlying fence.
> @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> if (submit->fence_id < 0) {
> ret = submit->fence_id = 0;
> submit->fence_id = 0;
> - goto out;
> }
>
> - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> struct sync_file *sync_file = sync_file_create(submit->user_fence);
> if (!sync_file) {
> ret = -ENOMEM;
> - goto out;
> + } else {
> + fd_install(out_fence_fd, sync_file->file);
> + args->fence_fd = out_fence_fd;
> }
> - fd_install(out_fence_fd, sync_file->file);
> - args->fence_fd = out_fence_fd;
I wonder if instead we should (approximately) undo "drm/msm/submit:
Simplify out-fence-fd handling" so that the point that it could fail
is moved up ahead of the drm_sched_job_arm()?
Also, does the dma_fence_get() work before drm_sched_job_arm()? From
a quick look, it looks like it won't, but I'm still playing catchup
and haven't had a chance to look at your entire series. If it doesn't
work before drm_sched_job_arm(), then there is really no way to
prevent a error path between the fence-init and job-submit.
But, prior to your series, wouldn't a failure after
drm_sched_job_init() but before the job is submitted just burn a
fence-id, and otherwise carry on it's merry way?
BR,
-R
> }
>
> submit_attach_object_fences(submit);
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
2021-08-05 23:02 ` [Intel-gfx] " Rob Clark
@ 2021-08-06 16:41 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-06 16:41 UTC (permalink / raw)
To: Rob Clark
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
>
> On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > Originally drm_sched_job_init was the point of no return, after which
> > drivers must submit a job. I've split that up, which allows us to fix
> > this issue pretty easily.
> >
> > Only thing we have to take care of is to not skip to error paths after
> > that. Other drivers do this the same for out-fence and similar things.
> >
> > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > Cc: Rob Clark <robdclark@chromium.org>
> > Cc: Rob Clark <robdclark@gmail.com>
> > Cc: Sean Paul <sean@poorly.run>
> > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: linux-arm-msm@vger.kernel.org
> > Cc: dri-devel@lists.freedesktop.org
> > Cc: freedreno@lists.freedesktop.org
> > Cc: linux-media@vger.kernel.org
> > Cc: linaro-mm-sig@lists.linaro.org
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > ---
> > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > 1 file changed, 7 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > return ERR_PTR(ret);
> > }
> >
> > - /* FIXME: this is way too early */
> > - drm_sched_job_arm(&job->base);
> > -
> > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> >
> > kref_init(&submit->ref);
> > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> >
> > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> >
> > + /* point of no return, we _have_ to submit no matter what */
> > + drm_sched_job_arm(&submit->base);
> > +
> > /*
> > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > * to the underlying fence.
> > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > if (submit->fence_id < 0) {
> > ret = submit->fence_id = 0;
> > submit->fence_id = 0;
> > - goto out;
> > }
> >
> > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > if (!sync_file) {
> > ret = -ENOMEM;
> > - goto out;
> > + } else {
> > + fd_install(out_fence_fd, sync_file->file);
> > + args->fence_fd = out_fence_fd;
> > }
> > - fd_install(out_fence_fd, sync_file->file);
> > - args->fence_fd = out_fence_fd;
>
> I wonder if instead we should (approximately) undo "drm/msm/submit:
> Simplify out-fence-fd handling" so that the point that it could fail
> is moved up ahead of the drm_sched_job_arm()?
Hm yeah. Up to you how you want to paint this shed, I think either is fine.
> Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> a quick look, it looks like it won't, but I'm still playing catchup
> and haven't had a chance to look at your entire series. If it doesn't
> work before drm_sched_job_arm(), then there is really no way to
> prevent a error path between the fence-init and job-submit.
Yes. I thought I've checked that I put the _arm() in the right spot,
but I guess I screwed up and you need the fence before the point where
I've put the job_arm()? And yes the error path cannot be avoided for
out-fences, that's what I tried to explain in the commit message.
> But, prior to your series, wouldn't a failure after
> drm_sched_job_init() but before the job is submitted just burn a
> fence-id, and otherwise carry on it's merry way?
Maybe? I'm not sure whether the scheduler gets confused about the gap
and freak out abou that. I'm fairly new to that code and learning
(which is part why I'm working on it). Since you look up in
fences/syncobj after job_init() it should be pretty easy to whip up a
testcase and see what happens. Also as long as nothing fails you won't
see an issue, that's for sure.
-Daniel
> BR,
> -R
>
> > }
> >
> > submit_attach_object_fences(submit);
> > --
> > 2.32.0
> >
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
@ 2021-08-06 16:41 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-06 16:41 UTC (permalink / raw)
To: Rob Clark
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
>
> On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > Originally drm_sched_job_init was the point of no return, after which
> > drivers must submit a job. I've split that up, which allows us to fix
> > this issue pretty easily.
> >
> > Only thing we have to take care of is to not skip to error paths after
> > that. Other drivers do this the same for out-fence and similar things.
> >
> > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > Cc: Rob Clark <robdclark@chromium.org>
> > Cc: Rob Clark <robdclark@gmail.com>
> > Cc: Sean Paul <sean@poorly.run>
> > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: linux-arm-msm@vger.kernel.org
> > Cc: dri-devel@lists.freedesktop.org
> > Cc: freedreno@lists.freedesktop.org
> > Cc: linux-media@vger.kernel.org
> > Cc: linaro-mm-sig@lists.linaro.org
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > ---
> > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > 1 file changed, 7 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > return ERR_PTR(ret);
> > }
> >
> > - /* FIXME: this is way too early */
> > - drm_sched_job_arm(&job->base);
> > -
> > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> >
> > kref_init(&submit->ref);
> > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> >
> > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> >
> > + /* point of no return, we _have_ to submit no matter what */
> > + drm_sched_job_arm(&submit->base);
> > +
> > /*
> > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > * to the underlying fence.
> > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > if (submit->fence_id < 0) {
> > ret = submit->fence_id = 0;
> > submit->fence_id = 0;
> > - goto out;
> > }
> >
> > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > if (!sync_file) {
> > ret = -ENOMEM;
> > - goto out;
> > + } else {
> > + fd_install(out_fence_fd, sync_file->file);
> > + args->fence_fd = out_fence_fd;
> > }
> > - fd_install(out_fence_fd, sync_file->file);
> > - args->fence_fd = out_fence_fd;
>
> I wonder if instead we should (approximately) undo "drm/msm/submit:
> Simplify out-fence-fd handling" so that the point that it could fail
> is moved up ahead of the drm_sched_job_arm()?
Hm yeah. Up to you how you want to paint this shed, I think either is fine.
> Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> a quick look, it looks like it won't, but I'm still playing catchup
> and haven't had a chance to look at your entire series. If it doesn't
> work before drm_sched_job_arm(), then there is really no way to
> prevent a error path between the fence-init and job-submit.
Yes. I thought I've checked that I put the _arm() in the right spot,
but I guess I screwed up and you need the fence before the point where
I've put the job_arm()? And yes the error path cannot be avoided for
out-fences, that's what I tried to explain in the commit message.
> But, prior to your series, wouldn't a failure after
> drm_sched_job_init() but before the job is submitted just burn a
> fence-id, and otherwise carry on it's merry way?
Maybe? I'm not sure whether the scheduler gets confused about the gap
and freak out abou that. I'm fairly new to that code and learning
(which is part why I'm working on it). Since you look up in
fences/syncobj after job_init() it should be pretty easy to whip up a
testcase and see what happens. Also as long as nothing fails you won't
see an issue, that's for sure.
-Daniel
> BR,
> -R
>
> > }
> >
> > submit_attach_object_fences(submit);
> > --
> > 2.32.0
> >
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
2021-08-06 16:41 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-06 17:19 ` Rob Clark
-1 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-06 17:19 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 9:42 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
> >
> > On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > >
> > > Originally drm_sched_job_init was the point of no return, after which
> > > drivers must submit a job. I've split that up, which allows us to fix
> > > this issue pretty easily.
> > >
> > > Only thing we have to take care of is to not skip to error paths after
> > > that. Other drivers do this the same for out-fence and similar things.
> > >
> > > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > > Cc: Rob Clark <robdclark@chromium.org>
> > > Cc: Rob Clark <robdclark@gmail.com>
> > > Cc: Sean Paul <sean@poorly.run>
> > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > Cc: "Christian König" <christian.koenig@amd.com>
> > > Cc: linux-arm-msm@vger.kernel.org
> > > Cc: dri-devel@lists.freedesktop.org
> > > Cc: freedreno@lists.freedesktop.org
> > > Cc: linux-media@vger.kernel.org
> > > Cc: linaro-mm-sig@lists.linaro.org
> > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > > ---
> > > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > > 1 file changed, 7 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > > return ERR_PTR(ret);
> > > }
> > >
> > > - /* FIXME: this is way too early */
> > > - drm_sched_job_arm(&job->base);
> > > -
> > > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> > >
> > > kref_init(&submit->ref);
> > > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > >
> > > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> > >
> > > + /* point of no return, we _have_ to submit no matter what */
> > > + drm_sched_job_arm(&submit->base);
> > > +
> > > /*
> > > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > > * to the underlying fence.
> > > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > if (submit->fence_id < 0) {
> > > ret = submit->fence_id = 0;
> > > submit->fence_id = 0;
> > > - goto out;
> > > }
> > >
> > > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > > if (!sync_file) {
> > > ret = -ENOMEM;
> > > - goto out;
> > > + } else {
> > > + fd_install(out_fence_fd, sync_file->file);
> > > + args->fence_fd = out_fence_fd;
> > > }
> > > - fd_install(out_fence_fd, sync_file->file);
> > > - args->fence_fd = out_fence_fd;
> >
> > I wonder if instead we should (approximately) undo "drm/msm/submit:
> > Simplify out-fence-fd handling" so that the point that it could fail
> > is moved up ahead of the drm_sched_job_arm()?
>
> Hm yeah. Up to you how you want to paint this shed, I think either is fine.
>
> > Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> > a quick look, it looks like it won't, but I'm still playing catchup
> > and haven't had a chance to look at your entire series. If it doesn't
> > work before drm_sched_job_arm(), then there is really no way to
> > prevent a error path between the fence-init and job-submit.
>
> Yes. I thought I've checked that I put the _arm() in the right spot,
> but I guess I screwed up and you need the fence before the point where
> I've put the job_arm()? And yes the error path cannot be avoided for
> out-fences, that's what I tried to explain in the commit message.
>
> > But, prior to your series, wouldn't a failure after
> > drm_sched_job_init() but before the job is submitted just burn a
> > fence-id, and otherwise carry on it's merry way?
>
> Maybe? I'm not sure whether the scheduler gets confused about the gap
> and freak out abou that. I'm fairly new to that code and learning
> (which is part why I'm working on it). Since you look up in
> fences/syncobj after job_init() it should be pretty easy to whip up a
> testcase and see what happens. Also as long as nothing fails you won't
> see an issue, that's for sure.
fair.. I'll try to come up with a test case.. pre-scheduler-conversion
it wasn't a problem to fail after the fence seqno was allocated (well,
I guess you might have problems if you had 2^31 failures in a row)
BR,
-R
> -Daniel
>
> > BR,
> > -R
> >
> > > }
> > >
> > > submit_attach_object_fences(submit);
> > > --
> > > 2.32.0
> > >
>
>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
@ 2021-08-06 17:19 ` Rob Clark
0 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-06 17:19 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 9:42 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
> >
> > On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > >
> > > Originally drm_sched_job_init was the point of no return, after which
> > > drivers must submit a job. I've split that up, which allows us to fix
> > > this issue pretty easily.
> > >
> > > Only thing we have to take care of is to not skip to error paths after
> > > that. Other drivers do this the same for out-fence and similar things.
> > >
> > > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > > Cc: Rob Clark <robdclark@chromium.org>
> > > Cc: Rob Clark <robdclark@gmail.com>
> > > Cc: Sean Paul <sean@poorly.run>
> > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > Cc: "Christian König" <christian.koenig@amd.com>
> > > Cc: linux-arm-msm@vger.kernel.org
> > > Cc: dri-devel@lists.freedesktop.org
> > > Cc: freedreno@lists.freedesktop.org
> > > Cc: linux-media@vger.kernel.org
> > > Cc: linaro-mm-sig@lists.linaro.org
> > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > > ---
> > > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > > 1 file changed, 7 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > > return ERR_PTR(ret);
> > > }
> > >
> > > - /* FIXME: this is way too early */
> > > - drm_sched_job_arm(&job->base);
> > > -
> > > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> > >
> > > kref_init(&submit->ref);
> > > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > >
> > > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> > >
> > > + /* point of no return, we _have_ to submit no matter what */
> > > + drm_sched_job_arm(&submit->base);
> > > +
> > > /*
> > > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > > * to the underlying fence.
> > > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > if (submit->fence_id < 0) {
> > > ret = submit->fence_id = 0;
> > > submit->fence_id = 0;
> > > - goto out;
> > > }
> > >
> > > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > > if (!sync_file) {
> > > ret = -ENOMEM;
> > > - goto out;
> > > + } else {
> > > + fd_install(out_fence_fd, sync_file->file);
> > > + args->fence_fd = out_fence_fd;
> > > }
> > > - fd_install(out_fence_fd, sync_file->file);
> > > - args->fence_fd = out_fence_fd;
> >
> > I wonder if instead we should (approximately) undo "drm/msm/submit:
> > Simplify out-fence-fd handling" so that the point that it could fail
> > is moved up ahead of the drm_sched_job_arm()?
>
> Hm yeah. Up to you how you want to paint this shed, I think either is fine.
>
> > Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> > a quick look, it looks like it won't, but I'm still playing catchup
> > and haven't had a chance to look at your entire series. If it doesn't
> > work before drm_sched_job_arm(), then there is really no way to
> > prevent a error path between the fence-init and job-submit.
>
> Yes. I thought I've checked that I put the _arm() in the right spot,
> but I guess I screwed up and you need the fence before the point where
> I've put the job_arm()? And yes the error path cannot be avoided for
> out-fences, that's what I tried to explain in the commit message.
>
> > But, prior to your series, wouldn't a failure after
> > drm_sched_job_init() but before the job is submitted just burn a
> > fence-id, and otherwise carry on it's merry way?
>
> Maybe? I'm not sure whether the scheduler gets confused about the gap
> and freak out abou that. I'm fairly new to that code and learning
> (which is part why I'm working on it). Since you look up in
> fences/syncobj after job_init() it should be pretty easy to whip up a
> testcase and see what happens. Also as long as nothing fails you won't
> see an issue, that's for sure.
fair.. I'll try to come up with a test case.. pre-scheduler-conversion
it wasn't a problem to fail after the fence seqno was allocated (well,
I guess you might have problems if you had 2^31 failures in a row)
BR,
-R
> -Daniel
>
> > BR,
> > -R
> >
> > > }
> > >
> > > submit_attach_object_fences(submit);
> > > --
> > > 2.32.0
> > >
>
>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
2021-08-06 17:19 ` [Intel-gfx] " Rob Clark
@ 2021-08-06 18:41 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-06 18:41 UTC (permalink / raw)
To: Rob Clark
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 7:15 PM Rob Clark <robdclark@gmail.com> wrote:
>
> On Fri, Aug 6, 2021 at 9:42 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
> > >
> > > On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > >
> > > > Originally drm_sched_job_init was the point of no return, after which
> > > > drivers must submit a job. I've split that up, which allows us to fix
> > > > this issue pretty easily.
> > > >
> > > > Only thing we have to take care of is to not skip to error paths after
> > > > that. Other drivers do this the same for out-fence and similar things.
> > > >
> > > > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > > > Cc: Rob Clark <robdclark@chromium.org>
> > > > Cc: Rob Clark <robdclark@gmail.com>
> > > > Cc: Sean Paul <sean@poorly.run>
> > > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > > Cc: "Christian König" <christian.koenig@amd.com>
> > > > Cc: linux-arm-msm@vger.kernel.org
> > > > Cc: dri-devel@lists.freedesktop.org
> > > > Cc: freedreno@lists.freedesktop.org
> > > > Cc: linux-media@vger.kernel.org
> > > > Cc: linaro-mm-sig@lists.linaro.org
> > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > > > ---
> > > > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > > > 1 file changed, 7 insertions(+), 8 deletions(-)
> > > >
> > > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > > > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > > > return ERR_PTR(ret);
> > > > }
> > > >
> > > > - /* FIXME: this is way too early */
> > > > - drm_sched_job_arm(&job->base);
> > > > -
> > > > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> > > >
> > > > kref_init(&submit->ref);
> > > > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > >
> > > > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> > > >
> > > > + /* point of no return, we _have_ to submit no matter what */
> > > > + drm_sched_job_arm(&submit->base);
> > > > +
> > > > /*
> > > > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > > > * to the underlying fence.
> > > > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > if (submit->fence_id < 0) {
> > > > ret = submit->fence_id = 0;
> > > > submit->fence_id = 0;
> > > > - goto out;
> > > > }
> > > >
> > > > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > > > if (!sync_file) {
> > > > ret = -ENOMEM;
> > > > - goto out;
> > > > + } else {
> > > > + fd_install(out_fence_fd, sync_file->file);
> > > > + args->fence_fd = out_fence_fd;
> > > > }
> > > > - fd_install(out_fence_fd, sync_file->file);
> > > > - args->fence_fd = out_fence_fd;
> > >
> > > I wonder if instead we should (approximately) undo "drm/msm/submit:
> > > Simplify out-fence-fd handling" so that the point that it could fail
> > > is moved up ahead of the drm_sched_job_arm()?
> >
> > Hm yeah. Up to you how you want to paint this shed, I think either is fine.
> >
> > > Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> > > a quick look, it looks like it won't, but I'm still playing catchup
> > > and haven't had a chance to look at your entire series. If it doesn't
> > > work before drm_sched_job_arm(), then there is really no way to
> > > prevent a error path between the fence-init and job-submit.
> >
> > Yes. I thought I've checked that I put the _arm() in the right spot,
> > but I guess I screwed up and you need the fence before the point where
> > I've put the job_arm()? And yes the error path cannot be avoided for
> > out-fences, that's what I tried to explain in the commit message.
> >
> > > But, prior to your series, wouldn't a failure after
> > > drm_sched_job_init() but before the job is submitted just burn a
> > > fence-id, and otherwise carry on it's merry way?
> >
> > Maybe? I'm not sure whether the scheduler gets confused about the gap
> > and freak out abou that. I'm fairly new to that code and learning
> > (which is part why I'm working on it). Since you look up in
> > fences/syncobj after job_init() it should be pretty easy to whip up a
> > testcase and see what happens. Also as long as nothing fails you won't
> > see an issue, that's for sure.
>
> fair.. I'll try to come up with a test case.. pre-scheduler-conversion
> it wasn't a problem to fail after the fence seqno was allocated (well,
> I guess you might have problems if you had 2^31 failures in a row)
Yeah one thing drm/sched forces you to do is have a very clear notion
about the point of no return in your submit ioctl. Which I think is a
Very Good Thing, at least looking at i915 execbuf where the point of
no return is a multi-stage thing with such interesting intermediate
points like "we submit the ruquest but without actually running the
batchbuffer". The downside is that the submit ioctl isn't perfectly
transaction anymore, but I don't think that matters for tha tail
stuff, which is generally just some out-fence installing. That
generally never fails.
-Daniel
>
> BR,
> -R
>
> > -Daniel
> >
> > > BR,
> > > -R
> > >
> > > > }
> > > >
> > > > submit_attach_object_fences(submit);
> > > > --
> > > > 2.32.0
> > > >
> >
> >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > http://blog.ffwll.ch
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
@ 2021-08-06 18:41 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-06 18:41 UTC (permalink / raw)
To: Rob Clark
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 7:15 PM Rob Clark <robdclark@gmail.com> wrote:
>
> On Fri, Aug 6, 2021 at 9:42 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
> > >
> > > On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > >
> > > > Originally drm_sched_job_init was the point of no return, after which
> > > > drivers must submit a job. I've split that up, which allows us to fix
> > > > this issue pretty easily.
> > > >
> > > > Only thing we have to take care of is to not skip to error paths after
> > > > that. Other drivers do this the same for out-fence and similar things.
> > > >
> > > > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > > > Cc: Rob Clark <robdclark@chromium.org>
> > > > Cc: Rob Clark <robdclark@gmail.com>
> > > > Cc: Sean Paul <sean@poorly.run>
> > > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > > Cc: "Christian König" <christian.koenig@amd.com>
> > > > Cc: linux-arm-msm@vger.kernel.org
> > > > Cc: dri-devel@lists.freedesktop.org
> > > > Cc: freedreno@lists.freedesktop.org
> > > > Cc: linux-media@vger.kernel.org
> > > > Cc: linaro-mm-sig@lists.linaro.org
> > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > > > ---
> > > > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > > > 1 file changed, 7 insertions(+), 8 deletions(-)
> > > >
> > > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > > > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > > > return ERR_PTR(ret);
> > > > }
> > > >
> > > > - /* FIXME: this is way too early */
> > > > - drm_sched_job_arm(&job->base);
> > > > -
> > > > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> > > >
> > > > kref_init(&submit->ref);
> > > > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > >
> > > > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> > > >
> > > > + /* point of no return, we _have_ to submit no matter what */
> > > > + drm_sched_job_arm(&submit->base);
> > > > +
> > > > /*
> > > > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > > > * to the underlying fence.
> > > > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > if (submit->fence_id < 0) {
> > > > ret = submit->fence_id = 0;
> > > > submit->fence_id = 0;
> > > > - goto out;
> > > > }
> > > >
> > > > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > > > if (!sync_file) {
> > > > ret = -ENOMEM;
> > > > - goto out;
> > > > + } else {
> > > > + fd_install(out_fence_fd, sync_file->file);
> > > > + args->fence_fd = out_fence_fd;
> > > > }
> > > > - fd_install(out_fence_fd, sync_file->file);
> > > > - args->fence_fd = out_fence_fd;
> > >
> > > I wonder if instead we should (approximately) undo "drm/msm/submit:
> > > Simplify out-fence-fd handling" so that the point that it could fail
> > > is moved up ahead of the drm_sched_job_arm()?
> >
> > Hm yeah. Up to you how you want to paint this shed, I think either is fine.
> >
> > > Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> > > a quick look, it looks like it won't, but I'm still playing catchup
> > > and haven't had a chance to look at your entire series. If it doesn't
> > > work before drm_sched_job_arm(), then there is really no way to
> > > prevent a error path between the fence-init and job-submit.
> >
> > Yes. I thought I've checked that I put the _arm() in the right spot,
> > but I guess I screwed up and you need the fence before the point where
> > I've put the job_arm()? And yes the error path cannot be avoided for
> > out-fences, that's what I tried to explain in the commit message.
> >
> > > But, prior to your series, wouldn't a failure after
> > > drm_sched_job_init() but before the job is submitted just burn a
> > > fence-id, and otherwise carry on it's merry way?
> >
> > Maybe? I'm not sure whether the scheduler gets confused about the gap
> > and freak out abou that. I'm fairly new to that code and learning
> > (which is part why I'm working on it). Since you look up in
> > fences/syncobj after job_init() it should be pretty easy to whip up a
> > testcase and see what happens. Also as long as nothing fails you won't
> > see an issue, that's for sure.
>
> fair.. I'll try to come up with a test case.. pre-scheduler-conversion
> it wasn't a problem to fail after the fence seqno was allocated (well,
> I guess you might have problems if you had 2^31 failures in a row)
Yeah one thing drm/sched forces you to do is have a very clear notion
about the point of no return in your submit ioctl. Which I think is a
Very Good Thing, at least looking at i915 execbuf where the point of
no return is a multi-stage thing with such interesting intermediate
points like "we submit the ruquest but without actually running the
batchbuffer". The downside is that the submit ioctl isn't perfectly
transaction anymore, but I don't think that matters for tha tail
stuff, which is generally just some out-fence installing. That
generally never fails.
-Daniel
>
> BR,
> -R
>
> > -Daniel
> >
> > > BR,
> > > -R
> > >
> > > > }
> > > >
> > > > submit_attach_object_fences(submit);
> > > > --
> > > > 2.32.0
> > > >
> >
> >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > http://blog.ffwll.ch
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
2021-08-06 18:41 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-06 19:01 ` Rob Clark
-1 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-06 19:01 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 11:41 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> On Fri, Aug 6, 2021 at 7:15 PM Rob Clark <robdclark@gmail.com> wrote:
> >
> > On Fri, Aug 6, 2021 at 9:42 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > >
> > > On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
> > > >
> > > > On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > > >
> > > > > Originally drm_sched_job_init was the point of no return, after which
> > > > > drivers must submit a job. I've split that up, which allows us to fix
> > > > > this issue pretty easily.
> > > > >
> > > > > Only thing we have to take care of is to not skip to error paths after
> > > > > that. Other drivers do this the same for out-fence and similar things.
> > > > >
> > > > > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > > > > Cc: Rob Clark <robdclark@chromium.org>
> > > > > Cc: Rob Clark <robdclark@gmail.com>
> > > > > Cc: Sean Paul <sean@poorly.run>
> > > > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > > > Cc: "Christian König" <christian.koenig@amd.com>
> > > > > Cc: linux-arm-msm@vger.kernel.org
> > > > > Cc: dri-devel@lists.freedesktop.org
> > > > > Cc: freedreno@lists.freedesktop.org
> > > > > Cc: linux-media@vger.kernel.org
> > > > > Cc: linaro-mm-sig@lists.linaro.org
> > > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > > > > ---
> > > > > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > > > > 1 file changed, 7 insertions(+), 8 deletions(-)
> > > > >
> > > > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > > > > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > > > > return ERR_PTR(ret);
> > > > > }
> > > > >
> > > > > - /* FIXME: this is way too early */
> > > > > - drm_sched_job_arm(&job->base);
> > > > > -
> > > > > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> > > > >
> > > > > kref_init(&submit->ref);
> > > > > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > >
> > > > > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> > > > >
> > > > > + /* point of no return, we _have_ to submit no matter what */
> > > > > + drm_sched_job_arm(&submit->base);
> > > > > +
> > > > > /*
> > > > > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > > > > * to the underlying fence.
> > > > > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > > if (submit->fence_id < 0) {
> > > > > ret = submit->fence_id = 0;
> > > > > submit->fence_id = 0;
> > > > > - goto out;
> > > > > }
> > > > >
> > > > > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > > > > if (!sync_file) {
> > > > > ret = -ENOMEM;
> > > > > - goto out;
> > > > > + } else {
> > > > > + fd_install(out_fence_fd, sync_file->file);
> > > > > + args->fence_fd = out_fence_fd;
> > > > > }
> > > > > - fd_install(out_fence_fd, sync_file->file);
> > > > > - args->fence_fd = out_fence_fd;
> > > >
> > > > I wonder if instead we should (approximately) undo "drm/msm/submit:
> > > > Simplify out-fence-fd handling" so that the point that it could fail
> > > > is moved up ahead of the drm_sched_job_arm()?
> > >
> > > Hm yeah. Up to you how you want to paint this shed, I think either is fine.
> > >
> > > > Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> > > > a quick look, it looks like it won't, but I'm still playing catchup
> > > > and haven't had a chance to look at your entire series. If it doesn't
> > > > work before drm_sched_job_arm(), then there is really no way to
> > > > prevent a error path between the fence-init and job-submit.
> > >
> > > Yes. I thought I've checked that I put the _arm() in the right spot,
> > > but I guess I screwed up and you need the fence before the point where
> > > I've put the job_arm()? And yes the error path cannot be avoided for
> > > out-fences, that's what I tried to explain in the commit message.
> > >
> > > > But, prior to your series, wouldn't a failure after
> > > > drm_sched_job_init() but before the job is submitted just burn a
> > > > fence-id, and otherwise carry on it's merry way?
> > >
> > > Maybe? I'm not sure whether the scheduler gets confused about the gap
> > > and freak out abou that. I'm fairly new to that code and learning
> > > (which is part why I'm working on it). Since you look up in
> > > fences/syncobj after job_init() it should be pretty easy to whip up a
> > > testcase and see what happens. Also as long as nothing fails you won't
> > > see an issue, that's for sure.
> >
> > fair.. I'll try to come up with a test case.. pre-scheduler-conversion
> > it wasn't a problem to fail after the fence seqno was allocated (well,
> > I guess you might have problems if you had 2^31 failures in a row)
>
> Yeah one thing drm/sched forces you to do is have a very clear notion
> about the point of no return in your submit ioctl. Which I think is a
> Very Good Thing, at least looking at i915 execbuf where the point of
> no return is a multi-stage thing with such interesting intermediate
> points like "we submit the ruquest but without actually running the
> batchbuffer". The downside is that the submit ioctl isn't perfectly
> transaction anymore, but I don't think that matters for tha tail
> stuff, which is generally just some out-fence installing. That
> generally never fails.
So I hacked up:
------
diff --git a/drivers/gpu/drm/scheduler/sched_fence.c
b/drivers/gpu/drm/scheduler/sched_fence.c
index 3aa6351d2101..88e66dbc9515 100644
--- a/drivers/gpu/drm/scheduler/sched_fence.c
+++ b/drivers/gpu/drm/scheduler/sched_fence.c
@@ -176,6 +176,7 @@ struct drm_sched_fence
*drm_sched_fence_create(struct drm_sched_entity *entity,
fence->sched = entity->rq->sched;
spin_lock_init(&fence->lock);
+ seq = atomic_inc_return(&entity->fence_seq);
seq = atomic_inc_return(&entity->fence_seq);
dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
&fence->lock, entity->fence_context, seq);
diff --git a/drivers/gpu/drm/scheduler/sched_main.c
b/drivers/gpu/drm/scheduler/sched_main.c
index fcc601962e92..583e85adbbe0 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -593,6 +593,7 @@ int drm_sched_job_init(struct drm_sched_job *job,
if (!job->s_fence)
return -ENOMEM;
job->id = atomic64_inc_return(&sched->job_id_count);
+ job->id = atomic64_inc_return(&sched->job_id_count);
INIT_LIST_HEAD(&job->list);
------
(I guess the job->id part shouldn't really be needed, that looks like
it is only used by amdgpu)
This didn't cause any problems that I could see. So I don't *think* a
failure after drm_sched_job_init() is really problematic, as long as
things are serialized between drm_sched_job_init() and
drm_sched_entity_push_job().
I also noticed that in the atomic commit path, the out-fences are
initialized before atomic-check.. so there should be plenty of
precedent for skipping fence seqno's.
BR,
-R
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
@ 2021-08-06 19:01 ` Rob Clark
0 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-06 19:01 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 11:41 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> On Fri, Aug 6, 2021 at 7:15 PM Rob Clark <robdclark@gmail.com> wrote:
> >
> > On Fri, Aug 6, 2021 at 9:42 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > >
> > > On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
> > > >
> > > > On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > > >
> > > > > Originally drm_sched_job_init was the point of no return, after which
> > > > > drivers must submit a job. I've split that up, which allows us to fix
> > > > > this issue pretty easily.
> > > > >
> > > > > Only thing we have to take care of is to not skip to error paths after
> > > > > that. Other drivers do this the same for out-fence and similar things.
> > > > >
> > > > > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > > > > Cc: Rob Clark <robdclark@chromium.org>
> > > > > Cc: Rob Clark <robdclark@gmail.com>
> > > > > Cc: Sean Paul <sean@poorly.run>
> > > > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > > > Cc: "Christian König" <christian.koenig@amd.com>
> > > > > Cc: linux-arm-msm@vger.kernel.org
> > > > > Cc: dri-devel@lists.freedesktop.org
> > > > > Cc: freedreno@lists.freedesktop.org
> > > > > Cc: linux-media@vger.kernel.org
> > > > > Cc: linaro-mm-sig@lists.linaro.org
> > > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > > > > ---
> > > > > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > > > > 1 file changed, 7 insertions(+), 8 deletions(-)
> > > > >
> > > > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > > > > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > > > > return ERR_PTR(ret);
> > > > > }
> > > > >
> > > > > - /* FIXME: this is way too early */
> > > > > - drm_sched_job_arm(&job->base);
> > > > > -
> > > > > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> > > > >
> > > > > kref_init(&submit->ref);
> > > > > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > >
> > > > > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> > > > >
> > > > > + /* point of no return, we _have_ to submit no matter what */
> > > > > + drm_sched_job_arm(&submit->base);
> > > > > +
> > > > > /*
> > > > > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > > > > * to the underlying fence.
> > > > > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > > if (submit->fence_id < 0) {
> > > > > ret = submit->fence_id = 0;
> > > > > submit->fence_id = 0;
> > > > > - goto out;
> > > > > }
> > > > >
> > > > > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > > > > if (!sync_file) {
> > > > > ret = -ENOMEM;
> > > > > - goto out;
> > > > > + } else {
> > > > > + fd_install(out_fence_fd, sync_file->file);
> > > > > + args->fence_fd = out_fence_fd;
> > > > > }
> > > > > - fd_install(out_fence_fd, sync_file->file);
> > > > > - args->fence_fd = out_fence_fd;
> > > >
> > > > I wonder if instead we should (approximately) undo "drm/msm/submit:
> > > > Simplify out-fence-fd handling" so that the point that it could fail
> > > > is moved up ahead of the drm_sched_job_arm()?
> > >
> > > Hm yeah. Up to you how you want to paint this shed, I think either is fine.
> > >
> > > > Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> > > > a quick look, it looks like it won't, but I'm still playing catchup
> > > > and haven't had a chance to look at your entire series. If it doesn't
> > > > work before drm_sched_job_arm(), then there is really no way to
> > > > prevent a error path between the fence-init and job-submit.
> > >
> > > Yes. I thought I've checked that I put the _arm() in the right spot,
> > > but I guess I screwed up and you need the fence before the point where
> > > I've put the job_arm()? And yes the error path cannot be avoided for
> > > out-fences, that's what I tried to explain in the commit message.
> > >
> > > > But, prior to your series, wouldn't a failure after
> > > > drm_sched_job_init() but before the job is submitted just burn a
> > > > fence-id, and otherwise carry on it's merry way?
> > >
> > > Maybe? I'm not sure whether the scheduler gets confused about the gap
> > > and freak out abou that. I'm fairly new to that code and learning
> > > (which is part why I'm working on it). Since you look up in
> > > fences/syncobj after job_init() it should be pretty easy to whip up a
> > > testcase and see what happens. Also as long as nothing fails you won't
> > > see an issue, that's for sure.
> >
> > fair.. I'll try to come up with a test case.. pre-scheduler-conversion
> > it wasn't a problem to fail after the fence seqno was allocated (well,
> > I guess you might have problems if you had 2^31 failures in a row)
>
> Yeah one thing drm/sched forces you to do is have a very clear notion
> about the point of no return in your submit ioctl. Which I think is a
> Very Good Thing, at least looking at i915 execbuf where the point of
> no return is a multi-stage thing with such interesting intermediate
> points like "we submit the ruquest but without actually running the
> batchbuffer". The downside is that the submit ioctl isn't perfectly
> transaction anymore, but I don't think that matters for tha tail
> stuff, which is generally just some out-fence installing. That
> generally never fails.
So I hacked up:
------
diff --git a/drivers/gpu/drm/scheduler/sched_fence.c
b/drivers/gpu/drm/scheduler/sched_fence.c
index 3aa6351d2101..88e66dbc9515 100644
--- a/drivers/gpu/drm/scheduler/sched_fence.c
+++ b/drivers/gpu/drm/scheduler/sched_fence.c
@@ -176,6 +176,7 @@ struct drm_sched_fence
*drm_sched_fence_create(struct drm_sched_entity *entity,
fence->sched = entity->rq->sched;
spin_lock_init(&fence->lock);
+ seq = atomic_inc_return(&entity->fence_seq);
seq = atomic_inc_return(&entity->fence_seq);
dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
&fence->lock, entity->fence_context, seq);
diff --git a/drivers/gpu/drm/scheduler/sched_main.c
b/drivers/gpu/drm/scheduler/sched_main.c
index fcc601962e92..583e85adbbe0 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -593,6 +593,7 @@ int drm_sched_job_init(struct drm_sched_job *job,
if (!job->s_fence)
return -ENOMEM;
job->id = atomic64_inc_return(&sched->job_id_count);
+ job->id = atomic64_inc_return(&sched->job_id_count);
INIT_LIST_HEAD(&job->list);
------
(I guess the job->id part shouldn't really be needed, that looks like
it is only used by amdgpu)
This didn't cause any problems that I could see. So I don't *think* a
failure after drm_sched_job_init() is really problematic, as long as
things are serialized between drm_sched_job_init() and
drm_sched_entity_push_job().
I also noticed that in the atomic commit path, the out-fences are
initialized before atomic-check.. so there should be plenty of
precedent for skipping fence seqno's.
BR,
-R
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
2021-08-06 19:01 ` [Intel-gfx] " Rob Clark
@ 2021-08-06 19:10 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-06 19:10 UTC (permalink / raw)
To: Rob Clark
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 8:57 PM Rob Clark <robdclark@gmail.com> wrote:
>
> On Fri, Aug 6, 2021 at 11:41 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > On Fri, Aug 6, 2021 at 7:15 PM Rob Clark <robdclark@gmail.com> wrote:
> > >
> > > On Fri, Aug 6, 2021 at 9:42 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > >
> > > > On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
> > > > >
> > > > > On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > > > >
> > > > > > Originally drm_sched_job_init was the point of no return, after which
> > > > > > drivers must submit a job. I've split that up, which allows us to fix
> > > > > > this issue pretty easily.
> > > > > >
> > > > > > Only thing we have to take care of is to not skip to error paths after
> > > > > > that. Other drivers do this the same for out-fence and similar things.
> > > > > >
> > > > > > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > > > > > Cc: Rob Clark <robdclark@chromium.org>
> > > > > > Cc: Rob Clark <robdclark@gmail.com>
> > > > > > Cc: Sean Paul <sean@poorly.run>
> > > > > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > > > > Cc: "Christian König" <christian.koenig@amd.com>
> > > > > > Cc: linux-arm-msm@vger.kernel.org
> > > > > > Cc: dri-devel@lists.freedesktop.org
> > > > > > Cc: freedreno@lists.freedesktop.org
> > > > > > Cc: linux-media@vger.kernel.org
> > > > > > Cc: linaro-mm-sig@lists.linaro.org
> > > > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > > > > > ---
> > > > > > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > > > > > 1 file changed, 7 insertions(+), 8 deletions(-)
> > > > > >
> > > > > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > > > > > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > > > > > return ERR_PTR(ret);
> > > > > > }
> > > > > >
> > > > > > - /* FIXME: this is way too early */
> > > > > > - drm_sched_job_arm(&job->base);
> > > > > > -
> > > > > > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> > > > > >
> > > > > > kref_init(&submit->ref);
> > > > > > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > > >
> > > > > > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> > > > > >
> > > > > > + /* point of no return, we _have_ to submit no matter what */
> > > > > > + drm_sched_job_arm(&submit->base);
> > > > > > +
> > > > > > /*
> > > > > > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > > > > > * to the underlying fence.
> > > > > > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > > > if (submit->fence_id < 0) {
> > > > > > ret = submit->fence_id = 0;
> > > > > > submit->fence_id = 0;
> > > > > > - goto out;
> > > > > > }
> > > > > >
> > > > > > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > > > > > if (!sync_file) {
> > > > > > ret = -ENOMEM;
> > > > > > - goto out;
> > > > > > + } else {
> > > > > > + fd_install(out_fence_fd, sync_file->file);
> > > > > > + args->fence_fd = out_fence_fd;
> > > > > > }
> > > > > > - fd_install(out_fence_fd, sync_file->file);
> > > > > > - args->fence_fd = out_fence_fd;
> > > > >
> > > > > I wonder if instead we should (approximately) undo "drm/msm/submit:
> > > > > Simplify out-fence-fd handling" so that the point that it could fail
> > > > > is moved up ahead of the drm_sched_job_arm()?
> > > >
> > > > Hm yeah. Up to you how you want to paint this shed, I think either is fine.
> > > >
> > > > > Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> > > > > a quick look, it looks like it won't, but I'm still playing catchup
> > > > > and haven't had a chance to look at your entire series. If it doesn't
> > > > > work before drm_sched_job_arm(), then there is really no way to
> > > > > prevent a error path between the fence-init and job-submit.
> > > >
> > > > Yes. I thought I've checked that I put the _arm() in the right spot,
> > > > but I guess I screwed up and you need the fence before the point where
> > > > I've put the job_arm()? And yes the error path cannot be avoided for
> > > > out-fences, that's what I tried to explain in the commit message.
> > > >
> > > > > But, prior to your series, wouldn't a failure after
> > > > > drm_sched_job_init() but before the job is submitted just burn a
> > > > > fence-id, and otherwise carry on it's merry way?
> > > >
> > > > Maybe? I'm not sure whether the scheduler gets confused about the gap
> > > > and freak out abou that. I'm fairly new to that code and learning
> > > > (which is part why I'm working on it). Since you look up in
> > > > fences/syncobj after job_init() it should be pretty easy to whip up a
> > > > testcase and see what happens. Also as long as nothing fails you won't
> > > > see an issue, that's for sure.
> > >
> > > fair.. I'll try to come up with a test case.. pre-scheduler-conversion
> > > it wasn't a problem to fail after the fence seqno was allocated (well,
> > > I guess you might have problems if you had 2^31 failures in a row)
> >
> > Yeah one thing drm/sched forces you to do is have a very clear notion
> > about the point of no return in your submit ioctl. Which I think is a
> > Very Good Thing, at least looking at i915 execbuf where the point of
> > no return is a multi-stage thing with such interesting intermediate
> > points like "we submit the ruquest but without actually running the
> > batchbuffer". The downside is that the submit ioctl isn't perfectly
> > transaction anymore, but I don't think that matters for tha tail
> > stuff, which is generally just some out-fence installing. That
> > generally never fails.
>
> So I hacked up:
>
> ------
> diff --git a/drivers/gpu/drm/scheduler/sched_fence.c
> b/drivers/gpu/drm/scheduler/sched_fence.c
> index 3aa6351d2101..88e66dbc9515 100644
> --- a/drivers/gpu/drm/scheduler/sched_fence.c
> +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> @@ -176,6 +176,7 @@ struct drm_sched_fence
> *drm_sched_fence_create(struct drm_sched_entity *entity,
> fence->sched = entity->rq->sched;
> spin_lock_init(&fence->lock);
>
> + seq = atomic_inc_return(&entity->fence_seq);
> seq = atomic_inc_return(&entity->fence_seq);
> dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
> &fence->lock, entity->fence_context, seq);
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c
> b/drivers/gpu/drm/scheduler/sched_main.c
> index fcc601962e92..583e85adbbe0 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -593,6 +593,7 @@ int drm_sched_job_init(struct drm_sched_job *job,
> if (!job->s_fence)
> return -ENOMEM;
> job->id = atomic64_inc_return(&sched->job_id_count);
> + job->id = atomic64_inc_return(&sched->job_id_count);
>
> INIT_LIST_HEAD(&job->list);
>
> ------
>
> (I guess the job->id part shouldn't really be needed, that looks like
> it is only used by amdgpu)
>
> This didn't cause any problems that I could see. So I don't *think* a
> failure after drm_sched_job_init() is really problematic, as long as
> things are serialized between drm_sched_job_init() and
> drm_sched_entity_push_job().
>
> I also noticed that in the atomic commit path, the out-fences are
> initialized before atomic-check.. so there should be plenty of
> precedent for skipping fence seqno's.
Oh I think I remember now. The reason why the split into init/arm is
so that you can keep your critical section only around job_arm() and
push_job(). My very first version just pulled the jobs_init() of that
for most drivers to where I needed it, and that would result in a bit
chaos because the fences would signal out of order potentially. But
yeah I guess bailing out is fine with the scheduler.
Do you want me to tune down the commit message a bit, it's not a must
to submit the job, but just makes a bit more sense than bailing out
with a fence seqno reserved?
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
@ 2021-08-06 19:10 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-06 19:10 UTC (permalink / raw)
To: Rob Clark
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 8:57 PM Rob Clark <robdclark@gmail.com> wrote:
>
> On Fri, Aug 6, 2021 at 11:41 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > On Fri, Aug 6, 2021 at 7:15 PM Rob Clark <robdclark@gmail.com> wrote:
> > >
> > > On Fri, Aug 6, 2021 at 9:42 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > >
> > > > On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
> > > > >
> > > > > On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > > > >
> > > > > > Originally drm_sched_job_init was the point of no return, after which
> > > > > > drivers must submit a job. I've split that up, which allows us to fix
> > > > > > this issue pretty easily.
> > > > > >
> > > > > > Only thing we have to take care of is to not skip to error paths after
> > > > > > that. Other drivers do this the same for out-fence and similar things.
> > > > > >
> > > > > > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > > > > > Cc: Rob Clark <robdclark@chromium.org>
> > > > > > Cc: Rob Clark <robdclark@gmail.com>
> > > > > > Cc: Sean Paul <sean@poorly.run>
> > > > > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > > > > Cc: "Christian König" <christian.koenig@amd.com>
> > > > > > Cc: linux-arm-msm@vger.kernel.org
> > > > > > Cc: dri-devel@lists.freedesktop.org
> > > > > > Cc: freedreno@lists.freedesktop.org
> > > > > > Cc: linux-media@vger.kernel.org
> > > > > > Cc: linaro-mm-sig@lists.linaro.org
> > > > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > > > > > ---
> > > > > > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > > > > > 1 file changed, 7 insertions(+), 8 deletions(-)
> > > > > >
> > > > > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > > > > > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > > > > > return ERR_PTR(ret);
> > > > > > }
> > > > > >
> > > > > > - /* FIXME: this is way too early */
> > > > > > - drm_sched_job_arm(&job->base);
> > > > > > -
> > > > > > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> > > > > >
> > > > > > kref_init(&submit->ref);
> > > > > > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > > >
> > > > > > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> > > > > >
> > > > > > + /* point of no return, we _have_ to submit no matter what */
> > > > > > + drm_sched_job_arm(&submit->base);
> > > > > > +
> > > > > > /*
> > > > > > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > > > > > * to the underlying fence.
> > > > > > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > > > if (submit->fence_id < 0) {
> > > > > > ret = submit->fence_id = 0;
> > > > > > submit->fence_id = 0;
> > > > > > - goto out;
> > > > > > }
> > > > > >
> > > > > > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > > > > > if (!sync_file) {
> > > > > > ret = -ENOMEM;
> > > > > > - goto out;
> > > > > > + } else {
> > > > > > + fd_install(out_fence_fd, sync_file->file);
> > > > > > + args->fence_fd = out_fence_fd;
> > > > > > }
> > > > > > - fd_install(out_fence_fd, sync_file->file);
> > > > > > - args->fence_fd = out_fence_fd;
> > > > >
> > > > > I wonder if instead we should (approximately) undo "drm/msm/submit:
> > > > > Simplify out-fence-fd handling" so that the point that it could fail
> > > > > is moved up ahead of the drm_sched_job_arm()?
> > > >
> > > > Hm yeah. Up to you how you want to paint this shed, I think either is fine.
> > > >
> > > > > Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> > > > > a quick look, it looks like it won't, but I'm still playing catchup
> > > > > and haven't had a chance to look at your entire series. If it doesn't
> > > > > work before drm_sched_job_arm(), then there is really no way to
> > > > > prevent a error path between the fence-init and job-submit.
> > > >
> > > > Yes. I thought I've checked that I put the _arm() in the right spot,
> > > > but I guess I screwed up and you need the fence before the point where
> > > > I've put the job_arm()? And yes the error path cannot be avoided for
> > > > out-fences, that's what I tried to explain in the commit message.
> > > >
> > > > > But, prior to your series, wouldn't a failure after
> > > > > drm_sched_job_init() but before the job is submitted just burn a
> > > > > fence-id, and otherwise carry on it's merry way?
> > > >
> > > > Maybe? I'm not sure whether the scheduler gets confused about the gap
> > > > and freak out abou that. I'm fairly new to that code and learning
> > > > (which is part why I'm working on it). Since you look up in
> > > > fences/syncobj after job_init() it should be pretty easy to whip up a
> > > > testcase and see what happens. Also as long as nothing fails you won't
> > > > see an issue, that's for sure.
> > >
> > > fair.. I'll try to come up with a test case.. pre-scheduler-conversion
> > > it wasn't a problem to fail after the fence seqno was allocated (well,
> > > I guess you might have problems if you had 2^31 failures in a row)
> >
> > Yeah one thing drm/sched forces you to do is have a very clear notion
> > about the point of no return in your submit ioctl. Which I think is a
> > Very Good Thing, at least looking at i915 execbuf where the point of
> > no return is a multi-stage thing with such interesting intermediate
> > points like "we submit the ruquest but without actually running the
> > batchbuffer". The downside is that the submit ioctl isn't perfectly
> > transaction anymore, but I don't think that matters for tha tail
> > stuff, which is generally just some out-fence installing. That
> > generally never fails.
>
> So I hacked up:
>
> ------
> diff --git a/drivers/gpu/drm/scheduler/sched_fence.c
> b/drivers/gpu/drm/scheduler/sched_fence.c
> index 3aa6351d2101..88e66dbc9515 100644
> --- a/drivers/gpu/drm/scheduler/sched_fence.c
> +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> @@ -176,6 +176,7 @@ struct drm_sched_fence
> *drm_sched_fence_create(struct drm_sched_entity *entity,
> fence->sched = entity->rq->sched;
> spin_lock_init(&fence->lock);
>
> + seq = atomic_inc_return(&entity->fence_seq);
> seq = atomic_inc_return(&entity->fence_seq);
> dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
> &fence->lock, entity->fence_context, seq);
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c
> b/drivers/gpu/drm/scheduler/sched_main.c
> index fcc601962e92..583e85adbbe0 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -593,6 +593,7 @@ int drm_sched_job_init(struct drm_sched_job *job,
> if (!job->s_fence)
> return -ENOMEM;
> job->id = atomic64_inc_return(&sched->job_id_count);
> + job->id = atomic64_inc_return(&sched->job_id_count);
>
> INIT_LIST_HEAD(&job->list);
>
> ------
>
> (I guess the job->id part shouldn't really be needed, that looks like
> it is only used by amdgpu)
>
> This didn't cause any problems that I could see. So I don't *think* a
> failure after drm_sched_job_init() is really problematic, as long as
> things are serialized between drm_sched_job_init() and
> drm_sched_entity_push_job().
>
> I also noticed that in the atomic commit path, the out-fences are
> initialized before atomic-check.. so there should be plenty of
> precedent for skipping fence seqno's.
Oh I think I remember now. The reason why the split into init/arm is
so that you can keep your critical section only around job_arm() and
push_job(). My very first version just pulled the jobs_init() of that
for most drivers to where I needed it, and that would result in a bit
chaos because the fences would signal out of order potentially. But
yeah I guess bailing out is fine with the scheduler.
Do you want me to tune down the commit message a bit, it's not a must
to submit the job, but just makes a bit more sense than bailing out
with a fence seqno reserved?
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
2021-08-06 19:10 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-06 19:59 ` Rob Clark
-1 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-06 19:59 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 12:11 PM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> On Fri, Aug 6, 2021 at 8:57 PM Rob Clark <robdclark@gmail.com> wrote:
> >
> > On Fri, Aug 6, 2021 at 11:41 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > >
> > > On Fri, Aug 6, 2021 at 7:15 PM Rob Clark <robdclark@gmail.com> wrote:
> > > >
> > > > On Fri, Aug 6, 2021 at 9:42 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > > >
> > > > > On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
> > > > > >
> > > > > > On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > > > > >
> > > > > > > Originally drm_sched_job_init was the point of no return, after which
> > > > > > > drivers must submit a job. I've split that up, which allows us to fix
> > > > > > > this issue pretty easily.
> > > > > > >
> > > > > > > Only thing we have to take care of is to not skip to error paths after
> > > > > > > that. Other drivers do this the same for out-fence and similar things.
> > > > > > >
> > > > > > > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > > > > > > Cc: Rob Clark <robdclark@chromium.org>
> > > > > > > Cc: Rob Clark <robdclark@gmail.com>
> > > > > > > Cc: Sean Paul <sean@poorly.run>
> > > > > > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > > > > > Cc: "Christian König" <christian.koenig@amd.com>
> > > > > > > Cc: linux-arm-msm@vger.kernel.org
> > > > > > > Cc: dri-devel@lists.freedesktop.org
> > > > > > > Cc: freedreno@lists.freedesktop.org
> > > > > > > Cc: linux-media@vger.kernel.org
> > > > > > > Cc: linaro-mm-sig@lists.linaro.org
> > > > > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > > > > > > ---
> > > > > > > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > > > > > > 1 file changed, 7 insertions(+), 8 deletions(-)
> > > > > > >
> > > > > > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > > > > > > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > > > > > > return ERR_PTR(ret);
> > > > > > > }
> > > > > > >
> > > > > > > - /* FIXME: this is way too early */
> > > > > > > - drm_sched_job_arm(&job->base);
> > > > > > > -
> > > > > > > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> > > > > > >
> > > > > > > kref_init(&submit->ref);
> > > > > > > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > > > >
> > > > > > > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> > > > > > >
> > > > > > > + /* point of no return, we _have_ to submit no matter what */
> > > > > > > + drm_sched_job_arm(&submit->base);
> > > > > > > +
> > > > > > > /*
> > > > > > > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > > > > > > * to the underlying fence.
> > > > > > > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > > > > if (submit->fence_id < 0) {
> > > > > > > ret = submit->fence_id = 0;
> > > > > > > submit->fence_id = 0;
> > > > > > > - goto out;
> > > > > > > }
> > > > > > >
> > > > > > > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > > > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > > > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > > > > > > if (!sync_file) {
> > > > > > > ret = -ENOMEM;
> > > > > > > - goto out;
> > > > > > > + } else {
> > > > > > > + fd_install(out_fence_fd, sync_file->file);
> > > > > > > + args->fence_fd = out_fence_fd;
> > > > > > > }
> > > > > > > - fd_install(out_fence_fd, sync_file->file);
> > > > > > > - args->fence_fd = out_fence_fd;
> > > > > >
> > > > > > I wonder if instead we should (approximately) undo "drm/msm/submit:
> > > > > > Simplify out-fence-fd handling" so that the point that it could fail
> > > > > > is moved up ahead of the drm_sched_job_arm()?
> > > > >
> > > > > Hm yeah. Up to you how you want to paint this shed, I think either is fine.
> > > > >
> > > > > > Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> > > > > > a quick look, it looks like it won't, but I'm still playing catchup
> > > > > > and haven't had a chance to look at your entire series. If it doesn't
> > > > > > work before drm_sched_job_arm(), then there is really no way to
> > > > > > prevent a error path between the fence-init and job-submit.
> > > > >
> > > > > Yes. I thought I've checked that I put the _arm() in the right spot,
> > > > > but I guess I screwed up and you need the fence before the point where
> > > > > I've put the job_arm()? And yes the error path cannot be avoided for
> > > > > out-fences, that's what I tried to explain in the commit message.
> > > > >
> > > > > > But, prior to your series, wouldn't a failure after
> > > > > > drm_sched_job_init() but before the job is submitted just burn a
> > > > > > fence-id, and otherwise carry on it's merry way?
> > > > >
> > > > > Maybe? I'm not sure whether the scheduler gets confused about the gap
> > > > > and freak out abou that. I'm fairly new to that code and learning
> > > > > (which is part why I'm working on it). Since you look up in
> > > > > fences/syncobj after job_init() it should be pretty easy to whip up a
> > > > > testcase and see what happens. Also as long as nothing fails you won't
> > > > > see an issue, that's for sure.
> > > >
> > > > fair.. I'll try to come up with a test case.. pre-scheduler-conversion
> > > > it wasn't a problem to fail after the fence seqno was allocated (well,
> > > > I guess you might have problems if you had 2^31 failures in a row)
> > >
> > > Yeah one thing drm/sched forces you to do is have a very clear notion
> > > about the point of no return in your submit ioctl. Which I think is a
> > > Very Good Thing, at least looking at i915 execbuf where the point of
> > > no return is a multi-stage thing with such interesting intermediate
> > > points like "we submit the ruquest but without actually running the
> > > batchbuffer". The downside is that the submit ioctl isn't perfectly
> > > transaction anymore, but I don't think that matters for tha tail
> > > stuff, which is generally just some out-fence installing. That
> > > generally never fails.
> >
> > So I hacked up:
> >
> > ------
> > diff --git a/drivers/gpu/drm/scheduler/sched_fence.c
> > b/drivers/gpu/drm/scheduler/sched_fence.c
> > index 3aa6351d2101..88e66dbc9515 100644
> > --- a/drivers/gpu/drm/scheduler/sched_fence.c
> > +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> > @@ -176,6 +176,7 @@ struct drm_sched_fence
> > *drm_sched_fence_create(struct drm_sched_entity *entity,
> > fence->sched = entity->rq->sched;
> > spin_lock_init(&fence->lock);
> >
> > + seq = atomic_inc_return(&entity->fence_seq);
> > seq = atomic_inc_return(&entity->fence_seq);
> > dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
> > &fence->lock, entity->fence_context, seq);
> > diff --git a/drivers/gpu/drm/scheduler/sched_main.c
> > b/drivers/gpu/drm/scheduler/sched_main.c
> > index fcc601962e92..583e85adbbe0 100644
> > --- a/drivers/gpu/drm/scheduler/sched_main.c
> > +++ b/drivers/gpu/drm/scheduler/sched_main.c
> > @@ -593,6 +593,7 @@ int drm_sched_job_init(struct drm_sched_job *job,
> > if (!job->s_fence)
> > return -ENOMEM;
> > job->id = atomic64_inc_return(&sched->job_id_count);
> > + job->id = atomic64_inc_return(&sched->job_id_count);
> >
> > INIT_LIST_HEAD(&job->list);
> >
> > ------
> >
> > (I guess the job->id part shouldn't really be needed, that looks like
> > it is only used by amdgpu)
> >
> > This didn't cause any problems that I could see. So I don't *think* a
> > failure after drm_sched_job_init() is really problematic, as long as
> > things are serialized between drm_sched_job_init() and
> > drm_sched_entity_push_job().
> >
> > I also noticed that in the atomic commit path, the out-fences are
> > initialized before atomic-check.. so there should be plenty of
> > precedent for skipping fence seqno's.
>
> Oh I think I remember now. The reason why the split into init/arm is
> so that you can keep your critical section only around job_arm() and
> push_job(). My very first version just pulled the jobs_init() of that
> for most drivers to where I needed it, and that would result in a bit
> chaos because the fences would signal out of order potentially. But
> yeah I guess bailing out is fine with the scheduler.
ahh, that makes more sense
> Do you want me to tune down the commit message a bit, it's not a must
> to submit the job, but just makes a bit more sense than bailing out
> with a fence seqno reserved?
yeah, and I guess drop the fixme comment in the drm/msm patch..
I wonder if it would make sense to split drm_fence_init().. most of
the things we need the out-fence for prior to
drm_sched_entity_push_job() is, I think, just to have a dma_fence
reference.. which doesn't require having a seqno assigned yet. Which
would let us move the critical section into
drm_sched_entity_push_job() itself. (OTOH I suppose that opens up an
exciting new class of bugs, to have fences floating around without an
initialized seqno.)
BR,
-R
> -Daniel
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 02/20] drm/msm: Fix drm/sched point of no return rules
@ 2021-08-06 19:59 ` Rob Clark
0 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-06 19:59 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Fri, Aug 6, 2021 at 12:11 PM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> On Fri, Aug 6, 2021 at 8:57 PM Rob Clark <robdclark@gmail.com> wrote:
> >
> > On Fri, Aug 6, 2021 at 11:41 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > >
> > > On Fri, Aug 6, 2021 at 7:15 PM Rob Clark <robdclark@gmail.com> wrote:
> > > >
> > > > On Fri, Aug 6, 2021 at 9:42 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > > >
> > > > > On Fri, Aug 6, 2021 at 12:58 AM Rob Clark <robdclark@gmail.com> wrote:
> > > > > >
> > > > > > On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > > > > > >
> > > > > > > Originally drm_sched_job_init was the point of no return, after which
> > > > > > > drivers must submit a job. I've split that up, which allows us to fix
> > > > > > > this issue pretty easily.
> > > > > > >
> > > > > > > Only thing we have to take care of is to not skip to error paths after
> > > > > > > that. Other drivers do this the same for out-fence and similar things.
> > > > > > >
> > > > > > > Fixes: 1d8a5ca436ee ("drm/msm: Conversion to drm scheduler")
> > > > > > > Cc: Rob Clark <robdclark@chromium.org>
> > > > > > > Cc: Rob Clark <robdclark@gmail.com>
> > > > > > > Cc: Sean Paul <sean@poorly.run>
> > > > > > > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > > > > > > Cc: "Christian König" <christian.koenig@amd.com>
> > > > > > > Cc: linux-arm-msm@vger.kernel.org
> > > > > > > Cc: dri-devel@lists.freedesktop.org
> > > > > > > Cc: freedreno@lists.freedesktop.org
> > > > > > > Cc: linux-media@vger.kernel.org
> > > > > > > Cc: linaro-mm-sig@lists.linaro.org
> > > > > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > > > > > > ---
> > > > > > > drivers/gpu/drm/msm/msm_gem_submit.c | 15 +++++++--------
> > > > > > > 1 file changed, 7 insertions(+), 8 deletions(-)
> > > > > > >
> > > > > > > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > > index 6d6c44f0e1f3..d0ed4ddc509e 100644
> > > > > > > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > > > > > > @@ -52,9 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > > > > > > return ERR_PTR(ret);
> > > > > > > }
> > > > > > >
> > > > > > > - /* FIXME: this is way too early */
> > > > > > > - drm_sched_job_arm(&job->base);
> > > > > > > -
> > > > > > > xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> > > > > > >
> > > > > > > kref_init(&submit->ref);
> > > > > > > @@ -883,6 +880,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > > > >
> > > > > > > submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
> > > > > > >
> > > > > > > + /* point of no return, we _have_ to submit no matter what */
> > > > > > > + drm_sched_job_arm(&submit->base);
> > > > > > > +
> > > > > > > /*
> > > > > > > * Allocate an id which can be used by WAIT_FENCE ioctl to map back
> > > > > > > * to the underlying fence.
> > > > > > > @@ -892,17 +892,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> > > > > > > if (submit->fence_id < 0) {
> > > > > > > ret = submit->fence_id = 0;
> > > > > > > submit->fence_id = 0;
> > > > > > > - goto out;
> > > > > > > }
> > > > > > >
> > > > > > > - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > > > + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> > > > > > > struct sync_file *sync_file = sync_file_create(submit->user_fence);
> > > > > > > if (!sync_file) {
> > > > > > > ret = -ENOMEM;
> > > > > > > - goto out;
> > > > > > > + } else {
> > > > > > > + fd_install(out_fence_fd, sync_file->file);
> > > > > > > + args->fence_fd = out_fence_fd;
> > > > > > > }
> > > > > > > - fd_install(out_fence_fd, sync_file->file);
> > > > > > > - args->fence_fd = out_fence_fd;
> > > > > >
> > > > > > I wonder if instead we should (approximately) undo "drm/msm/submit:
> > > > > > Simplify out-fence-fd handling" so that the point that it could fail
> > > > > > is moved up ahead of the drm_sched_job_arm()?
> > > > >
> > > > > Hm yeah. Up to you how you want to paint this shed, I think either is fine.
> > > > >
> > > > > > Also, does the dma_fence_get() work before drm_sched_job_arm()? From
> > > > > > a quick look, it looks like it won't, but I'm still playing catchup
> > > > > > and haven't had a chance to look at your entire series. If it doesn't
> > > > > > work before drm_sched_job_arm(), then there is really no way to
> > > > > > prevent a error path between the fence-init and job-submit.
> > > > >
> > > > > Yes. I thought I've checked that I put the _arm() in the right spot,
> > > > > but I guess I screwed up and you need the fence before the point where
> > > > > I've put the job_arm()? And yes the error path cannot be avoided for
> > > > > out-fences, that's what I tried to explain in the commit message.
> > > > >
> > > > > > But, prior to your series, wouldn't a failure after
> > > > > > drm_sched_job_init() but before the job is submitted just burn a
> > > > > > fence-id, and otherwise carry on it's merry way?
> > > > >
> > > > > Maybe? I'm not sure whether the scheduler gets confused about the gap
> > > > > and freak out abou that. I'm fairly new to that code and learning
> > > > > (which is part why I'm working on it). Since you look up in
> > > > > fences/syncobj after job_init() it should be pretty easy to whip up a
> > > > > testcase and see what happens. Also as long as nothing fails you won't
> > > > > see an issue, that's for sure.
> > > >
> > > > fair.. I'll try to come up with a test case.. pre-scheduler-conversion
> > > > it wasn't a problem to fail after the fence seqno was allocated (well,
> > > > I guess you might have problems if you had 2^31 failures in a row)
> > >
> > > Yeah one thing drm/sched forces you to do is have a very clear notion
> > > about the point of no return in your submit ioctl. Which I think is a
> > > Very Good Thing, at least looking at i915 execbuf where the point of
> > > no return is a multi-stage thing with such interesting intermediate
> > > points like "we submit the ruquest but without actually running the
> > > batchbuffer". The downside is that the submit ioctl isn't perfectly
> > > transaction anymore, but I don't think that matters for tha tail
> > > stuff, which is generally just some out-fence installing. That
> > > generally never fails.
> >
> > So I hacked up:
> >
> > ------
> > diff --git a/drivers/gpu/drm/scheduler/sched_fence.c
> > b/drivers/gpu/drm/scheduler/sched_fence.c
> > index 3aa6351d2101..88e66dbc9515 100644
> > --- a/drivers/gpu/drm/scheduler/sched_fence.c
> > +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> > @@ -176,6 +176,7 @@ struct drm_sched_fence
> > *drm_sched_fence_create(struct drm_sched_entity *entity,
> > fence->sched = entity->rq->sched;
> > spin_lock_init(&fence->lock);
> >
> > + seq = atomic_inc_return(&entity->fence_seq);
> > seq = atomic_inc_return(&entity->fence_seq);
> > dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled,
> > &fence->lock, entity->fence_context, seq);
> > diff --git a/drivers/gpu/drm/scheduler/sched_main.c
> > b/drivers/gpu/drm/scheduler/sched_main.c
> > index fcc601962e92..583e85adbbe0 100644
> > --- a/drivers/gpu/drm/scheduler/sched_main.c
> > +++ b/drivers/gpu/drm/scheduler/sched_main.c
> > @@ -593,6 +593,7 @@ int drm_sched_job_init(struct drm_sched_job *job,
> > if (!job->s_fence)
> > return -ENOMEM;
> > job->id = atomic64_inc_return(&sched->job_id_count);
> > + job->id = atomic64_inc_return(&sched->job_id_count);
> >
> > INIT_LIST_HEAD(&job->list);
> >
> > ------
> >
> > (I guess the job->id part shouldn't really be needed, that looks like
> > it is only used by amdgpu)
> >
> > This didn't cause any problems that I could see. So I don't *think* a
> > failure after drm_sched_job_init() is really problematic, as long as
> > things are serialized between drm_sched_job_init() and
> > drm_sched_entity_push_job().
> >
> > I also noticed that in the atomic commit path, the out-fences are
> > initialized before atomic-check.. so there should be plenty of
> > precedent for skipping fence seqno's.
>
> Oh I think I remember now. The reason why the split into init/arm is
> so that you can keep your critical section only around job_arm() and
> push_job(). My very first version just pulled the jobs_init() of that
> for most drivers to where I needed it, and that would result in a bit
> chaos because the fences would signal out of order potentially. But
> yeah I guess bailing out is fine with the scheduler.
ahh, that makes more sense
> Do you want me to tune down the commit message a bit, it's not a must
> to submit the job, but just makes a bit more sense than bailing out
> with a fence seqno reserved?
yeah, and I guess drop the fixme comment in the drm/msm patch..
I wonder if it would make sense to split drm_fence_init().. most of
the things we need the out-fence for prior to
drm_sched_entity_push_job() is, I think, just to have a dma_fence
reference.. which doesn't require having a seqno assigned yet. Which
would let us move the critical section into
drm_sched_entity_push_job() itself. (OTOH I suppose that opens up an
exciting new class of bugs, to have fences floating around without an
initialized seqno.)
BR,
-R
> -Daniel
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH] drm/msm: Improve drm/sched point of no return rules
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-17 8:53 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-17 8:53 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Rob Clark, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, linux-media, linaro-mm-sig, Daniel Vetter
Originally drm_sched_job_init was the point of no return, after which
drivers really should submit a job. I've split that up, which allows
us to fix this issue pretty easily.
Only thing we have to take care of is to not skip to error paths after
that. Other drivers do this the same for out-fence and similar things.
v2: It's not really a bugfix, just an improvement, since all
drm_sched_job_arm does is reserve the fence number. And gaps should be
fine, as long as the drm_sched_job doesn't escape anywhere at all.
For robustness it's still better to align with other drivers here and
not bail out after job_arm().
Cc: Rob Clark <robdclark@chromium.org>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: freedreno@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
drivers/gpu/drm/msm/msm_gem_submit.c | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 4d1c4d5f6a2a..371ed9154e58 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
- drm_sched_job_arm(&job->base);
-
xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
kref_init(&submit->ref);
@@ -882,6 +880,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
+ drm_sched_job_arm(&submit->base);
+
/*
* Allocate an id which can be used by WAIT_FENCE ioctl to map back
* to the underlying fence.
@@ -891,17 +891,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
if (submit->fence_id < 0) {
ret = submit->fence_id = 0;
submit->fence_id = 0;
- goto out;
}
- if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
+ if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
struct sync_file *sync_file = sync_file_create(submit->user_fence);
if (!sync_file) {
ret = -ENOMEM;
- goto out;
+ } else {
+ fd_install(out_fence_fd, sync_file->file);
+ args->fence_fd = out_fence_fd;
}
- fd_install(out_fence_fd, sync_file->file);
- args->fence_fd = out_fence_fd;
}
submit_attach_object_fences(submit);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH] drm/msm: Improve drm/sched point of no return rules
@ 2021-08-17 8:53 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-17 8:53 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Rob Clark, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, linux-media, linaro-mm-sig, Daniel Vetter
Originally drm_sched_job_init was the point of no return, after which
drivers really should submit a job. I've split that up, which allows
us to fix this issue pretty easily.
Only thing we have to take care of is to not skip to error paths after
that. Other drivers do this the same for out-fence and similar things.
v2: It's not really a bugfix, just an improvement, since all
drm_sched_job_arm does is reserve the fence number. And gaps should be
fine, as long as the drm_sched_job doesn't escape anywhere at all.
For robustness it's still better to align with other drivers here and
not bail out after job_arm().
Cc: Rob Clark <robdclark@chromium.org>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: freedreno@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
drivers/gpu/drm/msm/msm_gem_submit.c | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 4d1c4d5f6a2a..371ed9154e58 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
- drm_sched_job_arm(&job->base);
-
xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
kref_init(&submit->ref);
@@ -882,6 +880,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
+ drm_sched_job_arm(&submit->base);
+
/*
* Allocate an id which can be used by WAIT_FENCE ioctl to map back
* to the underlying fence.
@@ -891,17 +891,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
if (submit->fence_id < 0) {
ret = submit->fence_id = 0;
submit->fence_id = 0;
- goto out;
}
- if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
+ if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
struct sync_file *sync_file = sync_file_create(submit->user_fence);
if (!sync_file) {
ret = -ENOMEM;
- goto out;
+ } else {
+ fd_install(out_fence_fd, sync_file->file);
+ args->fence_fd = out_fence_fd;
}
- fd_install(out_fence_fd, sync_file->file);
- args->fence_fd = out_fence_fd;
}
submit_attach_object_fences(submit);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [PATCH] drm/msm: Improve drm/sched point of no return rules
2021-08-17 8:53 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-26 9:33 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-26 9:33 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Rob Clark, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, linux-media, linaro-mm-sig, Daniel Vetter
Originally drm_sched_job_init was the point of no return, after which
drivers really should submit a job. I've split that up, which allows
us to fix this issue pretty easily.
Only thing we have to take care of is to not skip to error paths after
that. Other drivers do this the same for out-fence and similar things.
v2: It's not really a bugfix, just an improvement, since all
drm_sched_job_arm does is reserve the fence number. And gaps should be
fine, as long as the drm_sched_job doesn't escape anywhere at all.
For robustness it's still better to align with other drivers here and
not bail out after job_arm().
v3: I misplaced drm_sched_job_arm by _one_ line! Thanks to Rob for
testing and debug help.
Cc: Rob Clark <robdclark@chromium.org>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: freedreno@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
drivers/gpu/drm/msm/msm_gem_submit.c | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 4d1c4d5f6a2a..71b8c8f752a3 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
- drm_sched_job_arm(&job->base);
-
xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
kref_init(&submit->ref);
@@ -880,6 +878,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->nr_cmds = i;
+ drm_sched_job_arm(&submit->base);
+
submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
/*
@@ -891,17 +891,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
if (submit->fence_id < 0) {
ret = submit->fence_id = 0;
submit->fence_id = 0;
- goto out;
}
- if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
+ if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
struct sync_file *sync_file = sync_file_create(submit->user_fence);
if (!sync_file) {
ret = -ENOMEM;
- goto out;
+ } else {
+ fd_install(out_fence_fd, sync_file->file);
+ args->fence_fd = out_fence_fd;
}
- fd_install(out_fence_fd, sync_file->file);
- args->fence_fd = out_fence_fd;
}
submit_attach_object_fences(submit);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH] drm/msm: Improve drm/sched point of no return rules
@ 2021-08-26 9:33 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-26 9:33 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Rob Clark, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, linux-media, linaro-mm-sig, Daniel Vetter
Originally drm_sched_job_init was the point of no return, after which
drivers really should submit a job. I've split that up, which allows
us to fix this issue pretty easily.
Only thing we have to take care of is to not skip to error paths after
that. Other drivers do this the same for out-fence and similar things.
v2: It's not really a bugfix, just an improvement, since all
drm_sched_job_arm does is reserve the fence number. And gaps should be
fine, as long as the drm_sched_job doesn't escape anywhere at all.
For robustness it's still better to align with other drivers here and
not bail out after job_arm().
v3: I misplaced drm_sched_job_arm by _one_ line! Thanks to Rob for
testing and debug help.
Cc: Rob Clark <robdclark@chromium.org>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: freedreno@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
drivers/gpu/drm/msm/msm_gem_submit.c | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 4d1c4d5f6a2a..71b8c8f752a3 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
- drm_sched_job_arm(&job->base);
-
xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
kref_init(&submit->ref);
@@ -880,6 +878,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->nr_cmds = i;
+ drm_sched_job_arm(&submit->base);
+
submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
/*
@@ -891,17 +891,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
if (submit->fence_id < 0) {
ret = submit->fence_id = 0;
submit->fence_id = 0;
- goto out;
}
- if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
+ if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
struct sync_file *sync_file = sync_file_create(submit->user_fence);
if (!sync_file) {
ret = -ENOMEM;
- goto out;
+ } else {
+ fd_install(out_fence_fd, sync_file->file);
+ args->fence_fd = out_fence_fd;
}
- fd_install(out_fence_fd, sync_file->file);
- args->fence_fd = out_fence_fd;
}
submit_attach_object_fences(submit);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH] drm/msm: Improve drm/sched point of no return rules
2021-08-26 9:33 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-26 15:38 ` Rob Clark
-1 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-26 15:38 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Thu, Aug 26, 2021 at 2:33 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> Originally drm_sched_job_init was the point of no return, after which
> drivers really should submit a job. I've split that up, which allows
> us to fix this issue pretty easily.
>
> Only thing we have to take care of is to not skip to error paths after
> that. Other drivers do this the same for out-fence and similar things.
>
> v2: It's not really a bugfix, just an improvement, since all
> drm_sched_job_arm does is reserve the fence number. And gaps should be
> fine, as long as the drm_sched_job doesn't escape anywhere at all.
>
> For robustness it's still better to align with other drivers here and
> not bail out after job_arm().
>
> v3: I misplaced drm_sched_job_arm by _one_ line! Thanks to Rob for
> testing and debug help.
>
> Cc: Rob Clark <robdclark@chromium.org>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: dri-devel@lists.freedesktop.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
t-b && r-b
BR,
-R
> ---
> drivers/gpu/drm/msm/msm_gem_submit.c | 13 ++++++-------
> 1 file changed, 6 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index 4d1c4d5f6a2a..71b8c8f752a3 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> - drm_sched_job_arm(&job->base);
> -
> xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
>
> kref_init(&submit->ref);
> @@ -880,6 +878,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
>
> submit->nr_cmds = i;
>
> + drm_sched_job_arm(&submit->base);
> +
> submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
>
> /*
> @@ -891,17 +891,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> if (submit->fence_id < 0) {
> ret = submit->fence_id = 0;
> submit->fence_id = 0;
> - goto out;
> }
>
> - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> struct sync_file *sync_file = sync_file_create(submit->user_fence);
> if (!sync_file) {
> ret = -ENOMEM;
> - goto out;
> + } else {
> + fd_install(out_fence_fd, sync_file->file);
> + args->fence_fd = out_fence_fd;
> }
> - fd_install(out_fence_fd, sync_file->file);
> - args->fence_fd = out_fence_fd;
> }
>
> submit_attach_object_fences(submit);
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH] drm/msm: Improve drm/sched point of no return rules
@ 2021-08-26 15:38 ` Rob Clark
0 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-26 15:38 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Rob Clark,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK, Daniel Vetter
On Thu, Aug 26, 2021 at 2:33 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> Originally drm_sched_job_init was the point of no return, after which
> drivers really should submit a job. I've split that up, which allows
> us to fix this issue pretty easily.
>
> Only thing we have to take care of is to not skip to error paths after
> that. Other drivers do this the same for out-fence and similar things.
>
> v2: It's not really a bugfix, just an improvement, since all
> drm_sched_job_arm does is reserve the fence number. And gaps should be
> fine, as long as the drm_sched_job doesn't escape anywhere at all.
>
> For robustness it's still better to align with other drivers here and
> not bail out after job_arm().
>
> v3: I misplaced drm_sched_job_arm by _one_ line! Thanks to Rob for
> testing and debug help.
>
> Cc: Rob Clark <robdclark@chromium.org>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: dri-devel@lists.freedesktop.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
t-b && r-b
BR,
-R
> ---
> drivers/gpu/drm/msm/msm_gem_submit.c | 13 ++++++-------
> 1 file changed, 6 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index 4d1c4d5f6a2a..71b8c8f752a3 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> - drm_sched_job_arm(&job->base);
> -
> xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
>
> kref_init(&submit->ref);
> @@ -880,6 +878,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
>
> submit->nr_cmds = i;
>
> + drm_sched_job_arm(&submit->base);
> +
> submit->user_fence = dma_fence_get(&submit->base.s_fence->finished);
>
> /*
> @@ -891,17 +891,16 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> if (submit->fence_id < 0) {
> ret = submit->fence_id = 0;
> submit->fence_id = 0;
> - goto out;
> }
>
> - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> + if (ret == 0 && args->flags & MSM_SUBMIT_FENCE_FD_OUT) {
> struct sync_file *sync_file = sync_file_create(submit->user_fence);
> if (!sync_file) {
> ret = -ENOMEM;
> - goto out;
> + } else {
> + fd_install(out_fence_fd, sync_file->file);
> + args->fence_fd = out_fence_fd;
> }
> - fd_install(out_fence_fd, sync_file->file);
> - args->fence_fd = out_fence_fd;
> }
>
> submit_attach_object_fences(submit);
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 03/20] drm/sched: Barriers are needed for entity->last_scheduled
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Daniel Vetter, Christian König, Steven Price,
Andrey Grodzovsky, Lee Jones, Boris Brezillon
It might be good enough on x86 with just READ_ONCE, but the write side
should then at least be WRITE_ONCE because x86 has total store order.
It's definitely not enough on arm.
Fix this proplery, which means
- explain the need for the barrier in both places
- point at the other side in each comment
Also pull out the !sched_list case as the first check, so that the
code flow is clearer.
While at it sprinkle some comments around because it was very
non-obvious to me what's actually going on here and why.
Note that we really need full barriers here, at first I thought
store-release and load-acquire on ->last_scheduled would be enough,
but we actually requiring ordering between that and the queue state.
v2: Put smp_rmp() in the right place and fix up comment (Andrey)
Acked-by: Melissa Wen <mwen@igalia.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
---
drivers/gpu/drm/scheduler/sched_entity.c | 27 ++++++++++++++++++++++--
1 file changed, 25 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index f7347c284886..89e3f6eaf519 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -439,8 +439,16 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
dma_fence_set_error(&sched_job->s_fence->finished, -ECANCELED);
dma_fence_put(entity->last_scheduled);
+
entity->last_scheduled = dma_fence_get(&sched_job->s_fence->finished);
+ /*
+ * If the queue is empty we allow drm_sched_entity_select_rq() to
+ * locklessly access ->last_scheduled. This only works if we set the
+ * pointer before we dequeue and if we a write barrier here.
+ */
+ smp_wmb();
+
spsc_queue_pop(&entity->job_queue);
return sched_job;
}
@@ -459,10 +467,25 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
struct drm_gpu_scheduler *sched;
struct drm_sched_rq *rq;
- if (spsc_queue_count(&entity->job_queue) || !entity->sched_list)
+ /* single possible engine and already selected */
+ if (!entity->sched_list)
+ return;
+
+ /* queue non-empty, stay on the same engine */
+ if (spsc_queue_count(&entity->job_queue))
return;
- fence = READ_ONCE(entity->last_scheduled);
+ /*
+ * Only when the queue is empty are we guaranteed that the scheduler
+ * thread cannot change ->last_scheduled. To enforce ordering we need
+ * a read barrier here. See drm_sched_entity_pop_job() for the other
+ * side.
+ */
+ smp_rmb();
+
+ fence = entity->last_scheduled;
+
+ /* stay on the same engine if the previous job hasn't finished */
if (fence && !dma_fence_is_signaled(fence))
return;
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* [Intel-gfx] [PATCH v5 03/20] drm/sched: Barriers are needed for entity->last_scheduled
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Daniel Vetter, Christian König, Steven Price,
Andrey Grodzovsky, Lee Jones, Boris Brezillon
It might be good enough on x86 with just READ_ONCE, but the write side
should then at least be WRITE_ONCE because x86 has total store order.
It's definitely not enough on arm.
Fix this proplery, which means
- explain the need for the barrier in both places
- point at the other side in each comment
Also pull out the !sched_list case as the first check, so that the
code flow is clearer.
While at it sprinkle some comments around because it was very
non-obvious to me what's actually going on here and why.
Note that we really need full barriers here, at first I thought
store-release and load-acquire on ->last_scheduled would be enough,
but we actually requiring ordering between that and the queue state.
v2: Put smp_rmp() in the right place and fix up comment (Andrey)
Acked-by: Melissa Wen <mwen@igalia.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
---
drivers/gpu/drm/scheduler/sched_entity.c | 27 ++++++++++++++++++++++--
1 file changed, 25 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index f7347c284886..89e3f6eaf519 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -439,8 +439,16 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
dma_fence_set_error(&sched_job->s_fence->finished, -ECANCELED);
dma_fence_put(entity->last_scheduled);
+
entity->last_scheduled = dma_fence_get(&sched_job->s_fence->finished);
+ /*
+ * If the queue is empty we allow drm_sched_entity_select_rq() to
+ * locklessly access ->last_scheduled. This only works if we set the
+ * pointer before we dequeue and if we a write barrier here.
+ */
+ smp_wmb();
+
spsc_queue_pop(&entity->job_queue);
return sched_job;
}
@@ -459,10 +467,25 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
struct drm_gpu_scheduler *sched;
struct drm_sched_rq *rq;
- if (spsc_queue_count(&entity->job_queue) || !entity->sched_list)
+ /* single possible engine and already selected */
+ if (!entity->sched_list)
+ return;
+
+ /* queue non-empty, stay on the same engine */
+ if (spsc_queue_count(&entity->job_queue))
return;
- fence = READ_ONCE(entity->last_scheduled);
+ /*
+ * Only when the queue is empty are we guaranteed that the scheduler
+ * thread cannot change ->last_scheduled. To enforce ordering we need
+ * a read barrier here. See drm_sched_entity_pop_job() for the other
+ * side.
+ */
+ smp_rmb();
+
+ fence = entity->last_scheduled;
+
+ /* stay on the same engine if the previous job hasn't finished */
if (fence && !dma_fence_is_signaled(fence))
return;
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* Re: [PATCH v5 03/20] drm/sched: Barriers are needed for entity->last_scheduled
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 13:45 ` Christian König
-1 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:45 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Melissa Wen, Daniel Vetter,
Steven Price, Andrey Grodzovsky, Lee Jones, Boris Brezillon
Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> It might be good enough on x86 with just READ_ONCE, but the write side
> should then at least be WRITE_ONCE because x86 has total store order.
>
> It's definitely not enough on arm.
>
> Fix this proplery, which means
> - explain the need for the barrier in both places
> - point at the other side in each comment
>
> Also pull out the !sched_list case as the first check, so that the
> code flow is clearer.
>
> While at it sprinkle some comments around because it was very
> non-obvious to me what's actually going on here and why.
>
> Note that we really need full barriers here, at first I thought
> store-release and load-acquire on ->last_scheduled would be enough,
> but we actually requiring ordering between that and the queue state.
>
> v2: Put smp_rmp() in the right place and fix up comment (Andrey)
>
> Acked-by: Melissa Wen <mwen@igalia.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Lee Jones <lee.jones@linaro.org>
> Cc: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
> ---
> drivers/gpu/drm/scheduler/sched_entity.c | 27 ++++++++++++++++++++++--
> 1 file changed, 25 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> index f7347c284886..89e3f6eaf519 100644
> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> @@ -439,8 +439,16 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
> dma_fence_set_error(&sched_job->s_fence->finished, -ECANCELED);
>
> dma_fence_put(entity->last_scheduled);
> +
> entity->last_scheduled = dma_fence_get(&sched_job->s_fence->finished);
>
> + /*
> + * If the queue is empty we allow drm_sched_entity_select_rq() to
> + * locklessly access ->last_scheduled. This only works if we set the
> + * pointer before we dequeue and if we a write barrier here.
> + */
> + smp_wmb();
> +
> spsc_queue_pop(&entity->job_queue);
> return sched_job;
> }
> @@ -459,10 +467,25 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
> struct drm_gpu_scheduler *sched;
> struct drm_sched_rq *rq;
>
> - if (spsc_queue_count(&entity->job_queue) || !entity->sched_list)
> + /* single possible engine and already selected */
> + if (!entity->sched_list)
> + return;
> +
> + /* queue non-empty, stay on the same engine */
> + if (spsc_queue_count(&entity->job_queue))
> return;
>
> - fence = READ_ONCE(entity->last_scheduled);
> + /*
> + * Only when the queue is empty are we guaranteed that the scheduler
> + * thread cannot change ->last_scheduled. To enforce ordering we need
> + * a read barrier here. See drm_sched_entity_pop_job() for the other
> + * side.
> + */
> + smp_rmb();
> +
> + fence = entity->last_scheduled;
> +
> + /* stay on the same engine if the previous job hasn't finished */
> if (fence && !dma_fence_is_signaled(fence))
> return;
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [Intel-gfx] [PATCH v5 03/20] drm/sched: Barriers are needed for entity->last_scheduled
@ 2021-08-05 13:45 ` Christian König
0 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:45 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Melissa Wen, Daniel Vetter,
Steven Price, Andrey Grodzovsky, Lee Jones, Boris Brezillon
Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> It might be good enough on x86 with just READ_ONCE, but the write side
> should then at least be WRITE_ONCE because x86 has total store order.
>
> It's definitely not enough on arm.
>
> Fix this proplery, which means
> - explain the need for the barrier in both places
> - point at the other side in each comment
>
> Also pull out the !sched_list case as the first check, so that the
> code flow is clearer.
>
> While at it sprinkle some comments around because it was very
> non-obvious to me what's actually going on here and why.
>
> Note that we really need full barriers here, at first I thought
> store-release and load-acquire on ->last_scheduled would be enough,
> but we actually requiring ordering between that and the queue state.
>
> v2: Put smp_rmp() in the right place and fix up comment (Andrey)
>
> Acked-by: Melissa Wen <mwen@igalia.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Lee Jones <lee.jones@linaro.org>
> Cc: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
> ---
> drivers/gpu/drm/scheduler/sched_entity.c | 27 ++++++++++++++++++++++--
> 1 file changed, 25 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> index f7347c284886..89e3f6eaf519 100644
> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> @@ -439,8 +439,16 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
> dma_fence_set_error(&sched_job->s_fence->finished, -ECANCELED);
>
> dma_fence_put(entity->last_scheduled);
> +
> entity->last_scheduled = dma_fence_get(&sched_job->s_fence->finished);
>
> + /*
> + * If the queue is empty we allow drm_sched_entity_select_rq() to
> + * locklessly access ->last_scheduled. This only works if we set the
> + * pointer before we dequeue and if we a write barrier here.
> + */
> + smp_wmb();
> +
> spsc_queue_pop(&entity->job_queue);
> return sched_job;
> }
> @@ -459,10 +467,25 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
> struct drm_gpu_scheduler *sched;
> struct drm_sched_rq *rq;
>
> - if (spsc_queue_count(&entity->job_queue) || !entity->sched_list)
> + /* single possible engine and already selected */
> + if (!entity->sched_list)
> + return;
> +
> + /* queue non-empty, stay on the same engine */
> + if (spsc_queue_count(&entity->job_queue))
> return;
>
> - fence = READ_ONCE(entity->last_scheduled);
> + /*
> + * Only when the queue is empty are we guaranteed that the scheduler
> + * thread cannot change ->last_scheduled. To enforce ordering we need
> + * a read barrier here. See drm_sched_entity_pop_job() for the other
> + * side.
> + */
> + smp_rmb();
> +
> + fence = entity->last_scheduled;
> +
> + /* stay on the same engine if the previous job hasn't finished */
> if (fence && !dma_fence_is_signaled(fence))
> return;
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 04/20] drm/sched: Add dependency tracking
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Boris Brezillon,
Steven Price, Melissa Wen, Daniel Vetter, David Airlie,
Daniel Vetter, Sumit Semwal, Christian König,
Andrey Grodzovsky, Lee Jones, Nirmoy Das, Luben Tuikov,
Alex Deucher, linux-media, linaro-mm-sig
Instead of just a callback we can just glue in the gem helpers that
panfrost, v3d and lima currently use. There's really not that many
ways to skin this cat.
v2/3: Rebased.
v4: Repaint this shed. The functions are now called _add_dependency()
and _add_implicit_dependency()
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v3)
Reviewed-by: Steven Price <steven.price@arm.com> (v1)
Acked-by: Melissa Wen <mwen@igalia.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Nirmoy Das <nirmoy.aiemd@gmail.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/gpu/drm/scheduler/sched_entity.c | 18 +++-
drivers/gpu/drm/scheduler/sched_main.c | 104 +++++++++++++++++++++++
include/drm/gpu_scheduler.h | 33 ++++++-
3 files changed, 149 insertions(+), 6 deletions(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 89e3f6eaf519..381fbf462ea7 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -211,6 +211,19 @@ static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
job->sched->ops->free_job(job);
}
+static struct dma_fence *
+drm_sched_job_dependency(struct drm_sched_job *job,
+ struct drm_sched_entity *entity)
+{
+ if (!xa_empty(&job->dependencies))
+ return xa_erase(&job->dependencies, job->last_dependency++);
+
+ if (job->sched->ops->dependency)
+ return job->sched->ops->dependency(job, entity);
+
+ return NULL;
+}
+
/**
* drm_sched_entity_kill_jobs - Make sure all remaining jobs are killed
*
@@ -229,7 +242,7 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
struct drm_sched_fence *s_fence = job->s_fence;
/* Wait for all dependencies to avoid data corruptions */
- while ((f = job->sched->ops->dependency(job, entity)))
+ while ((f = drm_sched_job_dependency(job, entity)))
dma_fence_wait(f, false);
drm_sched_fence_scheduled(s_fence);
@@ -419,7 +432,6 @@ static bool drm_sched_entity_add_dependency_cb(struct drm_sched_entity *entity)
*/
struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
{
- struct drm_gpu_scheduler *sched = entity->rq->sched;
struct drm_sched_job *sched_job;
sched_job = to_drm_sched_job(spsc_queue_peek(&entity->job_queue));
@@ -427,7 +439,7 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
return NULL;
while ((entity->dependency =
- sched->ops->dependency(sched_job, entity))) {
+ drm_sched_job_dependency(sched_job, entity))) {
trace_drm_sched_job_wait_dep(sched_job, entity->dependency);
if (drm_sched_entity_add_dependency_cb(entity))
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index 454cb6164bdc..f77456929139 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -603,6 +603,8 @@ int drm_sched_job_init(struct drm_sched_job *job,
INIT_LIST_HEAD(&job->list);
+ xa_init_flags(&job->dependencies, XA_FLAGS_ALLOC);
+
return 0;
}
EXPORT_SYMBOL(drm_sched_job_init);
@@ -637,6 +639,99 @@ void drm_sched_job_arm(struct drm_sched_job *job)
}
EXPORT_SYMBOL(drm_sched_job_arm);
+/**
+ * drm_sched_job_add_dependency - adds the fence as a job dependency
+ * @job: scheduler job to add the dependencies to
+ * @fence: the dma_fence to add to the list of dependencies.
+ *
+ * Note that @fence is consumed in both the success and error cases.
+ *
+ * Returns:
+ * 0 on success, or an error on failing to expand the array.
+ */
+int drm_sched_job_add_dependency(struct drm_sched_job *job,
+ struct dma_fence *fence)
+{
+ struct dma_fence *entry;
+ unsigned long index;
+ u32 id = 0;
+ int ret;
+
+ if (!fence)
+ return 0;
+
+ /* Deduplicate if we already depend on a fence from the same context.
+ * This lets the size of the array of deps scale with the number of
+ * engines involved, rather than the number of BOs.
+ */
+ xa_for_each(&job->dependencies, index, entry) {
+ if (entry->context != fence->context)
+ continue;
+
+ if (dma_fence_is_later(fence, entry)) {
+ dma_fence_put(entry);
+ xa_store(&job->dependencies, index, fence, GFP_KERNEL);
+ } else {
+ dma_fence_put(fence);
+ }
+ return 0;
+ }
+
+ ret = xa_alloc(&job->dependencies, &id, fence, xa_limit_32b, GFP_KERNEL);
+ if (ret != 0)
+ dma_fence_put(fence);
+
+ return ret;
+}
+EXPORT_SYMBOL(drm_sched_job_add_dependency);
+
+/**
+ * drm_sched_job_add_implicit_dependencies - adds implicit dependencies as job
+ * dependencies
+ * @job: scheduler job to add the dependencies to
+ * @obj: the gem object to add new dependencies from.
+ * @write: whether the job might write the object (so we need to depend on
+ * shared fences in the reservation object).
+ *
+ * This should be called after drm_gem_lock_reservations() on your array of
+ * GEM objects used in the job but before updating the reservations with your
+ * own fences.
+ *
+ * Returns:
+ * 0 on success, or an error on failing to expand the array.
+ */
+int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
+ struct drm_gem_object *obj,
+ bool write)
+{
+ int ret;
+ struct dma_fence **fences;
+ unsigned int i, fence_count;
+
+ if (!write) {
+ struct dma_fence *fence = dma_resv_get_excl_unlocked(obj->resv);
+
+ return drm_sched_job_add_dependency(job, fence);
+ }
+
+ ret = dma_resv_get_fences(obj->resv, NULL, &fence_count, &fences);
+ if (ret || !fence_count)
+ return ret;
+
+ for (i = 0; i < fence_count; i++) {
+ ret = drm_sched_job_add_dependency(job, fences[i]);
+ if (ret)
+ break;
+ }
+
+ for (; i < fence_count; i++)
+ dma_fence_put(fences[i]);
+ kfree(fences);
+ return ret;
+}
+EXPORT_SYMBOL(drm_sched_job_add_implicit_dependencies);
+
+
/**
* drm_sched_job_cleanup - clean up scheduler job resources
* @job: scheduler job to clean up
@@ -652,6 +747,9 @@ EXPORT_SYMBOL(drm_sched_job_arm);
*/
void drm_sched_job_cleanup(struct drm_sched_job *job)
{
+ struct dma_fence *fence;
+ unsigned long index;
+
if (kref_read(&job->s_fence->finished.refcount)) {
/* drm_sched_job_arm() has been called */
dma_fence_put(&job->s_fence->finished);
@@ -661,6 +759,12 @@ void drm_sched_job_cleanup(struct drm_sched_job *job)
}
job->s_fence = NULL;
+
+ xa_for_each(&job->dependencies, index, fence) {
+ dma_fence_put(fence);
+ }
+ xa_destroy(&job->dependencies);
+
}
EXPORT_SYMBOL(drm_sched_job_cleanup);
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index 83afc3aa8e2f..a47946f904b6 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -27,9 +27,12 @@
#include <drm/spsc_queue.h>
#include <linux/dma-fence.h>
#include <linux/completion.h>
+#include <linux/xarray.h>
#define MAX_WAIT_SCHED_ENTITY_Q_EMPTY msecs_to_jiffies(1000)
+struct drm_gem_object;
+
struct drm_gpu_scheduler;
struct drm_sched_rq;
@@ -198,6 +201,17 @@ struct drm_sched_job {
enum drm_sched_priority s_priority;
struct drm_sched_entity *entity;
struct dma_fence_cb cb;
+ /**
+ * @dependencies:
+ *
+ * Contains the dependencies as struct dma_fence for this job, see
+ * drm_sched_job_add_dependency() and
+ * drm_sched_job_add_implicit_dependencies().
+ */
+ struct xarray dependencies;
+
+ /** @last_dependency: tracks @dependencies as they signal */
+ unsigned long last_dependency;
};
static inline bool drm_sched_invalidate_job(struct drm_sched_job *s_job,
@@ -220,9 +234,15 @@ enum drm_gpu_sched_stat {
*/
struct drm_sched_backend_ops {
/**
- * @dependency: Called when the scheduler is considering scheduling
- * this job next, to get another struct dma_fence for this job to
- * block on. Once it returns NULL, run_job() may be called.
+ * @dependency:
+ *
+ * Called when the scheduler is considering scheduling this job next, to
+ * get another struct dma_fence for this job to block on. Once it
+ * returns NULL, run_job() may be called.
+ *
+ * If a driver exclusively uses drm_sched_job_add_dependency() and
+ * drm_sched_job_add_implicit_dependencies() this can be ommitted and
+ * left as NULL.
*/
struct dma_fence *(*dependency)(struct drm_sched_job *sched_job,
struct drm_sched_entity *s_entity);
@@ -349,6 +369,13 @@ int drm_sched_job_init(struct drm_sched_job *job,
struct drm_sched_entity *entity,
void *owner);
void drm_sched_job_arm(struct drm_sched_job *job);
+int drm_sched_job_add_dependency(struct drm_sched_job *job,
+ struct dma_fence *fence);
+int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
+ struct drm_gem_object *obj,
+ bool write);
+
+
void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
struct drm_gpu_scheduler **sched_list,
unsigned int num_sched_list);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 04/20] drm/sched: Add dependency tracking
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Boris Brezillon,
Steven Price, Melissa Wen, Daniel Vetter, David Airlie,
Daniel Vetter, Sumit Semwal, Christian König,
Andrey Grodzovsky, Lee Jones, Nirmoy Das, Luben Tuikov,
Alex Deucher, linux-media, linaro-mm-sig
Instead of just a callback we can just glue in the gem helpers that
panfrost, v3d and lima currently use. There's really not that many
ways to skin this cat.
v2/3: Rebased.
v4: Repaint this shed. The functions are now called _add_dependency()
and _add_implicit_dependency()
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v3)
Reviewed-by: Steven Price <steven.price@arm.com> (v1)
Acked-by: Melissa Wen <mwen@igalia.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Nirmoy Das <nirmoy.aiemd@gmail.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/gpu/drm/scheduler/sched_entity.c | 18 +++-
drivers/gpu/drm/scheduler/sched_main.c | 104 +++++++++++++++++++++++
include/drm/gpu_scheduler.h | 33 ++++++-
3 files changed, 149 insertions(+), 6 deletions(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 89e3f6eaf519..381fbf462ea7 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -211,6 +211,19 @@ static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
job->sched->ops->free_job(job);
}
+static struct dma_fence *
+drm_sched_job_dependency(struct drm_sched_job *job,
+ struct drm_sched_entity *entity)
+{
+ if (!xa_empty(&job->dependencies))
+ return xa_erase(&job->dependencies, job->last_dependency++);
+
+ if (job->sched->ops->dependency)
+ return job->sched->ops->dependency(job, entity);
+
+ return NULL;
+}
+
/**
* drm_sched_entity_kill_jobs - Make sure all remaining jobs are killed
*
@@ -229,7 +242,7 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
struct drm_sched_fence *s_fence = job->s_fence;
/* Wait for all dependencies to avoid data corruptions */
- while ((f = job->sched->ops->dependency(job, entity)))
+ while ((f = drm_sched_job_dependency(job, entity)))
dma_fence_wait(f, false);
drm_sched_fence_scheduled(s_fence);
@@ -419,7 +432,6 @@ static bool drm_sched_entity_add_dependency_cb(struct drm_sched_entity *entity)
*/
struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
{
- struct drm_gpu_scheduler *sched = entity->rq->sched;
struct drm_sched_job *sched_job;
sched_job = to_drm_sched_job(spsc_queue_peek(&entity->job_queue));
@@ -427,7 +439,7 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
return NULL;
while ((entity->dependency =
- sched->ops->dependency(sched_job, entity))) {
+ drm_sched_job_dependency(sched_job, entity))) {
trace_drm_sched_job_wait_dep(sched_job, entity->dependency);
if (drm_sched_entity_add_dependency_cb(entity))
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index 454cb6164bdc..f77456929139 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -603,6 +603,8 @@ int drm_sched_job_init(struct drm_sched_job *job,
INIT_LIST_HEAD(&job->list);
+ xa_init_flags(&job->dependencies, XA_FLAGS_ALLOC);
+
return 0;
}
EXPORT_SYMBOL(drm_sched_job_init);
@@ -637,6 +639,99 @@ void drm_sched_job_arm(struct drm_sched_job *job)
}
EXPORT_SYMBOL(drm_sched_job_arm);
+/**
+ * drm_sched_job_add_dependency - adds the fence as a job dependency
+ * @job: scheduler job to add the dependencies to
+ * @fence: the dma_fence to add to the list of dependencies.
+ *
+ * Note that @fence is consumed in both the success and error cases.
+ *
+ * Returns:
+ * 0 on success, or an error on failing to expand the array.
+ */
+int drm_sched_job_add_dependency(struct drm_sched_job *job,
+ struct dma_fence *fence)
+{
+ struct dma_fence *entry;
+ unsigned long index;
+ u32 id = 0;
+ int ret;
+
+ if (!fence)
+ return 0;
+
+ /* Deduplicate if we already depend on a fence from the same context.
+ * This lets the size of the array of deps scale with the number of
+ * engines involved, rather than the number of BOs.
+ */
+ xa_for_each(&job->dependencies, index, entry) {
+ if (entry->context != fence->context)
+ continue;
+
+ if (dma_fence_is_later(fence, entry)) {
+ dma_fence_put(entry);
+ xa_store(&job->dependencies, index, fence, GFP_KERNEL);
+ } else {
+ dma_fence_put(fence);
+ }
+ return 0;
+ }
+
+ ret = xa_alloc(&job->dependencies, &id, fence, xa_limit_32b, GFP_KERNEL);
+ if (ret != 0)
+ dma_fence_put(fence);
+
+ return ret;
+}
+EXPORT_SYMBOL(drm_sched_job_add_dependency);
+
+/**
+ * drm_sched_job_add_implicit_dependencies - adds implicit dependencies as job
+ * dependencies
+ * @job: scheduler job to add the dependencies to
+ * @obj: the gem object to add new dependencies from.
+ * @write: whether the job might write the object (so we need to depend on
+ * shared fences in the reservation object).
+ *
+ * This should be called after drm_gem_lock_reservations() on your array of
+ * GEM objects used in the job but before updating the reservations with your
+ * own fences.
+ *
+ * Returns:
+ * 0 on success, or an error on failing to expand the array.
+ */
+int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
+ struct drm_gem_object *obj,
+ bool write)
+{
+ int ret;
+ struct dma_fence **fences;
+ unsigned int i, fence_count;
+
+ if (!write) {
+ struct dma_fence *fence = dma_resv_get_excl_unlocked(obj->resv);
+
+ return drm_sched_job_add_dependency(job, fence);
+ }
+
+ ret = dma_resv_get_fences(obj->resv, NULL, &fence_count, &fences);
+ if (ret || !fence_count)
+ return ret;
+
+ for (i = 0; i < fence_count; i++) {
+ ret = drm_sched_job_add_dependency(job, fences[i]);
+ if (ret)
+ break;
+ }
+
+ for (; i < fence_count; i++)
+ dma_fence_put(fences[i]);
+ kfree(fences);
+ return ret;
+}
+EXPORT_SYMBOL(drm_sched_job_add_implicit_dependencies);
+
+
/**
* drm_sched_job_cleanup - clean up scheduler job resources
* @job: scheduler job to clean up
@@ -652,6 +747,9 @@ EXPORT_SYMBOL(drm_sched_job_arm);
*/
void drm_sched_job_cleanup(struct drm_sched_job *job)
{
+ struct dma_fence *fence;
+ unsigned long index;
+
if (kref_read(&job->s_fence->finished.refcount)) {
/* drm_sched_job_arm() has been called */
dma_fence_put(&job->s_fence->finished);
@@ -661,6 +759,12 @@ void drm_sched_job_cleanup(struct drm_sched_job *job)
}
job->s_fence = NULL;
+
+ xa_for_each(&job->dependencies, index, fence) {
+ dma_fence_put(fence);
+ }
+ xa_destroy(&job->dependencies);
+
}
EXPORT_SYMBOL(drm_sched_job_cleanup);
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index 83afc3aa8e2f..a47946f904b6 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -27,9 +27,12 @@
#include <drm/spsc_queue.h>
#include <linux/dma-fence.h>
#include <linux/completion.h>
+#include <linux/xarray.h>
#define MAX_WAIT_SCHED_ENTITY_Q_EMPTY msecs_to_jiffies(1000)
+struct drm_gem_object;
+
struct drm_gpu_scheduler;
struct drm_sched_rq;
@@ -198,6 +201,17 @@ struct drm_sched_job {
enum drm_sched_priority s_priority;
struct drm_sched_entity *entity;
struct dma_fence_cb cb;
+ /**
+ * @dependencies:
+ *
+ * Contains the dependencies as struct dma_fence for this job, see
+ * drm_sched_job_add_dependency() and
+ * drm_sched_job_add_implicit_dependencies().
+ */
+ struct xarray dependencies;
+
+ /** @last_dependency: tracks @dependencies as they signal */
+ unsigned long last_dependency;
};
static inline bool drm_sched_invalidate_job(struct drm_sched_job *s_job,
@@ -220,9 +234,15 @@ enum drm_gpu_sched_stat {
*/
struct drm_sched_backend_ops {
/**
- * @dependency: Called when the scheduler is considering scheduling
- * this job next, to get another struct dma_fence for this job to
- * block on. Once it returns NULL, run_job() may be called.
+ * @dependency:
+ *
+ * Called when the scheduler is considering scheduling this job next, to
+ * get another struct dma_fence for this job to block on. Once it
+ * returns NULL, run_job() may be called.
+ *
+ * If a driver exclusively uses drm_sched_job_add_dependency() and
+ * drm_sched_job_add_implicit_dependencies() this can be ommitted and
+ * left as NULL.
*/
struct dma_fence *(*dependency)(struct drm_sched_job *sched_job,
struct drm_sched_entity *s_entity);
@@ -349,6 +369,13 @@ int drm_sched_job_init(struct drm_sched_job *job,
struct drm_sched_entity *entity,
void *owner);
void drm_sched_job_arm(struct drm_sched_job *job);
+int drm_sched_job_add_dependency(struct drm_sched_job *job,
+ struct dma_fence *fence);
+int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
+ struct drm_gem_object *obj,
+ bool write);
+
+
void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
struct drm_gpu_scheduler **sched_list,
unsigned int num_sched_list);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 04/20] drm/sched: Add dependency tracking
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 13:47 ` Christian König
-1 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:47 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Boris Brezillon, Steven Price,
Melissa Wen, Daniel Vetter, David Airlie, Daniel Vetter,
Sumit Semwal, Andrey Grodzovsky, Lee Jones, Nirmoy Das,
Luben Tuikov, Alex Deucher, linux-media, linaro-mm-sig
Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> Instead of just a callback we can just glue in the gem helpers that
> panfrost, v3d and lima currently use. There's really not that many
> ways to skin this cat.
>
> v2/3: Rebased.
>
> v4: Repaint this shed. The functions are now called _add_dependency()
> and _add_implicit_dependency()
>
> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v3)
> Reviewed-by: Steven Price <steven.price@arm.com> (v1)
> Acked-by: Melissa Wen <mwen@igalia.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
> Cc: David Airlie <airlied@linux.ie>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Lee Jones <lee.jones@linaro.org>
> Cc: Nirmoy Das <nirmoy.aiemd@gmail.com>
> Cc: Boris Brezillon <boris.brezillon@collabora.com>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> ---
> drivers/gpu/drm/scheduler/sched_entity.c | 18 +++-
> drivers/gpu/drm/scheduler/sched_main.c | 104 +++++++++++++++++++++++
> include/drm/gpu_scheduler.h | 33 ++++++-
> 3 files changed, 149 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> index 89e3f6eaf519..381fbf462ea7 100644
> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> @@ -211,6 +211,19 @@ static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
> job->sched->ops->free_job(job);
> }
>
> +static struct dma_fence *
> +drm_sched_job_dependency(struct drm_sched_job *job,
> + struct drm_sched_entity *entity)
> +{
> + if (!xa_empty(&job->dependencies))
> + return xa_erase(&job->dependencies, job->last_dependency++);
> +
> + if (job->sched->ops->dependency)
> + return job->sched->ops->dependency(job, entity);
> +
> + return NULL;
> +}
> +
> /**
> * drm_sched_entity_kill_jobs - Make sure all remaining jobs are killed
> *
> @@ -229,7 +242,7 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
> struct drm_sched_fence *s_fence = job->s_fence;
>
> /* Wait for all dependencies to avoid data corruptions */
> - while ((f = job->sched->ops->dependency(job, entity)))
> + while ((f = drm_sched_job_dependency(job, entity)))
> dma_fence_wait(f, false);
>
> drm_sched_fence_scheduled(s_fence);
> @@ -419,7 +432,6 @@ static bool drm_sched_entity_add_dependency_cb(struct drm_sched_entity *entity)
> */
> struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
> {
> - struct drm_gpu_scheduler *sched = entity->rq->sched;
> struct drm_sched_job *sched_job;
>
> sched_job = to_drm_sched_job(spsc_queue_peek(&entity->job_queue));
> @@ -427,7 +439,7 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
> return NULL;
>
> while ((entity->dependency =
> - sched->ops->dependency(sched_job, entity))) {
> + drm_sched_job_dependency(sched_job, entity))) {
> trace_drm_sched_job_wait_dep(sched_job, entity->dependency);
>
> if (drm_sched_entity_add_dependency_cb(entity))
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> index 454cb6164bdc..f77456929139 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -603,6 +603,8 @@ int drm_sched_job_init(struct drm_sched_job *job,
>
> INIT_LIST_HEAD(&job->list);
>
> + xa_init_flags(&job->dependencies, XA_FLAGS_ALLOC);
> +
> return 0;
> }
> EXPORT_SYMBOL(drm_sched_job_init);
> @@ -637,6 +639,99 @@ void drm_sched_job_arm(struct drm_sched_job *job)
> }
> EXPORT_SYMBOL(drm_sched_job_arm);
>
> +/**
> + * drm_sched_job_add_dependency - adds the fence as a job dependency
> + * @job: scheduler job to add the dependencies to
> + * @fence: the dma_fence to add to the list of dependencies.
> + *
> + * Note that @fence is consumed in both the success and error cases.
> + *
> + * Returns:
> + * 0 on success, or an error on failing to expand the array.
> + */
> +int drm_sched_job_add_dependency(struct drm_sched_job *job,
> + struct dma_fence *fence)
> +{
> + struct dma_fence *entry;
> + unsigned long index;
> + u32 id = 0;
> + int ret;
> +
> + if (!fence)
> + return 0;
> +
> + /* Deduplicate if we already depend on a fence from the same context.
> + * This lets the size of the array of deps scale with the number of
> + * engines involved, rather than the number of BOs.
> + */
> + xa_for_each(&job->dependencies, index, entry) {
> + if (entry->context != fence->context)
> + continue;
> +
> + if (dma_fence_is_later(fence, entry)) {
> + dma_fence_put(entry);
> + xa_store(&job->dependencies, index, fence, GFP_KERNEL);
> + } else {
> + dma_fence_put(fence);
> + }
> + return 0;
> + }
> +
> + ret = xa_alloc(&job->dependencies, &id, fence, xa_limit_32b, GFP_KERNEL);
> + if (ret != 0)
> + dma_fence_put(fence);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL(drm_sched_job_add_dependency);
> +
> +/**
> + * drm_sched_job_add_implicit_dependencies - adds implicit dependencies as job
> + * dependencies
> + * @job: scheduler job to add the dependencies to
> + * @obj: the gem object to add new dependencies from.
> + * @write: whether the job might write the object (so we need to depend on
> + * shared fences in the reservation object).
> + *
> + * This should be called after drm_gem_lock_reservations() on your array of
> + * GEM objects used in the job but before updating the reservations with your
> + * own fences.
> + *
> + * Returns:
> + * 0 on success, or an error on failing to expand the array.
> + */
> +int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
> + struct drm_gem_object *obj,
> + bool write)
> +{
> + int ret;
> + struct dma_fence **fences;
> + unsigned int i, fence_count;
> +
> + if (!write) {
> + struct dma_fence *fence = dma_resv_get_excl_unlocked(obj->resv);
> +
> + return drm_sched_job_add_dependency(job, fence);
> + }
> +
> + ret = dma_resv_get_fences(obj->resv, NULL, &fence_count, &fences);
> + if (ret || !fence_count)
> + return ret;
> +
> + for (i = 0; i < fence_count; i++) {
> + ret = drm_sched_job_add_dependency(job, fences[i]);
> + if (ret)
> + break;
> + }
> +
> + for (; i < fence_count; i++)
> + dma_fence_put(fences[i]);
> + kfree(fences);
> + return ret;
> +}
> +EXPORT_SYMBOL(drm_sched_job_add_implicit_dependencies);
> +
> +
> /**
> * drm_sched_job_cleanup - clean up scheduler job resources
> * @job: scheduler job to clean up
> @@ -652,6 +747,9 @@ EXPORT_SYMBOL(drm_sched_job_arm);
> */
> void drm_sched_job_cleanup(struct drm_sched_job *job)
> {
> + struct dma_fence *fence;
> + unsigned long index;
> +
> if (kref_read(&job->s_fence->finished.refcount)) {
> /* drm_sched_job_arm() has been called */
> dma_fence_put(&job->s_fence->finished);
> @@ -661,6 +759,12 @@ void drm_sched_job_cleanup(struct drm_sched_job *job)
> }
>
> job->s_fence = NULL;
> +
> + xa_for_each(&job->dependencies, index, fence) {
> + dma_fence_put(fence);
> + }
> + xa_destroy(&job->dependencies);
> +
> }
> EXPORT_SYMBOL(drm_sched_job_cleanup);
>
> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> index 83afc3aa8e2f..a47946f904b6 100644
> --- a/include/drm/gpu_scheduler.h
> +++ b/include/drm/gpu_scheduler.h
> @@ -27,9 +27,12 @@
> #include <drm/spsc_queue.h>
> #include <linux/dma-fence.h>
> #include <linux/completion.h>
> +#include <linux/xarray.h>
>
> #define MAX_WAIT_SCHED_ENTITY_Q_EMPTY msecs_to_jiffies(1000)
>
> +struct drm_gem_object;
> +
> struct drm_gpu_scheduler;
> struct drm_sched_rq;
>
> @@ -198,6 +201,17 @@ struct drm_sched_job {
> enum drm_sched_priority s_priority;
> struct drm_sched_entity *entity;
> struct dma_fence_cb cb;
> + /**
> + * @dependencies:
> + *
> + * Contains the dependencies as struct dma_fence for this job, see
> + * drm_sched_job_add_dependency() and
> + * drm_sched_job_add_implicit_dependencies().
> + */
> + struct xarray dependencies;
> +
> + /** @last_dependency: tracks @dependencies as they signal */
> + unsigned long last_dependency;
> };
>
> static inline bool drm_sched_invalidate_job(struct drm_sched_job *s_job,
> @@ -220,9 +234,15 @@ enum drm_gpu_sched_stat {
> */
> struct drm_sched_backend_ops {
> /**
> - * @dependency: Called when the scheduler is considering scheduling
> - * this job next, to get another struct dma_fence for this job to
> - * block on. Once it returns NULL, run_job() may be called.
> + * @dependency:
> + *
> + * Called when the scheduler is considering scheduling this job next, to
> + * get another struct dma_fence for this job to block on. Once it
> + * returns NULL, run_job() may be called.
> + *
> + * If a driver exclusively uses drm_sched_job_add_dependency() and
> + * drm_sched_job_add_implicit_dependencies() this can be ommitted and
> + * left as NULL.
> */
> struct dma_fence *(*dependency)(struct drm_sched_job *sched_job,
> struct drm_sched_entity *s_entity);
> @@ -349,6 +369,13 @@ int drm_sched_job_init(struct drm_sched_job *job,
> struct drm_sched_entity *entity,
> void *owner);
> void drm_sched_job_arm(struct drm_sched_job *job);
> +int drm_sched_job_add_dependency(struct drm_sched_job *job,
> + struct dma_fence *fence);
> +int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
> + struct drm_gem_object *obj,
> + bool write);
> +
> +
> void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
> struct drm_gpu_scheduler **sched_list,
> unsigned int num_sched_list);
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 04/20] drm/sched: Add dependency tracking
@ 2021-08-05 13:47 ` Christian König
0 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:47 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Boris Brezillon, Steven Price,
Melissa Wen, Daniel Vetter, David Airlie, Daniel Vetter,
Sumit Semwal, Andrey Grodzovsky, Lee Jones, Nirmoy Das,
Luben Tuikov, Alex Deucher, linux-media, linaro-mm-sig
Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> Instead of just a callback we can just glue in the gem helpers that
> panfrost, v3d and lima currently use. There's really not that many
> ways to skin this cat.
>
> v2/3: Rebased.
>
> v4: Repaint this shed. The functions are now called _add_dependency()
> and _add_implicit_dependency()
>
> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v3)
> Reviewed-by: Steven Price <steven.price@arm.com> (v1)
> Acked-by: Melissa Wen <mwen@igalia.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
> Cc: David Airlie <airlied@linux.ie>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Lee Jones <lee.jones@linaro.org>
> Cc: Nirmoy Das <nirmoy.aiemd@gmail.com>
> Cc: Boris Brezillon <boris.brezillon@collabora.com>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> ---
> drivers/gpu/drm/scheduler/sched_entity.c | 18 +++-
> drivers/gpu/drm/scheduler/sched_main.c | 104 +++++++++++++++++++++++
> include/drm/gpu_scheduler.h | 33 ++++++-
> 3 files changed, 149 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> index 89e3f6eaf519..381fbf462ea7 100644
> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> @@ -211,6 +211,19 @@ static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
> job->sched->ops->free_job(job);
> }
>
> +static struct dma_fence *
> +drm_sched_job_dependency(struct drm_sched_job *job,
> + struct drm_sched_entity *entity)
> +{
> + if (!xa_empty(&job->dependencies))
> + return xa_erase(&job->dependencies, job->last_dependency++);
> +
> + if (job->sched->ops->dependency)
> + return job->sched->ops->dependency(job, entity);
> +
> + return NULL;
> +}
> +
> /**
> * drm_sched_entity_kill_jobs - Make sure all remaining jobs are killed
> *
> @@ -229,7 +242,7 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
> struct drm_sched_fence *s_fence = job->s_fence;
>
> /* Wait for all dependencies to avoid data corruptions */
> - while ((f = job->sched->ops->dependency(job, entity)))
> + while ((f = drm_sched_job_dependency(job, entity)))
> dma_fence_wait(f, false);
>
> drm_sched_fence_scheduled(s_fence);
> @@ -419,7 +432,6 @@ static bool drm_sched_entity_add_dependency_cb(struct drm_sched_entity *entity)
> */
> struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
> {
> - struct drm_gpu_scheduler *sched = entity->rq->sched;
> struct drm_sched_job *sched_job;
>
> sched_job = to_drm_sched_job(spsc_queue_peek(&entity->job_queue));
> @@ -427,7 +439,7 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
> return NULL;
>
> while ((entity->dependency =
> - sched->ops->dependency(sched_job, entity))) {
> + drm_sched_job_dependency(sched_job, entity))) {
> trace_drm_sched_job_wait_dep(sched_job, entity->dependency);
>
> if (drm_sched_entity_add_dependency_cb(entity))
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> index 454cb6164bdc..f77456929139 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -603,6 +603,8 @@ int drm_sched_job_init(struct drm_sched_job *job,
>
> INIT_LIST_HEAD(&job->list);
>
> + xa_init_flags(&job->dependencies, XA_FLAGS_ALLOC);
> +
> return 0;
> }
> EXPORT_SYMBOL(drm_sched_job_init);
> @@ -637,6 +639,99 @@ void drm_sched_job_arm(struct drm_sched_job *job)
> }
> EXPORT_SYMBOL(drm_sched_job_arm);
>
> +/**
> + * drm_sched_job_add_dependency - adds the fence as a job dependency
> + * @job: scheduler job to add the dependencies to
> + * @fence: the dma_fence to add to the list of dependencies.
> + *
> + * Note that @fence is consumed in both the success and error cases.
> + *
> + * Returns:
> + * 0 on success, or an error on failing to expand the array.
> + */
> +int drm_sched_job_add_dependency(struct drm_sched_job *job,
> + struct dma_fence *fence)
> +{
> + struct dma_fence *entry;
> + unsigned long index;
> + u32 id = 0;
> + int ret;
> +
> + if (!fence)
> + return 0;
> +
> + /* Deduplicate if we already depend on a fence from the same context.
> + * This lets the size of the array of deps scale with the number of
> + * engines involved, rather than the number of BOs.
> + */
> + xa_for_each(&job->dependencies, index, entry) {
> + if (entry->context != fence->context)
> + continue;
> +
> + if (dma_fence_is_later(fence, entry)) {
> + dma_fence_put(entry);
> + xa_store(&job->dependencies, index, fence, GFP_KERNEL);
> + } else {
> + dma_fence_put(fence);
> + }
> + return 0;
> + }
> +
> + ret = xa_alloc(&job->dependencies, &id, fence, xa_limit_32b, GFP_KERNEL);
> + if (ret != 0)
> + dma_fence_put(fence);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL(drm_sched_job_add_dependency);
> +
> +/**
> + * drm_sched_job_add_implicit_dependencies - adds implicit dependencies as job
> + * dependencies
> + * @job: scheduler job to add the dependencies to
> + * @obj: the gem object to add new dependencies from.
> + * @write: whether the job might write the object (so we need to depend on
> + * shared fences in the reservation object).
> + *
> + * This should be called after drm_gem_lock_reservations() on your array of
> + * GEM objects used in the job but before updating the reservations with your
> + * own fences.
> + *
> + * Returns:
> + * 0 on success, or an error on failing to expand the array.
> + */
> +int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
> + struct drm_gem_object *obj,
> + bool write)
> +{
> + int ret;
> + struct dma_fence **fences;
> + unsigned int i, fence_count;
> +
> + if (!write) {
> + struct dma_fence *fence = dma_resv_get_excl_unlocked(obj->resv);
> +
> + return drm_sched_job_add_dependency(job, fence);
> + }
> +
> + ret = dma_resv_get_fences(obj->resv, NULL, &fence_count, &fences);
> + if (ret || !fence_count)
> + return ret;
> +
> + for (i = 0; i < fence_count; i++) {
> + ret = drm_sched_job_add_dependency(job, fences[i]);
> + if (ret)
> + break;
> + }
> +
> + for (; i < fence_count; i++)
> + dma_fence_put(fences[i]);
> + kfree(fences);
> + return ret;
> +}
> +EXPORT_SYMBOL(drm_sched_job_add_implicit_dependencies);
> +
> +
> /**
> * drm_sched_job_cleanup - clean up scheduler job resources
> * @job: scheduler job to clean up
> @@ -652,6 +747,9 @@ EXPORT_SYMBOL(drm_sched_job_arm);
> */
> void drm_sched_job_cleanup(struct drm_sched_job *job)
> {
> + struct dma_fence *fence;
> + unsigned long index;
> +
> if (kref_read(&job->s_fence->finished.refcount)) {
> /* drm_sched_job_arm() has been called */
> dma_fence_put(&job->s_fence->finished);
> @@ -661,6 +759,12 @@ void drm_sched_job_cleanup(struct drm_sched_job *job)
> }
>
> job->s_fence = NULL;
> +
> + xa_for_each(&job->dependencies, index, fence) {
> + dma_fence_put(fence);
> + }
> + xa_destroy(&job->dependencies);
> +
> }
> EXPORT_SYMBOL(drm_sched_job_cleanup);
>
> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> index 83afc3aa8e2f..a47946f904b6 100644
> --- a/include/drm/gpu_scheduler.h
> +++ b/include/drm/gpu_scheduler.h
> @@ -27,9 +27,12 @@
> #include <drm/spsc_queue.h>
> #include <linux/dma-fence.h>
> #include <linux/completion.h>
> +#include <linux/xarray.h>
>
> #define MAX_WAIT_SCHED_ENTITY_Q_EMPTY msecs_to_jiffies(1000)
>
> +struct drm_gem_object;
> +
> struct drm_gpu_scheduler;
> struct drm_sched_rq;
>
> @@ -198,6 +201,17 @@ struct drm_sched_job {
> enum drm_sched_priority s_priority;
> struct drm_sched_entity *entity;
> struct dma_fence_cb cb;
> + /**
> + * @dependencies:
> + *
> + * Contains the dependencies as struct dma_fence for this job, see
> + * drm_sched_job_add_dependency() and
> + * drm_sched_job_add_implicit_dependencies().
> + */
> + struct xarray dependencies;
> +
> + /** @last_dependency: tracks @dependencies as they signal */
> + unsigned long last_dependency;
> };
>
> static inline bool drm_sched_invalidate_job(struct drm_sched_job *s_job,
> @@ -220,9 +234,15 @@ enum drm_gpu_sched_stat {
> */
> struct drm_sched_backend_ops {
> /**
> - * @dependency: Called when the scheduler is considering scheduling
> - * this job next, to get another struct dma_fence for this job to
> - * block on. Once it returns NULL, run_job() may be called.
> + * @dependency:
> + *
> + * Called when the scheduler is considering scheduling this job next, to
> + * get another struct dma_fence for this job to block on. Once it
> + * returns NULL, run_job() may be called.
> + *
> + * If a driver exclusively uses drm_sched_job_add_dependency() and
> + * drm_sched_job_add_implicit_dependencies() this can be ommitted and
> + * left as NULL.
> */
> struct dma_fence *(*dependency)(struct drm_sched_job *sched_job,
> struct drm_sched_entity *s_entity);
> @@ -349,6 +369,13 @@ int drm_sched_job_init(struct drm_sched_job *job,
> struct drm_sched_entity *entity,
> void *owner);
> void drm_sched_job_arm(struct drm_sched_job *job);
> +int drm_sched_job_add_dependency(struct drm_sched_job *job,
> + struct dma_fence *fence);
> +int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
> + struct drm_gem_object *obj,
> + bool write);
> +
> +
> void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
> struct drm_gpu_scheduler **sched_list,
> unsigned int num_sched_list);
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 05/20] drm/sched: drop entity parameter from drm_sched_push_job
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Emma Anholt,
Melissa Wen, Steven Price, Boris Brezillon, Daniel Vetter,
Lucas Stach, Russell King, Christian Gmeiner, Qiang Yu,
Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig, David Airlie,
Daniel Vetter, Sumit Semwal, Christian König, Alex Deucher,
Nirmoy Das, Dave Airlie, Chen Li, Lee Jones, Deepak R Varma,
Kevin Wang, Luben Tuikov, Marek Olšák,
Maarten Lankhorst, Andrey Grodzovsky, Dennis Li, etnaviv, lima,
linux-media, linaro-mm-sig, Rob Clark, Sean Paul, linux-arm-msm,
freedreno
Originally a job was only bound to the queue when we pushed this, but
now that's done in drm_sched_job_init, making that parameter entirely
redundant.
Remove it.
The same applies to the context parameter in
lima_sched_context_queue_task, simplify that too.
v2:
Rebase on top of msm adopting drm/sched
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Melissa Wen <mwen@igalia.com>
Reviewed-by: Steven Price <steven.price@arm.com> (v1)
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v1)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: Qiang Yu <yuq825@gmail.com>
Cc: Rob Herring <robh@kernel.org>
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Cc: Emma Anholt <emma@anholt.net>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Nirmoy Das <nirmoy.das@amd.com>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Chen Li <chenli@uniontech.com>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Deepak R Varma <mh12gx2825@gmail.com>
Cc: Kevin Wang <kevin1.wang@amd.com>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: "Marek Olšák" <marek.olsak@amd.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Dennis Li <Dennis.Li@amd.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: etnaviv@lists.freedesktop.org
Cc: lima@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Melissa Wen <mwen@igalia.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
---
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +-
drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +-
drivers/gpu/drm/lima/lima_gem.c | 3 +--
drivers/gpu/drm/lima/lima_sched.c | 5 ++---
drivers/gpu/drm/lima/lima_sched.h | 3 +--
drivers/gpu/drm/msm/msm_gem_submit.c | 2 +-
drivers/gpu/drm/panfrost/panfrost_job.c | 2 +-
drivers/gpu/drm/scheduler/sched_entity.c | 6 ++----
drivers/gpu/drm/v3d/v3d_gem.c | 2 +-
include/drm/gpu_scheduler.h | 3 +--
11 files changed, 13 insertions(+), 19 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 32e80bc6af22..1d8a914108af 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -1267,7 +1267,7 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
trace_amdgpu_cs_ioctl(job);
amdgpu_vm_bo_trace_cs(&fpriv->vm, &p->ticket);
- drm_sched_entity_push_job(&job->base, entity);
+ drm_sched_entity_push_job(&job->base);
amdgpu_vm_move_to_lru_tail(p->adev, &fpriv->vm);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index 5ddb955d2315..b8609cccc9c1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -174,7 +174,7 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
*f = dma_fence_get(&job->base.s_fence->finished);
amdgpu_job_free_resources(job);
- drm_sched_entity_push_job(&job->base, entity);
+ drm_sched_entity_push_job(&job->base);
return 0;
}
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index 05f412204118..180bb633d5c5 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -178,7 +178,7 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
/* the scheduler holds on to the job now */
kref_get(&submit->refcount);
- drm_sched_entity_push_job(&submit->sched_job, sched_entity);
+ drm_sched_entity_push_job(&submit->sched_job);
out_unlock:
mutex_unlock(&submit->gpu->fence_lock);
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index de62966243cd..c528f40981bb 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -359,8 +359,7 @@ int lima_gem_submit(struct drm_file *file, struct lima_submit *submit)
goto err_out2;
}
- fence = lima_sched_context_queue_task(
- submit->ctx->context + submit->pipe, submit->task);
+ fence = lima_sched_context_queue_task(submit->task);
for (i = 0; i < submit->nr_bos; i++) {
if (submit->bos[i].flags & LIMA_SUBMIT_BO_WRITE)
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index 38f755580507..e968b5a8f0b0 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -177,13 +177,12 @@ void lima_sched_context_fini(struct lima_sched_pipe *pipe,
drm_sched_entity_fini(&context->base);
}
-struct dma_fence *lima_sched_context_queue_task(struct lima_sched_context *context,
- struct lima_sched_task *task)
+struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task)
{
struct dma_fence *fence = dma_fence_get(&task->base.s_fence->finished);
trace_lima_task_submit(task);
- drm_sched_entity_push_job(&task->base, &context->base);
+ drm_sched_entity_push_job(&task->base);
return fence;
}
diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
index 90f03c48ef4a..ac70006b0e26 100644
--- a/drivers/gpu/drm/lima/lima_sched.h
+++ b/drivers/gpu/drm/lima/lima_sched.h
@@ -98,8 +98,7 @@ int lima_sched_context_init(struct lima_sched_pipe *pipe,
atomic_t *guilty);
void lima_sched_context_fini(struct lima_sched_pipe *pipe,
struct lima_sched_context *context);
-struct dma_fence *lima_sched_context_queue_task(struct lima_sched_context *context,
- struct lima_sched_task *task);
+struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task);
int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name);
void lima_sched_pipe_fini(struct lima_sched_pipe *pipe);
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index d0ed4ddc509e..96cea0ba4cfd 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -909,7 +909,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
/* The scheduler owns a ref now: */
msm_gem_submit_get(submit);
- drm_sched_entity_push_job(&submit->base, &queue->entity);
+ drm_sched_entity_push_job(&submit->base);
args->fence = submit->fence_id;
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 2992dc85325f..4bc962763e1f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -301,7 +301,7 @@ int panfrost_job_push(struct panfrost_job *job)
kref_get(&job->refcount); /* put by scheduler job completion */
- drm_sched_entity_push_job(&job->base, entity);
+ drm_sched_entity_push_job(&job->base);
mutex_unlock(&pfdev->sched_lock);
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 381fbf462ea7..e4d33db1eb45 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -516,9 +516,7 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
/**
* drm_sched_entity_push_job - Submit a job to the entity's job queue
- *
* @sched_job: job to submit
- * @entity: scheduler entity
*
* Note: To guarantee that the order of insertion to queue matches the job's
* fence sequence number this function should be called with drm_sched_job_arm()
@@ -526,9 +524,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
*
* Returns 0 for success, negative error code otherwise.
*/
-void drm_sched_entity_push_job(struct drm_sched_job *sched_job,
- struct drm_sched_entity *entity)
+void drm_sched_entity_push_job(struct drm_sched_job *sched_job)
{
+ struct drm_sched_entity *entity = sched_job->entity;
bool first;
trace_drm_sched_job(sched_job, entity);
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 2e808097b4d1..957228bef29c 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -487,7 +487,7 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
/* put by scheduler job completion */
kref_get(&job->refcount);
- drm_sched_entity_push_job(&job->base, &v3d_priv->sched_entity[queue]);
+ drm_sched_entity_push_job(&job->base);
return 0;
}
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index a47946f904b6..b72f73b375a2 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -409,8 +409,7 @@ void drm_sched_entity_fini(struct drm_sched_entity *entity);
void drm_sched_entity_destroy(struct drm_sched_entity *entity);
void drm_sched_entity_select_rq(struct drm_sched_entity *entity);
struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity);
-void drm_sched_entity_push_job(struct drm_sched_job *sched_job,
- struct drm_sched_entity *entity);
+void drm_sched_entity_push_job(struct drm_sched_job *sched_job);
void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
enum drm_sched_priority priority);
bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 05/20] drm/sched: drop entity parameter from drm_sched_push_job
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Emma Anholt,
Melissa Wen, Steven Price, Boris Brezillon, Daniel Vetter,
Lucas Stach, Russell King, Christian Gmeiner, Qiang Yu,
Rob Herring, Tomeu Vizoso, Alyssa Rosenzweig, David Airlie,
Daniel Vetter, Sumit Semwal, Christian König, Alex Deucher,
Nirmoy Das, Dave Airlie, Chen Li, Lee Jones, Deepak R Varma,
Kevin Wang, Luben Tuikov, Marek Olšák,
Maarten Lankhorst, Andrey Grodzovsky, Dennis Li, etnaviv, lima,
linux-media, linaro-mm-sig, Rob Clark, Sean Paul, linux-arm-msm,
freedreno
Originally a job was only bound to the queue when we pushed this, but
now that's done in drm_sched_job_init, making that parameter entirely
redundant.
Remove it.
The same applies to the context parameter in
lima_sched_context_queue_task, simplify that too.
v2:
Rebase on top of msm adopting drm/sched
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Melissa Wen <mwen@igalia.com>
Reviewed-by: Steven Price <steven.price@arm.com> (v1)
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v1)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: Qiang Yu <yuq825@gmail.com>
Cc: Rob Herring <robh@kernel.org>
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Cc: Emma Anholt <emma@anholt.net>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Nirmoy Das <nirmoy.das@amd.com>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Chen Li <chenli@uniontech.com>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Deepak R Varma <mh12gx2825@gmail.com>
Cc: Kevin Wang <kevin1.wang@amd.com>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: "Marek Olšák" <marek.olsak@amd.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Dennis Li <Dennis.Li@amd.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: etnaviv@lists.freedesktop.org
Cc: lima@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Melissa Wen <mwen@igalia.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
---
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +-
drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +-
drivers/gpu/drm/lima/lima_gem.c | 3 +--
drivers/gpu/drm/lima/lima_sched.c | 5 ++---
drivers/gpu/drm/lima/lima_sched.h | 3 +--
drivers/gpu/drm/msm/msm_gem_submit.c | 2 +-
drivers/gpu/drm/panfrost/panfrost_job.c | 2 +-
drivers/gpu/drm/scheduler/sched_entity.c | 6 ++----
drivers/gpu/drm/v3d/v3d_gem.c | 2 +-
include/drm/gpu_scheduler.h | 3 +--
11 files changed, 13 insertions(+), 19 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 32e80bc6af22..1d8a914108af 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -1267,7 +1267,7 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
trace_amdgpu_cs_ioctl(job);
amdgpu_vm_bo_trace_cs(&fpriv->vm, &p->ticket);
- drm_sched_entity_push_job(&job->base, entity);
+ drm_sched_entity_push_job(&job->base);
amdgpu_vm_move_to_lru_tail(p->adev, &fpriv->vm);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index 5ddb955d2315..b8609cccc9c1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -174,7 +174,7 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
*f = dma_fence_get(&job->base.s_fence->finished);
amdgpu_job_free_resources(job);
- drm_sched_entity_push_job(&job->base, entity);
+ drm_sched_entity_push_job(&job->base);
return 0;
}
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index 05f412204118..180bb633d5c5 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -178,7 +178,7 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
/* the scheduler holds on to the job now */
kref_get(&submit->refcount);
- drm_sched_entity_push_job(&submit->sched_job, sched_entity);
+ drm_sched_entity_push_job(&submit->sched_job);
out_unlock:
mutex_unlock(&submit->gpu->fence_lock);
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index de62966243cd..c528f40981bb 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -359,8 +359,7 @@ int lima_gem_submit(struct drm_file *file, struct lima_submit *submit)
goto err_out2;
}
- fence = lima_sched_context_queue_task(
- submit->ctx->context + submit->pipe, submit->task);
+ fence = lima_sched_context_queue_task(submit->task);
for (i = 0; i < submit->nr_bos; i++) {
if (submit->bos[i].flags & LIMA_SUBMIT_BO_WRITE)
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index 38f755580507..e968b5a8f0b0 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -177,13 +177,12 @@ void lima_sched_context_fini(struct lima_sched_pipe *pipe,
drm_sched_entity_fini(&context->base);
}
-struct dma_fence *lima_sched_context_queue_task(struct lima_sched_context *context,
- struct lima_sched_task *task)
+struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task)
{
struct dma_fence *fence = dma_fence_get(&task->base.s_fence->finished);
trace_lima_task_submit(task);
- drm_sched_entity_push_job(&task->base, &context->base);
+ drm_sched_entity_push_job(&task->base);
return fence;
}
diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
index 90f03c48ef4a..ac70006b0e26 100644
--- a/drivers/gpu/drm/lima/lima_sched.h
+++ b/drivers/gpu/drm/lima/lima_sched.h
@@ -98,8 +98,7 @@ int lima_sched_context_init(struct lima_sched_pipe *pipe,
atomic_t *guilty);
void lima_sched_context_fini(struct lima_sched_pipe *pipe,
struct lima_sched_context *context);
-struct dma_fence *lima_sched_context_queue_task(struct lima_sched_context *context,
- struct lima_sched_task *task);
+struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task);
int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name);
void lima_sched_pipe_fini(struct lima_sched_pipe *pipe);
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index d0ed4ddc509e..96cea0ba4cfd 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -909,7 +909,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
/* The scheduler owns a ref now: */
msm_gem_submit_get(submit);
- drm_sched_entity_push_job(&submit->base, &queue->entity);
+ drm_sched_entity_push_job(&submit->base);
args->fence = submit->fence_id;
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 2992dc85325f..4bc962763e1f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -301,7 +301,7 @@ int panfrost_job_push(struct panfrost_job *job)
kref_get(&job->refcount); /* put by scheduler job completion */
- drm_sched_entity_push_job(&job->base, entity);
+ drm_sched_entity_push_job(&job->base);
mutex_unlock(&pfdev->sched_lock);
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 381fbf462ea7..e4d33db1eb45 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -516,9 +516,7 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
/**
* drm_sched_entity_push_job - Submit a job to the entity's job queue
- *
* @sched_job: job to submit
- * @entity: scheduler entity
*
* Note: To guarantee that the order of insertion to queue matches the job's
* fence sequence number this function should be called with drm_sched_job_arm()
@@ -526,9 +524,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
*
* Returns 0 for success, negative error code otherwise.
*/
-void drm_sched_entity_push_job(struct drm_sched_job *sched_job,
- struct drm_sched_entity *entity)
+void drm_sched_entity_push_job(struct drm_sched_job *sched_job)
{
+ struct drm_sched_entity *entity = sched_job->entity;
bool first;
trace_drm_sched_job(sched_job, entity);
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 2e808097b4d1..957228bef29c 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -487,7 +487,7 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
/* put by scheduler job completion */
kref_get(&job->refcount);
- drm_sched_entity_push_job(&job->base, &v3d_priv->sched_entity[queue]);
+ drm_sched_entity_push_job(&job->base);
return 0;
}
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index a47946f904b6..b72f73b375a2 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -409,8 +409,7 @@ void drm_sched_entity_fini(struct drm_sched_entity *entity);
void drm_sched_entity_destroy(struct drm_sched_entity *entity);
void drm_sched_entity_select_rq(struct drm_sched_entity *entity);
struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity);
-void drm_sched_entity_push_job(struct drm_sched_job *sched_job,
- struct drm_sched_entity *entity);
+void drm_sched_entity_push_job(struct drm_sched_job *sched_job);
void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
enum drm_sched_priority priority);
bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 05/20] drm/sched: drop entity parameter from drm_sched_push_job
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 13:48 ` Christian König
-1 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:48 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Emma Anholt, Melissa Wen,
Steven Price, Boris Brezillon, Daniel Vetter, Lucas Stach,
Russell King, Christian Gmeiner, Qiang Yu, Rob Herring,
Tomeu Vizoso, Alyssa Rosenzweig, David Airlie, Daniel Vetter,
Sumit Semwal, Alex Deucher, Nirmoy Das, Dave Airlie, Chen Li,
Lee Jones, Deepak R Varma, Kevin Wang, Luben Tuikov,
Marek Olšák, Maarten Lankhorst, Andrey Grodzovsky,
Dennis Li, etnaviv, lima, linux-media, linaro-mm-sig, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> Originally a job was only bound to the queue when we pushed this, but
> now that's done in drm_sched_job_init, making that parameter entirely
> redundant.
>
> Remove it.
>
> The same applies to the context parameter in
> lima_sched_context_queue_task, simplify that too.
>
> v2:
> Rebase on top of msm adopting drm/sched
>
> Acked-by: Emma Anholt <emma@anholt.net>
> Acked-by: Melissa Wen <mwen@igalia.com>
> Reviewed-by: Steven Price <steven.price@arm.com> (v1)
> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v1)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: Emma Anholt <emma@anholt.net>
> Cc: David Airlie <airlied@linux.ie>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Nirmoy Das <nirmoy.das@amd.com>
> Cc: Dave Airlie <airlied@redhat.com>
> Cc: Chen Li <chenli@uniontech.com>
> Cc: Lee Jones <lee.jones@linaro.org>
> Cc: Deepak R Varma <mh12gx2825@gmail.com>
> Cc: Kevin Wang <kevin1.wang@amd.com>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: "Marek Olšák" <marek.olsak@amd.com>
> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Dennis Li <Dennis.Li@amd.com>
> Cc: Boris Brezillon <boris.brezillon@collabora.com>
> Cc: etnaviv@lists.freedesktop.org
> Cc: lima@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Melissa Wen <mwen@igalia.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +-
> drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +-
> drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +-
> drivers/gpu/drm/lima/lima_gem.c | 3 +--
> drivers/gpu/drm/lima/lima_sched.c | 5 ++---
> drivers/gpu/drm/lima/lima_sched.h | 3 +--
> drivers/gpu/drm/msm/msm_gem_submit.c | 2 +-
> drivers/gpu/drm/panfrost/panfrost_job.c | 2 +-
> drivers/gpu/drm/scheduler/sched_entity.c | 6 ++----
> drivers/gpu/drm/v3d/v3d_gem.c | 2 +-
> include/drm/gpu_scheduler.h | 3 +--
> 11 files changed, 13 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> index 32e80bc6af22..1d8a914108af 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> @@ -1267,7 +1267,7 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
>
> trace_amdgpu_cs_ioctl(job);
> amdgpu_vm_bo_trace_cs(&fpriv->vm, &p->ticket);
> - drm_sched_entity_push_job(&job->base, entity);
> + drm_sched_entity_push_job(&job->base);
>
> amdgpu_vm_move_to_lru_tail(p->adev, &fpriv->vm);
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> index 5ddb955d2315..b8609cccc9c1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> @@ -174,7 +174,7 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
>
> *f = dma_fence_get(&job->base.s_fence->finished);
> amdgpu_job_free_resources(job);
> - drm_sched_entity_push_job(&job->base, entity);
> + drm_sched_entity_push_job(&job->base);
>
> return 0;
> }
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> index 05f412204118..180bb633d5c5 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> @@ -178,7 +178,7 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> /* the scheduler holds on to the job now */
> kref_get(&submit->refcount);
>
> - drm_sched_entity_push_job(&submit->sched_job, sched_entity);
> + drm_sched_entity_push_job(&submit->sched_job);
>
> out_unlock:
> mutex_unlock(&submit->gpu->fence_lock);
> diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
> index de62966243cd..c528f40981bb 100644
> --- a/drivers/gpu/drm/lima/lima_gem.c
> +++ b/drivers/gpu/drm/lima/lima_gem.c
> @@ -359,8 +359,7 @@ int lima_gem_submit(struct drm_file *file, struct lima_submit *submit)
> goto err_out2;
> }
>
> - fence = lima_sched_context_queue_task(
> - submit->ctx->context + submit->pipe, submit->task);
> + fence = lima_sched_context_queue_task(submit->task);
>
> for (i = 0; i < submit->nr_bos; i++) {
> if (submit->bos[i].flags & LIMA_SUBMIT_BO_WRITE)
> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> index 38f755580507..e968b5a8f0b0 100644
> --- a/drivers/gpu/drm/lima/lima_sched.c
> +++ b/drivers/gpu/drm/lima/lima_sched.c
> @@ -177,13 +177,12 @@ void lima_sched_context_fini(struct lima_sched_pipe *pipe,
> drm_sched_entity_fini(&context->base);
> }
>
> -struct dma_fence *lima_sched_context_queue_task(struct lima_sched_context *context,
> - struct lima_sched_task *task)
> +struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task)
> {
> struct dma_fence *fence = dma_fence_get(&task->base.s_fence->finished);
>
> trace_lima_task_submit(task);
> - drm_sched_entity_push_job(&task->base, &context->base);
> + drm_sched_entity_push_job(&task->base);
> return fence;
> }
>
> diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
> index 90f03c48ef4a..ac70006b0e26 100644
> --- a/drivers/gpu/drm/lima/lima_sched.h
> +++ b/drivers/gpu/drm/lima/lima_sched.h
> @@ -98,8 +98,7 @@ int lima_sched_context_init(struct lima_sched_pipe *pipe,
> atomic_t *guilty);
> void lima_sched_context_fini(struct lima_sched_pipe *pipe,
> struct lima_sched_context *context);
> -struct dma_fence *lima_sched_context_queue_task(struct lima_sched_context *context,
> - struct lima_sched_task *task);
> +struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task);
>
> int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name);
> void lima_sched_pipe_fini(struct lima_sched_pipe *pipe);
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index d0ed4ddc509e..96cea0ba4cfd 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -909,7 +909,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> /* The scheduler owns a ref now: */
> msm_gem_submit_get(submit);
>
> - drm_sched_entity_push_job(&submit->base, &queue->entity);
> + drm_sched_entity_push_job(&submit->base);
>
> args->fence = submit->fence_id;
>
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> index 2992dc85325f..4bc962763e1f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> @@ -301,7 +301,7 @@ int panfrost_job_push(struct panfrost_job *job)
>
> kref_get(&job->refcount); /* put by scheduler job completion */
>
> - drm_sched_entity_push_job(&job->base, entity);
> + drm_sched_entity_push_job(&job->base);
>
> mutex_unlock(&pfdev->sched_lock);
>
> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> index 381fbf462ea7..e4d33db1eb45 100644
> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> @@ -516,9 +516,7 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
>
> /**
> * drm_sched_entity_push_job - Submit a job to the entity's job queue
> - *
> * @sched_job: job to submit
> - * @entity: scheduler entity
> *
> * Note: To guarantee that the order of insertion to queue matches the job's
> * fence sequence number this function should be called with drm_sched_job_arm()
> @@ -526,9 +524,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
> *
> * Returns 0 for success, negative error code otherwise.
> */
> -void drm_sched_entity_push_job(struct drm_sched_job *sched_job,
> - struct drm_sched_entity *entity)
> +void drm_sched_entity_push_job(struct drm_sched_job *sched_job)
> {
> + struct drm_sched_entity *entity = sched_job->entity;
> bool first;
>
> trace_drm_sched_job(sched_job, entity);
> diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
> index 2e808097b4d1..957228bef29c 100644
> --- a/drivers/gpu/drm/v3d/v3d_gem.c
> +++ b/drivers/gpu/drm/v3d/v3d_gem.c
> @@ -487,7 +487,7 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
> /* put by scheduler job completion */
> kref_get(&job->refcount);
>
> - drm_sched_entity_push_job(&job->base, &v3d_priv->sched_entity[queue]);
> + drm_sched_entity_push_job(&job->base);
>
> return 0;
> }
> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> index a47946f904b6..b72f73b375a2 100644
> --- a/include/drm/gpu_scheduler.h
> +++ b/include/drm/gpu_scheduler.h
> @@ -409,8 +409,7 @@ void drm_sched_entity_fini(struct drm_sched_entity *entity);
> void drm_sched_entity_destroy(struct drm_sched_entity *entity);
> void drm_sched_entity_select_rq(struct drm_sched_entity *entity);
> struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity);
> -void drm_sched_entity_push_job(struct drm_sched_job *sched_job,
> - struct drm_sched_entity *entity);
> +void drm_sched_entity_push_job(struct drm_sched_job *sched_job);
> void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> enum drm_sched_priority priority);
> bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 05/20] drm/sched: drop entity parameter from drm_sched_push_job
@ 2021-08-05 13:48 ` Christian König
0 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:48 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Emma Anholt, Melissa Wen,
Steven Price, Boris Brezillon, Daniel Vetter, Lucas Stach,
Russell King, Christian Gmeiner, Qiang Yu, Rob Herring,
Tomeu Vizoso, Alyssa Rosenzweig, David Airlie, Daniel Vetter,
Sumit Semwal, Alex Deucher, Nirmoy Das, Dave Airlie, Chen Li,
Lee Jones, Deepak R Varma, Kevin Wang, Luben Tuikov,
Marek Olšák, Maarten Lankhorst, Andrey Grodzovsky,
Dennis Li, etnaviv, lima, linux-media, linaro-mm-sig, Rob Clark,
Sean Paul, linux-arm-msm, freedreno
Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> Originally a job was only bound to the queue when we pushed this, but
> now that's done in drm_sched_job_init, making that parameter entirely
> redundant.
>
> Remove it.
>
> The same applies to the context parameter in
> lima_sched_context_queue_task, simplify that too.
>
> v2:
> Rebase on top of msm adopting drm/sched
>
> Acked-by: Emma Anholt <emma@anholt.net>
> Acked-by: Melissa Wen <mwen@igalia.com>
> Reviewed-by: Steven Price <steven.price@arm.com> (v1)
> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v1)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: Emma Anholt <emma@anholt.net>
> Cc: David Airlie <airlied@linux.ie>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Cc: Nirmoy Das <nirmoy.das@amd.com>
> Cc: Dave Airlie <airlied@redhat.com>
> Cc: Chen Li <chenli@uniontech.com>
> Cc: Lee Jones <lee.jones@linaro.org>
> Cc: Deepak R Varma <mh12gx2825@gmail.com>
> Cc: Kevin Wang <kevin1.wang@amd.com>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: "Marek Olšák" <marek.olsak@amd.com>
> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Dennis Li <Dennis.Li@amd.com>
> Cc: Boris Brezillon <boris.brezillon@collabora.com>
> Cc: etnaviv@lists.freedesktop.org
> Cc: lima@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Melissa Wen <mwen@igalia.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +-
> drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +-
> drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +-
> drivers/gpu/drm/lima/lima_gem.c | 3 +--
> drivers/gpu/drm/lima/lima_sched.c | 5 ++---
> drivers/gpu/drm/lima/lima_sched.h | 3 +--
> drivers/gpu/drm/msm/msm_gem_submit.c | 2 +-
> drivers/gpu/drm/panfrost/panfrost_job.c | 2 +-
> drivers/gpu/drm/scheduler/sched_entity.c | 6 ++----
> drivers/gpu/drm/v3d/v3d_gem.c | 2 +-
> include/drm/gpu_scheduler.h | 3 +--
> 11 files changed, 13 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> index 32e80bc6af22..1d8a914108af 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> @@ -1267,7 +1267,7 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
>
> trace_amdgpu_cs_ioctl(job);
> amdgpu_vm_bo_trace_cs(&fpriv->vm, &p->ticket);
> - drm_sched_entity_push_job(&job->base, entity);
> + drm_sched_entity_push_job(&job->base);
>
> amdgpu_vm_move_to_lru_tail(p->adev, &fpriv->vm);
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> index 5ddb955d2315..b8609cccc9c1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
> @@ -174,7 +174,7 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
>
> *f = dma_fence_get(&job->base.s_fence->finished);
> amdgpu_job_free_resources(job);
> - drm_sched_entity_push_job(&job->base, entity);
> + drm_sched_entity_push_job(&job->base);
>
> return 0;
> }
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> index 05f412204118..180bb633d5c5 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> @@ -178,7 +178,7 @@ int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> /* the scheduler holds on to the job now */
> kref_get(&submit->refcount);
>
> - drm_sched_entity_push_job(&submit->sched_job, sched_entity);
> + drm_sched_entity_push_job(&submit->sched_job);
>
> out_unlock:
> mutex_unlock(&submit->gpu->fence_lock);
> diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
> index de62966243cd..c528f40981bb 100644
> --- a/drivers/gpu/drm/lima/lima_gem.c
> +++ b/drivers/gpu/drm/lima/lima_gem.c
> @@ -359,8 +359,7 @@ int lima_gem_submit(struct drm_file *file, struct lima_submit *submit)
> goto err_out2;
> }
>
> - fence = lima_sched_context_queue_task(
> - submit->ctx->context + submit->pipe, submit->task);
> + fence = lima_sched_context_queue_task(submit->task);
>
> for (i = 0; i < submit->nr_bos; i++) {
> if (submit->bos[i].flags & LIMA_SUBMIT_BO_WRITE)
> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> index 38f755580507..e968b5a8f0b0 100644
> --- a/drivers/gpu/drm/lima/lima_sched.c
> +++ b/drivers/gpu/drm/lima/lima_sched.c
> @@ -177,13 +177,12 @@ void lima_sched_context_fini(struct lima_sched_pipe *pipe,
> drm_sched_entity_fini(&context->base);
> }
>
> -struct dma_fence *lima_sched_context_queue_task(struct lima_sched_context *context,
> - struct lima_sched_task *task)
> +struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task)
> {
> struct dma_fence *fence = dma_fence_get(&task->base.s_fence->finished);
>
> trace_lima_task_submit(task);
> - drm_sched_entity_push_job(&task->base, &context->base);
> + drm_sched_entity_push_job(&task->base);
> return fence;
> }
>
> diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
> index 90f03c48ef4a..ac70006b0e26 100644
> --- a/drivers/gpu/drm/lima/lima_sched.h
> +++ b/drivers/gpu/drm/lima/lima_sched.h
> @@ -98,8 +98,7 @@ int lima_sched_context_init(struct lima_sched_pipe *pipe,
> atomic_t *guilty);
> void lima_sched_context_fini(struct lima_sched_pipe *pipe,
> struct lima_sched_context *context);
> -struct dma_fence *lima_sched_context_queue_task(struct lima_sched_context *context,
> - struct lima_sched_task *task);
> +struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task);
>
> int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name);
> void lima_sched_pipe_fini(struct lima_sched_pipe *pipe);
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index d0ed4ddc509e..96cea0ba4cfd 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -909,7 +909,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> /* The scheduler owns a ref now: */
> msm_gem_submit_get(submit);
>
> - drm_sched_entity_push_job(&submit->base, &queue->entity);
> + drm_sched_entity_push_job(&submit->base);
>
> args->fence = submit->fence_id;
>
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> index 2992dc85325f..4bc962763e1f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> @@ -301,7 +301,7 @@ int panfrost_job_push(struct panfrost_job *job)
>
> kref_get(&job->refcount); /* put by scheduler job completion */
>
> - drm_sched_entity_push_job(&job->base, entity);
> + drm_sched_entity_push_job(&job->base);
>
> mutex_unlock(&pfdev->sched_lock);
>
> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> index 381fbf462ea7..e4d33db1eb45 100644
> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> @@ -516,9 +516,7 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
>
> /**
> * drm_sched_entity_push_job - Submit a job to the entity's job queue
> - *
> * @sched_job: job to submit
> - * @entity: scheduler entity
> *
> * Note: To guarantee that the order of insertion to queue matches the job's
> * fence sequence number this function should be called with drm_sched_job_arm()
> @@ -526,9 +524,9 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
> *
> * Returns 0 for success, negative error code otherwise.
> */
> -void drm_sched_entity_push_job(struct drm_sched_job *sched_job,
> - struct drm_sched_entity *entity)
> +void drm_sched_entity_push_job(struct drm_sched_job *sched_job)
> {
> + struct drm_sched_entity *entity = sched_job->entity;
> bool first;
>
> trace_drm_sched_job(sched_job, entity);
> diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
> index 2e808097b4d1..957228bef29c 100644
> --- a/drivers/gpu/drm/v3d/v3d_gem.c
> +++ b/drivers/gpu/drm/v3d/v3d_gem.c
> @@ -487,7 +487,7 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
> /* put by scheduler job completion */
> kref_get(&job->refcount);
>
> - drm_sched_entity_push_job(&job->base, &v3d_priv->sched_entity[queue]);
> + drm_sched_entity_push_job(&job->base);
>
> return 0;
> }
> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> index a47946f904b6..b72f73b375a2 100644
> --- a/include/drm/gpu_scheduler.h
> +++ b/include/drm/gpu_scheduler.h
> @@ -409,8 +409,7 @@ void drm_sched_entity_fini(struct drm_sched_entity *entity);
> void drm_sched_entity_destroy(struct drm_sched_entity *entity);
> void drm_sched_entity_select_rq(struct drm_sched_entity *entity);
> struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity);
> -void drm_sched_entity_push_job(struct drm_sched_job *sched_job,
> - struct drm_sched_entity *entity);
> +void drm_sched_entity_push_job(struct drm_sched_job *sched_job);
> void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> enum drm_sched_priority priority);
> bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 06/20] drm/sched: improve docs around drm_sched_entity
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Boris Brezillon, Daniel Vetter, Lucas Stach, David Airlie,
Daniel Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Christian König, Steven Price,
Emma Anholt, Lee Jones, Andrey Grodzovsky
I found a few too many things that are tricky and not documented, so I
started typing.
I found a few more things that looked broken while typing, see the
varios FIXME in drm_sched_entity.
Also some of the usual logics:
- actually include sched_entity.c declarations, that was lost in the
move here: 620e762f9a98 ("drm/scheduler: move entity handling into
separate file")
- Ditch the kerneldoc for internal functions, keep the comments where
they're describing more than what the function name already implies.
- Switch drm_sched_entity to inline docs.
Acked-by: Melissa Wen <mwen@igalia.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v1)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Emma Anholt <emma@anholt.net>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
Documentation/gpu/drm-mm.rst | 3 +
drivers/gpu/drm/scheduler/sched_entity.c | 85 ++++---------
include/drm/gpu_scheduler.h | 145 ++++++++++++++++++-----
3 files changed, 146 insertions(+), 87 deletions(-)
diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst
index d5a73fa2c9ef..0198fa43d254 100644
--- a/Documentation/gpu/drm-mm.rst
+++ b/Documentation/gpu/drm-mm.rst
@@ -504,3 +504,6 @@ Scheduler Function References
.. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c
:export:
+
+.. kernel-doc:: drivers/gpu/drm/scheduler/sched_entity.c
+ :export:
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index e4d33db1eb45..27e1573af96e 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -45,8 +45,14 @@
* @guilty: atomic_t set to 1 when a job on this queue
* is found to be guilty causing a timeout
*
- * Note: the sched_list should have at least one element to schedule
- * the entity
+ * Note that the &sched_list must have at least one element to schedule the entity.
+ *
+ * For changing @priority later on at runtime see
+ * drm_sched_entity_set_priority(). For changing the set of schedulers
+ * @sched_list at runtime see drm_sched_entity_modify_sched().
+ *
+ * An entity is cleaned up by callind drm_sched_entity_fini(). See also
+ * drm_sched_entity_destroy().
*
* Returns 0 on success or a negative error code on failure.
*/
@@ -92,6 +98,11 @@ EXPORT_SYMBOL(drm_sched_entity_init);
* @sched_list: the list of new drm scheds which will replace
* existing entity->sched_list
* @num_sched_list: number of drm sched in sched_list
+ *
+ * Note that this must be called under the same common lock for @entity as
+ * drm_sched_job_arm() and drm_sched_entity_push_job(), or the driver needs to
+ * guarantee through some other means that this is never called while new jobs
+ * can be pushed to @entity.
*/
void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
struct drm_gpu_scheduler **sched_list,
@@ -104,13 +115,6 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
}
EXPORT_SYMBOL(drm_sched_entity_modify_sched);
-/**
- * drm_sched_entity_is_idle - Check if entity is idle
- *
- * @entity: scheduler entity
- *
- * Returns true if the entity does not have any unscheduled jobs.
- */
static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity)
{
rmb(); /* for list_empty to work without lock */
@@ -123,13 +127,7 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity)
return false;
}
-/**
- * drm_sched_entity_is_ready - Check if entity is ready
- *
- * @entity: scheduler entity
- *
- * Return true if entity could provide a job.
- */
+/* Return true if entity could provide a job. */
bool drm_sched_entity_is_ready(struct drm_sched_entity *entity)
{
if (spsc_queue_peek(&entity->job_queue) == NULL)
@@ -192,14 +190,7 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout)
}
EXPORT_SYMBOL(drm_sched_entity_flush);
-/**
- * drm_sched_entity_kill_jobs_cb - helper for drm_sched_entity_kill_jobs
- *
- * @f: signaled fence
- * @cb: our callback structure
- *
- * Signal the scheduler finished fence when the entity in question is killed.
- */
+/* Signal the scheduler finished fence when the entity in question is killed. */
static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
struct dma_fence_cb *cb)
{
@@ -224,14 +215,6 @@ drm_sched_job_dependency(struct drm_sched_job *job,
return NULL;
}
-/**
- * drm_sched_entity_kill_jobs - Make sure all remaining jobs are killed
- *
- * @entity: entity which is cleaned up
- *
- * Makes sure that all remaining jobs in an entity are killed before it is
- * destroyed.
- */
static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
{
struct drm_sched_job *job;
@@ -273,9 +256,11 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
*
* @entity: scheduler entity
*
- * This should be called after @drm_sched_entity_do_release. It goes over the
- * entity and signals all jobs with an error code if the process was killed.
+ * Cleanups up @entity which has been initialized by drm_sched_entity_init().
*
+ * If there are potentially job still in flight or getting newly queued
+ * drm_sched_entity_flush() must be called first. This function then goes over
+ * the entity and signals all jobs with an error code if the process was killed.
*/
void drm_sched_entity_fini(struct drm_sched_entity *entity)
{
@@ -315,10 +300,10 @@ EXPORT_SYMBOL(drm_sched_entity_fini);
/**
* drm_sched_entity_destroy - Destroy a context entity
- *
* @entity: scheduler entity
*
- * Calls drm_sched_entity_do_release() and drm_sched_entity_cleanup()
+ * Calls drm_sched_entity_flush() and drm_sched_entity_fini() as a
+ * convenience wrapper.
*/
void drm_sched_entity_destroy(struct drm_sched_entity *entity)
{
@@ -327,9 +312,7 @@ void drm_sched_entity_destroy(struct drm_sched_entity *entity)
}
EXPORT_SYMBOL(drm_sched_entity_destroy);
-/*
- * drm_sched_entity_clear_dep - callback to clear the entities dependency
- */
+/* drm_sched_entity_clear_dep - callback to clear the entities dependency */
static void drm_sched_entity_clear_dep(struct dma_fence *f,
struct dma_fence_cb *cb)
{
@@ -371,11 +354,7 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
}
EXPORT_SYMBOL(drm_sched_entity_set_priority);
-/**
- * drm_sched_entity_add_dependency_cb - add callback for the entities dependency
- *
- * @entity: entity with dependency
- *
+/*
* Add a callback to the current dependency of the entity to wake up the
* scheduler when the entity becomes available.
*/
@@ -423,13 +402,6 @@ static bool drm_sched_entity_add_dependency_cb(struct drm_sched_entity *entity)
return false;
}
-/**
- * drm_sched_entity_pop_job - get a ready to be scheduled job from the entity
- *
- * @entity: entity to get the job from
- *
- * Process all dependencies and try to get one job from the entities queue.
- */
struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
{
struct drm_sched_job *sched_job;
@@ -465,14 +437,6 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
return sched_job;
}
-/**
- * drm_sched_entity_select_rq - select a new rq for the entity
- *
- * @entity: scheduler entity
- *
- * Check all prerequisites and select a new rq for the entity for load
- * balancing.
- */
void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
{
struct dma_fence *fence;
@@ -520,7 +484,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
*
* Note: To guarantee that the order of insertion to queue matches the job's
* fence sequence number this function should be called with drm_sched_job_arm()
- * under common lock.
+ * under common lock for the struct drm_sched_entity that was set up for
+ * @sched_job in drm_sched_job_init().
*
* Returns 0 for success, negative error code otherwise.
*/
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index b72f73b375a2..7f77a455722c 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -53,56 +53,147 @@ enum drm_sched_priority {
* struct drm_sched_entity - A wrapper around a job queue (typically
* attached to the DRM file_priv).
*
- * @list: used to append this struct to the list of entities in the
- * runqueue.
- * @rq: runqueue on which this entity is currently scheduled.
- * @sched_list: A list of schedulers (drm_gpu_schedulers).
- * Jobs from this entity can be scheduled on any scheduler
- * on this list.
- * @num_sched_list: number of drm_gpu_schedulers in the sched_list.
- * @priority: priority of the entity
- * @rq_lock: lock to modify the runqueue to which this entity belongs.
- * @job_queue: the list of jobs of this entity.
- * @fence_seq: a linearly increasing seqno incremented with each
- * new &drm_sched_fence which is part of the entity.
- * @fence_context: a unique context for all the fences which belong
- * to this entity.
- * The &drm_sched_fence.scheduled uses the
- * fence_context but &drm_sched_fence.finished uses
- * fence_context + 1.
- * @dependency: the dependency fence of the job which is on the top
- * of the job queue.
- * @cb: callback for the dependency fence above.
- * @guilty: points to ctx's guilty.
- * @fini_status: contains the exit status in case the process was signalled.
- * @last_scheduled: points to the finished fence of the last scheduled job.
- * @last_user: last group leader pushing a job into the entity.
- * @stopped: Marks the enity as removed from rq and destined for termination.
- * @entity_idle: Signals when enityt is not in use
- *
* Entities will emit jobs in order to their corresponding hardware
* ring, and the scheduler will alternate between entities based on
* scheduling policy.
*/
struct drm_sched_entity {
+ /**
+ * @list:
+ *
+ * Used to append this struct to the list of entities in the runqueue
+ * @rq under &drm_sched_rq.entities.
+ *
+ * Protected by &drm_sched_rq.lock of @rq.
+ */
struct list_head list;
+
+ /**
+ * @rq:
+ *
+ * Runqueue on which this entity is currently scheduled.
+ *
+ * FIXME: Locking is very unclear for this. Writers are protected by
+ * @rq_lock, but readers are generally lockless and seem to just race
+ * with not even a READ_ONCE.
+ */
struct drm_sched_rq *rq;
+
+ /**
+ * @sched_list:
+ *
+ * A list of schedulers (struct drm_gpu_scheduler). Jobs from this entity can
+ * be scheduled on any scheduler on this list.
+ *
+ * This can be modified by calling drm_sched_entity_modify_sched().
+ * Locking is entirely up to the driver, see the above function for more
+ * details.
+ *
+ * This will be set to NULL if &num_sched_list equals 1 and @rq has been
+ * set already.
+ *
+ * FIXME: This means priority changes through
+ * drm_sched_entity_set_priority() will be lost henceforth in this case.
+ */
struct drm_gpu_scheduler **sched_list;
+
+ /**
+ * @num_sched_list:
+ *
+ * Number of drm_gpu_schedulers in the @sched_list.
+ */
unsigned int num_sched_list;
+
+ /**
+ * @priority:
+ *
+ * Priority of the entity. This can be modified by calling
+ * drm_sched_entity_set_priority(). Protected by &rq_lock.
+ */
enum drm_sched_priority priority;
+
+ /**
+ * @rq_lock:
+ *
+ * Lock to modify the runqueue to which this entity belongs.
+ */
spinlock_t rq_lock;
+ /**
+ * @job_queue: the list of jobs of this entity.
+ */
struct spsc_queue job_queue;
+ /**
+ * @fence_seq:
+ *
+ * A linearly increasing seqno incremented with each new
+ * &drm_sched_fence which is part of the entity.
+ *
+ * FIXME: Callers of drm_sched_job_arm() need to ensure correct locking,
+ * this doesn't need to be atomic.
+ */
atomic_t fence_seq;
+
+ /**
+ * @fence_context:
+ *
+ * A unique context for all the fences which belong to this entity. The
+ * &drm_sched_fence.scheduled uses the fence_context but
+ * &drm_sched_fence.finished uses fence_context + 1.
+ */
uint64_t fence_context;
+ /**
+ * @dependency:
+ *
+ * The dependency fence of the job which is on the top of the job queue.
+ */
struct dma_fence *dependency;
+
+ /**
+ * @cb:
+ *
+ * Callback for the dependency fence above.
+ */
struct dma_fence_cb cb;
+
+ /**
+ * @guilty:
+ *
+ * Points to entities' guilty.
+ */
atomic_t *guilty;
+
+ /**
+ * @last_scheduled:
+ *
+ * Points to the finished fence of the last scheduled job. Only written
+ * by the scheduler thread, can be accessed locklessly from
+ * drm_sched_job_arm() iff the queue is empty.
+ */
struct dma_fence *last_scheduled;
+
+ /**
+ * @last_user: last group leader pushing a job into the entity.
+ */
struct task_struct *last_user;
+
+ /**
+ * @stopped:
+ *
+ * Marks the enity as removed from rq and destined for
+ * termination. This is set by calling drm_sched_entity_flush() and by
+ * drm_sched_fini().
+ */
bool stopped;
+
+ /**
+ * @entity_idle:
+ *
+ * Signals when entity is not in use, used to sequence entity cleanup in
+ * drm_sched_entity_fini().
+ */
struct completion entity_idle;
};
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 06/20] drm/sched: improve docs around drm_sched_entity
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Boris Brezillon, Daniel Vetter, Lucas Stach, David Airlie,
Daniel Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Christian König, Steven Price,
Emma Anholt, Lee Jones, Andrey Grodzovsky
I found a few too many things that are tricky and not documented, so I
started typing.
I found a few more things that looked broken while typing, see the
varios FIXME in drm_sched_entity.
Also some of the usual logics:
- actually include sched_entity.c declarations, that was lost in the
move here: 620e762f9a98 ("drm/scheduler: move entity handling into
separate file")
- Ditch the kerneldoc for internal functions, keep the comments where
they're describing more than what the function name already implies.
- Switch drm_sched_entity to inline docs.
Acked-by: Melissa Wen <mwen@igalia.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (v1)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Emma Anholt <emma@anholt.net>
Cc: Lee Jones <lee.jones@linaro.org>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
Documentation/gpu/drm-mm.rst | 3 +
drivers/gpu/drm/scheduler/sched_entity.c | 85 ++++---------
include/drm/gpu_scheduler.h | 145 ++++++++++++++++++-----
3 files changed, 146 insertions(+), 87 deletions(-)
diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst
index d5a73fa2c9ef..0198fa43d254 100644
--- a/Documentation/gpu/drm-mm.rst
+++ b/Documentation/gpu/drm-mm.rst
@@ -504,3 +504,6 @@ Scheduler Function References
.. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c
:export:
+
+.. kernel-doc:: drivers/gpu/drm/scheduler/sched_entity.c
+ :export:
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index e4d33db1eb45..27e1573af96e 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -45,8 +45,14 @@
* @guilty: atomic_t set to 1 when a job on this queue
* is found to be guilty causing a timeout
*
- * Note: the sched_list should have at least one element to schedule
- * the entity
+ * Note that the &sched_list must have at least one element to schedule the entity.
+ *
+ * For changing @priority later on at runtime see
+ * drm_sched_entity_set_priority(). For changing the set of schedulers
+ * @sched_list at runtime see drm_sched_entity_modify_sched().
+ *
+ * An entity is cleaned up by callind drm_sched_entity_fini(). See also
+ * drm_sched_entity_destroy().
*
* Returns 0 on success or a negative error code on failure.
*/
@@ -92,6 +98,11 @@ EXPORT_SYMBOL(drm_sched_entity_init);
* @sched_list: the list of new drm scheds which will replace
* existing entity->sched_list
* @num_sched_list: number of drm sched in sched_list
+ *
+ * Note that this must be called under the same common lock for @entity as
+ * drm_sched_job_arm() and drm_sched_entity_push_job(), or the driver needs to
+ * guarantee through some other means that this is never called while new jobs
+ * can be pushed to @entity.
*/
void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
struct drm_gpu_scheduler **sched_list,
@@ -104,13 +115,6 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,
}
EXPORT_SYMBOL(drm_sched_entity_modify_sched);
-/**
- * drm_sched_entity_is_idle - Check if entity is idle
- *
- * @entity: scheduler entity
- *
- * Returns true if the entity does not have any unscheduled jobs.
- */
static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity)
{
rmb(); /* for list_empty to work without lock */
@@ -123,13 +127,7 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity)
return false;
}
-/**
- * drm_sched_entity_is_ready - Check if entity is ready
- *
- * @entity: scheduler entity
- *
- * Return true if entity could provide a job.
- */
+/* Return true if entity could provide a job. */
bool drm_sched_entity_is_ready(struct drm_sched_entity *entity)
{
if (spsc_queue_peek(&entity->job_queue) == NULL)
@@ -192,14 +190,7 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout)
}
EXPORT_SYMBOL(drm_sched_entity_flush);
-/**
- * drm_sched_entity_kill_jobs_cb - helper for drm_sched_entity_kill_jobs
- *
- * @f: signaled fence
- * @cb: our callback structure
- *
- * Signal the scheduler finished fence when the entity in question is killed.
- */
+/* Signal the scheduler finished fence when the entity in question is killed. */
static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
struct dma_fence_cb *cb)
{
@@ -224,14 +215,6 @@ drm_sched_job_dependency(struct drm_sched_job *job,
return NULL;
}
-/**
- * drm_sched_entity_kill_jobs - Make sure all remaining jobs are killed
- *
- * @entity: entity which is cleaned up
- *
- * Makes sure that all remaining jobs in an entity are killed before it is
- * destroyed.
- */
static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
{
struct drm_sched_job *job;
@@ -273,9 +256,11 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
*
* @entity: scheduler entity
*
- * This should be called after @drm_sched_entity_do_release. It goes over the
- * entity and signals all jobs with an error code if the process was killed.
+ * Cleanups up @entity which has been initialized by drm_sched_entity_init().
*
+ * If there are potentially job still in flight or getting newly queued
+ * drm_sched_entity_flush() must be called first. This function then goes over
+ * the entity and signals all jobs with an error code if the process was killed.
*/
void drm_sched_entity_fini(struct drm_sched_entity *entity)
{
@@ -315,10 +300,10 @@ EXPORT_SYMBOL(drm_sched_entity_fini);
/**
* drm_sched_entity_destroy - Destroy a context entity
- *
* @entity: scheduler entity
*
- * Calls drm_sched_entity_do_release() and drm_sched_entity_cleanup()
+ * Calls drm_sched_entity_flush() and drm_sched_entity_fini() as a
+ * convenience wrapper.
*/
void drm_sched_entity_destroy(struct drm_sched_entity *entity)
{
@@ -327,9 +312,7 @@ void drm_sched_entity_destroy(struct drm_sched_entity *entity)
}
EXPORT_SYMBOL(drm_sched_entity_destroy);
-/*
- * drm_sched_entity_clear_dep - callback to clear the entities dependency
- */
+/* drm_sched_entity_clear_dep - callback to clear the entities dependency */
static void drm_sched_entity_clear_dep(struct dma_fence *f,
struct dma_fence_cb *cb)
{
@@ -371,11 +354,7 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
}
EXPORT_SYMBOL(drm_sched_entity_set_priority);
-/**
- * drm_sched_entity_add_dependency_cb - add callback for the entities dependency
- *
- * @entity: entity with dependency
- *
+/*
* Add a callback to the current dependency of the entity to wake up the
* scheduler when the entity becomes available.
*/
@@ -423,13 +402,6 @@ static bool drm_sched_entity_add_dependency_cb(struct drm_sched_entity *entity)
return false;
}
-/**
- * drm_sched_entity_pop_job - get a ready to be scheduled job from the entity
- *
- * @entity: entity to get the job from
- *
- * Process all dependencies and try to get one job from the entities queue.
- */
struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
{
struct drm_sched_job *sched_job;
@@ -465,14 +437,6 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)
return sched_job;
}
-/**
- * drm_sched_entity_select_rq - select a new rq for the entity
- *
- * @entity: scheduler entity
- *
- * Check all prerequisites and select a new rq for the entity for load
- * balancing.
- */
void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
{
struct dma_fence *fence;
@@ -520,7 +484,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)
*
* Note: To guarantee that the order of insertion to queue matches the job's
* fence sequence number this function should be called with drm_sched_job_arm()
- * under common lock.
+ * under common lock for the struct drm_sched_entity that was set up for
+ * @sched_job in drm_sched_job_init().
*
* Returns 0 for success, negative error code otherwise.
*/
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index b72f73b375a2..7f77a455722c 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -53,56 +53,147 @@ enum drm_sched_priority {
* struct drm_sched_entity - A wrapper around a job queue (typically
* attached to the DRM file_priv).
*
- * @list: used to append this struct to the list of entities in the
- * runqueue.
- * @rq: runqueue on which this entity is currently scheduled.
- * @sched_list: A list of schedulers (drm_gpu_schedulers).
- * Jobs from this entity can be scheduled on any scheduler
- * on this list.
- * @num_sched_list: number of drm_gpu_schedulers in the sched_list.
- * @priority: priority of the entity
- * @rq_lock: lock to modify the runqueue to which this entity belongs.
- * @job_queue: the list of jobs of this entity.
- * @fence_seq: a linearly increasing seqno incremented with each
- * new &drm_sched_fence which is part of the entity.
- * @fence_context: a unique context for all the fences which belong
- * to this entity.
- * The &drm_sched_fence.scheduled uses the
- * fence_context but &drm_sched_fence.finished uses
- * fence_context + 1.
- * @dependency: the dependency fence of the job which is on the top
- * of the job queue.
- * @cb: callback for the dependency fence above.
- * @guilty: points to ctx's guilty.
- * @fini_status: contains the exit status in case the process was signalled.
- * @last_scheduled: points to the finished fence of the last scheduled job.
- * @last_user: last group leader pushing a job into the entity.
- * @stopped: Marks the enity as removed from rq and destined for termination.
- * @entity_idle: Signals when enityt is not in use
- *
* Entities will emit jobs in order to their corresponding hardware
* ring, and the scheduler will alternate between entities based on
* scheduling policy.
*/
struct drm_sched_entity {
+ /**
+ * @list:
+ *
+ * Used to append this struct to the list of entities in the runqueue
+ * @rq under &drm_sched_rq.entities.
+ *
+ * Protected by &drm_sched_rq.lock of @rq.
+ */
struct list_head list;
+
+ /**
+ * @rq:
+ *
+ * Runqueue on which this entity is currently scheduled.
+ *
+ * FIXME: Locking is very unclear for this. Writers are protected by
+ * @rq_lock, but readers are generally lockless and seem to just race
+ * with not even a READ_ONCE.
+ */
struct drm_sched_rq *rq;
+
+ /**
+ * @sched_list:
+ *
+ * A list of schedulers (struct drm_gpu_scheduler). Jobs from this entity can
+ * be scheduled on any scheduler on this list.
+ *
+ * This can be modified by calling drm_sched_entity_modify_sched().
+ * Locking is entirely up to the driver, see the above function for more
+ * details.
+ *
+ * This will be set to NULL if &num_sched_list equals 1 and @rq has been
+ * set already.
+ *
+ * FIXME: This means priority changes through
+ * drm_sched_entity_set_priority() will be lost henceforth in this case.
+ */
struct drm_gpu_scheduler **sched_list;
+
+ /**
+ * @num_sched_list:
+ *
+ * Number of drm_gpu_schedulers in the @sched_list.
+ */
unsigned int num_sched_list;
+
+ /**
+ * @priority:
+ *
+ * Priority of the entity. This can be modified by calling
+ * drm_sched_entity_set_priority(). Protected by &rq_lock.
+ */
enum drm_sched_priority priority;
+
+ /**
+ * @rq_lock:
+ *
+ * Lock to modify the runqueue to which this entity belongs.
+ */
spinlock_t rq_lock;
+ /**
+ * @job_queue: the list of jobs of this entity.
+ */
struct spsc_queue job_queue;
+ /**
+ * @fence_seq:
+ *
+ * A linearly increasing seqno incremented with each new
+ * &drm_sched_fence which is part of the entity.
+ *
+ * FIXME: Callers of drm_sched_job_arm() need to ensure correct locking,
+ * this doesn't need to be atomic.
+ */
atomic_t fence_seq;
+
+ /**
+ * @fence_context:
+ *
+ * A unique context for all the fences which belong to this entity. The
+ * &drm_sched_fence.scheduled uses the fence_context but
+ * &drm_sched_fence.finished uses fence_context + 1.
+ */
uint64_t fence_context;
+ /**
+ * @dependency:
+ *
+ * The dependency fence of the job which is on the top of the job queue.
+ */
struct dma_fence *dependency;
+
+ /**
+ * @cb:
+ *
+ * Callback for the dependency fence above.
+ */
struct dma_fence_cb cb;
+
+ /**
+ * @guilty:
+ *
+ * Points to entities' guilty.
+ */
atomic_t *guilty;
+
+ /**
+ * @last_scheduled:
+ *
+ * Points to the finished fence of the last scheduled job. Only written
+ * by the scheduler thread, can be accessed locklessly from
+ * drm_sched_job_arm() iff the queue is empty.
+ */
struct dma_fence *last_scheduled;
+
+ /**
+ * @last_user: last group leader pushing a job into the entity.
+ */
struct task_struct *last_user;
+
+ /**
+ * @stopped:
+ *
+ * Marks the enity as removed from rq and destined for
+ * termination. This is set by calling drm_sched_entity_flush() and by
+ * drm_sched_fini().
+ */
bool stopped;
+
+ /**
+ * @entity_idle:
+ *
+ * Signals when entity is not in use, used to sequence entity cleanup in
+ * drm_sched_entity_fini().
+ */
struct completion entity_idle;
};
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* [PATCH v5 07/20] drm/panfrost: use scheduler dependency tracking
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Emma Anholt,
Steven Price, Daniel Vetter, Rob Herring, Tomeu Vizoso,
Alyssa Rosenzweig, Sumit Semwal, Christian König,
linux-media, linaro-mm-sig
Just deletes some code that's now more shared.
Note that thanks to the split into drm_sched_job_init/arm we can now
easily pull the _init() part from under the submission lock way ahead
where we're adding the sync file in-fences as dependencies.
v2: Correctly clean up the partially set up job, now that job_init()
and job_arm() are apart (Emma).
v3: Rebased over renamed functions for adding depdencies
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Steven Price <steven.price@arm.com> (v3)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Rob Herring <robh@kernel.org>
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: Emma Anholt <emma@anholt.net>
---
drivers/gpu/drm/panfrost/panfrost_drv.c | 16 ++++++++---
drivers/gpu/drm/panfrost/panfrost_job.c | 38 ++++---------------------
drivers/gpu/drm/panfrost/panfrost_job.h | 5 +---
3 files changed, 18 insertions(+), 41 deletions(-)
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index 1ffaef5ec5ff..16212b6b202e 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -218,7 +218,7 @@ panfrost_copy_in_sync(struct drm_device *dev,
if (ret)
goto fail;
- ret = drm_gem_fence_array_add(&job->deps, fence);
+ ret = drm_sched_job_add_dependency(&job->base, fence);
if (ret)
goto fail;
@@ -236,7 +236,7 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
struct drm_panfrost_submit *args = data;
struct drm_syncobj *sync_out = NULL;
struct panfrost_job *job;
- int ret = 0;
+ int ret = 0, slot;
if (!args->jc)
return -EINVAL;
@@ -258,14 +258,20 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
kref_init(&job->refcount);
- xa_init_flags(&job->deps, XA_FLAGS_ALLOC);
-
job->pfdev = pfdev;
job->jc = args->jc;
job->requirements = args->requirements;
job->flush_id = panfrost_gpu_get_latest_flush_id(pfdev);
job->file_priv = file->driver_priv;
+ slot = panfrost_job_get_slot(job);
+
+ ret = drm_sched_job_init(&job->base,
+ &job->file_priv->sched_entity[slot],
+ NULL);
+ if (ret)
+ goto fail_job_put;
+
ret = panfrost_copy_in_sync(dev, file, args, job);
if (ret)
goto fail_job;
@@ -283,6 +289,8 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
drm_syncobj_replace_fence(sync_out, job->render_done_fence);
fail_job:
+ drm_sched_job_cleanup(&job->base);
+fail_job_put:
panfrost_job_put(job);
fail_out_sync:
if (sync_out)
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 4bc962763e1f..a98f507dc779 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -102,7 +102,7 @@ static struct dma_fence *panfrost_fence_create(struct panfrost_device *pfdev, in
return &fence->base;
}
-static int panfrost_job_get_slot(struct panfrost_job *job)
+int panfrost_job_get_slot(struct panfrost_job *job)
{
/* JS0: fragment jobs.
* JS1: vertex/tiler jobs
@@ -242,13 +242,14 @@ static void panfrost_job_hw_submit(struct panfrost_job *job, int js)
static int panfrost_acquire_object_fences(struct drm_gem_object **bos,
int bo_count,
- struct xarray *deps)
+ struct drm_sched_job *job)
{
int i, ret;
for (i = 0; i < bo_count; i++) {
/* panfrost always uses write mode in its current uapi */
- ret = drm_gem_fence_array_add_implicit(deps, bos[i], true);
+ ret = drm_sched_job_add_implicit_dependencies(job, bos[i],
+ true);
if (ret)
return ret;
}
@@ -269,31 +270,21 @@ static void panfrost_attach_object_fences(struct drm_gem_object **bos,
int panfrost_job_push(struct panfrost_job *job)
{
struct panfrost_device *pfdev = job->pfdev;
- int slot = panfrost_job_get_slot(job);
- struct drm_sched_entity *entity = &job->file_priv->sched_entity[slot];
struct ww_acquire_ctx acquire_ctx;
int ret = 0;
-
ret = drm_gem_lock_reservations(job->bos, job->bo_count,
&acquire_ctx);
if (ret)
return ret;
mutex_lock(&pfdev->sched_lock);
-
- ret = drm_sched_job_init(&job->base, entity, NULL);
- if (ret) {
- mutex_unlock(&pfdev->sched_lock);
- goto unlock;
- }
-
drm_sched_job_arm(&job->base);
job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
- &job->deps);
+ &job->base);
if (ret) {
mutex_unlock(&pfdev->sched_lock);
goto unlock;
@@ -318,15 +309,8 @@ static void panfrost_job_cleanup(struct kref *ref)
{
struct panfrost_job *job = container_of(ref, struct panfrost_job,
refcount);
- struct dma_fence *fence;
- unsigned long index;
unsigned int i;
- xa_for_each(&job->deps, index, fence) {
- dma_fence_put(fence);
- }
- xa_destroy(&job->deps);
-
dma_fence_put(job->done_fence);
dma_fence_put(job->render_done_fence);
@@ -365,17 +349,6 @@ static void panfrost_job_free(struct drm_sched_job *sched_job)
panfrost_job_put(job);
}
-static struct dma_fence *panfrost_job_dependency(struct drm_sched_job *sched_job,
- struct drm_sched_entity *s_entity)
-{
- struct panfrost_job *job = to_panfrost_job(sched_job);
-
- if (!xa_empty(&job->deps))
- return xa_erase(&job->deps, job->last_dep++);
-
- return NULL;
-}
-
static struct dma_fence *panfrost_job_run(struct drm_sched_job *sched_job)
{
struct panfrost_job *job = to_panfrost_job(sched_job);
@@ -765,7 +738,6 @@ static void panfrost_reset_work(struct work_struct *work)
}
static const struct drm_sched_backend_ops panfrost_sched_ops = {
- .dependency = panfrost_job_dependency,
.run_job = panfrost_job_run,
.timedout_job = panfrost_job_timedout,
.free_job = panfrost_job_free
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.h b/drivers/gpu/drm/panfrost/panfrost_job.h
index 82306a03b57e..77e6d0e6f612 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.h
+++ b/drivers/gpu/drm/panfrost/panfrost_job.h
@@ -19,10 +19,6 @@ struct panfrost_job {
struct panfrost_device *pfdev;
struct panfrost_file_priv *file_priv;
- /* Contains both explicit and implicit fences */
- struct xarray deps;
- unsigned long last_dep;
-
/* Fence to be signaled by IRQ handler when the job is complete. */
struct dma_fence *done_fence;
@@ -42,6 +38,7 @@ int panfrost_job_init(struct panfrost_device *pfdev);
void panfrost_job_fini(struct panfrost_device *pfdev);
int panfrost_job_open(struct panfrost_file_priv *panfrost_priv);
void panfrost_job_close(struct panfrost_file_priv *panfrost_priv);
+int panfrost_job_get_slot(struct panfrost_job *job);
int panfrost_job_push(struct panfrost_job *job);
void panfrost_job_put(struct panfrost_job *job);
void panfrost_job_enable_interrupts(struct panfrost_device *pfdev);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 07/20] drm/panfrost: use scheduler dependency tracking
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Emma Anholt,
Steven Price, Daniel Vetter, Rob Herring, Tomeu Vizoso,
Alyssa Rosenzweig, Sumit Semwal, Christian König,
linux-media, linaro-mm-sig
Just deletes some code that's now more shared.
Note that thanks to the split into drm_sched_job_init/arm we can now
easily pull the _init() part from under the submission lock way ahead
where we're adding the sync file in-fences as dependencies.
v2: Correctly clean up the partially set up job, now that job_init()
and job_arm() are apart (Emma).
v3: Rebased over renamed functions for adding depdencies
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Steven Price <steven.price@arm.com> (v3)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Rob Herring <robh@kernel.org>
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: Emma Anholt <emma@anholt.net>
---
drivers/gpu/drm/panfrost/panfrost_drv.c | 16 ++++++++---
drivers/gpu/drm/panfrost/panfrost_job.c | 38 ++++---------------------
drivers/gpu/drm/panfrost/panfrost_job.h | 5 +---
3 files changed, 18 insertions(+), 41 deletions(-)
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index 1ffaef5ec5ff..16212b6b202e 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -218,7 +218,7 @@ panfrost_copy_in_sync(struct drm_device *dev,
if (ret)
goto fail;
- ret = drm_gem_fence_array_add(&job->deps, fence);
+ ret = drm_sched_job_add_dependency(&job->base, fence);
if (ret)
goto fail;
@@ -236,7 +236,7 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
struct drm_panfrost_submit *args = data;
struct drm_syncobj *sync_out = NULL;
struct panfrost_job *job;
- int ret = 0;
+ int ret = 0, slot;
if (!args->jc)
return -EINVAL;
@@ -258,14 +258,20 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
kref_init(&job->refcount);
- xa_init_flags(&job->deps, XA_FLAGS_ALLOC);
-
job->pfdev = pfdev;
job->jc = args->jc;
job->requirements = args->requirements;
job->flush_id = panfrost_gpu_get_latest_flush_id(pfdev);
job->file_priv = file->driver_priv;
+ slot = panfrost_job_get_slot(job);
+
+ ret = drm_sched_job_init(&job->base,
+ &job->file_priv->sched_entity[slot],
+ NULL);
+ if (ret)
+ goto fail_job_put;
+
ret = panfrost_copy_in_sync(dev, file, args, job);
if (ret)
goto fail_job;
@@ -283,6 +289,8 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
drm_syncobj_replace_fence(sync_out, job->render_done_fence);
fail_job:
+ drm_sched_job_cleanup(&job->base);
+fail_job_put:
panfrost_job_put(job);
fail_out_sync:
if (sync_out)
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 4bc962763e1f..a98f507dc779 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -102,7 +102,7 @@ static struct dma_fence *panfrost_fence_create(struct panfrost_device *pfdev, in
return &fence->base;
}
-static int panfrost_job_get_slot(struct panfrost_job *job)
+int panfrost_job_get_slot(struct panfrost_job *job)
{
/* JS0: fragment jobs.
* JS1: vertex/tiler jobs
@@ -242,13 +242,14 @@ static void panfrost_job_hw_submit(struct panfrost_job *job, int js)
static int panfrost_acquire_object_fences(struct drm_gem_object **bos,
int bo_count,
- struct xarray *deps)
+ struct drm_sched_job *job)
{
int i, ret;
for (i = 0; i < bo_count; i++) {
/* panfrost always uses write mode in its current uapi */
- ret = drm_gem_fence_array_add_implicit(deps, bos[i], true);
+ ret = drm_sched_job_add_implicit_dependencies(job, bos[i],
+ true);
if (ret)
return ret;
}
@@ -269,31 +270,21 @@ static void panfrost_attach_object_fences(struct drm_gem_object **bos,
int panfrost_job_push(struct panfrost_job *job)
{
struct panfrost_device *pfdev = job->pfdev;
- int slot = panfrost_job_get_slot(job);
- struct drm_sched_entity *entity = &job->file_priv->sched_entity[slot];
struct ww_acquire_ctx acquire_ctx;
int ret = 0;
-
ret = drm_gem_lock_reservations(job->bos, job->bo_count,
&acquire_ctx);
if (ret)
return ret;
mutex_lock(&pfdev->sched_lock);
-
- ret = drm_sched_job_init(&job->base, entity, NULL);
- if (ret) {
- mutex_unlock(&pfdev->sched_lock);
- goto unlock;
- }
-
drm_sched_job_arm(&job->base);
job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
- &job->deps);
+ &job->base);
if (ret) {
mutex_unlock(&pfdev->sched_lock);
goto unlock;
@@ -318,15 +309,8 @@ static void panfrost_job_cleanup(struct kref *ref)
{
struct panfrost_job *job = container_of(ref, struct panfrost_job,
refcount);
- struct dma_fence *fence;
- unsigned long index;
unsigned int i;
- xa_for_each(&job->deps, index, fence) {
- dma_fence_put(fence);
- }
- xa_destroy(&job->deps);
-
dma_fence_put(job->done_fence);
dma_fence_put(job->render_done_fence);
@@ -365,17 +349,6 @@ static void panfrost_job_free(struct drm_sched_job *sched_job)
panfrost_job_put(job);
}
-static struct dma_fence *panfrost_job_dependency(struct drm_sched_job *sched_job,
- struct drm_sched_entity *s_entity)
-{
- struct panfrost_job *job = to_panfrost_job(sched_job);
-
- if (!xa_empty(&job->deps))
- return xa_erase(&job->deps, job->last_dep++);
-
- return NULL;
-}
-
static struct dma_fence *panfrost_job_run(struct drm_sched_job *sched_job)
{
struct panfrost_job *job = to_panfrost_job(sched_job);
@@ -765,7 +738,6 @@ static void panfrost_reset_work(struct work_struct *work)
}
static const struct drm_sched_backend_ops panfrost_sched_ops = {
- .dependency = panfrost_job_dependency,
.run_job = panfrost_job_run,
.timedout_job = panfrost_job_timedout,
.free_job = panfrost_job_free
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.h b/drivers/gpu/drm/panfrost/panfrost_job.h
index 82306a03b57e..77e6d0e6f612 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.h
+++ b/drivers/gpu/drm/panfrost/panfrost_job.h
@@ -19,10 +19,6 @@ struct panfrost_job {
struct panfrost_device *pfdev;
struct panfrost_file_priv *file_priv;
- /* Contains both explicit and implicit fences */
- struct xarray deps;
- unsigned long last_dep;
-
/* Fence to be signaled by IRQ handler when the job is complete. */
struct dma_fence *done_fence;
@@ -42,6 +38,7 @@ int panfrost_job_init(struct panfrost_device *pfdev);
void panfrost_job_fini(struct panfrost_device *pfdev);
int panfrost_job_open(struct panfrost_file_priv *panfrost_priv);
void panfrost_job_close(struct panfrost_file_priv *panfrost_priv);
+int panfrost_job_get_slot(struct panfrost_job *job);
int panfrost_job_push(struct panfrost_job *job);
void panfrost_job_put(struct panfrost_job *job);
void panfrost_job_enable_interrupts(struct panfrost_device *pfdev);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 07/20] drm/panfrost: use scheduler dependency tracking
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 15:10 ` Alyssa Rosenzweig
-1 siblings, 0 replies; 122+ messages in thread
From: Alyssa Rosenzweig @ 2021-08-05 15:10 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Emma Anholt,
Steven Price, Daniel Vetter, Rob Herring, Tomeu Vizoso,
Alyssa Rosenzweig, Sumit Semwal, Christian K??nig, linux-media,
linaro-mm-sig
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
On Thu, Aug 05, 2021 at 12:46:52PM +0200, Daniel Vetter wrote:
> Just deletes some code that's now more shared.
>
> Note that thanks to the split into drm_sched_job_init/arm we can now
> easily pull the _init() part from under the submission lock way ahead
> where we're adding the sync file in-fences as dependencies.
>
> v2: Correctly clean up the partially set up job, now that job_init()
> and job_arm() are apart (Emma).
>
> v3: Rebased over renamed functions for adding depdencies
>
> Acked-by: Emma Anholt <emma@anholt.net>
> Reviewed-by: Steven Price <steven.price@arm.com> (v3)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian K??nig" <christian.koenig@amd.com>
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> Cc: Emma Anholt <emma@anholt.net>
> ---
> drivers/gpu/drm/panfrost/panfrost_drv.c | 16 ++++++++---
> drivers/gpu/drm/panfrost/panfrost_job.c | 38 ++++---------------------
> drivers/gpu/drm/panfrost/panfrost_job.h | 5 +---
> 3 files changed, 18 insertions(+), 41 deletions(-)
>
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 1ffaef5ec5ff..16212b6b202e 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -218,7 +218,7 @@ panfrost_copy_in_sync(struct drm_device *dev,
> if (ret)
> goto fail;
>
> - ret = drm_gem_fence_array_add(&job->deps, fence);
> + ret = drm_sched_job_add_dependency(&job->base, fence);
>
> if (ret)
> goto fail;
> @@ -236,7 +236,7 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
> struct drm_panfrost_submit *args = data;
> struct drm_syncobj *sync_out = NULL;
> struct panfrost_job *job;
> - int ret = 0;
> + int ret = 0, slot;
>
> if (!args->jc)
> return -EINVAL;
> @@ -258,14 +258,20 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
>
> kref_init(&job->refcount);
>
> - xa_init_flags(&job->deps, XA_FLAGS_ALLOC);
> -
> job->pfdev = pfdev;
> job->jc = args->jc;
> job->requirements = args->requirements;
> job->flush_id = panfrost_gpu_get_latest_flush_id(pfdev);
> job->file_priv = file->driver_priv;
>
> + slot = panfrost_job_get_slot(job);
> +
> + ret = drm_sched_job_init(&job->base,
> + &job->file_priv->sched_entity[slot],
> + NULL);
> + if (ret)
> + goto fail_job_put;
> +
> ret = panfrost_copy_in_sync(dev, file, args, job);
> if (ret)
> goto fail_job;
> @@ -283,6 +289,8 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
> drm_syncobj_replace_fence(sync_out, job->render_done_fence);
>
> fail_job:
> + drm_sched_job_cleanup(&job->base);
> +fail_job_put:
> panfrost_job_put(job);
> fail_out_sync:
> if (sync_out)
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> index 4bc962763e1f..a98f507dc779 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> @@ -102,7 +102,7 @@ static struct dma_fence *panfrost_fence_create(struct panfrost_device *pfdev, in
> return &fence->base;
> }
>
> -static int panfrost_job_get_slot(struct panfrost_job *job)
> +int panfrost_job_get_slot(struct panfrost_job *job)
> {
> /* JS0: fragment jobs.
> * JS1: vertex/tiler jobs
> @@ -242,13 +242,14 @@ static void panfrost_job_hw_submit(struct panfrost_job *job, int js)
>
> static int panfrost_acquire_object_fences(struct drm_gem_object **bos,
> int bo_count,
> - struct xarray *deps)
> + struct drm_sched_job *job)
> {
> int i, ret;
>
> for (i = 0; i < bo_count; i++) {
> /* panfrost always uses write mode in its current uapi */
> - ret = drm_gem_fence_array_add_implicit(deps, bos[i], true);
> + ret = drm_sched_job_add_implicit_dependencies(job, bos[i],
> + true);
> if (ret)
> return ret;
> }
> @@ -269,31 +270,21 @@ static void panfrost_attach_object_fences(struct drm_gem_object **bos,
> int panfrost_job_push(struct panfrost_job *job)
> {
> struct panfrost_device *pfdev = job->pfdev;
> - int slot = panfrost_job_get_slot(job);
> - struct drm_sched_entity *entity = &job->file_priv->sched_entity[slot];
> struct ww_acquire_ctx acquire_ctx;
> int ret = 0;
>
> -
> ret = drm_gem_lock_reservations(job->bos, job->bo_count,
> &acquire_ctx);
> if (ret)
> return ret;
>
> mutex_lock(&pfdev->sched_lock);
> -
> - ret = drm_sched_job_init(&job->base, entity, NULL);
> - if (ret) {
> - mutex_unlock(&pfdev->sched_lock);
> - goto unlock;
> - }
> -
> drm_sched_job_arm(&job->base);
>
> job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
>
> ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
> - &job->deps);
> + &job->base);
> if (ret) {
> mutex_unlock(&pfdev->sched_lock);
> goto unlock;
> @@ -318,15 +309,8 @@ static void panfrost_job_cleanup(struct kref *ref)
> {
> struct panfrost_job *job = container_of(ref, struct panfrost_job,
> refcount);
> - struct dma_fence *fence;
> - unsigned long index;
> unsigned int i;
>
> - xa_for_each(&job->deps, index, fence) {
> - dma_fence_put(fence);
> - }
> - xa_destroy(&job->deps);
> -
> dma_fence_put(job->done_fence);
> dma_fence_put(job->render_done_fence);
>
> @@ -365,17 +349,6 @@ static void panfrost_job_free(struct drm_sched_job *sched_job)
> panfrost_job_put(job);
> }
>
> -static struct dma_fence *panfrost_job_dependency(struct drm_sched_job *sched_job,
> - struct drm_sched_entity *s_entity)
> -{
> - struct panfrost_job *job = to_panfrost_job(sched_job);
> -
> - if (!xa_empty(&job->deps))
> - return xa_erase(&job->deps, job->last_dep++);
> -
> - return NULL;
> -}
> -
> static struct dma_fence *panfrost_job_run(struct drm_sched_job *sched_job)
> {
> struct panfrost_job *job = to_panfrost_job(sched_job);
> @@ -765,7 +738,6 @@ static void panfrost_reset_work(struct work_struct *work)
> }
>
> static const struct drm_sched_backend_ops panfrost_sched_ops = {
> - .dependency = panfrost_job_dependency,
> .run_job = panfrost_job_run,
> .timedout_job = panfrost_job_timedout,
> .free_job = panfrost_job_free
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.h b/drivers/gpu/drm/panfrost/panfrost_job.h
> index 82306a03b57e..77e6d0e6f612 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.h
> @@ -19,10 +19,6 @@ struct panfrost_job {
> struct panfrost_device *pfdev;
> struct panfrost_file_priv *file_priv;
>
> - /* Contains both explicit and implicit fences */
> - struct xarray deps;
> - unsigned long last_dep;
> -
> /* Fence to be signaled by IRQ handler when the job is complete. */
> struct dma_fence *done_fence;
>
> @@ -42,6 +38,7 @@ int panfrost_job_init(struct panfrost_device *pfdev);
> void panfrost_job_fini(struct panfrost_device *pfdev);
> int panfrost_job_open(struct panfrost_file_priv *panfrost_priv);
> void panfrost_job_close(struct panfrost_file_priv *panfrost_priv);
> +int panfrost_job_get_slot(struct panfrost_job *job);
> int panfrost_job_push(struct panfrost_job *job);
> void panfrost_job_put(struct panfrost_job *job);
> void panfrost_job_enable_interrupts(struct panfrost_device *pfdev);
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 07/20] drm/panfrost: use scheduler dependency tracking
@ 2021-08-05 15:10 ` Alyssa Rosenzweig
0 siblings, 0 replies; 122+ messages in thread
From: Alyssa Rosenzweig @ 2021-08-05 15:10 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Emma Anholt,
Steven Price, Daniel Vetter, Rob Herring, Tomeu Vizoso,
Alyssa Rosenzweig, Sumit Semwal, Christian K??nig, linux-media,
linaro-mm-sig
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
On Thu, Aug 05, 2021 at 12:46:52PM +0200, Daniel Vetter wrote:
> Just deletes some code that's now more shared.
>
> Note that thanks to the split into drm_sched_job_init/arm we can now
> easily pull the _init() part from under the submission lock way ahead
> where we're adding the sync file in-fences as dependencies.
>
> v2: Correctly clean up the partially set up job, now that job_init()
> and job_arm() are apart (Emma).
>
> v3: Rebased over renamed functions for adding depdencies
>
> Acked-by: Emma Anholt <emma@anholt.net>
> Reviewed-by: Steven Price <steven.price@arm.com> (v3)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian K??nig" <christian.koenig@amd.com>
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
> Cc: Emma Anholt <emma@anholt.net>
> ---
> drivers/gpu/drm/panfrost/panfrost_drv.c | 16 ++++++++---
> drivers/gpu/drm/panfrost/panfrost_job.c | 38 ++++---------------------
> drivers/gpu/drm/panfrost/panfrost_job.h | 5 +---
> 3 files changed, 18 insertions(+), 41 deletions(-)
>
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 1ffaef5ec5ff..16212b6b202e 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -218,7 +218,7 @@ panfrost_copy_in_sync(struct drm_device *dev,
> if (ret)
> goto fail;
>
> - ret = drm_gem_fence_array_add(&job->deps, fence);
> + ret = drm_sched_job_add_dependency(&job->base, fence);
>
> if (ret)
> goto fail;
> @@ -236,7 +236,7 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
> struct drm_panfrost_submit *args = data;
> struct drm_syncobj *sync_out = NULL;
> struct panfrost_job *job;
> - int ret = 0;
> + int ret = 0, slot;
>
> if (!args->jc)
> return -EINVAL;
> @@ -258,14 +258,20 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
>
> kref_init(&job->refcount);
>
> - xa_init_flags(&job->deps, XA_FLAGS_ALLOC);
> -
> job->pfdev = pfdev;
> job->jc = args->jc;
> job->requirements = args->requirements;
> job->flush_id = panfrost_gpu_get_latest_flush_id(pfdev);
> job->file_priv = file->driver_priv;
>
> + slot = panfrost_job_get_slot(job);
> +
> + ret = drm_sched_job_init(&job->base,
> + &job->file_priv->sched_entity[slot],
> + NULL);
> + if (ret)
> + goto fail_job_put;
> +
> ret = panfrost_copy_in_sync(dev, file, args, job);
> if (ret)
> goto fail_job;
> @@ -283,6 +289,8 @@ static int panfrost_ioctl_submit(struct drm_device *dev, void *data,
> drm_syncobj_replace_fence(sync_out, job->render_done_fence);
>
> fail_job:
> + drm_sched_job_cleanup(&job->base);
> +fail_job_put:
> panfrost_job_put(job);
> fail_out_sync:
> if (sync_out)
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> index 4bc962763e1f..a98f507dc779 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> @@ -102,7 +102,7 @@ static struct dma_fence *panfrost_fence_create(struct panfrost_device *pfdev, in
> return &fence->base;
> }
>
> -static int panfrost_job_get_slot(struct panfrost_job *job)
> +int panfrost_job_get_slot(struct panfrost_job *job)
> {
> /* JS0: fragment jobs.
> * JS1: vertex/tiler jobs
> @@ -242,13 +242,14 @@ static void panfrost_job_hw_submit(struct panfrost_job *job, int js)
>
> static int panfrost_acquire_object_fences(struct drm_gem_object **bos,
> int bo_count,
> - struct xarray *deps)
> + struct drm_sched_job *job)
> {
> int i, ret;
>
> for (i = 0; i < bo_count; i++) {
> /* panfrost always uses write mode in its current uapi */
> - ret = drm_gem_fence_array_add_implicit(deps, bos[i], true);
> + ret = drm_sched_job_add_implicit_dependencies(job, bos[i],
> + true);
> if (ret)
> return ret;
> }
> @@ -269,31 +270,21 @@ static void panfrost_attach_object_fences(struct drm_gem_object **bos,
> int panfrost_job_push(struct panfrost_job *job)
> {
> struct panfrost_device *pfdev = job->pfdev;
> - int slot = panfrost_job_get_slot(job);
> - struct drm_sched_entity *entity = &job->file_priv->sched_entity[slot];
> struct ww_acquire_ctx acquire_ctx;
> int ret = 0;
>
> -
> ret = drm_gem_lock_reservations(job->bos, job->bo_count,
> &acquire_ctx);
> if (ret)
> return ret;
>
> mutex_lock(&pfdev->sched_lock);
> -
> - ret = drm_sched_job_init(&job->base, entity, NULL);
> - if (ret) {
> - mutex_unlock(&pfdev->sched_lock);
> - goto unlock;
> - }
> -
> drm_sched_job_arm(&job->base);
>
> job->render_done_fence = dma_fence_get(&job->base.s_fence->finished);
>
> ret = panfrost_acquire_object_fences(job->bos, job->bo_count,
> - &job->deps);
> + &job->base);
> if (ret) {
> mutex_unlock(&pfdev->sched_lock);
> goto unlock;
> @@ -318,15 +309,8 @@ static void panfrost_job_cleanup(struct kref *ref)
> {
> struct panfrost_job *job = container_of(ref, struct panfrost_job,
> refcount);
> - struct dma_fence *fence;
> - unsigned long index;
> unsigned int i;
>
> - xa_for_each(&job->deps, index, fence) {
> - dma_fence_put(fence);
> - }
> - xa_destroy(&job->deps);
> -
> dma_fence_put(job->done_fence);
> dma_fence_put(job->render_done_fence);
>
> @@ -365,17 +349,6 @@ static void panfrost_job_free(struct drm_sched_job *sched_job)
> panfrost_job_put(job);
> }
>
> -static struct dma_fence *panfrost_job_dependency(struct drm_sched_job *sched_job,
> - struct drm_sched_entity *s_entity)
> -{
> - struct panfrost_job *job = to_panfrost_job(sched_job);
> -
> - if (!xa_empty(&job->deps))
> - return xa_erase(&job->deps, job->last_dep++);
> -
> - return NULL;
> -}
> -
> static struct dma_fence *panfrost_job_run(struct drm_sched_job *sched_job)
> {
> struct panfrost_job *job = to_panfrost_job(sched_job);
> @@ -765,7 +738,6 @@ static void panfrost_reset_work(struct work_struct *work)
> }
>
> static const struct drm_sched_backend_ops panfrost_sched_ops = {
> - .dependency = panfrost_job_dependency,
> .run_job = panfrost_job_run,
> .timedout_job = panfrost_job_timedout,
> .free_job = panfrost_job_free
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.h b/drivers/gpu/drm/panfrost/panfrost_job.h
> index 82306a03b57e..77e6d0e6f612 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.h
> @@ -19,10 +19,6 @@ struct panfrost_job {
> struct panfrost_device *pfdev;
> struct panfrost_file_priv *file_priv;
>
> - /* Contains both explicit and implicit fences */
> - struct xarray deps;
> - unsigned long last_dep;
> -
> /* Fence to be signaled by IRQ handler when the job is complete. */
> struct dma_fence *done_fence;
>
> @@ -42,6 +38,7 @@ int panfrost_job_init(struct panfrost_device *pfdev);
> void panfrost_job_fini(struct panfrost_device *pfdev);
> int panfrost_job_open(struct panfrost_file_priv *panfrost_priv);
> void panfrost_job_close(struct panfrost_file_priv *panfrost_priv);
> +int panfrost_job_get_slot(struct panfrost_job *job);
> int panfrost_job_push(struct panfrost_job *job);
> void panfrost_job_put(struct panfrost_job *job);
> void panfrost_job_enable_interrupts(struct panfrost_device *pfdev);
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 08/20] drm/lima: use scheduler dependency tracking
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Qiang Yu, Sumit Semwal, Christian König, lima, linux-media,
linaro-mm-sig
Nothing special going on here.
Aside reviewing the code, it seems like drm_sched_job_arm() should be
moved into lima_sched_context_queue_task and put under some mutex
together with drm_sched_push_job(). See the kerneldoc for
drm_sched_push_job().
v2: Rebase over renamed functions to add dependencies.
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Qiang Yu <yuq825@gmail.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: lima@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/gpu/drm/lima/lima_gem.c | 6 ++++--
drivers/gpu/drm/lima/lima_sched.c | 21 ---------------------
drivers/gpu/drm/lima/lima_sched.h | 3 ---
3 files changed, 4 insertions(+), 26 deletions(-)
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index c528f40981bb..640acc060467 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -267,7 +267,9 @@ static int lima_gem_sync_bo(struct lima_sched_task *task, struct lima_bo *bo,
if (explicit)
return 0;
- return drm_gem_fence_array_add_implicit(&task->deps, &bo->base.base, write);
+ return drm_sched_job_add_implicit_dependencies(&task->base,
+ &bo->base.base,
+ write);
}
static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
@@ -285,7 +287,7 @@ static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
if (err)
return err;
- err = drm_gem_fence_array_add(&submit->task->deps, fence);
+ err = drm_sched_job_add_dependency(&submit->task->base, fence);
if (err) {
dma_fence_put(fence);
return err;
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index e968b5a8f0b0..99d5f6f1a882 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -134,24 +134,15 @@ int lima_sched_task_init(struct lima_sched_task *task,
task->num_bos = num_bos;
task->vm = lima_vm_get(vm);
- xa_init_flags(&task->deps, XA_FLAGS_ALLOC);
-
return 0;
}
void lima_sched_task_fini(struct lima_sched_task *task)
{
- struct dma_fence *fence;
- unsigned long index;
int i;
drm_sched_job_cleanup(&task->base);
- xa_for_each(&task->deps, index, fence) {
- dma_fence_put(fence);
- }
- xa_destroy(&task->deps);
-
if (task->bos) {
for (i = 0; i < task->num_bos; i++)
drm_gem_object_put(&task->bos[i]->base.base);
@@ -186,17 +177,6 @@ struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task)
return fence;
}
-static struct dma_fence *lima_sched_dependency(struct drm_sched_job *job,
- struct drm_sched_entity *entity)
-{
- struct lima_sched_task *task = to_lima_task(job);
-
- if (!xa_empty(&task->deps))
- return xa_erase(&task->deps, task->last_dep++);
-
- return NULL;
-}
-
static int lima_pm_busy(struct lima_device *ldev)
{
int ret;
@@ -472,7 +452,6 @@ static void lima_sched_free_job(struct drm_sched_job *job)
}
static const struct drm_sched_backend_ops lima_sched_ops = {
- .dependency = lima_sched_dependency,
.run_job = lima_sched_run_job,
.timedout_job = lima_sched_timedout_job,
.free_job = lima_sched_free_job,
diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
index ac70006b0e26..6a11764d87b3 100644
--- a/drivers/gpu/drm/lima/lima_sched.h
+++ b/drivers/gpu/drm/lima/lima_sched.h
@@ -23,9 +23,6 @@ struct lima_sched_task {
struct lima_vm *vm;
void *frame;
- struct xarray deps;
- unsigned long last_dep;
-
struct lima_bo **bos;
int num_bos;
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 08/20] drm/lima: use scheduler dependency tracking
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Qiang Yu, Sumit Semwal, Christian König, lima, linux-media,
linaro-mm-sig
Nothing special going on here.
Aside reviewing the code, it seems like drm_sched_job_arm() should be
moved into lima_sched_context_queue_task and put under some mutex
together with drm_sched_push_job(). See the kerneldoc for
drm_sched_push_job().
v2: Rebase over renamed functions to add dependencies.
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Qiang Yu <yuq825@gmail.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: lima@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/gpu/drm/lima/lima_gem.c | 6 ++++--
drivers/gpu/drm/lima/lima_sched.c | 21 ---------------------
drivers/gpu/drm/lima/lima_sched.h | 3 ---
3 files changed, 4 insertions(+), 26 deletions(-)
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index c528f40981bb..640acc060467 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -267,7 +267,9 @@ static int lima_gem_sync_bo(struct lima_sched_task *task, struct lima_bo *bo,
if (explicit)
return 0;
- return drm_gem_fence_array_add_implicit(&task->deps, &bo->base.base, write);
+ return drm_sched_job_add_implicit_dependencies(&task->base,
+ &bo->base.base,
+ write);
}
static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
@@ -285,7 +287,7 @@ static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
if (err)
return err;
- err = drm_gem_fence_array_add(&submit->task->deps, fence);
+ err = drm_sched_job_add_dependency(&submit->task->base, fence);
if (err) {
dma_fence_put(fence);
return err;
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index e968b5a8f0b0..99d5f6f1a882 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -134,24 +134,15 @@ int lima_sched_task_init(struct lima_sched_task *task,
task->num_bos = num_bos;
task->vm = lima_vm_get(vm);
- xa_init_flags(&task->deps, XA_FLAGS_ALLOC);
-
return 0;
}
void lima_sched_task_fini(struct lima_sched_task *task)
{
- struct dma_fence *fence;
- unsigned long index;
int i;
drm_sched_job_cleanup(&task->base);
- xa_for_each(&task->deps, index, fence) {
- dma_fence_put(fence);
- }
- xa_destroy(&task->deps);
-
if (task->bos) {
for (i = 0; i < task->num_bos; i++)
drm_gem_object_put(&task->bos[i]->base.base);
@@ -186,17 +177,6 @@ struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task)
return fence;
}
-static struct dma_fence *lima_sched_dependency(struct drm_sched_job *job,
- struct drm_sched_entity *entity)
-{
- struct lima_sched_task *task = to_lima_task(job);
-
- if (!xa_empty(&task->deps))
- return xa_erase(&task->deps, task->last_dep++);
-
- return NULL;
-}
-
static int lima_pm_busy(struct lima_device *ldev)
{
int ret;
@@ -472,7 +452,6 @@ static void lima_sched_free_job(struct drm_sched_job *job)
}
static const struct drm_sched_backend_ops lima_sched_ops = {
- .dependency = lima_sched_dependency,
.run_job = lima_sched_run_job,
.timedout_job = lima_sched_timedout_job,
.free_job = lima_sched_free_job,
diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
index ac70006b0e26..6a11764d87b3 100644
--- a/drivers/gpu/drm/lima/lima_sched.h
+++ b/drivers/gpu/drm/lima/lima_sched.h
@@ -23,9 +23,6 @@ struct lima_sched_task {
struct lima_vm *vm;
void *frame;
- struct xarray deps;
- unsigned long last_dep;
-
struct lima_bo **bos;
int num_bos;
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 08/20] drm/lima: use scheduler dependency tracking
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-12 19:28 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-12 19:28 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Qiang Yu, Sumit Semwal, Christian König, lima, linux-media,
linaro-mm-sig
On Thu, Aug 05, 2021 at 12:46:53PM +0200, Daniel Vetter wrote:
> Nothing special going on here.
>
> Aside reviewing the code, it seems like drm_sched_job_arm() should be
> moved into lima_sched_context_queue_task and put under some mutex
> together with drm_sched_push_job(). See the kerneldoc for
> drm_sched_push_job().
>
> v2: Rebase over renamed functions to add dependencies.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: lima@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Ping for an ack here please. Testing would be even better ofc.
-Daniel
> ---
> drivers/gpu/drm/lima/lima_gem.c | 6 ++++--
> drivers/gpu/drm/lima/lima_sched.c | 21 ---------------------
> drivers/gpu/drm/lima/lima_sched.h | 3 ---
> 3 files changed, 4 insertions(+), 26 deletions(-)
>
> diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
> index c528f40981bb..640acc060467 100644
> --- a/drivers/gpu/drm/lima/lima_gem.c
> +++ b/drivers/gpu/drm/lima/lima_gem.c
> @@ -267,7 +267,9 @@ static int lima_gem_sync_bo(struct lima_sched_task *task, struct lima_bo *bo,
> if (explicit)
> return 0;
>
> - return drm_gem_fence_array_add_implicit(&task->deps, &bo->base.base, write);
> + return drm_sched_job_add_implicit_dependencies(&task->base,
> + &bo->base.base,
> + write);
> }
>
> static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
> @@ -285,7 +287,7 @@ static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
> if (err)
> return err;
>
> - err = drm_gem_fence_array_add(&submit->task->deps, fence);
> + err = drm_sched_job_add_dependency(&submit->task->base, fence);
> if (err) {
> dma_fence_put(fence);
> return err;
> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> index e968b5a8f0b0..99d5f6f1a882 100644
> --- a/drivers/gpu/drm/lima/lima_sched.c
> +++ b/drivers/gpu/drm/lima/lima_sched.c
> @@ -134,24 +134,15 @@ int lima_sched_task_init(struct lima_sched_task *task,
> task->num_bos = num_bos;
> task->vm = lima_vm_get(vm);
>
> - xa_init_flags(&task->deps, XA_FLAGS_ALLOC);
> -
> return 0;
> }
>
> void lima_sched_task_fini(struct lima_sched_task *task)
> {
> - struct dma_fence *fence;
> - unsigned long index;
> int i;
>
> drm_sched_job_cleanup(&task->base);
>
> - xa_for_each(&task->deps, index, fence) {
> - dma_fence_put(fence);
> - }
> - xa_destroy(&task->deps);
> -
> if (task->bos) {
> for (i = 0; i < task->num_bos; i++)
> drm_gem_object_put(&task->bos[i]->base.base);
> @@ -186,17 +177,6 @@ struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task)
> return fence;
> }
>
> -static struct dma_fence *lima_sched_dependency(struct drm_sched_job *job,
> - struct drm_sched_entity *entity)
> -{
> - struct lima_sched_task *task = to_lima_task(job);
> -
> - if (!xa_empty(&task->deps))
> - return xa_erase(&task->deps, task->last_dep++);
> -
> - return NULL;
> -}
> -
> static int lima_pm_busy(struct lima_device *ldev)
> {
> int ret;
> @@ -472,7 +452,6 @@ static void lima_sched_free_job(struct drm_sched_job *job)
> }
>
> static const struct drm_sched_backend_ops lima_sched_ops = {
> - .dependency = lima_sched_dependency,
> .run_job = lima_sched_run_job,
> .timedout_job = lima_sched_timedout_job,
> .free_job = lima_sched_free_job,
> diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
> index ac70006b0e26..6a11764d87b3 100644
> --- a/drivers/gpu/drm/lima/lima_sched.h
> +++ b/drivers/gpu/drm/lima/lima_sched.h
> @@ -23,9 +23,6 @@ struct lima_sched_task {
> struct lima_vm *vm;
> void *frame;
>
> - struct xarray deps;
> - unsigned long last_dep;
> -
> struct lima_bo **bos;
> int num_bos;
>
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 08/20] drm/lima: use scheduler dependency tracking
@ 2021-08-12 19:28 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-12 19:28 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Qiang Yu, Sumit Semwal, Christian König, lima, linux-media,
linaro-mm-sig
On Thu, Aug 05, 2021 at 12:46:53PM +0200, Daniel Vetter wrote:
> Nothing special going on here.
>
> Aside reviewing the code, it seems like drm_sched_job_arm() should be
> moved into lima_sched_context_queue_task and put under some mutex
> together with drm_sched_push_job(). See the kerneldoc for
> drm_sched_push_job().
>
> v2: Rebase over renamed functions to add dependencies.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Qiang Yu <yuq825@gmail.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: lima@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Ping for an ack here please. Testing would be even better ofc.
-Daniel
> ---
> drivers/gpu/drm/lima/lima_gem.c | 6 ++++--
> drivers/gpu/drm/lima/lima_sched.c | 21 ---------------------
> drivers/gpu/drm/lima/lima_sched.h | 3 ---
> 3 files changed, 4 insertions(+), 26 deletions(-)
>
> diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
> index c528f40981bb..640acc060467 100644
> --- a/drivers/gpu/drm/lima/lima_gem.c
> +++ b/drivers/gpu/drm/lima/lima_gem.c
> @@ -267,7 +267,9 @@ static int lima_gem_sync_bo(struct lima_sched_task *task, struct lima_bo *bo,
> if (explicit)
> return 0;
>
> - return drm_gem_fence_array_add_implicit(&task->deps, &bo->base.base, write);
> + return drm_sched_job_add_implicit_dependencies(&task->base,
> + &bo->base.base,
> + write);
> }
>
> static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
> @@ -285,7 +287,7 @@ static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
> if (err)
> return err;
>
> - err = drm_gem_fence_array_add(&submit->task->deps, fence);
> + err = drm_sched_job_add_dependency(&submit->task->base, fence);
> if (err) {
> dma_fence_put(fence);
> return err;
> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> index e968b5a8f0b0..99d5f6f1a882 100644
> --- a/drivers/gpu/drm/lima/lima_sched.c
> +++ b/drivers/gpu/drm/lima/lima_sched.c
> @@ -134,24 +134,15 @@ int lima_sched_task_init(struct lima_sched_task *task,
> task->num_bos = num_bos;
> task->vm = lima_vm_get(vm);
>
> - xa_init_flags(&task->deps, XA_FLAGS_ALLOC);
> -
> return 0;
> }
>
> void lima_sched_task_fini(struct lima_sched_task *task)
> {
> - struct dma_fence *fence;
> - unsigned long index;
> int i;
>
> drm_sched_job_cleanup(&task->base);
>
> - xa_for_each(&task->deps, index, fence) {
> - dma_fence_put(fence);
> - }
> - xa_destroy(&task->deps);
> -
> if (task->bos) {
> for (i = 0; i < task->num_bos; i++)
> drm_gem_object_put(&task->bos[i]->base.base);
> @@ -186,17 +177,6 @@ struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task)
> return fence;
> }
>
> -static struct dma_fence *lima_sched_dependency(struct drm_sched_job *job,
> - struct drm_sched_entity *entity)
> -{
> - struct lima_sched_task *task = to_lima_task(job);
> -
> - if (!xa_empty(&task->deps))
> - return xa_erase(&task->deps, task->last_dep++);
> -
> - return NULL;
> -}
> -
> static int lima_pm_busy(struct lima_device *ldev)
> {
> int ret;
> @@ -472,7 +452,6 @@ static void lima_sched_free_job(struct drm_sched_job *job)
> }
>
> static const struct drm_sched_backend_ops lima_sched_ops = {
> - .dependency = lima_sched_dependency,
> .run_job = lima_sched_run_job,
> .timedout_job = lima_sched_timedout_job,
> .free_job = lima_sched_free_job,
> diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
> index ac70006b0e26..6a11764d87b3 100644
> --- a/drivers/gpu/drm/lima/lima_sched.h
> +++ b/drivers/gpu/drm/lima/lima_sched.h
> @@ -23,9 +23,6 @@ struct lima_sched_task {
> struct lima_vm *vm;
> void *frame;
>
> - struct xarray deps;
> - unsigned long last_dep;
> -
> struct lima_bo **bos;
> int num_bos;
>
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 08/20] drm/lima: use scheduler dependency tracking
2021-08-12 19:28 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-14 2:45 ` Qiang Yu
-1 siblings, 0 replies; 122+ messages in thread
From: Qiang Yu @ 2021-08-14 2:45 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Daniel Vetter,
Daniel Vetter, Sumit Semwal, Christian König, lima,
linux-media, linaro-mm-sig
Thanks for the remind, indeed miss a lock. Patch is:
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Regards,
Qiang
On Fri, Aug 13, 2021 at 3:28 AM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Thu, Aug 05, 2021 at 12:46:53PM +0200, Daniel Vetter wrote:
> > Nothing special going on here.
> >
> > Aside reviewing the code, it seems like drm_sched_job_arm() should be
> > moved into lima_sched_context_queue_task and put under some mutex
> > together with drm_sched_push_job(). See the kerneldoc for
> > drm_sched_push_job().
> >
> > v2: Rebase over renamed functions to add dependencies.
> >
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: Qiang Yu <yuq825@gmail.com>
> > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: lima@lists.freedesktop.org
> > Cc: linux-media@vger.kernel.org
> > Cc: linaro-mm-sig@lists.linaro.org
>
> Ping for an ack here please. Testing would be even better ofc.
> -Daniel
>
> > ---
> > drivers/gpu/drm/lima/lima_gem.c | 6 ++++--
> > drivers/gpu/drm/lima/lima_sched.c | 21 ---------------------
> > drivers/gpu/drm/lima/lima_sched.h | 3 ---
> > 3 files changed, 4 insertions(+), 26 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
> > index c528f40981bb..640acc060467 100644
> > --- a/drivers/gpu/drm/lima/lima_gem.c
> > +++ b/drivers/gpu/drm/lima/lima_gem.c
> > @@ -267,7 +267,9 @@ static int lima_gem_sync_bo(struct lima_sched_task *task, struct lima_bo *bo,
> > if (explicit)
> > return 0;
> >
> > - return drm_gem_fence_array_add_implicit(&task->deps, &bo->base.base, write);
> > + return drm_sched_job_add_implicit_dependencies(&task->base,
> > + &bo->base.base,
> > + write);
> > }
> >
> > static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
> > @@ -285,7 +287,7 @@ static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
> > if (err)
> > return err;
> >
> > - err = drm_gem_fence_array_add(&submit->task->deps, fence);
> > + err = drm_sched_job_add_dependency(&submit->task->base, fence);
> > if (err) {
> > dma_fence_put(fence);
> > return err;
> > diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> > index e968b5a8f0b0..99d5f6f1a882 100644
> > --- a/drivers/gpu/drm/lima/lima_sched.c
> > +++ b/drivers/gpu/drm/lima/lima_sched.c
> > @@ -134,24 +134,15 @@ int lima_sched_task_init(struct lima_sched_task *task,
> > task->num_bos = num_bos;
> > task->vm = lima_vm_get(vm);
> >
> > - xa_init_flags(&task->deps, XA_FLAGS_ALLOC);
> > -
> > return 0;
> > }
> >
> > void lima_sched_task_fini(struct lima_sched_task *task)
> > {
> > - struct dma_fence *fence;
> > - unsigned long index;
> > int i;
> >
> > drm_sched_job_cleanup(&task->base);
> >
> > - xa_for_each(&task->deps, index, fence) {
> > - dma_fence_put(fence);
> > - }
> > - xa_destroy(&task->deps);
> > -
> > if (task->bos) {
> > for (i = 0; i < task->num_bos; i++)
> > drm_gem_object_put(&task->bos[i]->base.base);
> > @@ -186,17 +177,6 @@ struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task)
> > return fence;
> > }
> >
> > -static struct dma_fence *lima_sched_dependency(struct drm_sched_job *job,
> > - struct drm_sched_entity *entity)
> > -{
> > - struct lima_sched_task *task = to_lima_task(job);
> > -
> > - if (!xa_empty(&task->deps))
> > - return xa_erase(&task->deps, task->last_dep++);
> > -
> > - return NULL;
> > -}
> > -
> > static int lima_pm_busy(struct lima_device *ldev)
> > {
> > int ret;
> > @@ -472,7 +452,6 @@ static void lima_sched_free_job(struct drm_sched_job *job)
> > }
> >
> > static const struct drm_sched_backend_ops lima_sched_ops = {
> > - .dependency = lima_sched_dependency,
> > .run_job = lima_sched_run_job,
> > .timedout_job = lima_sched_timedout_job,
> > .free_job = lima_sched_free_job,
> > diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
> > index ac70006b0e26..6a11764d87b3 100644
> > --- a/drivers/gpu/drm/lima/lima_sched.h
> > +++ b/drivers/gpu/drm/lima/lima_sched.h
> > @@ -23,9 +23,6 @@ struct lima_sched_task {
> > struct lima_vm *vm;
> > void *frame;
> >
> > - struct xarray deps;
> > - unsigned long last_dep;
> > -
> > struct lima_bo **bos;
> > int num_bos;
> >
> > --
> > 2.32.0
> >
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 08/20] drm/lima: use scheduler dependency tracking
@ 2021-08-14 2:45 ` Qiang Yu
0 siblings, 0 replies; 122+ messages in thread
From: Qiang Yu @ 2021-08-14 2:45 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Daniel Vetter,
Daniel Vetter, Sumit Semwal, Christian König, lima,
linux-media, linaro-mm-sig
Thanks for the remind, indeed miss a lock. Patch is:
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Regards,
Qiang
On Fri, Aug 13, 2021 at 3:28 AM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Thu, Aug 05, 2021 at 12:46:53PM +0200, Daniel Vetter wrote:
> > Nothing special going on here.
> >
> > Aside reviewing the code, it seems like drm_sched_job_arm() should be
> > moved into lima_sched_context_queue_task and put under some mutex
> > together with drm_sched_push_job(). See the kerneldoc for
> > drm_sched_push_job().
> >
> > v2: Rebase over renamed functions to add dependencies.
> >
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: Qiang Yu <yuq825@gmail.com>
> > Cc: Sumit Semwal <sumit.semwal@linaro.org>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: lima@lists.freedesktop.org
> > Cc: linux-media@vger.kernel.org
> > Cc: linaro-mm-sig@lists.linaro.org
>
> Ping for an ack here please. Testing would be even better ofc.
> -Daniel
>
> > ---
> > drivers/gpu/drm/lima/lima_gem.c | 6 ++++--
> > drivers/gpu/drm/lima/lima_sched.c | 21 ---------------------
> > drivers/gpu/drm/lima/lima_sched.h | 3 ---
> > 3 files changed, 4 insertions(+), 26 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
> > index c528f40981bb..640acc060467 100644
> > --- a/drivers/gpu/drm/lima/lima_gem.c
> > +++ b/drivers/gpu/drm/lima/lima_gem.c
> > @@ -267,7 +267,9 @@ static int lima_gem_sync_bo(struct lima_sched_task *task, struct lima_bo *bo,
> > if (explicit)
> > return 0;
> >
> > - return drm_gem_fence_array_add_implicit(&task->deps, &bo->base.base, write);
> > + return drm_sched_job_add_implicit_dependencies(&task->base,
> > + &bo->base.base,
> > + write);
> > }
> >
> > static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
> > @@ -285,7 +287,7 @@ static int lima_gem_add_deps(struct drm_file *file, struct lima_submit *submit)
> > if (err)
> > return err;
> >
> > - err = drm_gem_fence_array_add(&submit->task->deps, fence);
> > + err = drm_sched_job_add_dependency(&submit->task->base, fence);
> > if (err) {
> > dma_fence_put(fence);
> > return err;
> > diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> > index e968b5a8f0b0..99d5f6f1a882 100644
> > --- a/drivers/gpu/drm/lima/lima_sched.c
> > +++ b/drivers/gpu/drm/lima/lima_sched.c
> > @@ -134,24 +134,15 @@ int lima_sched_task_init(struct lima_sched_task *task,
> > task->num_bos = num_bos;
> > task->vm = lima_vm_get(vm);
> >
> > - xa_init_flags(&task->deps, XA_FLAGS_ALLOC);
> > -
> > return 0;
> > }
> >
> > void lima_sched_task_fini(struct lima_sched_task *task)
> > {
> > - struct dma_fence *fence;
> > - unsigned long index;
> > int i;
> >
> > drm_sched_job_cleanup(&task->base);
> >
> > - xa_for_each(&task->deps, index, fence) {
> > - dma_fence_put(fence);
> > - }
> > - xa_destroy(&task->deps);
> > -
> > if (task->bos) {
> > for (i = 0; i < task->num_bos; i++)
> > drm_gem_object_put(&task->bos[i]->base.base);
> > @@ -186,17 +177,6 @@ struct dma_fence *lima_sched_context_queue_task(struct lima_sched_task *task)
> > return fence;
> > }
> >
> > -static struct dma_fence *lima_sched_dependency(struct drm_sched_job *job,
> > - struct drm_sched_entity *entity)
> > -{
> > - struct lima_sched_task *task = to_lima_task(job);
> > -
> > - if (!xa_empty(&task->deps))
> > - return xa_erase(&task->deps, task->last_dep++);
> > -
> > - return NULL;
> > -}
> > -
> > static int lima_pm_busy(struct lima_device *ldev)
> > {
> > int ret;
> > @@ -472,7 +452,6 @@ static void lima_sched_free_job(struct drm_sched_job *job)
> > }
> >
> > static const struct drm_sched_backend_ops lima_sched_ops = {
> > - .dependency = lima_sched_dependency,
> > .run_job = lima_sched_run_job,
> > .timedout_job = lima_sched_timedout_job,
> > .free_job = lima_sched_free_job,
> > diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
> > index ac70006b0e26..6a11764d87b3 100644
> > --- a/drivers/gpu/drm/lima/lima_sched.h
> > +++ b/drivers/gpu/drm/lima/lima_sched.h
> > @@ -23,9 +23,6 @@ struct lima_sched_task {
> > struct lima_vm *vm;
> > void *frame;
> >
> > - struct xarray deps;
> > - unsigned long last_dep;
> > -
> > struct lima_bo **bos;
> > int num_bos;
> >
> > --
> > 2.32.0
> >
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 09/20] drm/v3d: Move drm_sched_job_init to v3d_job_init
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Emma Anholt, Melissa Wen, Daniel Vetter
Prep work for using the scheduler dependency handling. We need to call
drm_sched_job_init earlier so we can use the new drm_sched_job_await*
functions for dependency handling here.
v2: Slightly better commit message and rebase to include the
drm_sched_job_arm() call (Emma).
v3: Cleanup jobs under construction correctly (Emma)
v4: Rebase over perfmon patch
Reviewed-by: Melissa Wen <mwen@igalia.com> (v3)
Acked-by: Emma Anholt <emma@anholt.net>
Cc: Melissa Wen <melissa.srw@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Emma Anholt <emma@anholt.net>
---
drivers/gpu/drm/v3d/v3d_drv.h | 1 +
drivers/gpu/drm/v3d/v3d_gem.c | 86 ++++++++++++++-------------------
drivers/gpu/drm/v3d/v3d_sched.c | 15 +++---
3 files changed, 44 insertions(+), 58 deletions(-)
diff --git a/drivers/gpu/drm/v3d/v3d_drv.h b/drivers/gpu/drm/v3d/v3d_drv.h
index 270134779073..c1d433b4cf93 100644
--- a/drivers/gpu/drm/v3d/v3d_drv.h
+++ b/drivers/gpu/drm/v3d/v3d_drv.h
@@ -379,6 +379,7 @@ int v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
int v3d_wait_bo_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
+void v3d_job_cleanup(struct v3d_job *job);
void v3d_job_put(struct v3d_job *job);
void v3d_reset(struct v3d_dev *v3d);
void v3d_invalidate_caches(struct v3d_dev *v3d);
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 957228bef29c..42587248c54e 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -397,6 +397,12 @@ v3d_render_job_free(struct kref *ref)
v3d_job_free(ref);
}
+void v3d_job_cleanup(struct v3d_job *job)
+{
+ drm_sched_job_cleanup(&job->base);
+ v3d_job_put(job);
+}
+
void v3d_job_put(struct v3d_job *job)
{
kref_put(&job->refcount, job->free);
@@ -438,9 +444,10 @@ v3d_wait_bo_ioctl(struct drm_device *dev, void *data,
static int
v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv,
struct v3d_job *job, void (*free)(struct kref *ref),
- u32 in_sync)
+ u32 in_sync, enum v3d_queue queue)
{
struct dma_fence *in_fence = NULL;
+ struct v3d_file_priv *v3d_priv = file_priv->driver_priv;
int ret;
job->v3d = v3d;
@@ -451,35 +458,33 @@ v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv,
return ret;
xa_init_flags(&job->deps, XA_FLAGS_ALLOC);
+ ret = drm_sched_job_init(&job->base, &v3d_priv->sched_entity[queue],
+ v3d_priv);
+ if (ret)
+ goto fail;
ret = drm_syncobj_find_fence(file_priv, in_sync, 0, 0, &in_fence);
if (ret == -EINVAL)
- goto fail;
+ goto fail_job;
ret = drm_gem_fence_array_add(&job->deps, in_fence);
if (ret)
- goto fail;
+ goto fail_job;
kref_init(&job->refcount);
return 0;
+fail_job:
+ drm_sched_job_cleanup(&job->base);
fail:
xa_destroy(&job->deps);
pm_runtime_put_autosuspend(v3d->drm.dev);
return ret;
}
-static int
-v3d_push_job(struct v3d_file_priv *v3d_priv,
- struct v3d_job *job, enum v3d_queue queue)
+static void
+v3d_push_job(struct v3d_job *job)
{
- int ret;
-
- ret = drm_sched_job_init(&job->base, &v3d_priv->sched_entity[queue],
- v3d_priv);
- if (ret)
- return ret;
-
drm_sched_job_arm(&job->base);
job->done_fence = dma_fence_get(&job->base.s_fence->finished);
@@ -488,8 +493,6 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
kref_get(&job->refcount);
drm_sched_entity_push_job(&job->base);
-
- return 0;
}
static void
@@ -564,7 +567,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
INIT_LIST_HEAD(&render->unref_list);
ret = v3d_job_init(v3d, file_priv, &render->base,
- v3d_render_job_free, args->in_sync_rcl);
+ v3d_render_job_free, args->in_sync_rcl, V3D_RENDER);
if (ret) {
kfree(render);
return ret;
@@ -578,7 +581,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
}
ret = v3d_job_init(v3d, file_priv, &bin->base,
- v3d_job_free, args->in_sync_bcl);
+ v3d_job_free, args->in_sync_bcl, V3D_BIN);
if (ret) {
v3d_job_put(&render->base);
kfree(bin);
@@ -600,7 +603,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
goto fail;
}
- ret = v3d_job_init(v3d, file_priv, clean_job, v3d_job_free, 0);
+ ret = v3d_job_init(v3d, file_priv, clean_job, v3d_job_free, 0, V3D_CACHE_CLEAN);
if (ret) {
kfree(clean_job);
clean_job = NULL;
@@ -635,9 +638,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
if (bin) {
bin->base.perfmon = render->base.perfmon;
v3d_perfmon_get(bin->base.perfmon);
- ret = v3d_push_job(v3d_priv, &bin->base, V3D_BIN);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(&bin->base);
ret = drm_gem_fence_array_add(&render->base.deps,
dma_fence_get(bin->base.done_fence));
@@ -645,9 +646,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
goto fail_unreserve;
}
- ret = v3d_push_job(v3d_priv, &render->base, V3D_RENDER);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(&render->base);
if (clean_job) {
struct dma_fence *render_fence =
@@ -657,9 +656,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
goto fail_unreserve;
clean_job->perfmon = render->base.perfmon;
v3d_perfmon_get(clean_job->perfmon);
- ret = v3d_push_job(v3d_priv, clean_job, V3D_CACHE_CLEAN);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(clean_job);
}
mutex_unlock(&v3d->sched_lock);
@@ -684,10 +681,10 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
last_job->bo_count, &acquire_ctx);
fail:
if (bin)
- v3d_job_put(&bin->base);
- v3d_job_put(&render->base);
+ v3d_job_cleanup(&bin->base);
+ v3d_job_cleanup(&render->base);
if (clean_job)
- v3d_job_put(clean_job);
+ v3d_job_cleanup(clean_job);
return ret;
}
@@ -706,7 +703,6 @@ v3d_submit_tfu_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv)
{
struct v3d_dev *v3d = to_v3d_dev(dev);
- struct v3d_file_priv *v3d_priv = file_priv->driver_priv;
struct drm_v3d_submit_tfu *args = data;
struct v3d_tfu_job *job;
struct ww_acquire_ctx acquire_ctx;
@@ -719,7 +715,7 @@ v3d_submit_tfu_ioctl(struct drm_device *dev, void *data,
return -ENOMEM;
ret = v3d_job_init(v3d, file_priv, &job->base,
- v3d_job_free, args->in_sync);
+ v3d_job_free, args->in_sync, V3D_TFU);
if (ret) {
kfree(job);
return ret;
@@ -763,9 +759,7 @@ v3d_submit_tfu_ioctl(struct drm_device *dev, void *data,
goto fail;
mutex_lock(&v3d->sched_lock);
- ret = v3d_push_job(v3d_priv, &job->base, V3D_TFU);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(&job->base);
mutex_unlock(&v3d->sched_lock);
v3d_attach_fences_and_unlock_reservation(file_priv,
@@ -777,12 +771,8 @@ v3d_submit_tfu_ioctl(struct drm_device *dev, void *data,
return 0;
-fail_unreserve:
- mutex_unlock(&v3d->sched_lock);
- drm_gem_unlock_reservations(job->base.bo, job->base.bo_count,
- &acquire_ctx);
fail:
- v3d_job_put(&job->base);
+ v3d_job_cleanup(&job->base);
return ret;
}
@@ -820,7 +810,7 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
return -ENOMEM;
ret = v3d_job_init(v3d, file_priv, &job->base,
- v3d_job_free, args->in_sync);
+ v3d_job_free, args->in_sync, V3D_CSD);
if (ret) {
kfree(job);
return ret;
@@ -833,7 +823,7 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
return -ENOMEM;
}
- ret = v3d_job_init(v3d, file_priv, clean_job, v3d_job_free, 0);
+ ret = v3d_job_init(v3d, file_priv, clean_job, v3d_job_free, 0, V3D_CACHE_CLEAN);
if (ret) {
v3d_job_put(&job->base);
kfree(clean_job);
@@ -861,18 +851,14 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
}
mutex_lock(&v3d->sched_lock);
- ret = v3d_push_job(v3d_priv, &job->base, V3D_CSD);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(&job->base);
ret = drm_gem_fence_array_add(&clean_job->deps,
dma_fence_get(job->base.done_fence));
if (ret)
goto fail_unreserve;
- ret = v3d_push_job(v3d_priv, clean_job, V3D_CACHE_CLEAN);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(clean_job);
mutex_unlock(&v3d->sched_lock);
v3d_attach_fences_and_unlock_reservation(file_priv,
@@ -891,8 +877,8 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
drm_gem_unlock_reservations(clean_job->bo, clean_job->bo_count,
&acquire_ctx);
fail:
- v3d_job_put(&job->base);
- v3d_job_put(clean_job);
+ v3d_job_cleanup(&job->base);
+ v3d_job_cleanup(clean_job);
return ret;
}
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index dd7fcc36d726..7969797793d0 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -55,12 +55,11 @@ to_csd_job(struct drm_sched_job *sched_job)
}
static void
-v3d_job_free(struct drm_sched_job *sched_job)
+v3d_sched_job_free(struct drm_sched_job *sched_job)
{
struct v3d_job *job = to_v3d_job(sched_job);
- drm_sched_job_cleanup(sched_job);
- v3d_job_put(job);
+ v3d_job_cleanup(job);
}
static void
@@ -376,35 +375,35 @@ static const struct drm_sched_backend_ops v3d_bin_sched_ops = {
.dependency = v3d_job_dependency,
.run_job = v3d_bin_job_run,
.timedout_job = v3d_bin_job_timedout,
- .free_job = v3d_job_free,
+ .free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_render_sched_ops = {
.dependency = v3d_job_dependency,
.run_job = v3d_render_job_run,
.timedout_job = v3d_render_job_timedout,
- .free_job = v3d_job_free,
+ .free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_tfu_sched_ops = {
.dependency = v3d_job_dependency,
.run_job = v3d_tfu_job_run,
.timedout_job = v3d_generic_job_timedout,
- .free_job = v3d_job_free,
+ .free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_csd_sched_ops = {
.dependency = v3d_job_dependency,
.run_job = v3d_csd_job_run,
.timedout_job = v3d_csd_job_timedout,
- .free_job = v3d_job_free
+ .free_job = v3d_sched_job_free
};
static const struct drm_sched_backend_ops v3d_cache_clean_sched_ops = {
.dependency = v3d_job_dependency,
.run_job = v3d_cache_clean_job_run,
.timedout_job = v3d_generic_job_timedout,
- .free_job = v3d_job_free
+ .free_job = v3d_sched_job_free
};
int
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 09/20] drm/v3d: Move drm_sched_job_init to v3d_job_init
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Emma Anholt, Melissa Wen, Daniel Vetter
Prep work for using the scheduler dependency handling. We need to call
drm_sched_job_init earlier so we can use the new drm_sched_job_await*
functions for dependency handling here.
v2: Slightly better commit message and rebase to include the
drm_sched_job_arm() call (Emma).
v3: Cleanup jobs under construction correctly (Emma)
v4: Rebase over perfmon patch
Reviewed-by: Melissa Wen <mwen@igalia.com> (v3)
Acked-by: Emma Anholt <emma@anholt.net>
Cc: Melissa Wen <melissa.srw@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Emma Anholt <emma@anholt.net>
---
drivers/gpu/drm/v3d/v3d_drv.h | 1 +
drivers/gpu/drm/v3d/v3d_gem.c | 86 ++++++++++++++-------------------
drivers/gpu/drm/v3d/v3d_sched.c | 15 +++---
3 files changed, 44 insertions(+), 58 deletions(-)
diff --git a/drivers/gpu/drm/v3d/v3d_drv.h b/drivers/gpu/drm/v3d/v3d_drv.h
index 270134779073..c1d433b4cf93 100644
--- a/drivers/gpu/drm/v3d/v3d_drv.h
+++ b/drivers/gpu/drm/v3d/v3d_drv.h
@@ -379,6 +379,7 @@ int v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
int v3d_wait_bo_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
+void v3d_job_cleanup(struct v3d_job *job);
void v3d_job_put(struct v3d_job *job);
void v3d_reset(struct v3d_dev *v3d);
void v3d_invalidate_caches(struct v3d_dev *v3d);
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 957228bef29c..42587248c54e 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -397,6 +397,12 @@ v3d_render_job_free(struct kref *ref)
v3d_job_free(ref);
}
+void v3d_job_cleanup(struct v3d_job *job)
+{
+ drm_sched_job_cleanup(&job->base);
+ v3d_job_put(job);
+}
+
void v3d_job_put(struct v3d_job *job)
{
kref_put(&job->refcount, job->free);
@@ -438,9 +444,10 @@ v3d_wait_bo_ioctl(struct drm_device *dev, void *data,
static int
v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv,
struct v3d_job *job, void (*free)(struct kref *ref),
- u32 in_sync)
+ u32 in_sync, enum v3d_queue queue)
{
struct dma_fence *in_fence = NULL;
+ struct v3d_file_priv *v3d_priv = file_priv->driver_priv;
int ret;
job->v3d = v3d;
@@ -451,35 +458,33 @@ v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv,
return ret;
xa_init_flags(&job->deps, XA_FLAGS_ALLOC);
+ ret = drm_sched_job_init(&job->base, &v3d_priv->sched_entity[queue],
+ v3d_priv);
+ if (ret)
+ goto fail;
ret = drm_syncobj_find_fence(file_priv, in_sync, 0, 0, &in_fence);
if (ret == -EINVAL)
- goto fail;
+ goto fail_job;
ret = drm_gem_fence_array_add(&job->deps, in_fence);
if (ret)
- goto fail;
+ goto fail_job;
kref_init(&job->refcount);
return 0;
+fail_job:
+ drm_sched_job_cleanup(&job->base);
fail:
xa_destroy(&job->deps);
pm_runtime_put_autosuspend(v3d->drm.dev);
return ret;
}
-static int
-v3d_push_job(struct v3d_file_priv *v3d_priv,
- struct v3d_job *job, enum v3d_queue queue)
+static void
+v3d_push_job(struct v3d_job *job)
{
- int ret;
-
- ret = drm_sched_job_init(&job->base, &v3d_priv->sched_entity[queue],
- v3d_priv);
- if (ret)
- return ret;
-
drm_sched_job_arm(&job->base);
job->done_fence = dma_fence_get(&job->base.s_fence->finished);
@@ -488,8 +493,6 @@ v3d_push_job(struct v3d_file_priv *v3d_priv,
kref_get(&job->refcount);
drm_sched_entity_push_job(&job->base);
-
- return 0;
}
static void
@@ -564,7 +567,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
INIT_LIST_HEAD(&render->unref_list);
ret = v3d_job_init(v3d, file_priv, &render->base,
- v3d_render_job_free, args->in_sync_rcl);
+ v3d_render_job_free, args->in_sync_rcl, V3D_RENDER);
if (ret) {
kfree(render);
return ret;
@@ -578,7 +581,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
}
ret = v3d_job_init(v3d, file_priv, &bin->base,
- v3d_job_free, args->in_sync_bcl);
+ v3d_job_free, args->in_sync_bcl, V3D_BIN);
if (ret) {
v3d_job_put(&render->base);
kfree(bin);
@@ -600,7 +603,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
goto fail;
}
- ret = v3d_job_init(v3d, file_priv, clean_job, v3d_job_free, 0);
+ ret = v3d_job_init(v3d, file_priv, clean_job, v3d_job_free, 0, V3D_CACHE_CLEAN);
if (ret) {
kfree(clean_job);
clean_job = NULL;
@@ -635,9 +638,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
if (bin) {
bin->base.perfmon = render->base.perfmon;
v3d_perfmon_get(bin->base.perfmon);
- ret = v3d_push_job(v3d_priv, &bin->base, V3D_BIN);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(&bin->base);
ret = drm_gem_fence_array_add(&render->base.deps,
dma_fence_get(bin->base.done_fence));
@@ -645,9 +646,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
goto fail_unreserve;
}
- ret = v3d_push_job(v3d_priv, &render->base, V3D_RENDER);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(&render->base);
if (clean_job) {
struct dma_fence *render_fence =
@@ -657,9 +656,7 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
goto fail_unreserve;
clean_job->perfmon = render->base.perfmon;
v3d_perfmon_get(clean_job->perfmon);
- ret = v3d_push_job(v3d_priv, clean_job, V3D_CACHE_CLEAN);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(clean_job);
}
mutex_unlock(&v3d->sched_lock);
@@ -684,10 +681,10 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
last_job->bo_count, &acquire_ctx);
fail:
if (bin)
- v3d_job_put(&bin->base);
- v3d_job_put(&render->base);
+ v3d_job_cleanup(&bin->base);
+ v3d_job_cleanup(&render->base);
if (clean_job)
- v3d_job_put(clean_job);
+ v3d_job_cleanup(clean_job);
return ret;
}
@@ -706,7 +703,6 @@ v3d_submit_tfu_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv)
{
struct v3d_dev *v3d = to_v3d_dev(dev);
- struct v3d_file_priv *v3d_priv = file_priv->driver_priv;
struct drm_v3d_submit_tfu *args = data;
struct v3d_tfu_job *job;
struct ww_acquire_ctx acquire_ctx;
@@ -719,7 +715,7 @@ v3d_submit_tfu_ioctl(struct drm_device *dev, void *data,
return -ENOMEM;
ret = v3d_job_init(v3d, file_priv, &job->base,
- v3d_job_free, args->in_sync);
+ v3d_job_free, args->in_sync, V3D_TFU);
if (ret) {
kfree(job);
return ret;
@@ -763,9 +759,7 @@ v3d_submit_tfu_ioctl(struct drm_device *dev, void *data,
goto fail;
mutex_lock(&v3d->sched_lock);
- ret = v3d_push_job(v3d_priv, &job->base, V3D_TFU);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(&job->base);
mutex_unlock(&v3d->sched_lock);
v3d_attach_fences_and_unlock_reservation(file_priv,
@@ -777,12 +771,8 @@ v3d_submit_tfu_ioctl(struct drm_device *dev, void *data,
return 0;
-fail_unreserve:
- mutex_unlock(&v3d->sched_lock);
- drm_gem_unlock_reservations(job->base.bo, job->base.bo_count,
- &acquire_ctx);
fail:
- v3d_job_put(&job->base);
+ v3d_job_cleanup(&job->base);
return ret;
}
@@ -820,7 +810,7 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
return -ENOMEM;
ret = v3d_job_init(v3d, file_priv, &job->base,
- v3d_job_free, args->in_sync);
+ v3d_job_free, args->in_sync, V3D_CSD);
if (ret) {
kfree(job);
return ret;
@@ -833,7 +823,7 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
return -ENOMEM;
}
- ret = v3d_job_init(v3d, file_priv, clean_job, v3d_job_free, 0);
+ ret = v3d_job_init(v3d, file_priv, clean_job, v3d_job_free, 0, V3D_CACHE_CLEAN);
if (ret) {
v3d_job_put(&job->base);
kfree(clean_job);
@@ -861,18 +851,14 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
}
mutex_lock(&v3d->sched_lock);
- ret = v3d_push_job(v3d_priv, &job->base, V3D_CSD);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(&job->base);
ret = drm_gem_fence_array_add(&clean_job->deps,
dma_fence_get(job->base.done_fence));
if (ret)
goto fail_unreserve;
- ret = v3d_push_job(v3d_priv, clean_job, V3D_CACHE_CLEAN);
- if (ret)
- goto fail_unreserve;
+ v3d_push_job(clean_job);
mutex_unlock(&v3d->sched_lock);
v3d_attach_fences_and_unlock_reservation(file_priv,
@@ -891,8 +877,8 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
drm_gem_unlock_reservations(clean_job->bo, clean_job->bo_count,
&acquire_ctx);
fail:
- v3d_job_put(&job->base);
- v3d_job_put(clean_job);
+ v3d_job_cleanup(&job->base);
+ v3d_job_cleanup(clean_job);
return ret;
}
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index dd7fcc36d726..7969797793d0 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -55,12 +55,11 @@ to_csd_job(struct drm_sched_job *sched_job)
}
static void
-v3d_job_free(struct drm_sched_job *sched_job)
+v3d_sched_job_free(struct drm_sched_job *sched_job)
{
struct v3d_job *job = to_v3d_job(sched_job);
- drm_sched_job_cleanup(sched_job);
- v3d_job_put(job);
+ v3d_job_cleanup(job);
}
static void
@@ -376,35 +375,35 @@ static const struct drm_sched_backend_ops v3d_bin_sched_ops = {
.dependency = v3d_job_dependency,
.run_job = v3d_bin_job_run,
.timedout_job = v3d_bin_job_timedout,
- .free_job = v3d_job_free,
+ .free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_render_sched_ops = {
.dependency = v3d_job_dependency,
.run_job = v3d_render_job_run,
.timedout_job = v3d_render_job_timedout,
- .free_job = v3d_job_free,
+ .free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_tfu_sched_ops = {
.dependency = v3d_job_dependency,
.run_job = v3d_tfu_job_run,
.timedout_job = v3d_generic_job_timedout,
- .free_job = v3d_job_free,
+ .free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_csd_sched_ops = {
.dependency = v3d_job_dependency,
.run_job = v3d_csd_job_run,
.timedout_job = v3d_csd_job_timedout,
- .free_job = v3d_job_free
+ .free_job = v3d_sched_job_free
};
static const struct drm_sched_backend_ops v3d_cache_clean_sched_ops = {
.dependency = v3d_job_dependency,
.run_job = v3d_cache_clean_job_run,
.timedout_job = v3d_generic_job_timedout,
- .free_job = v3d_job_free
+ .free_job = v3d_sched_job_free
};
int
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* [PATCH v5 10/20] drm/v3d: Use scheduler dependency handling
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Emma Anholt, Melissa Wen, Daniel Vetter
With the prep work out of the way this isn't tricky anymore.
Aside: The chaining of the various jobs is a bit awkward, with the
possibility of failure in bad places. I think with the
drm_sched_job_init/arm split and maybe preloading the
job->dependencies xarray this should be fixable.
v2: Rebase over renamed function names for adding dependencies.
Reviewed-by: Melissa Wen <mwen@igalia.com> (v1)
Acked-by: Emma Anholt <emma@anholt.net>
Cc: Melissa Wen <melissa.srw@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Emma Anholt <emma@anholt.net>
---
drivers/gpu/drm/v3d/v3d_drv.h | 5 -----
drivers/gpu/drm/v3d/v3d_gem.c | 26 +++++++++-----------------
drivers/gpu/drm/v3d/v3d_sched.c | 29 +----------------------------
3 files changed, 10 insertions(+), 50 deletions(-)
diff --git a/drivers/gpu/drm/v3d/v3d_drv.h b/drivers/gpu/drm/v3d/v3d_drv.h
index c1d433b4cf93..b900a050d5e2 100644
--- a/drivers/gpu/drm/v3d/v3d_drv.h
+++ b/drivers/gpu/drm/v3d/v3d_drv.h
@@ -234,11 +234,6 @@ struct v3d_job {
struct drm_gem_object **bo;
u32 bo_count;
- /* Array of struct dma_fence * to block on before submitting this job.
- */
- struct xarray deps;
- unsigned long last_dep;
-
/* v3d fence to be signaled by IRQ handler when the job is complete. */
struct dma_fence *irq_fence;
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 42587248c54e..a3529809d547 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -259,8 +259,8 @@ v3d_lock_bo_reservations(struct v3d_job *job,
return ret;
for (i = 0; i < job->bo_count; i++) {
- ret = drm_gem_fence_array_add_implicit(&job->deps,
- job->bo[i], true);
+ ret = drm_sched_job_add_implicit_dependencies(&job->base,
+ job->bo[i], true);
if (ret) {
drm_gem_unlock_reservations(job->bo, job->bo_count,
acquire_ctx);
@@ -356,8 +356,6 @@ static void
v3d_job_free(struct kref *ref)
{
struct v3d_job *job = container_of(ref, struct v3d_job, refcount);
- unsigned long index;
- struct dma_fence *fence;
int i;
for (i = 0; i < job->bo_count; i++) {
@@ -366,11 +364,6 @@ v3d_job_free(struct kref *ref)
}
kvfree(job->bo);
- xa_for_each(&job->deps, index, fence) {
- dma_fence_put(fence);
- }
- xa_destroy(&job->deps);
-
dma_fence_put(job->irq_fence);
dma_fence_put(job->done_fence);
@@ -457,7 +450,6 @@ v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv,
if (ret < 0)
return ret;
- xa_init_flags(&job->deps, XA_FLAGS_ALLOC);
ret = drm_sched_job_init(&job->base, &v3d_priv->sched_entity[queue],
v3d_priv);
if (ret)
@@ -467,7 +459,7 @@ v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv,
if (ret == -EINVAL)
goto fail_job;
- ret = drm_gem_fence_array_add(&job->deps, in_fence);
+ ret = drm_sched_job_add_dependency(&job->base, in_fence);
if (ret)
goto fail_job;
@@ -477,7 +469,6 @@ v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv,
fail_job:
drm_sched_job_cleanup(&job->base);
fail:
- xa_destroy(&job->deps);
pm_runtime_put_autosuspend(v3d->drm.dev);
return ret;
}
@@ -640,8 +631,8 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
v3d_perfmon_get(bin->base.perfmon);
v3d_push_job(&bin->base);
- ret = drm_gem_fence_array_add(&render->base.deps,
- dma_fence_get(bin->base.done_fence));
+ ret = drm_sched_job_add_dependency(&render->base.base,
+ dma_fence_get(bin->base.done_fence));
if (ret)
goto fail_unreserve;
}
@@ -651,7 +642,8 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
if (clean_job) {
struct dma_fence *render_fence =
dma_fence_get(render->base.done_fence);
- ret = drm_gem_fence_array_add(&clean_job->deps, render_fence);
+ ret = drm_sched_job_add_dependency(&clean_job->base,
+ render_fence);
if (ret)
goto fail_unreserve;
clean_job->perfmon = render->base.perfmon;
@@ -853,8 +845,8 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
mutex_lock(&v3d->sched_lock);
v3d_push_job(&job->base);
- ret = drm_gem_fence_array_add(&clean_job->deps,
- dma_fence_get(job->base.done_fence));
+ ret = drm_sched_job_add_dependency(&clean_job->base,
+ dma_fence_get(job->base.done_fence));
if (ret)
goto fail_unreserve;
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index 7969797793d0..e0cb7d0697a7 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -13,7 +13,7 @@
* jobs when bulk background jobs are queued up, we submit a new job
* to the HW only when it has completed the last one, instead of
* filling up the CT[01]Q FIFOs with jobs. Similarly, we use
- * v3d_job_dependency() to manage the dependency between bin and
+ * drm_sched_job_add_dependency() to manage the dependency between bin and
* render, instead of having the clients submit jobs using the HW's
* semaphores to interlock between them.
*/
@@ -72,28 +72,6 @@ v3d_switch_perfmon(struct v3d_dev *v3d, struct v3d_job *job)
v3d_perfmon_start(v3d, job->perfmon);
}
-/*
- * Returns the fences that the job depends on, one by one.
- *
- * If placed in the scheduler's .dependency method, the corresponding
- * .run_job won't be called until all of them have been signaled.
- */
-static struct dma_fence *
-v3d_job_dependency(struct drm_sched_job *sched_job,
- struct drm_sched_entity *s_entity)
-{
- struct v3d_job *job = to_v3d_job(sched_job);
-
- /* XXX: Wait on a fence for switching the GMP if necessary,
- * and then do so.
- */
-
- if (!xa_empty(&job->deps))
- return xa_erase(&job->deps, job->last_dep++);
-
- return NULL;
-}
-
static struct dma_fence *v3d_bin_job_run(struct drm_sched_job *sched_job)
{
struct v3d_bin_job *job = to_bin_job(sched_job);
@@ -372,35 +350,30 @@ v3d_csd_job_timedout(struct drm_sched_job *sched_job)
}
static const struct drm_sched_backend_ops v3d_bin_sched_ops = {
- .dependency = v3d_job_dependency,
.run_job = v3d_bin_job_run,
.timedout_job = v3d_bin_job_timedout,
.free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_render_sched_ops = {
- .dependency = v3d_job_dependency,
.run_job = v3d_render_job_run,
.timedout_job = v3d_render_job_timedout,
.free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_tfu_sched_ops = {
- .dependency = v3d_job_dependency,
.run_job = v3d_tfu_job_run,
.timedout_job = v3d_generic_job_timedout,
.free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_csd_sched_ops = {
- .dependency = v3d_job_dependency,
.run_job = v3d_csd_job_run,
.timedout_job = v3d_csd_job_timedout,
.free_job = v3d_sched_job_free
};
static const struct drm_sched_backend_ops v3d_cache_clean_sched_ops = {
- .dependency = v3d_job_dependency,
.run_job = v3d_cache_clean_job_run,
.timedout_job = v3d_generic_job_timedout,
.free_job = v3d_sched_job_free
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 10/20] drm/v3d: Use scheduler dependency handling
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Emma Anholt, Melissa Wen, Daniel Vetter
With the prep work out of the way this isn't tricky anymore.
Aside: The chaining of the various jobs is a bit awkward, with the
possibility of failure in bad places. I think with the
drm_sched_job_init/arm split and maybe preloading the
job->dependencies xarray this should be fixable.
v2: Rebase over renamed function names for adding dependencies.
Reviewed-by: Melissa Wen <mwen@igalia.com> (v1)
Acked-by: Emma Anholt <emma@anholt.net>
Cc: Melissa Wen <melissa.srw@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Emma Anholt <emma@anholt.net>
---
drivers/gpu/drm/v3d/v3d_drv.h | 5 -----
drivers/gpu/drm/v3d/v3d_gem.c | 26 +++++++++-----------------
drivers/gpu/drm/v3d/v3d_sched.c | 29 +----------------------------
3 files changed, 10 insertions(+), 50 deletions(-)
diff --git a/drivers/gpu/drm/v3d/v3d_drv.h b/drivers/gpu/drm/v3d/v3d_drv.h
index c1d433b4cf93..b900a050d5e2 100644
--- a/drivers/gpu/drm/v3d/v3d_drv.h
+++ b/drivers/gpu/drm/v3d/v3d_drv.h
@@ -234,11 +234,6 @@ struct v3d_job {
struct drm_gem_object **bo;
u32 bo_count;
- /* Array of struct dma_fence * to block on before submitting this job.
- */
- struct xarray deps;
- unsigned long last_dep;
-
/* v3d fence to be signaled by IRQ handler when the job is complete. */
struct dma_fence *irq_fence;
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index 42587248c54e..a3529809d547 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -259,8 +259,8 @@ v3d_lock_bo_reservations(struct v3d_job *job,
return ret;
for (i = 0; i < job->bo_count; i++) {
- ret = drm_gem_fence_array_add_implicit(&job->deps,
- job->bo[i], true);
+ ret = drm_sched_job_add_implicit_dependencies(&job->base,
+ job->bo[i], true);
if (ret) {
drm_gem_unlock_reservations(job->bo, job->bo_count,
acquire_ctx);
@@ -356,8 +356,6 @@ static void
v3d_job_free(struct kref *ref)
{
struct v3d_job *job = container_of(ref, struct v3d_job, refcount);
- unsigned long index;
- struct dma_fence *fence;
int i;
for (i = 0; i < job->bo_count; i++) {
@@ -366,11 +364,6 @@ v3d_job_free(struct kref *ref)
}
kvfree(job->bo);
- xa_for_each(&job->deps, index, fence) {
- dma_fence_put(fence);
- }
- xa_destroy(&job->deps);
-
dma_fence_put(job->irq_fence);
dma_fence_put(job->done_fence);
@@ -457,7 +450,6 @@ v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv,
if (ret < 0)
return ret;
- xa_init_flags(&job->deps, XA_FLAGS_ALLOC);
ret = drm_sched_job_init(&job->base, &v3d_priv->sched_entity[queue],
v3d_priv);
if (ret)
@@ -467,7 +459,7 @@ v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv,
if (ret == -EINVAL)
goto fail_job;
- ret = drm_gem_fence_array_add(&job->deps, in_fence);
+ ret = drm_sched_job_add_dependency(&job->base, in_fence);
if (ret)
goto fail_job;
@@ -477,7 +469,6 @@ v3d_job_init(struct v3d_dev *v3d, struct drm_file *file_priv,
fail_job:
drm_sched_job_cleanup(&job->base);
fail:
- xa_destroy(&job->deps);
pm_runtime_put_autosuspend(v3d->drm.dev);
return ret;
}
@@ -640,8 +631,8 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
v3d_perfmon_get(bin->base.perfmon);
v3d_push_job(&bin->base);
- ret = drm_gem_fence_array_add(&render->base.deps,
- dma_fence_get(bin->base.done_fence));
+ ret = drm_sched_job_add_dependency(&render->base.base,
+ dma_fence_get(bin->base.done_fence));
if (ret)
goto fail_unreserve;
}
@@ -651,7 +642,8 @@ v3d_submit_cl_ioctl(struct drm_device *dev, void *data,
if (clean_job) {
struct dma_fence *render_fence =
dma_fence_get(render->base.done_fence);
- ret = drm_gem_fence_array_add(&clean_job->deps, render_fence);
+ ret = drm_sched_job_add_dependency(&clean_job->base,
+ render_fence);
if (ret)
goto fail_unreserve;
clean_job->perfmon = render->base.perfmon;
@@ -853,8 +845,8 @@ v3d_submit_csd_ioctl(struct drm_device *dev, void *data,
mutex_lock(&v3d->sched_lock);
v3d_push_job(&job->base);
- ret = drm_gem_fence_array_add(&clean_job->deps,
- dma_fence_get(job->base.done_fence));
+ ret = drm_sched_job_add_dependency(&clean_job->base,
+ dma_fence_get(job->base.done_fence));
if (ret)
goto fail_unreserve;
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index 7969797793d0..e0cb7d0697a7 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -13,7 +13,7 @@
* jobs when bulk background jobs are queued up, we submit a new job
* to the HW only when it has completed the last one, instead of
* filling up the CT[01]Q FIFOs with jobs. Similarly, we use
- * v3d_job_dependency() to manage the dependency between bin and
+ * drm_sched_job_add_dependency() to manage the dependency between bin and
* render, instead of having the clients submit jobs using the HW's
* semaphores to interlock between them.
*/
@@ -72,28 +72,6 @@ v3d_switch_perfmon(struct v3d_dev *v3d, struct v3d_job *job)
v3d_perfmon_start(v3d, job->perfmon);
}
-/*
- * Returns the fences that the job depends on, one by one.
- *
- * If placed in the scheduler's .dependency method, the corresponding
- * .run_job won't be called until all of them have been signaled.
- */
-static struct dma_fence *
-v3d_job_dependency(struct drm_sched_job *sched_job,
- struct drm_sched_entity *s_entity)
-{
- struct v3d_job *job = to_v3d_job(sched_job);
-
- /* XXX: Wait on a fence for switching the GMP if necessary,
- * and then do so.
- */
-
- if (!xa_empty(&job->deps))
- return xa_erase(&job->deps, job->last_dep++);
-
- return NULL;
-}
-
static struct dma_fence *v3d_bin_job_run(struct drm_sched_job *sched_job)
{
struct v3d_bin_job *job = to_bin_job(sched_job);
@@ -372,35 +350,30 @@ v3d_csd_job_timedout(struct drm_sched_job *sched_job)
}
static const struct drm_sched_backend_ops v3d_bin_sched_ops = {
- .dependency = v3d_job_dependency,
.run_job = v3d_bin_job_run,
.timedout_job = v3d_bin_job_timedout,
.free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_render_sched_ops = {
- .dependency = v3d_job_dependency,
.run_job = v3d_render_job_run,
.timedout_job = v3d_render_job_timedout,
.free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_tfu_sched_ops = {
- .dependency = v3d_job_dependency,
.run_job = v3d_tfu_job_run,
.timedout_job = v3d_generic_job_timedout,
.free_job = v3d_sched_job_free,
};
static const struct drm_sched_backend_ops v3d_csd_sched_ops = {
- .dependency = v3d_job_dependency,
.run_job = v3d_csd_job_run,
.timedout_job = v3d_csd_job_timedout,
.free_job = v3d_sched_job_free
};
static const struct drm_sched_backend_ops v3d_cache_clean_sched_ops = {
- .dependency = v3d_job_dependency,
.run_job = v3d_cache_clean_job_run,
.timedout_job = v3d_generic_job_timedout,
.free_job = v3d_sched_job_free
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* [PATCH v5 11/20] drm/etnaviv: Use scheduler dependency handling
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Lucas Stach, Russell King, Christian Gmeiner, Sumit Semwal,
Christian König, etnaviv, linux-media, linaro-mm-sig
We need to pull the drm_sched_job_init much earlier, but that's very
minor surgery.
v2: Actually fix up cleanup paths by calling drm_sched_job_init, which
I wanted to to in the previous round (and did, for all other drivers).
Spotted by Lucas.
v3: Rebase over renamed functions to add dependencies.
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: etnaviv@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/gpu/drm/etnaviv/etnaviv_gem.h | 5 +-
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 60 ++++++++++---------
drivers/gpu/drm/etnaviv/etnaviv_sched.c | 63 +-------------------
drivers/gpu/drm/etnaviv/etnaviv_sched.h | 3 +-
4 files changed, 37 insertions(+), 94 deletions(-)
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.h b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
index 98e60df882b6..63688e6e4580 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
@@ -80,9 +80,6 @@ struct etnaviv_gem_submit_bo {
u64 va;
struct etnaviv_gem_object *obj;
struct etnaviv_vram_mapping *mapping;
- struct dma_fence *excl;
- unsigned int nr_shared;
- struct dma_fence **shared;
};
/* Created per submit-ioctl, to track bo's and cmdstream bufs, etc,
@@ -95,7 +92,7 @@ struct etnaviv_gem_submit {
struct etnaviv_file_private *ctx;
struct etnaviv_gpu *gpu;
struct etnaviv_iommu_context *mmu_context, *prev_mmu_context;
- struct dma_fence *out_fence, *in_fence;
+ struct dma_fence *out_fence;
int out_fence_id;
struct list_head node; /* GPU active submit list */
struct etnaviv_cmdbuf cmdbuf;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
index 4dd7d9d541c0..e3d43678eb09 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
@@ -188,16 +188,11 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)
if (submit->flags & ETNA_SUBMIT_NO_IMPLICIT)
continue;
- if (bo->flags & ETNA_SUBMIT_BO_WRITE) {
- ret = dma_resv_get_fences(robj, &bo->excl,
- &bo->nr_shared,
- &bo->shared);
- if (ret)
- return ret;
- } else {
- bo->excl = dma_resv_get_excl_unlocked(robj);
- }
-
+ ret = drm_sched_job_add_implicit_dependencies(&submit->sched_job,
+ &bo->obj->base,
+ bo->flags & ETNA_SUBMIT_BO_WRITE);
+ if (ret)
+ return ret;
}
return ret;
@@ -403,8 +398,6 @@ static void submit_cleanup(struct kref *kref)
wake_up_all(&submit->gpu->fence_event);
- if (submit->in_fence)
- dma_fence_put(submit->in_fence);
if (submit->out_fence) {
/* first remove from IDR, so fence can not be found anymore */
mutex_lock(&submit->gpu->fence_lock);
@@ -529,7 +522,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
ret = etnaviv_cmdbuf_init(priv->cmdbuf_suballoc, &submit->cmdbuf,
ALIGN(args->stream_size, 8) + 8);
if (ret)
- goto err_submit_objects;
+ goto err_submit_put;
submit->ctx = file->driver_priv;
etnaviv_iommu_context_get(submit->ctx->mmu);
@@ -537,51 +530,62 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->exec_state = args->exec_state;
submit->flags = args->flags;
+ ret = drm_sched_job_init(&submit->sched_job,
+ &ctx->sched_entity[args->pipe],
+ submit->ctx);
+ if (ret)
+ goto err_submit_put;
+
ret = submit_lookup_objects(submit, file, bos, args->nr_bos);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
if ((priv->mmu_global->version != ETNAVIV_IOMMU_V2) &&
!etnaviv_cmd_validate_one(gpu, stream, args->stream_size / 4,
relocs, args->nr_relocs)) {
ret = -EINVAL;
- goto err_submit_objects;
+ goto err_submit_job;
}
if (args->flags & ETNA_SUBMIT_FENCE_FD_IN) {
- submit->in_fence = sync_file_get_fence(args->fence_fd);
- if (!submit->in_fence) {
+ struct dma_fence *in_fence = sync_file_get_fence(args->fence_fd);
+ if (!in_fence) {
ret = -EINVAL;
- goto err_submit_objects;
+ goto err_submit_job;
}
+
+ ret = drm_sched_job_add_dependency(&submit->sched_job,
+ in_fence);
+ if (ret)
+ goto err_submit_job;
}
ret = submit_pin_objects(submit);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
ret = submit_reloc(submit, stream, args->stream_size / 4,
relocs, args->nr_relocs);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
ret = submit_perfmon_validate(submit, args->exec_state, pmrs);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
memcpy(submit->cmdbuf.vaddr, stream, args->stream_size);
ret = submit_lock_objects(submit, &ticket);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
ret = submit_fence_sync(submit);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
- ret = etnaviv_sched_push_job(&ctx->sched_entity[args->pipe], submit);
+ ret = etnaviv_sched_push_job(submit);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
submit_attach_object_fences(submit);
@@ -595,7 +599,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
sync_file = sync_file_create(submit->out_fence);
if (!sync_file) {
ret = -ENOMEM;
- goto err_submit_objects;
+ goto err_submit_job;
}
fd_install(out_fence_fd, sync_file->file);
}
@@ -603,7 +607,9 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
args->fence_fd = out_fence_fd;
args->fence = submit->out_fence_id;
-err_submit_objects:
+err_submit_job:
+ drm_sched_job_cleanup(&submit->sched_job);
+err_submit_put:
etnaviv_submit_put(submit);
err_submit_ww_acquire:
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index 180bb633d5c5..2bbbd6ccc95e 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -17,58 +17,6 @@ module_param_named(job_hang_limit, etnaviv_job_hang_limit, int , 0444);
static int etnaviv_hw_jobs_limit = 4;
module_param_named(hw_job_limit, etnaviv_hw_jobs_limit, int , 0444);
-static struct dma_fence *
-etnaviv_sched_dependency(struct drm_sched_job *sched_job,
- struct drm_sched_entity *entity)
-{
- struct etnaviv_gem_submit *submit = to_etnaviv_submit(sched_job);
- struct dma_fence *fence;
- int i;
-
- if (unlikely(submit->in_fence)) {
- fence = submit->in_fence;
- submit->in_fence = NULL;
-
- if (!dma_fence_is_signaled(fence))
- return fence;
-
- dma_fence_put(fence);
- }
-
- for (i = 0; i < submit->nr_bos; i++) {
- struct etnaviv_gem_submit_bo *bo = &submit->bos[i];
- int j;
-
- if (bo->excl) {
- fence = bo->excl;
- bo->excl = NULL;
-
- if (!dma_fence_is_signaled(fence))
- return fence;
-
- dma_fence_put(fence);
- }
-
- for (j = 0; j < bo->nr_shared; j++) {
- if (!bo->shared[j])
- continue;
-
- fence = bo->shared[j];
- bo->shared[j] = NULL;
-
- if (!dma_fence_is_signaled(fence))
- return fence;
-
- dma_fence_put(fence);
- }
- kfree(bo->shared);
- bo->nr_shared = 0;
- bo->shared = NULL;
- }
-
- return NULL;
-}
-
static struct dma_fence *etnaviv_sched_run_job(struct drm_sched_job *sched_job)
{
struct etnaviv_gem_submit *submit = to_etnaviv_submit(sched_job);
@@ -140,29 +88,22 @@ static void etnaviv_sched_free_job(struct drm_sched_job *sched_job)
}
static const struct drm_sched_backend_ops etnaviv_sched_ops = {
- .dependency = etnaviv_sched_dependency,
.run_job = etnaviv_sched_run_job,
.timedout_job = etnaviv_sched_timedout_job,
.free_job = etnaviv_sched_free_job,
};
-int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
- struct etnaviv_gem_submit *submit)
+int etnaviv_sched_push_job(struct etnaviv_gem_submit *submit)
{
int ret = 0;
/*
* Hold the fence lock across the whole operation to avoid jobs being
* pushed out of order with regard to their sched fence seqnos as
- * allocated in drm_sched_job_init.
+ * allocated in drm_sched_job_arm.
*/
mutex_lock(&submit->gpu->fence_lock);
- ret = drm_sched_job_init(&submit->sched_job, sched_entity,
- submit->ctx);
- if (ret)
- goto out_unlock;
-
drm_sched_job_arm(&submit->sched_job);
submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.h b/drivers/gpu/drm/etnaviv/etnaviv_sched.h
index c0a6796e22c9..baebfa069afc 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.h
@@ -18,7 +18,6 @@ struct etnaviv_gem_submit *to_etnaviv_submit(struct drm_sched_job *sched_job)
int etnaviv_sched_init(struct etnaviv_gpu *gpu);
void etnaviv_sched_fini(struct etnaviv_gpu *gpu);
-int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
- struct etnaviv_gem_submit *submit);
+int etnaviv_sched_push_job(struct etnaviv_gem_submit *submit);
#endif /* __ETNAVIV_SCHED_H__ */
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 11/20] drm/etnaviv: Use scheduler dependency handling
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Lucas Stach, Russell King, Christian Gmeiner, Sumit Semwal,
Christian König, etnaviv, linux-media, linaro-mm-sig
We need to pull the drm_sched_job_init much earlier, but that's very
minor surgery.
v2: Actually fix up cleanup paths by calling drm_sched_job_init, which
I wanted to to in the previous round (and did, for all other drivers).
Spotted by Lucas.
v3: Rebase over renamed functions to add dependencies.
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: etnaviv@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/gpu/drm/etnaviv/etnaviv_gem.h | 5 +-
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 60 ++++++++++---------
drivers/gpu/drm/etnaviv/etnaviv_sched.c | 63 +-------------------
drivers/gpu/drm/etnaviv/etnaviv_sched.h | 3 +-
4 files changed, 37 insertions(+), 94 deletions(-)
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.h b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
index 98e60df882b6..63688e6e4580 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
@@ -80,9 +80,6 @@ struct etnaviv_gem_submit_bo {
u64 va;
struct etnaviv_gem_object *obj;
struct etnaviv_vram_mapping *mapping;
- struct dma_fence *excl;
- unsigned int nr_shared;
- struct dma_fence **shared;
};
/* Created per submit-ioctl, to track bo's and cmdstream bufs, etc,
@@ -95,7 +92,7 @@ struct etnaviv_gem_submit {
struct etnaviv_file_private *ctx;
struct etnaviv_gpu *gpu;
struct etnaviv_iommu_context *mmu_context, *prev_mmu_context;
- struct dma_fence *out_fence, *in_fence;
+ struct dma_fence *out_fence;
int out_fence_id;
struct list_head node; /* GPU active submit list */
struct etnaviv_cmdbuf cmdbuf;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
index 4dd7d9d541c0..e3d43678eb09 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
@@ -188,16 +188,11 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)
if (submit->flags & ETNA_SUBMIT_NO_IMPLICIT)
continue;
- if (bo->flags & ETNA_SUBMIT_BO_WRITE) {
- ret = dma_resv_get_fences(robj, &bo->excl,
- &bo->nr_shared,
- &bo->shared);
- if (ret)
- return ret;
- } else {
- bo->excl = dma_resv_get_excl_unlocked(robj);
- }
-
+ ret = drm_sched_job_add_implicit_dependencies(&submit->sched_job,
+ &bo->obj->base,
+ bo->flags & ETNA_SUBMIT_BO_WRITE);
+ if (ret)
+ return ret;
}
return ret;
@@ -403,8 +398,6 @@ static void submit_cleanup(struct kref *kref)
wake_up_all(&submit->gpu->fence_event);
- if (submit->in_fence)
- dma_fence_put(submit->in_fence);
if (submit->out_fence) {
/* first remove from IDR, so fence can not be found anymore */
mutex_lock(&submit->gpu->fence_lock);
@@ -529,7 +522,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
ret = etnaviv_cmdbuf_init(priv->cmdbuf_suballoc, &submit->cmdbuf,
ALIGN(args->stream_size, 8) + 8);
if (ret)
- goto err_submit_objects;
+ goto err_submit_put;
submit->ctx = file->driver_priv;
etnaviv_iommu_context_get(submit->ctx->mmu);
@@ -537,51 +530,62 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->exec_state = args->exec_state;
submit->flags = args->flags;
+ ret = drm_sched_job_init(&submit->sched_job,
+ &ctx->sched_entity[args->pipe],
+ submit->ctx);
+ if (ret)
+ goto err_submit_put;
+
ret = submit_lookup_objects(submit, file, bos, args->nr_bos);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
if ((priv->mmu_global->version != ETNAVIV_IOMMU_V2) &&
!etnaviv_cmd_validate_one(gpu, stream, args->stream_size / 4,
relocs, args->nr_relocs)) {
ret = -EINVAL;
- goto err_submit_objects;
+ goto err_submit_job;
}
if (args->flags & ETNA_SUBMIT_FENCE_FD_IN) {
- submit->in_fence = sync_file_get_fence(args->fence_fd);
- if (!submit->in_fence) {
+ struct dma_fence *in_fence = sync_file_get_fence(args->fence_fd);
+ if (!in_fence) {
ret = -EINVAL;
- goto err_submit_objects;
+ goto err_submit_job;
}
+
+ ret = drm_sched_job_add_dependency(&submit->sched_job,
+ in_fence);
+ if (ret)
+ goto err_submit_job;
}
ret = submit_pin_objects(submit);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
ret = submit_reloc(submit, stream, args->stream_size / 4,
relocs, args->nr_relocs);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
ret = submit_perfmon_validate(submit, args->exec_state, pmrs);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
memcpy(submit->cmdbuf.vaddr, stream, args->stream_size);
ret = submit_lock_objects(submit, &ticket);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
ret = submit_fence_sync(submit);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
- ret = etnaviv_sched_push_job(&ctx->sched_entity[args->pipe], submit);
+ ret = etnaviv_sched_push_job(submit);
if (ret)
- goto err_submit_objects;
+ goto err_submit_job;
submit_attach_object_fences(submit);
@@ -595,7 +599,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
sync_file = sync_file_create(submit->out_fence);
if (!sync_file) {
ret = -ENOMEM;
- goto err_submit_objects;
+ goto err_submit_job;
}
fd_install(out_fence_fd, sync_file->file);
}
@@ -603,7 +607,9 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
args->fence_fd = out_fence_fd;
args->fence = submit->out_fence_id;
-err_submit_objects:
+err_submit_job:
+ drm_sched_job_cleanup(&submit->sched_job);
+err_submit_put:
etnaviv_submit_put(submit);
err_submit_ww_acquire:
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index 180bb633d5c5..2bbbd6ccc95e 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -17,58 +17,6 @@ module_param_named(job_hang_limit, etnaviv_job_hang_limit, int , 0444);
static int etnaviv_hw_jobs_limit = 4;
module_param_named(hw_job_limit, etnaviv_hw_jobs_limit, int , 0444);
-static struct dma_fence *
-etnaviv_sched_dependency(struct drm_sched_job *sched_job,
- struct drm_sched_entity *entity)
-{
- struct etnaviv_gem_submit *submit = to_etnaviv_submit(sched_job);
- struct dma_fence *fence;
- int i;
-
- if (unlikely(submit->in_fence)) {
- fence = submit->in_fence;
- submit->in_fence = NULL;
-
- if (!dma_fence_is_signaled(fence))
- return fence;
-
- dma_fence_put(fence);
- }
-
- for (i = 0; i < submit->nr_bos; i++) {
- struct etnaviv_gem_submit_bo *bo = &submit->bos[i];
- int j;
-
- if (bo->excl) {
- fence = bo->excl;
- bo->excl = NULL;
-
- if (!dma_fence_is_signaled(fence))
- return fence;
-
- dma_fence_put(fence);
- }
-
- for (j = 0; j < bo->nr_shared; j++) {
- if (!bo->shared[j])
- continue;
-
- fence = bo->shared[j];
- bo->shared[j] = NULL;
-
- if (!dma_fence_is_signaled(fence))
- return fence;
-
- dma_fence_put(fence);
- }
- kfree(bo->shared);
- bo->nr_shared = 0;
- bo->shared = NULL;
- }
-
- return NULL;
-}
-
static struct dma_fence *etnaviv_sched_run_job(struct drm_sched_job *sched_job)
{
struct etnaviv_gem_submit *submit = to_etnaviv_submit(sched_job);
@@ -140,29 +88,22 @@ static void etnaviv_sched_free_job(struct drm_sched_job *sched_job)
}
static const struct drm_sched_backend_ops etnaviv_sched_ops = {
- .dependency = etnaviv_sched_dependency,
.run_job = etnaviv_sched_run_job,
.timedout_job = etnaviv_sched_timedout_job,
.free_job = etnaviv_sched_free_job,
};
-int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
- struct etnaviv_gem_submit *submit)
+int etnaviv_sched_push_job(struct etnaviv_gem_submit *submit)
{
int ret = 0;
/*
* Hold the fence lock across the whole operation to avoid jobs being
* pushed out of order with regard to their sched fence seqnos as
- * allocated in drm_sched_job_init.
+ * allocated in drm_sched_job_arm.
*/
mutex_lock(&submit->gpu->fence_lock);
- ret = drm_sched_job_init(&submit->sched_job, sched_entity,
- submit->ctx);
- if (ret)
- goto out_unlock;
-
drm_sched_job_arm(&submit->sched_job);
submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.h b/drivers/gpu/drm/etnaviv/etnaviv_sched.h
index c0a6796e22c9..baebfa069afc 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.h
@@ -18,7 +18,6 @@ struct etnaviv_gem_submit *to_etnaviv_submit(struct drm_sched_job *sched_job)
int etnaviv_sched_init(struct etnaviv_gpu *gpu);
void etnaviv_sched_fini(struct etnaviv_gpu *gpu);
-int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
- struct etnaviv_gem_submit *submit);
+int etnaviv_sched_push_job(struct etnaviv_gem_submit *submit);
#endif /* __ETNAVIV_SCHED_H__ */
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 11/20] drm/etnaviv: Use scheduler dependency handling
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-12 19:28 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-12 19:28 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Lucas Stach, Russell King, Christian Gmeiner, Sumit Semwal,
Christian König, etnaviv, linux-media, linaro-mm-sig
On Thu, Aug 05, 2021 at 12:46:56PM +0200, Daniel Vetter wrote:
> We need to pull the drm_sched_job_init much earlier, but that's very
> minor surgery.
>
> v2: Actually fix up cleanup paths by calling drm_sched_job_init, which
> I wanted to to in the previous round (and did, for all other drivers).
> Spotted by Lucas.
>
> v3: Rebase over renamed functions to add dependencies.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: etnaviv@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Ping for an ack&testing please.
-Daniel
> ---
> drivers/gpu/drm/etnaviv/etnaviv_gem.h | 5 +-
> drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 60 ++++++++++---------
> drivers/gpu/drm/etnaviv/etnaviv_sched.c | 63 +-------------------
> drivers/gpu/drm/etnaviv/etnaviv_sched.h | 3 +-
> 4 files changed, 37 insertions(+), 94 deletions(-)
>
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.h b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
> index 98e60df882b6..63688e6e4580 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.h
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
> @@ -80,9 +80,6 @@ struct etnaviv_gem_submit_bo {
> u64 va;
> struct etnaviv_gem_object *obj;
> struct etnaviv_vram_mapping *mapping;
> - struct dma_fence *excl;
> - unsigned int nr_shared;
> - struct dma_fence **shared;
> };
>
> /* Created per submit-ioctl, to track bo's and cmdstream bufs, etc,
> @@ -95,7 +92,7 @@ struct etnaviv_gem_submit {
> struct etnaviv_file_private *ctx;
> struct etnaviv_gpu *gpu;
> struct etnaviv_iommu_context *mmu_context, *prev_mmu_context;
> - struct dma_fence *out_fence, *in_fence;
> + struct dma_fence *out_fence;
> int out_fence_id;
> struct list_head node; /* GPU active submit list */
> struct etnaviv_cmdbuf cmdbuf;
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
> index 4dd7d9d541c0..e3d43678eb09 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
> @@ -188,16 +188,11 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)
> if (submit->flags & ETNA_SUBMIT_NO_IMPLICIT)
> continue;
>
> - if (bo->flags & ETNA_SUBMIT_BO_WRITE) {
> - ret = dma_resv_get_fences(robj, &bo->excl,
> - &bo->nr_shared,
> - &bo->shared);
> - if (ret)
> - return ret;
> - } else {
> - bo->excl = dma_resv_get_excl_unlocked(robj);
> - }
> -
> + ret = drm_sched_job_add_implicit_dependencies(&submit->sched_job,
> + &bo->obj->base,
> + bo->flags & ETNA_SUBMIT_BO_WRITE);
> + if (ret)
> + return ret;
> }
>
> return ret;
> @@ -403,8 +398,6 @@ static void submit_cleanup(struct kref *kref)
>
> wake_up_all(&submit->gpu->fence_event);
>
> - if (submit->in_fence)
> - dma_fence_put(submit->in_fence);
> if (submit->out_fence) {
> /* first remove from IDR, so fence can not be found anymore */
> mutex_lock(&submit->gpu->fence_lock);
> @@ -529,7 +522,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
> ret = etnaviv_cmdbuf_init(priv->cmdbuf_suballoc, &submit->cmdbuf,
> ALIGN(args->stream_size, 8) + 8);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_put;
>
> submit->ctx = file->driver_priv;
> etnaviv_iommu_context_get(submit->ctx->mmu);
> @@ -537,51 +530,62 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
> submit->exec_state = args->exec_state;
> submit->flags = args->flags;
>
> + ret = drm_sched_job_init(&submit->sched_job,
> + &ctx->sched_entity[args->pipe],
> + submit->ctx);
> + if (ret)
> + goto err_submit_put;
> +
> ret = submit_lookup_objects(submit, file, bos, args->nr_bos);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> if ((priv->mmu_global->version != ETNAVIV_IOMMU_V2) &&
> !etnaviv_cmd_validate_one(gpu, stream, args->stream_size / 4,
> relocs, args->nr_relocs)) {
> ret = -EINVAL;
> - goto err_submit_objects;
> + goto err_submit_job;
> }
>
> if (args->flags & ETNA_SUBMIT_FENCE_FD_IN) {
> - submit->in_fence = sync_file_get_fence(args->fence_fd);
> - if (!submit->in_fence) {
> + struct dma_fence *in_fence = sync_file_get_fence(args->fence_fd);
> + if (!in_fence) {
> ret = -EINVAL;
> - goto err_submit_objects;
> + goto err_submit_job;
> }
> +
> + ret = drm_sched_job_add_dependency(&submit->sched_job,
> + in_fence);
> + if (ret)
> + goto err_submit_job;
> }
>
> ret = submit_pin_objects(submit);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> ret = submit_reloc(submit, stream, args->stream_size / 4,
> relocs, args->nr_relocs);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> ret = submit_perfmon_validate(submit, args->exec_state, pmrs);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> memcpy(submit->cmdbuf.vaddr, stream, args->stream_size);
>
> ret = submit_lock_objects(submit, &ticket);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> ret = submit_fence_sync(submit);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> - ret = etnaviv_sched_push_job(&ctx->sched_entity[args->pipe], submit);
> + ret = etnaviv_sched_push_job(submit);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> submit_attach_object_fences(submit);
>
> @@ -595,7 +599,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
> sync_file = sync_file_create(submit->out_fence);
> if (!sync_file) {
> ret = -ENOMEM;
> - goto err_submit_objects;
> + goto err_submit_job;
> }
> fd_install(out_fence_fd, sync_file->file);
> }
> @@ -603,7 +607,9 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
> args->fence_fd = out_fence_fd;
> args->fence = submit->out_fence_id;
>
> -err_submit_objects:
> +err_submit_job:
> + drm_sched_job_cleanup(&submit->sched_job);
> +err_submit_put:
> etnaviv_submit_put(submit);
>
> err_submit_ww_acquire:
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> index 180bb633d5c5..2bbbd6ccc95e 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> @@ -17,58 +17,6 @@ module_param_named(job_hang_limit, etnaviv_job_hang_limit, int , 0444);
> static int etnaviv_hw_jobs_limit = 4;
> module_param_named(hw_job_limit, etnaviv_hw_jobs_limit, int , 0444);
>
> -static struct dma_fence *
> -etnaviv_sched_dependency(struct drm_sched_job *sched_job,
> - struct drm_sched_entity *entity)
> -{
> - struct etnaviv_gem_submit *submit = to_etnaviv_submit(sched_job);
> - struct dma_fence *fence;
> - int i;
> -
> - if (unlikely(submit->in_fence)) {
> - fence = submit->in_fence;
> - submit->in_fence = NULL;
> -
> - if (!dma_fence_is_signaled(fence))
> - return fence;
> -
> - dma_fence_put(fence);
> - }
> -
> - for (i = 0; i < submit->nr_bos; i++) {
> - struct etnaviv_gem_submit_bo *bo = &submit->bos[i];
> - int j;
> -
> - if (bo->excl) {
> - fence = bo->excl;
> - bo->excl = NULL;
> -
> - if (!dma_fence_is_signaled(fence))
> - return fence;
> -
> - dma_fence_put(fence);
> - }
> -
> - for (j = 0; j < bo->nr_shared; j++) {
> - if (!bo->shared[j])
> - continue;
> -
> - fence = bo->shared[j];
> - bo->shared[j] = NULL;
> -
> - if (!dma_fence_is_signaled(fence))
> - return fence;
> -
> - dma_fence_put(fence);
> - }
> - kfree(bo->shared);
> - bo->nr_shared = 0;
> - bo->shared = NULL;
> - }
> -
> - return NULL;
> -}
> -
> static struct dma_fence *etnaviv_sched_run_job(struct drm_sched_job *sched_job)
> {
> struct etnaviv_gem_submit *submit = to_etnaviv_submit(sched_job);
> @@ -140,29 +88,22 @@ static void etnaviv_sched_free_job(struct drm_sched_job *sched_job)
> }
>
> static const struct drm_sched_backend_ops etnaviv_sched_ops = {
> - .dependency = etnaviv_sched_dependency,
> .run_job = etnaviv_sched_run_job,
> .timedout_job = etnaviv_sched_timedout_job,
> .free_job = etnaviv_sched_free_job,
> };
>
> -int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> - struct etnaviv_gem_submit *submit)
> +int etnaviv_sched_push_job(struct etnaviv_gem_submit *submit)
> {
> int ret = 0;
>
> /*
> * Hold the fence lock across the whole operation to avoid jobs being
> * pushed out of order with regard to their sched fence seqnos as
> - * allocated in drm_sched_job_init.
> + * allocated in drm_sched_job_arm.
> */
> mutex_lock(&submit->gpu->fence_lock);
>
> - ret = drm_sched_job_init(&submit->sched_job, sched_entity,
> - submit->ctx);
> - if (ret)
> - goto out_unlock;
> -
> drm_sched_job_arm(&submit->sched_job);
>
> submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.h b/drivers/gpu/drm/etnaviv/etnaviv_sched.h
> index c0a6796e22c9..baebfa069afc 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.h
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.h
> @@ -18,7 +18,6 @@ struct etnaviv_gem_submit *to_etnaviv_submit(struct drm_sched_job *sched_job)
>
> int etnaviv_sched_init(struct etnaviv_gpu *gpu);
> void etnaviv_sched_fini(struct etnaviv_gpu *gpu);
> -int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> - struct etnaviv_gem_submit *submit);
> +int etnaviv_sched_push_job(struct etnaviv_gem_submit *submit);
>
> #endif /* __ETNAVIV_SCHED_H__ */
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 11/20] drm/etnaviv: Use scheduler dependency handling
@ 2021-08-12 19:28 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-12 19:28 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Lucas Stach, Russell King, Christian Gmeiner, Sumit Semwal,
Christian König, etnaviv, linux-media, linaro-mm-sig
On Thu, Aug 05, 2021 at 12:46:56PM +0200, Daniel Vetter wrote:
> We need to pull the drm_sched_job_init much earlier, but that's very
> minor surgery.
>
> v2: Actually fix up cleanup paths by calling drm_sched_job_init, which
> I wanted to to in the previous round (and did, for all other drivers).
> Spotted by Lucas.
>
> v3: Rebase over renamed functions to add dependencies.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Cc: Russell King <linux+etnaviv@armlinux.org.uk>
> Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: etnaviv@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Ping for an ack&testing please.
-Daniel
> ---
> drivers/gpu/drm/etnaviv/etnaviv_gem.h | 5 +-
> drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 60 ++++++++++---------
> drivers/gpu/drm/etnaviv/etnaviv_sched.c | 63 +-------------------
> drivers/gpu/drm/etnaviv/etnaviv_sched.h | 3 +-
> 4 files changed, 37 insertions(+), 94 deletions(-)
>
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.h b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
> index 98e60df882b6..63688e6e4580 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.h
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
> @@ -80,9 +80,6 @@ struct etnaviv_gem_submit_bo {
> u64 va;
> struct etnaviv_gem_object *obj;
> struct etnaviv_vram_mapping *mapping;
> - struct dma_fence *excl;
> - unsigned int nr_shared;
> - struct dma_fence **shared;
> };
>
> /* Created per submit-ioctl, to track bo's and cmdstream bufs, etc,
> @@ -95,7 +92,7 @@ struct etnaviv_gem_submit {
> struct etnaviv_file_private *ctx;
> struct etnaviv_gpu *gpu;
> struct etnaviv_iommu_context *mmu_context, *prev_mmu_context;
> - struct dma_fence *out_fence, *in_fence;
> + struct dma_fence *out_fence;
> int out_fence_id;
> struct list_head node; /* GPU active submit list */
> struct etnaviv_cmdbuf cmdbuf;
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
> index 4dd7d9d541c0..e3d43678eb09 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
> @@ -188,16 +188,11 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)
> if (submit->flags & ETNA_SUBMIT_NO_IMPLICIT)
> continue;
>
> - if (bo->flags & ETNA_SUBMIT_BO_WRITE) {
> - ret = dma_resv_get_fences(robj, &bo->excl,
> - &bo->nr_shared,
> - &bo->shared);
> - if (ret)
> - return ret;
> - } else {
> - bo->excl = dma_resv_get_excl_unlocked(robj);
> - }
> -
> + ret = drm_sched_job_add_implicit_dependencies(&submit->sched_job,
> + &bo->obj->base,
> + bo->flags & ETNA_SUBMIT_BO_WRITE);
> + if (ret)
> + return ret;
> }
>
> return ret;
> @@ -403,8 +398,6 @@ static void submit_cleanup(struct kref *kref)
>
> wake_up_all(&submit->gpu->fence_event);
>
> - if (submit->in_fence)
> - dma_fence_put(submit->in_fence);
> if (submit->out_fence) {
> /* first remove from IDR, so fence can not be found anymore */
> mutex_lock(&submit->gpu->fence_lock);
> @@ -529,7 +522,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
> ret = etnaviv_cmdbuf_init(priv->cmdbuf_suballoc, &submit->cmdbuf,
> ALIGN(args->stream_size, 8) + 8);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_put;
>
> submit->ctx = file->driver_priv;
> etnaviv_iommu_context_get(submit->ctx->mmu);
> @@ -537,51 +530,62 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
> submit->exec_state = args->exec_state;
> submit->flags = args->flags;
>
> + ret = drm_sched_job_init(&submit->sched_job,
> + &ctx->sched_entity[args->pipe],
> + submit->ctx);
> + if (ret)
> + goto err_submit_put;
> +
> ret = submit_lookup_objects(submit, file, bos, args->nr_bos);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> if ((priv->mmu_global->version != ETNAVIV_IOMMU_V2) &&
> !etnaviv_cmd_validate_one(gpu, stream, args->stream_size / 4,
> relocs, args->nr_relocs)) {
> ret = -EINVAL;
> - goto err_submit_objects;
> + goto err_submit_job;
> }
>
> if (args->flags & ETNA_SUBMIT_FENCE_FD_IN) {
> - submit->in_fence = sync_file_get_fence(args->fence_fd);
> - if (!submit->in_fence) {
> + struct dma_fence *in_fence = sync_file_get_fence(args->fence_fd);
> + if (!in_fence) {
> ret = -EINVAL;
> - goto err_submit_objects;
> + goto err_submit_job;
> }
> +
> + ret = drm_sched_job_add_dependency(&submit->sched_job,
> + in_fence);
> + if (ret)
> + goto err_submit_job;
> }
>
> ret = submit_pin_objects(submit);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> ret = submit_reloc(submit, stream, args->stream_size / 4,
> relocs, args->nr_relocs);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> ret = submit_perfmon_validate(submit, args->exec_state, pmrs);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> memcpy(submit->cmdbuf.vaddr, stream, args->stream_size);
>
> ret = submit_lock_objects(submit, &ticket);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> ret = submit_fence_sync(submit);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> - ret = etnaviv_sched_push_job(&ctx->sched_entity[args->pipe], submit);
> + ret = etnaviv_sched_push_job(submit);
> if (ret)
> - goto err_submit_objects;
> + goto err_submit_job;
>
> submit_attach_object_fences(submit);
>
> @@ -595,7 +599,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
> sync_file = sync_file_create(submit->out_fence);
> if (!sync_file) {
> ret = -ENOMEM;
> - goto err_submit_objects;
> + goto err_submit_job;
> }
> fd_install(out_fence_fd, sync_file->file);
> }
> @@ -603,7 +607,9 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
> args->fence_fd = out_fence_fd;
> args->fence = submit->out_fence_id;
>
> -err_submit_objects:
> +err_submit_job:
> + drm_sched_job_cleanup(&submit->sched_job);
> +err_submit_put:
> etnaviv_submit_put(submit);
>
> err_submit_ww_acquire:
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> index 180bb633d5c5..2bbbd6ccc95e 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
> @@ -17,58 +17,6 @@ module_param_named(job_hang_limit, etnaviv_job_hang_limit, int , 0444);
> static int etnaviv_hw_jobs_limit = 4;
> module_param_named(hw_job_limit, etnaviv_hw_jobs_limit, int , 0444);
>
> -static struct dma_fence *
> -etnaviv_sched_dependency(struct drm_sched_job *sched_job,
> - struct drm_sched_entity *entity)
> -{
> - struct etnaviv_gem_submit *submit = to_etnaviv_submit(sched_job);
> - struct dma_fence *fence;
> - int i;
> -
> - if (unlikely(submit->in_fence)) {
> - fence = submit->in_fence;
> - submit->in_fence = NULL;
> -
> - if (!dma_fence_is_signaled(fence))
> - return fence;
> -
> - dma_fence_put(fence);
> - }
> -
> - for (i = 0; i < submit->nr_bos; i++) {
> - struct etnaviv_gem_submit_bo *bo = &submit->bos[i];
> - int j;
> -
> - if (bo->excl) {
> - fence = bo->excl;
> - bo->excl = NULL;
> -
> - if (!dma_fence_is_signaled(fence))
> - return fence;
> -
> - dma_fence_put(fence);
> - }
> -
> - for (j = 0; j < bo->nr_shared; j++) {
> - if (!bo->shared[j])
> - continue;
> -
> - fence = bo->shared[j];
> - bo->shared[j] = NULL;
> -
> - if (!dma_fence_is_signaled(fence))
> - return fence;
> -
> - dma_fence_put(fence);
> - }
> - kfree(bo->shared);
> - bo->nr_shared = 0;
> - bo->shared = NULL;
> - }
> -
> - return NULL;
> -}
> -
> static struct dma_fence *etnaviv_sched_run_job(struct drm_sched_job *sched_job)
> {
> struct etnaviv_gem_submit *submit = to_etnaviv_submit(sched_job);
> @@ -140,29 +88,22 @@ static void etnaviv_sched_free_job(struct drm_sched_job *sched_job)
> }
>
> static const struct drm_sched_backend_ops etnaviv_sched_ops = {
> - .dependency = etnaviv_sched_dependency,
> .run_job = etnaviv_sched_run_job,
> .timedout_job = etnaviv_sched_timedout_job,
> .free_job = etnaviv_sched_free_job,
> };
>
> -int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> - struct etnaviv_gem_submit *submit)
> +int etnaviv_sched_push_job(struct etnaviv_gem_submit *submit)
> {
> int ret = 0;
>
> /*
> * Hold the fence lock across the whole operation to avoid jobs being
> * pushed out of order with regard to their sched fence seqnos as
> - * allocated in drm_sched_job_init.
> + * allocated in drm_sched_job_arm.
> */
> mutex_lock(&submit->gpu->fence_lock);
>
> - ret = drm_sched_job_init(&submit->sched_job, sched_entity,
> - submit->ctx);
> - if (ret)
> - goto out_unlock;
> -
> drm_sched_job_arm(&submit->sched_job);
>
> submit->out_fence = dma_fence_get(&submit->sched_job.s_fence->finished);
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.h b/drivers/gpu/drm/etnaviv/etnaviv_sched.h
> index c0a6796e22c9..baebfa069afc 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.h
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.h
> @@ -18,7 +18,6 @@ struct etnaviv_gem_submit *to_etnaviv_submit(struct drm_sched_job *sched_job)
>
> int etnaviv_sched_init(struct etnaviv_gpu *gpu);
> void etnaviv_sched_fini(struct etnaviv_gpu *gpu);
> -int etnaviv_sched_push_job(struct drm_sched_entity *sched_entity,
> - struct etnaviv_gem_submit *submit);
> +int etnaviv_sched_push_job(struct etnaviv_gem_submit *submit);
>
> #endif /* __ETNAVIV_SCHED_H__ */
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 12/20] drm/msm: Use scheduler dependency handling
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Rob Clark, Sean Paul, Sumit Semwal, Christian König,
linux-arm-msm, freedreno, linux-media, linaro-mm-sig
drm_sched_job_init is already at the right place, so this boils down
to deleting code.
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/gpu/drm/msm/msm_gem.h | 5 -----
drivers/gpu/drm/msm/msm_gem_submit.c | 19 +++++--------------
drivers/gpu/drm/msm/msm_ringbuffer.c | 12 ------------
3 files changed, 5 insertions(+), 31 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
index f9e3ffb2309a..8bf0ac707fd7 100644
--- a/drivers/gpu/drm/msm/msm_gem.h
+++ b/drivers/gpu/drm/msm/msm_gem.h
@@ -312,11 +312,6 @@ struct msm_gem_submit {
struct ww_acquire_ctx ticket;
uint32_t seqno; /* Sequence number of the submit on the ring */
- /* Array of struct dma_fence * to block on before submitting this job.
- */
- struct xarray deps;
- unsigned long last_dep;
-
/* Hw fence, which is created when the scheduler executes the job, and
* is signaled when the hw finishes (via seqno write from cmdstream)
*/
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 96cea0ba4cfd..fb5a2eab27a2 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
- xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
-
kref_init(&submit->ref);
submit->dev = dev;
submit->aspace = queue->ctx->aspace;
@@ -72,8 +70,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
{
struct msm_gem_submit *submit =
container_of(kref, struct msm_gem_submit, ref);
- unsigned long index;
- struct dma_fence *fence;
unsigned i;
if (submit->fence_id) {
@@ -82,12 +78,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
mutex_unlock(&submit->queue->lock);
}
- xa_for_each (&submit->deps, index, fence) {
- dma_fence_put(fence);
- }
-
- xa_destroy(&submit->deps);
-
dma_fence_put(submit->user_fence);
dma_fence_put(submit->hw_fence);
@@ -343,8 +333,9 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
if (no_implicit)
continue;
- ret = drm_gem_fence_array_add_implicit(&submit->deps, obj,
- write);
+ ret = drm_sched_job_add_implicit_dependencies(&submit->base,
+ obj,
+ write);
if (ret)
break;
}
@@ -588,7 +579,7 @@ static struct drm_syncobj **msm_parse_deps(struct msm_gem_submit *submit,
if (ret)
break;
- ret = drm_gem_fence_array_add(&submit->deps, fence);
+ ret = drm_sched_job_add_dependency(&submit->base, fence);
if (ret)
break;
@@ -798,7 +789,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
goto out_unlock;
}
- ret = drm_gem_fence_array_add(&submit->deps, in_fence);
+ ret = drm_sched_job_add_dependency(&submit->base, in_fence);
if (ret)
goto out_unlock;
}
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
index bd54c1412649..652b1dedd7c1 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.c
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
@@ -11,17 +11,6 @@ static uint num_hw_submissions = 8;
MODULE_PARM_DESC(num_hw_submissions, "The max # of jobs to write into ringbuffer (default 8)");
module_param(num_hw_submissions, uint, 0600);
-static struct dma_fence *msm_job_dependency(struct drm_sched_job *job,
- struct drm_sched_entity *s_entity)
-{
- struct msm_gem_submit *submit = to_msm_submit(job);
-
- if (!xa_empty(&submit->deps))
- return xa_erase(&submit->deps, submit->last_dep++);
-
- return NULL;
-}
-
static struct dma_fence *msm_job_run(struct drm_sched_job *job)
{
struct msm_gem_submit *submit = to_msm_submit(job);
@@ -52,7 +41,6 @@ static void msm_job_free(struct drm_sched_job *job)
}
const struct drm_sched_backend_ops msm_sched_ops = {
- .dependency = msm_job_dependency,
.run_job = msm_job_run,
.free_job = msm_job_free
};
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 12/20] drm/msm: Use scheduler dependency handling
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Rob Clark, Sean Paul, Sumit Semwal, Christian König,
linux-arm-msm, freedreno, linux-media, linaro-mm-sig
drm_sched_job_init is already at the right place, so this boils down
to deleting code.
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/gpu/drm/msm/msm_gem.h | 5 -----
drivers/gpu/drm/msm/msm_gem_submit.c | 19 +++++--------------
drivers/gpu/drm/msm/msm_ringbuffer.c | 12 ------------
3 files changed, 5 insertions(+), 31 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
index f9e3ffb2309a..8bf0ac707fd7 100644
--- a/drivers/gpu/drm/msm/msm_gem.h
+++ b/drivers/gpu/drm/msm/msm_gem.h
@@ -312,11 +312,6 @@ struct msm_gem_submit {
struct ww_acquire_ctx ticket;
uint32_t seqno; /* Sequence number of the submit on the ring */
- /* Array of struct dma_fence * to block on before submitting this job.
- */
- struct xarray deps;
- unsigned long last_dep;
-
/* Hw fence, which is created when the scheduler executes the job, and
* is signaled when the hw finishes (via seqno write from cmdstream)
*/
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 96cea0ba4cfd..fb5a2eab27a2 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
return ERR_PTR(ret);
}
- xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
-
kref_init(&submit->ref);
submit->dev = dev;
submit->aspace = queue->ctx->aspace;
@@ -72,8 +70,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
{
struct msm_gem_submit *submit =
container_of(kref, struct msm_gem_submit, ref);
- unsigned long index;
- struct dma_fence *fence;
unsigned i;
if (submit->fence_id) {
@@ -82,12 +78,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
mutex_unlock(&submit->queue->lock);
}
- xa_for_each (&submit->deps, index, fence) {
- dma_fence_put(fence);
- }
-
- xa_destroy(&submit->deps);
-
dma_fence_put(submit->user_fence);
dma_fence_put(submit->hw_fence);
@@ -343,8 +333,9 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
if (no_implicit)
continue;
- ret = drm_gem_fence_array_add_implicit(&submit->deps, obj,
- write);
+ ret = drm_sched_job_add_implicit_dependencies(&submit->base,
+ obj,
+ write);
if (ret)
break;
}
@@ -588,7 +579,7 @@ static struct drm_syncobj **msm_parse_deps(struct msm_gem_submit *submit,
if (ret)
break;
- ret = drm_gem_fence_array_add(&submit->deps, fence);
+ ret = drm_sched_job_add_dependency(&submit->base, fence);
if (ret)
break;
@@ -798,7 +789,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
goto out_unlock;
}
- ret = drm_gem_fence_array_add(&submit->deps, in_fence);
+ ret = drm_sched_job_add_dependency(&submit->base, in_fence);
if (ret)
goto out_unlock;
}
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
index bd54c1412649..652b1dedd7c1 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.c
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
@@ -11,17 +11,6 @@ static uint num_hw_submissions = 8;
MODULE_PARM_DESC(num_hw_submissions, "The max # of jobs to write into ringbuffer (default 8)");
module_param(num_hw_submissions, uint, 0600);
-static struct dma_fence *msm_job_dependency(struct drm_sched_job *job,
- struct drm_sched_entity *s_entity)
-{
- struct msm_gem_submit *submit = to_msm_submit(job);
-
- if (!xa_empty(&submit->deps))
- return xa_erase(&submit->deps, submit->last_dep++);
-
- return NULL;
-}
-
static struct dma_fence *msm_job_run(struct drm_sched_job *job)
{
struct msm_gem_submit *submit = to_msm_submit(job);
@@ -52,7 +41,6 @@ static void msm_job_free(struct drm_sched_job *job)
}
const struct drm_sched_backend_ops msm_sched_ops = {
- .dependency = msm_job_dependency,
.run_job = msm_job_run,
.free_job = msm_job_free
};
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 12/20] drm/msm: Use scheduler dependency handling
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-12 19:29 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-12 19:29 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Rob Clark, Sean Paul, Sumit Semwal, Christian König,
linux-arm-msm, freedreno, linux-media, linaro-mm-sig
On Thu, Aug 05, 2021 at 12:46:57PM +0200, Daniel Vetter wrote:
> drm_sched_job_init is already at the right place, so this boils down
> to deleting code.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Ping for ack&testing please.
-Daniel
> ---
> drivers/gpu/drm/msm/msm_gem.h | 5 -----
> drivers/gpu/drm/msm/msm_gem_submit.c | 19 +++++--------------
> drivers/gpu/drm/msm/msm_ringbuffer.c | 12 ------------
> 3 files changed, 5 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
> index f9e3ffb2309a..8bf0ac707fd7 100644
> --- a/drivers/gpu/drm/msm/msm_gem.h
> +++ b/drivers/gpu/drm/msm/msm_gem.h
> @@ -312,11 +312,6 @@ struct msm_gem_submit {
> struct ww_acquire_ctx ticket;
> uint32_t seqno; /* Sequence number of the submit on the ring */
>
> - /* Array of struct dma_fence * to block on before submitting this job.
> - */
> - struct xarray deps;
> - unsigned long last_dep;
> -
> /* Hw fence, which is created when the scheduler executes the job, and
> * is signaled when the hw finishes (via seqno write from cmdstream)
> */
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index 96cea0ba4cfd..fb5a2eab27a2 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> - xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> -
> kref_init(&submit->ref);
> submit->dev = dev;
> submit->aspace = queue->ctx->aspace;
> @@ -72,8 +70,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> {
> struct msm_gem_submit *submit =
> container_of(kref, struct msm_gem_submit, ref);
> - unsigned long index;
> - struct dma_fence *fence;
> unsigned i;
>
> if (submit->fence_id) {
> @@ -82,12 +78,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> mutex_unlock(&submit->queue->lock);
> }
>
> - xa_for_each (&submit->deps, index, fence) {
> - dma_fence_put(fence);
> - }
> -
> - xa_destroy(&submit->deps);
> -
> dma_fence_put(submit->user_fence);
> dma_fence_put(submit->hw_fence);
>
> @@ -343,8 +333,9 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> if (no_implicit)
> continue;
>
> - ret = drm_gem_fence_array_add_implicit(&submit->deps, obj,
> - write);
> + ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> + obj,
> + write);
> if (ret)
> break;
> }
> @@ -588,7 +579,7 @@ static struct drm_syncobj **msm_parse_deps(struct msm_gem_submit *submit,
> if (ret)
> break;
>
> - ret = drm_gem_fence_array_add(&submit->deps, fence);
> + ret = drm_sched_job_add_dependency(&submit->base, fence);
> if (ret)
> break;
>
> @@ -798,7 +789,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> goto out_unlock;
> }
>
> - ret = drm_gem_fence_array_add(&submit->deps, in_fence);
> + ret = drm_sched_job_add_dependency(&submit->base, in_fence);
> if (ret)
> goto out_unlock;
> }
> diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
> index bd54c1412649..652b1dedd7c1 100644
> --- a/drivers/gpu/drm/msm/msm_ringbuffer.c
> +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
> @@ -11,17 +11,6 @@ static uint num_hw_submissions = 8;
> MODULE_PARM_DESC(num_hw_submissions, "The max # of jobs to write into ringbuffer (default 8)");
> module_param(num_hw_submissions, uint, 0600);
>
> -static struct dma_fence *msm_job_dependency(struct drm_sched_job *job,
> - struct drm_sched_entity *s_entity)
> -{
> - struct msm_gem_submit *submit = to_msm_submit(job);
> -
> - if (!xa_empty(&submit->deps))
> - return xa_erase(&submit->deps, submit->last_dep++);
> -
> - return NULL;
> -}
> -
> static struct dma_fence *msm_job_run(struct drm_sched_job *job)
> {
> struct msm_gem_submit *submit = to_msm_submit(job);
> @@ -52,7 +41,6 @@ static void msm_job_free(struct drm_sched_job *job)
> }
>
> const struct drm_sched_backend_ops msm_sched_ops = {
> - .dependency = msm_job_dependency,
> .run_job = msm_job_run,
> .free_job = msm_job_free
> };
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 12/20] drm/msm: Use scheduler dependency handling
@ 2021-08-12 19:29 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-12 19:29 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Rob Clark, Sean Paul, Sumit Semwal, Christian König,
linux-arm-msm, freedreno, linux-media, linaro-mm-sig
On Thu, Aug 05, 2021 at 12:46:57PM +0200, Daniel Vetter wrote:
> drm_sched_job_init is already at the right place, so this boils down
> to deleting code.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Ping for ack&testing please.
-Daniel
> ---
> drivers/gpu/drm/msm/msm_gem.h | 5 -----
> drivers/gpu/drm/msm/msm_gem_submit.c | 19 +++++--------------
> drivers/gpu/drm/msm/msm_ringbuffer.c | 12 ------------
> 3 files changed, 5 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
> index f9e3ffb2309a..8bf0ac707fd7 100644
> --- a/drivers/gpu/drm/msm/msm_gem.h
> +++ b/drivers/gpu/drm/msm/msm_gem.h
> @@ -312,11 +312,6 @@ struct msm_gem_submit {
> struct ww_acquire_ctx ticket;
> uint32_t seqno; /* Sequence number of the submit on the ring */
>
> - /* Array of struct dma_fence * to block on before submitting this job.
> - */
> - struct xarray deps;
> - unsigned long last_dep;
> -
> /* Hw fence, which is created when the scheduler executes the job, and
> * is signaled when the hw finishes (via seqno write from cmdstream)
> */
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index 96cea0ba4cfd..fb5a2eab27a2 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> - xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> -
> kref_init(&submit->ref);
> submit->dev = dev;
> submit->aspace = queue->ctx->aspace;
> @@ -72,8 +70,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> {
> struct msm_gem_submit *submit =
> container_of(kref, struct msm_gem_submit, ref);
> - unsigned long index;
> - struct dma_fence *fence;
> unsigned i;
>
> if (submit->fence_id) {
> @@ -82,12 +78,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> mutex_unlock(&submit->queue->lock);
> }
>
> - xa_for_each (&submit->deps, index, fence) {
> - dma_fence_put(fence);
> - }
> -
> - xa_destroy(&submit->deps);
> -
> dma_fence_put(submit->user_fence);
> dma_fence_put(submit->hw_fence);
>
> @@ -343,8 +333,9 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> if (no_implicit)
> continue;
>
> - ret = drm_gem_fence_array_add_implicit(&submit->deps, obj,
> - write);
> + ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> + obj,
> + write);
> if (ret)
> break;
> }
> @@ -588,7 +579,7 @@ static struct drm_syncobj **msm_parse_deps(struct msm_gem_submit *submit,
> if (ret)
> break;
>
> - ret = drm_gem_fence_array_add(&submit->deps, fence);
> + ret = drm_sched_job_add_dependency(&submit->base, fence);
> if (ret)
> break;
>
> @@ -798,7 +789,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> goto out_unlock;
> }
>
> - ret = drm_gem_fence_array_add(&submit->deps, in_fence);
> + ret = drm_sched_job_add_dependency(&submit->base, in_fence);
> if (ret)
> goto out_unlock;
> }
> diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
> index bd54c1412649..652b1dedd7c1 100644
> --- a/drivers/gpu/drm/msm/msm_ringbuffer.c
> +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
> @@ -11,17 +11,6 @@ static uint num_hw_submissions = 8;
> MODULE_PARM_DESC(num_hw_submissions, "The max # of jobs to write into ringbuffer (default 8)");
> module_param(num_hw_submissions, uint, 0600);
>
> -static struct dma_fence *msm_job_dependency(struct drm_sched_job *job,
> - struct drm_sched_entity *s_entity)
> -{
> - struct msm_gem_submit *submit = to_msm_submit(job);
> -
> - if (!xa_empty(&submit->deps))
> - return xa_erase(&submit->deps, submit->last_dep++);
> -
> - return NULL;
> -}
> -
> static struct dma_fence *msm_job_run(struct drm_sched_job *job)
> {
> struct msm_gem_submit *submit = to_msm_submit(job);
> @@ -52,7 +41,6 @@ static void msm_job_free(struct drm_sched_job *job)
> }
>
> const struct drm_sched_backend_ops msm_sched_ops = {
> - .dependency = msm_job_dependency,
> .run_job = msm_job_run,
> .free_job = msm_job_free
> };
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [PATCH v5 12/20] drm/msm: Use scheduler dependency handling
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-26 16:12 ` Rob Clark
-1 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-26 16:12 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Daniel Vetter,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK
On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> drm_sched_job_init is already at the right place, so this boils down
> to deleting code.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
r-b
> ---
> drivers/gpu/drm/msm/msm_gem.h | 5 -----
> drivers/gpu/drm/msm/msm_gem_submit.c | 19 +++++--------------
> drivers/gpu/drm/msm/msm_ringbuffer.c | 12 ------------
> 3 files changed, 5 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
> index f9e3ffb2309a..8bf0ac707fd7 100644
> --- a/drivers/gpu/drm/msm/msm_gem.h
> +++ b/drivers/gpu/drm/msm/msm_gem.h
> @@ -312,11 +312,6 @@ struct msm_gem_submit {
> struct ww_acquire_ctx ticket;
> uint32_t seqno; /* Sequence number of the submit on the ring */
>
> - /* Array of struct dma_fence * to block on before submitting this job.
> - */
> - struct xarray deps;
> - unsigned long last_dep;
> -
> /* Hw fence, which is created when the scheduler executes the job, and
> * is signaled when the hw finishes (via seqno write from cmdstream)
> */
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index 96cea0ba4cfd..fb5a2eab27a2 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> - xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> -
> kref_init(&submit->ref);
> submit->dev = dev;
> submit->aspace = queue->ctx->aspace;
> @@ -72,8 +70,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> {
> struct msm_gem_submit *submit =
> container_of(kref, struct msm_gem_submit, ref);
> - unsigned long index;
> - struct dma_fence *fence;
> unsigned i;
>
> if (submit->fence_id) {
> @@ -82,12 +78,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> mutex_unlock(&submit->queue->lock);
> }
>
> - xa_for_each (&submit->deps, index, fence) {
> - dma_fence_put(fence);
> - }
> -
> - xa_destroy(&submit->deps);
> -
> dma_fence_put(submit->user_fence);
> dma_fence_put(submit->hw_fence);
>
> @@ -343,8 +333,9 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> if (no_implicit)
> continue;
>
> - ret = drm_gem_fence_array_add_implicit(&submit->deps, obj,
> - write);
> + ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> + obj,
> + write);
> if (ret)
> break;
> }
> @@ -588,7 +579,7 @@ static struct drm_syncobj **msm_parse_deps(struct msm_gem_submit *submit,
> if (ret)
> break;
>
> - ret = drm_gem_fence_array_add(&submit->deps, fence);
> + ret = drm_sched_job_add_dependency(&submit->base, fence);
> if (ret)
> break;
>
> @@ -798,7 +789,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> goto out_unlock;
> }
>
> - ret = drm_gem_fence_array_add(&submit->deps, in_fence);
> + ret = drm_sched_job_add_dependency(&submit->base, in_fence);
> if (ret)
> goto out_unlock;
> }
> diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
> index bd54c1412649..652b1dedd7c1 100644
> --- a/drivers/gpu/drm/msm/msm_ringbuffer.c
> +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
> @@ -11,17 +11,6 @@ static uint num_hw_submissions = 8;
> MODULE_PARM_DESC(num_hw_submissions, "The max # of jobs to write into ringbuffer (default 8)");
> module_param(num_hw_submissions, uint, 0600);
>
> -static struct dma_fence *msm_job_dependency(struct drm_sched_job *job,
> - struct drm_sched_entity *s_entity)
> -{
> - struct msm_gem_submit *submit = to_msm_submit(job);
> -
> - if (!xa_empty(&submit->deps))
> - return xa_erase(&submit->deps, submit->last_dep++);
> -
> - return NULL;
> -}
> -
> static struct dma_fence *msm_job_run(struct drm_sched_job *job)
> {
> struct msm_gem_submit *submit = to_msm_submit(job);
> @@ -52,7 +41,6 @@ static void msm_job_free(struct drm_sched_job *job)
> }
>
> const struct drm_sched_backend_ops msm_sched_ops = {
> - .dependency = msm_job_dependency,
> .run_job = msm_job_run,
> .free_job = msm_job_free
> };
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 12/20] drm/msm: Use scheduler dependency handling
@ 2021-08-26 16:12 ` Rob Clark
0 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-26 16:12 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Daniel Vetter,
Sean Paul, Sumit Semwal, Christian König, linux-arm-msm,
freedreno, open list:DMA BUFFER SHARING FRAMEWORK,
moderated list:DMA BUFFER SHARING FRAMEWORK
On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> drm_sched_job_init is already at the right place, so this boils down
> to deleting code.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
r-b
> ---
> drivers/gpu/drm/msm/msm_gem.h | 5 -----
> drivers/gpu/drm/msm/msm_gem_submit.c | 19 +++++--------------
> drivers/gpu/drm/msm/msm_ringbuffer.c | 12 ------------
> 3 files changed, 5 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
> index f9e3ffb2309a..8bf0ac707fd7 100644
> --- a/drivers/gpu/drm/msm/msm_gem.h
> +++ b/drivers/gpu/drm/msm/msm_gem.h
> @@ -312,11 +312,6 @@ struct msm_gem_submit {
> struct ww_acquire_ctx ticket;
> uint32_t seqno; /* Sequence number of the submit on the ring */
>
> - /* Array of struct dma_fence * to block on before submitting this job.
> - */
> - struct xarray deps;
> - unsigned long last_dep;
> -
> /* Hw fence, which is created when the scheduler executes the job, and
> * is signaled when the hw finishes (via seqno write from cmdstream)
> */
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index 96cea0ba4cfd..fb5a2eab27a2 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> - xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> -
> kref_init(&submit->ref);
> submit->dev = dev;
> submit->aspace = queue->ctx->aspace;
> @@ -72,8 +70,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> {
> struct msm_gem_submit *submit =
> container_of(kref, struct msm_gem_submit, ref);
> - unsigned long index;
> - struct dma_fence *fence;
> unsigned i;
>
> if (submit->fence_id) {
> @@ -82,12 +78,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> mutex_unlock(&submit->queue->lock);
> }
>
> - xa_for_each (&submit->deps, index, fence) {
> - dma_fence_put(fence);
> - }
> -
> - xa_destroy(&submit->deps);
> -
> dma_fence_put(submit->user_fence);
> dma_fence_put(submit->hw_fence);
>
> @@ -343,8 +333,9 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> if (no_implicit)
> continue;
>
> - ret = drm_gem_fence_array_add_implicit(&submit->deps, obj,
> - write);
> + ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> + obj,
> + write);
> if (ret)
> break;
> }
> @@ -588,7 +579,7 @@ static struct drm_syncobj **msm_parse_deps(struct msm_gem_submit *submit,
> if (ret)
> break;
>
> - ret = drm_gem_fence_array_add(&submit->deps, fence);
> + ret = drm_sched_job_add_dependency(&submit->base, fence);
> if (ret)
> break;
>
> @@ -798,7 +789,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> goto out_unlock;
> }
>
> - ret = drm_gem_fence_array_add(&submit->deps, in_fence);
> + ret = drm_sched_job_add_dependency(&submit->base, in_fence);
> if (ret)
> goto out_unlock;
> }
> diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
> index bd54c1412649..652b1dedd7c1 100644
> --- a/drivers/gpu/drm/msm/msm_ringbuffer.c
> +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
> @@ -11,17 +11,6 @@ static uint num_hw_submissions = 8;
> MODULE_PARM_DESC(num_hw_submissions, "The max # of jobs to write into ringbuffer (default 8)");
> module_param(num_hw_submissions, uint, 0600);
>
> -static struct dma_fence *msm_job_dependency(struct drm_sched_job *job,
> - struct drm_sched_entity *s_entity)
> -{
> - struct msm_gem_submit *submit = to_msm_submit(job);
> -
> - if (!xa_empty(&submit->deps))
> - return xa_erase(&submit->deps, submit->last_dep++);
> -
> - return NULL;
> -}
> -
> static struct dma_fence *msm_job_run(struct drm_sched_job *job)
> {
> struct msm_gem_submit *submit = to_msm_submit(job);
> @@ -52,7 +41,6 @@ static void msm_job_free(struct drm_sched_job *job)
> }
>
> const struct drm_sched_backend_ops msm_sched_ops = {
> - .dependency = msm_job_dependency,
> .run_job = msm_job_run,
> .free_job = msm_job_free
> };
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [PATCH v5 12/20] drm/msm: Use scheduler dependency handling
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-30 9:01 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-30 9:01 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Rob Clark, Sean Paul, Sumit Semwal, Christian König,
linux-arm-msm, freedreno, linux-media, linaro-mm-sig
On Thu, Aug 05, 2021 at 12:46:57PM +0200, Daniel Vetter wrote:
> drm_sched_job_init is already at the right place, so this boils down
> to deleting code.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Merged up to this patch, except for etnaviv.
-Daniel
> ---
> drivers/gpu/drm/msm/msm_gem.h | 5 -----
> drivers/gpu/drm/msm/msm_gem_submit.c | 19 +++++--------------
> drivers/gpu/drm/msm/msm_ringbuffer.c | 12 ------------
> 3 files changed, 5 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
> index f9e3ffb2309a..8bf0ac707fd7 100644
> --- a/drivers/gpu/drm/msm/msm_gem.h
> +++ b/drivers/gpu/drm/msm/msm_gem.h
> @@ -312,11 +312,6 @@ struct msm_gem_submit {
> struct ww_acquire_ctx ticket;
> uint32_t seqno; /* Sequence number of the submit on the ring */
>
> - /* Array of struct dma_fence * to block on before submitting this job.
> - */
> - struct xarray deps;
> - unsigned long last_dep;
> -
> /* Hw fence, which is created when the scheduler executes the job, and
> * is signaled when the hw finishes (via seqno write from cmdstream)
> */
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index 96cea0ba4cfd..fb5a2eab27a2 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> - xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> -
> kref_init(&submit->ref);
> submit->dev = dev;
> submit->aspace = queue->ctx->aspace;
> @@ -72,8 +70,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> {
> struct msm_gem_submit *submit =
> container_of(kref, struct msm_gem_submit, ref);
> - unsigned long index;
> - struct dma_fence *fence;
> unsigned i;
>
> if (submit->fence_id) {
> @@ -82,12 +78,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> mutex_unlock(&submit->queue->lock);
> }
>
> - xa_for_each (&submit->deps, index, fence) {
> - dma_fence_put(fence);
> - }
> -
> - xa_destroy(&submit->deps);
> -
> dma_fence_put(submit->user_fence);
> dma_fence_put(submit->hw_fence);
>
> @@ -343,8 +333,9 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> if (no_implicit)
> continue;
>
> - ret = drm_gem_fence_array_add_implicit(&submit->deps, obj,
> - write);
> + ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> + obj,
> + write);
> if (ret)
> break;
> }
> @@ -588,7 +579,7 @@ static struct drm_syncobj **msm_parse_deps(struct msm_gem_submit *submit,
> if (ret)
> break;
>
> - ret = drm_gem_fence_array_add(&submit->deps, fence);
> + ret = drm_sched_job_add_dependency(&submit->base, fence);
> if (ret)
> break;
>
> @@ -798,7 +789,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> goto out_unlock;
> }
>
> - ret = drm_gem_fence_array_add(&submit->deps, in_fence);
> + ret = drm_sched_job_add_dependency(&submit->base, in_fence);
> if (ret)
> goto out_unlock;
> }
> diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
> index bd54c1412649..652b1dedd7c1 100644
> --- a/drivers/gpu/drm/msm/msm_ringbuffer.c
> +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
> @@ -11,17 +11,6 @@ static uint num_hw_submissions = 8;
> MODULE_PARM_DESC(num_hw_submissions, "The max # of jobs to write into ringbuffer (default 8)");
> module_param(num_hw_submissions, uint, 0600);
>
> -static struct dma_fence *msm_job_dependency(struct drm_sched_job *job,
> - struct drm_sched_entity *s_entity)
> -{
> - struct msm_gem_submit *submit = to_msm_submit(job);
> -
> - if (!xa_empty(&submit->deps))
> - return xa_erase(&submit->deps, submit->last_dep++);
> -
> - return NULL;
> -}
> -
> static struct dma_fence *msm_job_run(struct drm_sched_job *job)
> {
> struct msm_gem_submit *submit = to_msm_submit(job);
> @@ -52,7 +41,6 @@ static void msm_job_free(struct drm_sched_job *job)
> }
>
> const struct drm_sched_backend_ops msm_sched_ops = {
> - .dependency = msm_job_dependency,
> .run_job = msm_job_run,
> .free_job = msm_job_free
> };
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 12/20] drm/msm: Use scheduler dependency handling
@ 2021-08-30 9:01 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-30 9:01 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Rob Clark, Sean Paul, Sumit Semwal, Christian König,
linux-arm-msm, freedreno, linux-media, linaro-mm-sig
On Thu, Aug 05, 2021 at 12:46:57PM +0200, Daniel Vetter wrote:
> drm_sched_job_init is already at the right place, so this boils down
> to deleting code.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Merged up to this patch, except for etnaviv.
-Daniel
> ---
> drivers/gpu/drm/msm/msm_gem.h | 5 -----
> drivers/gpu/drm/msm/msm_gem_submit.c | 19 +++++--------------
> drivers/gpu/drm/msm/msm_ringbuffer.c | 12 ------------
> 3 files changed, 5 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
> index f9e3ffb2309a..8bf0ac707fd7 100644
> --- a/drivers/gpu/drm/msm/msm_gem.h
> +++ b/drivers/gpu/drm/msm/msm_gem.h
> @@ -312,11 +312,6 @@ struct msm_gem_submit {
> struct ww_acquire_ctx ticket;
> uint32_t seqno; /* Sequence number of the submit on the ring */
>
> - /* Array of struct dma_fence * to block on before submitting this job.
> - */
> - struct xarray deps;
> - unsigned long last_dep;
> -
> /* Hw fence, which is created when the scheduler executes the job, and
> * is signaled when the hw finishes (via seqno write from cmdstream)
> */
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index 96cea0ba4cfd..fb5a2eab27a2 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -52,8 +52,6 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> return ERR_PTR(ret);
> }
>
> - xa_init_flags(&submit->deps, XA_FLAGS_ALLOC);
> -
> kref_init(&submit->ref);
> submit->dev = dev;
> submit->aspace = queue->ctx->aspace;
> @@ -72,8 +70,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> {
> struct msm_gem_submit *submit =
> container_of(kref, struct msm_gem_submit, ref);
> - unsigned long index;
> - struct dma_fence *fence;
> unsigned i;
>
> if (submit->fence_id) {
> @@ -82,12 +78,6 @@ void __msm_gem_submit_destroy(struct kref *kref)
> mutex_unlock(&submit->queue->lock);
> }
>
> - xa_for_each (&submit->deps, index, fence) {
> - dma_fence_put(fence);
> - }
> -
> - xa_destroy(&submit->deps);
> -
> dma_fence_put(submit->user_fence);
> dma_fence_put(submit->hw_fence);
>
> @@ -343,8 +333,9 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> if (no_implicit)
> continue;
>
> - ret = drm_gem_fence_array_add_implicit(&submit->deps, obj,
> - write);
> + ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> + obj,
> + write);
> if (ret)
> break;
> }
> @@ -588,7 +579,7 @@ static struct drm_syncobj **msm_parse_deps(struct msm_gem_submit *submit,
> if (ret)
> break;
>
> - ret = drm_gem_fence_array_add(&submit->deps, fence);
> + ret = drm_sched_job_add_dependency(&submit->base, fence);
> if (ret)
> break;
>
> @@ -798,7 +789,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
> goto out_unlock;
> }
>
> - ret = drm_gem_fence_array_add(&submit->deps, in_fence);
> + ret = drm_sched_job_add_dependency(&submit->base, in_fence);
> if (ret)
> goto out_unlock;
> }
> diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
> index bd54c1412649..652b1dedd7c1 100644
> --- a/drivers/gpu/drm/msm/msm_ringbuffer.c
> +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
> @@ -11,17 +11,6 @@ static uint num_hw_submissions = 8;
> MODULE_PARM_DESC(num_hw_submissions, "The max # of jobs to write into ringbuffer (default 8)");
> module_param(num_hw_submissions, uint, 0600);
>
> -static struct dma_fence *msm_job_dependency(struct drm_sched_job *job,
> - struct drm_sched_entity *s_entity)
> -{
> - struct msm_gem_submit *submit = to_msm_submit(job);
> -
> - if (!xa_empty(&submit->deps))
> - return xa_erase(&submit->deps, submit->last_dep++);
> -
> - return NULL;
> -}
> -
> static struct dma_fence *msm_job_run(struct drm_sched_job *job)
> {
> struct msm_gem_submit *submit = to_msm_submit(job);
> @@ -52,7 +41,6 @@ static void msm_job_free(struct drm_sched_job *job)
> }
>
> const struct drm_sched_backend_ops msm_sched_ops = {
> - .dependency = msm_job_dependency,
> .run_job = msm_job_run,
> .free_job = msm_job_free
> };
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 13/20] drm/gem: Delete gem array fencing helpers
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
David Airlie, Daniel Vetter, Sumit Semwal, Christian König,
linux-media, linaro-mm-sig
Integrated into the scheduler now and all users converted over.
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/gpu/drm/drm_gem.c | 96 ---------------------------------------
include/drm/drm_gem.h | 5 --
2 files changed, 101 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 09c820045859..37e2e2820f08 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1272,99 +1272,3 @@ drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
ww_acquire_fini(acquire_ctx);
}
EXPORT_SYMBOL(drm_gem_unlock_reservations);
-
-/**
- * drm_gem_fence_array_add - Adds the fence to an array of fences to be
- * waited on, deduplicating fences from the same context.
- *
- * @fence_array: array of dma_fence * for the job to block on.
- * @fence: the dma_fence to add to the list of dependencies.
- *
- * This functions consumes the reference for @fence both on success and error
- * cases.
- *
- * Returns:
- * 0 on success, or an error on failing to expand the array.
- */
-int drm_gem_fence_array_add(struct xarray *fence_array,
- struct dma_fence *fence)
-{
- struct dma_fence *entry;
- unsigned long index;
- u32 id = 0;
- int ret;
-
- if (!fence)
- return 0;
-
- /* Deduplicate if we already depend on a fence from the same context.
- * This lets the size of the array of deps scale with the number of
- * engines involved, rather than the number of BOs.
- */
- xa_for_each(fence_array, index, entry) {
- if (entry->context != fence->context)
- continue;
-
- if (dma_fence_is_later(fence, entry)) {
- dma_fence_put(entry);
- xa_store(fence_array, index, fence, GFP_KERNEL);
- } else {
- dma_fence_put(fence);
- }
- return 0;
- }
-
- ret = xa_alloc(fence_array, &id, fence, xa_limit_32b, GFP_KERNEL);
- if (ret != 0)
- dma_fence_put(fence);
-
- return ret;
-}
-EXPORT_SYMBOL(drm_gem_fence_array_add);
-
-/**
- * drm_gem_fence_array_add_implicit - Adds the implicit dependencies tracked
- * in the GEM object's reservation object to an array of dma_fences for use in
- * scheduling a rendering job.
- *
- * This should be called after drm_gem_lock_reservations() on your array of
- * GEM objects used in the job but before updating the reservations with your
- * own fences.
- *
- * @fence_array: array of dma_fence * for the job to block on.
- * @obj: the gem object to add new dependencies from.
- * @write: whether the job might write the object (so we need to depend on
- * shared fences in the reservation object).
- */
-int drm_gem_fence_array_add_implicit(struct xarray *fence_array,
- struct drm_gem_object *obj,
- bool write)
-{
- int ret;
- struct dma_fence **fences;
- unsigned int i, fence_count;
-
- if (!write) {
- struct dma_fence *fence =
- dma_resv_get_excl_unlocked(obj->resv);
-
- return drm_gem_fence_array_add(fence_array, fence);
- }
-
- ret = dma_resv_get_fences(obj->resv, NULL,
- &fence_count, &fences);
- if (ret || !fence_count)
- return ret;
-
- for (i = 0; i < fence_count; i++) {
- ret = drm_gem_fence_array_add(fence_array, fences[i]);
- if (ret)
- break;
- }
-
- for (; i < fence_count; i++)
- dma_fence_put(fences[i]);
- kfree(fences);
- return ret;
-}
-EXPORT_SYMBOL(drm_gem_fence_array_add_implicit);
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index 35e7f44c2a75..e55a767188af 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -407,11 +407,6 @@ int drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
struct ww_acquire_ctx *acquire_ctx);
void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
struct ww_acquire_ctx *acquire_ctx);
-int drm_gem_fence_array_add(struct xarray *fence_array,
- struct dma_fence *fence);
-int drm_gem_fence_array_add_implicit(struct xarray *fence_array,
- struct drm_gem_object *obj,
- bool write);
int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
u32 handle, u64 *offset);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 13/20] drm/gem: Delete gem array fencing helpers
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
David Airlie, Daniel Vetter, Sumit Semwal, Christian König,
linux-media, linaro-mm-sig
Integrated into the scheduler now and all users converted over.
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/gpu/drm/drm_gem.c | 96 ---------------------------------------
include/drm/drm_gem.h | 5 --
2 files changed, 101 deletions(-)
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 09c820045859..37e2e2820f08 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1272,99 +1272,3 @@ drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
ww_acquire_fini(acquire_ctx);
}
EXPORT_SYMBOL(drm_gem_unlock_reservations);
-
-/**
- * drm_gem_fence_array_add - Adds the fence to an array of fences to be
- * waited on, deduplicating fences from the same context.
- *
- * @fence_array: array of dma_fence * for the job to block on.
- * @fence: the dma_fence to add to the list of dependencies.
- *
- * This functions consumes the reference for @fence both on success and error
- * cases.
- *
- * Returns:
- * 0 on success, or an error on failing to expand the array.
- */
-int drm_gem_fence_array_add(struct xarray *fence_array,
- struct dma_fence *fence)
-{
- struct dma_fence *entry;
- unsigned long index;
- u32 id = 0;
- int ret;
-
- if (!fence)
- return 0;
-
- /* Deduplicate if we already depend on a fence from the same context.
- * This lets the size of the array of deps scale with the number of
- * engines involved, rather than the number of BOs.
- */
- xa_for_each(fence_array, index, entry) {
- if (entry->context != fence->context)
- continue;
-
- if (dma_fence_is_later(fence, entry)) {
- dma_fence_put(entry);
- xa_store(fence_array, index, fence, GFP_KERNEL);
- } else {
- dma_fence_put(fence);
- }
- return 0;
- }
-
- ret = xa_alloc(fence_array, &id, fence, xa_limit_32b, GFP_KERNEL);
- if (ret != 0)
- dma_fence_put(fence);
-
- return ret;
-}
-EXPORT_SYMBOL(drm_gem_fence_array_add);
-
-/**
- * drm_gem_fence_array_add_implicit - Adds the implicit dependencies tracked
- * in the GEM object's reservation object to an array of dma_fences for use in
- * scheduling a rendering job.
- *
- * This should be called after drm_gem_lock_reservations() on your array of
- * GEM objects used in the job but before updating the reservations with your
- * own fences.
- *
- * @fence_array: array of dma_fence * for the job to block on.
- * @obj: the gem object to add new dependencies from.
- * @write: whether the job might write the object (so we need to depend on
- * shared fences in the reservation object).
- */
-int drm_gem_fence_array_add_implicit(struct xarray *fence_array,
- struct drm_gem_object *obj,
- bool write)
-{
- int ret;
- struct dma_fence **fences;
- unsigned int i, fence_count;
-
- if (!write) {
- struct dma_fence *fence =
- dma_resv_get_excl_unlocked(obj->resv);
-
- return drm_gem_fence_array_add(fence_array, fence);
- }
-
- ret = dma_resv_get_fences(obj->resv, NULL,
- &fence_count, &fences);
- if (ret || !fence_count)
- return ret;
-
- for (i = 0; i < fence_count; i++) {
- ret = drm_gem_fence_array_add(fence_array, fences[i]);
- if (ret)
- break;
- }
-
- for (; i < fence_count; i++)
- dma_fence_put(fences[i]);
- kfree(fences);
- return ret;
-}
-EXPORT_SYMBOL(drm_gem_fence_array_add_implicit);
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index 35e7f44c2a75..e55a767188af 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -407,11 +407,6 @@ int drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
struct ww_acquire_ctx *acquire_ctx);
void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
struct ww_acquire_ctx *acquire_ctx);
-int drm_gem_fence_array_add(struct xarray *fence_array,
- struct dma_fence *fence);
-int drm_gem_fence_array_add_implicit(struct xarray *fence_array,
- struct drm_gem_object *obj,
- bool write);
int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
u32 handle, u64 *offset);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 13/20] drm/gem: Delete gem array fencing helpers
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-12 19:29 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-12 19:29 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
David Airlie, Daniel Vetter, Sumit Semwal, Christian König,
linux-media, linaro-mm-sig
On Thu, Aug 05, 2021 at 12:46:58PM +0200, Daniel Vetter wrote:
> Integrated into the scheduler now and all users converted over.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> Cc: Maxime Ripard <mripard@kernel.org>
> Cc: Thomas Zimmermann <tzimmermann@suse.de>
> Cc: David Airlie <airlied@linux.ie>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Some acks would be great here.
-Daniel
> ---
> drivers/gpu/drm/drm_gem.c | 96 ---------------------------------------
> include/drm/drm_gem.h | 5 --
> 2 files changed, 101 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index 09c820045859..37e2e2820f08 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1272,99 +1272,3 @@ drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
> ww_acquire_fini(acquire_ctx);
> }
> EXPORT_SYMBOL(drm_gem_unlock_reservations);
> -
> -/**
> - * drm_gem_fence_array_add - Adds the fence to an array of fences to be
> - * waited on, deduplicating fences from the same context.
> - *
> - * @fence_array: array of dma_fence * for the job to block on.
> - * @fence: the dma_fence to add to the list of dependencies.
> - *
> - * This functions consumes the reference for @fence both on success and error
> - * cases.
> - *
> - * Returns:
> - * 0 on success, or an error on failing to expand the array.
> - */
> -int drm_gem_fence_array_add(struct xarray *fence_array,
> - struct dma_fence *fence)
> -{
> - struct dma_fence *entry;
> - unsigned long index;
> - u32 id = 0;
> - int ret;
> -
> - if (!fence)
> - return 0;
> -
> - /* Deduplicate if we already depend on a fence from the same context.
> - * This lets the size of the array of deps scale with the number of
> - * engines involved, rather than the number of BOs.
> - */
> - xa_for_each(fence_array, index, entry) {
> - if (entry->context != fence->context)
> - continue;
> -
> - if (dma_fence_is_later(fence, entry)) {
> - dma_fence_put(entry);
> - xa_store(fence_array, index, fence, GFP_KERNEL);
> - } else {
> - dma_fence_put(fence);
> - }
> - return 0;
> - }
> -
> - ret = xa_alloc(fence_array, &id, fence, xa_limit_32b, GFP_KERNEL);
> - if (ret != 0)
> - dma_fence_put(fence);
> -
> - return ret;
> -}
> -EXPORT_SYMBOL(drm_gem_fence_array_add);
> -
> -/**
> - * drm_gem_fence_array_add_implicit - Adds the implicit dependencies tracked
> - * in the GEM object's reservation object to an array of dma_fences for use in
> - * scheduling a rendering job.
> - *
> - * This should be called after drm_gem_lock_reservations() on your array of
> - * GEM objects used in the job but before updating the reservations with your
> - * own fences.
> - *
> - * @fence_array: array of dma_fence * for the job to block on.
> - * @obj: the gem object to add new dependencies from.
> - * @write: whether the job might write the object (so we need to depend on
> - * shared fences in the reservation object).
> - */
> -int drm_gem_fence_array_add_implicit(struct xarray *fence_array,
> - struct drm_gem_object *obj,
> - bool write)
> -{
> - int ret;
> - struct dma_fence **fences;
> - unsigned int i, fence_count;
> -
> - if (!write) {
> - struct dma_fence *fence =
> - dma_resv_get_excl_unlocked(obj->resv);
> -
> - return drm_gem_fence_array_add(fence_array, fence);
> - }
> -
> - ret = dma_resv_get_fences(obj->resv, NULL,
> - &fence_count, &fences);
> - if (ret || !fence_count)
> - return ret;
> -
> - for (i = 0; i < fence_count; i++) {
> - ret = drm_gem_fence_array_add(fence_array, fences[i]);
> - if (ret)
> - break;
> - }
> -
> - for (; i < fence_count; i++)
> - dma_fence_put(fences[i]);
> - kfree(fences);
> - return ret;
> -}
> -EXPORT_SYMBOL(drm_gem_fence_array_add_implicit);
> diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
> index 35e7f44c2a75..e55a767188af 100644
> --- a/include/drm/drm_gem.h
> +++ b/include/drm/drm_gem.h
> @@ -407,11 +407,6 @@ int drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
> struct ww_acquire_ctx *acquire_ctx);
> void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
> struct ww_acquire_ctx *acquire_ctx);
> -int drm_gem_fence_array_add(struct xarray *fence_array,
> - struct dma_fence *fence);
> -int drm_gem_fence_array_add_implicit(struct xarray *fence_array,
> - struct drm_gem_object *obj,
> - bool write);
> int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
> u32 handle, u64 *offset);
>
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 13/20] drm/gem: Delete gem array fencing helpers
@ 2021-08-12 19:29 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-12 19:29 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
David Airlie, Daniel Vetter, Sumit Semwal, Christian König,
linux-media, linaro-mm-sig
On Thu, Aug 05, 2021 at 12:46:58PM +0200, Daniel Vetter wrote:
> Integrated into the scheduler now and all users converted over.
>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> Cc: Maxime Ripard <mripard@kernel.org>
> Cc: Thomas Zimmermann <tzimmermann@suse.de>
> Cc: David Airlie <airlied@linux.ie>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Some acks would be great here.
-Daniel
> ---
> drivers/gpu/drm/drm_gem.c | 96 ---------------------------------------
> include/drm/drm_gem.h | 5 --
> 2 files changed, 101 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index 09c820045859..37e2e2820f08 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1272,99 +1272,3 @@ drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
> ww_acquire_fini(acquire_ctx);
> }
> EXPORT_SYMBOL(drm_gem_unlock_reservations);
> -
> -/**
> - * drm_gem_fence_array_add - Adds the fence to an array of fences to be
> - * waited on, deduplicating fences from the same context.
> - *
> - * @fence_array: array of dma_fence * for the job to block on.
> - * @fence: the dma_fence to add to the list of dependencies.
> - *
> - * This functions consumes the reference for @fence both on success and error
> - * cases.
> - *
> - * Returns:
> - * 0 on success, or an error on failing to expand the array.
> - */
> -int drm_gem_fence_array_add(struct xarray *fence_array,
> - struct dma_fence *fence)
> -{
> - struct dma_fence *entry;
> - unsigned long index;
> - u32 id = 0;
> - int ret;
> -
> - if (!fence)
> - return 0;
> -
> - /* Deduplicate if we already depend on a fence from the same context.
> - * This lets the size of the array of deps scale with the number of
> - * engines involved, rather than the number of BOs.
> - */
> - xa_for_each(fence_array, index, entry) {
> - if (entry->context != fence->context)
> - continue;
> -
> - if (dma_fence_is_later(fence, entry)) {
> - dma_fence_put(entry);
> - xa_store(fence_array, index, fence, GFP_KERNEL);
> - } else {
> - dma_fence_put(fence);
> - }
> - return 0;
> - }
> -
> - ret = xa_alloc(fence_array, &id, fence, xa_limit_32b, GFP_KERNEL);
> - if (ret != 0)
> - dma_fence_put(fence);
> -
> - return ret;
> -}
> -EXPORT_SYMBOL(drm_gem_fence_array_add);
> -
> -/**
> - * drm_gem_fence_array_add_implicit - Adds the implicit dependencies tracked
> - * in the GEM object's reservation object to an array of dma_fences for use in
> - * scheduling a rendering job.
> - *
> - * This should be called after drm_gem_lock_reservations() on your array of
> - * GEM objects used in the job but before updating the reservations with your
> - * own fences.
> - *
> - * @fence_array: array of dma_fence * for the job to block on.
> - * @obj: the gem object to add new dependencies from.
> - * @write: whether the job might write the object (so we need to depend on
> - * shared fences in the reservation object).
> - */
> -int drm_gem_fence_array_add_implicit(struct xarray *fence_array,
> - struct drm_gem_object *obj,
> - bool write)
> -{
> - int ret;
> - struct dma_fence **fences;
> - unsigned int i, fence_count;
> -
> - if (!write) {
> - struct dma_fence *fence =
> - dma_resv_get_excl_unlocked(obj->resv);
> -
> - return drm_gem_fence_array_add(fence_array, fence);
> - }
> -
> - ret = dma_resv_get_fences(obj->resv, NULL,
> - &fence_count, &fences);
> - if (ret || !fence_count)
> - return ret;
> -
> - for (i = 0; i < fence_count; i++) {
> - ret = drm_gem_fence_array_add(fence_array, fences[i]);
> - if (ret)
> - break;
> - }
> -
> - for (; i < fence_count; i++)
> - dma_fence_put(fences[i]);
> - kfree(fences);
> - return ret;
> -}
> -EXPORT_SYMBOL(drm_gem_fence_array_add_implicit);
> diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
> index 35e7f44c2a75..e55a767188af 100644
> --- a/include/drm/drm_gem.h
> +++ b/include/drm/drm_gem.h
> @@ -407,11 +407,6 @@ int drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
> struct ww_acquire_ctx *acquire_ctx);
> void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
> struct ww_acquire_ctx *acquire_ctx);
> -int drm_gem_fence_array_add(struct xarray *fence_array,
> - struct dma_fence *fence);
> -int drm_gem_fence_array_add_implicit(struct xarray *fence_array,
> - struct drm_gem_object *obj,
> - bool write);
> int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
> u32 handle, u64 *offset);
>
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 14/20] drm/sched: Don't store self-dependencies
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:46 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Lucas Stach,
Melissa Wen, Daniel Vetter, Christian König, Luben Tuikov,
Andrey Grodzovsky, Alex Deucher
This is essentially part of drm_sched_dependency_optimized(), which
only amdgpu seems to make use of. Use it a bit more.
This would mean that as-is amdgpu can't use the dependency helpers, at
least not with the current approach amdgpu has for deciding whether a
vm_flush is needed. Since amdgpu also has very special rules around
implicit fencing it can't use those helpers either, and adding a
drm_sched_job_await_fence_always or similar for amdgpu wouldn't be too
onerous. That way the special case handling for amdgpu sticks even
more out and we have higher chances that reviewers that go across all
drivers wont miss it.
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Acked-by: Melissa Wen <mwen@igalia.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
---
drivers/gpu/drm/scheduler/sched_main.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index f77456929139..49e507f91ec0 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -660,6 +660,13 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
if (!fence)
return 0;
+ /* if it's a fence from us it's guaranteed to be earlier */
+ if (fence->context == job->entity->fence_context ||
+ fence->context == job->entity->fence_context + 1) {
+ dma_fence_put(fence);
+ return 0;
+ }
+
/* Deduplicate if we already depend on a fence from the same context.
* This lets the size of the array of deps scale with the number of
* engines involved, rather than the number of BOs.
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 14/20] drm/sched: Don't store self-dependencies
@ 2021-08-05 10:46 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:46 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Lucas Stach,
Melissa Wen, Daniel Vetter, Christian König, Luben Tuikov,
Andrey Grodzovsky, Alex Deucher
This is essentially part of drm_sched_dependency_optimized(), which
only amdgpu seems to make use of. Use it a bit more.
This would mean that as-is amdgpu can't use the dependency helpers, at
least not with the current approach amdgpu has for deciding whether a
vm_flush is needed. Since amdgpu also has very special rules around
implicit fencing it can't use those helpers either, and adding a
drm_sched_job_await_fence_always or similar for amdgpu wouldn't be too
onerous. That way the special case handling for amdgpu sticks even
more out and we have higher chances that reviewers that go across all
drivers wont miss it.
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Acked-by: Melissa Wen <mwen@igalia.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
---
drivers/gpu/drm/scheduler/sched_main.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index f77456929139..49e507f91ec0 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -660,6 +660,13 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
if (!fence)
return 0;
+ /* if it's a fence from us it's guaranteed to be earlier */
+ if (fence->context == job->entity->fence_context ||
+ fence->context == job->entity->fence_context + 1) {
+ dma_fence_put(fence);
+ return 0;
+ }
+
/* Deduplicate if we already depend on a fence from the same context.
* This lets the size of the array of deps scale with the number of
* engines involved, rather than the number of BOs.
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 14/20] drm/sched: Don't store self-dependencies
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 13:18 ` Christian König
-1 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:18 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Lucas Stach, Melissa Wen,
Daniel Vetter, Luben Tuikov, Andrey Grodzovsky, Alex Deucher
Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> This is essentially part of drm_sched_dependency_optimized(), which
> only amdgpu seems to make use of. Use it a bit more.
>
> This would mean that as-is amdgpu can't use the dependency helpers, at
> least not with the current approach amdgpu has for deciding whether a
> vm_flush is needed. Since amdgpu also has very special rules around
> implicit fencing it can't use those helpers either, and adding a
> drm_sched_job_await_fence_always or similar for amdgpu wouldn't be too
> onerous. That way the special case handling for amdgpu sticks even
> more out and we have higher chances that reviewers that go across all
> drivers wont miss it.
Well you should probably drop the sentence about the vm_flush, this is
completely unrelated.
Additional to that I still don't think that this is a good idea.
Dependency handling is something completely driver specific.
E.g. even when you have submitted jobs back to back they still might
need a cache flush in between and that is not only for amdgpu like this.
What you can do is to optimize for while looking at the fences later on
and then note that you have done so and what the last hw fence is you
used instead.
Regards,
Christian.
>
> Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
> Acked-by: Melissa Wen <mwen@igalia.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> ---
> drivers/gpu/drm/scheduler/sched_main.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> index f77456929139..49e507f91ec0 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -660,6 +660,13 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
> if (!fence)
> return 0;
>
> + /* if it's a fence from us it's guaranteed to be earlier */
> + if (fence->context == job->entity->fence_context ||
> + fence->context == job->entity->fence_context + 1) {
> + dma_fence_put(fence);
> + return 0;
> + }
> +
> /* Deduplicate if we already depend on a fence from the same context.
> * This lets the size of the array of deps scale with the number of
> * engines involved, rather than the number of BOs.
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 14/20] drm/sched: Don't store self-dependencies
@ 2021-08-05 13:18 ` Christian König
0 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:18 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Lucas Stach, Melissa Wen,
Daniel Vetter, Luben Tuikov, Andrey Grodzovsky, Alex Deucher
Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> This is essentially part of drm_sched_dependency_optimized(), which
> only amdgpu seems to make use of. Use it a bit more.
>
> This would mean that as-is amdgpu can't use the dependency helpers, at
> least not with the current approach amdgpu has for deciding whether a
> vm_flush is needed. Since amdgpu also has very special rules around
> implicit fencing it can't use those helpers either, and adding a
> drm_sched_job_await_fence_always or similar for amdgpu wouldn't be too
> onerous. That way the special case handling for amdgpu sticks even
> more out and we have higher chances that reviewers that go across all
> drivers wont miss it.
Well you should probably drop the sentence about the vm_flush, this is
completely unrelated.
Additional to that I still don't think that this is a good idea.
Dependency handling is something completely driver specific.
E.g. even when you have submitted jobs back to back they still might
need a cache flush in between and that is not only for amdgpu like this.
What you can do is to optimize for while looking at the fences later on
and then note that you have done so and what the last hw fence is you
used instead.
Regards,
Christian.
>
> Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
> Acked-by: Melissa Wen <mwen@igalia.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> ---
> drivers/gpu/drm/scheduler/sched_main.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> index f77456929139..49e507f91ec0 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -660,6 +660,13 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
> if (!fence)
> return 0;
>
> + /* if it's a fence from us it's guaranteed to be earlier */
> + if (fence->context == job->entity->fence_context ||
> + fence->context == job->entity->fence_context + 1) {
> + dma_fence_put(fence);
> + return 0;
> + }
> +
> /* Deduplicate if we already depend on a fence from the same context.
> * This lets the size of the array of deps scale with the number of
> * engines involved, rather than the number of BOs.
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 14/20] drm/sched: Don't store self-dependencies
2021-08-05 13:18 ` [Intel-gfx] " Christian König
@ 2021-08-05 13:25 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 13:25 UTC (permalink / raw)
To: Christian König
Cc: DRI Development, Intel Graphics Development, Lucas Stach,
Melissa Wen, Daniel Vetter, Luben Tuikov, Andrey Grodzovsky,
Alex Deucher
On Thu, Aug 5, 2021 at 3:18 PM Christian König <christian.koenig@amd.com> wrote:
>
>
>
> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> > This is essentially part of drm_sched_dependency_optimized(), which
> > only amdgpu seems to make use of. Use it a bit more.
> >
> > This would mean that as-is amdgpu can't use the dependency helpers, at
> > least not with the current approach amdgpu has for deciding whether a
> > vm_flush is needed. Since amdgpu also has very special rules around
> > implicit fencing it can't use those helpers either, and adding a
> > drm_sched_job_await_fence_always or similar for amdgpu wouldn't be too
> > onerous. That way the special case handling for amdgpu sticks even
> > more out and we have higher chances that reviewers that go across all
> > drivers wont miss it.
>
> Well you should probably drop the sentence about the vm_flush, this is
> completely unrelated.
>
> Additional to that I still don't think that this is a good idea.
> Dependency handling is something completely driver specific.
>
> E.g. even when you have submitted jobs back to back they still might
> need a cache flush in between and that is not only for amdgpu like this.
>
> What you can do is to optimize for while looking at the fences later on
> and then note that you have done so and what the last hw fence is you
> used instead.
Out of 6 drivers using drm/sched 5 can use this. When we get i915
over, that one will be added to the list. amdgpu can't use any of this
anyway due to the vm_id allocation requirements, which is why I
mention that. Also note that all the callbacks are still there, so you
can just ignore this all and still build your own. Like amdgpu does.
So I'm not sure what exactly your object is, aside from "this doesn't
fit for amdgpu", which a) I know b) the commit message explains c)
doesn't actually hurt amdgpu in the slightest. And we still get the
benefit that for most drivers it's a nice optimization.
-Daniel
> Regards,
> Christian.
>
> >
> > Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
> > Acked-by: Melissa Wen <mwen@igalia.com>
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Cc: Luben Tuikov <luben.tuikov@amd.com>
> > Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > Cc: Alex Deucher <alexander.deucher@amd.com>
> > ---
> > drivers/gpu/drm/scheduler/sched_main.c | 7 +++++++
> > 1 file changed, 7 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> > index f77456929139..49e507f91ec0 100644
> > --- a/drivers/gpu/drm/scheduler/sched_main.c
> > +++ b/drivers/gpu/drm/scheduler/sched_main.c
> > @@ -660,6 +660,13 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
> > if (!fence)
> > return 0;
> >
> > + /* if it's a fence from us it's guaranteed to be earlier */
> > + if (fence->context == job->entity->fence_context ||
> > + fence->context == job->entity->fence_context + 1) {
> > + dma_fence_put(fence);
> > + return 0;
> > + }
> > +
> > /* Deduplicate if we already depend on a fence from the same context.
> > * This lets the size of the array of deps scale with the number of
> > * engines involved, rather than the number of BOs.
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 14/20] drm/sched: Don't store self-dependencies
@ 2021-08-05 13:25 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 13:25 UTC (permalink / raw)
To: Christian König
Cc: DRI Development, Intel Graphics Development, Lucas Stach,
Melissa Wen, Daniel Vetter, Luben Tuikov, Andrey Grodzovsky,
Alex Deucher
On Thu, Aug 5, 2021 at 3:18 PM Christian König <christian.koenig@amd.com> wrote:
>
>
>
> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> > This is essentially part of drm_sched_dependency_optimized(), which
> > only amdgpu seems to make use of. Use it a bit more.
> >
> > This would mean that as-is amdgpu can't use the dependency helpers, at
> > least not with the current approach amdgpu has for deciding whether a
> > vm_flush is needed. Since amdgpu also has very special rules around
> > implicit fencing it can't use those helpers either, and adding a
> > drm_sched_job_await_fence_always or similar for amdgpu wouldn't be too
> > onerous. That way the special case handling for amdgpu sticks even
> > more out and we have higher chances that reviewers that go across all
> > drivers wont miss it.
>
> Well you should probably drop the sentence about the vm_flush, this is
> completely unrelated.
>
> Additional to that I still don't think that this is a good idea.
> Dependency handling is something completely driver specific.
>
> E.g. even when you have submitted jobs back to back they still might
> need a cache flush in between and that is not only for amdgpu like this.
>
> What you can do is to optimize for while looking at the fences later on
> and then note that you have done so and what the last hw fence is you
> used instead.
Out of 6 drivers using drm/sched 5 can use this. When we get i915
over, that one will be added to the list. amdgpu can't use any of this
anyway due to the vm_id allocation requirements, which is why I
mention that. Also note that all the callbacks are still there, so you
can just ignore this all and still build your own. Like amdgpu does.
So I'm not sure what exactly your object is, aside from "this doesn't
fit for amdgpu", which a) I know b) the commit message explains c)
doesn't actually hurt amdgpu in the slightest. And we still get the
benefit that for most drivers it's a nice optimization.
-Daniel
> Regards,
> Christian.
>
> >
> > Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
> > Acked-by: Melissa Wen <mwen@igalia.com>
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Cc: Luben Tuikov <luben.tuikov@amd.com>
> > Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > Cc: Alex Deucher <alexander.deucher@amd.com>
> > ---
> > drivers/gpu/drm/scheduler/sched_main.c | 7 +++++++
> > 1 file changed, 7 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> > index f77456929139..49e507f91ec0 100644
> > --- a/drivers/gpu/drm/scheduler/sched_main.c
> > +++ b/drivers/gpu/drm/scheduler/sched_main.c
> > @@ -660,6 +660,13 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
> > if (!fence)
> > return 0;
> >
> > + /* if it's a fence from us it's guaranteed to be earlier */
> > + if (fence->context == job->entity->fence_context ||
> > + fence->context == job->entity->fence_context + 1) {
> > + dma_fence_put(fence);
> > + return 0;
> > + }
> > +
> > /* Deduplicate if we already depend on a fence from the same context.
> > * This lets the size of the array of deps scale with the number of
> > * engines involved, rather than the number of BOs.
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 14/20] drm/sched: Don't store self-dependencies
2021-08-05 13:25 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 13:57 ` Christian König
-1 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:57 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Lucas Stach,
Melissa Wen, Daniel Vetter, Luben Tuikov, Andrey Grodzovsky,
Alex Deucher
Am 05.08.21 um 15:25 schrieb Daniel Vetter:
> On Thu, Aug 5, 2021 at 3:18 PM Christian König <christian.koenig@amd.com> wrote:
>>
>>
>> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
>>> This is essentially part of drm_sched_dependency_optimized(), which
>>> only amdgpu seems to make use of. Use it a bit more.
>>>
>>> This would mean that as-is amdgpu can't use the dependency helpers, at
>>> least not with the current approach amdgpu has for deciding whether a
>>> vm_flush is needed. Since amdgpu also has very special rules around
>>> implicit fencing it can't use those helpers either, and adding a
>>> drm_sched_job_await_fence_always or similar for amdgpu wouldn't be too
>>> onerous. That way the special case handling for amdgpu sticks even
>>> more out and we have higher chances that reviewers that go across all
>>> drivers wont miss it.
>> Well you should probably drop the sentence about the vm_flush, this is
>> completely unrelated.
>>
>> Additional to that I still don't think that this is a good idea.
>> Dependency handling is something completely driver specific.
>>
>> E.g. even when you have submitted jobs back to back they still might
>> need a cache flush in between and that is not only for amdgpu like this.
>>
>> What you can do is to optimize for while looking at the fences later on
>> and then note that you have done so and what the last hw fence is you
>> used instead.
> Out of 6 drivers using drm/sched 5 can use this. When we get i915
> over, that one will be added to the list. amdgpu can't use any of this
> anyway due to the vm_id allocation requirements, which is why I
> mention that. Also note that all the callbacks are still there, so you
> can just ignore this all and still build your own. Like amdgpu does.
The VMID allocation stuff is rather easy to handle, that's why I noted
we should remove that sentence.
The problematic stuff is handling the cache flush and pipeline sync
which you make impossible with this here.
> So I'm not sure what exactly your object is, aside from "this doesn't
> fit for amdgpu", which a) I know b) the commit message explains c)
> doesn't actually hurt amdgpu in the slightest. And we still get the
> benefit that for most drivers it's a nice optimization.
Well exactly that's what I wanted to avoid. We still can use this in
amdgpu even with the VMID allocation stuff and I still hope to do so.
Can't we add this as a wrapper or similar?
Christian.
> -Daniel
>
>> Regards,
>> Christian.
>>
>>> Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
>>> Acked-by: Melissa Wen <mwen@igalia.com>
>>> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
>>> Cc: "Christian König" <christian.koenig@amd.com>
>>> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
>>> Cc: Luben Tuikov <luben.tuikov@amd.com>
>>> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>> Cc: Alex Deucher <alexander.deucher@amd.com>
>>> ---
>>> drivers/gpu/drm/scheduler/sched_main.c | 7 +++++++
>>> 1 file changed, 7 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
>>> index f77456929139..49e507f91ec0 100644
>>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>>> @@ -660,6 +660,13 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
>>> if (!fence)
>>> return 0;
>>>
>>> + /* if it's a fence from us it's guaranteed to be earlier */
>>> + if (fence->context == job->entity->fence_context ||
>>> + fence->context == job->entity->fence_context + 1) {
>>> + dma_fence_put(fence);
>>> + return 0;
>>> + }
>>> +
>>> /* Deduplicate if we already depend on a fence from the same context.
>>> * This lets the size of the array of deps scale with the number of
>>> * engines involved, rather than the number of BOs.
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 14/20] drm/sched: Don't store self-dependencies
@ 2021-08-05 13:57 ` Christian König
0 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:57 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Lucas Stach,
Melissa Wen, Daniel Vetter, Luben Tuikov, Andrey Grodzovsky,
Alex Deucher
Am 05.08.21 um 15:25 schrieb Daniel Vetter:
> On Thu, Aug 5, 2021 at 3:18 PM Christian König <christian.koenig@amd.com> wrote:
>>
>>
>> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
>>> This is essentially part of drm_sched_dependency_optimized(), which
>>> only amdgpu seems to make use of. Use it a bit more.
>>>
>>> This would mean that as-is amdgpu can't use the dependency helpers, at
>>> least not with the current approach amdgpu has for deciding whether a
>>> vm_flush is needed. Since amdgpu also has very special rules around
>>> implicit fencing it can't use those helpers either, and adding a
>>> drm_sched_job_await_fence_always or similar for amdgpu wouldn't be too
>>> onerous. That way the special case handling for amdgpu sticks even
>>> more out and we have higher chances that reviewers that go across all
>>> drivers wont miss it.
>> Well you should probably drop the sentence about the vm_flush, this is
>> completely unrelated.
>>
>> Additional to that I still don't think that this is a good idea.
>> Dependency handling is something completely driver specific.
>>
>> E.g. even when you have submitted jobs back to back they still might
>> need a cache flush in between and that is not only for amdgpu like this.
>>
>> What you can do is to optimize for while looking at the fences later on
>> and then note that you have done so and what the last hw fence is you
>> used instead.
> Out of 6 drivers using drm/sched 5 can use this. When we get i915
> over, that one will be added to the list. amdgpu can't use any of this
> anyway due to the vm_id allocation requirements, which is why I
> mention that. Also note that all the callbacks are still there, so you
> can just ignore this all and still build your own. Like amdgpu does.
The VMID allocation stuff is rather easy to handle, that's why I noted
we should remove that sentence.
The problematic stuff is handling the cache flush and pipeline sync
which you make impossible with this here.
> So I'm not sure what exactly your object is, aside from "this doesn't
> fit for amdgpu", which a) I know b) the commit message explains c)
> doesn't actually hurt amdgpu in the slightest. And we still get the
> benefit that for most drivers it's a nice optimization.
Well exactly that's what I wanted to avoid. We still can use this in
amdgpu even with the VMID allocation stuff and I still hope to do so.
Can't we add this as a wrapper or similar?
Christian.
> -Daniel
>
>> Regards,
>> Christian.
>>
>>> Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
>>> Acked-by: Melissa Wen <mwen@igalia.com>
>>> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
>>> Cc: "Christian König" <christian.koenig@amd.com>
>>> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
>>> Cc: Luben Tuikov <luben.tuikov@amd.com>
>>> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>> Cc: Alex Deucher <alexander.deucher@amd.com>
>>> ---
>>> drivers/gpu/drm/scheduler/sched_main.c | 7 +++++++
>>> 1 file changed, 7 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
>>> index f77456929139..49e507f91ec0 100644
>>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>>> @@ -660,6 +660,13 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
>>> if (!fence)
>>> return 0;
>>>
>>> + /* if it's a fence from us it's guaranteed to be earlier */
>>> + if (fence->context == job->entity->fence_context ||
>>> + fence->context == job->entity->fence_context + 1) {
>>> + dma_fence_put(fence);
>>> + return 0;
>>> + }
>>> +
>>> /* Deduplicate if we already depend on a fence from the same context.
>>> * This lets the size of the array of deps scale with the number of
>>> * engines involved, rather than the number of BOs.
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 14/20] drm/sched: Don't store self-dependencies
2021-08-05 13:57 ` [Intel-gfx] " Christian König
@ 2021-08-05 15:06 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 15:06 UTC (permalink / raw)
To: Christian König
Cc: DRI Development, Intel Graphics Development, Lucas Stach,
Melissa Wen, Daniel Vetter, Luben Tuikov, Andrey Grodzovsky,
Alex Deucher
On Thu, Aug 5, 2021 at 3:57 PM Christian König <christian.koenig@amd.com> wrote:
> Am 05.08.21 um 15:25 schrieb Daniel Vetter:
> > On Thu, Aug 5, 2021 at 3:18 PM Christian König <christian.koenig@amd.com> wrote:
> >>
> >>
> >> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> >>> This is essentially part of drm_sched_dependency_optimized(), which
> >>> only amdgpu seems to make use of. Use it a bit more.
> >>>
> >>> This would mean that as-is amdgpu can't use the dependency helpers, at
> >>> least not with the current approach amdgpu has for deciding whether a
> >>> vm_flush is needed. Since amdgpu also has very special rules around
> >>> implicit fencing it can't use those helpers either, and adding a
> >>> drm_sched_job_await_fence_always or similar for amdgpu wouldn't be too
> >>> onerous. That way the special case handling for amdgpu sticks even
> >>> more out and we have higher chances that reviewers that go across all
> >>> drivers wont miss it.
> >> Well you should probably drop the sentence about the vm_flush, this is
> >> completely unrelated.
> >>
> >> Additional to that I still don't think that this is a good idea.
> >> Dependency handling is something completely driver specific.
> >>
> >> E.g. even when you have submitted jobs back to back they still might
> >> need a cache flush in between and that is not only for amdgpu like this.
> >>
> >> What you can do is to optimize for while looking at the fences later on
> >> and then note that you have done so and what the last hw fence is you
> >> used instead.
> > Out of 6 drivers using drm/sched 5 can use this. When we get i915
> > over, that one will be added to the list. amdgpu can't use any of this
> > anyway due to the vm_id allocation requirements, which is why I
> > mention that. Also note that all the callbacks are still there, so you
> > can just ignore this all and still build your own. Like amdgpu does.
>
> The VMID allocation stuff is rather easy to handle, that's why I noted
> we should remove that sentence.
>
> The problematic stuff is handling the cache flush and pipeline sync
> which you make impossible with this here.
Well the vmid is tied to the flush, but yeah the commit message
doesn't make this clear.
> > So I'm not sure what exactly your object is, aside from "this doesn't
> > fit for amdgpu", which a) I know b) the commit message explains c)
> > doesn't actually hurt amdgpu in the slightest. And we still get the
> > benefit that for most drivers it's a nice optimization.
>
> Well exactly that's what I wanted to avoid. We still can use this in
> amdgpu even with the VMID allocation stuff and I still hope to do so.
>
> Can't we add this as a wrapper or similar?
This patch is not the only thing that will prevent you from using
these helpers, because amdgpu also needs to keep track of all the
fences in the xarray, which these helpers don't - they get cleared out
as we hand them off to the scheduler. So it's more surgery than just
not having this, and I'm honestly not sure it's worth it since you'd
need to duplicate quite a bit more than just the functions to add
dependencies.
-Daniel
-Daniel
> Christian.
>
> > -Daniel
> >
> >> Regards,
> >> Christian.
> >>
> >>> Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
> >>> Acked-by: Melissa Wen <mwen@igalia.com>
> >>> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> >>> Cc: "Christian König" <christian.koenig@amd.com>
> >>> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> >>> Cc: Luben Tuikov <luben.tuikov@amd.com>
> >>> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> >>> Cc: Alex Deucher <alexander.deucher@amd.com>
> >>> ---
> >>> drivers/gpu/drm/scheduler/sched_main.c | 7 +++++++
> >>> 1 file changed, 7 insertions(+)
> >>>
> >>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> >>> index f77456929139..49e507f91ec0 100644
> >>> --- a/drivers/gpu/drm/scheduler/sched_main.c
> >>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> >>> @@ -660,6 +660,13 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
> >>> if (!fence)
> >>> return 0;
> >>>
> >>> + /* if it's a fence from us it's guaranteed to be earlier */
> >>> + if (fence->context == job->entity->fence_context ||
> >>> + fence->context == job->entity->fence_context + 1) {
> >>> + dma_fence_put(fence);
> >>> + return 0;
> >>> + }
> >>> +
> >>> /* Deduplicate if we already depend on a fence from the same context.
> >>> * This lets the size of the array of deps scale with the number of
> >>> * engines involved, rather than the number of BOs.
> >
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 14/20] drm/sched: Don't store self-dependencies
@ 2021-08-05 15:06 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 15:06 UTC (permalink / raw)
To: Christian König
Cc: DRI Development, Intel Graphics Development, Lucas Stach,
Melissa Wen, Daniel Vetter, Luben Tuikov, Andrey Grodzovsky,
Alex Deucher
On Thu, Aug 5, 2021 at 3:57 PM Christian König <christian.koenig@amd.com> wrote:
> Am 05.08.21 um 15:25 schrieb Daniel Vetter:
> > On Thu, Aug 5, 2021 at 3:18 PM Christian König <christian.koenig@amd.com> wrote:
> >>
> >>
> >> Am 05.08.21 um 12:46 schrieb Daniel Vetter:
> >>> This is essentially part of drm_sched_dependency_optimized(), which
> >>> only amdgpu seems to make use of. Use it a bit more.
> >>>
> >>> This would mean that as-is amdgpu can't use the dependency helpers, at
> >>> least not with the current approach amdgpu has for deciding whether a
> >>> vm_flush is needed. Since amdgpu also has very special rules around
> >>> implicit fencing it can't use those helpers either, and adding a
> >>> drm_sched_job_await_fence_always or similar for amdgpu wouldn't be too
> >>> onerous. That way the special case handling for amdgpu sticks even
> >>> more out and we have higher chances that reviewers that go across all
> >>> drivers wont miss it.
> >> Well you should probably drop the sentence about the vm_flush, this is
> >> completely unrelated.
> >>
> >> Additional to that I still don't think that this is a good idea.
> >> Dependency handling is something completely driver specific.
> >>
> >> E.g. even when you have submitted jobs back to back they still might
> >> need a cache flush in between and that is not only for amdgpu like this.
> >>
> >> What you can do is to optimize for while looking at the fences later on
> >> and then note that you have done so and what the last hw fence is you
> >> used instead.
> > Out of 6 drivers using drm/sched 5 can use this. When we get i915
> > over, that one will be added to the list. amdgpu can't use any of this
> > anyway due to the vm_id allocation requirements, which is why I
> > mention that. Also note that all the callbacks are still there, so you
> > can just ignore this all and still build your own. Like amdgpu does.
>
> The VMID allocation stuff is rather easy to handle, that's why I noted
> we should remove that sentence.
>
> The problematic stuff is handling the cache flush and pipeline sync
> which you make impossible with this here.
Well the vmid is tied to the flush, but yeah the commit message
doesn't make this clear.
> > So I'm not sure what exactly your object is, aside from "this doesn't
> > fit for amdgpu", which a) I know b) the commit message explains c)
> > doesn't actually hurt amdgpu in the slightest. And we still get the
> > benefit that for most drivers it's a nice optimization.
>
> Well exactly that's what I wanted to avoid. We still can use this in
> amdgpu even with the VMID allocation stuff and I still hope to do so.
>
> Can't we add this as a wrapper or similar?
This patch is not the only thing that will prevent you from using
these helpers, because amdgpu also needs to keep track of all the
fences in the xarray, which these helpers don't - they get cleared out
as we hand them off to the scheduler. So it's more surgery than just
not having this, and I'm honestly not sure it's worth it since you'd
need to duplicate quite a bit more than just the functions to add
dependencies.
-Daniel
-Daniel
> Christian.
>
> > -Daniel
> >
> >> Regards,
> >> Christian.
> >>
> >>> Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
> >>> Acked-by: Melissa Wen <mwen@igalia.com>
> >>> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> >>> Cc: "Christian König" <christian.koenig@amd.com>
> >>> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> >>> Cc: Luben Tuikov <luben.tuikov@amd.com>
> >>> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> >>> Cc: Alex Deucher <alexander.deucher@amd.com>
> >>> ---
> >>> drivers/gpu/drm/scheduler/sched_main.c | 7 +++++++
> >>> 1 file changed, 7 insertions(+)
> >>>
> >>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> >>> index f77456929139..49e507f91ec0 100644
> >>> --- a/drivers/gpu/drm/scheduler/sched_main.c
> >>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> >>> @@ -660,6 +660,13 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job,
> >>> if (!fence)
> >>> return 0;
> >>>
> >>> + /* if it's a fence from us it's guaranteed to be earlier */
> >>> + if (fence->context == job->entity->fence_context ||
> >>> + fence->context == job->entity->fence_context + 1) {
> >>> + dma_fence_put(fence);
> >>> + return 0;
> >>> + }
> >>> +
> >>> /* Deduplicate if we already depend on a fence from the same context.
> >>> * This lets the size of the array of deps scale with the number of
> >>> * engines involved, rather than the number of BOs.
> >
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 15/20] drm/sched: Check locking in drm_sched_job_await_implicit
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:47 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Daniel Vetter, Christian König, Luben Tuikov,
Andrey Grodzovsky, Alex Deucher
You really need to hold the reservation here or all kinds of funny
things can happen between grabbing the dependencies and inserting the
new fences.
Acked-by: Melissa Wen <mwen@igalia.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
---
drivers/gpu/drm/scheduler/sched_main.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index 49e507f91ec0..1abb40b07324 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -715,6 +715,8 @@ int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
struct dma_fence **fences;
unsigned int i, fence_count;
+ dma_resv_assert_held(obj->resv);
+
if (!write) {
struct dma_fence *fence = dma_resv_get_excl_unlocked(obj->resv);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 15/20] drm/sched: Check locking in drm_sched_job_await_implicit
@ 2021-08-05 10:47 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Melissa Wen,
Daniel Vetter, Christian König, Luben Tuikov,
Andrey Grodzovsky, Alex Deucher
You really need to hold the reservation here or all kinds of funny
things can happen between grabbing the dependencies and inserting the
new fences.
Acked-by: Melissa Wen <mwen@igalia.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
---
drivers/gpu/drm/scheduler/sched_main.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index 49e507f91ec0..1abb40b07324 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -715,6 +715,8 @@ int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
struct dma_fence **fences;
unsigned int i, fence_count;
+ dma_resv_assert_held(obj->resv);
+
if (!write) {
struct dma_fence *fence = dma_resv_get_excl_unlocked(obj->resv);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 15/20] drm/sched: Check locking in drm_sched_job_await_implicit
2021-08-05 10:47 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 13:19 ` Christian König
-1 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:19 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Melissa Wen, Daniel Vetter,
Luben Tuikov, Andrey Grodzovsky, Alex Deucher
Am 05.08.21 um 12:47 schrieb Daniel Vetter:
> You really need to hold the reservation here or all kinds of funny
> things can happen between grabbing the dependencies and inserting the
> new fences.
>
> Acked-by: Melissa Wen <mwen@igalia.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
The function name in the subject line should be updated, apart from that
feel free to add my rb to this patch.
Christian.
> ---
> drivers/gpu/drm/scheduler/sched_main.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> index 49e507f91ec0..1abb40b07324 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -715,6 +715,8 @@ int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
> struct dma_fence **fences;
> unsigned int i, fence_count;
>
> + dma_resv_assert_held(obj->resv);
> +
> if (!write) {
> struct dma_fence *fence = dma_resv_get_excl_unlocked(obj->resv);
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 15/20] drm/sched: Check locking in drm_sched_job_await_implicit
@ 2021-08-05 13:19 ` Christian König
0 siblings, 0 replies; 122+ messages in thread
From: Christian König @ 2021-08-05 13:19 UTC (permalink / raw)
To: Daniel Vetter, DRI Development
Cc: Intel Graphics Development, Melissa Wen, Daniel Vetter,
Luben Tuikov, Andrey Grodzovsky, Alex Deucher
Am 05.08.21 um 12:47 schrieb Daniel Vetter:
> You really need to hold the reservation here or all kinds of funny
> things can happen between grabbing the dependencies and inserting the
> new fences.
>
> Acked-by: Melissa Wen <mwen@igalia.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
The function name in the subject line should be updated, apart from that
feel free to add my rb to this patch.
Christian.
> ---
> drivers/gpu/drm/scheduler/sched_main.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> index 49e507f91ec0..1abb40b07324 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -715,6 +715,8 @@ int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
> struct dma_fence **fences;
> unsigned int i, fence_count;
>
> + dma_resv_assert_held(obj->resv);
> +
> if (!write) {
> struct dma_fence *fence = dma_resv_get_excl_unlocked(obj->resv);
>
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [PATCH v5 15/20] drm/sched: Check locking in drm_sched_job_await_implicit
2021-08-05 13:19 ` [Intel-gfx] " Christian König
@ 2021-08-05 13:27 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 13:27 UTC (permalink / raw)
To: Christian König
Cc: DRI Development, Intel Graphics Development, Melissa Wen,
Daniel Vetter, Luben Tuikov, Andrey Grodzovsky, Alex Deucher
On Thu, Aug 5, 2021 at 3:19 PM Christian König <christian.koenig@amd.com> wrote:
>
> Am 05.08.21 um 12:47 schrieb Daniel Vetter:
> > You really need to hold the reservation here or all kinds of funny
> > things can happen between grabbing the dependencies and inserting the
> > new fences.
> >
> > Acked-by: Melissa Wen <mwen@igalia.com>
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Cc: Luben Tuikov <luben.tuikov@amd.com>
> > Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > Cc: Alex Deucher <alexander.deucher@amd.com>
>
> The function name in the subject line should be updated, apart from that
> feel free to add my rb to this patch.
Fixed locally and r-b added, I think the later parts of this series
will need to be resent anyway. Thanks for your review.
-Daniel
>
> Christian.
>
> > ---
> > drivers/gpu/drm/scheduler/sched_main.c | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> > index 49e507f91ec0..1abb40b07324 100644
> > --- a/drivers/gpu/drm/scheduler/sched_main.c
> > +++ b/drivers/gpu/drm/scheduler/sched_main.c
> > @@ -715,6 +715,8 @@ int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
> > struct dma_fence **fences;
> > unsigned int i, fence_count;
> >
> > + dma_resv_assert_held(obj->resv);
> > +
> > if (!write) {
> > struct dma_fence *fence = dma_resv_get_excl_unlocked(obj->resv);
> >
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 15/20] drm/sched: Check locking in drm_sched_job_await_implicit
@ 2021-08-05 13:27 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 13:27 UTC (permalink / raw)
To: Christian König
Cc: DRI Development, Intel Graphics Development, Melissa Wen,
Daniel Vetter, Luben Tuikov, Andrey Grodzovsky, Alex Deucher
On Thu, Aug 5, 2021 at 3:19 PM Christian König <christian.koenig@amd.com> wrote:
>
> Am 05.08.21 um 12:47 schrieb Daniel Vetter:
> > You really need to hold the reservation here or all kinds of funny
> > things can happen between grabbing the dependencies and inserting the
> > new fences.
> >
> > Acked-by: Melissa Wen <mwen@igalia.com>
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: "Christian König" <christian.koenig@amd.com>
> > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Cc: Luben Tuikov <luben.tuikov@amd.com>
> > Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > Cc: Alex Deucher <alexander.deucher@amd.com>
>
> The function name in the subject line should be updated, apart from that
> feel free to add my rb to this patch.
Fixed locally and r-b added, I think the later parts of this series
will need to be resent anyway. Thanks for your review.
-Daniel
>
> Christian.
>
> > ---
> > drivers/gpu/drm/scheduler/sched_main.c | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> > index 49e507f91ec0..1abb40b07324 100644
> > --- a/drivers/gpu/drm/scheduler/sched_main.c
> > +++ b/drivers/gpu/drm/scheduler/sched_main.c
> > @@ -715,6 +715,8 @@ int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,
> > struct dma_fence **fences;
> > unsigned int i, fence_count;
> >
> > + dma_resv_assert_held(obj->resv);
> > +
> > if (!write) {
> > struct dma_fence *fence = dma_resv_get_excl_unlocked(obj->resv);
> >
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 16/20] drm/msm: Don't break exclusive fence ordering
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:47 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Lucas Stach,
Daniel Vetter, Rob Clark, Sean Paul, linux-arm-msm, freedreno
There's only one exclusive slot, and we must not break the ordering.
Adding a new exclusive fence drops all previous fences from the
dma_resv. To avoid violating the signalling order we err on the side of
over-synchronizing by waiting for the existing fences, even if
userspace asked us to ignore them.
A better fix would be to us a dma_fence_chain or _array like e.g.
amdgpu now uses, but
- msm has a synchronous dma_fence_wait for anything from another
context, so doesn't seem to care much,
- and it probably makes sense to lift this into dma-resv.c code as a
proper concept, so that drivers don't have to hack up their own
solution each on their own.
v2: Improve commit message per Lucas' suggestion.
Cc: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
---
drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index fb5a2eab27a2..66633dfd58a2 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -330,7 +330,8 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
return ret;
}
- if (no_implicit)
+ /* exclusive fences must be ordered */
+ if (no_implicit && !write)
continue;
ret = drm_sched_job_add_implicit_dependencies(&submit->base,
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* [Intel-gfx] [PATCH v5 16/20] drm/msm: Don't break exclusive fence ordering
@ 2021-08-05 10:47 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Lucas Stach,
Daniel Vetter, Rob Clark, Sean Paul, linux-arm-msm, freedreno
There's only one exclusive slot, and we must not break the ordering.
Adding a new exclusive fence drops all previous fences from the
dma_resv. To avoid violating the signalling order we err on the side of
over-synchronizing by waiting for the existing fences, even if
userspace asked us to ignore them.
A better fix would be to us a dma_fence_chain or _array like e.g.
amdgpu now uses, but
- msm has a synchronous dma_fence_wait for anything from another
context, so doesn't seem to care much,
- and it probably makes sense to lift this into dma-resv.c code as a
proper concept, so that drivers don't have to hack up their own
solution each on their own.
v2: Improve commit message per Lucas' suggestion.
Cc: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
---
drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index fb5a2eab27a2..66633dfd58a2 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -330,7 +330,8 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
return ret;
}
- if (no_implicit)
+ /* exclusive fences must be ordered */
+ if (no_implicit && !write)
continue;
ret = drm_sched_job_add_implicit_dependencies(&submit->base,
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* Re: [PATCH v5 16/20] drm/msm: Don't break exclusive fence ordering
2021-08-05 10:47 ` [Intel-gfx] " Daniel Vetter
(?)
@ 2021-08-26 16:16 ` Rob Clark
-1 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-26 16:16 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Lucas Stach,
Daniel Vetter, Sean Paul, linux-arm-msm, freedreno
On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> There's only one exclusive slot, and we must not break the ordering.
>
> Adding a new exclusive fence drops all previous fences from the
> dma_resv. To avoid violating the signalling order we err on the side of
> over-synchronizing by waiting for the existing fences, even if
> userspace asked us to ignore them.
>
> A better fix would be to us a dma_fence_chain or _array like e.g.
> amdgpu now uses, but
> - msm has a synchronous dma_fence_wait for anything from another
> context, so doesn't seem to care much,
> - and it probably makes sense to lift this into dma-resv.c code as a
> proper concept, so that drivers don't have to hack up their own
> solution each on their own.
>
> v2: Improve commit message per Lucas' suggestion.
>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
a-b
> ---
> drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index fb5a2eab27a2..66633dfd58a2 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -330,7 +330,8 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> return ret;
> }
>
> - if (no_implicit)
> + /* exclusive fences must be ordered */
> + if (no_implicit && !write)
> continue;
>
> ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [Intel-gfx] [PATCH v5 16/20] drm/msm: Don't break exclusive fence ordering
@ 2021-08-26 16:16 ` Rob Clark
0 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-26 16:16 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Lucas Stach,
Daniel Vetter, Sean Paul, linux-arm-msm, freedreno
On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> There's only one exclusive slot, and we must not break the ordering.
>
> Adding a new exclusive fence drops all previous fences from the
> dma_resv. To avoid violating the signalling order we err on the side of
> over-synchronizing by waiting for the existing fences, even if
> userspace asked us to ignore them.
>
> A better fix would be to us a dma_fence_chain or _array like e.g.
> amdgpu now uses, but
> - msm has a synchronous dma_fence_wait for anything from another
> context, so doesn't seem to care much,
> - and it probably makes sense to lift this into dma-resv.c code as a
> proper concept, so that drivers don't have to hack up their own
> solution each on their own.
>
> v2: Improve commit message per Lucas' suggestion.
>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
a-b
> ---
> drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index fb5a2eab27a2..66633dfd58a2 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -330,7 +330,8 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> return ret;
> }
>
> - if (no_implicit)
> + /* exclusive fences must be ordered */
> + if (no_implicit && !write)
> continue;
>
> ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [PATCH v5 16/20] drm/msm: Don't break exclusive fence ordering
@ 2021-08-26 16:16 ` Rob Clark
0 siblings, 0 replies; 122+ messages in thread
From: Rob Clark @ 2021-08-26 16:16 UTC (permalink / raw)
To: Daniel Vetter
Cc: DRI Development, Intel Graphics Development, Lucas Stach,
Daniel Vetter, Sean Paul, linux-arm-msm, freedreno
On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> There's only one exclusive slot, and we must not break the ordering.
>
> Adding a new exclusive fence drops all previous fences from the
> dma_resv. To avoid violating the signalling order we err on the side of
> over-synchronizing by waiting for the existing fences, even if
> userspace asked us to ignore them.
>
> A better fix would be to us a dma_fence_chain or _array like e.g.
> amdgpu now uses, but
> - msm has a synchronous dma_fence_wait for anything from another
> context, so doesn't seem to care much,
> - and it probably makes sense to lift this into dma-resv.c code as a
> proper concept, so that drivers don't have to hack up their own
> solution each on their own.
>
> v2: Improve commit message per Lucas' suggestion.
>
> Cc: Lucas Stach <l.stach@pengutronix.de>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Rob Clark <robdclark@gmail.com>
> Cc: Sean Paul <sean@poorly.run>
> Cc: linux-arm-msm@vger.kernel.org
> Cc: freedreno@lists.freedesktop.org
a-b
> ---
> drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index fb5a2eab27a2..66633dfd58a2 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -330,7 +330,8 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> return ret;
> }
>
> - if (no_implicit)
> + /* exclusive fences must be ordered */
> + if (no_implicit && !write)
> continue;
>
> ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [PATCH v5 16/20] drm/msm: Don't break exclusive fence ordering
2021-08-26 16:16 ` Rob Clark
@ 2021-08-30 9:02 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-30 9:02 UTC (permalink / raw)
To: Rob Clark
Cc: Daniel Vetter, DRI Development, Intel Graphics Development,
Lucas Stach, Daniel Vetter, Sean Paul, linux-arm-msm, freedreno
On Thu, Aug 26, 2021 at 09:16:25AM -0700, Rob Clark wrote:
> On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > There's only one exclusive slot, and we must not break the ordering.
> >
> > Adding a new exclusive fence drops all previous fences from the
> > dma_resv. To avoid violating the signalling order we err on the side of
> > over-synchronizing by waiting for the existing fences, even if
> > userspace asked us to ignore them.
> >
> > A better fix would be to us a dma_fence_chain or _array like e.g.
> > amdgpu now uses, but
> > - msm has a synchronous dma_fence_wait for anything from another
> > context, so doesn't seem to care much,
> > - and it probably makes sense to lift this into dma-resv.c code as a
> > proper concept, so that drivers don't have to hack up their own
> > solution each on their own.
> >
> > v2: Improve commit message per Lucas' suggestion.
> >
> > Cc: Lucas Stach <l.stach@pengutronix.de>
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: Rob Clark <robdclark@gmail.com>
> > Cc: Sean Paul <sean@poorly.run>
> > Cc: linux-arm-msm@vger.kernel.org
> > Cc: freedreno@lists.freedesktop.org
>
> a-b
Also pushed to drm-misc-next, thanks for review&testing.
-Daniel
>
> > ---
> > drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > index fb5a2eab27a2..66633dfd58a2 100644
> > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > @@ -330,7 +330,8 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> > return ret;
> > }
> >
> > - if (no_implicit)
> > + /* exclusive fences must be ordered */
> > + if (no_implicit && !write)
> > continue;
> >
> > ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> > --
> > 2.32.0
> >
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* Re: [Intel-gfx] [PATCH v5 16/20] drm/msm: Don't break exclusive fence ordering
@ 2021-08-30 9:02 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-30 9:02 UTC (permalink / raw)
To: Rob Clark
Cc: Daniel Vetter, DRI Development, Intel Graphics Development,
Lucas Stach, Daniel Vetter, Sean Paul, linux-arm-msm, freedreno
On Thu, Aug 26, 2021 at 09:16:25AM -0700, Rob Clark wrote:
> On Thu, Aug 5, 2021 at 3:47 AM Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >
> > There's only one exclusive slot, and we must not break the ordering.
> >
> > Adding a new exclusive fence drops all previous fences from the
> > dma_resv. To avoid violating the signalling order we err on the side of
> > over-synchronizing by waiting for the existing fences, even if
> > userspace asked us to ignore them.
> >
> > A better fix would be to us a dma_fence_chain or _array like e.g.
> > amdgpu now uses, but
> > - msm has a synchronous dma_fence_wait for anything from another
> > context, so doesn't seem to care much,
> > - and it probably makes sense to lift this into dma-resv.c code as a
> > proper concept, so that drivers don't have to hack up their own
> > solution each on their own.
> >
> > v2: Improve commit message per Lucas' suggestion.
> >
> > Cc: Lucas Stach <l.stach@pengutronix.de>
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > Cc: Rob Clark <robdclark@gmail.com>
> > Cc: Sean Paul <sean@poorly.run>
> > Cc: linux-arm-msm@vger.kernel.org
> > Cc: freedreno@lists.freedesktop.org
>
> a-b
Also pushed to drm-misc-next, thanks for review&testing.
-Daniel
>
> > ---
> > drivers/gpu/drm/msm/msm_gem_submit.c | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > index fb5a2eab27a2..66633dfd58a2 100644
> > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > @@ -330,7 +330,8 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)
> > return ret;
> > }
> >
> > - if (no_implicit)
> > + /* exclusive fences must be ordered */
> > + if (no_implicit && !write)
> > continue;
> >
> > ret = drm_sched_job_add_implicit_dependencies(&submit->base,
> > --
> > 2.32.0
> >
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* [PATCH v5 17/20] drm/etnaviv: Don't break exclusive fence ordering
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:47 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Lucas Stach, Russell King, Christian Gmeiner, etnaviv
There's only one exclusive slot, and we must not break the ordering.
Adding a new exclusive fence drops all previous fences from the
dma_resv. To avoid violating the signalling order we err on the side of
over-synchronizing by waiting for the existing fences, even if
userspace asked us to ignore them.
A better fix would be to us a dma_fence_chain or _array like e.g.
amdgpu now uses, but it probably makes sense to lift this into
dma-resv.c code as a proper concept, so that drivers don't have to
hack up their own solution each on their own. Hence go with the simple
fix for now.
Another option is the fence import ioctl from Jason:
https://lore.kernel.org/dri-devel/20210610210925.642582-7-jason@jlekstrand.net/
v2: Improve commit message per Lucas' suggestion.
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: etnaviv@lists.freedesktop.org
---
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
index e3d43678eb09..8d1703da971a 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
@@ -178,19 +178,21 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)
for (i = 0; i < submit->nr_bos; i++) {
struct etnaviv_gem_submit_bo *bo = &submit->bos[i];
struct dma_resv *robj = bo->obj->base.resv;
+ bool write = bo->flags & ETNA_SUBMIT_BO_WRITE;
- if (!(bo->flags & ETNA_SUBMIT_BO_WRITE)) {
+ if (!(write)) {
ret = dma_resv_reserve_shared(robj, 1);
if (ret)
return ret;
}
- if (submit->flags & ETNA_SUBMIT_NO_IMPLICIT)
+ /* exclusive fences must be ordered */
+ if (submit->flags & ETNA_SUBMIT_NO_IMPLICIT && !write)
continue;
ret = drm_sched_job_add_implicit_dependencies(&submit->sched_job,
&bo->obj->base,
- bo->flags & ETNA_SUBMIT_BO_WRITE);
+ write);
if (ret)
return ret;
}
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 17/20] drm/etnaviv: Don't break exclusive fence ordering
@ 2021-08-05 10:47 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Lucas Stach, Russell King, Christian Gmeiner, etnaviv
There's only one exclusive slot, and we must not break the ordering.
Adding a new exclusive fence drops all previous fences from the
dma_resv. To avoid violating the signalling order we err on the side of
over-synchronizing by waiting for the existing fences, even if
userspace asked us to ignore them.
A better fix would be to us a dma_fence_chain or _array like e.g.
amdgpu now uses, but it probably makes sense to lift this into
dma-resv.c code as a proper concept, so that drivers don't have to
hack up their own solution each on their own. Hence go with the simple
fix for now.
Another option is the fence import ioctl from Jason:
https://lore.kernel.org/dri-devel/20210610210925.642582-7-jason@jlekstrand.net/
v2: Improve commit message per Lucas' suggestion.
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: etnaviv@lists.freedesktop.org
---
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
index e3d43678eb09..8d1703da971a 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
@@ -178,19 +178,21 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)
for (i = 0; i < submit->nr_bos; i++) {
struct etnaviv_gem_submit_bo *bo = &submit->bos[i];
struct dma_resv *robj = bo->obj->base.resv;
+ bool write = bo->flags & ETNA_SUBMIT_BO_WRITE;
- if (!(bo->flags & ETNA_SUBMIT_BO_WRITE)) {
+ if (!(write)) {
ret = dma_resv_reserve_shared(robj, 1);
if (ret)
return ret;
}
- if (submit->flags & ETNA_SUBMIT_NO_IMPLICIT)
+ /* exclusive fences must be ordered */
+ if (submit->flags & ETNA_SUBMIT_NO_IMPLICIT && !write)
continue;
ret = drm_sched_job_add_implicit_dependencies(&submit->sched_job,
&bo->obj->base,
- bo->flags & ETNA_SUBMIT_BO_WRITE);
+ write);
if (ret)
return ret;
}
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* [PATCH v5 18/20] drm/i915: delete exclude argument from i915_sw_fence_await_reservation
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:47 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Maarten Lankhorst, Thomas Hellström, Jason Ekstrand
No longer used, the last user disappeared with
commit d07f0e59b2c762584478920cd2d11fba2980a94a
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date: Fri Oct 28 13:58:44 2016 +0100
drm/i915: Move GEM activity tracking into a common struct reservation_object
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: "Thomas Hellström" <thomas.hellstrom@linux.intel.com>
Cc: Jason Ekstrand <jason@jlekstrand.net>
---
drivers/gpu/drm/i915/display/intel_display.c | 4 ++--
drivers/gpu/drm/i915/gem/i915_gem_clflush.c | 2 +-
drivers/gpu/drm/i915/i915_sw_fence.c | 6 +-----
drivers/gpu/drm/i915/i915_sw_fence.h | 1 -
4 files changed, 4 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/i915/display/intel_display.c b/drivers/gpu/drm/i915/display/intel_display.c
index 86b86deca701..0ec736026132 100644
--- a/drivers/gpu/drm/i915/display/intel_display.c
+++ b/drivers/gpu/drm/i915/display/intel_display.c
@@ -11248,7 +11248,7 @@ intel_prepare_plane_fb(struct drm_plane *_plane,
*/
if (intel_crtc_needs_modeset(crtc_state)) {
ret = i915_sw_fence_await_reservation(&state->commit_ready,
- old_obj->base.resv, NULL,
+ old_obj->base.resv,
false, 0,
GFP_KERNEL);
if (ret < 0)
@@ -11282,7 +11282,7 @@ intel_prepare_plane_fb(struct drm_plane *_plane,
struct dma_fence *fence;
ret = i915_sw_fence_await_reservation(&state->commit_ready,
- obj->base.resv, NULL,
+ obj->base.resv,
false,
i915_fence_timeout(dev_priv),
GFP_KERNEL);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_clflush.c b/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
index f0435c6feb68..fde88fa90780 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
@@ -104,7 +104,7 @@ bool i915_gem_clflush_object(struct drm_i915_gem_object *obj,
clflush = clflush_work_create(obj);
if (clflush) {
i915_sw_fence_await_reservation(&clflush->base.chain,
- obj->base.resv, NULL, true,
+ obj->base.resv, true,
i915_fence_timeout(to_i915(obj->base.dev)),
I915_FENCE_GFP);
dma_resv_add_excl_fence(obj->base.resv, &clflush->base.dma);
diff --git a/drivers/gpu/drm/i915/i915_sw_fence.c b/drivers/gpu/drm/i915/i915_sw_fence.c
index c589a681da77..91711a46b1c7 100644
--- a/drivers/gpu/drm/i915/i915_sw_fence.c
+++ b/drivers/gpu/drm/i915/i915_sw_fence.c
@@ -567,7 +567,6 @@ int __i915_sw_fence_await_dma_fence(struct i915_sw_fence *fence,
int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
struct dma_resv *resv,
- const struct dma_fence_ops *exclude,
bool write,
unsigned long timeout,
gfp_t gfp)
@@ -587,9 +586,6 @@ int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
return ret;
for (i = 0; i < count; i++) {
- if (shared[i]->ops == exclude)
- continue;
-
pending = i915_sw_fence_await_dma_fence(fence,
shared[i],
timeout,
@@ -609,7 +605,7 @@ int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
excl = dma_resv_get_excl_unlocked(resv);
}
- if (ret >= 0 && excl && excl->ops != exclude) {
+ if (ret >= 0 && excl) {
pending = i915_sw_fence_await_dma_fence(fence,
excl,
timeout,
diff --git a/drivers/gpu/drm/i915/i915_sw_fence.h b/drivers/gpu/drm/i915/i915_sw_fence.h
index 30a863353ee6..6572f01668e4 100644
--- a/drivers/gpu/drm/i915/i915_sw_fence.h
+++ b/drivers/gpu/drm/i915/i915_sw_fence.h
@@ -86,7 +86,6 @@ int i915_sw_fence_await_dma_fence(struct i915_sw_fence *fence,
int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
struct dma_resv *resv,
- const struct dma_fence_ops *exclude,
bool write,
unsigned long timeout,
gfp_t gfp);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 18/20] drm/i915: delete exclude argument from i915_sw_fence_await_reservation
@ 2021-08-05 10:47 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Daniel Vetter,
Maarten Lankhorst, Thomas Hellström, Jason Ekstrand
No longer used, the last user disappeared with
commit d07f0e59b2c762584478920cd2d11fba2980a94a
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date: Fri Oct 28 13:58:44 2016 +0100
drm/i915: Move GEM activity tracking into a common struct reservation_object
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: "Thomas Hellström" <thomas.hellstrom@linux.intel.com>
Cc: Jason Ekstrand <jason@jlekstrand.net>
---
drivers/gpu/drm/i915/display/intel_display.c | 4 ++--
drivers/gpu/drm/i915/gem/i915_gem_clflush.c | 2 +-
drivers/gpu/drm/i915/i915_sw_fence.c | 6 +-----
drivers/gpu/drm/i915/i915_sw_fence.h | 1 -
4 files changed, 4 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/i915/display/intel_display.c b/drivers/gpu/drm/i915/display/intel_display.c
index 86b86deca701..0ec736026132 100644
--- a/drivers/gpu/drm/i915/display/intel_display.c
+++ b/drivers/gpu/drm/i915/display/intel_display.c
@@ -11248,7 +11248,7 @@ intel_prepare_plane_fb(struct drm_plane *_plane,
*/
if (intel_crtc_needs_modeset(crtc_state)) {
ret = i915_sw_fence_await_reservation(&state->commit_ready,
- old_obj->base.resv, NULL,
+ old_obj->base.resv,
false, 0,
GFP_KERNEL);
if (ret < 0)
@@ -11282,7 +11282,7 @@ intel_prepare_plane_fb(struct drm_plane *_plane,
struct dma_fence *fence;
ret = i915_sw_fence_await_reservation(&state->commit_ready,
- obj->base.resv, NULL,
+ obj->base.resv,
false,
i915_fence_timeout(dev_priv),
GFP_KERNEL);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_clflush.c b/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
index f0435c6feb68..fde88fa90780 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_clflush.c
@@ -104,7 +104,7 @@ bool i915_gem_clflush_object(struct drm_i915_gem_object *obj,
clflush = clflush_work_create(obj);
if (clflush) {
i915_sw_fence_await_reservation(&clflush->base.chain,
- obj->base.resv, NULL, true,
+ obj->base.resv, true,
i915_fence_timeout(to_i915(obj->base.dev)),
I915_FENCE_GFP);
dma_resv_add_excl_fence(obj->base.resv, &clflush->base.dma);
diff --git a/drivers/gpu/drm/i915/i915_sw_fence.c b/drivers/gpu/drm/i915/i915_sw_fence.c
index c589a681da77..91711a46b1c7 100644
--- a/drivers/gpu/drm/i915/i915_sw_fence.c
+++ b/drivers/gpu/drm/i915/i915_sw_fence.c
@@ -567,7 +567,6 @@ int __i915_sw_fence_await_dma_fence(struct i915_sw_fence *fence,
int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
struct dma_resv *resv,
- const struct dma_fence_ops *exclude,
bool write,
unsigned long timeout,
gfp_t gfp)
@@ -587,9 +586,6 @@ int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
return ret;
for (i = 0; i < count; i++) {
- if (shared[i]->ops == exclude)
- continue;
-
pending = i915_sw_fence_await_dma_fence(fence,
shared[i],
timeout,
@@ -609,7 +605,7 @@ int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
excl = dma_resv_get_excl_unlocked(resv);
}
- if (ret >= 0 && excl && excl->ops != exclude) {
+ if (ret >= 0 && excl) {
pending = i915_sw_fence_await_dma_fence(fence,
excl,
timeout,
diff --git a/drivers/gpu/drm/i915/i915_sw_fence.h b/drivers/gpu/drm/i915/i915_sw_fence.h
index 30a863353ee6..6572f01668e4 100644
--- a/drivers/gpu/drm/i915/i915_sw_fence.h
+++ b/drivers/gpu/drm/i915/i915_sw_fence.h
@@ -86,7 +86,6 @@ int i915_sw_fence_await_dma_fence(struct i915_sw_fence *fence,
int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,
struct dma_resv *resv,
- const struct dma_fence_ops *exclude,
bool write,
unsigned long timeout,
gfp_t gfp);
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* [PATCH v5 19/20] drm/i915: Don't break exclusive fence ordering
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:47 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Lucas Stach,
Daniel Vetter, Maarten Lankhorst, Thomas Hellström,
Jason Ekstrand
There's only one exclusive slot, and we must not break the ordering.
Adding a new exclusive fence drops all previous fences from the
dma_resv. To avoid violating the signalling order we err on the side of
over-synchronizing by waiting for the existing fences, even if
userspace asked us to ignore them.
A better fix would be to us a dma_fence_chain or _array like e.g.
amdgpu now uses, but it probably makes sense to lift this into
dma-resv.c code as a proper concept, so that drivers don't have to
hack up their own solution each on their own. Hence go with the simple
fix for now.
Another option is the fence import ioctl from Jason:
https://lore.kernel.org/dri-devel/20210610210925.642582-7-jason@jlekstrand.net/
v2: Improve commit message per Lucas' suggestion.
Cc: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: "Thomas Hellström" <thomas.hellstrom@linux.intel.com>
Cc: Jason Ekstrand <jason@jlekstrand.net>
---
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index 1ed7475de454..25ba2765d27d 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -2240,6 +2240,7 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
struct i915_vma *vma = ev->vma;
unsigned int flags = ev->flags;
struct drm_i915_gem_object *obj = vma->obj;
+ bool async, write;
assert_vma_held(vma);
@@ -2271,7 +2272,10 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
flags &= ~EXEC_OBJECT_ASYNC;
}
- if (err == 0 && !(flags & EXEC_OBJECT_ASYNC)) {
+ async = flags & EXEC_OBJECT_ASYNC;
+ write = flags & EXEC_OBJECT_WRITE;
+
+ if (err == 0 && (!async || write)) {
err = i915_request_await_object
(eb->request, obj, flags & EXEC_OBJECT_WRITE);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 19/20] drm/i915: Don't break exclusive fence ordering
@ 2021-08-05 10:47 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Lucas Stach,
Daniel Vetter, Maarten Lankhorst, Thomas Hellström,
Jason Ekstrand
There's only one exclusive slot, and we must not break the ordering.
Adding a new exclusive fence drops all previous fences from the
dma_resv. To avoid violating the signalling order we err on the side of
over-synchronizing by waiting for the existing fences, even if
userspace asked us to ignore them.
A better fix would be to us a dma_fence_chain or _array like e.g.
amdgpu now uses, but it probably makes sense to lift this into
dma-resv.c code as a proper concept, so that drivers don't have to
hack up their own solution each on their own. Hence go with the simple
fix for now.
Another option is the fence import ioctl from Jason:
https://lore.kernel.org/dri-devel/20210610210925.642582-7-jason@jlekstrand.net/
v2: Improve commit message per Lucas' suggestion.
Cc: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: "Thomas Hellström" <thomas.hellstrom@linux.intel.com>
Cc: Jason Ekstrand <jason@jlekstrand.net>
---
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index 1ed7475de454..25ba2765d27d 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -2240,6 +2240,7 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
struct i915_vma *vma = ev->vma;
unsigned int flags = ev->flags;
struct drm_i915_gem_object *obj = vma->obj;
+ bool async, write;
assert_vma_held(vma);
@@ -2271,7 +2272,10 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
flags &= ~EXEC_OBJECT_ASYNC;
}
- if (err == 0 && !(flags & EXEC_OBJECT_ASYNC)) {
+ async = flags & EXEC_OBJECT_ASYNC;
+ write = flags & EXEC_OBJECT_WRITE;
+
+ if (err == 0 && (!async || write)) {
err = i915_request_await_object
(eb->request, obj, flags & EXEC_OBJECT_WRITE);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread
* [PATCH v5 20/20] dma-resv: Give the docs a do-over
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-05 10:47 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Christian König,
Jason Ekstrand, Matthew Auld, Daniel Vetter, Sumit Semwal,
linux-media, linaro-mm-sig
Specifically document the new/clarified rules around how the shared
fences do not have any ordering requirements against the exclusive
fence.
But also document all the things a bit better, given how central
struct dma_resv to dynamic buffer management the docs have been very
inadequat.
- Lots more links to other pieces of the puzzle. Unfortunately
ttm_buffer_object has no docs, so no links :-(
- Explain/complain a bit about dma_resv_locking_ctx(). I still don't
like that one, but fixing the ttm call chains is going to be
horrible. Plus we want to plug in real slowpath locking when we do
that anyway.
- Main part of the patch is some actual docs for struct dma_resv.
Overall I think we still have a lot of bad naming in this area (e.g.
dma_resv.fence is singular, but contains the multiple shared fences),
but I think that's more indicative of how the semantics and rules are
just not great.
Another thing that's real awkard is how chaining exclusive fences
right now means direct dma_resv.exclusive_fence pointer access with an
rcu_assign_pointer. Not so great either.
v2:
- Fix a pile of typos (Matt, Jason)
- Hammer it in that breaking the rules leads to use-after-free issues
around dma-buf sharing (Christian)
Reviewed-by: Christian König <christian.koenig@amd.com>
Cc: Jason Ekstrand <jason@jlekstrand.net>
Cc: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/dma-buf/dma-resv.c | 24 ++++++---
include/linux/dma-buf.h | 7 +++
include/linux/dma-resv.h | 104 +++++++++++++++++++++++++++++++++++--
3 files changed, 124 insertions(+), 11 deletions(-)
diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
index e744fd87c63c..84fbe60629e3 100644
--- a/drivers/dma-buf/dma-resv.c
+++ b/drivers/dma-buf/dma-resv.c
@@ -48,6 +48,8 @@
* write operations) or N shared fences (read operations). The RCU
* mechanism is used to protect read access to fences from locked
* write-side updates.
+ *
+ * See struct dma_resv for more details.
*/
DEFINE_WD_CLASS(reservation_ww_class);
@@ -137,7 +139,11 @@ EXPORT_SYMBOL(dma_resv_fini);
* @num_fences: number of fences we want to add
*
* Should be called before dma_resv_add_shared_fence(). Must
- * be called with obj->lock held.
+ * be called with @obj locked through dma_resv_lock().
+ *
+ * Note that the preallocated slots need to be re-reserved if @obj is unlocked
+ * at any time before calling dma_resv_add_shared_fence(). This is validated
+ * when CONFIG_DEBUG_MUTEXES is enabled.
*
* RETURNS
* Zero for success, or -errno
@@ -234,8 +240,10 @@ EXPORT_SYMBOL(dma_resv_reset_shared_max);
* @obj: the reservation object
* @fence: the shared fence to add
*
- * Add a fence to a shared slot, obj->lock must be held, and
+ * Add a fence to a shared slot, @obj must be locked with dma_resv_lock(), and
* dma_resv_reserve_shared() has been called.
+ *
+ * See also &dma_resv.fence for a discussion of the semantics.
*/
void dma_resv_add_shared_fence(struct dma_resv *obj, struct dma_fence *fence)
{
@@ -278,9 +286,11 @@ EXPORT_SYMBOL(dma_resv_add_shared_fence);
/**
* dma_resv_add_excl_fence - Add an exclusive fence.
* @obj: the reservation object
- * @fence: the shared fence to add
+ * @fence: the exclusive fence to add
*
- * Add a fence to the exclusive slot. The obj->lock must be held.
+ * Add a fence to the exclusive slot. @obj must be locked with dma_resv_lock().
+ * Note that this function replaces all fences attached to @obj, see also
+ * &dma_resv.fence_excl for a discussion of the semantics.
*/
void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence)
{
@@ -609,9 +619,11 @@ static inline int dma_resv_test_signaled_single(struct dma_fence *passed_fence)
* fence
*
* Callers are not required to hold specific locks, but maybe hold
- * dma_resv_lock() already
+ * dma_resv_lock() already.
+ *
* RETURNS
- * true if all fences signaled, else false
+ *
+ * True if all fences signaled, else false.
*/
bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all)
{
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index 678b2006be78..fc62b5f9980c 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -420,6 +420,13 @@ struct dma_buf {
* - Dynamic importers should set fences for any access that they can't
* disable immediately from their &dma_buf_attach_ops.move_notify
* callback.
+ *
+ * IMPORTANT:
+ *
+ * All drivers must obey the struct dma_resv rules, specifically the
+ * rules for updating fences, see &dma_resv.fence_excl and
+ * &dma_resv.fence. If these dependency rules are broken access tracking
+ * can be lost resulting in use after free issues.
*/
struct dma_resv *resv;
diff --git a/include/linux/dma-resv.h b/include/linux/dma-resv.h
index e1ca2080a1ff..9100dd3dc21f 100644
--- a/include/linux/dma-resv.h
+++ b/include/linux/dma-resv.h
@@ -62,16 +62,90 @@ struct dma_resv_list {
/**
* struct dma_resv - a reservation object manages fences for a buffer
- * @lock: update side lock
- * @seq: sequence count for managing RCU read-side synchronization
- * @fence_excl: the exclusive fence, if there is one currently
- * @fence: list of current shared fences
+ *
+ * There are multiple uses for this, with sometimes slightly different rules in
+ * how the fence slots are used.
+ *
+ * One use is to synchronize cross-driver access to a struct dma_buf, either for
+ * dynamic buffer management or just to handle implicit synchronization between
+ * different users of the buffer in userspace. See &dma_buf.resv for a more
+ * in-depth discussion.
+ *
+ * The other major use is to manage access and locking within a driver in a
+ * buffer based memory manager. struct ttm_buffer_object is the canonical
+ * example here, since this is where reservation objects originated from. But
+ * use in drivers is spreading and some drivers also manage struct
+ * drm_gem_object with the same scheme.
*/
struct dma_resv {
+ /**
+ * @lock:
+ *
+ * Update side lock. Don't use directly, instead use the wrapper
+ * functions like dma_resv_lock() and dma_resv_unlock().
+ *
+ * Drivers which use the reservation object to manage memory dynamically
+ * also use this lock to protect buffer object state like placement,
+ * allocation policies or throughout command submission.
+ */
struct ww_mutex lock;
+
+ /**
+ * @seq:
+ *
+ * Sequence count for managing RCU read-side synchronization, allows
+ * read-only access to @fence_excl and @fence while ensuring we take a
+ * consistent snapshot.
+ */
seqcount_ww_mutex_t seq;
+ /**
+ * @fence_excl:
+ *
+ * The exclusive fence, if there is one currently.
+ *
+ * There are two ways to update this fence:
+ *
+ * - First by calling dma_resv_add_excl_fence(), which replaces all
+ * fences attached to the reservation object. To guarantee that no
+ * fences are lost, this new fence must signal only after all previous
+ * fences, both shared and exclusive, have signalled. In some cases it
+ * is convenient to achieve that by attaching a struct dma_fence_array
+ * with all the new and old fences.
+ *
+ * - Alternatively the fence can be set directly, which leaves the
+ * shared fences unchanged. To guarantee that no fences are lost, this
+ * new fence must signal only after the previous exclusive fence has
+ * signalled. Since the shared fences are staying intact, it is not
+ * necessary to maintain any ordering against those. If semantically
+ * only a new access is added without actually treating the previous
+ * one as a dependency the exclusive fences can be strung together
+ * using struct dma_fence_chain.
+ *
+ * Note that actual semantics of what an exclusive or shared fence mean
+ * is defined by the user, for reservation objects shared across drivers
+ * see &dma_buf.resv.
+ */
struct dma_fence __rcu *fence_excl;
+
+ /**
+ * @fence:
+ *
+ * List of current shared fences.
+ *
+ * There are no ordering constraints of shared fences against the
+ * exclusive fence slot. If a waiter needs to wait for all access, it
+ * has to wait for both sets of fences to signal.
+ *
+ * A new fence is added by calling dma_resv_add_shared_fence(). Since
+ * this often needs to be done past the point of no return in command
+ * submission it cannot fail, and therefore sufficient slots need to be
+ * reserved by calling dma_resv_reserve_shared().
+ *
+ * Note that actual semantics of what an exclusive or shared fence mean
+ * is defined by the user, for reservation objects shared across drivers
+ * see &dma_buf.resv.
+ */
struct dma_resv_list __rcu *fence;
};
@@ -98,6 +172,13 @@ static inline void dma_resv_reset_shared_max(struct dma_resv *obj) {}
* undefined order, a #ww_acquire_ctx is passed to unwind if a cycle
* is detected. See ww_mutex_lock() and ww_acquire_init(). A reservation
* object may be locked by itself by passing NULL as @ctx.
+ *
+ * When a die situation is indicated by returning -EDEADLK all locks held by
+ * @ctx must be unlocked and then dma_resv_lock_slow() called on @obj.
+ *
+ * Unlocked by calling dma_resv_unlock().
+ *
+ * See also dma_resv_lock_interruptible() for the interruptible variant.
*/
static inline int dma_resv_lock(struct dma_resv *obj,
struct ww_acquire_ctx *ctx)
@@ -119,6 +200,12 @@ static inline int dma_resv_lock(struct dma_resv *obj,
* undefined order, a #ww_acquire_ctx is passed to unwind if a cycle
* is detected. See ww_mutex_lock() and ww_acquire_init(). A reservation
* object may be locked by itself by passing NULL as @ctx.
+ *
+ * When a die situation is indicated by returning -EDEADLK all locks held by
+ * @ctx must be unlocked and then dma_resv_lock_slow_interruptible() called on
+ * @obj.
+ *
+ * Unlocked by calling dma_resv_unlock().
*/
static inline int dma_resv_lock_interruptible(struct dma_resv *obj,
struct ww_acquire_ctx *ctx)
@@ -134,6 +221,8 @@ static inline int dma_resv_lock_interruptible(struct dma_resv *obj,
* Acquires the reservation object after a die case. This function
* will sleep until the lock becomes available. See dma_resv_lock() as
* well.
+ *
+ * See also dma_resv_lock_slow_interruptible() for the interruptible variant.
*/
static inline void dma_resv_lock_slow(struct dma_resv *obj,
struct ww_acquire_ctx *ctx)
@@ -167,7 +256,7 @@ static inline int dma_resv_lock_slow_interruptible(struct dma_resv *obj,
* if they overlap with a writer.
*
* Also note that since no context is provided, no deadlock protection is
- * possible.
+ * possible, which is also not needed for a trylock.
*
* Returns true if the lock was acquired, false otherwise.
*/
@@ -193,6 +282,11 @@ static inline bool dma_resv_is_locked(struct dma_resv *obj)
*
* Returns the context used to lock a reservation object or NULL if no context
* was used or the object is not locked at all.
+ *
+ * WARNING: This interface is pretty horrible, but TTM needs it because it
+ * doesn't pass the struct ww_acquire_ctx around in some very long callchains.
+ * Everyone else just uses it to check whether they're holding a reservation or
+ * not.
*/
static inline struct ww_acquire_ctx *dma_resv_locking_ctx(struct dma_resv *obj)
{
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* [Intel-gfx] [PATCH v5 20/20] dma-resv: Give the docs a do-over
@ 2021-08-05 10:47 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-05 10:47 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Christian König,
Jason Ekstrand, Matthew Auld, Daniel Vetter, Sumit Semwal,
linux-media, linaro-mm-sig
Specifically document the new/clarified rules around how the shared
fences do not have any ordering requirements against the exclusive
fence.
But also document all the things a bit better, given how central
struct dma_resv to dynamic buffer management the docs have been very
inadequat.
- Lots more links to other pieces of the puzzle. Unfortunately
ttm_buffer_object has no docs, so no links :-(
- Explain/complain a bit about dma_resv_locking_ctx(). I still don't
like that one, but fixing the ttm call chains is going to be
horrible. Plus we want to plug in real slowpath locking when we do
that anyway.
- Main part of the patch is some actual docs for struct dma_resv.
Overall I think we still have a lot of bad naming in this area (e.g.
dma_resv.fence is singular, but contains the multiple shared fences),
but I think that's more indicative of how the semantics and rules are
just not great.
Another thing that's real awkard is how chaining exclusive fences
right now means direct dma_resv.exclusive_fence pointer access with an
rcu_assign_pointer. Not so great either.
v2:
- Fix a pile of typos (Matt, Jason)
- Hammer it in that breaking the rules leads to use-after-free issues
around dma-buf sharing (Christian)
Reviewed-by: Christian König <christian.koenig@amd.com>
Cc: Jason Ekstrand <jason@jlekstrand.net>
Cc: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
---
drivers/dma-buf/dma-resv.c | 24 ++++++---
include/linux/dma-buf.h | 7 +++
include/linux/dma-resv.h | 104 +++++++++++++++++++++++++++++++++++--
3 files changed, 124 insertions(+), 11 deletions(-)
diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
index e744fd87c63c..84fbe60629e3 100644
--- a/drivers/dma-buf/dma-resv.c
+++ b/drivers/dma-buf/dma-resv.c
@@ -48,6 +48,8 @@
* write operations) or N shared fences (read operations). The RCU
* mechanism is used to protect read access to fences from locked
* write-side updates.
+ *
+ * See struct dma_resv for more details.
*/
DEFINE_WD_CLASS(reservation_ww_class);
@@ -137,7 +139,11 @@ EXPORT_SYMBOL(dma_resv_fini);
* @num_fences: number of fences we want to add
*
* Should be called before dma_resv_add_shared_fence(). Must
- * be called with obj->lock held.
+ * be called with @obj locked through dma_resv_lock().
+ *
+ * Note that the preallocated slots need to be re-reserved if @obj is unlocked
+ * at any time before calling dma_resv_add_shared_fence(). This is validated
+ * when CONFIG_DEBUG_MUTEXES is enabled.
*
* RETURNS
* Zero for success, or -errno
@@ -234,8 +240,10 @@ EXPORT_SYMBOL(dma_resv_reset_shared_max);
* @obj: the reservation object
* @fence: the shared fence to add
*
- * Add a fence to a shared slot, obj->lock must be held, and
+ * Add a fence to a shared slot, @obj must be locked with dma_resv_lock(), and
* dma_resv_reserve_shared() has been called.
+ *
+ * See also &dma_resv.fence for a discussion of the semantics.
*/
void dma_resv_add_shared_fence(struct dma_resv *obj, struct dma_fence *fence)
{
@@ -278,9 +286,11 @@ EXPORT_SYMBOL(dma_resv_add_shared_fence);
/**
* dma_resv_add_excl_fence - Add an exclusive fence.
* @obj: the reservation object
- * @fence: the shared fence to add
+ * @fence: the exclusive fence to add
*
- * Add a fence to the exclusive slot. The obj->lock must be held.
+ * Add a fence to the exclusive slot. @obj must be locked with dma_resv_lock().
+ * Note that this function replaces all fences attached to @obj, see also
+ * &dma_resv.fence_excl for a discussion of the semantics.
*/
void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence)
{
@@ -609,9 +619,11 @@ static inline int dma_resv_test_signaled_single(struct dma_fence *passed_fence)
* fence
*
* Callers are not required to hold specific locks, but maybe hold
- * dma_resv_lock() already
+ * dma_resv_lock() already.
+ *
* RETURNS
- * true if all fences signaled, else false
+ *
+ * True if all fences signaled, else false.
*/
bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all)
{
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index 678b2006be78..fc62b5f9980c 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -420,6 +420,13 @@ struct dma_buf {
* - Dynamic importers should set fences for any access that they can't
* disable immediately from their &dma_buf_attach_ops.move_notify
* callback.
+ *
+ * IMPORTANT:
+ *
+ * All drivers must obey the struct dma_resv rules, specifically the
+ * rules for updating fences, see &dma_resv.fence_excl and
+ * &dma_resv.fence. If these dependency rules are broken access tracking
+ * can be lost resulting in use after free issues.
*/
struct dma_resv *resv;
diff --git a/include/linux/dma-resv.h b/include/linux/dma-resv.h
index e1ca2080a1ff..9100dd3dc21f 100644
--- a/include/linux/dma-resv.h
+++ b/include/linux/dma-resv.h
@@ -62,16 +62,90 @@ struct dma_resv_list {
/**
* struct dma_resv - a reservation object manages fences for a buffer
- * @lock: update side lock
- * @seq: sequence count for managing RCU read-side synchronization
- * @fence_excl: the exclusive fence, if there is one currently
- * @fence: list of current shared fences
+ *
+ * There are multiple uses for this, with sometimes slightly different rules in
+ * how the fence slots are used.
+ *
+ * One use is to synchronize cross-driver access to a struct dma_buf, either for
+ * dynamic buffer management or just to handle implicit synchronization between
+ * different users of the buffer in userspace. See &dma_buf.resv for a more
+ * in-depth discussion.
+ *
+ * The other major use is to manage access and locking within a driver in a
+ * buffer based memory manager. struct ttm_buffer_object is the canonical
+ * example here, since this is where reservation objects originated from. But
+ * use in drivers is spreading and some drivers also manage struct
+ * drm_gem_object with the same scheme.
*/
struct dma_resv {
+ /**
+ * @lock:
+ *
+ * Update side lock. Don't use directly, instead use the wrapper
+ * functions like dma_resv_lock() and dma_resv_unlock().
+ *
+ * Drivers which use the reservation object to manage memory dynamically
+ * also use this lock to protect buffer object state like placement,
+ * allocation policies or throughout command submission.
+ */
struct ww_mutex lock;
+
+ /**
+ * @seq:
+ *
+ * Sequence count for managing RCU read-side synchronization, allows
+ * read-only access to @fence_excl and @fence while ensuring we take a
+ * consistent snapshot.
+ */
seqcount_ww_mutex_t seq;
+ /**
+ * @fence_excl:
+ *
+ * The exclusive fence, if there is one currently.
+ *
+ * There are two ways to update this fence:
+ *
+ * - First by calling dma_resv_add_excl_fence(), which replaces all
+ * fences attached to the reservation object. To guarantee that no
+ * fences are lost, this new fence must signal only after all previous
+ * fences, both shared and exclusive, have signalled. In some cases it
+ * is convenient to achieve that by attaching a struct dma_fence_array
+ * with all the new and old fences.
+ *
+ * - Alternatively the fence can be set directly, which leaves the
+ * shared fences unchanged. To guarantee that no fences are lost, this
+ * new fence must signal only after the previous exclusive fence has
+ * signalled. Since the shared fences are staying intact, it is not
+ * necessary to maintain any ordering against those. If semantically
+ * only a new access is added without actually treating the previous
+ * one as a dependency the exclusive fences can be strung together
+ * using struct dma_fence_chain.
+ *
+ * Note that actual semantics of what an exclusive or shared fence mean
+ * is defined by the user, for reservation objects shared across drivers
+ * see &dma_buf.resv.
+ */
struct dma_fence __rcu *fence_excl;
+
+ /**
+ * @fence:
+ *
+ * List of current shared fences.
+ *
+ * There are no ordering constraints of shared fences against the
+ * exclusive fence slot. If a waiter needs to wait for all access, it
+ * has to wait for both sets of fences to signal.
+ *
+ * A new fence is added by calling dma_resv_add_shared_fence(). Since
+ * this often needs to be done past the point of no return in command
+ * submission it cannot fail, and therefore sufficient slots need to be
+ * reserved by calling dma_resv_reserve_shared().
+ *
+ * Note that actual semantics of what an exclusive or shared fence mean
+ * is defined by the user, for reservation objects shared across drivers
+ * see &dma_buf.resv.
+ */
struct dma_resv_list __rcu *fence;
};
@@ -98,6 +172,13 @@ static inline void dma_resv_reset_shared_max(struct dma_resv *obj) {}
* undefined order, a #ww_acquire_ctx is passed to unwind if a cycle
* is detected. See ww_mutex_lock() and ww_acquire_init(). A reservation
* object may be locked by itself by passing NULL as @ctx.
+ *
+ * When a die situation is indicated by returning -EDEADLK all locks held by
+ * @ctx must be unlocked and then dma_resv_lock_slow() called on @obj.
+ *
+ * Unlocked by calling dma_resv_unlock().
+ *
+ * See also dma_resv_lock_interruptible() for the interruptible variant.
*/
static inline int dma_resv_lock(struct dma_resv *obj,
struct ww_acquire_ctx *ctx)
@@ -119,6 +200,12 @@ static inline int dma_resv_lock(struct dma_resv *obj,
* undefined order, a #ww_acquire_ctx is passed to unwind if a cycle
* is detected. See ww_mutex_lock() and ww_acquire_init(). A reservation
* object may be locked by itself by passing NULL as @ctx.
+ *
+ * When a die situation is indicated by returning -EDEADLK all locks held by
+ * @ctx must be unlocked and then dma_resv_lock_slow_interruptible() called on
+ * @obj.
+ *
+ * Unlocked by calling dma_resv_unlock().
*/
static inline int dma_resv_lock_interruptible(struct dma_resv *obj,
struct ww_acquire_ctx *ctx)
@@ -134,6 +221,8 @@ static inline int dma_resv_lock_interruptible(struct dma_resv *obj,
* Acquires the reservation object after a die case. This function
* will sleep until the lock becomes available. See dma_resv_lock() as
* well.
+ *
+ * See also dma_resv_lock_slow_interruptible() for the interruptible variant.
*/
static inline void dma_resv_lock_slow(struct dma_resv *obj,
struct ww_acquire_ctx *ctx)
@@ -167,7 +256,7 @@ static inline int dma_resv_lock_slow_interruptible(struct dma_resv *obj,
* if they overlap with a writer.
*
* Also note that since no context is provided, no deadlock protection is
- * possible.
+ * possible, which is also not needed for a trylock.
*
* Returns true if the lock was acquired, false otherwise.
*/
@@ -193,6 +282,11 @@ static inline bool dma_resv_is_locked(struct dma_resv *obj)
*
* Returns the context used to lock a reservation object or NULL if no context
* was used or the object is not locked at all.
+ *
+ * WARNING: This interface is pretty horrible, but TTM needs it because it
+ * doesn't pass the struct ww_acquire_ctx around in some very long callchains.
+ * Everyone else just uses it to check whether they're holding a reservation or
+ * not.
*/
static inline struct ww_acquire_ctx *dma_resv_locking_ctx(struct dma_resv *obj)
{
--
2.32.0
^ permalink raw reply related [flat|nested] 122+ messages in thread* Re: [PATCH v5 20/20] dma-resv: Give the docs a do-over
2021-08-05 10:47 ` [Intel-gfx] " Daniel Vetter
@ 2021-08-30 19:38 ` Daniel Vetter
-1 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-30 19:38 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Christian König,
Jason Ekstrand, Matthew Auld, Daniel Vetter, Sumit Semwal,
linux-media, linaro-mm-sig
On Thu, Aug 05, 2021 at 12:47:05PM +0200, Daniel Vetter wrote:
> Specifically document the new/clarified rules around how the shared
> fences do not have any ordering requirements against the exclusive
> fence.
>
> But also document all the things a bit better, given how central
> struct dma_resv to dynamic buffer management the docs have been very
> inadequat.
>
> - Lots more links to other pieces of the puzzle. Unfortunately
> ttm_buffer_object has no docs, so no links :-(
>
> - Explain/complain a bit about dma_resv_locking_ctx(). I still don't
> like that one, but fixing the ttm call chains is going to be
> horrible. Plus we want to plug in real slowpath locking when we do
> that anyway.
>
> - Main part of the patch is some actual docs for struct dma_resv.
>
> Overall I think we still have a lot of bad naming in this area (e.g.
> dma_resv.fence is singular, but contains the multiple shared fences),
> but I think that's more indicative of how the semantics and rules are
> just not great.
>
> Another thing that's real awkard is how chaining exclusive fences
> right now means direct dma_resv.exclusive_fence pointer access with an
> rcu_assign_pointer. Not so great either.
>
> v2:
> - Fix a pile of typos (Matt, Jason)
> - Hammer it in that breaking the rules leads to use-after-free issues
> around dma-buf sharing (Christian)
>
> Reviewed-by: Christian König <christian.koenig@amd.com>
> Cc: Jason Ekstrand <jason@jlekstrand.net>
> Cc: Matthew Auld <matthew.auld@intel.com>
> Reviewed-by: Matthew Auld <matthew.auld@intel.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Also pushed to drm-misc-next.
-Daniel
> ---
> drivers/dma-buf/dma-resv.c | 24 ++++++---
> include/linux/dma-buf.h | 7 +++
> include/linux/dma-resv.h | 104 +++++++++++++++++++++++++++++++++++--
> 3 files changed, 124 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
> index e744fd87c63c..84fbe60629e3 100644
> --- a/drivers/dma-buf/dma-resv.c
> +++ b/drivers/dma-buf/dma-resv.c
> @@ -48,6 +48,8 @@
> * write operations) or N shared fences (read operations). The RCU
> * mechanism is used to protect read access to fences from locked
> * write-side updates.
> + *
> + * See struct dma_resv for more details.
> */
>
> DEFINE_WD_CLASS(reservation_ww_class);
> @@ -137,7 +139,11 @@ EXPORT_SYMBOL(dma_resv_fini);
> * @num_fences: number of fences we want to add
> *
> * Should be called before dma_resv_add_shared_fence(). Must
> - * be called with obj->lock held.
> + * be called with @obj locked through dma_resv_lock().
> + *
> + * Note that the preallocated slots need to be re-reserved if @obj is unlocked
> + * at any time before calling dma_resv_add_shared_fence(). This is validated
> + * when CONFIG_DEBUG_MUTEXES is enabled.
> *
> * RETURNS
> * Zero for success, or -errno
> @@ -234,8 +240,10 @@ EXPORT_SYMBOL(dma_resv_reset_shared_max);
> * @obj: the reservation object
> * @fence: the shared fence to add
> *
> - * Add a fence to a shared slot, obj->lock must be held, and
> + * Add a fence to a shared slot, @obj must be locked with dma_resv_lock(), and
> * dma_resv_reserve_shared() has been called.
> + *
> + * See also &dma_resv.fence for a discussion of the semantics.
> */
> void dma_resv_add_shared_fence(struct dma_resv *obj, struct dma_fence *fence)
> {
> @@ -278,9 +286,11 @@ EXPORT_SYMBOL(dma_resv_add_shared_fence);
> /**
> * dma_resv_add_excl_fence - Add an exclusive fence.
> * @obj: the reservation object
> - * @fence: the shared fence to add
> + * @fence: the exclusive fence to add
> *
> - * Add a fence to the exclusive slot. The obj->lock must be held.
> + * Add a fence to the exclusive slot. @obj must be locked with dma_resv_lock().
> + * Note that this function replaces all fences attached to @obj, see also
> + * &dma_resv.fence_excl for a discussion of the semantics.
> */
> void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence)
> {
> @@ -609,9 +619,11 @@ static inline int dma_resv_test_signaled_single(struct dma_fence *passed_fence)
> * fence
> *
> * Callers are not required to hold specific locks, but maybe hold
> - * dma_resv_lock() already
> + * dma_resv_lock() already.
> + *
> * RETURNS
> - * true if all fences signaled, else false
> + *
> + * True if all fences signaled, else false.
> */
> bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all)
> {
> diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> index 678b2006be78..fc62b5f9980c 100644
> --- a/include/linux/dma-buf.h
> +++ b/include/linux/dma-buf.h
> @@ -420,6 +420,13 @@ struct dma_buf {
> * - Dynamic importers should set fences for any access that they can't
> * disable immediately from their &dma_buf_attach_ops.move_notify
> * callback.
> + *
> + * IMPORTANT:
> + *
> + * All drivers must obey the struct dma_resv rules, specifically the
> + * rules for updating fences, see &dma_resv.fence_excl and
> + * &dma_resv.fence. If these dependency rules are broken access tracking
> + * can be lost resulting in use after free issues.
> */
> struct dma_resv *resv;
>
> diff --git a/include/linux/dma-resv.h b/include/linux/dma-resv.h
> index e1ca2080a1ff..9100dd3dc21f 100644
> --- a/include/linux/dma-resv.h
> +++ b/include/linux/dma-resv.h
> @@ -62,16 +62,90 @@ struct dma_resv_list {
>
> /**
> * struct dma_resv - a reservation object manages fences for a buffer
> - * @lock: update side lock
> - * @seq: sequence count for managing RCU read-side synchronization
> - * @fence_excl: the exclusive fence, if there is one currently
> - * @fence: list of current shared fences
> + *
> + * There are multiple uses for this, with sometimes slightly different rules in
> + * how the fence slots are used.
> + *
> + * One use is to synchronize cross-driver access to a struct dma_buf, either for
> + * dynamic buffer management or just to handle implicit synchronization between
> + * different users of the buffer in userspace. See &dma_buf.resv for a more
> + * in-depth discussion.
> + *
> + * The other major use is to manage access and locking within a driver in a
> + * buffer based memory manager. struct ttm_buffer_object is the canonical
> + * example here, since this is where reservation objects originated from. But
> + * use in drivers is spreading and some drivers also manage struct
> + * drm_gem_object with the same scheme.
> */
> struct dma_resv {
> + /**
> + * @lock:
> + *
> + * Update side lock. Don't use directly, instead use the wrapper
> + * functions like dma_resv_lock() and dma_resv_unlock().
> + *
> + * Drivers which use the reservation object to manage memory dynamically
> + * also use this lock to protect buffer object state like placement,
> + * allocation policies or throughout command submission.
> + */
> struct ww_mutex lock;
> +
> + /**
> + * @seq:
> + *
> + * Sequence count for managing RCU read-side synchronization, allows
> + * read-only access to @fence_excl and @fence while ensuring we take a
> + * consistent snapshot.
> + */
> seqcount_ww_mutex_t seq;
>
> + /**
> + * @fence_excl:
> + *
> + * The exclusive fence, if there is one currently.
> + *
> + * There are two ways to update this fence:
> + *
> + * - First by calling dma_resv_add_excl_fence(), which replaces all
> + * fences attached to the reservation object. To guarantee that no
> + * fences are lost, this new fence must signal only after all previous
> + * fences, both shared and exclusive, have signalled. In some cases it
> + * is convenient to achieve that by attaching a struct dma_fence_array
> + * with all the new and old fences.
> + *
> + * - Alternatively the fence can be set directly, which leaves the
> + * shared fences unchanged. To guarantee that no fences are lost, this
> + * new fence must signal only after the previous exclusive fence has
> + * signalled. Since the shared fences are staying intact, it is not
> + * necessary to maintain any ordering against those. If semantically
> + * only a new access is added without actually treating the previous
> + * one as a dependency the exclusive fences can be strung together
> + * using struct dma_fence_chain.
> + *
> + * Note that actual semantics of what an exclusive or shared fence mean
> + * is defined by the user, for reservation objects shared across drivers
> + * see &dma_buf.resv.
> + */
> struct dma_fence __rcu *fence_excl;
> +
> + /**
> + * @fence:
> + *
> + * List of current shared fences.
> + *
> + * There are no ordering constraints of shared fences against the
> + * exclusive fence slot. If a waiter needs to wait for all access, it
> + * has to wait for both sets of fences to signal.
> + *
> + * A new fence is added by calling dma_resv_add_shared_fence(). Since
> + * this often needs to be done past the point of no return in command
> + * submission it cannot fail, and therefore sufficient slots need to be
> + * reserved by calling dma_resv_reserve_shared().
> + *
> + * Note that actual semantics of what an exclusive or shared fence mean
> + * is defined by the user, for reservation objects shared across drivers
> + * see &dma_buf.resv.
> + */
> struct dma_resv_list __rcu *fence;
> };
>
> @@ -98,6 +172,13 @@ static inline void dma_resv_reset_shared_max(struct dma_resv *obj) {}
> * undefined order, a #ww_acquire_ctx is passed to unwind if a cycle
> * is detected. See ww_mutex_lock() and ww_acquire_init(). A reservation
> * object may be locked by itself by passing NULL as @ctx.
> + *
> + * When a die situation is indicated by returning -EDEADLK all locks held by
> + * @ctx must be unlocked and then dma_resv_lock_slow() called on @obj.
> + *
> + * Unlocked by calling dma_resv_unlock().
> + *
> + * See also dma_resv_lock_interruptible() for the interruptible variant.
> */
> static inline int dma_resv_lock(struct dma_resv *obj,
> struct ww_acquire_ctx *ctx)
> @@ -119,6 +200,12 @@ static inline int dma_resv_lock(struct dma_resv *obj,
> * undefined order, a #ww_acquire_ctx is passed to unwind if a cycle
> * is detected. See ww_mutex_lock() and ww_acquire_init(). A reservation
> * object may be locked by itself by passing NULL as @ctx.
> + *
> + * When a die situation is indicated by returning -EDEADLK all locks held by
> + * @ctx must be unlocked and then dma_resv_lock_slow_interruptible() called on
> + * @obj.
> + *
> + * Unlocked by calling dma_resv_unlock().
> */
> static inline int dma_resv_lock_interruptible(struct dma_resv *obj,
> struct ww_acquire_ctx *ctx)
> @@ -134,6 +221,8 @@ static inline int dma_resv_lock_interruptible(struct dma_resv *obj,
> * Acquires the reservation object after a die case. This function
> * will sleep until the lock becomes available. See dma_resv_lock() as
> * well.
> + *
> + * See also dma_resv_lock_slow_interruptible() for the interruptible variant.
> */
> static inline void dma_resv_lock_slow(struct dma_resv *obj,
> struct ww_acquire_ctx *ctx)
> @@ -167,7 +256,7 @@ static inline int dma_resv_lock_slow_interruptible(struct dma_resv *obj,
> * if they overlap with a writer.
> *
> * Also note that since no context is provided, no deadlock protection is
> - * possible.
> + * possible, which is also not needed for a trylock.
> *
> * Returns true if the lock was acquired, false otherwise.
> */
> @@ -193,6 +282,11 @@ static inline bool dma_resv_is_locked(struct dma_resv *obj)
> *
> * Returns the context used to lock a reservation object or NULL if no context
> * was used or the object is not locked at all.
> + *
> + * WARNING: This interface is pretty horrible, but TTM needs it because it
> + * doesn't pass the struct ww_acquire_ctx around in some very long callchains.
> + * Everyone else just uses it to check whether they're holding a reservation or
> + * not.
> */
> static inline struct ww_acquire_ctx *dma_resv_locking_ctx(struct dma_resv *obj)
> {
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread* Re: [Intel-gfx] [PATCH v5 20/20] dma-resv: Give the docs a do-over
@ 2021-08-30 19:38 ` Daniel Vetter
0 siblings, 0 replies; 122+ messages in thread
From: Daniel Vetter @ 2021-08-30 19:38 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Daniel Vetter, Christian König,
Jason Ekstrand, Matthew Auld, Daniel Vetter, Sumit Semwal,
linux-media, linaro-mm-sig
On Thu, Aug 05, 2021 at 12:47:05PM +0200, Daniel Vetter wrote:
> Specifically document the new/clarified rules around how the shared
> fences do not have any ordering requirements against the exclusive
> fence.
>
> But also document all the things a bit better, given how central
> struct dma_resv to dynamic buffer management the docs have been very
> inadequat.
>
> - Lots more links to other pieces of the puzzle. Unfortunately
> ttm_buffer_object has no docs, so no links :-(
>
> - Explain/complain a bit about dma_resv_locking_ctx(). I still don't
> like that one, but fixing the ttm call chains is going to be
> horrible. Plus we want to plug in real slowpath locking when we do
> that anyway.
>
> - Main part of the patch is some actual docs for struct dma_resv.
>
> Overall I think we still have a lot of bad naming in this area (e.g.
> dma_resv.fence is singular, but contains the multiple shared fences),
> but I think that's more indicative of how the semantics and rules are
> just not great.
>
> Another thing that's real awkard is how chaining exclusive fences
> right now means direct dma_resv.exclusive_fence pointer access with an
> rcu_assign_pointer. Not so great either.
>
> v2:
> - Fix a pile of typos (Matt, Jason)
> - Hammer it in that breaking the rules leads to use-after-free issues
> around dma-buf sharing (Christian)
>
> Reviewed-by: Christian König <christian.koenig@amd.com>
> Cc: Jason Ekstrand <jason@jlekstrand.net>
> Cc: Matthew Auld <matthew.auld@intel.com>
> Reviewed-by: Matthew Auld <matthew.auld@intel.com>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: "Christian König" <christian.koenig@amd.com>
> Cc: linux-media@vger.kernel.org
> Cc: linaro-mm-sig@lists.linaro.org
Also pushed to drm-misc-next.
-Daniel
> ---
> drivers/dma-buf/dma-resv.c | 24 ++++++---
> include/linux/dma-buf.h | 7 +++
> include/linux/dma-resv.h | 104 +++++++++++++++++++++++++++++++++++--
> 3 files changed, 124 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
> index e744fd87c63c..84fbe60629e3 100644
> --- a/drivers/dma-buf/dma-resv.c
> +++ b/drivers/dma-buf/dma-resv.c
> @@ -48,6 +48,8 @@
> * write operations) or N shared fences (read operations). The RCU
> * mechanism is used to protect read access to fences from locked
> * write-side updates.
> + *
> + * See struct dma_resv for more details.
> */
>
> DEFINE_WD_CLASS(reservation_ww_class);
> @@ -137,7 +139,11 @@ EXPORT_SYMBOL(dma_resv_fini);
> * @num_fences: number of fences we want to add
> *
> * Should be called before dma_resv_add_shared_fence(). Must
> - * be called with obj->lock held.
> + * be called with @obj locked through dma_resv_lock().
> + *
> + * Note that the preallocated slots need to be re-reserved if @obj is unlocked
> + * at any time before calling dma_resv_add_shared_fence(). This is validated
> + * when CONFIG_DEBUG_MUTEXES is enabled.
> *
> * RETURNS
> * Zero for success, or -errno
> @@ -234,8 +240,10 @@ EXPORT_SYMBOL(dma_resv_reset_shared_max);
> * @obj: the reservation object
> * @fence: the shared fence to add
> *
> - * Add a fence to a shared slot, obj->lock must be held, and
> + * Add a fence to a shared slot, @obj must be locked with dma_resv_lock(), and
> * dma_resv_reserve_shared() has been called.
> + *
> + * See also &dma_resv.fence for a discussion of the semantics.
> */
> void dma_resv_add_shared_fence(struct dma_resv *obj, struct dma_fence *fence)
> {
> @@ -278,9 +286,11 @@ EXPORT_SYMBOL(dma_resv_add_shared_fence);
> /**
> * dma_resv_add_excl_fence - Add an exclusive fence.
> * @obj: the reservation object
> - * @fence: the shared fence to add
> + * @fence: the exclusive fence to add
> *
> - * Add a fence to the exclusive slot. The obj->lock must be held.
> + * Add a fence to the exclusive slot. @obj must be locked with dma_resv_lock().
> + * Note that this function replaces all fences attached to @obj, see also
> + * &dma_resv.fence_excl for a discussion of the semantics.
> */
> void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence)
> {
> @@ -609,9 +619,11 @@ static inline int dma_resv_test_signaled_single(struct dma_fence *passed_fence)
> * fence
> *
> * Callers are not required to hold specific locks, but maybe hold
> - * dma_resv_lock() already
> + * dma_resv_lock() already.
> + *
> * RETURNS
> - * true if all fences signaled, else false
> + *
> + * True if all fences signaled, else false.
> */
> bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all)
> {
> diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> index 678b2006be78..fc62b5f9980c 100644
> --- a/include/linux/dma-buf.h
> +++ b/include/linux/dma-buf.h
> @@ -420,6 +420,13 @@ struct dma_buf {
> * - Dynamic importers should set fences for any access that they can't
> * disable immediately from their &dma_buf_attach_ops.move_notify
> * callback.
> + *
> + * IMPORTANT:
> + *
> + * All drivers must obey the struct dma_resv rules, specifically the
> + * rules for updating fences, see &dma_resv.fence_excl and
> + * &dma_resv.fence. If these dependency rules are broken access tracking
> + * can be lost resulting in use after free issues.
> */
> struct dma_resv *resv;
>
> diff --git a/include/linux/dma-resv.h b/include/linux/dma-resv.h
> index e1ca2080a1ff..9100dd3dc21f 100644
> --- a/include/linux/dma-resv.h
> +++ b/include/linux/dma-resv.h
> @@ -62,16 +62,90 @@ struct dma_resv_list {
>
> /**
> * struct dma_resv - a reservation object manages fences for a buffer
> - * @lock: update side lock
> - * @seq: sequence count for managing RCU read-side synchronization
> - * @fence_excl: the exclusive fence, if there is one currently
> - * @fence: list of current shared fences
> + *
> + * There are multiple uses for this, with sometimes slightly different rules in
> + * how the fence slots are used.
> + *
> + * One use is to synchronize cross-driver access to a struct dma_buf, either for
> + * dynamic buffer management or just to handle implicit synchronization between
> + * different users of the buffer in userspace. See &dma_buf.resv for a more
> + * in-depth discussion.
> + *
> + * The other major use is to manage access and locking within a driver in a
> + * buffer based memory manager. struct ttm_buffer_object is the canonical
> + * example here, since this is where reservation objects originated from. But
> + * use in drivers is spreading and some drivers also manage struct
> + * drm_gem_object with the same scheme.
> */
> struct dma_resv {
> + /**
> + * @lock:
> + *
> + * Update side lock. Don't use directly, instead use the wrapper
> + * functions like dma_resv_lock() and dma_resv_unlock().
> + *
> + * Drivers which use the reservation object to manage memory dynamically
> + * also use this lock to protect buffer object state like placement,
> + * allocation policies or throughout command submission.
> + */
> struct ww_mutex lock;
> +
> + /**
> + * @seq:
> + *
> + * Sequence count for managing RCU read-side synchronization, allows
> + * read-only access to @fence_excl and @fence while ensuring we take a
> + * consistent snapshot.
> + */
> seqcount_ww_mutex_t seq;
>
> + /**
> + * @fence_excl:
> + *
> + * The exclusive fence, if there is one currently.
> + *
> + * There are two ways to update this fence:
> + *
> + * - First by calling dma_resv_add_excl_fence(), which replaces all
> + * fences attached to the reservation object. To guarantee that no
> + * fences are lost, this new fence must signal only after all previous
> + * fences, both shared and exclusive, have signalled. In some cases it
> + * is convenient to achieve that by attaching a struct dma_fence_array
> + * with all the new and old fences.
> + *
> + * - Alternatively the fence can be set directly, which leaves the
> + * shared fences unchanged. To guarantee that no fences are lost, this
> + * new fence must signal only after the previous exclusive fence has
> + * signalled. Since the shared fences are staying intact, it is not
> + * necessary to maintain any ordering against those. If semantically
> + * only a new access is added without actually treating the previous
> + * one as a dependency the exclusive fences can be strung together
> + * using struct dma_fence_chain.
> + *
> + * Note that actual semantics of what an exclusive or shared fence mean
> + * is defined by the user, for reservation objects shared across drivers
> + * see &dma_buf.resv.
> + */
> struct dma_fence __rcu *fence_excl;
> +
> + /**
> + * @fence:
> + *
> + * List of current shared fences.
> + *
> + * There are no ordering constraints of shared fences against the
> + * exclusive fence slot. If a waiter needs to wait for all access, it
> + * has to wait for both sets of fences to signal.
> + *
> + * A new fence is added by calling dma_resv_add_shared_fence(). Since
> + * this often needs to be done past the point of no return in command
> + * submission it cannot fail, and therefore sufficient slots need to be
> + * reserved by calling dma_resv_reserve_shared().
> + *
> + * Note that actual semantics of what an exclusive or shared fence mean
> + * is defined by the user, for reservation objects shared across drivers
> + * see &dma_buf.resv.
> + */
> struct dma_resv_list __rcu *fence;
> };
>
> @@ -98,6 +172,13 @@ static inline void dma_resv_reset_shared_max(struct dma_resv *obj) {}
> * undefined order, a #ww_acquire_ctx is passed to unwind if a cycle
> * is detected. See ww_mutex_lock() and ww_acquire_init(). A reservation
> * object may be locked by itself by passing NULL as @ctx.
> + *
> + * When a die situation is indicated by returning -EDEADLK all locks held by
> + * @ctx must be unlocked and then dma_resv_lock_slow() called on @obj.
> + *
> + * Unlocked by calling dma_resv_unlock().
> + *
> + * See also dma_resv_lock_interruptible() for the interruptible variant.
> */
> static inline int dma_resv_lock(struct dma_resv *obj,
> struct ww_acquire_ctx *ctx)
> @@ -119,6 +200,12 @@ static inline int dma_resv_lock(struct dma_resv *obj,
> * undefined order, a #ww_acquire_ctx is passed to unwind if a cycle
> * is detected. See ww_mutex_lock() and ww_acquire_init(). A reservation
> * object may be locked by itself by passing NULL as @ctx.
> + *
> + * When a die situation is indicated by returning -EDEADLK all locks held by
> + * @ctx must be unlocked and then dma_resv_lock_slow_interruptible() called on
> + * @obj.
> + *
> + * Unlocked by calling dma_resv_unlock().
> */
> static inline int dma_resv_lock_interruptible(struct dma_resv *obj,
> struct ww_acquire_ctx *ctx)
> @@ -134,6 +221,8 @@ static inline int dma_resv_lock_interruptible(struct dma_resv *obj,
> * Acquires the reservation object after a die case. This function
> * will sleep until the lock becomes available. See dma_resv_lock() as
> * well.
> + *
> + * See also dma_resv_lock_slow_interruptible() for the interruptible variant.
> */
> static inline void dma_resv_lock_slow(struct dma_resv *obj,
> struct ww_acquire_ctx *ctx)
> @@ -167,7 +256,7 @@ static inline int dma_resv_lock_slow_interruptible(struct dma_resv *obj,
> * if they overlap with a writer.
> *
> * Also note that since no context is provided, no deadlock protection is
> - * possible.
> + * possible, which is also not needed for a trylock.
> *
> * Returns true if the lock was acquired, false otherwise.
> */
> @@ -193,6 +282,11 @@ static inline bool dma_resv_is_locked(struct dma_resv *obj)
> *
> * Returns the context used to lock a reservation object or NULL if no context
> * was used or the object is not locked at all.
> + *
> + * WARNING: This interface is pretty horrible, but TTM needs it because it
> + * doesn't pass the struct ww_acquire_ctx around in some very long callchains.
> + * Everyone else just uses it to check whether they're holding a reservation or
> + * not.
> */
> static inline struct ww_acquire_ctx *dma_resv_locking_ctx(struct dma_resv *obj)
> {
> --
> 2.32.0
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 122+ messages in thread
* [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/sched dependency handling and implicit sync fixes
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
` (20 preceding siblings ...)
(?)
@ 2021-08-05 13:58 ` Patchwork
-1 siblings, 0 replies; 122+ messages in thread
From: Patchwork @ 2021-08-05 13:58 UTC (permalink / raw)
To: Daniel Vetter; +Cc: intel-gfx
== Series Details ==
Series: drm/sched dependency handling and implicit sync fixes
URL : https://patchwork.freedesktop.org/series/93415/
State : warning
== Summary ==
$ dim checkpatch origin/drm-tip
fbee9b54db37 drm/sched: Split drm_sched_job_init
-:237: WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of 'unsigned'
#237: FILE: drivers/gpu/drm/scheduler/sched_fence.c:173:
+ unsigned seq;
-:333: WARNING:AVOID_BUG: Avoid crashing the kernel - try using WARN_ON & recovery code rather than BUG() or BUG_ON()
#333: FILE: drivers/gpu/drm/scheduler/sched_main.c:623:
+ BUG_ON(!entity);
-:402: CHECK:OPEN_ENDED_LINE: Lines should not end with a '('
#402: FILE: include/drm/gpu_scheduler.h:391:
+struct drm_sched_fence *drm_sched_fence_alloc(
-:410: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 3 warnings, 1 checks, 249 lines checked
e8aa4762329c drm/msm: Fix drm/sched point of no return rules
-:74: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 39 lines checked
590f3db49271 drm/sched: Barriers are needed for entity->last_scheduled
-:88: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 43 lines checked
bcde98968570 drm/sched: Add dependency tracking
-:195: CHECK:LINE_SPACING: Please don't use multiple blank lines
#195: FILE: drivers/gpu/drm/scheduler/sched_main.c:729:
+
+
-:271: WARNING:TYPO_SPELLING: 'ommitted' may be misspelled - perhaps 'omitted'?
#271: FILE: include/drm/gpu_scheduler.h:244:
+ * drm_sched_job_add_implicit_dependencies() this can be ommitted and
^^^^^^^^
-:286: CHECK:LINE_SPACING: Please don't use multiple blank lines
#286: FILE: include/drm/gpu_scheduler.h:378:
+
+
-:289: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 2 checks, 230 lines checked
4578cd7fba03 drm/sched: drop entity parameter from drm_sched_push_job
-:227: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 110 lines checked
3ed6103f784e drm/sched: improve docs around drm_sched_entity
-:17: ERROR:GIT_COMMIT_ID: Please use git commit description style 'commit <12+ chars of sha1> ("<title line>")' - ie: 'commit 620e762f9a98 ("drm/scheduler: move entity handling into separate file")'
#17:
move here: 620e762f9a98 ("drm/scheduler: move entity handling into
-:413: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 1 errors, 1 warnings, 0 checks, 346 lines checked
42d75ea77670 drm/panfrost: use scheduler dependency tracking
-:214: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 158 lines checked
1074a5fc4e39 drm/lima: use scheduler dependency tracking
-:118: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 75 lines checked
e94f03075601 drm/v3d: Move drm_sched_job_init to v3d_job_init
-:344: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 288 lines checked
43177403c5b4 drm/v3d: Use scheduler dependency handling
-:207: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 162 lines checked
94e8ea6bdac4 drm/etnaviv: Use scheduler dependency handling
-:13: WARNING:REPEATED_WORD: Possible repeated word: 'to'
#13:
I wanted to to in the previous round (and did, for all other drivers).
-:122: WARNING:LINE_SPACING: Missing a blank line after declarations
#122: FILE: drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c:552:
+ struct dma_fence *in_fence = sync_file_get_fence(args->fence_fd);
+ if (!in_fence) {
-:297: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 3 warnings, 0 checks, 243 lines checked
914d59644238 drm/msm: Use scheduler dependency handling
-:132: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 90 lines checked
c4596cb48171 drm/gem: Delete gem array fencing helpers
-:141: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 14 lines checked
a0a15e60e8a4 drm/sched: Don't store self-dependencies
-:19: WARNING:TYPO_SPELLING: 'wont' may be misspelled - perhaps 'won't'?
#19:
drivers wont miss it.
^^^^
-:47: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 0 checks, 13 lines checked
7768c5a01737 drm/sched: Check locking in drm_sched_job_await_implicit
-:33: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 8 lines checked
0691454484ff drm/msm: Don't break exclusive fence ordering
-:43: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 9 lines checked
8d1e08eee56b drm/etnaviv: Don't break exclusive fence ordering
-:20: WARNING:COMMIT_LOG_LONG_LINE: Possible unwrapped commit description (prefer a maximum 75 chars per line)
#20:
https://lore.kernel.org/dri-devel/20210610210925.642582-7-jason@jlekstrand.net/
-:58: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 0 checks, 24 lines checked
9a678298fbf9 drm/i915: delete exclude argument from i915_sw_fence_await_reservation
-:12: ERROR:GIT_COMMIT_ID: Please use git commit description style 'commit <12+ chars of sha1> ("<title line>")' - ie: 'commit d07f0e59b2c7 ("drm/i915: Move GEM activity tracking into a common struct reservation_object")'
#12:
commit d07f0e59b2c762584478920cd2d11fba2980a94a
-:100: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 1 errors, 1 warnings, 0 checks, 55 lines checked
c023ae95f5f1 drm/i915: Don't break exclusive fence ordering
-:23: WARNING:COMMIT_LOG_LONG_LINE: Possible unwrapped commit description (prefer a maximum 75 chars per line)
#23:
https://lore.kernel.org/dri-devel/20210610210925.642582-7-jason@jlekstrand.net/
-:56: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 0 checks, 18 lines checked
2836aa5b3f16 dma-resv: Give the docs a do-over
-:290: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 216 lines checked
^ permalink raw reply [flat|nested] 122+ messages in thread* [Intel-gfx] ✗ Fi.CI.BAT: failure for drm/sched dependency handling and implicit sync fixes
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
` (21 preceding siblings ...)
(?)
@ 2021-08-05 14:29 ` Patchwork
-1 siblings, 0 replies; 122+ messages in thread
From: Patchwork @ 2021-08-05 14:29 UTC (permalink / raw)
To: Daniel Vetter; +Cc: intel-gfx
[-- Attachment #1: Type: text/plain, Size: 4271 bytes --]
== Series Details ==
Series: drm/sched dependency handling and implicit sync fixes
URL : https://patchwork.freedesktop.org/series/93415/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_10450 -> Patchwork_20773
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_20773 absolutely need to be
verified manually.
If you think the reported changes have nothing to do with the changes
introduced in Patchwork_20773, please notify your bug team to allow them
to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20773/index.html
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_20773:
### IGT changes ###
#### Possible regressions ####
* igt@i915_selftest@live@workarounds:
- fi-rkl-guc: [PASS][1] -> [DMESG-FAIL][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10450/fi-rkl-guc/igt@i915_selftest@live@workarounds.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20773/fi-rkl-guc/igt@i915_selftest@live@workarounds.html
Known issues
------------
Here are the changes found in Patchwork_20773 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@amdgpu/amd_basic@query-info:
- fi-bsw-kefka: NOTRUN -> [SKIP][3] ([fdo#109271]) +17 similar issues
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20773/fi-bsw-kefka/igt@amdgpu/amd_basic@query-info.html
* igt@runner@aborted:
- fi-rkl-guc: NOTRUN -> [FAIL][4] ([i915#3462])
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20773/fi-rkl-guc/igt@runner@aborted.html
#### Possible fixes ####
* igt@i915_selftest@live@execlists:
- fi-bsw-kefka: [INCOMPLETE][5] ([i915#2940]) -> [PASS][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10450/fi-bsw-kefka/igt@i915_selftest@live@execlists.html
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20773/fi-bsw-kefka/igt@i915_selftest@live@execlists.html
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[i915#2940]: https://gitlab.freedesktop.org/drm/intel/issues/2940
[i915#3462]: https://gitlab.freedesktop.org/drm/intel/issues/3462
Participating hosts (40 -> 34)
------------------------------
Missing (6): fi-ilk-m540 fi-hsw-4200u fi-bsw-cyan fi-ctg-p8600 fi-bdw-samus bat-jsl-1
Build changes
-------------
* Linux: CI_DRM_10450 -> Patchwork_20773
CI-20190529: 20190529
CI_DRM_10450: 51d9c8293e8446e921b74d996982ade862fcfa5c @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6160: 4287344dd6a39d9036c5fb9a047a7d8f10bee981 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_20773: 2836aa5b3f16292d5043778e200038dc658fa8b1 @ git://anongit.freedesktop.org/gfx-ci/linux
== Linux commits ==
2836aa5b3f16 dma-resv: Give the docs a do-over
c023ae95f5f1 drm/i915: Don't break exclusive fence ordering
9a678298fbf9 drm/i915: delete exclude argument from i915_sw_fence_await_reservation
8d1e08eee56b drm/etnaviv: Don't break exclusive fence ordering
0691454484ff drm/msm: Don't break exclusive fence ordering
7768c5a01737 drm/sched: Check locking in drm_sched_job_await_implicit
a0a15e60e8a4 drm/sched: Don't store self-dependencies
c4596cb48171 drm/gem: Delete gem array fencing helpers
914d59644238 drm/msm: Use scheduler dependency handling
94e8ea6bdac4 drm/etnaviv: Use scheduler dependency handling
43177403c5b4 drm/v3d: Use scheduler dependency handling
e94f03075601 drm/v3d: Move drm_sched_job_init to v3d_job_init
1074a5fc4e39 drm/lima: use scheduler dependency tracking
42d75ea77670 drm/panfrost: use scheduler dependency tracking
3ed6103f784e drm/sched: improve docs around drm_sched_entity
4578cd7fba03 drm/sched: drop entity parameter from drm_sched_push_job
bcde98968570 drm/sched: Add dependency tracking
590f3db49271 drm/sched: Barriers are needed for entity->last_scheduled
e8aa4762329c drm/msm: Fix drm/sched point of no return rules
fbee9b54db37 drm/sched: Split drm_sched_job_init
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20773/index.html
[-- Attachment #2: Type: text/html, Size: 5082 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread* [Intel-gfx] ✗ Fi.CI.BUILD: failure for drm/sched dependency handling and implicit sync fixes (rev2)
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
` (22 preceding siblings ...)
(?)
@ 2021-08-06 19:14 ` Patchwork
-1 siblings, 0 replies; 122+ messages in thread
From: Patchwork @ 2021-08-06 19:14 UTC (permalink / raw)
To: Rob Clark; +Cc: intel-gfx
== Series Details ==
Series: drm/sched dependency handling and implicit sync fixes (rev2)
URL : https://patchwork.freedesktop.org/series/93415/
State : failure
== Summary ==
Applying: drm/sched: Split drm_sched_job_init
Applying: drm/msm: Fix drm/sched point of no return rules
error: git diff header lacks filename information when removing 1 leading pathname component (line 2)
error: could not build fake ancestor
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0002 drm/msm: Fix drm/sched point of no return rules
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
^ permalink raw reply [flat|nested] 122+ messages in thread* [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/sched dependency handling and implicit sync fixes (rev4)
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
` (23 preceding siblings ...)
(?)
@ 2021-08-17 16:27 ` Patchwork
-1 siblings, 0 replies; 122+ messages in thread
From: Patchwork @ 2021-08-17 16:27 UTC (permalink / raw)
To: Daniel Vetter; +Cc: intel-gfx
== Series Details ==
Series: drm/sched dependency handling and implicit sync fixes (rev4)
URL : https://patchwork.freedesktop.org/series/93415/
State : warning
== Summary ==
$ dim checkpatch origin/drm-tip
0d891258e40a drm/sched: Split drm_sched_job_init
-:240: WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of 'unsigned'
#240: FILE: drivers/gpu/drm/scheduler/sched_fence.c:173:
+ unsigned seq;
-:336: WARNING:AVOID_BUG: Avoid crashing the kernel - try using WARN_ON & recovery code rather than BUG() or BUG_ON()
#336: FILE: drivers/gpu/drm/scheduler/sched_main.c:623:
+ BUG_ON(!entity);
-:405: CHECK:OPEN_ENDED_LINE: Lines should not end with a '('
#405: FILE: include/drm/gpu_scheduler.h:391:
+struct drm_sched_fence *drm_sched_fence_alloc(
-:413: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 3 warnings, 1 checks, 248 lines checked
27dbe1a630f0 drm/msm: Improve drm/sched point of no return rules
-:78: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 37 lines checked
5b5164ff17f2 drm/sched: Barriers are needed for entity->last_scheduled
-:88: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 43 lines checked
255f53586a60 drm/sched: Add dependency tracking
-:195: CHECK:LINE_SPACING: Please don't use multiple blank lines
#195: FILE: drivers/gpu/drm/scheduler/sched_main.c:729:
+
+
-:271: WARNING:TYPO_SPELLING: 'ommitted' may be misspelled - perhaps 'omitted'?
#271: FILE: include/drm/gpu_scheduler.h:244:
+ * drm_sched_job_add_implicit_dependencies() this can be ommitted and
^^^^^^^^
-:286: CHECK:LINE_SPACING: Please don't use multiple blank lines
#286: FILE: include/drm/gpu_scheduler.h:378:
+
+
-:289: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 2 checks, 230 lines checked
ebfbb6077485 drm/sched: drop entity parameter from drm_sched_push_job
-:228: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 110 lines checked
f4b4e005b964 drm/sched: improve docs around drm_sched_entity
-:17: ERROR:GIT_COMMIT_ID: Please use git commit description style 'commit <12+ chars of sha1> ("<title line>")' - ie: 'commit 620e762f9a98 ("drm/scheduler: move entity handling into separate file")'
#17:
move here: 620e762f9a98 ("drm/scheduler: move entity handling into
-:413: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 1 errors, 1 warnings, 0 checks, 346 lines checked
6687763f729e drm/panfrost: use scheduler dependency tracking
-:215: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 158 lines checked
ca1c5aec7cde drm/lima: use scheduler dependency tracking
-:119: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 75 lines checked
1663a9467a04 drm/v3d: Move drm_sched_job_init to v3d_job_init
-:344: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 288 lines checked
2c445c2eb7fe drm/v3d: Use scheduler dependency handling
-:207: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 162 lines checked
e63e89cc8ac5 drm/etnaviv: Use scheduler dependency handling
-:13: WARNING:REPEATED_WORD: Possible repeated word: 'to'
#13:
I wanted to to in the previous round (and did, for all other drivers).
-:122: WARNING:LINE_SPACING: Missing a blank line after declarations
#122: FILE: drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c:552:
+ struct dma_fence *in_fence = sync_file_get_fence(args->fence_fd);
+ if (!in_fence) {
-:297: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 3 warnings, 0 checks, 243 lines checked
9f2a9ecfea9c drm/msm: Use scheduler dependency handling
-:132: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 90 lines checked
e5b7840798ca drm/gem: Delete gem array fencing helpers
-:141: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 14 lines checked
8f51a69d4e74 drm/sched: Don't store self-dependencies
-:19: WARNING:TYPO_SPELLING: 'wont' may be misspelled - perhaps 'won't'?
#19:
drivers wont miss it.
^^^^
-:47: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 0 checks, 13 lines checked
7650a6f74b18 drm/sched: Check locking in drm_sched_job_await_implicit
-:33: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 8 lines checked
a3c9153955a4 drm/msm: Don't break exclusive fence ordering
-:43: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 9 lines checked
2eb5ca447466 drm/etnaviv: Don't break exclusive fence ordering
-:20: WARNING:COMMIT_LOG_LONG_LINE: Possible unwrapped commit description (prefer a maximum 75 chars per line)
#20:
https://lore.kernel.org/dri-devel/20210610210925.642582-7-jason@jlekstrand.net/
-:58: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 0 checks, 24 lines checked
ab0a3f52cac3 drm/i915: delete exclude argument from i915_sw_fence_await_reservation
-:12: ERROR:GIT_COMMIT_ID: Please use git commit description style 'commit <12+ chars of sha1> ("<title line>")' - ie: 'commit d07f0e59b2c7 ("drm/i915: Move GEM activity tracking into a common struct reservation_object")'
#12:
commit d07f0e59b2c762584478920cd2d11fba2980a94a
-:100: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 1 errors, 1 warnings, 0 checks, 55 lines checked
8c4011c29394 drm/i915: Don't break exclusive fence ordering
-:23: WARNING:COMMIT_LOG_LONG_LINE: Possible unwrapped commit description (prefer a maximum 75 chars per line)
#23:
https://lore.kernel.org/dri-devel/20210610210925.642582-7-jason@jlekstrand.net/
-:56: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 0 checks, 18 lines checked
724ee6ec9713 dma-resv: Give the docs a do-over
-:290: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 216 lines checked
^ permalink raw reply [flat|nested] 122+ messages in thread* [Intel-gfx] ✓ Fi.CI.BAT: success for drm/sched dependency handling and implicit sync fixes (rev4)
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
` (24 preceding siblings ...)
(?)
@ 2021-08-17 16:57 ` Patchwork
-1 siblings, 0 replies; 122+ messages in thread
From: Patchwork @ 2021-08-17 16:57 UTC (permalink / raw)
To: Daniel Vetter; +Cc: intel-gfx
[-- Attachment #1: Type: text/plain, Size: 3334 bytes --]
== Series Details ==
Series: drm/sched dependency handling and implicit sync fixes (rev4)
URL : https://patchwork.freedesktop.org/series/93415/
State : success
== Summary ==
CI Bug Log - changes from CI_DRM_10490 -> Patchwork_20836
====================================================
Summary
-------
**SUCCESS**
No regressions found.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/index.html
Known issues
------------
Here are the changes found in Patchwork_20836 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@amdgpu/amd_basic@semaphore:
- fi-bdw-5557u: NOTRUN -> [SKIP][1] ([fdo#109271]) +27 similar issues
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/fi-bdw-5557u/igt@amdgpu/amd_basic@semaphore.html
* igt@core_hotunplug@unbind-rebind:
- fi-bdw-5557u: NOTRUN -> [WARN][2] ([i915#3718])
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/fi-bdw-5557u/igt@core_hotunplug@unbind-rebind.html
* igt@kms_chamelium@dp-crc-fast:
- fi-bdw-5557u: NOTRUN -> [SKIP][3] ([fdo#109271] / [fdo#111827]) +8 similar issues
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/fi-bdw-5557u/igt@kms_chamelium@dp-crc-fast.html
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[i915#3718]: https://gitlab.freedesktop.org/drm/intel/issues/3718
Participating hosts (36 -> 34)
------------------------------
Missing (2): fi-bsw-cyan fi-bdw-samus
Build changes
-------------
* Linux: CI_DRM_10490 -> Patchwork_20836
CI-20190529: 20190529
CI_DRM_10490: 3bd74b377986fcb89cf4563629f97c5b3199ca6f @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6177: f474644e7226dd319195ca03b3cde82ad10ac54c @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_20836: 724ee6ec97135c8a4fd57f8b19d9802834ad62fc @ git://anongit.freedesktop.org/gfx-ci/linux
== Linux commits ==
724ee6ec9713 dma-resv: Give the docs a do-over
8c4011c29394 drm/i915: Don't break exclusive fence ordering
ab0a3f52cac3 drm/i915: delete exclude argument from i915_sw_fence_await_reservation
2eb5ca447466 drm/etnaviv: Don't break exclusive fence ordering
a3c9153955a4 drm/msm: Don't break exclusive fence ordering
7650a6f74b18 drm/sched: Check locking in drm_sched_job_await_implicit
8f51a69d4e74 drm/sched: Don't store self-dependencies
e5b7840798ca drm/gem: Delete gem array fencing helpers
9f2a9ecfea9c drm/msm: Use scheduler dependency handling
e63e89cc8ac5 drm/etnaviv: Use scheduler dependency handling
2c445c2eb7fe drm/v3d: Use scheduler dependency handling
1663a9467a04 drm/v3d: Move drm_sched_job_init to v3d_job_init
ca1c5aec7cde drm/lima: use scheduler dependency tracking
6687763f729e drm/panfrost: use scheduler dependency tracking
f4b4e005b964 drm/sched: improve docs around drm_sched_entity
ebfbb6077485 drm/sched: drop entity parameter from drm_sched_push_job
255f53586a60 drm/sched: Add dependency tracking
5b5164ff17f2 drm/sched: Barriers are needed for entity->last_scheduled
27dbe1a630f0 drm/msm: Improve drm/sched point of no return rules
0d891258e40a drm/sched: Split drm_sched_job_init
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/index.html
[-- Attachment #2: Type: text/html, Size: 4169 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread* [Intel-gfx] ✗ Fi.CI.IGT: failure for drm/sched dependency handling and implicit sync fixes (rev4)
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
` (25 preceding siblings ...)
(?)
@ 2021-08-17 18:15 ` Patchwork
-1 siblings, 0 replies; 122+ messages in thread
From: Patchwork @ 2021-08-17 18:15 UTC (permalink / raw)
To: Daniel Vetter; +Cc: intel-gfx
[-- Attachment #1: Type: text/plain, Size: 30283 bytes --]
== Series Details ==
Series: drm/sched dependency handling and implicit sync fixes (rev4)
URL : https://patchwork.freedesktop.org/series/93415/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_10490_full -> Patchwork_20836_full
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_20836_full absolutely need to be
verified manually.
If you think the reported changes have nothing to do with the changes
introduced in Patchwork_20836_full, please notify your bug team to allow them
to document this new failure mode, which will reduce false positives in CI.
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_20836_full:
### IGT changes ###
#### Possible regressions ####
* igt@gem_exec_async@concurrent-writes@bcs0:
- shard-tglb: [PASS][1] -> [FAIL][2] +4 similar issues
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-tglb3/igt@gem_exec_async@concurrent-writes@bcs0.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb7/igt@gem_exec_async@concurrent-writes@bcs0.html
* igt@gem_exec_async@concurrent-writes@vcs1:
- shard-kbl: [PASS][3] -> [FAIL][4] +1 similar issue
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-kbl7/igt@gem_exec_async@concurrent-writes@vcs1.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl4/igt@gem_exec_async@concurrent-writes@vcs1.html
* igt@gem_exec_async@concurrent-writes@vecs0:
- shard-iclb: [PASS][5] -> [FAIL][6] +3 similar issues
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-iclb7/igt@gem_exec_async@concurrent-writes@vecs0.html
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb7/igt@gem_exec_async@concurrent-writes@vecs0.html
* igt@gem_exec_async@forked-writes@bcs0:
- shard-snb: [PASS][7] -> [FAIL][8] +2 similar issues
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-snb5/igt@gem_exec_async@forked-writes@bcs0.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-snb6/igt@gem_exec_async@forked-writes@bcs0.html
Known issues
------------
Here are the changes found in Patchwork_20836_full that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_create@create-massive:
- shard-snb: NOTRUN -> [DMESG-WARN][9] ([i915#3002])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-snb6/igt@gem_create@create-massive.html
* igt@gem_ctx_persistence@legacy-engines-queued:
- shard-snb: NOTRUN -> [SKIP][10] ([fdo#109271] / [i915#1099]) +4 similar issues
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-snb2/igt@gem_ctx_persistence@legacy-engines-queued.html
* igt@gem_exec_fair@basic-deadline:
- shard-apl: NOTRUN -> [FAIL][11] ([i915#2846])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl2/igt@gem_exec_fair@basic-deadline.html
* igt@gem_exec_fair@basic-none-solo@rcs0:
- shard-kbl: [PASS][12] -> [FAIL][13] ([i915#2842]) +3 similar issues
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-kbl6/igt@gem_exec_fair@basic-none-solo@rcs0.html
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl7/igt@gem_exec_fair@basic-none-solo@rcs0.html
* igt@gem_exec_fair@basic-none-vip@rcs0:
- shard-glk: [PASS][14] -> [FAIL][15] ([i915#2842])
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-glk5/igt@gem_exec_fair@basic-none-vip@rcs0.html
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-glk3/igt@gem_exec_fair@basic-none-vip@rcs0.html
* igt@gem_exec_fair@basic-pace@vcs0:
- shard-kbl: [PASS][16] -> [SKIP][17] ([fdo#109271])
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-kbl7/igt@gem_exec_fair@basic-pace@vcs0.html
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl3/igt@gem_exec_fair@basic-pace@vcs0.html
* igt@gem_mmap_gtt@cpuset-big-copy-odd:
- shard-iclb: [PASS][18] -> [FAIL][19] ([i915#2428])
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-iclb6/igt@gem_mmap_gtt@cpuset-big-copy-odd.html
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb6/igt@gem_mmap_gtt@cpuset-big-copy-odd.html
* igt@gem_pread@exhaustion:
- shard-apl: NOTRUN -> [WARN][20] ([i915#2658])
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl6/igt@gem_pread@exhaustion.html
* igt@gem_pwrite@basic-exhaustion:
- shard-tglb: NOTRUN -> [WARN][21] ([i915#2658])
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb5/igt@gem_pwrite@basic-exhaustion.html
* igt@gem_render_copy@y-tiled-to-vebox-y-tiled:
- shard-iclb: NOTRUN -> [SKIP][22] ([i915#768])
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@gem_render_copy@y-tiled-to-vebox-y-tiled.html
* igt@gem_userptr_blits@access-control:
- shard-tglb: NOTRUN -> [SKIP][23] ([i915#3297]) +1 similar issue
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb6/igt@gem_userptr_blits@access-control.html
- shard-iclb: NOTRUN -> [SKIP][24] ([i915#3297]) +1 similar issue
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@gem_userptr_blits@access-control.html
* igt@kms_addfb_basic@invalid-smem-bo-on-discrete:
- shard-tglb: NOTRUN -> [SKIP][25] ([i915#3826])
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb6/igt@kms_addfb_basic@invalid-smem-bo-on-discrete.html
- shard-iclb: NOTRUN -> [SKIP][26] ([i915#3826])
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_addfb_basic@invalid-smem-bo-on-discrete.html
* igt@kms_atomic@plane-primary-overlay-mutable-zpos:
- shard-tglb: NOTRUN -> [SKIP][27] ([i915#404])
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb6/igt@kms_atomic@plane-primary-overlay-mutable-zpos.html
- shard-iclb: NOTRUN -> [SKIP][28] ([i915#404])
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_atomic@plane-primary-overlay-mutable-zpos.html
* igt@kms_big_fb@x-tiled-max-hw-stride-32bpp-rotate-0-hflip:
- shard-skl: NOTRUN -> [SKIP][29] ([fdo#109271] / [i915#3777])
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl8/igt@kms_big_fb@x-tiled-max-hw-stride-32bpp-rotate-0-hflip.html
* igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-0-hflip:
- shard-apl: NOTRUN -> [SKIP][30] ([fdo#109271] / [i915#3777]) +2 similar issues
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl2/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-0-hflip.html
* igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-0-hflip-async-flip:
- shard-iclb: [PASS][31] -> [DMESG-WARN][32] ([i915#3621])
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-iclb4/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-0-hflip-async-flip.html
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb1/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-0-hflip-async-flip.html
* igt@kms_big_fb@yf-tiled-8bpp-rotate-180:
- shard-tglb: NOTRUN -> [SKIP][33] ([fdo#111615])
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb5/igt@kms_big_fb@yf-tiled-8bpp-rotate-180.html
* igt@kms_ccs@pipe-a-ccs-on-another-bo-y_tiled_gen12_mc_ccs:
- shard-apl: NOTRUN -> [SKIP][34] ([fdo#109271] / [i915#3886]) +8 similar issues
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl6/igt@kms_ccs@pipe-a-ccs-on-another-bo-y_tiled_gen12_mc_ccs.html
* igt@kms_ccs@pipe-b-crc-primary-rotation-180-y_tiled_ccs:
- shard-tglb: NOTRUN -> [SKIP][35] ([i915#3689]) +2 similar issues
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb6/igt@kms_ccs@pipe-b-crc-primary-rotation-180-y_tiled_ccs.html
* igt@kms_ccs@pipe-b-crc-sprite-planes-basic-y_tiled_gen12_rc_ccs_cc:
- shard-iclb: NOTRUN -> [SKIP][36] ([fdo#109278] / [i915#3886])
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_ccs@pipe-b-crc-sprite-planes-basic-y_tiled_gen12_rc_ccs_cc.html
* igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc:
- shard-kbl: NOTRUN -> [SKIP][37] ([fdo#109271] / [i915#3886]) +1 similar issue
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl4/igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc.html
* igt@kms_ccs@pipe-c-crc-sprite-planes-basic-y_tiled_gen12_mc_ccs:
- shard-tglb: NOTRUN -> [SKIP][38] ([i915#3689] / [i915#3886])
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb5/igt@kms_ccs@pipe-c-crc-sprite-planes-basic-y_tiled_gen12_mc_ccs.html
* igt@kms_ccs@pipe-d-bad-pixel-format-y_tiled_ccs:
- shard-snb: NOTRUN -> [SKIP][39] ([fdo#109271]) +301 similar issues
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-snb6/igt@kms_ccs@pipe-d-bad-pixel-format-y_tiled_ccs.html
* igt@kms_ccs@pipe-d-crc-primary-rotation-180-y_tiled_ccs:
- shard-iclb: NOTRUN -> [SKIP][40] ([fdo#109278])
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_ccs@pipe-d-crc-primary-rotation-180-y_tiled_ccs.html
* igt@kms_cdclk@plane-scaling:
- shard-iclb: NOTRUN -> [SKIP][41] ([i915#3742])
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_cdclk@plane-scaling.html
- shard-tglb: NOTRUN -> [SKIP][42] ([i915#3742])
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb6/igt@kms_cdclk@plane-scaling.html
* igt@kms_chamelium@dp-audio-edid:
- shard-skl: NOTRUN -> [SKIP][43] ([fdo#109271] / [fdo#111827]) +1 similar issue
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl5/igt@kms_chamelium@dp-audio-edid.html
* igt@kms_color@pipe-c-ctm-0-25:
- shard-skl: [PASS][44] -> [DMESG-WARN][45] ([i915#1982]) +1 similar issue
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-skl5/igt@kms_color@pipe-c-ctm-0-25.html
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl7/igt@kms_color@pipe-c-ctm-0-25.html
* igt@kms_color_chamelium@pipe-a-ctm-0-25:
- shard-snb: NOTRUN -> [SKIP][46] ([fdo#109271] / [fdo#111827]) +17 similar issues
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-snb2/igt@kms_color_chamelium@pipe-a-ctm-0-25.html
* igt@kms_color_chamelium@pipe-a-ctm-0-5:
- shard-apl: NOTRUN -> [SKIP][47] ([fdo#109271] / [fdo#111827]) +17 similar issues
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl6/igt@kms_color_chamelium@pipe-a-ctm-0-5.html
* igt@kms_color_chamelium@pipe-a-ctm-blue-to-red:
- shard-kbl: NOTRUN -> [SKIP][48] ([fdo#109271] / [fdo#111827])
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl4/igt@kms_color_chamelium@pipe-a-ctm-blue-to-red.html
* igt@kms_cursor_crc@pipe-a-cursor-32x32-random:
- shard-tglb: NOTRUN -> [SKIP][49] ([i915#3319])
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb5/igt@kms_cursor_crc@pipe-a-cursor-32x32-random.html
* igt@kms_cursor_crc@pipe-a-cursor-512x512-rapid-movement:
- shard-iclb: NOTRUN -> [SKIP][50] ([fdo#109278] / [fdo#109279])
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_cursor_crc@pipe-a-cursor-512x512-rapid-movement.html
- shard-tglb: NOTRUN -> [SKIP][51] ([fdo#109279] / [i915#3359])
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb6/igt@kms_cursor_crc@pipe-a-cursor-512x512-rapid-movement.html
* igt@kms_cursor_crc@pipe-d-cursor-256x256-rapid-movement:
- shard-kbl: NOTRUN -> [SKIP][52] ([fdo#109271]) +18 similar issues
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl4/igt@kms_cursor_crc@pipe-d-cursor-256x256-rapid-movement.html
* igt@kms_cursor_legacy@pipe-d-torture-bo:
- shard-apl: NOTRUN -> [SKIP][53] ([fdo#109271] / [i915#533])
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl3/igt@kms_cursor_legacy@pipe-d-torture-bo.html
* igt@kms_dsc@basic-dsc-enable@edp-1-pipe-c:
- shard-iclb: NOTRUN -> [DMESG-WARN][54] ([i915#1226]) +2 similar issues
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_dsc@basic-dsc-enable@edp-1-pipe-c.html
* igt@kms_flip@2x-flip-vs-panning-vs-hang:
- shard-skl: NOTRUN -> [SKIP][55] ([fdo#109271]) +23 similar issues
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl8/igt@kms_flip@2x-flip-vs-panning-vs-hang.html
* igt@kms_flip@2x-plain-flip:
- shard-iclb: NOTRUN -> [SKIP][56] ([fdo#109274])
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_flip@2x-plain-flip.html
* igt@kms_flip@flip-vs-expired-vblank-interruptible@a-edp1:
- shard-skl: [PASS][57] -> [FAIL][58] ([i915#79])
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-skl10/igt@kms_flip@flip-vs-expired-vblank-interruptible@a-edp1.html
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl8/igt@kms_flip@flip-vs-expired-vblank-interruptible@a-edp1.html
* igt@kms_flip@flip-vs-suspend-interruptible@a-dp1:
- shard-kbl: [PASS][59] -> [DMESG-WARN][60] ([i915#180]) +4 similar issues
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-kbl3/igt@kms_flip@flip-vs-suspend-interruptible@a-dp1.html
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl1/igt@kms_flip@flip-vs-suspend-interruptible@a-dp1.html
* igt@kms_flip@flip-vs-suspend-interruptible@a-hdmi-a1:
- shard-glk: [PASS][61] -> [INCOMPLETE][62] ([i915#2199])
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-glk3/igt@kms_flip@flip-vs-suspend-interruptible@a-hdmi-a1.html
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-glk7/igt@kms_flip@flip-vs-suspend-interruptible@a-hdmi-a1.html
* igt@kms_frontbuffer_tracking@fbc-2p-primscrn-spr-indfb-draw-mmap-cpu:
- shard-iclb: NOTRUN -> [SKIP][63] ([fdo#109280]) +4 similar issues
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-spr-indfb-draw-mmap-cpu.html
* igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-spr-indfb-draw-blt:
- shard-apl: NOTRUN -> [SKIP][64] ([fdo#109271]) +208 similar issues
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl6/igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-spr-indfb-draw-blt.html
* igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-cur-indfb-draw-mmap-cpu:
- shard-tglb: NOTRUN -> [SKIP][65] ([fdo#111825]) +8 similar issues
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb6/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-cur-indfb-draw-mmap-cpu.html
* igt@kms_hdr@bpc-switch:
- shard-skl: [PASS][66] -> [FAIL][67] ([i915#1188])
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-skl8/igt@kms_hdr@bpc-switch.html
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl9/igt@kms_hdr@bpc-switch.html
* igt@kms_pipe_crc_basic@read-crc-pipe-d:
- shard-kbl: NOTRUN -> [SKIP][68] ([fdo#109271] / [i915#533])
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl4/igt@kms_pipe_crc_basic@read-crc-pipe-d.html
* igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes:
- shard-apl: [PASS][69] -> [DMESG-WARN][70] ([i915#180]) +1 similar issue
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-apl1/igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes.html
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl2/igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes.html
* igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb:
- shard-apl: NOTRUN -> [FAIL][71] ([fdo#108145] / [i915#265]) +2 similar issues
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl2/igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb.html
* igt@kms_plane_alpha_blend@pipe-b-coverage-7efc:
- shard-skl: [PASS][72] -> [FAIL][73] ([fdo#108145] / [i915#265])
[72]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-skl3/igt@kms_plane_alpha_blend@pipe-b-coverage-7efc.html
[73]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl4/igt@kms_plane_alpha_blend@pipe-b-coverage-7efc.html
* igt@kms_plane_lowres@pipe-a-tiling-x:
- shard-iclb: NOTRUN -> [SKIP][74] ([i915#3536])
[74]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_plane_lowres@pipe-a-tiling-x.html
- shard-tglb: NOTRUN -> [SKIP][75] ([i915#3536])
[75]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb6/igt@kms_plane_lowres@pipe-a-tiling-x.html
* igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-2:
- shard-apl: NOTRUN -> [SKIP][76] ([fdo#109271] / [i915#658]) +3 similar issues
[76]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl8/igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-2.html
* igt@kms_psr2_sf@plane-move-sf-dmg-area-2:
- shard-tglb: NOTRUN -> [SKIP][77] ([i915#2920])
[77]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb6/igt@kms_psr2_sf@plane-move-sf-dmg-area-2.html
- shard-iclb: NOTRUN -> [SKIP][78] ([i915#2920])
[78]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_psr2_sf@plane-move-sf-dmg-area-2.html
* igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-3:
- shard-skl: NOTRUN -> [SKIP][79] ([fdo#109271] / [i915#658])
[79]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl8/igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-3.html
* igt@kms_vrr@flipline:
- shard-tglb: NOTRUN -> [SKIP][80] ([fdo#109502])
[80]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb6/igt@kms_vrr@flipline.html
- shard-iclb: NOTRUN -> [SKIP][81] ([fdo#109502])
[81]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_vrr@flipline.html
* igt@sysfs_clients@sema-10:
- shard-skl: NOTRUN -> [SKIP][82] ([fdo#109271] / [i915#2994])
[82]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl8/igt@sysfs_clients@sema-10.html
- shard-apl: NOTRUN -> [SKIP][83] ([fdo#109271] / [i915#2994]) +2 similar issues
[83]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl3/igt@sysfs_clients@sema-10.html
#### Possible fixes ####
* igt@gem_exec_fair@basic-none@vecs0:
- shard-apl: [FAIL][84] ([i915#2842] / [i915#3468]) -> [PASS][85]
[84]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-apl3/igt@gem_exec_fair@basic-none@vecs0.html
[85]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl6/igt@gem_exec_fair@basic-none@vecs0.html
* igt@gem_exec_fair@basic-pace-share@rcs0:
- shard-tglb: [FAIL][86] ([i915#2842]) -> [PASS][87] +1 similar issue
[86]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-tglb8/igt@gem_exec_fair@basic-pace-share@rcs0.html
[87]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-tglb5/igt@gem_exec_fair@basic-pace-share@rcs0.html
* igt@gem_exec_fair@basic-pace@rcs0:
- shard-kbl: [FAIL][88] ([i915#2842]) -> [PASS][89]
[88]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-kbl7/igt@gem_exec_fair@basic-pace@rcs0.html
[89]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl3/igt@gem_exec_fair@basic-pace@rcs0.html
* igt@gem_ppgtt@flink-and-close-vma-leak:
- shard-skl: [FAIL][90] ([i915#644]) -> [PASS][91]
[90]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-skl2/igt@gem_ppgtt@flink-and-close-vma-leak.html
[91]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl6/igt@gem_ppgtt@flink-and-close-vma-leak.html
* igt@i915_pm_rpm@system-suspend:
- shard-skl: [INCOMPLETE][92] ([i915#151]) -> [PASS][93]
[92]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-skl6/igt@i915_pm_rpm@system-suspend.html
[93]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl8/igt@i915_pm_rpm@system-suspend.html
* igt@kms_cursor_crc@pipe-b-cursor-suspend:
- shard-kbl: [DMESG-WARN][94] ([i915#180]) -> [PASS][95]
[94]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-kbl7/igt@kms_cursor_crc@pipe-b-cursor-suspend.html
[95]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl4/igt@kms_cursor_crc@pipe-b-cursor-suspend.html
* igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions:
- shard-skl: [FAIL][96] ([i915#2346]) -> [PASS][97]
[96]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-skl5/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html
[97]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl7/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html
* igt@kms_flip@flip-vs-expired-vblank@a-edp1:
- shard-skl: [FAIL][98] ([i915#79]) -> [PASS][99] +2 similar issues
[98]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-skl2/igt@kms_flip@flip-vs-expired-vblank@a-edp1.html
[99]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl4/igt@kms_flip@flip-vs-expired-vblank@a-edp1.html
* igt@kms_plane_alpha_blend@pipe-a-constant-alpha-min:
- shard-skl: [FAIL][100] ([fdo#108145] / [i915#265]) -> [PASS][101]
[100]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-skl4/igt@kms_plane_alpha_blend@pipe-a-constant-alpha-min.html
[101]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-skl6/igt@kms_plane_alpha_blend@pipe-a-constant-alpha-min.html
* igt@kms_psr@psr2_cursor_render:
- shard-iclb: [SKIP][102] ([fdo#109441]) -> [PASS][103]
[102]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-iclb6/igt@kms_psr@psr2_cursor_render.html
[103]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_psr@psr2_cursor_render.html
#### Warnings ####
* igt@i915_pm_rc6_residency@rc6-fence:
- shard-iclb: [WARN][104] ([i915#2684]) -> [WARN][105] ([i915#1804] / [i915#2684]) +1 similar issue
[104]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-iclb8/igt@i915_pm_rc6_residency@rc6-fence.html
[105]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb3/igt@i915_pm_rc6_residency@rc6-fence.html
* igt@i915_suspend@forcewake:
- shard-kbl: [INCOMPLETE][106] ([i915#636]) -> [INCOMPLETE][107] ([i915#155] / [i915#636])
[106]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-kbl4/igt@i915_suspend@forcewake.html
[107]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl2/igt@i915_suspend@forcewake.html
* igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-2:
- shard-iclb: [SKIP][108] ([i915#2920]) -> [SKIP][109] ([i915#658])
[108]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-iclb2/igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-2.html
[109]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb5/igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-2.html
* igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-1:
- shard-iclb: [SKIP][110] ([i915#658]) -> [SKIP][111] ([i915#2920])
[110]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-iclb6/igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-1.html
[111]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb2/igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-1.html
* igt@runner@aborted:
- shard-kbl: ([FAIL][112], [FAIL][113], [FAIL][114]) ([i915#1814] / [i915#2505] / [i915#3002] / [i915#3363]) -> ([FAIL][115], [FAIL][116], [FAIL][117], [FAIL][118], [FAIL][119]) ([i915#1436] / [i915#180] / [i915#1814] / [i915#2505] / [i915#3002] / [i915#3363])
[112]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-kbl6/igt@runner@aborted.html
[113]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-kbl3/igt@runner@aborted.html
[114]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-kbl7/igt@runner@aborted.html
[115]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl3/igt@runner@aborted.html
[116]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl6/igt@runner@aborted.html
[117]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl1/igt@runner@aborted.html
[118]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl1/igt@runner@aborted.html
[119]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-kbl1/igt@runner@aborted.html
- shard-iclb: ([FAIL][120], [FAIL][121]) ([i915#3002]) -> ([FAIL][122], [FAIL][123], [FAIL][124]) ([i915#1814] / [i915#3002])
[120]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-iclb7/igt@runner@aborted.html
[121]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-iclb5/igt@runner@aborted.html
[122]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb5/igt@runner@aborted.html
[123]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb7/igt@runner@aborted.html
[124]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-iclb1/igt@runner@aborted.html
- shard-apl: ([FAIL][125], [FAIL][126], [FAIL][127], [FAIL][128], [FAIL][129]) ([fdo#109271] / [i915#180] / [i915#1814] / [i915#3002] / [i915#3363]) -> ([FAIL][130], [FAIL][131], [FAIL][132], [FAIL][133], [FAIL][134]) ([i915#1610] / [i915#180] / [i915#1814] / [i915#3002] / [i915#3363])
[125]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-apl8/igt@runner@aborted.html
[126]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-apl1/igt@runner@aborted.html
[127]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-apl2/igt@runner@aborted.html
[128]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-apl1/igt@runner@aborted.html
[129]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10490/shard-apl6/igt@runner@aborted.html
[130]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl2/igt@runner@aborted.html
[131]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl8/igt@runner@aborted.html
[132]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl2/igt@runner@aborted.html
[133]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl8/igt@runner@aborted.html
[134]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/shard-apl8/igt@runner@aborted.html
[fdo#108145]: https://bugs.freedesktop.org/show_bug.cgi?id=108145
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#109274]: https://bugs.freedesktop.org/show_bug.cgi?id=109274
[fdo#109278]: https://bugs.freedesktop.org/show_bug.cgi?id=109278
[fdo#109279]: https://bugs.freedesktop.org/show_bug.cgi?id=109279
[fdo#109280]: https://bugs.freedesktop.org/show_bug.cgi?id=109280
[fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
[fdo#109502]: https://bugs.freedesktop.org/show_bug.cgi?id=109502
[fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
[fdo#111825]: https://bugs.freedesktop.org/show_bug.cgi?id=111825
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[i915#1099]: https://gitlab.freedesktop.org/drm/intel/issues/1099
[i915#1188]: https://gitlab.freedesktop.org/drm/intel/issues/1188
[i915#1226]: https://gitlab.freedesktop.org/drm/intel/issues/1226
[i915#1436]: https://gitlab.freedesktop.org/drm/intel/issues/1436
[i915#151]: https://gitlab.freedesktop.org/drm/intel/issues/151
[i915#155]: https://gitlab.freedesktop.org/drm/intel/issues/155
[i915#1610]: https://gitlab.freedesktop.org/drm/intel/issues/1610
[i915#180]: https://gitlab.freedesktop.org/drm/intel/issues/180
[i915#1804]: https://gitlab.freedesktop.org/drm/intel/issues/1804
[i915#1814]: https://gitlab.freedesktop.org/drm/intel/issues/1814
[i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
[i915#2199]: https://gitlab.freedesktop.org/drm/intel/issues/2199
[i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
[i915#2428]: https://gitlab.freedesktop.org/drm/intel/issues/2428
[i915#2505]: https://gitlab.freedesktop.org/drm/intel/issues/2505
[i915#265]: https://gitlab.freedesktop.org/drm/intel/issues/265
[i915#2658]: https://gitlab.freedesktop.org/drm/intel/issues/2658
[i915#2684]: https://gitlab.freedesktop.org/drm/intel/issues/2684
[i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
[i915#2846]: https://gitlab.freedesktop.org/drm/intel/issues/2846
[i915#2920]: https://gitlab.freedesktop.org/drm/intel/issues/2920
[i915#2994]: https://gitlab.freedesktop.org/drm/intel/issues/2994
[i915#3002]: https://gitlab.freedesktop.org/drm/intel/issues/3002
[i915#3297]: https://gitlab.freedesktop.org/drm/intel/issues/3297
[i915#3319]: https://gitlab.freedesktop.org/drm/intel/issues/3319
[i915#3359]: https://gitlab.freedesktop.org/drm
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20836/index.html
[-- Attachment #2: Type: text/html, Size: 37269 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread* [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/sched dependency handling and implicit sync fixes (rev5)
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
` (26 preceding siblings ...)
(?)
@ 2021-08-26 13:17 ` Patchwork
-1 siblings, 0 replies; 122+ messages in thread
From: Patchwork @ 2021-08-26 13:17 UTC (permalink / raw)
To: Daniel Vetter; +Cc: intel-gfx
== Series Details ==
Series: drm/sched dependency handling and implicit sync fixes (rev5)
URL : https://patchwork.freedesktop.org/series/93415/
State : warning
== Summary ==
$ dim checkpatch origin/drm-tip
5cdf79bc4298 drm/sched: Split drm_sched_job_init
-:240: WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of 'unsigned'
#240: FILE: drivers/gpu/drm/scheduler/sched_fence.c:173:
+ unsigned seq;
-:336: WARNING:AVOID_BUG: Avoid crashing the kernel - try using WARN_ON & recovery code rather than BUG() or BUG_ON()
#336: FILE: drivers/gpu/drm/scheduler/sched_main.c:623:
+ BUG_ON(!entity);
-:405: CHECK:OPEN_ENDED_LINE: Lines should not end with a '('
#405: FILE: include/drm/gpu_scheduler.h:391:
+struct drm_sched_fence *drm_sched_fence_alloc(
-:413: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 3 warnings, 1 checks, 248 lines checked
b5c679582c27 drm/msm: Improve drm/sched point of no return rules
-:81: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 37 lines checked
b4bf87348863 drm/sched: Barriers are needed for entity->last_scheduled
-:88: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 43 lines checked
4bfa4c96b80c drm/sched: Add dependency tracking
-:195: CHECK:LINE_SPACING: Please don't use multiple blank lines
#195: FILE: drivers/gpu/drm/scheduler/sched_main.c:729:
+
+
-:271: WARNING:TYPO_SPELLING: 'ommitted' may be misspelled - perhaps 'omitted'?
#271: FILE: include/drm/gpu_scheduler.h:244:
+ * drm_sched_job_add_implicit_dependencies() this can be ommitted and
^^^^^^^^
-:286: CHECK:LINE_SPACING: Please don't use multiple blank lines
#286: FILE: include/drm/gpu_scheduler.h:378:
+
+
-:289: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 2 checks, 230 lines checked
fcd4fea4cae3 drm/sched: drop entity parameter from drm_sched_push_job
-:228: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 110 lines checked
ea09670e5838 drm/sched: improve docs around drm_sched_entity
-:17: ERROR:GIT_COMMIT_ID: Please use git commit description style 'commit <12+ chars of sha1> ("<title line>")' - ie: 'commit 620e762f9a98 ("drm/scheduler: move entity handling into separate file")'
#17:
move here: 620e762f9a98 ("drm/scheduler: move entity handling into
-:413: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 1 errors, 1 warnings, 0 checks, 346 lines checked
e31bfc2e79ff drm/panfrost: use scheduler dependency tracking
-:215: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 158 lines checked
c8f3842d2afd drm/lima: use scheduler dependency tracking
-:119: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 75 lines checked
563fd199edf5 drm/v3d: Move drm_sched_job_init to v3d_job_init
-:344: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 288 lines checked
79d00ee75ee5 drm/v3d: Use scheduler dependency handling
-:207: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 162 lines checked
164bbe5be15b drm/etnaviv: Use scheduler dependency handling
-:13: WARNING:REPEATED_WORD: Possible repeated word: 'to'
#13:
I wanted to to in the previous round (and did, for all other drivers).
-:122: WARNING:LINE_SPACING: Missing a blank line after declarations
#122: FILE: drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c:552:
+ struct dma_fence *in_fence = sync_file_get_fence(args->fence_fd);
+ if (!in_fence) {
-:297: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 3 warnings, 0 checks, 243 lines checked
2e8edd04f89c drm/msm: Use scheduler dependency handling
-:132: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 90 lines checked
9de878031434 drm/gem: Delete gem array fencing helpers
-:141: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 14 lines checked
3d4cc3335c0e drm/sched: Don't store self-dependencies
-:19: WARNING:TYPO_SPELLING: 'wont' may be misspelled - perhaps 'won't'?
#19:
drivers wont miss it.
^^^^
-:47: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 0 checks, 13 lines checked
89463913c4ad drm/sched: Check locking in drm_sched_job_await_implicit
-:33: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 8 lines checked
9578a0679e6b drm/msm: Don't break exclusive fence ordering
-:43: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 9 lines checked
335ff7e05f44 drm/etnaviv: Don't break exclusive fence ordering
-:20: WARNING:COMMIT_LOG_LONG_LINE: Possible unwrapped commit description (prefer a maximum 75 chars per line)
#20:
https://lore.kernel.org/dri-devel/20210610210925.642582-7-jason@jlekstrand.net/
-:58: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 0 checks, 24 lines checked
6f297c479190 drm/i915: delete exclude argument from i915_sw_fence_await_reservation
-:12: ERROR:GIT_COMMIT_ID: Please use git commit description style 'commit <12+ chars of sha1> ("<title line>")' - ie: 'commit d07f0e59b2c7 ("drm/i915: Move GEM activity tracking into a common struct reservation_object")'
#12:
commit d07f0e59b2c762584478920cd2d11fba2980a94a
-:100: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 1 errors, 1 warnings, 0 checks, 55 lines checked
4f9da5ea4882 drm/i915: Don't break exclusive fence ordering
-:23: WARNING:COMMIT_LOG_LONG_LINE: Possible unwrapped commit description (prefer a maximum 75 chars per line)
#23:
https://lore.kernel.org/dri-devel/20210610210925.642582-7-jason@jlekstrand.net/
-:56: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 2 warnings, 0 checks, 18 lines checked
d83acd41677b dma-resv: Give the docs a do-over
-:290: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Daniel Vetter <daniel.vetter@ffwll.ch>' != 'Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>'
total: 0 errors, 1 warnings, 0 checks, 216 lines checked
^ permalink raw reply [flat|nested] 122+ messages in thread* [Intel-gfx] ✓ Fi.CI.BAT: success for drm/sched dependency handling and implicit sync fixes (rev5)
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
` (27 preceding siblings ...)
(?)
@ 2021-08-26 13:48 ` Patchwork
-1 siblings, 0 replies; 122+ messages in thread
From: Patchwork @ 2021-08-26 13:48 UTC (permalink / raw)
To: Daniel Vetter; +Cc: intel-gfx
[-- Attachment #1: Type: text/plain, Size: 3489 bytes --]
== Series Details ==
Series: drm/sched dependency handling and implicit sync fixes (rev5)
URL : https://patchwork.freedesktop.org/series/93415/
State : success
== Summary ==
CI Bug Log - changes from CI_DRM_10525 -> Patchwork_20901
====================================================
Summary
-------
**SUCCESS**
No regressions found.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/index.html
Known issues
------------
Here are the changes found in Patchwork_20901 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@i915_selftest@live@gt_lrc:
- fi-bsw-n3050: [PASS][1] -> [DMESG-FAIL][2] ([i915#2373])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/fi-bsw-n3050/igt@i915_selftest@live@gt_lrc.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/fi-bsw-n3050/igt@i915_selftest@live@gt_lrc.html
* igt@i915_selftest@live@workarounds:
- fi-rkl-guc: [PASS][3] -> [DMESG-FAIL][4] ([i915#3928])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/fi-rkl-guc/igt@i915_selftest@live@workarounds.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/fi-rkl-guc/igt@i915_selftest@live@workarounds.html
* igt@runner@aborted:
- fi-rkl-guc: NOTRUN -> [FAIL][5] ([i915#3928])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/fi-rkl-guc/igt@runner@aborted.html
[i915#2373]: https://gitlab.freedesktop.org/drm/intel/issues/2373
[i915#3928]: https://gitlab.freedesktop.org/drm/intel/issues/3928
Participating hosts (40 -> 33)
------------------------------
Missing (7): fi-ilk-m540 bat-adls-5 fi-hsw-4200u fi-tgl-1115g4 fi-bsw-cyan fi-bdw-samus bat-jsl-1
Build changes
-------------
* Linux: CI_DRM_10525 -> Patchwork_20901
CI-20190529: 20190529
CI_DRM_10525: 059309d37ac2de5d93cf6d71fd7fe33c9c2c66ea @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6186: 250081b306c6fa8f95405fab6a7604f1968dd4ec @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_20901: d83acd41677bc9968b313704bf42b3c12ea6d018 @ git://anongit.freedesktop.org/gfx-ci/linux
== Linux commits ==
d83acd41677b dma-resv: Give the docs a do-over
4f9da5ea4882 drm/i915: Don't break exclusive fence ordering
6f297c479190 drm/i915: delete exclude argument from i915_sw_fence_await_reservation
335ff7e05f44 drm/etnaviv: Don't break exclusive fence ordering
9578a0679e6b drm/msm: Don't break exclusive fence ordering
89463913c4ad drm/sched: Check locking in drm_sched_job_await_implicit
3d4cc3335c0e drm/sched: Don't store self-dependencies
9de878031434 drm/gem: Delete gem array fencing helpers
2e8edd04f89c drm/msm: Use scheduler dependency handling
164bbe5be15b drm/etnaviv: Use scheduler dependency handling
79d00ee75ee5 drm/v3d: Use scheduler dependency handling
563fd199edf5 drm/v3d: Move drm_sched_job_init to v3d_job_init
c8f3842d2afd drm/lima: use scheduler dependency tracking
e31bfc2e79ff drm/panfrost: use scheduler dependency tracking
ea09670e5838 drm/sched: improve docs around drm_sched_entity
fcd4fea4cae3 drm/sched: drop entity parameter from drm_sched_push_job
4bfa4c96b80c drm/sched: Add dependency tracking
b4bf87348863 drm/sched: Barriers are needed for entity->last_scheduled
b5c679582c27 drm/msm: Improve drm/sched point of no return rules
5cdf79bc4298 drm/sched: Split drm_sched_job_init
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/index.html
[-- Attachment #2: Type: text/html, Size: 4332 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread* [Intel-gfx] ✗ Fi.CI.IGT: failure for drm/sched dependency handling and implicit sync fixes (rev5)
2021-08-05 10:46 ` [Intel-gfx] " Daniel Vetter
` (28 preceding siblings ...)
(?)
@ 2021-08-26 21:46 ` Patchwork
-1 siblings, 0 replies; 122+ messages in thread
From: Patchwork @ 2021-08-26 21:46 UTC (permalink / raw)
To: Daniel Vetter; +Cc: intel-gfx
[-- Attachment #1: Type: text/plain, Size: 30283 bytes --]
== Series Details ==
Series: drm/sched dependency handling and implicit sync fixes (rev5)
URL : https://patchwork.freedesktop.org/series/93415/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_10525_full -> Patchwork_20901_full
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_20901_full absolutely need to be
verified manually.
If you think the reported changes have nothing to do with the changes
introduced in Patchwork_20901_full, please notify your bug team to allow them
to document this new failure mode, which will reduce false positives in CI.
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_20901_full:
### IGT changes ###
#### Possible regressions ####
* igt@gem_exec_async@concurrent-writes@bcs0:
- shard-tglb: [PASS][1] -> [FAIL][2] +4 similar issues
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-tglb8/igt@gem_exec_async@concurrent-writes@bcs0.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb6/igt@gem_exec_async@concurrent-writes@bcs0.html
* igt@gem_exec_async@concurrent-writes@rcs0:
- shard-snb: [PASS][3] -> [FAIL][4] +2 similar issues
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-snb2/igt@gem_exec_async@concurrent-writes@rcs0.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-snb2/igt@gem_exec_async@concurrent-writes@rcs0.html
* igt@gem_exec_async@concurrent-writes@vcs0:
- shard-kbl: NOTRUN -> [FAIL][5] +3 similar issues
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl3/igt@gem_exec_async@concurrent-writes@vcs0.html
* igt@gem_exec_async@concurrent-writes@vecs0:
- shard-iclb: [PASS][6] -> [FAIL][7] +2 similar issues
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-iclb8/igt@gem_exec_async@concurrent-writes@vecs0.html
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb7/igt@gem_exec_async@concurrent-writes@vecs0.html
* igt@i915_pm_rpm@system-suspend-modeset:
- shard-glk: [PASS][8] -> [DMESG-WARN][9]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-glk4/igt@i915_pm_rpm@system-suspend-modeset.html
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-glk9/igt@i915_pm_rpm@system-suspend-modeset.html
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses.
They do not affect the overall result.
* igt@gem_exec_async@concurrent-writes@rcs0:
- {shard-rkl}: [PASS][10] -> [FAIL][11] +2 similar issues
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-rkl-2/igt@gem_exec_async@concurrent-writes@rcs0.html
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-rkl-6/igt@gem_exec_async@concurrent-writes@rcs0.html
Known issues
------------
Here are the changes found in Patchwork_20901_full that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@feature_discovery@psr2:
- shard-iclb: [PASS][12] -> [SKIP][13] ([i915#658])
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-iclb2/igt@feature_discovery@psr2.html
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb1/igt@feature_discovery@psr2.html
* igt@gem_ctx_persistence@legacy-engines-hang:
- shard-snb: NOTRUN -> [SKIP][14] ([fdo#109271] / [i915#1099])
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-snb6/igt@gem_ctx_persistence@legacy-engines-hang.html
* igt@gem_ctx_sseu@mmap-args:
- shard-tglb: NOTRUN -> [SKIP][15] ([i915#280])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@gem_ctx_sseu@mmap-args.html
* igt@gem_exec_fair@basic-deadline:
- shard-apl: NOTRUN -> [FAIL][16] ([i915#2846])
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl2/igt@gem_exec_fair@basic-deadline.html
* igt@gem_exec_fair@basic-flow@rcs0:
- shard-tglb: [PASS][17] -> [FAIL][18] ([i915#2842])
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-tglb6/igt@gem_exec_fair@basic-flow@rcs0.html
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb8/igt@gem_exec_fair@basic-flow@rcs0.html
* igt@gem_exec_fair@basic-none-solo@rcs0:
- shard-tglb: NOTRUN -> [FAIL][19] ([i915#2842])
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb6/igt@gem_exec_fair@basic-none-solo@rcs0.html
* igt@gem_exec_fair@basic-pace@rcs0:
- shard-kbl: [PASS][20] -> [FAIL][21] ([i915#2842]) +1 similar issue
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-kbl4/igt@gem_exec_fair@basic-pace@rcs0.html
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl7/igt@gem_exec_fair@basic-pace@rcs0.html
* igt@gem_exec_fair@basic-pace@vcs1:
- shard-iclb: NOTRUN -> [FAIL][22] ([i915#2842])
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb4/igt@gem_exec_fair@basic-pace@vcs1.html
* igt@gem_exec_flush@basic-batch-kernel-default-cmd:
- shard-snb: NOTRUN -> [SKIP][23] ([fdo#109271]) +204 similar issues
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-snb6/igt@gem_exec_flush@basic-batch-kernel-default-cmd.html
* igt@gem_exec_params@secure-non-master:
- shard-tglb: NOTRUN -> [SKIP][24] ([fdo#112283])
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@gem_exec_params@secure-non-master.html
* igt@gem_mmap_gtt@cpuset-big-copy:
- shard-iclb: [PASS][25] -> [FAIL][26] ([i915#307])
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-iclb8/igt@gem_mmap_gtt@cpuset-big-copy.html
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb2/igt@gem_mmap_gtt@cpuset-big-copy.html
* igt@gem_pread@exhaustion:
- shard-apl: NOTRUN -> [WARN][27] ([i915#2658]) +1 similar issue
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl1/igt@gem_pread@exhaustion.html
* igt@gem_render_copy@yf-tiled-to-vebox-linear:
- shard-skl: NOTRUN -> [SKIP][28] ([fdo#109271]) +20 similar issues
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl10/igt@gem_render_copy@yf-tiled-to-vebox-linear.html
- shard-iclb: NOTRUN -> [SKIP][29] ([i915#768])
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@gem_render_copy@yf-tiled-to-vebox-linear.html
* igt@gem_userptr_blits@dmabuf-sync:
- shard-kbl: NOTRUN -> [SKIP][30] ([fdo#109271] / [i915#3323])
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl6/igt@gem_userptr_blits@dmabuf-sync.html
* igt@gem_userptr_blits@input-checking:
- shard-snb: NOTRUN -> [DMESG-WARN][31] ([i915#3002])
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-snb7/igt@gem_userptr_blits@input-checking.html
* igt@gem_userptr_blits@readonly-pwrite-unsync:
- shard-tglb: NOTRUN -> [SKIP][32] ([i915#3297])
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb6/igt@gem_userptr_blits@readonly-pwrite-unsync.html
- shard-iclb: NOTRUN -> [SKIP][33] ([i915#3297])
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@gem_userptr_blits@readonly-pwrite-unsync.html
* igt@gen3_render_tiledy_blits:
- shard-tglb: NOTRUN -> [SKIP][34] ([fdo#109289])
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@gen3_render_tiledy_blits.html
* igt@i915_pm_dc@dc6-dpms:
- shard-kbl: NOTRUN -> [FAIL][35] ([i915#454])
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl7/igt@i915_pm_dc@dc6-dpms.html
* igt@i915_pm_lpsp@kms-lpsp@kms-lpsp-dp:
- shard-apl: NOTRUN -> [SKIP][36] ([fdo#109271] / [i915#1937])
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl1/igt@i915_pm_lpsp@kms-lpsp@kms-lpsp-dp.html
* igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait:
- shard-tglb: NOTRUN -> [SKIP][37] ([fdo#111644] / [i915#1397] / [i915#2411])
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@i915_pm_rpm@modeset-non-lpsp-stress-no-wait.html
* igt@kms_big_fb@x-tiled-max-hw-stride-32bpp-rotate-0-hflip:
- shard-skl: NOTRUN -> [SKIP][38] ([fdo#109271] / [i915#3777])
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl10/igt@kms_big_fb@x-tiled-max-hw-stride-32bpp-rotate-0-hflip.html
* igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-hflip:
- shard-kbl: NOTRUN -> [SKIP][39] ([fdo#109271] / [i915#3777])
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl6/igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-hflip.html
* igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-180-hflip:
- shard-apl: NOTRUN -> [SKIP][40] ([fdo#109271] / [i915#3777]) +2 similar issues
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl1/igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-180-hflip.html
* igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0:
- shard-apl: NOTRUN -> [SKIP][41] ([fdo#109271]) +338 similar issues
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl2/igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0.html
* igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0-hflip:
- shard-tglb: NOTRUN -> [SKIP][42] ([fdo#111615])
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0-hflip.html
* igt@kms_ccs@pipe-a-bad-pixel-format-y_tiled_gen12_rc_ccs_cc:
- shard-skl: NOTRUN -> [SKIP][43] ([fdo#109271] / [i915#3886]) +2 similar issues
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl4/igt@kms_ccs@pipe-a-bad-pixel-format-y_tiled_gen12_rc_ccs_cc.html
* igt@kms_ccs@pipe-a-crc-sprite-planes-basic-y_tiled_gen12_rc_ccs_cc:
- shard-kbl: NOTRUN -> [SKIP][44] ([fdo#109271] / [i915#3886]) +3 similar issues
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl6/igt@kms_ccs@pipe-a-crc-sprite-planes-basic-y_tiled_gen12_rc_ccs_cc.html
* igt@kms_ccs@pipe-a-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc:
- shard-apl: NOTRUN -> [SKIP][45] ([fdo#109271] / [i915#3886]) +18 similar issues
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl8/igt@kms_ccs@pipe-a-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc.html
* igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc:
- shard-iclb: NOTRUN -> [SKIP][46] ([fdo#109278] / [i915#3886])
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc.html
* igt@kms_ccs@pipe-c-random-ccs-data-y_tiled_gen12_mc_ccs:
- shard-tglb: NOTRUN -> [SKIP][47] ([i915#3689] / [i915#3886])
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb6/igt@kms_ccs@pipe-c-random-ccs-data-y_tiled_gen12_mc_ccs.html
* igt@kms_chamelium@dp-audio:
- shard-tglb: NOTRUN -> [SKIP][48] ([fdo#109284] / [fdo#111827]) +1 similar issue
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@kms_chamelium@dp-audio.html
* igt@kms_chamelium@hdmi-hpd-fast:
- shard-iclb: NOTRUN -> [SKIP][49] ([fdo#109284] / [fdo#111827]) +1 similar issue
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@kms_chamelium@hdmi-hpd-fast.html
* igt@kms_chamelium@vga-hpd:
- shard-apl: NOTRUN -> [SKIP][50] ([fdo#109271] / [fdo#111827]) +29 similar issues
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl2/igt@kms_chamelium@vga-hpd.html
* igt@kms_chamelium@vga-hpd-without-ddc:
- shard-snb: NOTRUN -> [SKIP][51] ([fdo#109271] / [fdo#111827]) +9 similar issues
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-snb7/igt@kms_chamelium@vga-hpd-without-ddc.html
* igt@kms_color@pipe-d-ctm-green-to-red:
- shard-iclb: NOTRUN -> [SKIP][52] ([fdo#109278] / [i915#1149])
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@kms_color@pipe-d-ctm-green-to-red.html
* igt@kms_color_chamelium@pipe-a-ctm-blue-to-red:
- shard-kbl: NOTRUN -> [SKIP][53] ([fdo#109271] / [fdo#111827]) +12 similar issues
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl6/igt@kms_color_chamelium@pipe-a-ctm-blue-to-red.html
* igt@kms_content_protection@atomic:
- shard-apl: NOTRUN -> [TIMEOUT][54] ([i915#1319]) +1 similar issue
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl2/igt@kms_content_protection@atomic.html
* igt@kms_content_protection@atomic-dpms:
- shard-tglb: NOTRUN -> [SKIP][55] ([fdo#111828]) +1 similar issue
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb6/igt@kms_content_protection@atomic-dpms.html
- shard-iclb: NOTRUN -> [SKIP][56] ([fdo#109300] / [fdo#111066])
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@kms_content_protection@atomic-dpms.html
* igt@kms_content_protection@uevent:
- shard-apl: NOTRUN -> [FAIL][57] ([i915#2105])
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl2/igt@kms_content_protection@uevent.html
* igt@kms_cursor_crc@pipe-a-cursor-32x32-sliding:
- shard-tglb: NOTRUN -> [SKIP][58] ([i915#3319])
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@kms_cursor_crc@pipe-a-cursor-32x32-sliding.html
* igt@kms_cursor_crc@pipe-b-cursor-size-change:
- shard-apl: NOTRUN -> [FAIL][59] ([i915#3444])
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl2/igt@kms_cursor_crc@pipe-b-cursor-size-change.html
* igt@kms_cursor_crc@pipe-c-cursor-32x10-onscreen:
- shard-tglb: NOTRUN -> [SKIP][60] ([i915#3359])
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@kms_cursor_crc@pipe-c-cursor-32x10-onscreen.html
* igt@kms_cursor_crc@pipe-c-cursor-512x512-random:
- shard-tglb: NOTRUN -> [SKIP][61] ([fdo#109279] / [i915#3359])
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@kms_cursor_crc@pipe-c-cursor-512x512-random.html
* igt@kms_cursor_legacy@cursor-vs-flip-atomic:
- shard-skl: [PASS][62] -> [DMESG-WARN][63] ([i915#1982])
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-skl10/igt@kms_cursor_legacy@cursor-vs-flip-atomic.html
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl6/igt@kms_cursor_legacy@cursor-vs-flip-atomic.html
* igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size:
- shard-skl: [PASS][64] -> [FAIL][65] ([i915#2346] / [i915#533])
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-skl8/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl8/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
* igt@kms_draw_crc@draw-method-xrgb2101010-mmap-gtt-xtiled:
- shard-skl: [PASS][66] -> [FAIL][67] ([i915#1888])
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-skl9/igt@kms_draw_crc@draw-method-xrgb2101010-mmap-gtt-xtiled.html
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl3/igt@kms_draw_crc@draw-method-xrgb2101010-mmap-gtt-xtiled.html
* igt@kms_flip@2x-single-buffer-flip-vs-dpms-off-vs-modeset-interruptible:
- shard-iclb: NOTRUN -> [SKIP][68] ([fdo#109274])
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@kms_flip@2x-single-buffer-flip-vs-dpms-off-vs-modeset-interruptible.html
* igt@kms_flip@flip-vs-expired-vblank@a-edp1:
- shard-skl: [PASS][69] -> [FAIL][70] ([i915#79])
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-skl9/igt@kms_flip@flip-vs-expired-vblank@a-edp1.html
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl3/igt@kms_flip@flip-vs-expired-vblank@a-edp1.html
* igt@kms_flip@flip-vs-suspend-interruptible@a-dp1:
- shard-kbl: [PASS][71] -> [DMESG-WARN][72] ([i915#180]) +9 similar issues
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-kbl6/igt@kms_flip@flip-vs-suspend-interruptible@a-dp1.html
[72]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl4/igt@kms_flip@flip-vs-suspend-interruptible@a-dp1.html
* igt@kms_flip@plain-flip-fb-recreate-interruptible@a-edp1:
- shard-skl: [PASS][73] -> [FAIL][74] ([i915#2122])
[73]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-skl4/igt@kms_flip@plain-flip-fb-recreate-interruptible@a-edp1.html
[74]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl10/igt@kms_flip@plain-flip-fb-recreate-interruptible@a-edp1.html
* igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytilegen12rcccs:
- shard-apl: NOTRUN -> [SKIP][75] ([fdo#109271] / [i915#2672]) +1 similar issue
[75]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl2/igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytilegen12rcccs.html
* igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-indfb-draw-mmap-gtt:
- shard-iclb: NOTRUN -> [SKIP][76] ([fdo#109280]) +2 similar issues
[76]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-indfb-draw-mmap-gtt.html
- shard-tglb: NOTRUN -> [SKIP][77] ([fdo#111825]) +2 similar issues
[77]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb6/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-indfb-draw-mmap-gtt.html
* igt@kms_frontbuffer_tracking@psr-1p-primscrn-spr-indfb-draw-mmap-cpu:
- shard-kbl: NOTRUN -> [SKIP][78] ([fdo#109271]) +104 similar issues
[78]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl7/igt@kms_frontbuffer_tracking@psr-1p-primscrn-spr-indfb-draw-mmap-cpu.html
* igt@kms_hdr@bpc-switch-dpms:
- shard-skl: [PASS][79] -> [FAIL][80] ([i915#1188])
[79]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-skl6/igt@kms_hdr@bpc-switch-dpms.html
[80]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl4/igt@kms_hdr@bpc-switch-dpms.html
* igt@kms_pipe_crc_basic@suspend-read-crc-pipe-d:
- shard-apl: NOTRUN -> [SKIP][81] ([fdo#109271] / [i915#533]) +4 similar issues
[81]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl1/igt@kms_pipe_crc_basic@suspend-read-crc-pipe-d.html
* igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb:
- shard-apl: NOTRUN -> [FAIL][82] ([fdo#108145] / [i915#265]) +4 similar issues
[82]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl6/igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb.html
* igt@kms_plane_alpha_blend@pipe-b-alpha-transparent-fb:
- shard-apl: NOTRUN -> [FAIL][83] ([i915#265])
[83]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl1/igt@kms_plane_alpha_blend@pipe-b-alpha-transparent-fb.html
* igt@kms_plane_alpha_blend@pipe-b-constant-alpha-min:
- shard-skl: [PASS][84] -> [FAIL][85] ([fdo#108145] / [i915#265])
[84]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-skl2/igt@kms_plane_alpha_blend@pipe-b-constant-alpha-min.html
[85]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl8/igt@kms_plane_alpha_blend@pipe-b-constant-alpha-min.html
* igt@kms_plane_alpha_blend@pipe-c-alpha-basic:
- shard-kbl: NOTRUN -> [FAIL][86] ([fdo#108145] / [i915#265]) +1 similar issue
[86]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl1/igt@kms_plane_alpha_blend@pipe-c-alpha-basic.html
* igt@kms_plane_cursor@pipe-d-viewport-size-64:
- shard-iclb: NOTRUN -> [SKIP][87] ([fdo#109278]) +7 similar issues
[87]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@kms_plane_cursor@pipe-d-viewport-size-64.html
* igt@kms_plane_lowres@pipe-d-tiling-none:
- shard-tglb: NOTRUN -> [SKIP][88] ([i915#3536])
[88]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@kms_plane_lowres@pipe-d-tiling-none.html
* igt@kms_prime@basic-crc@first-to-second:
- shard-tglb: NOTRUN -> [SKIP][89] ([i915#1836])
[89]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@kms_prime@basic-crc@first-to-second.html
* igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-1:
- shard-tglb: NOTRUN -> [SKIP][90] ([i915#2920])
[90]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb6/igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-1.html
- shard-skl: NOTRUN -> [SKIP][91] ([fdo#109271] / [i915#658])
[91]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl10/igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-1.html
- shard-iclb: NOTRUN -> [SKIP][92] ([i915#658])
[92]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-1.html
* igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-5:
- shard-apl: NOTRUN -> [SKIP][93] ([fdo#109271] / [i915#658]) +6 similar issues
[93]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl8/igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-5.html
* igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-5:
- shard-kbl: NOTRUN -> [SKIP][94] ([fdo#109271] / [i915#658]) +1 similar issue
[94]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl3/igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-5.html
* igt@kms_psr2_su@frontbuffer:
- shard-iclb: [PASS][95] -> [SKIP][96] ([fdo#109642] / [fdo#111068] / [i915#658])
[95]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-iclb2/igt@kms_psr2_su@frontbuffer.html
[96]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb3/igt@kms_psr2_su@frontbuffer.html
* igt@kms_psr@psr2_sprite_mmap_gtt:
- shard-iclb: NOTRUN -> [SKIP][97] ([fdo#109441])
[97]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb4/igt@kms_psr@psr2_sprite_mmap_gtt.html
* igt@kms_psr@psr2_sprite_plane_move:
- shard-iclb: [PASS][98] -> [SKIP][99] ([fdo#109441]) +3 similar issues
[98]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-iclb2/igt@kms_psr@psr2_sprite_plane_move.html
[99]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb1/igt@kms_psr@psr2_sprite_plane_move.html
* igt@kms_sysfs_edid_timing:
- shard-apl: NOTRUN -> [FAIL][100] ([IGT#2])
[100]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl2/igt@kms_sysfs_edid_timing.html
* igt@kms_vblank@pipe-b-ts-continuation-suspend:
- shard-apl: [PASS][101] -> [DMESG-WARN][102] ([i915#180])
[101]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-apl3/igt@kms_vblank@pipe-b-ts-continuation-suspend.html
[102]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl6/igt@kms_vblank@pipe-b-ts-continuation-suspend.html
* igt@kms_vblank@pipe-c-ts-continuation-dpms-suspend:
- shard-skl: [PASS][103] -> [INCOMPLETE][104] ([i915#198])
[103]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-skl4/igt@kms_vblank@pipe-c-ts-continuation-dpms-suspend.html
[104]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl10/igt@kms_vblank@pipe-c-ts-continuation-dpms-suspend.html
* igt@kms_vblank@pipe-d-wait-idle:
- shard-kbl: NOTRUN -> [SKIP][105] ([fdo#109271] / [i915#533])
[105]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl6/igt@kms_vblank@pipe-d-wait-idle.html
* igt@kms_writeback@writeback-check-output:
- shard-iclb: NOTRUN -> [SKIP][106] ([i915#2437])
[106]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb5/igt@kms_writeback@writeback-check-output.html
- shard-skl: NOTRUN -> [SKIP][107] ([fdo#109271] / [i915#2437])
[107]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl10/igt@kms_writeback@writeback-check-output.html
- shard-tglb: NOTRUN -> [SKIP][108] ([i915#2437])
[108]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb6/igt@kms_writeback@writeback-check-output.html
* igt@kms_writeback@writeback-fb-id:
- shard-apl: NOTRUN -> [SKIP][109] ([fdo#109271] / [i915#2437])
[109]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl2/igt@kms_writeback@writeback-fb-id.html
* igt@nouveau_crc@pipe-d-ctx-flip-detection:
- shard-tglb: NOTRUN -> [SKIP][110] ([i915#2530])
[110]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb6/igt@nouveau_crc@pipe-d-ctx-flip-detection.html
* igt@prime_nv_api@i915_nv_reimport_twice_check_flink_name:
- shard-tglb: NOTRUN -> [SKIP][111] ([fdo#109291])
[111]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb3/igt@prime_nv_api@i915_nv_reimport_twice_check_flink_name.html
* igt@prime_nv_test@i915_import_gtt_mmap:
- shard-iclb: NOTRUN -> [SKIP][112] ([fdo#109291])
[112]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb4/igt@prime_nv_test@i915_import_gtt_mmap.html
* igt@sysfs_clients@fair-1:
- shard-apl: NOTRUN -> [SKIP][113] ([fdo#109271] / [i915#2994]) +6 similar issues
[113]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-apl6/igt@sysfs_clients@fair-1.html
- shard-skl: NOTRUN -> [SKIP][114] ([fdo#109271] / [i915#2994])
[114]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-skl4/igt@sysfs_clients@fair-1.html
* igt@sysfs_clients@sema-10:
- shard-tglb: NOTRUN -> [SKIP][115] ([i915#2994]) +1 similar issue
[115]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb6/igt@sysfs_clients@sema-10.html
* igt@sysfs_clients@split-50:
- shard-kbl: NOTRUN -> [SKIP][116] ([fdo#109271] / [i915#2994]) +1 similar issue
[116]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl6/igt@sysfs_clients@split-50.html
#### Possible fixes ####
* igt@fbdev@info:
- {shard-rkl}: [SKIP][117] ([i915#2582]) -> [PASS][118]
[117]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-rkl-2/igt@fbdev@info.html
[118]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-rkl-6/igt@fbdev@info.html
* igt@feature_discovery@psr2:
- {shard-rkl}: [SKIP][119] ([i915#658]) -> [PASS][120]
[119]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-rkl-5/igt@feature_discovery@psr2.html
[120]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-rkl-6/igt@feature_discovery@psr2.html
* igt@gem_eio@reset-stress:
- {shard-rkl}: [FAIL][121] ([i915#3115]) -> [PASS][122] +1 similar issue
[121]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-rkl-5/igt@gem_eio@reset-stress.html
[122]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-rkl-1/igt@gem_eio@reset-stress.html
* igt@gem_eio@unwedge-stress:
- shard-tglb: [TIMEOUT][123] ([i915#2369] / [i915#3063] / [i915#3648]) -> [PASS][124]
[123]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-tglb1/igt@gem_eio@unwedge-stress.html
[124]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-tglb2/igt@gem_eio@unwedge-stress.html
* igt@gem_exec_fair@basic-pace@rcs0:
- {shard-rkl}: [FAIL][125] ([i915#2842]) -> [PASS][126]
[125]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-rkl-2/igt@gem_exec_fair@basic-pace@rcs0.html
[126]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-rkl-5/igt@gem_exec_fair@basic-pace@rcs0.html
- shard-glk: [FAIL][127] ([i915#2842]) -> [PASS][128]
[127]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-glk9/igt@gem_exec_fair@basic-pace@rcs0.html
[128]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-glk8/igt@gem_exec_fair@basic-pace@rcs0.html
* igt@gem_exec_fair@basic-pace@vcs1:
- shard-kbl: [SKIP][129] ([fdo#109271]) -> [PASS][130]
[129]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-kbl4/igt@gem_exec_fair@basic-pace@vcs1.html
[130]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-kbl7/igt@gem_exec_fair@basic-pace@vcs1.html
* igt@gem_exec_fair@basic-throttle@rcs0:
- shard-iclb: [FAIL][131] ([i915#2849]) -> [PASS][132]
[131]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-iclb8/igt@gem_exec_fair@basic-throttle@rcs0.html
[132]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-iclb2/igt@gem_exec_fair@basic-throttle@rcs0.html
* igt@gem_mmap_gtt@cpuset-big-copy-xy:
- shard-glk: [FAIL][133] ([i915#1888] / [i915#307]) -> [PASS][134]
[133]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10525/shard-glk4/igt@gem_mmap_gtt@cpuset-big-copy-xy.html
[134]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/shard-glk9/igt@gem_mmap_gtt@cpuset-big-copy-xy.html
* igt@i915_pm_backlight@fade:
- {shard-rkl}: [SKIP][135] ([i915#3012]) -> [PASS][136]
[135]: h
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20901/index.html
[-- Attachment #2: Type: text/html, Size: 33738 bytes --]
^ permalink raw reply [flat|nested] 122+ messages in thread