* [PATCH 0/9] Introduce i915_sched_engine object
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter, daniele.ceraolospurio
As discussed in [1] we are breaking that large series into a several
smaller ones. This series is stand alone patch part of step #4 which has
no other dependencies or patches relevant to it.
v2:
(Daniel Vetter):
- Split into several smaller patches
- Add kernel doc for i915_sched_engine
(Matthew Brost):
- Drop wrapper functions for tasklet as eventually tasklet will be
dropped
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
[1] https://patchwork.freedesktop.org/series/89844/
Matthew Brost (9):
drm/i915: Move priolist to new i915_sched_engine object
drm/i915: Add i915_sched_engine_is_empty function
drm/i915: Add i915_sched_engine_reset_on_empty function
drm/i915: Move active tracking to i915_sched_engine
drm/i915: Move engine->schedule to i915_sched_engine
drm/i915: Add kick_backend function to i915_sched_engine
drm/i915: Update i915_scheduler to operate on i915_sched_engine
drm/i915: Move submission tasklet to i915_sched_engine
drm/i915/doc: Add kernel doc for i915_sched_engine
Documentation/gpu/i915.rst | 6 +
drivers/gpu/drm/i915/gem/i915_gem_wait.c | 4 +-
drivers/gpu/drm/i915/gt/intel_engine.h | 16 -
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 72 ++--
.../gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 +-
drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
drivers/gpu/drm/i915/gt/intel_engine_types.h | 47 +--
drivers/gpu/drm/i915/gt/intel_engine_user.c | 2 +-
.../drm/i915/gt/intel_execlists_submission.c | 323 +++++++++++-------
.../gpu/drm/i915/gt/intel_ring_submission.c | 12 +-
drivers/gpu/drm/i915/gt/mock_engine.c | 17 +-
drivers/gpu/drm/i915/gt/selftest_execlists.c | 36 +-
drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 6 +-
drivers/gpu/drm/i915/gt/selftest_lrc.c | 6 +-
drivers/gpu/drm/i915/gt/selftest_reset.c | 2 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 70 ++--
drivers/gpu/drm/i915/i915_gpu_error.c | 4 +-
drivers/gpu/drm/i915/i915_request.c | 42 +--
drivers/gpu/drm/i915/i915_request.h | 2 +-
drivers/gpu/drm/i915/i915_scheduler.c | 168 +++++----
drivers/gpu/drm/i915/i915_scheduler.h | 47 ++-
drivers/gpu/drm/i915/i915_scheduler_types.h | 89 +++++
22 files changed, 554 insertions(+), 425 deletions(-)
--
2.28.0
^ permalink raw reply [flat|nested] 64+ messages in thread
* [Intel-gfx] [PATCH 0/9] Introduce i915_sched_engine object
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter
As discussed in [1] we are breaking that large series into a several
smaller ones. This series is stand alone patch part of step #4 which has
no other dependencies or patches relevant to it.
v2:
(Daniel Vetter):
- Split into several smaller patches
- Add kernel doc for i915_sched_engine
(Matthew Brost):
- Drop wrapper functions for tasklet as eventually tasklet will be
dropped
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
[1] https://patchwork.freedesktop.org/series/89844/
Matthew Brost (9):
drm/i915: Move priolist to new i915_sched_engine object
drm/i915: Add i915_sched_engine_is_empty function
drm/i915: Add i915_sched_engine_reset_on_empty function
drm/i915: Move active tracking to i915_sched_engine
drm/i915: Move engine->schedule to i915_sched_engine
drm/i915: Add kick_backend function to i915_sched_engine
drm/i915: Update i915_scheduler to operate on i915_sched_engine
drm/i915: Move submission tasklet to i915_sched_engine
drm/i915/doc: Add kernel doc for i915_sched_engine
Documentation/gpu/i915.rst | 6 +
drivers/gpu/drm/i915/gem/i915_gem_wait.c | 4 +-
drivers/gpu/drm/i915/gt/intel_engine.h | 16 -
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 72 ++--
.../gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 +-
drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
drivers/gpu/drm/i915/gt/intel_engine_types.h | 47 +--
drivers/gpu/drm/i915/gt/intel_engine_user.c | 2 +-
.../drm/i915/gt/intel_execlists_submission.c | 323 +++++++++++-------
.../gpu/drm/i915/gt/intel_ring_submission.c | 12 +-
drivers/gpu/drm/i915/gt/mock_engine.c | 17 +-
drivers/gpu/drm/i915/gt/selftest_execlists.c | 36 +-
drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 6 +-
drivers/gpu/drm/i915/gt/selftest_lrc.c | 6 +-
drivers/gpu/drm/i915/gt/selftest_reset.c | 2 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 70 ++--
drivers/gpu/drm/i915/i915_gpu_error.c | 4 +-
drivers/gpu/drm/i915/i915_request.c | 42 +--
drivers/gpu/drm/i915/i915_request.h | 2 +-
drivers/gpu/drm/i915/i915_scheduler.c | 168 +++++----
drivers/gpu/drm/i915/i915_scheduler.h | 47 ++-
drivers/gpu/drm/i915/i915_scheduler_types.h | 89 +++++
22 files changed, 554 insertions(+), 425 deletions(-)
--
2.28.0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-03 21:27 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter, daniele.ceraolospurio
Introduce i915_sched_engine object which is lower level data structure
that i915_scheduler / generic code can operate on without touching
execlist specific structures. This allows additional submission backends
to be added without breaking the layering.
This is a bit of detour to integrating the i915 with the DRM scheduler
but this object will still exist when the DRM scheduler lands in the
i915. It will however look a bit different. It will encapsulate the
drm_gpu_scheduler object plus and common variables (to the backends)
related to scheduling. Regardless this is a step in the right direction.
This patch starts the aforementioned transition by moving the the
priolist into the i915_sched_engine object.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
.../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
9 files changed, 169 insertions(+), 90 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 3f9a811eb02b..d0f3814440f6 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
memset(execlists->pending, 0, sizeof(execlists->pending));
execlists->active =
memset(execlists->inflight, 0, sizeof(execlists->inflight));
-
- execlists->queue_priority_hint = INT_MIN;
- execlists->queue = RB_ROOT_CACHED;
}
static void cleanup_status_page(struct intel_engine_cs *engine)
@@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
goto err_status;
}
+ engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
+ if (!engine->sched_engine) {
+ err = -ENOMEM;
+ goto err_sched_engine;
+ }
+
err = intel_engine_init_cmd_parser(engine);
if (err)
goto err_cmd_parser;
@@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
return 0;
err_cmd_parser:
+ i915_sched_engine_put(engine->sched_engine);
+err_sched_engine:
intel_breadcrumbs_free(engine->breadcrumbs);
err_status:
cleanup_status_page(engine);
@@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
GEM_BUG_ON(!list_empty(&engine->active.requests));
tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
+ i915_sched_engine_put(engine->sched_engine);
intel_breadcrumbs_free(engine->breadcrumbs);
intel_engine_fini_retire(engine);
@@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
intel_engine_flush_submission(engine);
/* ELSP is empty, but there are ready requests? E.g. after reset */
- if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
+ if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
return false;
/* Ring stopped? */
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
index 47f4397095e5..b6a00dd72808 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
@@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
intel_breadcrumbs_park(engine->breadcrumbs);
/* Must be reset upon idling, or we may miss the busy wakeup. */
- GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
+ GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
if (engine->park)
engine->park(engine);
- engine->execlists.no_priolist = false;
+ engine->sched_engine->no_priolist = false;
/* While gt calls i915_vma_parked(), we have to break the lock cycle */
intel_gt_pm_put_async(engine->gt);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index 9ef349cd5cea..86b41ddec373 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -59,6 +59,7 @@ struct drm_i915_reg_table;
struct i915_gem_context;
struct i915_request;
struct i915_sched_attr;
+struct i915_sched_engine;
struct intel_gt;
struct intel_ring;
struct intel_uncore;
@@ -152,11 +153,6 @@ struct intel_engine_execlists {
*/
struct timer_list preempt;
- /**
- * @default_priolist: priority list for I915_PRIORITY_NORMAL
- */
- struct i915_priolist default_priolist;
-
/**
* @ccid: identifier for contexts submitted to this engine
*/
@@ -191,11 +187,6 @@ struct intel_engine_execlists {
*/
u32 reset_ccid;
- /**
- * @no_priolist: priority lists disabled
- */
- bool no_priolist;
-
/**
* @submit_reg: gen-specific execlist submission register
* set to the ExecList Submission Port (elsp) register pre-Gen11 and to
@@ -238,23 +229,8 @@ struct intel_engine_execlists {
unsigned int port_mask;
/**
- * @queue_priority_hint: Highest pending priority.
- *
- * When we add requests into the queue, or adjust the priority of
- * executing requests, we compute the maximum priority of those
- * pending requests. We can then use this value to determine if
- * we need to preempt the executing requests to service the queue.
- * However, since the we may have recorded the priority of an inflight
- * request we wanted to preempt but since completed, at the time of
- * dequeuing the priority hint may no longer may match the highest
- * available request priority.
+ * @virtual: virtual of requests, in priority lists
*/
- int queue_priority_hint;
-
- /**
- * @queue: queue of requests, in priority lists
- */
- struct rb_root_cached queue;
struct rb_root_cached virtual;
/**
@@ -332,6 +308,8 @@ struct intel_engine_cs {
struct list_head hold; /* ready requests, but on hold */
} active;
+ struct i915_sched_engine *sched_engine;
+
/* keep a request in reserve for a [pm] barrier under oom */
struct i915_request *request_pool;
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 8db200422950..d1dc1db3e378 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
return prio;
}
-static int queue_prio(const struct intel_engine_execlists *execlists)
+static int queue_prio(const struct i915_sched_engine *sched_engine)
{
struct rb_node *rb;
- rb = rb_first_cached(&execlists->queue);
+ rb = rb_first_cached(&sched_engine->queue);
if (!rb)
return INT_MIN;
@@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
* to preserve FIFO ordering of dependencies.
*/
last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
- if (engine->execlists.queue_priority_hint <= last_prio)
+ if (engine->sched_engine->queue_priority_hint <= last_prio)
return false;
/*
@@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
* context, it's priority would not exceed ELSP[0] aka last_prio.
*/
return max(virtual_prio(&engine->execlists),
- queue_prio(&engine->execlists)) > last_prio;
+ queue_prio(engine->sched_engine)) > last_prio;
}
__maybe_unused static bool
@@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
prio = rq_prio(rq);
pl = i915_sched_lookup_priolist(engine, prio);
}
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
+ GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
list_move(&rq->sched.link, pl);
set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
@@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
}
/* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
- if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
+ if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
ENGINE_TRACE(engine, "timeslice required for queue\n");
return true;
}
@@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
static void execlists_dequeue(struct intel_engine_cs *engine)
{
struct intel_engine_execlists * const execlists = &engine->execlists;
+ struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_request **port = execlists->pending;
struct i915_request ** const last_port = port + execlists->port_mask;
struct i915_request *last, * const *active;
@@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
last->fence.context,
last->fence.seqno,
last->sched.attr.priority,
- execlists->queue_priority_hint);
+ sched_engine->queue_priority_hint);
record_preemption(execlists);
/*
@@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
yesno(timer_expired(&execlists->timer)),
last->fence.context, last->fence.seqno,
rq_prio(last),
- execlists->queue_priority_hint,
+ sched_engine->queue_priority_hint,
yesno(timeslice_yield(execlists, last)));
/*
@@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
GEM_BUG_ON(rq->engine != &ve->base);
GEM_BUG_ON(rq->context != &ve->context);
- if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
+ if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
spin_unlock(&ve->base.active.lock);
break;
}
@@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
yesno(engine != ve->siblings[0]));
WRITE_ONCE(ve->request, NULL);
- WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
+ WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
rb = &ve->nodes[engine->id].rb;
rb_erase_cached(rb, &execlists->virtual);
@@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
break;
}
- while ((rb = rb_first_cached(&execlists->queue))) {
+ while ((rb = rb_first_cached(&sched_engine->queue))) {
struct i915_priolist *p = to_priolist(rb);
struct i915_request *rq, *rn;
@@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
}
}
- rb_erase_cached(&p->node, &execlists->queue);
+ rb_erase_cached(&p->node, &sched_engine->queue);
i915_priolist_free(p);
}
done:
@@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
* request triggering preemption on the next dequeue (or subsequent
* interrupt for secondary ports).
*/
- execlists->queue_priority_hint = queue_prio(execlists);
+ sched_engine->queue_priority_hint = queue_prio(sched_engine);
spin_unlock(&engine->active.lock);
/*
@@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
*/
__execlists_unhold(rq);
- if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
- engine->execlists.queue_priority_hint = rq_prio(rq);
+ if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
+ engine->sched_engine->queue_priority_hint = rq_prio(rq);
tasklet_hi_schedule(&engine->execlists.tasklet);
}
@@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
static bool submit_queue(struct intel_engine_cs *engine,
const struct i915_request *rq)
{
- struct intel_engine_execlists *execlists = &engine->execlists;
+ struct i915_sched_engine *sched_engine = engine->sched_engine;
- if (rq_prio(rq) <= execlists->queue_priority_hint)
+ if (rq_prio(rq) <= sched_engine->queue_priority_hint)
return false;
- execlists->queue_priority_hint = rq_prio(rq);
+ sched_engine->queue_priority_hint = rq_prio(rq);
return true;
}
@@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
} else {
queue_request(engine, request);
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
+ GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
GEM_BUG_ON(list_empty(&request->sched.link));
if (submit_queue(engine, request))
@@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
from_tasklet(engine, t, execlists.tasklet);
/* The driver is wedged; don't process any more events. */
- WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
+ WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
}
static void execlists_reset_cancel(struct intel_engine_cs *engine)
{
struct intel_engine_execlists * const execlists = &engine->execlists;
+ struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_request *rq, *rn;
struct rb_node *rb;
unsigned long flags;
@@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
intel_engine_signal_breadcrumbs(engine);
/* Flush the queued requests to the timeline list (for retiring). */
- while ((rb = rb_first_cached(&execlists->queue))) {
+ while ((rb = rb_first_cached(&sched_engine->queue))) {
struct i915_priolist *p = to_priolist(rb);
priolist_for_each_request_consume(rq, rn, p) {
@@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
}
}
- rb_erase_cached(&p->node, &execlists->queue);
+ rb_erase_cached(&p->node, &sched_engine->queue);
i915_priolist_free(p);
}
@@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
}
i915_request_put(rq);
- ve->base.execlists.queue_priority_hint = INT_MIN;
+ ve->base.sched_engine->queue_priority_hint = INT_MIN;
}
spin_unlock(&ve->base.active.lock);
}
/* Remaining _unready_ requests will be nop'ed when submitted */
- execlists->queue_priority_hint = INT_MIN;
- execlists->queue = RB_ROOT_CACHED;
+ sched_engine->queue_priority_hint = INT_MIN;
+ sched_engine->queue = RB_ROOT_CACHED;
GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
execlists->tasklet.callback = nop_submission_tasklet;
@@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
static struct list_head *virtual_queue(struct virtual_engine *ve)
{
- return &ve->base.execlists.default_priolist.requests;
+ return &ve->base.sched_engine->default_priolist.requests;
}
static void rcu_virtual_context_destroy(struct work_struct *wrk)
@@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
lrc_fini(&ve->context);
intel_context_fini(&ve->context);
- intel_breadcrumbs_free(ve->base.breadcrumbs);
+ if (ve->base.breadcrumbs)
+ intel_breadcrumbs_free(ve->base.breadcrumbs);
+ if (ve->base.sched_engine)
+ i915_sched_engine_put(ve->base.sched_engine);
intel_engine_free_request_pool(&ve->base);
kfree(ve->bonds);
@@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
rq->fence.context, rq->fence.seqno,
- mask, ve->base.execlists.queue_priority_hint);
+ mask, ve->base.sched_engine->queue_priority_hint);
return mask;
}
@@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
{
struct virtual_engine * const ve =
from_tasklet(ve, t, base.execlists.tasklet);
- const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
+ const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
intel_engine_mask_t mask;
unsigned int n;
@@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
submit_engine:
GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
node->prio = prio;
- if (first && prio > sibling->execlists.queue_priority_hint)
+ if (first && prio > sibling->sched_engine->queue_priority_hint)
tasklet_hi_schedule(&sibling->execlists.tasklet);
unlock_engine:
@@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
i915_request_put(ve->request);
}
- ve->base.execlists.queue_priority_hint = rq_prio(rq);
+ ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
ve->request = i915_request_get(rq);
GEM_BUG_ON(!list_empty(virtual_queue(ve)));
@@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
intel_engine_init_execlists(&ve->base);
+ ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
+ if (!ve->base.sched_engine) {
+ err = -ENOMEM;
+ goto err_put;
+ }
+
ve->base.cops = &virtual_context_ops;
ve->base.request_alloc = execlists_request_alloc;
@@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
ve->base.bond_execute = virtual_bond_execute;
INIT_LIST_HEAD(virtual_queue(ve));
- ve->base.execlists.queue_priority_hint = INT_MIN;
tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
intel_context_init(&ve->context, &ve->base);
@@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
unsigned int max)
{
const struct intel_engine_execlists *execlists = &engine->execlists;
+ const struct i915_sched_engine *sched_engine = engine->sched_engine;
struct i915_request *rq, *last;
unsigned long flags;
unsigned int count;
@@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
show_request(m, last, "\t\t", 0);
}
- if (execlists->queue_priority_hint != INT_MIN)
+ if (sched_engine->queue_priority_hint != INT_MIN)
drm_printf(m, "\t\tQueue priority hint: %d\n",
- READ_ONCE(execlists->queue_priority_hint));
+ READ_ONCE(sched_engine->queue_priority_hint));
last = NULL;
count = 0;
- for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
+ for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
priolist_for_each_request(rq, p) {
diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
index 32589c6625e1..b1fdba13e900 100644
--- a/drivers/gpu/drm/i915/gt/mock_engine.c
+++ b/drivers/gpu/drm/i915/gt/mock_engine.c
@@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
GEM_BUG_ON(timer_pending(&mock->hw_delay));
+ i915_sched_engine_put(engine->sched_engine);
intel_breadcrumbs_free(engine->breadcrumbs);
intel_context_unpin(engine->kernel_context);
@@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
{
struct intel_context *ce;
+ engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
+ if (!engine->sched_engine)
+ return -ENOMEM;
+
intel_engine_init_active(engine, ENGINE_MOCK);
intel_engine_init_execlists(engine);
intel_engine_init__pm(engine);
@@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
engine->breadcrumbs = intel_breadcrumbs_create(NULL);
if (!engine->breadcrumbs)
- return -ENOMEM;
+ goto err_schedule;
ce = create_kernel_context(engine);
if (IS_ERR(ce))
@@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
err_breadcrumbs:
intel_breadcrumbs_free(engine->breadcrumbs);
+err_schedule:
+ i915_sched_engine_put(engine->sched_engine);
return -ENOMEM;
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 335719f17490..d42dea79ee64 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
static void __guc_dequeue(struct intel_engine_cs *engine)
{
struct intel_engine_execlists * const execlists = &engine->execlists;
+ struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_request **first = execlists->inflight;
struct i915_request ** const last_port = first + execlists->port_mask;
struct i915_request *last = first[0];
@@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
* event.
*/
port = first;
- while ((rb = rb_first_cached(&execlists->queue))) {
+ while ((rb = rb_first_cached(&sched_engine->queue))) {
struct i915_priolist *p = to_priolist(rb);
struct i915_request *rq, *rn;
@@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
last = rq;
}
- rb_erase_cached(&p->node, &execlists->queue);
+ rb_erase_cached(&p->node, &sched_engine->queue);
i915_priolist_free(p);
}
done:
- execlists->queue_priority_hint =
+ sched_engine->queue_priority_hint =
rb ? to_priolist(rb)->priority : INT_MIN;
if (submit) {
*port = schedule_in(last, port - execlists->inflight);
@@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
static void guc_reset_cancel(struct intel_engine_cs *engine)
{
- struct intel_engine_execlists * const execlists = &engine->execlists;
+ struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_request *rq, *rn;
struct rb_node *rb;
unsigned long flags;
@@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
}
/* Flush the queued requests to the timeline list (for retiring). */
- while ((rb = rb_first_cached(&execlists->queue))) {
+ while ((rb = rb_first_cached(&sched_engine->queue))) {
struct i915_priolist *p = to_priolist(rb);
priolist_for_each_request_consume(rq, rn, p) {
@@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
i915_request_mark_complete(rq);
}
- rb_erase_cached(&p->node, &execlists->queue);
+ rb_erase_cached(&p->node, &sched_engine->queue);
i915_priolist_free(p);
}
/* Remaining _unready_ requests will be nop'ed when submitted */
- execlists->queue_priority_hint = INT_MIN;
- execlists->queue = RB_ROOT_CACHED;
+ sched_engine->queue_priority_hint = INT_MIN;
+ sched_engine->queue = RB_ROOT_CACHED;
spin_unlock_irqrestore(&engine->active.lock, flags);
}
@@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
queue_request(engine, rq, rq_prio(rq));
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
+ GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
GEM_BUG_ON(list_empty(&rq->sched.link));
tasklet_hi_schedule(&engine->execlists.tasklet);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index efa638c3acc7..4953874a9ba6 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
return rb_entry(rb, struct i915_priolist, node);
}
-static void assert_priolists(struct intel_engine_execlists * const execlists)
+static void assert_priolists(struct i915_sched_engine * const sched_engine)
{
struct rb_node *rb;
long last_prio;
@@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
return;
- GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
- rb_first(&execlists->queue.rb_root));
+ GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
+ rb_first(&sched_engine->queue.rb_root));
last_prio = INT_MAX;
- for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
+ for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
const struct i915_priolist *p = to_priolist(rb);
GEM_BUG_ON(p->priority > last_prio);
@@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
struct list_head *
i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
{
- struct intel_engine_execlists * const execlists = &engine->execlists;
+ struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_priolist *p;
struct rb_node **parent, *rb;
bool first = true;
lockdep_assert_held(&engine->active.lock);
- assert_priolists(execlists);
+ assert_priolists(sched_engine);
- if (unlikely(execlists->no_priolist))
+ if (unlikely(sched_engine->no_priolist))
prio = I915_PRIORITY_NORMAL;
find_priolist:
/* most positive priority is scheduled first, equal priorities fifo */
rb = NULL;
- parent = &execlists->queue.rb_root.rb_node;
+ parent = &sched_engine->queue.rb_root.rb_node;
while (*parent) {
rb = *parent;
p = to_priolist(rb);
@@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
}
if (prio == I915_PRIORITY_NORMAL) {
- p = &execlists->default_priolist;
+ p = &sched_engine->default_priolist;
} else {
p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
/* Convert an allocation failure to a priority bump */
@@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
* requests, so if userspace lied about their
* dependencies that reordering may be visible.
*/
- execlists->no_priolist = true;
+ sched_engine->no_priolist = true;
goto find_priolist;
}
}
@@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
INIT_LIST_HEAD(&p->requests);
rb_link_node(&p->node, rb, parent);
- rb_insert_color_cached(&p->node, &execlists->queue, first);
+ rb_insert_color_cached(&p->node, &sched_engine->queue, first);
return &p->requests;
}
@@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
* We only need to kick the tasklet once for the high priority
* new context we add into the queue.
*/
- if (prio <= engine->execlists.queue_priority_hint)
+ if (prio <= engine->sched_engine->queue_priority_hint)
return;
rcu_read_lock();
@@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
inflight->fence.context, inflight->fence.seqno,
inflight->sched.attr.priority);
- engine->execlists.queue_priority_hint = prio;
+ engine->sched_engine->queue_priority_hint = prio;
if (need_preempt(prio, rq_prio(inflight)))
tasklet_hi_schedule(&engine->execlists.tasklet);
@@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
rcu_read_unlock();
}
+void i915_sched_engine_free(struct kref *kref)
+{
+ struct i915_sched_engine *sched_engine =
+ container_of(kref, typeof(*sched_engine), ref);
+
+ kfree(sched_engine);
+}
+
+struct i915_sched_engine *
+i915_sched_engine_create(unsigned int subclass)
+{
+ struct i915_sched_engine *sched_engine;
+
+ sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
+ if (!sched_engine)
+ return NULL;
+
+ kref_init(&sched_engine->ref);
+
+ sched_engine->queue = RB_ROOT_CACHED;
+ sched_engine->queue_priority_hint = INT_MIN;
+
+ return sched_engine;
+}
+
static void i915_global_scheduler_shrink(void)
{
kmem_cache_shrink(global.slab_dependencies);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 858a0938f47a..91a04e34cac5 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
__i915_priolist_free(p);
}
+struct i915_sched_engine *
+i915_sched_engine_create(unsigned int subclass);
+
+void i915_sched_engine_free(struct kref *kref);
+
+static inline struct i915_sched_engine *
+i915_sched_engine_get(struct i915_sched_engine *sched_engine)
+{
+ kref_get(&sched_engine->ref);
+ return sched_engine;
+}
+
+static inline void
+i915_sched_engine_put(struct i915_sched_engine *sched_engine)
+{
+ kref_put(&sched_engine->ref, i915_sched_engine_free);
+}
+
void i915_request_show_with_schedule(struct drm_printer *m,
const struct i915_request *rq,
const char *prefix,
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 343ed44d5ed4..16a054e67760 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -91,4 +91,37 @@ struct i915_dependency {
&(rq__)->sched.signalers_list, \
signal_link)
+struct i915_sched_engine {
+ struct kref ref;
+
+ /**
+ * @default_priolist: priority list for I915_PRIORITY_NORMAL
+ */
+ struct i915_priolist default_priolist;
+
+ /**
+ * @queue_priority_hint: Highest pending priority.
+ *
+ * When we add requests into the queue, or adjust the priority of
+ * executing requests, we compute the maximum priority of those
+ * pending requests. We can then use this value to determine if
+ * we need to preempt the executing requests to service the queue.
+ * However, since the we may have recorded the priority of an inflight
+ * request we wanted to preempt but since completed, at the time of
+ * dequeuing the priority hint may no longer may match the highest
+ * available request priority.
+ */
+ int queue_priority_hint;
+
+ /**
+ * @queue: queue of requests, in priority lists
+ */
+ struct rb_root_cached queue;
+
+ /**
+ * @no_priolist: priority lists disabled
+ */
+ bool no_priolist;
+};
+
#endif /* _I915_SCHEDULER_TYPES_H_ */
--
2.28.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Intel-gfx] [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter
Introduce i915_sched_engine object which is lower level data structure
that i915_scheduler / generic code can operate on without touching
execlist specific structures. This allows additional submission backends
to be added without breaking the layering.
This is a bit of detour to integrating the i915 with the DRM scheduler
but this object will still exist when the DRM scheduler lands in the
i915. It will however look a bit different. It will encapsulate the
drm_gpu_scheduler object plus and common variables (to the backends)
related to scheduling. Regardless this is a step in the right direction.
This patch starts the aforementioned transition by moving the the
priolist into the i915_sched_engine object.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
.../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
9 files changed, 169 insertions(+), 90 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 3f9a811eb02b..d0f3814440f6 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
memset(execlists->pending, 0, sizeof(execlists->pending));
execlists->active =
memset(execlists->inflight, 0, sizeof(execlists->inflight));
-
- execlists->queue_priority_hint = INT_MIN;
- execlists->queue = RB_ROOT_CACHED;
}
static void cleanup_status_page(struct intel_engine_cs *engine)
@@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
goto err_status;
}
+ engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
+ if (!engine->sched_engine) {
+ err = -ENOMEM;
+ goto err_sched_engine;
+ }
+
err = intel_engine_init_cmd_parser(engine);
if (err)
goto err_cmd_parser;
@@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
return 0;
err_cmd_parser:
+ i915_sched_engine_put(engine->sched_engine);
+err_sched_engine:
intel_breadcrumbs_free(engine->breadcrumbs);
err_status:
cleanup_status_page(engine);
@@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
GEM_BUG_ON(!list_empty(&engine->active.requests));
tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
+ i915_sched_engine_put(engine->sched_engine);
intel_breadcrumbs_free(engine->breadcrumbs);
intel_engine_fini_retire(engine);
@@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
intel_engine_flush_submission(engine);
/* ELSP is empty, but there are ready requests? E.g. after reset */
- if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
+ if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
return false;
/* Ring stopped? */
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
index 47f4397095e5..b6a00dd72808 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
@@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
intel_breadcrumbs_park(engine->breadcrumbs);
/* Must be reset upon idling, or we may miss the busy wakeup. */
- GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
+ GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
if (engine->park)
engine->park(engine);
- engine->execlists.no_priolist = false;
+ engine->sched_engine->no_priolist = false;
/* While gt calls i915_vma_parked(), we have to break the lock cycle */
intel_gt_pm_put_async(engine->gt);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index 9ef349cd5cea..86b41ddec373 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -59,6 +59,7 @@ struct drm_i915_reg_table;
struct i915_gem_context;
struct i915_request;
struct i915_sched_attr;
+struct i915_sched_engine;
struct intel_gt;
struct intel_ring;
struct intel_uncore;
@@ -152,11 +153,6 @@ struct intel_engine_execlists {
*/
struct timer_list preempt;
- /**
- * @default_priolist: priority list for I915_PRIORITY_NORMAL
- */
- struct i915_priolist default_priolist;
-
/**
* @ccid: identifier for contexts submitted to this engine
*/
@@ -191,11 +187,6 @@ struct intel_engine_execlists {
*/
u32 reset_ccid;
- /**
- * @no_priolist: priority lists disabled
- */
- bool no_priolist;
-
/**
* @submit_reg: gen-specific execlist submission register
* set to the ExecList Submission Port (elsp) register pre-Gen11 and to
@@ -238,23 +229,8 @@ struct intel_engine_execlists {
unsigned int port_mask;
/**
- * @queue_priority_hint: Highest pending priority.
- *
- * When we add requests into the queue, or adjust the priority of
- * executing requests, we compute the maximum priority of those
- * pending requests. We can then use this value to determine if
- * we need to preempt the executing requests to service the queue.
- * However, since the we may have recorded the priority of an inflight
- * request we wanted to preempt but since completed, at the time of
- * dequeuing the priority hint may no longer may match the highest
- * available request priority.
+ * @virtual: virtual of requests, in priority lists
*/
- int queue_priority_hint;
-
- /**
- * @queue: queue of requests, in priority lists
- */
- struct rb_root_cached queue;
struct rb_root_cached virtual;
/**
@@ -332,6 +308,8 @@ struct intel_engine_cs {
struct list_head hold; /* ready requests, but on hold */
} active;
+ struct i915_sched_engine *sched_engine;
+
/* keep a request in reserve for a [pm] barrier under oom */
struct i915_request *request_pool;
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 8db200422950..d1dc1db3e378 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
return prio;
}
-static int queue_prio(const struct intel_engine_execlists *execlists)
+static int queue_prio(const struct i915_sched_engine *sched_engine)
{
struct rb_node *rb;
- rb = rb_first_cached(&execlists->queue);
+ rb = rb_first_cached(&sched_engine->queue);
if (!rb)
return INT_MIN;
@@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
* to preserve FIFO ordering of dependencies.
*/
last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
- if (engine->execlists.queue_priority_hint <= last_prio)
+ if (engine->sched_engine->queue_priority_hint <= last_prio)
return false;
/*
@@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
* context, it's priority would not exceed ELSP[0] aka last_prio.
*/
return max(virtual_prio(&engine->execlists),
- queue_prio(&engine->execlists)) > last_prio;
+ queue_prio(engine->sched_engine)) > last_prio;
}
__maybe_unused static bool
@@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
prio = rq_prio(rq);
pl = i915_sched_lookup_priolist(engine, prio);
}
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
+ GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
list_move(&rq->sched.link, pl);
set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
@@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
}
/* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
- if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
+ if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
ENGINE_TRACE(engine, "timeslice required for queue\n");
return true;
}
@@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
static void execlists_dequeue(struct intel_engine_cs *engine)
{
struct intel_engine_execlists * const execlists = &engine->execlists;
+ struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_request **port = execlists->pending;
struct i915_request ** const last_port = port + execlists->port_mask;
struct i915_request *last, * const *active;
@@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
last->fence.context,
last->fence.seqno,
last->sched.attr.priority,
- execlists->queue_priority_hint);
+ sched_engine->queue_priority_hint);
record_preemption(execlists);
/*
@@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
yesno(timer_expired(&execlists->timer)),
last->fence.context, last->fence.seqno,
rq_prio(last),
- execlists->queue_priority_hint,
+ sched_engine->queue_priority_hint,
yesno(timeslice_yield(execlists, last)));
/*
@@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
GEM_BUG_ON(rq->engine != &ve->base);
GEM_BUG_ON(rq->context != &ve->context);
- if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
+ if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
spin_unlock(&ve->base.active.lock);
break;
}
@@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
yesno(engine != ve->siblings[0]));
WRITE_ONCE(ve->request, NULL);
- WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
+ WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
rb = &ve->nodes[engine->id].rb;
rb_erase_cached(rb, &execlists->virtual);
@@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
break;
}
- while ((rb = rb_first_cached(&execlists->queue))) {
+ while ((rb = rb_first_cached(&sched_engine->queue))) {
struct i915_priolist *p = to_priolist(rb);
struct i915_request *rq, *rn;
@@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
}
}
- rb_erase_cached(&p->node, &execlists->queue);
+ rb_erase_cached(&p->node, &sched_engine->queue);
i915_priolist_free(p);
}
done:
@@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
* request triggering preemption on the next dequeue (or subsequent
* interrupt for secondary ports).
*/
- execlists->queue_priority_hint = queue_prio(execlists);
+ sched_engine->queue_priority_hint = queue_prio(sched_engine);
spin_unlock(&engine->active.lock);
/*
@@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
*/
__execlists_unhold(rq);
- if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
- engine->execlists.queue_priority_hint = rq_prio(rq);
+ if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
+ engine->sched_engine->queue_priority_hint = rq_prio(rq);
tasklet_hi_schedule(&engine->execlists.tasklet);
}
@@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
static bool submit_queue(struct intel_engine_cs *engine,
const struct i915_request *rq)
{
- struct intel_engine_execlists *execlists = &engine->execlists;
+ struct i915_sched_engine *sched_engine = engine->sched_engine;
- if (rq_prio(rq) <= execlists->queue_priority_hint)
+ if (rq_prio(rq) <= sched_engine->queue_priority_hint)
return false;
- execlists->queue_priority_hint = rq_prio(rq);
+ sched_engine->queue_priority_hint = rq_prio(rq);
return true;
}
@@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
} else {
queue_request(engine, request);
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
+ GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
GEM_BUG_ON(list_empty(&request->sched.link));
if (submit_queue(engine, request))
@@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
from_tasklet(engine, t, execlists.tasklet);
/* The driver is wedged; don't process any more events. */
- WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
+ WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
}
static void execlists_reset_cancel(struct intel_engine_cs *engine)
{
struct intel_engine_execlists * const execlists = &engine->execlists;
+ struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_request *rq, *rn;
struct rb_node *rb;
unsigned long flags;
@@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
intel_engine_signal_breadcrumbs(engine);
/* Flush the queued requests to the timeline list (for retiring). */
- while ((rb = rb_first_cached(&execlists->queue))) {
+ while ((rb = rb_first_cached(&sched_engine->queue))) {
struct i915_priolist *p = to_priolist(rb);
priolist_for_each_request_consume(rq, rn, p) {
@@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
}
}
- rb_erase_cached(&p->node, &execlists->queue);
+ rb_erase_cached(&p->node, &sched_engine->queue);
i915_priolist_free(p);
}
@@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
}
i915_request_put(rq);
- ve->base.execlists.queue_priority_hint = INT_MIN;
+ ve->base.sched_engine->queue_priority_hint = INT_MIN;
}
spin_unlock(&ve->base.active.lock);
}
/* Remaining _unready_ requests will be nop'ed when submitted */
- execlists->queue_priority_hint = INT_MIN;
- execlists->queue = RB_ROOT_CACHED;
+ sched_engine->queue_priority_hint = INT_MIN;
+ sched_engine->queue = RB_ROOT_CACHED;
GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
execlists->tasklet.callback = nop_submission_tasklet;
@@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
static struct list_head *virtual_queue(struct virtual_engine *ve)
{
- return &ve->base.execlists.default_priolist.requests;
+ return &ve->base.sched_engine->default_priolist.requests;
}
static void rcu_virtual_context_destroy(struct work_struct *wrk)
@@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
lrc_fini(&ve->context);
intel_context_fini(&ve->context);
- intel_breadcrumbs_free(ve->base.breadcrumbs);
+ if (ve->base.breadcrumbs)
+ intel_breadcrumbs_free(ve->base.breadcrumbs);
+ if (ve->base.sched_engine)
+ i915_sched_engine_put(ve->base.sched_engine);
intel_engine_free_request_pool(&ve->base);
kfree(ve->bonds);
@@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
rq->fence.context, rq->fence.seqno,
- mask, ve->base.execlists.queue_priority_hint);
+ mask, ve->base.sched_engine->queue_priority_hint);
return mask;
}
@@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
{
struct virtual_engine * const ve =
from_tasklet(ve, t, base.execlists.tasklet);
- const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
+ const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
intel_engine_mask_t mask;
unsigned int n;
@@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
submit_engine:
GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
node->prio = prio;
- if (first && prio > sibling->execlists.queue_priority_hint)
+ if (first && prio > sibling->sched_engine->queue_priority_hint)
tasklet_hi_schedule(&sibling->execlists.tasklet);
unlock_engine:
@@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
i915_request_put(ve->request);
}
- ve->base.execlists.queue_priority_hint = rq_prio(rq);
+ ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
ve->request = i915_request_get(rq);
GEM_BUG_ON(!list_empty(virtual_queue(ve)));
@@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
intel_engine_init_execlists(&ve->base);
+ ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
+ if (!ve->base.sched_engine) {
+ err = -ENOMEM;
+ goto err_put;
+ }
+
ve->base.cops = &virtual_context_ops;
ve->base.request_alloc = execlists_request_alloc;
@@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
ve->base.bond_execute = virtual_bond_execute;
INIT_LIST_HEAD(virtual_queue(ve));
- ve->base.execlists.queue_priority_hint = INT_MIN;
tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
intel_context_init(&ve->context, &ve->base);
@@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
unsigned int max)
{
const struct intel_engine_execlists *execlists = &engine->execlists;
+ const struct i915_sched_engine *sched_engine = engine->sched_engine;
struct i915_request *rq, *last;
unsigned long flags;
unsigned int count;
@@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
show_request(m, last, "\t\t", 0);
}
- if (execlists->queue_priority_hint != INT_MIN)
+ if (sched_engine->queue_priority_hint != INT_MIN)
drm_printf(m, "\t\tQueue priority hint: %d\n",
- READ_ONCE(execlists->queue_priority_hint));
+ READ_ONCE(sched_engine->queue_priority_hint));
last = NULL;
count = 0;
- for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
+ for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
priolist_for_each_request(rq, p) {
diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
index 32589c6625e1..b1fdba13e900 100644
--- a/drivers/gpu/drm/i915/gt/mock_engine.c
+++ b/drivers/gpu/drm/i915/gt/mock_engine.c
@@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
GEM_BUG_ON(timer_pending(&mock->hw_delay));
+ i915_sched_engine_put(engine->sched_engine);
intel_breadcrumbs_free(engine->breadcrumbs);
intel_context_unpin(engine->kernel_context);
@@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
{
struct intel_context *ce;
+ engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
+ if (!engine->sched_engine)
+ return -ENOMEM;
+
intel_engine_init_active(engine, ENGINE_MOCK);
intel_engine_init_execlists(engine);
intel_engine_init__pm(engine);
@@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
engine->breadcrumbs = intel_breadcrumbs_create(NULL);
if (!engine->breadcrumbs)
- return -ENOMEM;
+ goto err_schedule;
ce = create_kernel_context(engine);
if (IS_ERR(ce))
@@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
err_breadcrumbs:
intel_breadcrumbs_free(engine->breadcrumbs);
+err_schedule:
+ i915_sched_engine_put(engine->sched_engine);
return -ENOMEM;
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 335719f17490..d42dea79ee64 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
static void __guc_dequeue(struct intel_engine_cs *engine)
{
struct intel_engine_execlists * const execlists = &engine->execlists;
+ struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_request **first = execlists->inflight;
struct i915_request ** const last_port = first + execlists->port_mask;
struct i915_request *last = first[0];
@@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
* event.
*/
port = first;
- while ((rb = rb_first_cached(&execlists->queue))) {
+ while ((rb = rb_first_cached(&sched_engine->queue))) {
struct i915_priolist *p = to_priolist(rb);
struct i915_request *rq, *rn;
@@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
last = rq;
}
- rb_erase_cached(&p->node, &execlists->queue);
+ rb_erase_cached(&p->node, &sched_engine->queue);
i915_priolist_free(p);
}
done:
- execlists->queue_priority_hint =
+ sched_engine->queue_priority_hint =
rb ? to_priolist(rb)->priority : INT_MIN;
if (submit) {
*port = schedule_in(last, port - execlists->inflight);
@@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
static void guc_reset_cancel(struct intel_engine_cs *engine)
{
- struct intel_engine_execlists * const execlists = &engine->execlists;
+ struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_request *rq, *rn;
struct rb_node *rb;
unsigned long flags;
@@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
}
/* Flush the queued requests to the timeline list (for retiring). */
- while ((rb = rb_first_cached(&execlists->queue))) {
+ while ((rb = rb_first_cached(&sched_engine->queue))) {
struct i915_priolist *p = to_priolist(rb);
priolist_for_each_request_consume(rq, rn, p) {
@@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
i915_request_mark_complete(rq);
}
- rb_erase_cached(&p->node, &execlists->queue);
+ rb_erase_cached(&p->node, &sched_engine->queue);
i915_priolist_free(p);
}
/* Remaining _unready_ requests will be nop'ed when submitted */
- execlists->queue_priority_hint = INT_MIN;
- execlists->queue = RB_ROOT_CACHED;
+ sched_engine->queue_priority_hint = INT_MIN;
+ sched_engine->queue = RB_ROOT_CACHED;
spin_unlock_irqrestore(&engine->active.lock, flags);
}
@@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
queue_request(engine, rq, rq_prio(rq));
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
+ GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
GEM_BUG_ON(list_empty(&rq->sched.link));
tasklet_hi_schedule(&engine->execlists.tasklet);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index efa638c3acc7..4953874a9ba6 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
return rb_entry(rb, struct i915_priolist, node);
}
-static void assert_priolists(struct intel_engine_execlists * const execlists)
+static void assert_priolists(struct i915_sched_engine * const sched_engine)
{
struct rb_node *rb;
long last_prio;
@@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
return;
- GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
- rb_first(&execlists->queue.rb_root));
+ GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
+ rb_first(&sched_engine->queue.rb_root));
last_prio = INT_MAX;
- for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
+ for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
const struct i915_priolist *p = to_priolist(rb);
GEM_BUG_ON(p->priority > last_prio);
@@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
struct list_head *
i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
{
- struct intel_engine_execlists * const execlists = &engine->execlists;
+ struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_priolist *p;
struct rb_node **parent, *rb;
bool first = true;
lockdep_assert_held(&engine->active.lock);
- assert_priolists(execlists);
+ assert_priolists(sched_engine);
- if (unlikely(execlists->no_priolist))
+ if (unlikely(sched_engine->no_priolist))
prio = I915_PRIORITY_NORMAL;
find_priolist:
/* most positive priority is scheduled first, equal priorities fifo */
rb = NULL;
- parent = &execlists->queue.rb_root.rb_node;
+ parent = &sched_engine->queue.rb_root.rb_node;
while (*parent) {
rb = *parent;
p = to_priolist(rb);
@@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
}
if (prio == I915_PRIORITY_NORMAL) {
- p = &execlists->default_priolist;
+ p = &sched_engine->default_priolist;
} else {
p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
/* Convert an allocation failure to a priority bump */
@@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
* requests, so if userspace lied about their
* dependencies that reordering may be visible.
*/
- execlists->no_priolist = true;
+ sched_engine->no_priolist = true;
goto find_priolist;
}
}
@@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
INIT_LIST_HEAD(&p->requests);
rb_link_node(&p->node, rb, parent);
- rb_insert_color_cached(&p->node, &execlists->queue, first);
+ rb_insert_color_cached(&p->node, &sched_engine->queue, first);
return &p->requests;
}
@@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
* We only need to kick the tasklet once for the high priority
* new context we add into the queue.
*/
- if (prio <= engine->execlists.queue_priority_hint)
+ if (prio <= engine->sched_engine->queue_priority_hint)
return;
rcu_read_lock();
@@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
inflight->fence.context, inflight->fence.seqno,
inflight->sched.attr.priority);
- engine->execlists.queue_priority_hint = prio;
+ engine->sched_engine->queue_priority_hint = prio;
if (need_preempt(prio, rq_prio(inflight)))
tasklet_hi_schedule(&engine->execlists.tasklet);
@@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
rcu_read_unlock();
}
+void i915_sched_engine_free(struct kref *kref)
+{
+ struct i915_sched_engine *sched_engine =
+ container_of(kref, typeof(*sched_engine), ref);
+
+ kfree(sched_engine);
+}
+
+struct i915_sched_engine *
+i915_sched_engine_create(unsigned int subclass)
+{
+ struct i915_sched_engine *sched_engine;
+
+ sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
+ if (!sched_engine)
+ return NULL;
+
+ kref_init(&sched_engine->ref);
+
+ sched_engine->queue = RB_ROOT_CACHED;
+ sched_engine->queue_priority_hint = INT_MIN;
+
+ return sched_engine;
+}
+
static void i915_global_scheduler_shrink(void)
{
kmem_cache_shrink(global.slab_dependencies);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 858a0938f47a..91a04e34cac5 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
__i915_priolist_free(p);
}
+struct i915_sched_engine *
+i915_sched_engine_create(unsigned int subclass);
+
+void i915_sched_engine_free(struct kref *kref);
+
+static inline struct i915_sched_engine *
+i915_sched_engine_get(struct i915_sched_engine *sched_engine)
+{
+ kref_get(&sched_engine->ref);
+ return sched_engine;
+}
+
+static inline void
+i915_sched_engine_put(struct i915_sched_engine *sched_engine)
+{
+ kref_put(&sched_engine->ref, i915_sched_engine_free);
+}
+
void i915_request_show_with_schedule(struct drm_printer *m,
const struct i915_request *rq,
const char *prefix,
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 343ed44d5ed4..16a054e67760 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -91,4 +91,37 @@ struct i915_dependency {
&(rq__)->sched.signalers_list, \
signal_link)
+struct i915_sched_engine {
+ struct kref ref;
+
+ /**
+ * @default_priolist: priority list for I915_PRIORITY_NORMAL
+ */
+ struct i915_priolist default_priolist;
+
+ /**
+ * @queue_priority_hint: Highest pending priority.
+ *
+ * When we add requests into the queue, or adjust the priority of
+ * executing requests, we compute the maximum priority of those
+ * pending requests. We can then use this value to determine if
+ * we need to preempt the executing requests to service the queue.
+ * However, since the we may have recorded the priority of an inflight
+ * request we wanted to preempt but since completed, at the time of
+ * dequeuing the priority hint may no longer may match the highest
+ * available request priority.
+ */
+ int queue_priority_hint;
+
+ /**
+ * @queue: queue of requests, in priority lists
+ */
+ struct rb_root_cached queue;
+
+ /**
+ * @no_priolist: priority lists disabled
+ */
+ bool no_priolist;
+};
+
#endif /* _I915_SCHEDULER_TYPES_H_ */
--
2.28.0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 2/9] drm/i915: Add i915_sched_engine_is_empty function
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-03 21:27 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter, daniele.ceraolospurio
Add wrapper function around RB tree to determine if i915_sched_engine is
empty.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +-
drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 6 +++---
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
drivers/gpu/drm/i915/i915_scheduler.h | 6 ++++++
4 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index d0f3814440f6..85f2effe9dc6 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -1287,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
intel_engine_flush_submission(engine);
/* ELSP is empty, but there are ready requests? E.g. after reset */
- if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
+ if (!i915_sched_engine_is_empty(engine->sched_engine))
return false;
/* Ring stopped? */
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index d1dc1db3e378..2326a73af6d3 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
prio = rq_prio(rq);
pl = i915_sched_lookup_priolist(engine, prio);
}
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
+ GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
list_move(&rq->sched.link, pl);
set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
@@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
}
/* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
- if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
+ if (!i915_sched_engine_is_empty(engine->sched_engine)) {
ENGINE_TRACE(engine, "timeslice required for queue\n");
return true;
}
@@ -2487,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
} else {
queue_request(engine, request);
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
+ GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
GEM_BUG_ON(list_empty(&request->sched.link));
if (submit_queue(engine, request))
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index d42dea79ee64..5d00f2e3c1de 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -541,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
queue_request(engine, rq, rq_prio(rq));
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
+ GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
GEM_BUG_ON(list_empty(&rq->sched.link));
tasklet_hi_schedule(&engine->execlists.tasklet);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 91a04e34cac5..5bec7b3b8456 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -66,6 +66,12 @@ i915_sched_engine_put(struct i915_sched_engine *sched_engine)
kref_put(&sched_engine->ref, i915_sched_engine_free);
}
+static inline bool
+i915_sched_engine_is_empty(struct i915_sched_engine *sched_engine)
+{
+ return RB_EMPTY_ROOT(&sched_engine->queue.rb_root);
+}
+
void i915_request_show_with_schedule(struct drm_printer *m,
const struct i915_request *rq,
const char *prefix,
--
2.28.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Intel-gfx] [PATCH 2/9] drm/i915: Add i915_sched_engine_is_empty function
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter
Add wrapper function around RB tree to determine if i915_sched_engine is
empty.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +-
drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 6 +++---
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
drivers/gpu/drm/i915/i915_scheduler.h | 6 ++++++
4 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index d0f3814440f6..85f2effe9dc6 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -1287,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
intel_engine_flush_submission(engine);
/* ELSP is empty, but there are ready requests? E.g. after reset */
- if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
+ if (!i915_sched_engine_is_empty(engine->sched_engine))
return false;
/* Ring stopped? */
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index d1dc1db3e378..2326a73af6d3 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
prio = rq_prio(rq);
pl = i915_sched_lookup_priolist(engine, prio);
}
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
+ GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
list_move(&rq->sched.link, pl);
set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
@@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
}
/* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
- if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
+ if (!i915_sched_engine_is_empty(engine->sched_engine)) {
ENGINE_TRACE(engine, "timeslice required for queue\n");
return true;
}
@@ -2487,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
} else {
queue_request(engine, request);
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
+ GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
GEM_BUG_ON(list_empty(&request->sched.link));
if (submit_queue(engine, request))
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index d42dea79ee64..5d00f2e3c1de 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -541,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
queue_request(engine, rq, rq_prio(rq));
- GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
+ GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
GEM_BUG_ON(list_empty(&rq->sched.link));
tasklet_hi_schedule(&engine->execlists.tasklet);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 91a04e34cac5..5bec7b3b8456 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -66,6 +66,12 @@ i915_sched_engine_put(struct i915_sched_engine *sched_engine)
kref_put(&sched_engine->ref, i915_sched_engine_free);
}
+static inline bool
+i915_sched_engine_is_empty(struct i915_sched_engine *sched_engine)
+{
+ return RB_EMPTY_ROOT(&sched_engine->queue.rb_root);
+}
+
void i915_request_show_with_schedule(struct drm_printer *m,
const struct i915_request *rq,
const char *prefix,
--
2.28.0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 3/9] drm/i915: Add i915_sched_engine_reset_on_empty function
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-03 21:27 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter, daniele.ceraolospurio
Rather than touching schedule state in the generic PM code, reset the
priolist allocation when empty in the submission code. Add a wrapper
function to do this and update the backends to call it in the correct
place.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine_pm.c | 2 --
drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 1 +
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 ++
drivers/gpu/drm/i915/i915_scheduler.h | 7 +++++++
4 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
index b6a00dd72808..1f07ac4e0672 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
@@ -280,8 +280,6 @@ static int __engine_park(struct intel_wakeref *wf)
if (engine->park)
engine->park(engine);
- engine->sched_engine->no_priolist = false;
-
/* While gt calls i915_vma_parked(), we have to break the lock cycle */
intel_gt_pm_put_async(engine->gt);
return 0;
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 2326a73af6d3..609753b5401a 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -1553,6 +1553,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
* interrupt for secondary ports).
*/
sched_engine->queue_priority_hint = queue_prio(sched_engine);
+ i915_sched_engine_reset_on_empty(sched_engine);
spin_unlock(&engine->active.lock);
/*
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 5d00f2e3c1de..f4a6fbfaf82e 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -263,6 +263,8 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
__guc_dequeue(engine);
+ i915_sched_engine_reset_on_empty(engine->sched_engine);
+
spin_unlock_irqrestore(&engine->active.lock, flags);
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 5bec7b3b8456..713c38c99de9 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -72,6 +72,13 @@ i915_sched_engine_is_empty(struct i915_sched_engine *sched_engine)
return RB_EMPTY_ROOT(&sched_engine->queue.rb_root);
}
+static inline void
+i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
+{
+ if (i915_sched_engine_is_empty(sched_engine))
+ sched_engine->no_priolist = false;
+}
+
void i915_request_show_with_schedule(struct drm_printer *m,
const struct i915_request *rq,
const char *prefix,
--
2.28.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Intel-gfx] [PATCH 3/9] drm/i915: Add i915_sched_engine_reset_on_empty function
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter
Rather than touching schedule state in the generic PM code, reset the
priolist allocation when empty in the submission code. Add a wrapper
function to do this and update the backends to call it in the correct
place.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine_pm.c | 2 --
drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 1 +
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 ++
drivers/gpu/drm/i915/i915_scheduler.h | 7 +++++++
4 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
index b6a00dd72808..1f07ac4e0672 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
@@ -280,8 +280,6 @@ static int __engine_park(struct intel_wakeref *wf)
if (engine->park)
engine->park(engine);
- engine->sched_engine->no_priolist = false;
-
/* While gt calls i915_vma_parked(), we have to break the lock cycle */
intel_gt_pm_put_async(engine->gt);
return 0;
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 2326a73af6d3..609753b5401a 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -1553,6 +1553,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
* interrupt for secondary ports).
*/
sched_engine->queue_priority_hint = queue_prio(sched_engine);
+ i915_sched_engine_reset_on_empty(sched_engine);
spin_unlock(&engine->active.lock);
/*
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 5d00f2e3c1de..f4a6fbfaf82e 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -263,6 +263,8 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
__guc_dequeue(engine);
+ i915_sched_engine_reset_on_empty(engine->sched_engine);
+
spin_unlock_irqrestore(&engine->active.lock, flags);
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 5bec7b3b8456..713c38c99de9 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -72,6 +72,13 @@ i915_sched_engine_is_empty(struct i915_sched_engine *sched_engine)
return RB_EMPTY_ROOT(&sched_engine->queue.rb_root);
}
+static inline void
+i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
+{
+ if (i915_sched_engine_is_empty(sched_engine))
+ sched_engine->no_priolist = false;
+}
+
void i915_request_show_with_schedule(struct drm_printer *m,
const struct i915_request *rq,
const char *prefix,
--
2.28.0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 4/9] drm/i915: Move active tracking to i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-03 21:27 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter, daniele.ceraolospurio
Move active request tracking and its lock to i915_sched_engine. This
lock is also the submission lock so having it in the i915_sched_engine
is the correct place.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine.h | 2 -
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 43 +++-----
drivers/gpu/drm/i915/gt/intel_engine_types.h | 6 --
.../drm/i915/gt/intel_execlists_submission.c | 98 ++++++++++---------
.../gpu/drm/i915/gt/intel_ring_submission.c | 12 +--
drivers/gpu/drm/i915/gt/mock_engine.c | 7 +-
drivers/gpu/drm/i915/gt/selftest_execlists.c | 4 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 20 ++--
drivers/gpu/drm/i915/i915_gpu_error.c | 4 +-
drivers/gpu/drm/i915/i915_request.c | 32 +++---
drivers/gpu/drm/i915/i915_request.h | 2 +-
drivers/gpu/drm/i915/i915_scheduler.c | 30 ++++--
drivers/gpu/drm/i915/i915_scheduler_types.h | 9 ++
13 files changed, 134 insertions(+), 135 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
index 8d9184920c51..a8b2174b4395 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine.h
@@ -257,8 +257,6 @@ intel_engine_find_active_request(struct intel_engine_cs *engine);
u32 intel_engine_context_size(struct intel_gt *gt, u8 class);
-void intel_engine_init_active(struct intel_engine_cs *engine,
- unsigned int subclass);
#define ENGINE_PHYSICAL 0
#define ENGINE_MOCK 1
#define ENGINE_VIRTUAL 2
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 85f2effe9dc6..33d879137908 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -719,7 +719,6 @@ static int engine_setup_common(struct intel_engine_cs *engine)
if (err)
goto err_cmd_parser;
- intel_engine_init_active(engine, ENGINE_PHYSICAL);
intel_engine_init_execlists(engine);
intel_engine_init__pm(engine);
intel_engine_init_retire(engine);
@@ -778,11 +777,11 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
frame->rq.ring = &frame->ring;
mutex_lock(&ce->timeline->mutex);
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
dw = engine->emit_fini_breadcrumb(&frame->rq, frame->cs) - frame->cs;
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
mutex_unlock(&ce->timeline->mutex);
GEM_BUG_ON(dw & 1); /* RING_TAIL must be qword aligned */
@@ -791,28 +790,6 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
return dw;
}
-void
-intel_engine_init_active(struct intel_engine_cs *engine, unsigned int subclass)
-{
- INIT_LIST_HEAD(&engine->active.requests);
- INIT_LIST_HEAD(&engine->active.hold);
-
- spin_lock_init(&engine->active.lock);
- lockdep_set_subclass(&engine->active.lock, subclass);
-
- /*
- * Due to an interesting quirk in lockdep's internal debug tracking,
- * after setting a subclass we must ensure the lock is used. Otherwise,
- * nr_unused_locks is incremented once too often.
- */
-#ifdef CONFIG_DEBUG_LOCK_ALLOC
- local_irq_disable();
- lock_map_acquire(&engine->active.lock.dep_map);
- lock_map_release(&engine->active.lock.dep_map);
- local_irq_enable();
-#endif
-}
-
static struct intel_context *
create_pinned_context(struct intel_engine_cs *engine,
unsigned int hwsp,
@@ -960,7 +937,7 @@ int intel_engines_init(struct intel_gt *gt)
*/
void intel_engine_cleanup_common(struct intel_engine_cs *engine)
{
- GEM_BUG_ON(!list_empty(&engine->active.requests));
+ GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
i915_sched_engine_put(engine->sched_engine);
@@ -1353,7 +1330,7 @@ static struct intel_timeline *get_timeline(struct i915_request *rq)
struct intel_timeline *tl;
/*
- * Even though we are holding the engine->active.lock here, there
+ * Even though we are holding the engine->sched_engine->lock here, there
* is no control over the submission queue per-se and we are
* inspecting the active state at a random point in time, with an
* unknown queue. Play safe and make sure the timeline remains valid.
@@ -1700,7 +1677,7 @@ void intel_engine_dump(struct intel_engine_cs *engine,
drm_printf(m, "\tRequests:\n");
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
rq = intel_engine_find_active_request(engine);
if (rq) {
struct intel_timeline *tl = get_timeline(rq);
@@ -1731,8 +1708,9 @@ void intel_engine_dump(struct intel_engine_cs *engine,
hexdump(m, rq->context->lrc_reg_state, PAGE_SIZE);
}
}
- drm_printf(m, "\tOn hold?: %lu\n", list_count(&engine->active.hold));
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ drm_printf(m, "\tOn hold?: %lu\n",
+ list_count(&engine->sched_engine->hold));
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
drm_printf(m, "\tMMIO base: 0x%08x\n", engine->mmio_base);
wakeref = intel_runtime_pm_get_if_in_use(engine->uncore->rpm);
@@ -1812,7 +1790,7 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
* At all other times, we must assume the GPU is still running, but
* we only care about the snapshot of this moment.
*/
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
rcu_read_lock();
request = execlists_active(&engine->execlists);
@@ -1830,7 +1808,8 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
if (active)
return active;
- list_for_each_entry(request, &engine->active.requests, sched.link) {
+ list_for_each_entry(request, &engine->sched_engine->requests,
+ sched.link) {
if (__i915_request_is_complete(request))
continue;
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index 86b41ddec373..7197b9fa5e35 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -302,12 +302,6 @@ struct intel_engine_cs {
struct intel_sseu sseu;
- struct {
- spinlock_t lock;
- struct list_head requests;
- struct list_head hold; /* ready requests, but on hold */
- } active;
-
struct i915_sched_engine *sched_engine;
/* keep a request in reserve for a [pm] barrier under oom */
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 609753b5401a..0413ba87cc1e 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -325,7 +325,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
* Check against the first request in ELSP[1], it will, thanks to the
* power of PI, be the highest priority of that context.
*/
- if (!list_is_last(&rq->sched.link, &engine->active.requests) &&
+ if (!list_is_last(&rq->sched.link, &engine->sched_engine->requests) &&
rq_prio(list_next_entry(rq, sched.link)) > last_prio)
return true;
@@ -367,10 +367,10 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
struct list_head *pl;
int prio = I915_PRIORITY_INVALID;
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
list_for_each_entry_safe_reverse(rq, rn,
- &engine->active.requests,
+ &engine->sched_engine->requests,
sched.link) {
if (__i915_request_is_complete(rq)) {
list_del_init(&rq->sched.link);
@@ -534,13 +534,13 @@ resubmit_virtual_request(struct i915_request *rq, struct virtual_engine *ve)
{
struct intel_engine_cs *engine = rq->engine;
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
WRITE_ONCE(rq->engine, &ve->base);
ve->base.submit_request(rq);
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
}
static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
@@ -579,7 +579,7 @@ static void __execlists_schedule_out(struct i915_request * const rq,
unsigned int ccid;
/*
- * NB process_csb() is not under the engine->active.lock and hence
+ * NB process_csb() is not under the engine->sched_engine->lock and hence
* schedule_out can race with schedule_in meaning that we should
* refrain from doing non-trivial work here.
*/
@@ -1133,7 +1133,8 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
return false;
/* If ELSP[1] is occupied, always check to see if worth slicing */
- if (!list_is_last_rcu(&rq->sched.link, &engine->active.requests)) {
+ if (!list_is_last_rcu(&rq->sched.link,
+ &engine->sched_engine->requests)) {
ENGINE_TRACE(engine, "timeslice required for second inflight context\n");
return true;
}
@@ -1266,7 +1267,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
* and context switches) submission.
*/
- spin_lock(&engine->active.lock);
+ spin_lock(&sched_engine->lock);
/*
* If the queue is higher priority than the last
@@ -1366,7 +1367,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
* Even if ELSP[1] is occupied and not worthy
* of timeslices, our queue might be.
*/
- spin_unlock(&engine->active.lock);
+ spin_unlock(&sched_engine->lock);
return;
}
}
@@ -1376,7 +1377,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
while ((ve = first_virtual_engine(engine))) {
struct i915_request *rq;
- spin_lock(&ve->base.active.lock);
+ spin_lock(&ve->base.sched_engine->lock);
rq = ve->request;
if (unlikely(!virtual_matches(ve, rq, engine)))
@@ -1386,13 +1387,13 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
GEM_BUG_ON(rq->context != &ve->context);
if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
- spin_unlock(&ve->base.active.lock);
+ spin_unlock(&ve->base.sched_engine->lock);
break;
}
if (last && !can_merge_rq(last, rq)) {
- spin_unlock(&ve->base.active.lock);
- spin_unlock(&engine->active.lock);
+ spin_unlock(&ve->base.sched_engine->lock);
+ spin_unlock(&engine->sched_engine->lock);
return; /* leave this for another sibling */
}
@@ -1438,7 +1439,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
i915_request_put(rq);
unlock:
- spin_unlock(&ve->base.active.lock);
+ spin_unlock(&ve->base.sched_engine->lock);
/*
* Hmm, we have a bunch of virtual engine requests,
@@ -1554,7 +1555,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
*/
sched_engine->queue_priority_hint = queue_prio(sched_engine);
i915_sched_engine_reset_on_empty(sched_engine);
- spin_unlock(&engine->active.lock);
+ spin_unlock(&sched_engine->lock);
/*
* We can skip poking the HW if we ended up with exactly the same set
@@ -1981,7 +1982,8 @@ static void __execlists_hold(struct i915_request *rq)
__i915_request_unsubmit(rq);
clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
- list_move_tail(&rq->sched.link, &rq->engine->active.hold);
+ list_move_tail(&rq->sched.link,
+ &rq->engine->sched_engine->hold);
i915_request_set_hold(rq);
RQ_TRACE(rq, "on hold\n");
@@ -2018,7 +2020,7 @@ static bool execlists_hold(struct intel_engine_cs *engine,
if (i915_request_on_hold(rq))
return false;
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
if (__i915_request_is_complete(rq)) { /* too late! */
rq = NULL;
@@ -2034,10 +2036,10 @@ static bool execlists_hold(struct intel_engine_cs *engine,
GEM_BUG_ON(i915_request_on_hold(rq));
GEM_BUG_ON(rq->engine != engine);
__execlists_hold(rq);
- GEM_BUG_ON(list_empty(&engine->active.hold));
+ GEM_BUG_ON(list_empty(&engine->sched_engine->hold));
unlock:
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
return rq;
}
@@ -2117,7 +2119,7 @@ static void __execlists_unhold(struct i915_request *rq)
static void execlists_unhold(struct intel_engine_cs *engine,
struct i915_request *rq)
{
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
/*
* Move this request back to the priority queue, and all of its
@@ -2130,7 +2132,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
tasklet_hi_schedule(&engine->execlists.tasklet);
}
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
}
struct execlists_capture {
@@ -2260,13 +2262,13 @@ static void execlists_capture(struct intel_engine_cs *engine)
if (!cap)
return;
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
cap->rq = active_context(engine, active_ccid(engine));
if (cap->rq) {
cap->rq = active_request(cap->rq->context->timeline, cap->rq);
cap->rq = i915_request_get_rcu(cap->rq);
}
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
if (!cap->rq)
goto err_free;
@@ -2470,7 +2472,7 @@ static bool ancestor_on_hold(const struct intel_engine_cs *engine,
const struct i915_request *rq)
{
GEM_BUG_ON(i915_request_on_hold(rq));
- return !list_empty(&engine->active.hold) && hold_request(rq);
+ return !list_empty(&engine->sched_engine->hold) && hold_request(rq);
}
static void execlists_submit_request(struct i915_request *request)
@@ -2479,11 +2481,12 @@ static void execlists_submit_request(struct i915_request *request)
unsigned long flags;
/* Will be called from irq-context when using foreign fences. */
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
if (unlikely(ancestor_on_hold(engine, request))) {
RQ_TRACE(request, "ancestor on hold\n");
- list_add_tail(&request->sched.link, &engine->active.hold);
+ list_add_tail(&request->sched.link,
+ &engine->sched_engine->hold);
i915_request_set_hold(request);
} else {
queue_request(engine, request);
@@ -2495,7 +2498,7 @@ static void execlists_submit_request(struct i915_request *request)
__execlists_kick(&engine->execlists);
}
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static int
@@ -2959,9 +2962,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
/* Push back any incomplete requests for replay after the reset. */
rcu_read_lock();
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
__unwind_incomplete_requests(engine);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
rcu_read_unlock();
}
@@ -3001,10 +3004,10 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
execlists_reset_csb(engine, true);
rcu_read_lock();
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
/* Mark all executing requests as skipped. */
- list_for_each_entry(rq, &engine->active.requests, sched.link)
+ list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
i915_request_put(i915_request_mark_eio(rq));
intel_engine_signal_breadcrumbs(engine);
@@ -3024,7 +3027,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
}
/* On-hold requests will be flushed to timeline upon their release */
- list_for_each_entry(rq, &engine->active.hold, sched.link)
+ list_for_each_entry(rq, &sched_engine->hold, sched.link)
i915_request_put(i915_request_mark_eio(rq));
/* Cancel all attached virtual engines */
@@ -3035,7 +3038,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
rb_erase_cached(rb, &execlists->virtual);
RB_CLEAR_NODE(rb);
- spin_lock(&ve->base.active.lock);
+ spin_lock(&ve->base.sched_engine->lock);
rq = fetch_and_zero(&ve->request);
if (rq) {
if (i915_request_mark_eio(rq)) {
@@ -3047,7 +3050,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
ve->base.sched_engine->queue_priority_hint = INT_MIN;
}
- spin_unlock(&ve->base.active.lock);
+ spin_unlock(&ve->base.sched_engine->lock);
}
/* Remaining _unready_ requests will be nop'ed when submitted */
@@ -3058,7 +3061,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
execlists->tasklet.callback = nop_submission_tasklet;
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
rcu_read_unlock();
}
@@ -3304,7 +3307,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
if (unlikely(ve->request)) {
struct i915_request *old;
- spin_lock_irq(&ve->base.active.lock);
+ spin_lock_irq(&ve->base.sched_engine->lock);
old = fetch_and_zero(&ve->request);
if (old) {
@@ -3313,7 +3316,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
i915_request_put(old);
}
- spin_unlock_irq(&ve->base.active.lock);
+ spin_unlock_irq(&ve->base.sched_engine->lock);
}
/*
@@ -3333,13 +3336,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
if (RB_EMPTY_NODE(node))
continue;
- spin_lock_irq(&sibling->active.lock);
+ spin_lock_irq(&sibling->sched_engine->lock);
/* Detachment is lazily performed in the execlists tasklet */
if (!RB_EMPTY_NODE(node))
rb_erase_cached(node, &sibling->execlists.virtual);
- spin_unlock_irq(&sibling->active.lock);
+ spin_unlock_irq(&sibling->sched_engine->lock);
}
GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
GEM_BUG_ON(!list_empty(virtual_queue(ve)));
@@ -3509,7 +3512,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
if (!READ_ONCE(ve->request))
break; /* already handled by a sibling's tasklet */
- spin_lock_irq(&sibling->active.lock);
+ spin_lock_irq(&sibling->sched_engine->lock);
if (unlikely(!(mask & sibling->mask))) {
if (!RB_EMPTY_NODE(&node->rb)) {
@@ -3562,7 +3565,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
tasklet_hi_schedule(&sibling->execlists.tasklet);
unlock_engine:
- spin_unlock_irq(&sibling->active.lock);
+ spin_unlock_irq(&sibling->sched_engine->lock);
if (intel_context_inflight(&ve->context))
break;
@@ -3580,7 +3583,7 @@ static void virtual_submit_request(struct i915_request *rq)
GEM_BUG_ON(ve->base.submit_request != virtual_submit_request);
- spin_lock_irqsave(&ve->base.active.lock, flags);
+ spin_lock_irqsave(&ve->base.sched_engine->lock, flags);
/* By the time we resubmit a request, it may be completed */
if (__i915_request_is_complete(rq)) {
@@ -3603,7 +3606,7 @@ static void virtual_submit_request(struct i915_request *rq)
tasklet_hi_schedule(&ve->base.execlists.tasklet);
unlock:
- spin_unlock_irqrestore(&ve->base.active.lock, flags);
+ spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
}
static struct ve_bond *
@@ -3687,7 +3690,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
snprintf(ve->base.name, sizeof(ve->base.name), "virtual");
- intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
intel_engine_init_execlists(&ve->base);
ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
@@ -3860,17 +3862,17 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
unsigned int max)
{
const struct intel_engine_execlists *execlists = &engine->execlists;
- const struct i915_sched_engine *sched_engine = engine->sched_engine;
+ struct i915_sched_engine *sched_engine = engine->sched_engine;
struct i915_request *rq, *last;
unsigned long flags;
unsigned int count;
struct rb_node *rb;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&sched_engine->lock, flags);
last = NULL;
count = 0;
- list_for_each_entry(rq, &engine->active.requests, sched.link) {
+ list_for_each_entry(rq, &sched_engine->requests, sched.link) {
if (count++ < max - 1)
show_request(m, rq, "\t\t", 0);
else
@@ -3933,7 +3935,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
show_request(m, last, "\t\t", 0);
}
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&sched_engine->lock, flags);
}
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
index 2b6dffcc2262..c98ae0f58790 100644
--- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
@@ -339,9 +339,9 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
u32 head;
rq = NULL;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
rcu_read_lock();
- list_for_each_entry(pos, &engine->active.requests, sched.link) {
+ list_for_each_entry(pos, &engine->sched_engine->requests, sched.link) {
if (!__i915_request_is_complete(pos)) {
rq = pos;
break;
@@ -396,7 +396,7 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
}
engine->legacy.ring->head = intel_ring_wrap(engine->legacy.ring, head);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void reset_finish(struct intel_engine_cs *engine)
@@ -408,16 +408,16 @@ static void reset_cancel(struct intel_engine_cs *engine)
struct i915_request *request;
unsigned long flags;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
/* Mark all submitted requests as skipped. */
- list_for_each_entry(request, &engine->active.requests, sched.link)
+ list_for_each_entry(request, &engine->sched_engine->requests, sched.link)
i915_request_put(i915_request_mark_eio(request));
intel_engine_signal_breadcrumbs(engine);
/* Remaining _unready_ requests will be nop'ed when submitted */
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void i9xx_submit_request(struct i915_request *request)
diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
index b1fdba13e900..a49fd3039f13 100644
--- a/drivers/gpu/drm/i915/gt/mock_engine.c
+++ b/drivers/gpu/drm/i915/gt/mock_engine.c
@@ -253,10 +253,10 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
del_timer_sync(&mock->hw_delay);
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
/* Mark all submitted requests as skipped. */
- list_for_each_entry(rq, &engine->active.requests, sched.link)
+ list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
i915_request_put(i915_request_mark_eio(rq));
intel_engine_signal_breadcrumbs(engine);
@@ -269,7 +269,7 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
}
INIT_LIST_HEAD(&mock->hw_queue);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void mock_reset_finish(struct intel_engine_cs *engine)
@@ -350,7 +350,6 @@ int mock_engine_init(struct intel_engine_cs *engine)
if (!engine->sched_engine)
return -ENOMEM;
- intel_engine_init_active(engine, ENGINE_MOCK);
intel_engine_init_execlists(engine);
intel_engine_init__pm(engine);
intel_engine_init_retire(engine);
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index 1f93591a8c69..fd824fa329c2 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -4599,9 +4599,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
/* Fake a preemption event; failed of course */
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
__unwind_incomplete_requests(engine);
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
GEM_BUG_ON(rq->engine != engine);
/* Reset the engine while keeping our active request on hold */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index f4a6fbfaf82e..2551019ee217 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -190,7 +190,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
bool submit = false;
struct rb_node *rb;
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&sched_engine->lock);
if (last) {
if (*++first)
@@ -247,7 +247,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
struct i915_request **port, *rq;
unsigned long flags;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
for (port = execlists->inflight; (rq = *port); port++) {
if (!i915_request_completed(rq))
@@ -265,7 +265,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
i915_sched_engine_reset_on_empty(engine->sched_engine);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
@@ -322,7 +322,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
struct i915_request *rq;
unsigned long flags;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
/* Push back any incomplete requests for replay after the reset. */
rq = execlists_unwind_incomplete_requests(execlists);
@@ -336,7 +336,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
guc_reset_state(rq->context, engine, rq->head, stalled);
out_unlock:
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void guc_reset_cancel(struct intel_engine_cs *engine)
@@ -362,10 +362,10 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
* submission's irq state, we also wish to remind ourselves that
* it is irq state.)
*/
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&sched_engine->lock, flags);
/* Mark all executing requests as skipped. */
- list_for_each_entry(rq, &engine->active.requests, sched.link) {
+ list_for_each_entry(rq, &sched_engine->requests, sched.link) {
i915_request_set_error_once(rq, -EIO);
i915_request_mark_complete(rq);
}
@@ -390,7 +390,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
sched_engine->queue_priority_hint = INT_MIN;
sched_engine->queue = RB_ROOT_CACHED;
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&sched_engine->lock, flags);
}
static void guc_reset_finish(struct intel_engine_cs *engine)
@@ -539,7 +539,7 @@ static void guc_submit_request(struct i915_request *rq)
unsigned long flags;
/* Will be called from irq-context when using foreign fences. */
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
queue_request(engine, rq, rq_prio(rq));
@@ -548,7 +548,7 @@ static void guc_submit_request(struct i915_request *rq)
tasklet_hi_schedule(&engine->execlists.tasklet);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void sanitize_hwsp(struct intel_engine_cs *engine)
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 833d3e8b7631..cd005f0a203c 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -1436,12 +1436,12 @@ capture_engine(struct intel_engine_cs *engine,
if (!ee)
return NULL;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
rq = intel_engine_find_active_request(engine);
if (rq)
capture = intel_engine_coredump_add_request(ee, rq,
ATOMIC_MAYFAIL);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
if (!capture) {
kfree(ee);
return NULL;
diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
index 970d8f4986bb..95e859c319ca 100644
--- a/drivers/gpu/drm/i915/i915_request.c
+++ b/drivers/gpu/drm/i915/i915_request.c
@@ -272,11 +272,11 @@ i915_request_active_engine(struct i915_request *rq,
* check that we have acquired the lock on the final engine.
*/
locked = READ_ONCE(rq->engine);
- spin_lock_irq(&locked->active.lock);
+ spin_lock_irq(&locked->sched_engine->lock);
while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
- spin_unlock(&locked->active.lock);
+ spin_unlock(&locked->sched_engine->lock);
locked = engine;
- spin_lock(&locked->active.lock);
+ spin_lock(&locked->sched_engine->lock);
}
if (i915_request_is_active(rq)) {
@@ -285,7 +285,7 @@ i915_request_active_engine(struct i915_request *rq,
ret = true;
}
- spin_unlock_irq(&locked->active.lock);
+ spin_unlock_irq(&locked->sched_engine->lock);
return ret;
}
@@ -302,10 +302,10 @@ static void remove_from_engine(struct i915_request *rq)
* check that the rq still belongs to the newly locked engine.
*/
locked = READ_ONCE(rq->engine);
- spin_lock_irq(&locked->active.lock);
+ spin_lock_irq(&locked->sched_engine->lock);
while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
- spin_unlock(&locked->active.lock);
- spin_lock(&engine->active.lock);
+ spin_unlock(&locked->sched_engine->lock);
+ spin_lock(&engine->sched_engine->lock);
locked = engine;
}
list_del_init(&rq->sched.link);
@@ -316,7 +316,7 @@ static void remove_from_engine(struct i915_request *rq)
/* Prevent further __await_execution() registering a cb, then flush */
set_bit(I915_FENCE_FLAG_ACTIVE, &rq->fence.flags);
- spin_unlock_irq(&locked->active.lock);
+ spin_unlock_irq(&locked->sched_engine->lock);
__notify_execute_cb_imm(rq);
}
@@ -637,7 +637,7 @@ bool __i915_request_submit(struct i915_request *request)
RQ_TRACE(request, "\n");
GEM_BUG_ON(!irqs_disabled());
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
/*
* With the advent of preempt-to-busy, we frequently encounter
@@ -649,7 +649,7 @@ bool __i915_request_submit(struct i915_request *request)
*
* We must remove the request from the caller's priority queue,
* and the caller must only call us when the request is in their
- * priority queue, under the active.lock. This ensures that the
+ * priority queue, under the sched_engine->lock. This ensures that the
* request has *not* yet been retired and we can safely move
* the request into the engine->active.list where it will be
* dropped upon retiring. (Otherwise if resubmit a *retired*
@@ -694,7 +694,7 @@ bool __i915_request_submit(struct i915_request *request)
result = true;
GEM_BUG_ON(test_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags));
- list_move_tail(&request->sched.link, &engine->active.requests);
+ list_move_tail(&request->sched.link, &engine->sched_engine->requests);
active:
clear_bit(I915_FENCE_FLAG_PQUEUE, &request->fence.flags);
set_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags);
@@ -724,11 +724,11 @@ void i915_request_submit(struct i915_request *request)
unsigned long flags;
/* Will be called from irq-context when using foreign fences. */
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
__i915_request_submit(request);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
void __i915_request_unsubmit(struct i915_request *request)
@@ -742,7 +742,7 @@ void __i915_request_unsubmit(struct i915_request *request)
RQ_TRACE(request, "\n");
GEM_BUG_ON(!irqs_disabled());
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
/*
* Before we remove this breadcrumb from the signal list, we have
@@ -775,11 +775,11 @@ void i915_request_unsubmit(struct i915_request *request)
unsigned long flags;
/* Will be called from irq-context when using foreign fences. */
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
__i915_request_unsubmit(request);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void __cancel_request(struct i915_request *rq)
diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
index 270f6cd37650..239964bec1fa 100644
--- a/drivers/gpu/drm/i915/i915_request.h
+++ b/drivers/gpu/drm/i915/i915_request.h
@@ -613,7 +613,7 @@ i915_request_active_timeline(const struct i915_request *rq)
* this submission.
*/
return rcu_dereference_protected(rq->timeline,
- lockdep_is_held(&rq->engine->active.lock));
+ lockdep_is_held(&rq->engine->sched_engine->lock));
}
static inline u32
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 4953874a9ba6..4bc6969f6a97 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -68,7 +68,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
struct rb_node **parent, *rb;
bool first = true;
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
assert_priolists(sched_engine);
if (unlikely(sched_engine->no_priolist))
@@ -147,9 +147,9 @@ sched_lock_engine(const struct i915_sched_node *node,
* check that the rq still belongs to the newly locked engine.
*/
while (locked != (engine = READ_ONCE(rq->engine))) {
- spin_unlock(&locked->active.lock);
+ spin_unlock(&locked->sched_engine->lock);
memset(cache, 0, sizeof(*cache));
- spin_lock(&engine->active.lock);
+ spin_lock(&engine->sched_engine->lock);
locked = engine;
}
@@ -296,7 +296,7 @@ static void __i915_schedule(struct i915_sched_node *node,
memset(&cache, 0, sizeof(cache));
engine = node_to_request(node)->engine;
- spin_lock(&engine->active.lock);
+ spin_lock(&engine->sched_engine->lock);
/* Fifo and depth-first replacement ensure our deps execute before us */
engine = sched_lock_engine(node, engine, &cache);
@@ -305,7 +305,7 @@ static void __i915_schedule(struct i915_sched_node *node,
node = dep->signaler;
engine = sched_lock_engine(node, engine, &cache);
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
/* Recheck after acquiring the engine->timeline.lock */
if (prio <= node->attr.priority || node_signaled(node))
@@ -338,7 +338,7 @@ static void __i915_schedule(struct i915_sched_node *node,
kick_submission(engine, node_to_request(node), prio);
}
- spin_unlock(&engine->active.lock);
+ spin_unlock(&engine->sched_engine->lock);
}
void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
@@ -511,6 +511,24 @@ i915_sched_engine_create(unsigned int subclass)
sched_engine->queue = RB_ROOT_CACHED;
sched_engine->queue_priority_hint = INT_MIN;
+ INIT_LIST_HEAD(&sched_engine->requests);
+ INIT_LIST_HEAD(&sched_engine->hold);
+
+ spin_lock_init(&sched_engine->lock);
+ lockdep_set_subclass(&sched_engine->lock, subclass);
+
+ /*
+ * Due to an interesting quirk in lockdep's internal debug tracking,
+ * after setting a subclass we must ensure the lock is used. Otherwise,
+ * nr_unused_locks is incremented once too often.
+ */
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+ local_irq_disable();
+ lock_map_acquire(&sched_engine->lock.dep_map);
+ lock_map_release(&sched_engine->lock.dep_map);
+ local_irq_enable();
+#endif
+
return sched_engine;
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 16a054e67760..5cb0b1c98d6b 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -94,6 +94,15 @@ struct i915_dependency {
struct i915_sched_engine {
struct kref ref;
+ /**
+ * @lock: protects requests in priority lists, requests, hold and
+ * tasklet while running
+ */
+ spinlock_t lock;
+
+ struct list_head requests;
+ struct list_head hold; /* ready requests, but on hold */
+
/**
* @default_priolist: priority list for I915_PRIORITY_NORMAL
*/
--
2.28.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Intel-gfx] [PATCH 4/9] drm/i915: Move active tracking to i915_sched_engine
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter
Move active request tracking and its lock to i915_sched_engine. This
lock is also the submission lock so having it in the i915_sched_engine
is the correct place.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine.h | 2 -
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 43 +++-----
drivers/gpu/drm/i915/gt/intel_engine_types.h | 6 --
.../drm/i915/gt/intel_execlists_submission.c | 98 ++++++++++---------
.../gpu/drm/i915/gt/intel_ring_submission.c | 12 +--
drivers/gpu/drm/i915/gt/mock_engine.c | 7 +-
drivers/gpu/drm/i915/gt/selftest_execlists.c | 4 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 20 ++--
drivers/gpu/drm/i915/i915_gpu_error.c | 4 +-
drivers/gpu/drm/i915/i915_request.c | 32 +++---
drivers/gpu/drm/i915/i915_request.h | 2 +-
drivers/gpu/drm/i915/i915_scheduler.c | 30 ++++--
drivers/gpu/drm/i915/i915_scheduler_types.h | 9 ++
13 files changed, 134 insertions(+), 135 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
index 8d9184920c51..a8b2174b4395 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine.h
@@ -257,8 +257,6 @@ intel_engine_find_active_request(struct intel_engine_cs *engine);
u32 intel_engine_context_size(struct intel_gt *gt, u8 class);
-void intel_engine_init_active(struct intel_engine_cs *engine,
- unsigned int subclass);
#define ENGINE_PHYSICAL 0
#define ENGINE_MOCK 1
#define ENGINE_VIRTUAL 2
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 85f2effe9dc6..33d879137908 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -719,7 +719,6 @@ static int engine_setup_common(struct intel_engine_cs *engine)
if (err)
goto err_cmd_parser;
- intel_engine_init_active(engine, ENGINE_PHYSICAL);
intel_engine_init_execlists(engine);
intel_engine_init__pm(engine);
intel_engine_init_retire(engine);
@@ -778,11 +777,11 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
frame->rq.ring = &frame->ring;
mutex_lock(&ce->timeline->mutex);
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
dw = engine->emit_fini_breadcrumb(&frame->rq, frame->cs) - frame->cs;
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
mutex_unlock(&ce->timeline->mutex);
GEM_BUG_ON(dw & 1); /* RING_TAIL must be qword aligned */
@@ -791,28 +790,6 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
return dw;
}
-void
-intel_engine_init_active(struct intel_engine_cs *engine, unsigned int subclass)
-{
- INIT_LIST_HEAD(&engine->active.requests);
- INIT_LIST_HEAD(&engine->active.hold);
-
- spin_lock_init(&engine->active.lock);
- lockdep_set_subclass(&engine->active.lock, subclass);
-
- /*
- * Due to an interesting quirk in lockdep's internal debug tracking,
- * after setting a subclass we must ensure the lock is used. Otherwise,
- * nr_unused_locks is incremented once too often.
- */
-#ifdef CONFIG_DEBUG_LOCK_ALLOC
- local_irq_disable();
- lock_map_acquire(&engine->active.lock.dep_map);
- lock_map_release(&engine->active.lock.dep_map);
- local_irq_enable();
-#endif
-}
-
static struct intel_context *
create_pinned_context(struct intel_engine_cs *engine,
unsigned int hwsp,
@@ -960,7 +937,7 @@ int intel_engines_init(struct intel_gt *gt)
*/
void intel_engine_cleanup_common(struct intel_engine_cs *engine)
{
- GEM_BUG_ON(!list_empty(&engine->active.requests));
+ GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
i915_sched_engine_put(engine->sched_engine);
@@ -1353,7 +1330,7 @@ static struct intel_timeline *get_timeline(struct i915_request *rq)
struct intel_timeline *tl;
/*
- * Even though we are holding the engine->active.lock here, there
+ * Even though we are holding the engine->sched_engine->lock here, there
* is no control over the submission queue per-se and we are
* inspecting the active state at a random point in time, with an
* unknown queue. Play safe and make sure the timeline remains valid.
@@ -1700,7 +1677,7 @@ void intel_engine_dump(struct intel_engine_cs *engine,
drm_printf(m, "\tRequests:\n");
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
rq = intel_engine_find_active_request(engine);
if (rq) {
struct intel_timeline *tl = get_timeline(rq);
@@ -1731,8 +1708,9 @@ void intel_engine_dump(struct intel_engine_cs *engine,
hexdump(m, rq->context->lrc_reg_state, PAGE_SIZE);
}
}
- drm_printf(m, "\tOn hold?: %lu\n", list_count(&engine->active.hold));
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ drm_printf(m, "\tOn hold?: %lu\n",
+ list_count(&engine->sched_engine->hold));
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
drm_printf(m, "\tMMIO base: 0x%08x\n", engine->mmio_base);
wakeref = intel_runtime_pm_get_if_in_use(engine->uncore->rpm);
@@ -1812,7 +1790,7 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
* At all other times, we must assume the GPU is still running, but
* we only care about the snapshot of this moment.
*/
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
rcu_read_lock();
request = execlists_active(&engine->execlists);
@@ -1830,7 +1808,8 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
if (active)
return active;
- list_for_each_entry(request, &engine->active.requests, sched.link) {
+ list_for_each_entry(request, &engine->sched_engine->requests,
+ sched.link) {
if (__i915_request_is_complete(request))
continue;
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index 86b41ddec373..7197b9fa5e35 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -302,12 +302,6 @@ struct intel_engine_cs {
struct intel_sseu sseu;
- struct {
- spinlock_t lock;
- struct list_head requests;
- struct list_head hold; /* ready requests, but on hold */
- } active;
-
struct i915_sched_engine *sched_engine;
/* keep a request in reserve for a [pm] barrier under oom */
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 609753b5401a..0413ba87cc1e 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -325,7 +325,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
* Check against the first request in ELSP[1], it will, thanks to the
* power of PI, be the highest priority of that context.
*/
- if (!list_is_last(&rq->sched.link, &engine->active.requests) &&
+ if (!list_is_last(&rq->sched.link, &engine->sched_engine->requests) &&
rq_prio(list_next_entry(rq, sched.link)) > last_prio)
return true;
@@ -367,10 +367,10 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
struct list_head *pl;
int prio = I915_PRIORITY_INVALID;
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
list_for_each_entry_safe_reverse(rq, rn,
- &engine->active.requests,
+ &engine->sched_engine->requests,
sched.link) {
if (__i915_request_is_complete(rq)) {
list_del_init(&rq->sched.link);
@@ -534,13 +534,13 @@ resubmit_virtual_request(struct i915_request *rq, struct virtual_engine *ve)
{
struct intel_engine_cs *engine = rq->engine;
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
WRITE_ONCE(rq->engine, &ve->base);
ve->base.submit_request(rq);
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
}
static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
@@ -579,7 +579,7 @@ static void __execlists_schedule_out(struct i915_request * const rq,
unsigned int ccid;
/*
- * NB process_csb() is not under the engine->active.lock and hence
+ * NB process_csb() is not under the engine->sched_engine->lock and hence
* schedule_out can race with schedule_in meaning that we should
* refrain from doing non-trivial work here.
*/
@@ -1133,7 +1133,8 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
return false;
/* If ELSP[1] is occupied, always check to see if worth slicing */
- if (!list_is_last_rcu(&rq->sched.link, &engine->active.requests)) {
+ if (!list_is_last_rcu(&rq->sched.link,
+ &engine->sched_engine->requests)) {
ENGINE_TRACE(engine, "timeslice required for second inflight context\n");
return true;
}
@@ -1266,7 +1267,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
* and context switches) submission.
*/
- spin_lock(&engine->active.lock);
+ spin_lock(&sched_engine->lock);
/*
* If the queue is higher priority than the last
@@ -1366,7 +1367,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
* Even if ELSP[1] is occupied and not worthy
* of timeslices, our queue might be.
*/
- spin_unlock(&engine->active.lock);
+ spin_unlock(&sched_engine->lock);
return;
}
}
@@ -1376,7 +1377,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
while ((ve = first_virtual_engine(engine))) {
struct i915_request *rq;
- spin_lock(&ve->base.active.lock);
+ spin_lock(&ve->base.sched_engine->lock);
rq = ve->request;
if (unlikely(!virtual_matches(ve, rq, engine)))
@@ -1386,13 +1387,13 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
GEM_BUG_ON(rq->context != &ve->context);
if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
- spin_unlock(&ve->base.active.lock);
+ spin_unlock(&ve->base.sched_engine->lock);
break;
}
if (last && !can_merge_rq(last, rq)) {
- spin_unlock(&ve->base.active.lock);
- spin_unlock(&engine->active.lock);
+ spin_unlock(&ve->base.sched_engine->lock);
+ spin_unlock(&engine->sched_engine->lock);
return; /* leave this for another sibling */
}
@@ -1438,7 +1439,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
i915_request_put(rq);
unlock:
- spin_unlock(&ve->base.active.lock);
+ spin_unlock(&ve->base.sched_engine->lock);
/*
* Hmm, we have a bunch of virtual engine requests,
@@ -1554,7 +1555,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
*/
sched_engine->queue_priority_hint = queue_prio(sched_engine);
i915_sched_engine_reset_on_empty(sched_engine);
- spin_unlock(&engine->active.lock);
+ spin_unlock(&sched_engine->lock);
/*
* We can skip poking the HW if we ended up with exactly the same set
@@ -1981,7 +1982,8 @@ static void __execlists_hold(struct i915_request *rq)
__i915_request_unsubmit(rq);
clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
- list_move_tail(&rq->sched.link, &rq->engine->active.hold);
+ list_move_tail(&rq->sched.link,
+ &rq->engine->sched_engine->hold);
i915_request_set_hold(rq);
RQ_TRACE(rq, "on hold\n");
@@ -2018,7 +2020,7 @@ static bool execlists_hold(struct intel_engine_cs *engine,
if (i915_request_on_hold(rq))
return false;
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
if (__i915_request_is_complete(rq)) { /* too late! */
rq = NULL;
@@ -2034,10 +2036,10 @@ static bool execlists_hold(struct intel_engine_cs *engine,
GEM_BUG_ON(i915_request_on_hold(rq));
GEM_BUG_ON(rq->engine != engine);
__execlists_hold(rq);
- GEM_BUG_ON(list_empty(&engine->active.hold));
+ GEM_BUG_ON(list_empty(&engine->sched_engine->hold));
unlock:
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
return rq;
}
@@ -2117,7 +2119,7 @@ static void __execlists_unhold(struct i915_request *rq)
static void execlists_unhold(struct intel_engine_cs *engine,
struct i915_request *rq)
{
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
/*
* Move this request back to the priority queue, and all of its
@@ -2130,7 +2132,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
tasklet_hi_schedule(&engine->execlists.tasklet);
}
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
}
struct execlists_capture {
@@ -2260,13 +2262,13 @@ static void execlists_capture(struct intel_engine_cs *engine)
if (!cap)
return;
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
cap->rq = active_context(engine, active_ccid(engine));
if (cap->rq) {
cap->rq = active_request(cap->rq->context->timeline, cap->rq);
cap->rq = i915_request_get_rcu(cap->rq);
}
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
if (!cap->rq)
goto err_free;
@@ -2470,7 +2472,7 @@ static bool ancestor_on_hold(const struct intel_engine_cs *engine,
const struct i915_request *rq)
{
GEM_BUG_ON(i915_request_on_hold(rq));
- return !list_empty(&engine->active.hold) && hold_request(rq);
+ return !list_empty(&engine->sched_engine->hold) && hold_request(rq);
}
static void execlists_submit_request(struct i915_request *request)
@@ -2479,11 +2481,12 @@ static void execlists_submit_request(struct i915_request *request)
unsigned long flags;
/* Will be called from irq-context when using foreign fences. */
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
if (unlikely(ancestor_on_hold(engine, request))) {
RQ_TRACE(request, "ancestor on hold\n");
- list_add_tail(&request->sched.link, &engine->active.hold);
+ list_add_tail(&request->sched.link,
+ &engine->sched_engine->hold);
i915_request_set_hold(request);
} else {
queue_request(engine, request);
@@ -2495,7 +2498,7 @@ static void execlists_submit_request(struct i915_request *request)
__execlists_kick(&engine->execlists);
}
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static int
@@ -2959,9 +2962,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
/* Push back any incomplete requests for replay after the reset. */
rcu_read_lock();
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
__unwind_incomplete_requests(engine);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
rcu_read_unlock();
}
@@ -3001,10 +3004,10 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
execlists_reset_csb(engine, true);
rcu_read_lock();
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
/* Mark all executing requests as skipped. */
- list_for_each_entry(rq, &engine->active.requests, sched.link)
+ list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
i915_request_put(i915_request_mark_eio(rq));
intel_engine_signal_breadcrumbs(engine);
@@ -3024,7 +3027,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
}
/* On-hold requests will be flushed to timeline upon their release */
- list_for_each_entry(rq, &engine->active.hold, sched.link)
+ list_for_each_entry(rq, &sched_engine->hold, sched.link)
i915_request_put(i915_request_mark_eio(rq));
/* Cancel all attached virtual engines */
@@ -3035,7 +3038,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
rb_erase_cached(rb, &execlists->virtual);
RB_CLEAR_NODE(rb);
- spin_lock(&ve->base.active.lock);
+ spin_lock(&ve->base.sched_engine->lock);
rq = fetch_and_zero(&ve->request);
if (rq) {
if (i915_request_mark_eio(rq)) {
@@ -3047,7 +3050,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
ve->base.sched_engine->queue_priority_hint = INT_MIN;
}
- spin_unlock(&ve->base.active.lock);
+ spin_unlock(&ve->base.sched_engine->lock);
}
/* Remaining _unready_ requests will be nop'ed when submitted */
@@ -3058,7 +3061,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
execlists->tasklet.callback = nop_submission_tasklet;
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
rcu_read_unlock();
}
@@ -3304,7 +3307,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
if (unlikely(ve->request)) {
struct i915_request *old;
- spin_lock_irq(&ve->base.active.lock);
+ spin_lock_irq(&ve->base.sched_engine->lock);
old = fetch_and_zero(&ve->request);
if (old) {
@@ -3313,7 +3316,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
i915_request_put(old);
}
- spin_unlock_irq(&ve->base.active.lock);
+ spin_unlock_irq(&ve->base.sched_engine->lock);
}
/*
@@ -3333,13 +3336,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
if (RB_EMPTY_NODE(node))
continue;
- spin_lock_irq(&sibling->active.lock);
+ spin_lock_irq(&sibling->sched_engine->lock);
/* Detachment is lazily performed in the execlists tasklet */
if (!RB_EMPTY_NODE(node))
rb_erase_cached(node, &sibling->execlists.virtual);
- spin_unlock_irq(&sibling->active.lock);
+ spin_unlock_irq(&sibling->sched_engine->lock);
}
GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
GEM_BUG_ON(!list_empty(virtual_queue(ve)));
@@ -3509,7 +3512,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
if (!READ_ONCE(ve->request))
break; /* already handled by a sibling's tasklet */
- spin_lock_irq(&sibling->active.lock);
+ spin_lock_irq(&sibling->sched_engine->lock);
if (unlikely(!(mask & sibling->mask))) {
if (!RB_EMPTY_NODE(&node->rb)) {
@@ -3562,7 +3565,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
tasklet_hi_schedule(&sibling->execlists.tasklet);
unlock_engine:
- spin_unlock_irq(&sibling->active.lock);
+ spin_unlock_irq(&sibling->sched_engine->lock);
if (intel_context_inflight(&ve->context))
break;
@@ -3580,7 +3583,7 @@ static void virtual_submit_request(struct i915_request *rq)
GEM_BUG_ON(ve->base.submit_request != virtual_submit_request);
- spin_lock_irqsave(&ve->base.active.lock, flags);
+ spin_lock_irqsave(&ve->base.sched_engine->lock, flags);
/* By the time we resubmit a request, it may be completed */
if (__i915_request_is_complete(rq)) {
@@ -3603,7 +3606,7 @@ static void virtual_submit_request(struct i915_request *rq)
tasklet_hi_schedule(&ve->base.execlists.tasklet);
unlock:
- spin_unlock_irqrestore(&ve->base.active.lock, flags);
+ spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
}
static struct ve_bond *
@@ -3687,7 +3690,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
snprintf(ve->base.name, sizeof(ve->base.name), "virtual");
- intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
intel_engine_init_execlists(&ve->base);
ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
@@ -3860,17 +3862,17 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
unsigned int max)
{
const struct intel_engine_execlists *execlists = &engine->execlists;
- const struct i915_sched_engine *sched_engine = engine->sched_engine;
+ struct i915_sched_engine *sched_engine = engine->sched_engine;
struct i915_request *rq, *last;
unsigned long flags;
unsigned int count;
struct rb_node *rb;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&sched_engine->lock, flags);
last = NULL;
count = 0;
- list_for_each_entry(rq, &engine->active.requests, sched.link) {
+ list_for_each_entry(rq, &sched_engine->requests, sched.link) {
if (count++ < max - 1)
show_request(m, rq, "\t\t", 0);
else
@@ -3933,7 +3935,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
show_request(m, last, "\t\t", 0);
}
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&sched_engine->lock, flags);
}
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
index 2b6dffcc2262..c98ae0f58790 100644
--- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
@@ -339,9 +339,9 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
u32 head;
rq = NULL;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
rcu_read_lock();
- list_for_each_entry(pos, &engine->active.requests, sched.link) {
+ list_for_each_entry(pos, &engine->sched_engine->requests, sched.link) {
if (!__i915_request_is_complete(pos)) {
rq = pos;
break;
@@ -396,7 +396,7 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
}
engine->legacy.ring->head = intel_ring_wrap(engine->legacy.ring, head);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void reset_finish(struct intel_engine_cs *engine)
@@ -408,16 +408,16 @@ static void reset_cancel(struct intel_engine_cs *engine)
struct i915_request *request;
unsigned long flags;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
/* Mark all submitted requests as skipped. */
- list_for_each_entry(request, &engine->active.requests, sched.link)
+ list_for_each_entry(request, &engine->sched_engine->requests, sched.link)
i915_request_put(i915_request_mark_eio(request));
intel_engine_signal_breadcrumbs(engine);
/* Remaining _unready_ requests will be nop'ed when submitted */
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void i9xx_submit_request(struct i915_request *request)
diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
index b1fdba13e900..a49fd3039f13 100644
--- a/drivers/gpu/drm/i915/gt/mock_engine.c
+++ b/drivers/gpu/drm/i915/gt/mock_engine.c
@@ -253,10 +253,10 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
del_timer_sync(&mock->hw_delay);
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
/* Mark all submitted requests as skipped. */
- list_for_each_entry(rq, &engine->active.requests, sched.link)
+ list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
i915_request_put(i915_request_mark_eio(rq));
intel_engine_signal_breadcrumbs(engine);
@@ -269,7 +269,7 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
}
INIT_LIST_HEAD(&mock->hw_queue);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void mock_reset_finish(struct intel_engine_cs *engine)
@@ -350,7 +350,6 @@ int mock_engine_init(struct intel_engine_cs *engine)
if (!engine->sched_engine)
return -ENOMEM;
- intel_engine_init_active(engine, ENGINE_MOCK);
intel_engine_init_execlists(engine);
intel_engine_init__pm(engine);
intel_engine_init_retire(engine);
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index 1f93591a8c69..fd824fa329c2 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -4599,9 +4599,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
/* Fake a preemption event; failed of course */
- spin_lock_irq(&engine->active.lock);
+ spin_lock_irq(&engine->sched_engine->lock);
__unwind_incomplete_requests(engine);
- spin_unlock_irq(&engine->active.lock);
+ spin_unlock_irq(&engine->sched_engine->lock);
GEM_BUG_ON(rq->engine != engine);
/* Reset the engine while keeping our active request on hold */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index f4a6fbfaf82e..2551019ee217 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -190,7 +190,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
bool submit = false;
struct rb_node *rb;
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&sched_engine->lock);
if (last) {
if (*++first)
@@ -247,7 +247,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
struct i915_request **port, *rq;
unsigned long flags;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
for (port = execlists->inflight; (rq = *port); port++) {
if (!i915_request_completed(rq))
@@ -265,7 +265,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
i915_sched_engine_reset_on_empty(engine->sched_engine);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
@@ -322,7 +322,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
struct i915_request *rq;
unsigned long flags;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
/* Push back any incomplete requests for replay after the reset. */
rq = execlists_unwind_incomplete_requests(execlists);
@@ -336,7 +336,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
guc_reset_state(rq->context, engine, rq->head, stalled);
out_unlock:
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void guc_reset_cancel(struct intel_engine_cs *engine)
@@ -362,10 +362,10 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
* submission's irq state, we also wish to remind ourselves that
* it is irq state.)
*/
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&sched_engine->lock, flags);
/* Mark all executing requests as skipped. */
- list_for_each_entry(rq, &engine->active.requests, sched.link) {
+ list_for_each_entry(rq, &sched_engine->requests, sched.link) {
i915_request_set_error_once(rq, -EIO);
i915_request_mark_complete(rq);
}
@@ -390,7 +390,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
sched_engine->queue_priority_hint = INT_MIN;
sched_engine->queue = RB_ROOT_CACHED;
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&sched_engine->lock, flags);
}
static void guc_reset_finish(struct intel_engine_cs *engine)
@@ -539,7 +539,7 @@ static void guc_submit_request(struct i915_request *rq)
unsigned long flags;
/* Will be called from irq-context when using foreign fences. */
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
queue_request(engine, rq, rq_prio(rq));
@@ -548,7 +548,7 @@ static void guc_submit_request(struct i915_request *rq)
tasklet_hi_schedule(&engine->execlists.tasklet);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void sanitize_hwsp(struct intel_engine_cs *engine)
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 833d3e8b7631..cd005f0a203c 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -1436,12 +1436,12 @@ capture_engine(struct intel_engine_cs *engine,
if (!ee)
return NULL;
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
rq = intel_engine_find_active_request(engine);
if (rq)
capture = intel_engine_coredump_add_request(ee, rq,
ATOMIC_MAYFAIL);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
if (!capture) {
kfree(ee);
return NULL;
diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
index 970d8f4986bb..95e859c319ca 100644
--- a/drivers/gpu/drm/i915/i915_request.c
+++ b/drivers/gpu/drm/i915/i915_request.c
@@ -272,11 +272,11 @@ i915_request_active_engine(struct i915_request *rq,
* check that we have acquired the lock on the final engine.
*/
locked = READ_ONCE(rq->engine);
- spin_lock_irq(&locked->active.lock);
+ spin_lock_irq(&locked->sched_engine->lock);
while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
- spin_unlock(&locked->active.lock);
+ spin_unlock(&locked->sched_engine->lock);
locked = engine;
- spin_lock(&locked->active.lock);
+ spin_lock(&locked->sched_engine->lock);
}
if (i915_request_is_active(rq)) {
@@ -285,7 +285,7 @@ i915_request_active_engine(struct i915_request *rq,
ret = true;
}
- spin_unlock_irq(&locked->active.lock);
+ spin_unlock_irq(&locked->sched_engine->lock);
return ret;
}
@@ -302,10 +302,10 @@ static void remove_from_engine(struct i915_request *rq)
* check that the rq still belongs to the newly locked engine.
*/
locked = READ_ONCE(rq->engine);
- spin_lock_irq(&locked->active.lock);
+ spin_lock_irq(&locked->sched_engine->lock);
while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
- spin_unlock(&locked->active.lock);
- spin_lock(&engine->active.lock);
+ spin_unlock(&locked->sched_engine->lock);
+ spin_lock(&engine->sched_engine->lock);
locked = engine;
}
list_del_init(&rq->sched.link);
@@ -316,7 +316,7 @@ static void remove_from_engine(struct i915_request *rq)
/* Prevent further __await_execution() registering a cb, then flush */
set_bit(I915_FENCE_FLAG_ACTIVE, &rq->fence.flags);
- spin_unlock_irq(&locked->active.lock);
+ spin_unlock_irq(&locked->sched_engine->lock);
__notify_execute_cb_imm(rq);
}
@@ -637,7 +637,7 @@ bool __i915_request_submit(struct i915_request *request)
RQ_TRACE(request, "\n");
GEM_BUG_ON(!irqs_disabled());
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
/*
* With the advent of preempt-to-busy, we frequently encounter
@@ -649,7 +649,7 @@ bool __i915_request_submit(struct i915_request *request)
*
* We must remove the request from the caller's priority queue,
* and the caller must only call us when the request is in their
- * priority queue, under the active.lock. This ensures that the
+ * priority queue, under the sched_engine->lock. This ensures that the
* request has *not* yet been retired and we can safely move
* the request into the engine->active.list where it will be
* dropped upon retiring. (Otherwise if resubmit a *retired*
@@ -694,7 +694,7 @@ bool __i915_request_submit(struct i915_request *request)
result = true;
GEM_BUG_ON(test_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags));
- list_move_tail(&request->sched.link, &engine->active.requests);
+ list_move_tail(&request->sched.link, &engine->sched_engine->requests);
active:
clear_bit(I915_FENCE_FLAG_PQUEUE, &request->fence.flags);
set_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags);
@@ -724,11 +724,11 @@ void i915_request_submit(struct i915_request *request)
unsigned long flags;
/* Will be called from irq-context when using foreign fences. */
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
__i915_request_submit(request);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
void __i915_request_unsubmit(struct i915_request *request)
@@ -742,7 +742,7 @@ void __i915_request_unsubmit(struct i915_request *request)
RQ_TRACE(request, "\n");
GEM_BUG_ON(!irqs_disabled());
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
/*
* Before we remove this breadcrumb from the signal list, we have
@@ -775,11 +775,11 @@ void i915_request_unsubmit(struct i915_request *request)
unsigned long flags;
/* Will be called from irq-context when using foreign fences. */
- spin_lock_irqsave(&engine->active.lock, flags);
+ spin_lock_irqsave(&engine->sched_engine->lock, flags);
__i915_request_unsubmit(request);
- spin_unlock_irqrestore(&engine->active.lock, flags);
+ spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
static void __cancel_request(struct i915_request *rq)
diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
index 270f6cd37650..239964bec1fa 100644
--- a/drivers/gpu/drm/i915/i915_request.h
+++ b/drivers/gpu/drm/i915/i915_request.h
@@ -613,7 +613,7 @@ i915_request_active_timeline(const struct i915_request *rq)
* this submission.
*/
return rcu_dereference_protected(rq->timeline,
- lockdep_is_held(&rq->engine->active.lock));
+ lockdep_is_held(&rq->engine->sched_engine->lock));
}
static inline u32
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 4953874a9ba6..4bc6969f6a97 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -68,7 +68,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
struct rb_node **parent, *rb;
bool first = true;
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
assert_priolists(sched_engine);
if (unlikely(sched_engine->no_priolist))
@@ -147,9 +147,9 @@ sched_lock_engine(const struct i915_sched_node *node,
* check that the rq still belongs to the newly locked engine.
*/
while (locked != (engine = READ_ONCE(rq->engine))) {
- spin_unlock(&locked->active.lock);
+ spin_unlock(&locked->sched_engine->lock);
memset(cache, 0, sizeof(*cache));
- spin_lock(&engine->active.lock);
+ spin_lock(&engine->sched_engine->lock);
locked = engine;
}
@@ -296,7 +296,7 @@ static void __i915_schedule(struct i915_sched_node *node,
memset(&cache, 0, sizeof(cache));
engine = node_to_request(node)->engine;
- spin_lock(&engine->active.lock);
+ spin_lock(&engine->sched_engine->lock);
/* Fifo and depth-first replacement ensure our deps execute before us */
engine = sched_lock_engine(node, engine, &cache);
@@ -305,7 +305,7 @@ static void __i915_schedule(struct i915_sched_node *node,
node = dep->signaler;
engine = sched_lock_engine(node, engine, &cache);
- lockdep_assert_held(&engine->active.lock);
+ lockdep_assert_held(&engine->sched_engine->lock);
/* Recheck after acquiring the engine->timeline.lock */
if (prio <= node->attr.priority || node_signaled(node))
@@ -338,7 +338,7 @@ static void __i915_schedule(struct i915_sched_node *node,
kick_submission(engine, node_to_request(node), prio);
}
- spin_unlock(&engine->active.lock);
+ spin_unlock(&engine->sched_engine->lock);
}
void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
@@ -511,6 +511,24 @@ i915_sched_engine_create(unsigned int subclass)
sched_engine->queue = RB_ROOT_CACHED;
sched_engine->queue_priority_hint = INT_MIN;
+ INIT_LIST_HEAD(&sched_engine->requests);
+ INIT_LIST_HEAD(&sched_engine->hold);
+
+ spin_lock_init(&sched_engine->lock);
+ lockdep_set_subclass(&sched_engine->lock, subclass);
+
+ /*
+ * Due to an interesting quirk in lockdep's internal debug tracking,
+ * after setting a subclass we must ensure the lock is used. Otherwise,
+ * nr_unused_locks is incremented once too often.
+ */
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+ local_irq_disable();
+ lock_map_acquire(&sched_engine->lock.dep_map);
+ lock_map_release(&sched_engine->lock.dep_map);
+ local_irq_enable();
+#endif
+
return sched_engine;
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 16a054e67760..5cb0b1c98d6b 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -94,6 +94,15 @@ struct i915_dependency {
struct i915_sched_engine {
struct kref ref;
+ /**
+ * @lock: protects requests in priority lists, requests, hold and
+ * tasklet while running
+ */
+ spinlock_t lock;
+
+ struct list_head requests;
+ struct list_head hold; /* ready requests, but on hold */
+
/**
* @default_priolist: priority list for I915_PRIORITY_NORMAL
*/
--
2.28.0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 5/9] drm/i915: Move engine->schedule to i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-03 21:27 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter, daniele.ceraolospurio
The schedule function should be in the schedule object.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_wait.c | 4 ++--
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 3 ---
drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++--
drivers/gpu/drm/i915/gt/intel_engine_types.h | 8 --------
drivers/gpu/drm/i915/gt/intel_engine_user.c | 2 +-
.../gpu/drm/i915/gt/intel_execlists_submission.c | 4 ++--
drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++++++++--------
drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 4 ++--
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
drivers/gpu/drm/i915/i915_request.c | 10 +++++-----
drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++++++++
11 files changed, 31 insertions(+), 34 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_wait.c b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
index 4b9856d5ba14..af1fbf8e2a9a 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_wait.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
@@ -104,8 +104,8 @@ static void fence_set_priority(struct dma_fence *fence,
engine = rq->engine;
rcu_read_lock(); /* RCU serialisation for set-wedged protection */
- if (engine->schedule)
- engine->schedule(rq, attr);
+ if (engine->sched_engine->schedule)
+ engine->sched_engine->schedule(rq, attr);
rcu_read_unlock();
}
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 33d879137908..b480fcb1aad1 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -326,9 +326,6 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
if (engine->context_size)
DRIVER_CAPS(i915)->has_logical_contexts = true;
- /* Nothing to do here, execute in order of dependencies */
- engine->schedule = NULL;
-
ewma__engine_latency_init(&engine->latency);
seqcount_init(&engine->stats.lock);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
index b99ac41695f3..b6a305e6a974 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
@@ -121,7 +121,7 @@ static void heartbeat(struct work_struct *wrk)
* but all other contexts, including the kernel
* context are stuck waiting for the signal.
*/
- } else if (engine->schedule &&
+ } else if (engine->sched_engine->schedule &&
rq->sched.attr.priority < I915_PRIORITY_BARRIER) {
/*
* Gradually raise the priority of the heartbeat to
@@ -136,7 +136,7 @@ static void heartbeat(struct work_struct *wrk)
attr.priority = I915_PRIORITY_BARRIER;
local_bh_disable();
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
local_bh_enable();
} else {
if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index 7197b9fa5e35..f1b14aff5118 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -426,14 +426,6 @@ struct intel_engine_cs {
void (*bond_execute)(struct i915_request *rq,
struct dma_fence *signal);
- /*
- * Call when the priority on a request has changed and it and its
- * dependencies may need rescheduling. Note the request itself may
- * not be ready to run!
- */
- void (*schedule)(struct i915_request *request,
- const struct i915_sched_attr *attr);
-
void (*release)(struct intel_engine_cs *engine);
struct intel_engine_execlists execlists;
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c
index 3cca7ea2d6ea..84142127ebd8 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_user.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c
@@ -108,7 +108,7 @@ static void set_scheduler_caps(struct drm_i915_private *i915)
for_each_uabi_engine(engine, i915) { /* all engines must agree! */
int i;
- if (engine->schedule)
+ if (engine->sched_engine->schedule)
enabled |= (I915_SCHEDULER_CAP_ENABLED |
I915_SCHEDULER_CAP_PRIORITY);
else
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 0413ba87cc1e..23fd03815ad0 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -3119,7 +3119,7 @@ static bool can_preempt(struct intel_engine_cs *engine)
static void execlists_set_default_submission(struct intel_engine_cs *engine)
{
engine->submit_request = execlists_submit_request;
- engine->schedule = i915_schedule;
+ engine->sched_engine->schedule = i915_schedule;
engine->execlists.tasklet.callback = execlists_submission_tasklet;
}
@@ -3701,7 +3701,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
ve->base.cops = &virtual_context_ops;
ve->base.request_alloc = execlists_request_alloc;
- ve->base.schedule = i915_schedule;
+ ve->base.sched_engine->schedule = i915_schedule;
ve->base.submit_request = virtual_submit_request;
ve->base.bond_execute = virtual_bond_execute;
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index fd824fa329c2..5cc7648d1e5a 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -273,7 +273,7 @@ static int live_unlite_restore(struct intel_gt *gt, int prio)
};
/* Alternatively preempt the spinner with ce[1] */
- engine->schedule(rq[1], &attr);
+ engine->sched_engine->schedule(rq[1], &attr);
}
/* And switch back to ce[0] for good measure */
@@ -900,7 +900,7 @@ release_queue(struct intel_engine_cs *engine,
i915_request_add(rq);
local_bh_disable();
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
local_bh_enable(); /* kick tasklet */
i915_request_put(rq);
@@ -1325,7 +1325,7 @@ static int live_timeslice_queue(void *arg)
err = PTR_ERR(rq);
goto err_heartbeat;
}
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
err = wait_for_submit(engine, rq, HZ / 2);
if (err) {
pr_err("%s: Timed out trying to submit semaphores\n",
@@ -1867,7 +1867,7 @@ static int live_late_preempt(void *arg)
}
attr.priority = I915_PRIORITY_MAX;
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
if (!igt_wait_for_spinner(&spin_hi, rq)) {
pr_err("High priority context failed to preempt the low priority context\n");
@@ -2480,7 +2480,7 @@ static int live_suppress_self_preempt(void *arg)
i915_request_add(rq_b);
GEM_BUG_ON(i915_request_completed(rq_a));
- engine->schedule(rq_a, &attr);
+ engine->sched_engine->schedule(rq_a, &attr);
igt_spinner_end(&a.spin);
if (!igt_wait_for_spinner(&b.spin, rq_b)) {
@@ -2612,7 +2612,7 @@ static int live_chain_preempt(void *arg)
i915_request_get(rq);
i915_request_add(rq);
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
igt_spinner_end(&hi.spin);
if (i915_request_wait(rq, 0, HZ / 5) < 0) {
@@ -2971,7 +2971,7 @@ static int live_preempt_gang(void *arg)
break;
/* Submit each spinner at increasing priority */
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
} while (prio <= I915_PRIORITY_MAX &&
!__igt_timeout(end_time, NULL));
pr_debug("%s: Preempt chain of %d requests\n",
@@ -3219,7 +3219,7 @@ static int preempt_user(struct intel_engine_cs *engine,
i915_request_get(rq);
i915_request_add(rq);
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
if (i915_request_wait(rq, 0, HZ / 2) < 0)
err = -ETIME;
diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
index 5b63d4df8c93..e57dc900ae8d 100644
--- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
+++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
@@ -858,12 +858,12 @@ static int active_engine(void *data)
rq[idx] = i915_request_get(new);
i915_request_add(new);
- if (engine->schedule && arg->flags & TEST_PRIORITY) {
+ if (engine->sched_engine->schedule && arg->flags & TEST_PRIORITY) {
struct i915_sched_attr attr = {
.priority =
i915_prandom_u32_max_state(512, &prng),
};
- engine->schedule(rq[idx], &attr);
+ engine->sched_engine->schedule(rq[idx], &attr);
}
err = active_request_put(old);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 2551019ee217..4c5bbec0775d 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -641,7 +641,7 @@ static void guc_default_vfuncs(struct intel_engine_cs *engine)
engine->cops = &guc_context_ops;
engine->request_alloc = guc_request_alloc;
- engine->schedule = i915_schedule;
+ engine->sched_engine->schedule = i915_schedule;
engine->reset.prepare = guc_reset_prepare;
engine->reset.rewind = guc_reset_rewind;
diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
index 95e859c319ca..f0cb8820fb97 100644
--- a/drivers/gpu/drm/i915/i915_request.c
+++ b/drivers/gpu/drm/i915/i915_request.c
@@ -1343,7 +1343,7 @@ __i915_request_await_execution(struct i915_request *to,
}
/* Couple the dependency tree for PI on this exposed to->fence */
- if (to->engine->schedule) {
+ if (to->engine->sched_engine->schedule) {
err = i915_sched_node_add_dependency(&to->sched,
&from->sched,
I915_DEPENDENCY_WEAK);
@@ -1484,7 +1484,7 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from)
return 0;
}
- if (to->engine->schedule) {
+ if (to->engine->sched_engine->schedule) {
ret = i915_sched_node_add_dependency(&to->sched,
&from->sched,
I915_DEPENDENCY_EXTERNAL);
@@ -1671,7 +1671,7 @@ __i915_request_add_to_timeline(struct i915_request *rq)
__i915_sw_fence_await_dma_fence(&rq->submit,
&prev->fence,
&rq->dmaq);
- if (rq->engine->schedule)
+ if (rq->engine->sched_engine->schedule)
__i915_sched_node_add_dependency(&rq->sched,
&prev->sched,
&rq->dep,
@@ -1743,8 +1743,8 @@ void __i915_request_queue(struct i915_request *rq,
* decide whether to preempt the entire chain so that it is ready to
* run at the earliest possible convenience.
*/
- if (attr && rq->engine->schedule)
- rq->engine->schedule(rq, attr);
+ if (attr && rq->engine->sched_engine->schedule)
+ rq->engine->sched_engine->schedule(rq, attr);
local_bh_disable();
__i915_request_queue_bh(rq);
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 5cb0b1c98d6b..3f462f8b06f2 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -131,6 +131,14 @@ struct i915_sched_engine {
* @no_priolist: priority lists disabled
*/
bool no_priolist;
+
+ /*
+ * Call when the priority on a request has changed and it and its
+ * dependencies may need rescheduling. Note the request itself may
+ * not be ready to run!
+ */
+ void (*schedule)(struct i915_request *request,
+ const struct i915_sched_attr *attr);
};
#endif /* _I915_SCHEDULER_TYPES_H_ */
--
2.28.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Intel-gfx] [PATCH 5/9] drm/i915: Move engine->schedule to i915_sched_engine
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter
The schedule function should be in the schedule object.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_wait.c | 4 ++--
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 3 ---
drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++--
drivers/gpu/drm/i915/gt/intel_engine_types.h | 8 --------
drivers/gpu/drm/i915/gt/intel_engine_user.c | 2 +-
.../gpu/drm/i915/gt/intel_execlists_submission.c | 4 ++--
drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++++++++--------
drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 4 ++--
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
drivers/gpu/drm/i915/i915_request.c | 10 +++++-----
drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++++++++
11 files changed, 31 insertions(+), 34 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_wait.c b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
index 4b9856d5ba14..af1fbf8e2a9a 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_wait.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
@@ -104,8 +104,8 @@ static void fence_set_priority(struct dma_fence *fence,
engine = rq->engine;
rcu_read_lock(); /* RCU serialisation for set-wedged protection */
- if (engine->schedule)
- engine->schedule(rq, attr);
+ if (engine->sched_engine->schedule)
+ engine->sched_engine->schedule(rq, attr);
rcu_read_unlock();
}
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 33d879137908..b480fcb1aad1 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -326,9 +326,6 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
if (engine->context_size)
DRIVER_CAPS(i915)->has_logical_contexts = true;
- /* Nothing to do here, execute in order of dependencies */
- engine->schedule = NULL;
-
ewma__engine_latency_init(&engine->latency);
seqcount_init(&engine->stats.lock);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
index b99ac41695f3..b6a305e6a974 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
@@ -121,7 +121,7 @@ static void heartbeat(struct work_struct *wrk)
* but all other contexts, including the kernel
* context are stuck waiting for the signal.
*/
- } else if (engine->schedule &&
+ } else if (engine->sched_engine->schedule &&
rq->sched.attr.priority < I915_PRIORITY_BARRIER) {
/*
* Gradually raise the priority of the heartbeat to
@@ -136,7 +136,7 @@ static void heartbeat(struct work_struct *wrk)
attr.priority = I915_PRIORITY_BARRIER;
local_bh_disable();
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
local_bh_enable();
} else {
if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index 7197b9fa5e35..f1b14aff5118 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -426,14 +426,6 @@ struct intel_engine_cs {
void (*bond_execute)(struct i915_request *rq,
struct dma_fence *signal);
- /*
- * Call when the priority on a request has changed and it and its
- * dependencies may need rescheduling. Note the request itself may
- * not be ready to run!
- */
- void (*schedule)(struct i915_request *request,
- const struct i915_sched_attr *attr);
-
void (*release)(struct intel_engine_cs *engine);
struct intel_engine_execlists execlists;
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c
index 3cca7ea2d6ea..84142127ebd8 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_user.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c
@@ -108,7 +108,7 @@ static void set_scheduler_caps(struct drm_i915_private *i915)
for_each_uabi_engine(engine, i915) { /* all engines must agree! */
int i;
- if (engine->schedule)
+ if (engine->sched_engine->schedule)
enabled |= (I915_SCHEDULER_CAP_ENABLED |
I915_SCHEDULER_CAP_PRIORITY);
else
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 0413ba87cc1e..23fd03815ad0 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -3119,7 +3119,7 @@ static bool can_preempt(struct intel_engine_cs *engine)
static void execlists_set_default_submission(struct intel_engine_cs *engine)
{
engine->submit_request = execlists_submit_request;
- engine->schedule = i915_schedule;
+ engine->sched_engine->schedule = i915_schedule;
engine->execlists.tasklet.callback = execlists_submission_tasklet;
}
@@ -3701,7 +3701,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
ve->base.cops = &virtual_context_ops;
ve->base.request_alloc = execlists_request_alloc;
- ve->base.schedule = i915_schedule;
+ ve->base.sched_engine->schedule = i915_schedule;
ve->base.submit_request = virtual_submit_request;
ve->base.bond_execute = virtual_bond_execute;
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index fd824fa329c2..5cc7648d1e5a 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -273,7 +273,7 @@ static int live_unlite_restore(struct intel_gt *gt, int prio)
};
/* Alternatively preempt the spinner with ce[1] */
- engine->schedule(rq[1], &attr);
+ engine->sched_engine->schedule(rq[1], &attr);
}
/* And switch back to ce[0] for good measure */
@@ -900,7 +900,7 @@ release_queue(struct intel_engine_cs *engine,
i915_request_add(rq);
local_bh_disable();
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
local_bh_enable(); /* kick tasklet */
i915_request_put(rq);
@@ -1325,7 +1325,7 @@ static int live_timeslice_queue(void *arg)
err = PTR_ERR(rq);
goto err_heartbeat;
}
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
err = wait_for_submit(engine, rq, HZ / 2);
if (err) {
pr_err("%s: Timed out trying to submit semaphores\n",
@@ -1867,7 +1867,7 @@ static int live_late_preempt(void *arg)
}
attr.priority = I915_PRIORITY_MAX;
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
if (!igt_wait_for_spinner(&spin_hi, rq)) {
pr_err("High priority context failed to preempt the low priority context\n");
@@ -2480,7 +2480,7 @@ static int live_suppress_self_preempt(void *arg)
i915_request_add(rq_b);
GEM_BUG_ON(i915_request_completed(rq_a));
- engine->schedule(rq_a, &attr);
+ engine->sched_engine->schedule(rq_a, &attr);
igt_spinner_end(&a.spin);
if (!igt_wait_for_spinner(&b.spin, rq_b)) {
@@ -2612,7 +2612,7 @@ static int live_chain_preempt(void *arg)
i915_request_get(rq);
i915_request_add(rq);
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
igt_spinner_end(&hi.spin);
if (i915_request_wait(rq, 0, HZ / 5) < 0) {
@@ -2971,7 +2971,7 @@ static int live_preempt_gang(void *arg)
break;
/* Submit each spinner at increasing priority */
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
} while (prio <= I915_PRIORITY_MAX &&
!__igt_timeout(end_time, NULL));
pr_debug("%s: Preempt chain of %d requests\n",
@@ -3219,7 +3219,7 @@ static int preempt_user(struct intel_engine_cs *engine,
i915_request_get(rq);
i915_request_add(rq);
- engine->schedule(rq, &attr);
+ engine->sched_engine->schedule(rq, &attr);
if (i915_request_wait(rq, 0, HZ / 2) < 0)
err = -ETIME;
diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
index 5b63d4df8c93..e57dc900ae8d 100644
--- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
+++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
@@ -858,12 +858,12 @@ static int active_engine(void *data)
rq[idx] = i915_request_get(new);
i915_request_add(new);
- if (engine->schedule && arg->flags & TEST_PRIORITY) {
+ if (engine->sched_engine->schedule && arg->flags & TEST_PRIORITY) {
struct i915_sched_attr attr = {
.priority =
i915_prandom_u32_max_state(512, &prng),
};
- engine->schedule(rq[idx], &attr);
+ engine->sched_engine->schedule(rq[idx], &attr);
}
err = active_request_put(old);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 2551019ee217..4c5bbec0775d 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -641,7 +641,7 @@ static void guc_default_vfuncs(struct intel_engine_cs *engine)
engine->cops = &guc_context_ops;
engine->request_alloc = guc_request_alloc;
- engine->schedule = i915_schedule;
+ engine->sched_engine->schedule = i915_schedule;
engine->reset.prepare = guc_reset_prepare;
engine->reset.rewind = guc_reset_rewind;
diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
index 95e859c319ca..f0cb8820fb97 100644
--- a/drivers/gpu/drm/i915/i915_request.c
+++ b/drivers/gpu/drm/i915/i915_request.c
@@ -1343,7 +1343,7 @@ __i915_request_await_execution(struct i915_request *to,
}
/* Couple the dependency tree for PI on this exposed to->fence */
- if (to->engine->schedule) {
+ if (to->engine->sched_engine->schedule) {
err = i915_sched_node_add_dependency(&to->sched,
&from->sched,
I915_DEPENDENCY_WEAK);
@@ -1484,7 +1484,7 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from)
return 0;
}
- if (to->engine->schedule) {
+ if (to->engine->sched_engine->schedule) {
ret = i915_sched_node_add_dependency(&to->sched,
&from->sched,
I915_DEPENDENCY_EXTERNAL);
@@ -1671,7 +1671,7 @@ __i915_request_add_to_timeline(struct i915_request *rq)
__i915_sw_fence_await_dma_fence(&rq->submit,
&prev->fence,
&rq->dmaq);
- if (rq->engine->schedule)
+ if (rq->engine->sched_engine->schedule)
__i915_sched_node_add_dependency(&rq->sched,
&prev->sched,
&rq->dep,
@@ -1743,8 +1743,8 @@ void __i915_request_queue(struct i915_request *rq,
* decide whether to preempt the entire chain so that it is ready to
* run at the earliest possible convenience.
*/
- if (attr && rq->engine->schedule)
- rq->engine->schedule(rq, attr);
+ if (attr && rq->engine->sched_engine->schedule)
+ rq->engine->sched_engine->schedule(rq, attr);
local_bh_disable();
__i915_request_queue_bh(rq);
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 5cb0b1c98d6b..3f462f8b06f2 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -131,6 +131,14 @@ struct i915_sched_engine {
* @no_priolist: priority lists disabled
*/
bool no_priolist;
+
+ /*
+ * Call when the priority on a request has changed and it and its
+ * dependencies may need rescheduling. Note the request itself may
+ * not be ready to run!
+ */
+ void (*schedule)(struct i915_request *request,
+ const struct i915_sched_attr *attr);
};
#endif /* _I915_SCHEDULER_TYPES_H_ */
--
2.28.0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 6/9] drm/i915: Add kick_backend function to i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-03 21:27 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter, daniele.ceraolospurio
Rather than touching execlist specific structures in the generic
scheduling code, add a callback function in the backend.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
.../drm/i915/gt/intel_execlists_submission.c | 52 ++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.c | 62 +------------------
drivers/gpu/drm/i915/i915_scheduler_types.h | 4 ++
3 files changed, 58 insertions(+), 60 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 23fd03815ad0..3fac17103837 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -3116,10 +3116,61 @@ static bool can_preempt(struct intel_engine_cs *engine)
return engine->class != RENDER_CLASS;
}
+static void kick_execlists(const struct i915_request *rq, int prio)
+{
+ struct intel_engine_cs *engine = rq->engine;
+ struct i915_sched_engine *sched_engine = engine->sched_engine;
+ const struct i915_request *inflight;
+
+ /*
+ * We only need to kick the tasklet once for the high priority
+ * new context we add into the queue.
+ */
+ if (prio <= sched_engine->queue_priority_hint)
+ return;
+
+ rcu_read_lock();
+
+ /* Nothing currently active? We're overdue for a submission! */
+ inflight = execlists_active(&engine->execlists);
+ if (!inflight)
+ goto unlock;
+
+ /*
+ * If we are already the currently executing context, don't
+ * bother evaluating if we should preempt ourselves.
+ */
+ if (inflight->context == rq->context)
+ goto unlock;
+
+ ENGINE_TRACE(engine,
+ "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
+ prio,
+ rq->fence.context, rq->fence.seqno,
+ inflight->fence.context, inflight->fence.seqno,
+ inflight->sched.attr.priority);
+
+ sched_engine->queue_priority_hint = prio;
+
+ /*
+ * Allow preemption of low -> normal -> high, but we do
+ * not allow low priority tasks to preempt other low priority
+ * tasks under the impression that latency for low priority
+ * tasks does not matter (as much as background throughput),
+ * so kiss.
+ */
+ if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
+ tasklet_hi_schedule(&engine->execlists.tasklet);
+
+unlock:
+ rcu_read_unlock();
+}
+
static void execlists_set_default_submission(struct intel_engine_cs *engine)
{
engine->submit_request = execlists_submit_request;
engine->sched_engine->schedule = i915_schedule;
+ engine->sched_engine->kick_backend = kick_execlists;
engine->execlists.tasklet.callback = execlists_submission_tasklet;
}
@@ -3702,6 +3753,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
ve->base.request_alloc = execlists_request_alloc;
ve->base.sched_engine->schedule = i915_schedule;
+ ve->base.sched_engine->kick_backend = kick_execlists;
ve->base.submit_request = virtual_submit_request;
ve->base.bond_execute = virtual_bond_execute;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 4bc6969f6a97..035b88f2e4aa 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -157,65 +157,6 @@ sched_lock_engine(const struct i915_sched_node *node,
return locked;
}
-static inline int rq_prio(const struct i915_request *rq)
-{
- return rq->sched.attr.priority;
-}
-
-static inline bool need_preempt(int prio, int active)
-{
- /*
- * Allow preemption of low -> normal -> high, but we do
- * not allow low priority tasks to preempt other low priority
- * tasks under the impression that latency for low priority
- * tasks does not matter (as much as background throughput),
- * so kiss.
- */
- return prio >= max(I915_PRIORITY_NORMAL, active);
-}
-
-static void kick_submission(struct intel_engine_cs *engine,
- const struct i915_request *rq,
- int prio)
-{
- const struct i915_request *inflight;
-
- /*
- * We only need to kick the tasklet once for the high priority
- * new context we add into the queue.
- */
- if (prio <= engine->sched_engine->queue_priority_hint)
- return;
-
- rcu_read_lock();
-
- /* Nothing currently active? We're overdue for a submission! */
- inflight = execlists_active(&engine->execlists);
- if (!inflight)
- goto unlock;
-
- /*
- * If we are already the currently executing context, don't
- * bother evaluating if we should preempt ourselves.
- */
- if (inflight->context == rq->context)
- goto unlock;
-
- ENGINE_TRACE(engine,
- "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
- prio,
- rq->fence.context, rq->fence.seqno,
- inflight->fence.context, inflight->fence.seqno,
- inflight->sched.attr.priority);
-
- engine->sched_engine->queue_priority_hint = prio;
- if (need_preempt(prio, rq_prio(inflight)))
- tasklet_hi_schedule(&engine->execlists.tasklet);
-
-unlock:
- rcu_read_unlock();
-}
-
static void __i915_schedule(struct i915_sched_node *node,
const struct i915_sched_attr *attr)
{
@@ -335,7 +276,8 @@ static void __i915_schedule(struct i915_sched_node *node,
}
/* Defer (tasklet) submission until after all of our updates. */
- kick_submission(engine, node_to_request(node), prio);
+ if (engine->sched_engine->kick_backend)
+ engine->sched_engine->kick_backend(node_to_request(node), prio);
}
spin_unlock(&engine->sched_engine->lock);
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 3f462f8b06f2..8b3af536e6cf 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -132,6 +132,10 @@ struct i915_sched_engine {
*/
bool no_priolist;
+ /* Kick backend */
+ void (*kick_backend)(const struct i915_request *rq,
+ int prio);
+
/*
* Call when the priority on a request has changed and it and its
* dependencies may need rescheduling. Note the request itself may
--
2.28.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Intel-gfx] [PATCH 6/9] drm/i915: Add kick_backend function to i915_sched_engine
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter
Rather than touching execlist specific structures in the generic
scheduling code, add a callback function in the backend.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
.../drm/i915/gt/intel_execlists_submission.c | 52 ++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.c | 62 +------------------
drivers/gpu/drm/i915/i915_scheduler_types.h | 4 ++
3 files changed, 58 insertions(+), 60 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 23fd03815ad0..3fac17103837 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -3116,10 +3116,61 @@ static bool can_preempt(struct intel_engine_cs *engine)
return engine->class != RENDER_CLASS;
}
+static void kick_execlists(const struct i915_request *rq, int prio)
+{
+ struct intel_engine_cs *engine = rq->engine;
+ struct i915_sched_engine *sched_engine = engine->sched_engine;
+ const struct i915_request *inflight;
+
+ /*
+ * We only need to kick the tasklet once for the high priority
+ * new context we add into the queue.
+ */
+ if (prio <= sched_engine->queue_priority_hint)
+ return;
+
+ rcu_read_lock();
+
+ /* Nothing currently active? We're overdue for a submission! */
+ inflight = execlists_active(&engine->execlists);
+ if (!inflight)
+ goto unlock;
+
+ /*
+ * If we are already the currently executing context, don't
+ * bother evaluating if we should preempt ourselves.
+ */
+ if (inflight->context == rq->context)
+ goto unlock;
+
+ ENGINE_TRACE(engine,
+ "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
+ prio,
+ rq->fence.context, rq->fence.seqno,
+ inflight->fence.context, inflight->fence.seqno,
+ inflight->sched.attr.priority);
+
+ sched_engine->queue_priority_hint = prio;
+
+ /*
+ * Allow preemption of low -> normal -> high, but we do
+ * not allow low priority tasks to preempt other low priority
+ * tasks under the impression that latency for low priority
+ * tasks does not matter (as much as background throughput),
+ * so kiss.
+ */
+ if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
+ tasklet_hi_schedule(&engine->execlists.tasklet);
+
+unlock:
+ rcu_read_unlock();
+}
+
static void execlists_set_default_submission(struct intel_engine_cs *engine)
{
engine->submit_request = execlists_submit_request;
engine->sched_engine->schedule = i915_schedule;
+ engine->sched_engine->kick_backend = kick_execlists;
engine->execlists.tasklet.callback = execlists_submission_tasklet;
}
@@ -3702,6 +3753,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
ve->base.request_alloc = execlists_request_alloc;
ve->base.sched_engine->schedule = i915_schedule;
+ ve->base.sched_engine->kick_backend = kick_execlists;
ve->base.submit_request = virtual_submit_request;
ve->base.bond_execute = virtual_bond_execute;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 4bc6969f6a97..035b88f2e4aa 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -157,65 +157,6 @@ sched_lock_engine(const struct i915_sched_node *node,
return locked;
}
-static inline int rq_prio(const struct i915_request *rq)
-{
- return rq->sched.attr.priority;
-}
-
-static inline bool need_preempt(int prio, int active)
-{
- /*
- * Allow preemption of low -> normal -> high, but we do
- * not allow low priority tasks to preempt other low priority
- * tasks under the impression that latency for low priority
- * tasks does not matter (as much as background throughput),
- * so kiss.
- */
- return prio >= max(I915_PRIORITY_NORMAL, active);
-}
-
-static void kick_submission(struct intel_engine_cs *engine,
- const struct i915_request *rq,
- int prio)
-{
- const struct i915_request *inflight;
-
- /*
- * We only need to kick the tasklet once for the high priority
- * new context we add into the queue.
- */
- if (prio <= engine->sched_engine->queue_priority_hint)
- return;
-
- rcu_read_lock();
-
- /* Nothing currently active? We're overdue for a submission! */
- inflight = execlists_active(&engine->execlists);
- if (!inflight)
- goto unlock;
-
- /*
- * If we are already the currently executing context, don't
- * bother evaluating if we should preempt ourselves.
- */
- if (inflight->context == rq->context)
- goto unlock;
-
- ENGINE_TRACE(engine,
- "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
- prio,
- rq->fence.context, rq->fence.seqno,
- inflight->fence.context, inflight->fence.seqno,
- inflight->sched.attr.priority);
-
- engine->sched_engine->queue_priority_hint = prio;
- if (need_preempt(prio, rq_prio(inflight)))
- tasklet_hi_schedule(&engine->execlists.tasklet);
-
-unlock:
- rcu_read_unlock();
-}
-
static void __i915_schedule(struct i915_sched_node *node,
const struct i915_sched_attr *attr)
{
@@ -335,7 +276,8 @@ static void __i915_schedule(struct i915_sched_node *node,
}
/* Defer (tasklet) submission until after all of our updates. */
- kick_submission(engine, node_to_request(node), prio);
+ if (engine->sched_engine->kick_backend)
+ engine->sched_engine->kick_backend(node_to_request(node), prio);
}
spin_unlock(&engine->sched_engine->lock);
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 3f462f8b06f2..8b3af536e6cf 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -132,6 +132,10 @@ struct i915_sched_engine {
*/
bool no_priolist;
+ /* Kick backend */
+ void (*kick_backend)(const struct i915_request *rq,
+ int prio);
+
/*
* Call when the priority on a request has changed and it and its
* dependencies may need rescheduling. Note the request itself may
--
2.28.0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 7/9] drm/i915: Update i915_scheduler to operate on i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-03 21:27 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter, daniele.ceraolospurio
Rather passing around an intel_engine_cs in the scheduling code, pass
around a i915_sched_engine.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
.../drm/i915/gt/intel_execlists_submission.c | 11 +++--
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
drivers/gpu/drm/i915/i915_scheduler.c | 46 +++++++++----------
drivers/gpu/drm/i915/i915_scheduler.h | 2 +-
4 files changed, 32 insertions(+), 29 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 3fac17103837..7240c153be35 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -382,7 +382,8 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
GEM_BUG_ON(rq_prio(rq) == I915_PRIORITY_INVALID);
if (rq_prio(rq) != prio) {
prio = rq_prio(rq);
- pl = i915_sched_lookup_priolist(engine, prio);
+ pl = i915_sched_lookup_priolist(engine->sched_engine,
+ prio);
}
GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
@@ -1096,7 +1097,8 @@ static void defer_active(struct intel_engine_cs *engine)
if (!rq)
return;
- defer_request(rq, i915_sched_lookup_priolist(engine, rq_prio(rq)));
+ defer_request(rq, i915_sched_lookup_priolist(engine->sched_engine,
+ rq_prio(rq)));
}
static bool
@@ -2083,7 +2085,7 @@ static void __execlists_unhold(struct i915_request *rq)
i915_request_clear_hold(rq);
list_move_tail(&rq->sched.link,
- i915_sched_lookup_priolist(rq->engine,
+ i915_sched_lookup_priolist(rq->engine->sched_engine,
rq_prio(rq)));
set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
@@ -2452,7 +2454,8 @@ static void queue_request(struct intel_engine_cs *engine,
{
GEM_BUG_ON(!list_empty(&rq->sched.link));
list_add_tail(&rq->sched.link,
- i915_sched_lookup_priolist(engine, rq_prio(rq)));
+ i915_sched_lookup_priolist(engine->sched_engine,
+ rq_prio(rq)));
set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 4c5bbec0775d..7ed21670ef14 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -529,7 +529,7 @@ static inline void queue_request(struct intel_engine_cs *engine,
{
GEM_BUG_ON(!list_empty(&rq->sched.link));
list_add_tail(&rq->sched.link,
- i915_sched_lookup_priolist(engine, prio));
+ i915_sched_lookup_priolist(engine->sched_engine, prio));
set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 035b88f2e4aa..3d36e4447b5d 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -61,14 +61,13 @@ static void assert_priolists(struct i915_sched_engine * const sched_engine)
}
struct list_head *
-i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
+i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio)
{
- struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_priolist *p;
struct rb_node **parent, *rb;
bool first = true;
- lockdep_assert_held(&engine->sched_engine->lock);
+ lockdep_assert_held(&sched_engine->lock);
assert_priolists(sched_engine);
if (unlikely(sched_engine->no_priolist))
@@ -130,13 +129,13 @@ struct sched_cache {
struct list_head *priolist;
};
-static struct intel_engine_cs *
-sched_lock_engine(const struct i915_sched_node *node,
- struct intel_engine_cs *locked,
+static struct i915_sched_engine *
+lock_sched_engine(struct i915_sched_node *node,
+ struct i915_sched_engine *locked,
struct sched_cache *cache)
{
const struct i915_request *rq = node_to_request(node);
- struct intel_engine_cs *engine;
+ struct i915_sched_engine *sched_engine;
GEM_BUG_ON(!locked);
@@ -146,14 +145,14 @@ sched_lock_engine(const struct i915_sched_node *node,
* engine lock. The simple ploy we use is to take the lock then
* check that the rq still belongs to the newly locked engine.
*/
- while (locked != (engine = READ_ONCE(rq->engine))) {
- spin_unlock(&locked->sched_engine->lock);
+ while (locked != (sched_engine = rq->engine->sched_engine)) {
+ spin_unlock(&locked->lock);
memset(cache, 0, sizeof(*cache));
- spin_lock(&engine->sched_engine->lock);
- locked = engine;
+ spin_lock(&sched_engine->lock);
+ locked = sched_engine;
}
- GEM_BUG_ON(locked != engine);
+ GEM_BUG_ON(locked != sched_engine);
return locked;
}
@@ -161,7 +160,7 @@ static void __i915_schedule(struct i915_sched_node *node,
const struct i915_sched_attr *attr)
{
const int prio = max(attr->priority, node->attr.priority);
- struct intel_engine_cs *engine;
+ struct i915_sched_engine *sched_engine;
struct i915_dependency *dep, *p;
struct i915_dependency stack;
struct sched_cache cache;
@@ -236,23 +235,24 @@ static void __i915_schedule(struct i915_sched_node *node,
}
memset(&cache, 0, sizeof(cache));
- engine = node_to_request(node)->engine;
- spin_lock(&engine->sched_engine->lock);
+ sched_engine = node_to_request(node)->engine->sched_engine;
+ spin_lock(&sched_engine->lock);
/* Fifo and depth-first replacement ensure our deps execute before us */
- engine = sched_lock_engine(node, engine, &cache);
+ sched_engine = lock_sched_engine(node, sched_engine, &cache);
list_for_each_entry_safe_reverse(dep, p, &dfs, dfs_link) {
INIT_LIST_HEAD(&dep->dfs_link);
node = dep->signaler;
- engine = sched_lock_engine(node, engine, &cache);
- lockdep_assert_held(&engine->sched_engine->lock);
+ sched_engine = lock_sched_engine(node, sched_engine, &cache);
+ lockdep_assert_held(&sched_engine->lock);
/* Recheck after acquiring the engine->timeline.lock */
if (prio <= node->attr.priority || node_signaled(node))
continue;
- GEM_BUG_ON(node_to_request(node)->engine != engine);
+ GEM_BUG_ON(node_to_request(node)->engine->sched_engine !=
+ sched_engine);
WRITE_ONCE(node->attr.priority, prio);
@@ -270,17 +270,17 @@ static void __i915_schedule(struct i915_sched_node *node,
if (i915_request_in_priority_queue(node_to_request(node))) {
if (!cache.priolist)
cache.priolist =
- i915_sched_lookup_priolist(engine,
+ i915_sched_lookup_priolist(sched_engine,
prio);
list_move_tail(&node->link, cache.priolist);
}
/* Defer (tasklet) submission until after all of our updates. */
- if (engine->sched_engine->kick_backend)
- engine->sched_engine->kick_backend(node_to_request(node), prio);
+ if (sched_engine->kick_backend)
+ sched_engine->kick_backend(node_to_request(node), prio);
}
- spin_unlock(&engine->sched_engine->lock);
+ spin_unlock(&sched_engine->lock);
}
void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 713c38c99de9..0014745bda30 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -39,7 +39,7 @@ void i915_schedule(struct i915_request *request,
const struct i915_sched_attr *attr);
struct list_head *
-i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio);
+i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio);
void __i915_priolist_free(struct i915_priolist *p);
static inline void i915_priolist_free(struct i915_priolist *p)
--
2.28.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Intel-gfx] [PATCH 7/9] drm/i915: Update i915_scheduler to operate on i915_sched_engine
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter
Rather passing around an intel_engine_cs in the scheduling code, pass
around a i915_sched_engine.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
.../drm/i915/gt/intel_execlists_submission.c | 11 +++--
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
drivers/gpu/drm/i915/i915_scheduler.c | 46 +++++++++----------
drivers/gpu/drm/i915/i915_scheduler.h | 2 +-
4 files changed, 32 insertions(+), 29 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 3fac17103837..7240c153be35 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -382,7 +382,8 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
GEM_BUG_ON(rq_prio(rq) == I915_PRIORITY_INVALID);
if (rq_prio(rq) != prio) {
prio = rq_prio(rq);
- pl = i915_sched_lookup_priolist(engine, prio);
+ pl = i915_sched_lookup_priolist(engine->sched_engine,
+ prio);
}
GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
@@ -1096,7 +1097,8 @@ static void defer_active(struct intel_engine_cs *engine)
if (!rq)
return;
- defer_request(rq, i915_sched_lookup_priolist(engine, rq_prio(rq)));
+ defer_request(rq, i915_sched_lookup_priolist(engine->sched_engine,
+ rq_prio(rq)));
}
static bool
@@ -2083,7 +2085,7 @@ static void __execlists_unhold(struct i915_request *rq)
i915_request_clear_hold(rq);
list_move_tail(&rq->sched.link,
- i915_sched_lookup_priolist(rq->engine,
+ i915_sched_lookup_priolist(rq->engine->sched_engine,
rq_prio(rq)));
set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
@@ -2452,7 +2454,8 @@ static void queue_request(struct intel_engine_cs *engine,
{
GEM_BUG_ON(!list_empty(&rq->sched.link));
list_add_tail(&rq->sched.link,
- i915_sched_lookup_priolist(engine, rq_prio(rq)));
+ i915_sched_lookup_priolist(engine->sched_engine,
+ rq_prio(rq)));
set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 4c5bbec0775d..7ed21670ef14 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -529,7 +529,7 @@ static inline void queue_request(struct intel_engine_cs *engine,
{
GEM_BUG_ON(!list_empty(&rq->sched.link));
list_add_tail(&rq->sched.link,
- i915_sched_lookup_priolist(engine, prio));
+ i915_sched_lookup_priolist(engine->sched_engine, prio));
set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 035b88f2e4aa..3d36e4447b5d 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -61,14 +61,13 @@ static void assert_priolists(struct i915_sched_engine * const sched_engine)
}
struct list_head *
-i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
+i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio)
{
- struct i915_sched_engine * const sched_engine = engine->sched_engine;
struct i915_priolist *p;
struct rb_node **parent, *rb;
bool first = true;
- lockdep_assert_held(&engine->sched_engine->lock);
+ lockdep_assert_held(&sched_engine->lock);
assert_priolists(sched_engine);
if (unlikely(sched_engine->no_priolist))
@@ -130,13 +129,13 @@ struct sched_cache {
struct list_head *priolist;
};
-static struct intel_engine_cs *
-sched_lock_engine(const struct i915_sched_node *node,
- struct intel_engine_cs *locked,
+static struct i915_sched_engine *
+lock_sched_engine(struct i915_sched_node *node,
+ struct i915_sched_engine *locked,
struct sched_cache *cache)
{
const struct i915_request *rq = node_to_request(node);
- struct intel_engine_cs *engine;
+ struct i915_sched_engine *sched_engine;
GEM_BUG_ON(!locked);
@@ -146,14 +145,14 @@ sched_lock_engine(const struct i915_sched_node *node,
* engine lock. The simple ploy we use is to take the lock then
* check that the rq still belongs to the newly locked engine.
*/
- while (locked != (engine = READ_ONCE(rq->engine))) {
- spin_unlock(&locked->sched_engine->lock);
+ while (locked != (sched_engine = rq->engine->sched_engine)) {
+ spin_unlock(&locked->lock);
memset(cache, 0, sizeof(*cache));
- spin_lock(&engine->sched_engine->lock);
- locked = engine;
+ spin_lock(&sched_engine->lock);
+ locked = sched_engine;
}
- GEM_BUG_ON(locked != engine);
+ GEM_BUG_ON(locked != sched_engine);
return locked;
}
@@ -161,7 +160,7 @@ static void __i915_schedule(struct i915_sched_node *node,
const struct i915_sched_attr *attr)
{
const int prio = max(attr->priority, node->attr.priority);
- struct intel_engine_cs *engine;
+ struct i915_sched_engine *sched_engine;
struct i915_dependency *dep, *p;
struct i915_dependency stack;
struct sched_cache cache;
@@ -236,23 +235,24 @@ static void __i915_schedule(struct i915_sched_node *node,
}
memset(&cache, 0, sizeof(cache));
- engine = node_to_request(node)->engine;
- spin_lock(&engine->sched_engine->lock);
+ sched_engine = node_to_request(node)->engine->sched_engine;
+ spin_lock(&sched_engine->lock);
/* Fifo and depth-first replacement ensure our deps execute before us */
- engine = sched_lock_engine(node, engine, &cache);
+ sched_engine = lock_sched_engine(node, sched_engine, &cache);
list_for_each_entry_safe_reverse(dep, p, &dfs, dfs_link) {
INIT_LIST_HEAD(&dep->dfs_link);
node = dep->signaler;
- engine = sched_lock_engine(node, engine, &cache);
- lockdep_assert_held(&engine->sched_engine->lock);
+ sched_engine = lock_sched_engine(node, sched_engine, &cache);
+ lockdep_assert_held(&sched_engine->lock);
/* Recheck after acquiring the engine->timeline.lock */
if (prio <= node->attr.priority || node_signaled(node))
continue;
- GEM_BUG_ON(node_to_request(node)->engine != engine);
+ GEM_BUG_ON(node_to_request(node)->engine->sched_engine !=
+ sched_engine);
WRITE_ONCE(node->attr.priority, prio);
@@ -270,17 +270,17 @@ static void __i915_schedule(struct i915_sched_node *node,
if (i915_request_in_priority_queue(node_to_request(node))) {
if (!cache.priolist)
cache.priolist =
- i915_sched_lookup_priolist(engine,
+ i915_sched_lookup_priolist(sched_engine,
prio);
list_move_tail(&node->link, cache.priolist);
}
/* Defer (tasklet) submission until after all of our updates. */
- if (engine->sched_engine->kick_backend)
- engine->sched_engine->kick_backend(node_to_request(node), prio);
+ if (sched_engine->kick_backend)
+ sched_engine->kick_backend(node_to_request(node), prio);
}
- spin_unlock(&engine->sched_engine->lock);
+ spin_unlock(&sched_engine->lock);
}
void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 713c38c99de9..0014745bda30 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -39,7 +39,7 @@ void i915_schedule(struct i915_request *request,
const struct i915_sched_attr *attr);
struct list_head *
-i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio);
+i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio);
void __i915_priolist_free(struct i915_priolist *p);
static inline void i915_priolist_free(struct i915_priolist *p)
--
2.28.0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 8/9] drm/i915: Move submission tasklet to i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-03 21:27 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter, daniele.ceraolospurio
The submission tasklet operates on i915_sched_engine, thus it is the
correct place for it.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine.h | 14 ---
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 12 +--
drivers/gpu/drm/i915/gt/intel_engine_types.h | 5 --
.../drm/i915/gt/intel_execlists_submission.c | 86 ++++++++++---------
drivers/gpu/drm/i915/gt/mock_engine.c | 1 +
drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++--
drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 2 +-
drivers/gpu/drm/i915/gt/selftest_lrc.c | 6 +-
drivers/gpu/drm/i915/gt/selftest_reset.c | 2 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 25 +++---
drivers/gpu/drm/i915/i915_scheduler.c | 1 +
drivers/gpu/drm/i915/i915_scheduler.h | 14 +++
drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++
13 files changed, 99 insertions(+), 93 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
index a8b2174b4395..988d9688ae4d 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine.h
@@ -123,20 +123,6 @@ execlists_active(const struct intel_engine_execlists *execlists)
return active;
}
-static inline void
-execlists_active_lock_bh(struct intel_engine_execlists *execlists)
-{
- local_bh_disable(); /* prevent local softirq and lock recursion */
- tasklet_lock(&execlists->tasklet);
-}
-
-static inline void
-execlists_active_unlock_bh(struct intel_engine_execlists *execlists)
-{
- tasklet_unlock(&execlists->tasklet);
- local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
-}
-
struct i915_request *
execlists_unwind_incomplete_requests(struct intel_engine_execlists *execlists);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index b480fcb1aad1..837d2132e31b 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -711,6 +711,7 @@ static int engine_setup_common(struct intel_engine_cs *engine)
err = -ENOMEM;
goto err_sched_engine;
}
+ engine->sched_engine->engine = engine;
err = intel_engine_init_cmd_parser(engine);
if (err)
@@ -935,7 +936,6 @@ int intel_engines_init(struct intel_gt *gt)
void intel_engine_cleanup_common(struct intel_engine_cs *engine)
{
GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
- tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
i915_sched_engine_put(engine->sched_engine);
intel_breadcrumbs_free(engine->breadcrumbs);
@@ -1221,7 +1221,7 @@ static bool ring_is_idle(struct intel_engine_cs *engine)
void __intel_engine_flush_submission(struct intel_engine_cs *engine, bool sync)
{
- struct tasklet_struct *t = &engine->execlists.tasklet;
+ struct tasklet_struct *t = &engine->sched_engine->tasklet;
if (!t->callback)
return;
@@ -1482,8 +1482,8 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
drm_printf(m, "\tExeclist tasklet queued? %s (%s), preempt? %s, timeslice? %s\n",
yesno(test_bit(TASKLET_STATE_SCHED,
- &engine->execlists.tasklet.state)),
- enableddisabled(!atomic_read(&engine->execlists.tasklet.count)),
+ &engine->sched_engine->tasklet.state)),
+ enableddisabled(!atomic_read(&engine->sched_engine->tasklet.count)),
repr_timer(&engine->execlists.preempt),
repr_timer(&engine->execlists.timer));
@@ -1507,7 +1507,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
idx, hws[idx * 2], hws[idx * 2 + 1]);
}
- execlists_active_lock_bh(execlists);
+ i915_sched_engine_active_lock_bh(engine->sched_engine);
rcu_read_lock();
for (port = execlists->active; (rq = *port); port++) {
char hdr[160];
@@ -1538,7 +1538,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
i915_request_show(m, rq, hdr, 0);
}
rcu_read_unlock();
- execlists_active_unlock_bh(execlists);
+ i915_sched_engine_active_unlock_bh(engine->sched_engine);
} else if (INTEL_GEN(dev_priv) > 6) {
drm_printf(m, "\tPP_DIR_BASE: 0x%08x\n",
ENGINE_READ(engine, RING_PP_DIR_BASE));
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index f1b14aff5118..6f2fd6f13ac4 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -138,11 +138,6 @@ struct st_preempt_hang {
* driver and the hardware state for execlist mode of submission.
*/
struct intel_engine_execlists {
- /**
- * @tasklet: softirq tasklet for bottom handler
- */
- struct tasklet_struct tasklet;
-
/**
* @timer: kick the current context if its timeslice expires
*/
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 7240c153be35..7a93aec3f6a9 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -570,7 +570,7 @@ static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
resubmit_virtual_request(rq, ve);
if (READ_ONCE(ve->request))
- tasklet_hi_schedule(&ve->base.execlists.tasklet);
+ tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
}
static void __execlists_schedule_out(struct i915_request * const rq,
@@ -739,9 +739,9 @@ trace_ports(const struct intel_engine_execlists *execlists,
}
static bool
-reset_in_progress(const struct intel_engine_execlists *execlists)
+reset_in_progress(const struct intel_engine_cs *engine)
{
- return unlikely(!__tasklet_is_enabled(&execlists->tasklet));
+ return unlikely(!__tasklet_is_enabled(&engine->sched_engine->tasklet));
}
static __maybe_unused noinline bool
@@ -757,7 +757,7 @@ assert_pending_valid(const struct intel_engine_execlists *execlists,
trace_ports(execlists, msg, execlists->pending);
/* We may be messing around with the lists during reset, lalala */
- if (reset_in_progress(execlists))
+ if (reset_in_progress(engine))
return true;
if (!execlists->pending[0]) {
@@ -1190,7 +1190,7 @@ static void start_timeslice(struct intel_engine_cs *engine)
* its timeslice, so recheck.
*/
if (!timer_pending(&el->timer))
- tasklet_hi_schedule(&el->tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
return;
}
@@ -1772,8 +1772,8 @@ process_csb(struct intel_engine_cs *engine, struct i915_request **inactive)
* access. Either we are inside the tasklet, or the tasklet is disabled
* and we assume that is only inside the reset paths and so serialised.
*/
- GEM_BUG_ON(!tasklet_is_locked(&execlists->tasklet) &&
- !reset_in_progress(execlists));
+ GEM_BUG_ON(!tasklet_is_locked(&engine->sched_engine->tasklet) &&
+ !reset_in_progress(engine));
/*
* Note that csb_write, csb_status may be either in HWSP or mmio.
@@ -2131,7 +2131,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
engine->sched_engine->queue_priority_hint = rq_prio(rq);
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
}
spin_unlock_irq(&engine->sched_engine->lock);
@@ -2322,13 +2322,13 @@ static void execlists_reset(struct intel_engine_cs *engine, const char *msg)
ENGINE_TRACE(engine, "reset for %s\n", msg);
/* Mark this tasklet as disabled to avoid waiting for it to complete */
- tasklet_disable_nosync(&engine->execlists.tasklet);
+ tasklet_disable_nosync(&engine->sched_engine->tasklet);
ring_set_paused(engine, 1); /* Freeze the current request in place */
execlists_capture(engine);
intel_engine_reset(engine, msg);
- tasklet_enable(&engine->execlists.tasklet);
+ tasklet_enable(&engine->sched_engine->tasklet);
clear_and_wake_up_bit(bit, lock);
}
@@ -2351,8 +2351,9 @@ static bool preempt_timeout(const struct intel_engine_cs *const engine)
*/
static void execlists_submission_tasklet(struct tasklet_struct *t)
{
- struct intel_engine_cs * const engine =
- from_tasklet(engine, t, execlists.tasklet);
+ struct i915_sched_engine *sched_engine =
+ from_tasklet(sched_engine, t, tasklet);
+ struct intel_engine_cs * const engine = sched_engine->engine;
struct i915_request *post[2 * EXECLIST_MAX_PORTS];
struct i915_request **inactive;
@@ -2427,13 +2428,16 @@ static void execlists_irq_handler(struct intel_engine_cs *engine, u16 iir)
intel_engine_signal_breadcrumbs(engine);
if (tasklet)
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
}
static void __execlists_kick(struct intel_engine_execlists *execlists)
{
+ struct intel_engine_cs *engine =
+ container_of(execlists, typeof(*engine), execlists);
+
/* Kick the tasklet for some interrupt coalescing and reset handling */
- tasklet_hi_schedule(&execlists->tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
}
#define execlists_kick(t, member) \
@@ -2808,10 +2812,8 @@ static int execlists_resume(struct intel_engine_cs *engine)
static void execlists_reset_prepare(struct intel_engine_cs *engine)
{
- struct intel_engine_execlists * const execlists = &engine->execlists;
-
ENGINE_TRACE(engine, "depth<-%d\n",
- atomic_read(&execlists->tasklet.count));
+ atomic_read(&engine->sched_engine->tasklet.count));
/*
* Prevent request submission to the hardware until we have
@@ -2822,8 +2824,8 @@ static void execlists_reset_prepare(struct intel_engine_cs *engine)
* Turning off the execlists->tasklet until the reset is over
* prevents the race.
*/
- __tasklet_disable_sync_once(&execlists->tasklet);
- GEM_BUG_ON(!reset_in_progress(execlists));
+ __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
+ GEM_BUG_ON(!reset_in_progress(engine));
/*
* We stop engines, otherwise we might get failed reset and a
@@ -2973,8 +2975,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
static void nop_submission_tasklet(struct tasklet_struct *t)
{
- struct intel_engine_cs * const engine =
- from_tasklet(engine, t, execlists.tasklet);
+ struct i915_sched_engine *sched_engine =
+ from_tasklet(sched_engine, t, tasklet);
+ struct intel_engine_cs * const engine = sched_engine->engine;
/* The driver is wedged; don't process any more events. */
WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
@@ -3061,8 +3064,8 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
sched_engine->queue_priority_hint = INT_MIN;
sched_engine->queue = RB_ROOT_CACHED;
- GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
- execlists->tasklet.callback = nop_submission_tasklet;
+ GEM_BUG_ON(__tasklet_is_enabled(&engine->sched_engine->tasklet));
+ engine->sched_engine->tasklet.callback = nop_submission_tasklet;
spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
rcu_read_unlock();
@@ -3082,14 +3085,14 @@ static void execlists_reset_finish(struct intel_engine_cs *engine)
* reset as the next level of recovery, and as a final resort we
* will declare the device wedged.
*/
- GEM_BUG_ON(!reset_in_progress(execlists));
+ GEM_BUG_ON(!reset_in_progress(engine));
/* And kick in case we missed a new request submission. */
- if (__tasklet_enable(&execlists->tasklet))
+ if (__tasklet_enable(&engine->sched_engine->tasklet))
__execlists_kick(execlists);
ENGINE_TRACE(engine, "depth->%d\n",
- atomic_read(&execlists->tasklet.count));
+ atomic_read(&engine->sched_engine->tasklet.count));
}
static void gen8_logical_ring_enable_irq(struct intel_engine_cs *engine)
@@ -3153,8 +3156,6 @@ static void kick_execlists(const struct i915_request *rq, int prio)
inflight->fence.context, inflight->fence.seqno,
inflight->sched.attr.priority);
- sched_engine->queue_priority_hint = prio;
-
/*
* Allow preemption of low -> normal -> high, but we do
* not allow low priority tasks to preempt other low priority
@@ -3163,7 +3164,7 @@ static void kick_execlists(const struct i915_request *rq, int prio)
* so kiss.
*/
if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&sched_engine->tasklet);
unlock:
rcu_read_unlock();
@@ -3174,7 +3175,7 @@ static void execlists_set_default_submission(struct intel_engine_cs *engine)
engine->submit_request = execlists_submit_request;
engine->sched_engine->schedule = i915_schedule;
engine->sched_engine->kick_backend = kick_execlists;
- engine->execlists.tasklet.callback = execlists_submission_tasklet;
+ engine->sched_engine->tasklet.callback = execlists_submission_tasklet;
}
static void execlists_shutdown(struct intel_engine_cs *engine)
@@ -3182,7 +3183,7 @@ static void execlists_shutdown(struct intel_engine_cs *engine)
/* Synchronise with residual timers and any softirq they raise */
del_timer_sync(&engine->execlists.timer);
del_timer_sync(&engine->execlists.preempt);
- tasklet_kill(&engine->execlists.tasklet);
+ tasklet_kill(&engine->sched_engine->tasklet);
}
static void execlists_release(struct intel_engine_cs *engine)
@@ -3298,7 +3299,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
struct intel_uncore *uncore = engine->uncore;
u32 base = engine->mmio_base;
- tasklet_setup(&engine->execlists.tasklet, execlists_submission_tasklet);
+ tasklet_setup(&engine->sched_engine->tasklet, execlists_submission_tasklet);
timer_setup(&engine->execlists.timer, execlists_timeslice, 0);
timer_setup(&engine->execlists.preempt, execlists_preempt, 0);
@@ -3380,7 +3381,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
* rbtrees as in the case it is running in parallel, it may reinsert
* the rb_node into a sibling.
*/
- tasklet_kill(&ve->base.execlists.tasklet);
+ tasklet_kill(&ve->base.sched_engine->tasklet);
/* Decouple ourselves from the siblings, no more access allowed. */
for (n = 0; n < ve->num_siblings; n++) {
@@ -3392,13 +3393,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
spin_lock_irq(&sibling->sched_engine->lock);
- /* Detachment is lazily performed in the execlists tasklet */
+ /* Detachment is lazily performed in the sched_engine->tasklet */
if (!RB_EMPTY_NODE(node))
rb_erase_cached(node, &sibling->execlists.virtual);
spin_unlock_irq(&sibling->sched_engine->lock);
}
- GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
+ GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.sched_engine->tasklet));
GEM_BUG_ON(!list_empty(virtual_queue(ve)));
lrc_fini(&ve->context);
@@ -3545,9 +3546,11 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
static void virtual_submission_tasklet(struct tasklet_struct *t)
{
+ struct i915_sched_engine *sched_engine =
+ from_tasklet(sched_engine, t, tasklet);
struct virtual_engine * const ve =
- from_tasklet(ve, t, base.execlists.tasklet);
- const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
+ (struct virtual_engine *)sched_engine->engine;
+ const int prio = READ_ONCE(sched_engine->queue_priority_hint);
intel_engine_mask_t mask;
unsigned int n;
@@ -3616,7 +3619,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
node->prio = prio;
if (first && prio > sibling->sched_engine->queue_priority_hint)
- tasklet_hi_schedule(&sibling->execlists.tasklet);
+ tasklet_hi_schedule(&sibling->sched_engine->tasklet);
unlock_engine:
spin_unlock_irq(&sibling->sched_engine->lock);
@@ -3657,7 +3660,7 @@ static void virtual_submit_request(struct i915_request *rq)
GEM_BUG_ON(!list_empty(virtual_queue(ve)));
list_move_tail(&rq->sched.link, virtual_queue(ve));
- tasklet_hi_schedule(&ve->base.execlists.tasklet);
+ tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
unlock:
spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
@@ -3751,6 +3754,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
err = -ENOMEM;
goto err_put;
}
+ ve->base.sched_engine->engine = &ve->base;
ve->base.cops = &virtual_context_ops;
ve->base.request_alloc = execlists_request_alloc;
@@ -3761,7 +3765,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
ve->base.bond_execute = virtual_bond_execute;
INIT_LIST_HEAD(virtual_queue(ve));
- tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
+ tasklet_setup(&ve->base.sched_engine->tasklet, virtual_submission_tasklet);
intel_context_init(&ve->context, &ve->base);
@@ -3789,7 +3793,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
* layering if we handle cloning of the requests and
* submitting a copy into each backend.
*/
- if (sibling->execlists.tasklet.callback !=
+ if (sibling->sched_engine->tasklet.callback !=
execlists_submission_tasklet) {
err = -ENODEV;
goto err_put;
diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
index a49fd3039f13..bd005c1b6fd5 100644
--- a/drivers/gpu/drm/i915/gt/mock_engine.c
+++ b/drivers/gpu/drm/i915/gt/mock_engine.c
@@ -349,6 +349,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
if (!engine->sched_engine)
return -ENOMEM;
+ engine->sched_engine->engine = engine;
intel_engine_init_execlists(engine);
intel_engine_init__pm(engine);
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index 5cc7648d1e5a..2554a2f343b4 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -43,7 +43,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
unsigned long timeout)
{
/* Ignore our own attempts to suppress excess tasklets */
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
timeout += jiffies;
do {
@@ -606,9 +606,9 @@ static int live_hold_reset(void *arg)
err = -EBUSY;
goto out;
}
- tasklet_disable(&engine->execlists.tasklet);
+ tasklet_disable(&engine->sched_engine->tasklet);
- engine->execlists.tasklet.callback(&engine->execlists.tasklet);
+ engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
i915_request_get(rq);
@@ -618,7 +618,7 @@ static int live_hold_reset(void *arg)
__intel_engine_reset_bh(engine, NULL);
GEM_BUG_ON(rq->fence.error != -EIO);
- tasklet_enable(&engine->execlists.tasklet);
+ tasklet_enable(&engine->sched_engine->tasklet);
clear_and_wake_up_bit(I915_RESET_ENGINE + id,
>->reset.flags);
local_bh_enable();
@@ -1183,7 +1183,7 @@ static int live_timeslice_rewind(void *arg)
while (i915_request_is_active(rq[A2])) { /* semaphore yield! */
/* Wait for the timeslice to kick in */
del_timer(&engine->execlists.timer);
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
intel_engine_flush_submission(engine);
}
/* -> ELSP[] = { { A:rq1 }, { B:rq1 } } */
@@ -4593,9 +4593,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
err = -EBUSY;
goto out_heartbeat;
}
- tasklet_disable(&engine->execlists.tasklet);
+ tasklet_disable(&engine->sched_engine->tasklet);
- engine->execlists.tasklet.callback(&engine->execlists.tasklet);
+ engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
/* Fake a preemption event; failed of course */
@@ -4612,7 +4612,7 @@ static int reset_virtual_engine(struct intel_gt *gt,
GEM_BUG_ON(rq->fence.error != -EIO);
/* Release our grasp on the engine, letting CS flow again */
- tasklet_enable(&engine->execlists.tasklet);
+ tasklet_enable(&engine->sched_engine->tasklet);
clear_and_wake_up_bit(I915_RESET_ENGINE + engine->id, >->reset.flags);
local_bh_enable();
diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
index e57dc900ae8d..cbcb800e2ca0 100644
--- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
+++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
@@ -1702,7 +1702,7 @@ static int __igt_atomic_reset_engine(struct intel_engine_cs *engine,
const struct igt_atomic_section *p,
const char *mode)
{
- struct tasklet_struct * const t = &engine->execlists.tasklet;
+ struct tasklet_struct * const t = &engine->sched_engine->tasklet;
int err;
GEM_TRACE("i915_reset_engine(%s:%s) under %s\n",
diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c
index d8f6623524e8..6a3a0d89dcd2 100644
--- a/drivers/gpu/drm/i915/gt/selftest_lrc.c
+++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c
@@ -49,7 +49,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
unsigned long timeout)
{
/* Ignore our own attempts to suppress excess tasklets */
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
timeout += jiffies;
do {
@@ -1613,12 +1613,12 @@ static void garbage_reset(struct intel_engine_cs *engine,
local_bh_disable();
if (!test_and_set_bit(bit, lock)) {
- tasklet_disable(&engine->execlists.tasklet);
+ tasklet_disable(&engine->sched_engine->tasklet);
if (!rq->fence.error)
__intel_engine_reset_bh(engine, NULL);
- tasklet_enable(&engine->execlists.tasklet);
+ tasklet_enable(&engine->sched_engine->tasklet);
clear_and_wake_up_bit(bit, lock);
}
local_bh_enable();
diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c
index 8784257ec808..7a50c9f4071b 100644
--- a/drivers/gpu/drm/i915/gt/selftest_reset.c
+++ b/drivers/gpu/drm/i915/gt/selftest_reset.c
@@ -321,7 +321,7 @@ static int igt_atomic_engine_reset(void *arg)
goto out_unlock;
for_each_engine(engine, gt, id) {
- struct tasklet_struct *t = &engine->execlists.tasklet;
+ struct tasklet_struct *t = &engine->sched_engine->tasklet;
if (t->func)
tasklet_disable(t);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 7ed21670ef14..95c6f6af4047 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -241,8 +241,9 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
static void guc_submission_tasklet(struct tasklet_struct *t)
{
- struct intel_engine_cs * const engine =
- from_tasklet(engine, t, execlists.tasklet);
+ struct i915_sched_engine *sched_engine =
+ from_tasklet(sched_engine, t, tasklet);
+ struct intel_engine_cs * const engine = sched_engine->engine;
struct intel_engine_execlists * const execlists = &engine->execlists;
struct i915_request **port, *rq;
unsigned long flags;
@@ -272,14 +273,12 @@ static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
{
if (iir & GT_RENDER_USER_INTERRUPT) {
intel_engine_signal_breadcrumbs(engine);
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
}
}
static void guc_reset_prepare(struct intel_engine_cs *engine)
{
- struct intel_engine_execlists * const execlists = &engine->execlists;
-
ENGINE_TRACE(engine, "\n");
/*
@@ -291,7 +290,7 @@ static void guc_reset_prepare(struct intel_engine_cs *engine)
* Turning off the execlists->tasklet until the reset is over
* prevents the race.
*/
- __tasklet_disable_sync_once(&execlists->tasklet);
+ __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
}
static void guc_reset_state(struct intel_context *ce,
@@ -395,14 +394,12 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
static void guc_reset_finish(struct intel_engine_cs *engine)
{
- struct intel_engine_execlists * const execlists = &engine->execlists;
-
- if (__tasklet_enable(&execlists->tasklet))
+ if (__tasklet_enable(&engine->sched_engine->tasklet))
/* And kick in case we missed a new request submission. */
- tasklet_hi_schedule(&execlists->tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
ENGINE_TRACE(engine, "depth->%d\n",
- atomic_read(&execlists->tasklet.count));
+ atomic_read(&engine->sched_engine->tasklet.count));
}
/*
@@ -546,7 +543,7 @@ static void guc_submit_request(struct i915_request *rq)
GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
GEM_BUG_ON(list_empty(&rq->sched.link));
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
@@ -626,7 +623,7 @@ static void guc_release(struct intel_engine_cs *engine)
{
engine->sanitize = NULL; /* no longer in control, nothing to sanitize */
- tasklet_kill(&engine->execlists.tasklet);
+ tasklet_kill(&engine->sched_engine->tasklet);
intel_engine_cleanup_common(engine);
lrc_fini_wa_ctx(engine);
@@ -705,7 +702,7 @@ int intel_guc_submission_setup(struct intel_engine_cs *engine)
*/
GEM_BUG_ON(INTEL_GEN(i915) < 11);
- tasklet_setup(&engine->execlists.tasklet, guc_submission_tasklet);
+ tasklet_setup(&engine->sched_engine->tasklet, guc_submission_tasklet);
guc_default_vfuncs(engine);
guc_default_irqs(engine);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 3d36e4447b5d..6f082579ee9e 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -436,6 +436,7 @@ void i915_sched_engine_free(struct kref *kref)
struct i915_sched_engine *sched_engine =
container_of(kref, typeof(*sched_engine), ref);
+ tasklet_kill(&sched_engine->tasklet); /* flush the callback */
kfree(sched_engine);
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 0014745bda30..650ab8e0db9f 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -79,6 +79,20 @@ i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
sched_engine->no_priolist = false;
}
+static inline void
+i915_sched_engine_active_lock_bh(struct i915_sched_engine *sched_engine)
+{
+ local_bh_disable(); /* prevent local softirq and lock recursion */
+ tasklet_lock(&sched_engine->tasklet);
+}
+
+static inline void
+i915_sched_engine_active_unlock_bh(struct i915_sched_engine *sched_engine)
+{
+ tasklet_unlock(&sched_engine->tasklet);
+ local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
+}
+
void i915_request_show_with_schedule(struct drm_printer *m,
const struct i915_request *rq,
const char *prefix,
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 8b3af536e6cf..9d79514450de 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -103,6 +103,11 @@ struct i915_sched_engine {
struct list_head requests;
struct list_head hold; /* ready requests, but on hold */
+ /**
+ * @tasklet: softirq tasklet for bottom handler
+ */
+ struct tasklet_struct tasklet;
+
/**
* @default_priolist: priority list for I915_PRIORITY_NORMAL
*/
@@ -132,6 +137,9 @@ struct i915_sched_engine {
*/
bool no_priolist;
+ /* Back pointer to engine */
+ struct intel_engine_cs *engine;
+
/* Kick backend */
void (*kick_backend)(const struct i915_request *rq,
int prio);
--
2.28.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Intel-gfx] [PATCH 8/9] drm/i915: Move submission tasklet to i915_sched_engine
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter
The submission tasklet operates on i915_sched_engine, thus it is the
correct place for it.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine.h | 14 ---
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 12 +--
drivers/gpu/drm/i915/gt/intel_engine_types.h | 5 --
.../drm/i915/gt/intel_execlists_submission.c | 86 ++++++++++---------
drivers/gpu/drm/i915/gt/mock_engine.c | 1 +
drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++--
drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 2 +-
drivers/gpu/drm/i915/gt/selftest_lrc.c | 6 +-
drivers/gpu/drm/i915/gt/selftest_reset.c | 2 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 25 +++---
drivers/gpu/drm/i915/i915_scheduler.c | 1 +
drivers/gpu/drm/i915/i915_scheduler.h | 14 +++
drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++
13 files changed, 99 insertions(+), 93 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
index a8b2174b4395..988d9688ae4d 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine.h
@@ -123,20 +123,6 @@ execlists_active(const struct intel_engine_execlists *execlists)
return active;
}
-static inline void
-execlists_active_lock_bh(struct intel_engine_execlists *execlists)
-{
- local_bh_disable(); /* prevent local softirq and lock recursion */
- tasklet_lock(&execlists->tasklet);
-}
-
-static inline void
-execlists_active_unlock_bh(struct intel_engine_execlists *execlists)
-{
- tasklet_unlock(&execlists->tasklet);
- local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
-}
-
struct i915_request *
execlists_unwind_incomplete_requests(struct intel_engine_execlists *execlists);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index b480fcb1aad1..837d2132e31b 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -711,6 +711,7 @@ static int engine_setup_common(struct intel_engine_cs *engine)
err = -ENOMEM;
goto err_sched_engine;
}
+ engine->sched_engine->engine = engine;
err = intel_engine_init_cmd_parser(engine);
if (err)
@@ -935,7 +936,6 @@ int intel_engines_init(struct intel_gt *gt)
void intel_engine_cleanup_common(struct intel_engine_cs *engine)
{
GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
- tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
i915_sched_engine_put(engine->sched_engine);
intel_breadcrumbs_free(engine->breadcrumbs);
@@ -1221,7 +1221,7 @@ static bool ring_is_idle(struct intel_engine_cs *engine)
void __intel_engine_flush_submission(struct intel_engine_cs *engine, bool sync)
{
- struct tasklet_struct *t = &engine->execlists.tasklet;
+ struct tasklet_struct *t = &engine->sched_engine->tasklet;
if (!t->callback)
return;
@@ -1482,8 +1482,8 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
drm_printf(m, "\tExeclist tasklet queued? %s (%s), preempt? %s, timeslice? %s\n",
yesno(test_bit(TASKLET_STATE_SCHED,
- &engine->execlists.tasklet.state)),
- enableddisabled(!atomic_read(&engine->execlists.tasklet.count)),
+ &engine->sched_engine->tasklet.state)),
+ enableddisabled(!atomic_read(&engine->sched_engine->tasklet.count)),
repr_timer(&engine->execlists.preempt),
repr_timer(&engine->execlists.timer));
@@ -1507,7 +1507,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
idx, hws[idx * 2], hws[idx * 2 + 1]);
}
- execlists_active_lock_bh(execlists);
+ i915_sched_engine_active_lock_bh(engine->sched_engine);
rcu_read_lock();
for (port = execlists->active; (rq = *port); port++) {
char hdr[160];
@@ -1538,7 +1538,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
i915_request_show(m, rq, hdr, 0);
}
rcu_read_unlock();
- execlists_active_unlock_bh(execlists);
+ i915_sched_engine_active_unlock_bh(engine->sched_engine);
} else if (INTEL_GEN(dev_priv) > 6) {
drm_printf(m, "\tPP_DIR_BASE: 0x%08x\n",
ENGINE_READ(engine, RING_PP_DIR_BASE));
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index f1b14aff5118..6f2fd6f13ac4 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -138,11 +138,6 @@ struct st_preempt_hang {
* driver and the hardware state for execlist mode of submission.
*/
struct intel_engine_execlists {
- /**
- * @tasklet: softirq tasklet for bottom handler
- */
- struct tasklet_struct tasklet;
-
/**
* @timer: kick the current context if its timeslice expires
*/
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 7240c153be35..7a93aec3f6a9 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -570,7 +570,7 @@ static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
resubmit_virtual_request(rq, ve);
if (READ_ONCE(ve->request))
- tasklet_hi_schedule(&ve->base.execlists.tasklet);
+ tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
}
static void __execlists_schedule_out(struct i915_request * const rq,
@@ -739,9 +739,9 @@ trace_ports(const struct intel_engine_execlists *execlists,
}
static bool
-reset_in_progress(const struct intel_engine_execlists *execlists)
+reset_in_progress(const struct intel_engine_cs *engine)
{
- return unlikely(!__tasklet_is_enabled(&execlists->tasklet));
+ return unlikely(!__tasklet_is_enabled(&engine->sched_engine->tasklet));
}
static __maybe_unused noinline bool
@@ -757,7 +757,7 @@ assert_pending_valid(const struct intel_engine_execlists *execlists,
trace_ports(execlists, msg, execlists->pending);
/* We may be messing around with the lists during reset, lalala */
- if (reset_in_progress(execlists))
+ if (reset_in_progress(engine))
return true;
if (!execlists->pending[0]) {
@@ -1190,7 +1190,7 @@ static void start_timeslice(struct intel_engine_cs *engine)
* its timeslice, so recheck.
*/
if (!timer_pending(&el->timer))
- tasklet_hi_schedule(&el->tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
return;
}
@@ -1772,8 +1772,8 @@ process_csb(struct intel_engine_cs *engine, struct i915_request **inactive)
* access. Either we are inside the tasklet, or the tasklet is disabled
* and we assume that is only inside the reset paths and so serialised.
*/
- GEM_BUG_ON(!tasklet_is_locked(&execlists->tasklet) &&
- !reset_in_progress(execlists));
+ GEM_BUG_ON(!tasklet_is_locked(&engine->sched_engine->tasklet) &&
+ !reset_in_progress(engine));
/*
* Note that csb_write, csb_status may be either in HWSP or mmio.
@@ -2131,7 +2131,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
engine->sched_engine->queue_priority_hint = rq_prio(rq);
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
}
spin_unlock_irq(&engine->sched_engine->lock);
@@ -2322,13 +2322,13 @@ static void execlists_reset(struct intel_engine_cs *engine, const char *msg)
ENGINE_TRACE(engine, "reset for %s\n", msg);
/* Mark this tasklet as disabled to avoid waiting for it to complete */
- tasklet_disable_nosync(&engine->execlists.tasklet);
+ tasklet_disable_nosync(&engine->sched_engine->tasklet);
ring_set_paused(engine, 1); /* Freeze the current request in place */
execlists_capture(engine);
intel_engine_reset(engine, msg);
- tasklet_enable(&engine->execlists.tasklet);
+ tasklet_enable(&engine->sched_engine->tasklet);
clear_and_wake_up_bit(bit, lock);
}
@@ -2351,8 +2351,9 @@ static bool preempt_timeout(const struct intel_engine_cs *const engine)
*/
static void execlists_submission_tasklet(struct tasklet_struct *t)
{
- struct intel_engine_cs * const engine =
- from_tasklet(engine, t, execlists.tasklet);
+ struct i915_sched_engine *sched_engine =
+ from_tasklet(sched_engine, t, tasklet);
+ struct intel_engine_cs * const engine = sched_engine->engine;
struct i915_request *post[2 * EXECLIST_MAX_PORTS];
struct i915_request **inactive;
@@ -2427,13 +2428,16 @@ static void execlists_irq_handler(struct intel_engine_cs *engine, u16 iir)
intel_engine_signal_breadcrumbs(engine);
if (tasklet)
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
}
static void __execlists_kick(struct intel_engine_execlists *execlists)
{
+ struct intel_engine_cs *engine =
+ container_of(execlists, typeof(*engine), execlists);
+
/* Kick the tasklet for some interrupt coalescing and reset handling */
- tasklet_hi_schedule(&execlists->tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
}
#define execlists_kick(t, member) \
@@ -2808,10 +2812,8 @@ static int execlists_resume(struct intel_engine_cs *engine)
static void execlists_reset_prepare(struct intel_engine_cs *engine)
{
- struct intel_engine_execlists * const execlists = &engine->execlists;
-
ENGINE_TRACE(engine, "depth<-%d\n",
- atomic_read(&execlists->tasklet.count));
+ atomic_read(&engine->sched_engine->tasklet.count));
/*
* Prevent request submission to the hardware until we have
@@ -2822,8 +2824,8 @@ static void execlists_reset_prepare(struct intel_engine_cs *engine)
* Turning off the execlists->tasklet until the reset is over
* prevents the race.
*/
- __tasklet_disable_sync_once(&execlists->tasklet);
- GEM_BUG_ON(!reset_in_progress(execlists));
+ __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
+ GEM_BUG_ON(!reset_in_progress(engine));
/*
* We stop engines, otherwise we might get failed reset and a
@@ -2973,8 +2975,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
static void nop_submission_tasklet(struct tasklet_struct *t)
{
- struct intel_engine_cs * const engine =
- from_tasklet(engine, t, execlists.tasklet);
+ struct i915_sched_engine *sched_engine =
+ from_tasklet(sched_engine, t, tasklet);
+ struct intel_engine_cs * const engine = sched_engine->engine;
/* The driver is wedged; don't process any more events. */
WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
@@ -3061,8 +3064,8 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
sched_engine->queue_priority_hint = INT_MIN;
sched_engine->queue = RB_ROOT_CACHED;
- GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
- execlists->tasklet.callback = nop_submission_tasklet;
+ GEM_BUG_ON(__tasklet_is_enabled(&engine->sched_engine->tasklet));
+ engine->sched_engine->tasklet.callback = nop_submission_tasklet;
spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
rcu_read_unlock();
@@ -3082,14 +3085,14 @@ static void execlists_reset_finish(struct intel_engine_cs *engine)
* reset as the next level of recovery, and as a final resort we
* will declare the device wedged.
*/
- GEM_BUG_ON(!reset_in_progress(execlists));
+ GEM_BUG_ON(!reset_in_progress(engine));
/* And kick in case we missed a new request submission. */
- if (__tasklet_enable(&execlists->tasklet))
+ if (__tasklet_enable(&engine->sched_engine->tasklet))
__execlists_kick(execlists);
ENGINE_TRACE(engine, "depth->%d\n",
- atomic_read(&execlists->tasklet.count));
+ atomic_read(&engine->sched_engine->tasklet.count));
}
static void gen8_logical_ring_enable_irq(struct intel_engine_cs *engine)
@@ -3153,8 +3156,6 @@ static void kick_execlists(const struct i915_request *rq, int prio)
inflight->fence.context, inflight->fence.seqno,
inflight->sched.attr.priority);
- sched_engine->queue_priority_hint = prio;
-
/*
* Allow preemption of low -> normal -> high, but we do
* not allow low priority tasks to preempt other low priority
@@ -3163,7 +3164,7 @@ static void kick_execlists(const struct i915_request *rq, int prio)
* so kiss.
*/
if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&sched_engine->tasklet);
unlock:
rcu_read_unlock();
@@ -3174,7 +3175,7 @@ static void execlists_set_default_submission(struct intel_engine_cs *engine)
engine->submit_request = execlists_submit_request;
engine->sched_engine->schedule = i915_schedule;
engine->sched_engine->kick_backend = kick_execlists;
- engine->execlists.tasklet.callback = execlists_submission_tasklet;
+ engine->sched_engine->tasklet.callback = execlists_submission_tasklet;
}
static void execlists_shutdown(struct intel_engine_cs *engine)
@@ -3182,7 +3183,7 @@ static void execlists_shutdown(struct intel_engine_cs *engine)
/* Synchronise with residual timers and any softirq they raise */
del_timer_sync(&engine->execlists.timer);
del_timer_sync(&engine->execlists.preempt);
- tasklet_kill(&engine->execlists.tasklet);
+ tasklet_kill(&engine->sched_engine->tasklet);
}
static void execlists_release(struct intel_engine_cs *engine)
@@ -3298,7 +3299,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
struct intel_uncore *uncore = engine->uncore;
u32 base = engine->mmio_base;
- tasklet_setup(&engine->execlists.tasklet, execlists_submission_tasklet);
+ tasklet_setup(&engine->sched_engine->tasklet, execlists_submission_tasklet);
timer_setup(&engine->execlists.timer, execlists_timeslice, 0);
timer_setup(&engine->execlists.preempt, execlists_preempt, 0);
@@ -3380,7 +3381,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
* rbtrees as in the case it is running in parallel, it may reinsert
* the rb_node into a sibling.
*/
- tasklet_kill(&ve->base.execlists.tasklet);
+ tasklet_kill(&ve->base.sched_engine->tasklet);
/* Decouple ourselves from the siblings, no more access allowed. */
for (n = 0; n < ve->num_siblings; n++) {
@@ -3392,13 +3393,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
spin_lock_irq(&sibling->sched_engine->lock);
- /* Detachment is lazily performed in the execlists tasklet */
+ /* Detachment is lazily performed in the sched_engine->tasklet */
if (!RB_EMPTY_NODE(node))
rb_erase_cached(node, &sibling->execlists.virtual);
spin_unlock_irq(&sibling->sched_engine->lock);
}
- GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
+ GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.sched_engine->tasklet));
GEM_BUG_ON(!list_empty(virtual_queue(ve)));
lrc_fini(&ve->context);
@@ -3545,9 +3546,11 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
static void virtual_submission_tasklet(struct tasklet_struct *t)
{
+ struct i915_sched_engine *sched_engine =
+ from_tasklet(sched_engine, t, tasklet);
struct virtual_engine * const ve =
- from_tasklet(ve, t, base.execlists.tasklet);
- const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
+ (struct virtual_engine *)sched_engine->engine;
+ const int prio = READ_ONCE(sched_engine->queue_priority_hint);
intel_engine_mask_t mask;
unsigned int n;
@@ -3616,7 +3619,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
node->prio = prio;
if (first && prio > sibling->sched_engine->queue_priority_hint)
- tasklet_hi_schedule(&sibling->execlists.tasklet);
+ tasklet_hi_schedule(&sibling->sched_engine->tasklet);
unlock_engine:
spin_unlock_irq(&sibling->sched_engine->lock);
@@ -3657,7 +3660,7 @@ static void virtual_submit_request(struct i915_request *rq)
GEM_BUG_ON(!list_empty(virtual_queue(ve)));
list_move_tail(&rq->sched.link, virtual_queue(ve));
- tasklet_hi_schedule(&ve->base.execlists.tasklet);
+ tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
unlock:
spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
@@ -3751,6 +3754,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
err = -ENOMEM;
goto err_put;
}
+ ve->base.sched_engine->engine = &ve->base;
ve->base.cops = &virtual_context_ops;
ve->base.request_alloc = execlists_request_alloc;
@@ -3761,7 +3765,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
ve->base.bond_execute = virtual_bond_execute;
INIT_LIST_HEAD(virtual_queue(ve));
- tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
+ tasklet_setup(&ve->base.sched_engine->tasklet, virtual_submission_tasklet);
intel_context_init(&ve->context, &ve->base);
@@ -3789,7 +3793,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
* layering if we handle cloning of the requests and
* submitting a copy into each backend.
*/
- if (sibling->execlists.tasklet.callback !=
+ if (sibling->sched_engine->tasklet.callback !=
execlists_submission_tasklet) {
err = -ENODEV;
goto err_put;
diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
index a49fd3039f13..bd005c1b6fd5 100644
--- a/drivers/gpu/drm/i915/gt/mock_engine.c
+++ b/drivers/gpu/drm/i915/gt/mock_engine.c
@@ -349,6 +349,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
if (!engine->sched_engine)
return -ENOMEM;
+ engine->sched_engine->engine = engine;
intel_engine_init_execlists(engine);
intel_engine_init__pm(engine);
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index 5cc7648d1e5a..2554a2f343b4 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -43,7 +43,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
unsigned long timeout)
{
/* Ignore our own attempts to suppress excess tasklets */
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
timeout += jiffies;
do {
@@ -606,9 +606,9 @@ static int live_hold_reset(void *arg)
err = -EBUSY;
goto out;
}
- tasklet_disable(&engine->execlists.tasklet);
+ tasklet_disable(&engine->sched_engine->tasklet);
- engine->execlists.tasklet.callback(&engine->execlists.tasklet);
+ engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
i915_request_get(rq);
@@ -618,7 +618,7 @@ static int live_hold_reset(void *arg)
__intel_engine_reset_bh(engine, NULL);
GEM_BUG_ON(rq->fence.error != -EIO);
- tasklet_enable(&engine->execlists.tasklet);
+ tasklet_enable(&engine->sched_engine->tasklet);
clear_and_wake_up_bit(I915_RESET_ENGINE + id,
>->reset.flags);
local_bh_enable();
@@ -1183,7 +1183,7 @@ static int live_timeslice_rewind(void *arg)
while (i915_request_is_active(rq[A2])) { /* semaphore yield! */
/* Wait for the timeslice to kick in */
del_timer(&engine->execlists.timer);
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
intel_engine_flush_submission(engine);
}
/* -> ELSP[] = { { A:rq1 }, { B:rq1 } } */
@@ -4593,9 +4593,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
err = -EBUSY;
goto out_heartbeat;
}
- tasklet_disable(&engine->execlists.tasklet);
+ tasklet_disable(&engine->sched_engine->tasklet);
- engine->execlists.tasklet.callback(&engine->execlists.tasklet);
+ engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
/* Fake a preemption event; failed of course */
@@ -4612,7 +4612,7 @@ static int reset_virtual_engine(struct intel_gt *gt,
GEM_BUG_ON(rq->fence.error != -EIO);
/* Release our grasp on the engine, letting CS flow again */
- tasklet_enable(&engine->execlists.tasklet);
+ tasklet_enable(&engine->sched_engine->tasklet);
clear_and_wake_up_bit(I915_RESET_ENGINE + engine->id, >->reset.flags);
local_bh_enable();
diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
index e57dc900ae8d..cbcb800e2ca0 100644
--- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
+++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
@@ -1702,7 +1702,7 @@ static int __igt_atomic_reset_engine(struct intel_engine_cs *engine,
const struct igt_atomic_section *p,
const char *mode)
{
- struct tasklet_struct * const t = &engine->execlists.tasklet;
+ struct tasklet_struct * const t = &engine->sched_engine->tasklet;
int err;
GEM_TRACE("i915_reset_engine(%s:%s) under %s\n",
diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c
index d8f6623524e8..6a3a0d89dcd2 100644
--- a/drivers/gpu/drm/i915/gt/selftest_lrc.c
+++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c
@@ -49,7 +49,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
unsigned long timeout)
{
/* Ignore our own attempts to suppress excess tasklets */
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
timeout += jiffies;
do {
@@ -1613,12 +1613,12 @@ static void garbage_reset(struct intel_engine_cs *engine,
local_bh_disable();
if (!test_and_set_bit(bit, lock)) {
- tasklet_disable(&engine->execlists.tasklet);
+ tasklet_disable(&engine->sched_engine->tasklet);
if (!rq->fence.error)
__intel_engine_reset_bh(engine, NULL);
- tasklet_enable(&engine->execlists.tasklet);
+ tasklet_enable(&engine->sched_engine->tasklet);
clear_and_wake_up_bit(bit, lock);
}
local_bh_enable();
diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c
index 8784257ec808..7a50c9f4071b 100644
--- a/drivers/gpu/drm/i915/gt/selftest_reset.c
+++ b/drivers/gpu/drm/i915/gt/selftest_reset.c
@@ -321,7 +321,7 @@ static int igt_atomic_engine_reset(void *arg)
goto out_unlock;
for_each_engine(engine, gt, id) {
- struct tasklet_struct *t = &engine->execlists.tasklet;
+ struct tasklet_struct *t = &engine->sched_engine->tasklet;
if (t->func)
tasklet_disable(t);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 7ed21670ef14..95c6f6af4047 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -241,8 +241,9 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
static void guc_submission_tasklet(struct tasklet_struct *t)
{
- struct intel_engine_cs * const engine =
- from_tasklet(engine, t, execlists.tasklet);
+ struct i915_sched_engine *sched_engine =
+ from_tasklet(sched_engine, t, tasklet);
+ struct intel_engine_cs * const engine = sched_engine->engine;
struct intel_engine_execlists * const execlists = &engine->execlists;
struct i915_request **port, *rq;
unsigned long flags;
@@ -272,14 +273,12 @@ static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
{
if (iir & GT_RENDER_USER_INTERRUPT) {
intel_engine_signal_breadcrumbs(engine);
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
}
}
static void guc_reset_prepare(struct intel_engine_cs *engine)
{
- struct intel_engine_execlists * const execlists = &engine->execlists;
-
ENGINE_TRACE(engine, "\n");
/*
@@ -291,7 +290,7 @@ static void guc_reset_prepare(struct intel_engine_cs *engine)
* Turning off the execlists->tasklet until the reset is over
* prevents the race.
*/
- __tasklet_disable_sync_once(&execlists->tasklet);
+ __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
}
static void guc_reset_state(struct intel_context *ce,
@@ -395,14 +394,12 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
static void guc_reset_finish(struct intel_engine_cs *engine)
{
- struct intel_engine_execlists * const execlists = &engine->execlists;
-
- if (__tasklet_enable(&execlists->tasklet))
+ if (__tasklet_enable(&engine->sched_engine->tasklet))
/* And kick in case we missed a new request submission. */
- tasklet_hi_schedule(&execlists->tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
ENGINE_TRACE(engine, "depth->%d\n",
- atomic_read(&execlists->tasklet.count));
+ atomic_read(&engine->sched_engine->tasklet.count));
}
/*
@@ -546,7 +543,7 @@ static void guc_submit_request(struct i915_request *rq)
GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
GEM_BUG_ON(list_empty(&rq->sched.link));
- tasklet_hi_schedule(&engine->execlists.tasklet);
+ tasklet_hi_schedule(&engine->sched_engine->tasklet);
spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
}
@@ -626,7 +623,7 @@ static void guc_release(struct intel_engine_cs *engine)
{
engine->sanitize = NULL; /* no longer in control, nothing to sanitize */
- tasklet_kill(&engine->execlists.tasklet);
+ tasklet_kill(&engine->sched_engine->tasklet);
intel_engine_cleanup_common(engine);
lrc_fini_wa_ctx(engine);
@@ -705,7 +702,7 @@ int intel_guc_submission_setup(struct intel_engine_cs *engine)
*/
GEM_BUG_ON(INTEL_GEN(i915) < 11);
- tasklet_setup(&engine->execlists.tasklet, guc_submission_tasklet);
+ tasklet_setup(&engine->sched_engine->tasklet, guc_submission_tasklet);
guc_default_vfuncs(engine);
guc_default_irqs(engine);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 3d36e4447b5d..6f082579ee9e 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -436,6 +436,7 @@ void i915_sched_engine_free(struct kref *kref)
struct i915_sched_engine *sched_engine =
container_of(kref, typeof(*sched_engine), ref);
+ tasklet_kill(&sched_engine->tasklet); /* flush the callback */
kfree(sched_engine);
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 0014745bda30..650ab8e0db9f 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -79,6 +79,20 @@ i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
sched_engine->no_priolist = false;
}
+static inline void
+i915_sched_engine_active_lock_bh(struct i915_sched_engine *sched_engine)
+{
+ local_bh_disable(); /* prevent local softirq and lock recursion */
+ tasklet_lock(&sched_engine->tasklet);
+}
+
+static inline void
+i915_sched_engine_active_unlock_bh(struct i915_sched_engine *sched_engine)
+{
+ tasklet_unlock(&sched_engine->tasklet);
+ local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
+}
+
void i915_request_show_with_schedule(struct drm_printer *m,
const struct i915_request *rq,
const char *prefix,
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 8b3af536e6cf..9d79514450de 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -103,6 +103,11 @@ struct i915_sched_engine {
struct list_head requests;
struct list_head hold; /* ready requests, but on hold */
+ /**
+ * @tasklet: softirq tasklet for bottom handler
+ */
+ struct tasklet_struct tasklet;
+
/**
* @default_priolist: priority list for I915_PRIORITY_NORMAL
*/
@@ -132,6 +137,9 @@ struct i915_sched_engine {
*/
bool no_priolist;
+ /* Back pointer to engine */
+ struct intel_engine_cs *engine;
+
/* Kick backend */
void (*kick_backend)(const struct i915_request *rq,
int prio);
--
2.28.0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 9/9] drm/i915/doc: Add kernel doc for i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-03 21:27 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter, daniele.ceraolospurio
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
Documentation/gpu/i915.rst | 6 ++++
drivers/gpu/drm/i915/i915_scheduler_types.h | 37 ++++++++++++++++++---
2 files changed, 38 insertions(+), 5 deletions(-)
diff --git a/Documentation/gpu/i915.rst b/Documentation/gpu/i915.rst
index 42ce0196930a..8f4f5471a05b 100644
--- a/Documentation/gpu/i915.rst
+++ b/Documentation/gpu/i915.rst
@@ -425,6 +425,12 @@ User Batchbuffer Execution
.. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
:doc: User command execution
+Scheduling
+----------
+.. kernel-doc:: drivers/gpu/drm/i915/i915_scheduler_types.h
+ :functions: i915_sched_engine
+
+
Logical Rings, Logical Ring Contexts and Execlists
--------------------------------------------------
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 9d79514450de..e3da7517853f 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -91,7 +91,21 @@ struct i915_dependency {
&(rq__)->sched.signalers_list, \
signal_link)
+/**
+ * sturct i915_sched_engine - scheduler engine
+ *
+ * A schedule engine represents a submission queue with different priority
+ * bands. It contains all the common state (relative to the backend) to queue,
+ * track, and submit a request.
+ *
+ * This object at the moment is quite i915 specific but will transition into a
+ * container for the drm_gpu_scheduler plus a few other variables once the i915
+ * is integrated with the DRM scheduler.
+ */
struct i915_sched_engine {
+ /**
+ * @ref: reference count of schedule engine object
+ */
struct kref ref;
/**
@@ -100,11 +114,18 @@ struct i915_sched_engine {
*/
spinlock_t lock;
+ /**
+ * @requests: list of requests inflight on this schedule engine
+ */
struct list_head requests;
- struct list_head hold; /* ready requests, but on hold */
/**
- * @tasklet: softirq tasklet for bottom handler
+ * @hold: list of requests on hold.
+ */
+ struct list_head hold;
+
+ /**
+ * @tasklet: softirq tasklet for submission
*/
struct tasklet_struct tasklet;
@@ -137,14 +158,20 @@ struct i915_sched_engine {
*/
bool no_priolist;
- /* Back pointer to engine */
+ /**
+ * @engine: back pointer to engine
+ */
struct intel_engine_cs *engine;
- /* Kick backend */
+ /**
+ * @kick_backed: kick back after a request's priority has changed
+ */
void (*kick_backend)(const struct i915_request *rq,
int prio);
- /*
+ /**
+ * @schedule: schedule function to adjust priority of request
+ *
* Call when the priority on a request has changed and it and its
* dependencies may need rescheduling. Note the request itself may
* not be ready to run!
--
2.28.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Intel-gfx] [PATCH 9/9] drm/i915/doc: Add kernel doc for i915_sched_engine
@ 2021-06-03 21:27 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-03 21:27 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: daniel.vetter
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
Documentation/gpu/i915.rst | 6 ++++
drivers/gpu/drm/i915/i915_scheduler_types.h | 37 ++++++++++++++++++---
2 files changed, 38 insertions(+), 5 deletions(-)
diff --git a/Documentation/gpu/i915.rst b/Documentation/gpu/i915.rst
index 42ce0196930a..8f4f5471a05b 100644
--- a/Documentation/gpu/i915.rst
+++ b/Documentation/gpu/i915.rst
@@ -425,6 +425,12 @@ User Batchbuffer Execution
.. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
:doc: User command execution
+Scheduling
+----------
+.. kernel-doc:: drivers/gpu/drm/i915/i915_scheduler_types.h
+ :functions: i915_sched_engine
+
+
Logical Rings, Logical Ring Contexts and Execlists
--------------------------------------------------
diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
index 9d79514450de..e3da7517853f 100644
--- a/drivers/gpu/drm/i915/i915_scheduler_types.h
+++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
@@ -91,7 +91,21 @@ struct i915_dependency {
&(rq__)->sched.signalers_list, \
signal_link)
+/**
+ * sturct i915_sched_engine - scheduler engine
+ *
+ * A schedule engine represents a submission queue with different priority
+ * bands. It contains all the common state (relative to the backend) to queue,
+ * track, and submit a request.
+ *
+ * This object at the moment is quite i915 specific but will transition into a
+ * container for the drm_gpu_scheduler plus a few other variables once the i915
+ * is integrated with the DRM scheduler.
+ */
struct i915_sched_engine {
+ /**
+ * @ref: reference count of schedule engine object
+ */
struct kref ref;
/**
@@ -100,11 +114,18 @@ struct i915_sched_engine {
*/
spinlock_t lock;
+ /**
+ * @requests: list of requests inflight on this schedule engine
+ */
struct list_head requests;
- struct list_head hold; /* ready requests, but on hold */
/**
- * @tasklet: softirq tasklet for bottom handler
+ * @hold: list of requests on hold.
+ */
+ struct list_head hold;
+
+ /**
+ * @tasklet: softirq tasklet for submission
*/
struct tasklet_struct tasklet;
@@ -137,14 +158,20 @@ struct i915_sched_engine {
*/
bool no_priolist;
- /* Back pointer to engine */
+ /**
+ * @engine: back pointer to engine
+ */
struct intel_engine_cs *engine;
- /* Kick backend */
+ /**
+ * @kick_backed: kick back after a request's priority has changed
+ */
void (*kick_backend)(const struct i915_request *rq,
int prio);
- /*
+ /**
+ * @schedule: schedule function to adjust priority of request
+ *
* Call when the priority on a request has changed and it and its
* dependencies may need rescheduling. Note the request itself may
* not be ready to run!
--
2.28.0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Introduce i915_sched_engine object (rev2)
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
` (9 preceding siblings ...)
(?)
@ 2021-06-03 23:32 ` Patchwork
-1 siblings, 0 replies; 64+ messages in thread
From: Patchwork @ 2021-06-03 23:32 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-gfx
== Series Details ==
Series: Introduce i915_sched_engine object (rev2)
URL : https://patchwork.freedesktop.org/series/90630/
State : warning
== Summary ==
$ dim checkpatch origin/drm-tip
8225af1a908b drm/i915: Move priolist to new i915_sched_engine object
-:17: WARNING:REPEATED_WORD: Possible repeated word: 'the'
#17:
This patch starts the aforementioned transition by moving the the
total: 0 errors, 1 warnings, 0 checks, 682 lines checked
ce8653e69398 drm/i915: Add i915_sched_engine_is_empty function
148a2142a7c3 drm/i915: Add i915_sched_engine_reset_on_empty function
e7485728abe3 drm/i915: Move active tracking to i915_sched_engine
f6110f5321ac drm/i915: Move engine->schedule to i915_sched_engine
6e59cded3265 drm/i915: Add kick_backend function to i915_sched_engine
b0932841cafb drm/i915: Update i915_scheduler to operate on i915_sched_engine
ec126054a0f5 drm/i915: Move submission tasklet to i915_sched_engine
378f81235475 drm/i915/doc: Add kernel doc for i915_sched_engine
-:7: WARNING:COMMIT_MESSAGE: Missing commit description - Add an appropriate one
total: 0 errors, 1 warnings, 0 checks, 76 lines checked
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* [Intel-gfx] ✗ Fi.CI.DOCS: warning for Introduce i915_sched_engine object (rev2)
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
` (10 preceding siblings ...)
(?)
@ 2021-06-03 23:37 ` Patchwork
-1 siblings, 0 replies; 64+ messages in thread
From: Patchwork @ 2021-06-03 23:37 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-gfx
== Series Details ==
Series: Introduce i915_sched_engine object (rev2)
URL : https://patchwork.freedesktop.org/series/90630/
State : warning
== Summary ==
$ make htmldocs 2>&1 > /dev/null | grep i915
./drivers/gpu/drm/i915/i915_scheduler_types.h:105: warning: cannot understand function prototype: 'struct i915_sched_engine '
./drivers/gpu/drm/i915/i915_scheduler_types.h:1: warning: 'i915_sched_engine' not found
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* [Intel-gfx] ✓ Fi.CI.BAT: success for Introduce i915_sched_engine object (rev2)
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
` (11 preceding siblings ...)
(?)
@ 2021-06-04 0:03 ` Patchwork
-1 siblings, 0 replies; 64+ messages in thread
From: Patchwork @ 2021-06-04 0:03 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-gfx
[-- Attachment #1.1: Type: text/plain, Size: 5191 bytes --]
== Series Details ==
Series: Introduce i915_sched_engine object (rev2)
URL : https://patchwork.freedesktop.org/series/90630/
State : success
== Summary ==
CI Bug Log - changes from CI_DRM_10168 -> Patchwork_20279
====================================================
Summary
-------
**SUCCESS**
No regressions found.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/index.html
Known issues
------------
Here are the changes found in Patchwork_20279 that come from known issues:
### IGT changes ###
#### Possible fixes ####
* igt@i915_selftest@live@gt_heartbeat:
- fi-glk-dsi: [DMESG-FAIL][1] ([i915#2291] / [i915#541]) -> [PASS][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/fi-glk-dsi/igt@i915_selftest@live@gt_heartbeat.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/fi-glk-dsi/igt@i915_selftest@live@gt_heartbeat.html
#### Warnings ####
* igt@runner@aborted:
- fi-bdw-5557u: [FAIL][3] ([i915#3462]) -> [FAIL][4] ([i915#2426] / [i915#3462])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/fi-bdw-5557u/igt@runner@aborted.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/fi-bdw-5557u/igt@runner@aborted.html
- fi-cml-u2: [FAIL][5] ([i915#3363] / [i915#3462]) -> [FAIL][6] ([i915#2082] / [i915#2426] / [i915#3363] / [i915#3462])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/fi-cml-u2/igt@runner@aborted.html
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/fi-cml-u2/igt@runner@aborted.html
- fi-bxt-dsi: [FAIL][7] ([i915#3363]) -> [FAIL][8] ([i915#2426] / [i915#3363])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/fi-bxt-dsi/igt@runner@aborted.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/fi-bxt-dsi/igt@runner@aborted.html
- fi-cml-s: [FAIL][9] ([i915#2082] / [i915#2426] / [i915#3363] / [i915#3462]) -> [FAIL][10] ([i915#3363] / [i915#3462])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/fi-cml-s/igt@runner@aborted.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/fi-cml-s/igt@runner@aborted.html
- fi-cfl-guc: [FAIL][11] ([i915#3363]) -> [FAIL][12] ([i915#2426] / [i915#3363])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/fi-cfl-guc/igt@runner@aborted.html
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/fi-cfl-guc/igt@runner@aborted.html
- fi-kbl-7567u: [FAIL][13] ([i915#1436] / [i915#3363]) -> [FAIL][14] ([i915#1436] / [i915#2426] / [i915#3363])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/fi-kbl-7567u/igt@runner@aborted.html
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/fi-kbl-7567u/igt@runner@aborted.html
- fi-skl-guc: [FAIL][15] ([i915#1436] / [i915#2426] / [i915#3363]) -> [FAIL][16] ([i915#1436] / [i915#3363])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/fi-skl-guc/igt@runner@aborted.html
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/fi-skl-guc/igt@runner@aborted.html
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[i915#1436]: https://gitlab.freedesktop.org/drm/intel/issues/1436
[i915#2082]: https://gitlab.freedesktop.org/drm/intel/issues/2082
[i915#2291]: https://gitlab.freedesktop.org/drm/intel/issues/2291
[i915#2426]: https://gitlab.freedesktop.org/drm/intel/issues/2426
[i915#2932]: https://gitlab.freedesktop.org/drm/intel/issues/2932
[i915#2966]: https://gitlab.freedesktop.org/drm/intel/issues/2966
[i915#3363]: https://gitlab.freedesktop.org/drm/intel/issues/3363
[i915#3462]: https://gitlab.freedesktop.org/drm/intel/issues/3462
[i915#541]: https://gitlab.freedesktop.org/drm/intel/issues/541
Participating hosts (45 -> 41)
------------------------------
Missing (4): fi-ilk-m540 fi-bsw-cyan fi-bdw-samus fi-hsw-4200u
Build changes
-------------
* Linux: CI_DRM_10168 -> Patchwork_20279
CI-20190529: 20190529
CI_DRM_10168: ad6cb7b043355cf535e0027ff90c4a6aacb790d1 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6098: 1fbc1e7d602f96a7f4e2b95057eef994656b8e74 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
Patchwork_20279: 378f81235475bf26dcbe8c9daaafd01aea52fb13 @ git://anongit.freedesktop.org/gfx-ci/linux
== Linux commits ==
378f81235475 drm/i915/doc: Add kernel doc for i915_sched_engine
ec126054a0f5 drm/i915: Move submission tasklet to i915_sched_engine
b0932841cafb drm/i915: Update i915_scheduler to operate on i915_sched_engine
6e59cded3265 drm/i915: Add kick_backend function to i915_sched_engine
f6110f5321ac drm/i915: Move engine->schedule to i915_sched_engine
e7485728abe3 drm/i915: Move active tracking to i915_sched_engine
148a2142a7c3 drm/i915: Add i915_sched_engine_reset_on_empty function
ce8653e69398 drm/i915: Add i915_sched_engine_is_empty function
8225af1a908b drm/i915: Move priolist to new i915_sched_engine object
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/index.html
[-- Attachment #1.2: Type: text/html, Size: 7516 bytes --]
[-- Attachment #2: Type: text/plain, Size: 160 bytes --]
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* [Intel-gfx] ✗ Fi.CI.IGT: failure for Introduce i915_sched_engine object (rev2)
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
` (12 preceding siblings ...)
(?)
@ 2021-06-04 2:03 ` Patchwork
-1 siblings, 0 replies; 64+ messages in thread
From: Patchwork @ 2021-06-04 2:03 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-gfx
[-- Attachment #1.1: Type: text/plain, Size: 29917 bytes --]
== Series Details ==
Series: Introduce i915_sched_engine object (rev2)
URL : https://patchwork.freedesktop.org/series/90630/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_10168_full -> Patchwork_20279_full
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_20279_full absolutely need to be
verified manually.
If you think the reported changes have nothing to do with the changes
introduced in Patchwork_20279_full, please notify your bug team to allow them
to document this new failure mode, which will reduce false positives in CI.
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_20279_full:
### IGT changes ###
#### Possible regressions ####
* igt@kms_draw_crc@draw-method-xrgb8888-blt-ytiled:
- shard-skl: [PASS][1] -> [FAIL][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl6/igt@kms_draw_crc@draw-method-xrgb8888-blt-ytiled.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl3/igt@kms_draw_crc@draw-method-xrgb8888-blt-ytiled.html
Known issues
------------
Here are the changes found in Patchwork_20279_full that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_create@create-clear:
- shard-skl: [PASS][3] -> [FAIL][4] ([i915#3160])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl6/igt@gem_create@create-clear.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl3/igt@gem_create@create-clear.html
* igt@gem_ctx_isolation@preservation-s3@vcs0:
- shard-kbl: [PASS][5] -> [DMESG-WARN][6] ([i915#180]) +1 similar issue
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl1/igt@gem_ctx_isolation@preservation-s3@vcs0.html
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl1/igt@gem_ctx_isolation@preservation-s3@vcs0.html
* igt@gem_ctx_persistence@legacy-engines-mixed:
- shard-snb: NOTRUN -> [SKIP][7] ([fdo#109271] / [i915#1099]) +3 similar issues
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-snb5/igt@gem_ctx_persistence@legacy-engines-mixed.html
* igt@gem_ctx_shared@q-in-order:
- shard-snb: NOTRUN -> [SKIP][8] ([fdo#109271]) +315 similar issues
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-snb6/igt@gem_ctx_shared@q-in-order.html
* igt@gem_exec_fair@basic-deadline:
- shard-glk: [PASS][9] -> [FAIL][10] ([i915#2846])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-glk9/igt@gem_exec_fair@basic-deadline.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-glk1/igt@gem_exec_fair@basic-deadline.html
* igt@gem_exec_fair@basic-none-share@rcs0:
- shard-iclb: [PASS][11] -> [FAIL][12] ([i915#2842])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-iclb7/igt@gem_exec_fair@basic-none-share@rcs0.html
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-iclb8/igt@gem_exec_fair@basic-none-share@rcs0.html
* igt@gem_exec_fair@basic-none@rcs0:
- shard-glk: [PASS][13] -> [FAIL][14] ([i915#2842])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-glk7/igt@gem_exec_fair@basic-none@rcs0.html
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-glk1/igt@gem_exec_fair@basic-none@rcs0.html
* igt@gem_exec_fair@basic-pace@rcs0:
- shard-kbl: [PASS][15] -> [FAIL][16] ([i915#2842])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl3/igt@gem_exec_fair@basic-pace@rcs0.html
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl1/igt@gem_exec_fair@basic-pace@rcs0.html
* igt@gem_exec_fair@basic-throttle@rcs0:
- shard-iclb: [PASS][17] -> [FAIL][18] ([i915#2849])
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-iclb3/igt@gem_exec_fair@basic-throttle@rcs0.html
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-iclb3/igt@gem_exec_fair@basic-throttle@rcs0.html
* igt@gem_exec_reloc@basic-wide-active@bcs0:
- shard-apl: NOTRUN -> [FAIL][19] ([i915#2389]) +3 similar issues
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl7/igt@gem_exec_reloc@basic-wide-active@bcs0.html
* igt@gem_mmap_gtt@cpuset-big-copy-odd:
- shard-iclb: [PASS][20] -> [FAIL][21] ([i915#307]) +1 similar issue
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-iclb2/igt@gem_mmap_gtt@cpuset-big-copy-odd.html
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-iclb1/igt@gem_mmap_gtt@cpuset-big-copy-odd.html
* igt@gem_mmap_gtt@cpuset-medium-copy-odd:
- shard-iclb: [PASS][22] -> [FAIL][23] ([i915#2428])
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-iclb7/igt@gem_mmap_gtt@cpuset-medium-copy-odd.html
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-iclb8/igt@gem_mmap_gtt@cpuset-medium-copy-odd.html
* igt@gem_userptr_blits@vma-merge:
- shard-snb: NOTRUN -> [FAIL][24] ([i915#2724])
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-snb6/igt@gem_userptr_blits@vma-merge.html
- shard-apl: NOTRUN -> [FAIL][25] ([i915#3318])
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl8/igt@gem_userptr_blits@vma-merge.html
* igt@gem_workarounds@suspend-resume:
- shard-apl: [PASS][26] -> [DMESG-WARN][27] ([i915#180])
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-apl1/igt@gem_workarounds@suspend-resume.html
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl6/igt@gem_workarounds@suspend-resume.html
* igt@gen9_exec_parse@allowed-single:
- shard-skl: [PASS][28] -> [DMESG-WARN][29] ([i915#1436] / [i915#716])
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl6/igt@gen9_exec_parse@allowed-single.html
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl3/igt@gen9_exec_parse@allowed-single.html
* igt@i915_pm_dc@dc6-psr:
- shard-skl: NOTRUN -> [FAIL][30] ([i915#454])
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl7/igt@i915_pm_dc@dc6-psr.html
* igt@i915_selftest@live@hangcheck:
- shard-snb: NOTRUN -> [INCOMPLETE][31] ([i915#2782])
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-snb5/igt@i915_selftest@live@hangcheck.html
* igt@kms_big_joiner@invalid-modeset:
- shard-apl: NOTRUN -> [SKIP][32] ([fdo#109271] / [i915#2705])
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl7/igt@kms_big_joiner@invalid-modeset.html
* igt@kms_chamelium@hdmi-mode-timings:
- shard-snb: NOTRUN -> [SKIP][33] ([fdo#109271] / [fdo#111827]) +20 similar issues
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-snb6/igt@kms_chamelium@hdmi-mode-timings.html
* igt@kms_chamelium@vga-edid-read:
- shard-apl: NOTRUN -> [SKIP][34] ([fdo#109271] / [fdo#111827]) +18 similar issues
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl8/igt@kms_chamelium@vga-edid-read.html
* igt@kms_color_chamelium@pipe-d-ctm-0-25:
- shard-skl: NOTRUN -> [SKIP][35] ([fdo#109271] / [fdo#111827]) +3 similar issues
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl7/igt@kms_color_chamelium@pipe-d-ctm-0-25.html
* igt@kms_cursor_legacy@flip-vs-cursor-atomic:
- shard-skl: [PASS][36] -> [FAIL][37] ([i915#2346])
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl3/igt@kms_cursor_legacy@flip-vs-cursor-atomic.html
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl2/igt@kms_cursor_legacy@flip-vs-cursor-atomic.html
* igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size:
- shard-skl: [PASS][38] -> [FAIL][39] ([i915#2346] / [i915#533])
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl9/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl5/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
* igt@kms_fbcon_fbt@fbc-suspend:
- shard-kbl: [PASS][40] -> [INCOMPLETE][41] ([i915#155] / [i915#180] / [i915#636])
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl1/igt@kms_fbcon_fbt@fbc-suspend.html
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl4/igt@kms_fbcon_fbt@fbc-suspend.html
* igt@kms_flip@flip-vs-rmfb-interruptible@a-edp1:
- shard-skl: [PASS][42] -> [DMESG-WARN][43] ([i915#1982])
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl1/igt@kms_flip@flip-vs-rmfb-interruptible@a-edp1.html
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl4/igt@kms_flip@flip-vs-rmfb-interruptible@a-edp1.html
* igt@kms_flip@flip-vs-suspend-interruptible@c-dp1:
- shard-apl: NOTRUN -> [DMESG-WARN][44] ([i915#180])
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl8/igt@kms_flip@flip-vs-suspend-interruptible@c-dp1.html
* igt@kms_flip@flip-vs-suspend@c-dp1:
- shard-kbl: NOTRUN -> [DMESG-WARN][45] ([i915#180]) +2 similar issues
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl2/igt@kms_flip@flip-vs-suspend@c-dp1.html
* igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytilegen12rcccs:
- shard-skl: NOTRUN -> [SKIP][46] ([fdo#109271] / [i915#2672])
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl7/igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytilegen12rcccs.html
* igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-pri-indfb-draw-blt:
- shard-skl: NOTRUN -> [SKIP][47] ([fdo#109271]) +32 similar issues
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl5/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-pri-indfb-draw-blt.html
* igt@kms_frontbuffer_tracking@psr-rgb101010-draw-mmap-wc:
- shard-kbl: NOTRUN -> [SKIP][48] ([fdo#109271]) +4 similar issues
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl1/igt@kms_frontbuffer_tracking@psr-rgb101010-draw-mmap-wc.html
* igt@kms_hdr@bpc-switch:
- shard-skl: [PASS][49] -> [FAIL][50] ([i915#1188])
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl1/igt@kms_hdr@bpc-switch.html
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl5/igt@kms_hdr@bpc-switch.html
* igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-d:
- shard-apl: NOTRUN -> [SKIP][51] ([fdo#109271] / [i915#533]) +1 similar issue
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl7/igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-d.html
* igt@kms_pipe_crc_basic@read-crc-pipe-d:
- shard-skl: NOTRUN -> [SKIP][52] ([fdo#109271] / [i915#533])
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl7/igt@kms_pipe_crc_basic@read-crc-pipe-d.html
* igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb:
- shard-skl: NOTRUN -> [FAIL][53] ([fdo#108145] / [i915#265])
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl7/igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb.html
* igt@kms_plane_alpha_blend@pipe-a-coverage-7efc:
- shard-skl: [PASS][54] -> [FAIL][55] ([fdo#108145] / [i915#265])
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl6/igt@kms_plane_alpha_blend@pipe-a-coverage-7efc.html
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl3/igt@kms_plane_alpha_blend@pipe-a-coverage-7efc.html
* igt@kms_plane_alpha_blend@pipe-c-alpha-basic:
- shard-apl: NOTRUN -> [FAIL][56] ([fdo#108145] / [i915#265]) +2 similar issues
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl8/igt@kms_plane_alpha_blend@pipe-c-alpha-basic.html
* igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-4:
- shard-apl: NOTRUN -> [SKIP][57] ([fdo#109271] / [i915#658]) +2 similar issues
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl2/igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-4.html
* igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-4:
- shard-skl: NOTRUN -> [SKIP][58] ([fdo#109271] / [i915#658])
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl7/igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-4.html
* igt@kms_psr@psr2_no_drrs:
- shard-iclb: [PASS][59] -> [SKIP][60] ([fdo#109441]) +2 similar issues
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-iclb2/igt@kms_psr@psr2_no_drrs.html
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-iclb7/igt@kms_psr@psr2_no_drrs.html
* igt@kms_vblank@pipe-d-wait-forked-hang:
- shard-apl: NOTRUN -> [SKIP][61] ([fdo#109271]) +198 similar issues
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl7/igt@kms_vblank@pipe-d-wait-forked-hang.html
* igt@kms_writeback@writeback-pixel-formats:
- shard-apl: NOTRUN -> [SKIP][62] ([fdo#109271] / [i915#2437])
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl8/igt@kms_writeback@writeback-pixel-formats.html
* igt@sysfs_clients@create:
- shard-apl: NOTRUN -> [SKIP][63] ([fdo#109271] / [i915#2994]) +2 similar issues
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl8/igt@sysfs_clients@create.html
* igt@sysfs_clients@fair-1:
- shard-skl: NOTRUN -> [SKIP][64] ([fdo#109271] / [i915#2994])
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl7/igt@sysfs_clients@fair-1.html
#### Possible fixes ####
* igt@gem_ctx_isolation@preservation-s3@bcs0:
- shard-kbl: [DMESG-WARN][65] ([i915#180]) -> [PASS][66] +2 similar issues
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl1/igt@gem_ctx_isolation@preservation-s3@bcs0.html
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl1/igt@gem_ctx_isolation@preservation-s3@bcs0.html
* igt@gem_ctx_isolation@preservation-s3@vcs0:
- shard-skl: [INCOMPLETE][67] ([i915#198]) -> [PASS][68]
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl7/igt@gem_ctx_isolation@preservation-s3@vcs0.html
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl7/igt@gem_ctx_isolation@preservation-s3@vcs0.html
* igt@gem_ctx_persistence@many-contexts:
- shard-tglb: [FAIL][69] ([i915#2410]) -> [PASS][70]
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-tglb2/igt@gem_ctx_persistence@many-contexts.html
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-tglb7/igt@gem_ctx_persistence@many-contexts.html
* igt@gem_exec_fair@basic-none@vcs0:
- shard-kbl: [FAIL][71] ([i915#2842]) -> [PASS][72] +2 similar issues
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl2/igt@gem_exec_fair@basic-none@vcs0.html
[72]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl2/igt@gem_exec_fair@basic-none@vcs0.html
* igt@gem_exec_fair@basic-pace-share@rcs0:
- shard-tglb: [FAIL][73] ([i915#2842]) -> [PASS][74]
[73]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-tglb8/igt@gem_exec_fair@basic-pace-share@rcs0.html
[74]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-tglb3/igt@gem_exec_fair@basic-pace-share@rcs0.html
* igt@gem_exec_fair@basic-throttle@rcs0:
- shard-glk: [FAIL][75] ([i915#2842]) -> [PASS][76]
[75]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-glk2/igt@gem_exec_fair@basic-throttle@rcs0.html
[76]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-glk6/igt@gem_exec_fair@basic-throttle@rcs0.html
* igt@i915_pm_dc@dc6-psr:
- shard-iclb: [FAIL][77] ([i915#454]) -> [PASS][78]
[77]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-iclb4/igt@i915_pm_dc@dc6-psr.html
[78]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-iclb4/igt@i915_pm_dc@dc6-psr.html
* igt@kms_async_flips@alternate-sync-async-flip:
- shard-skl: [FAIL][79] ([i915#2521]) -> [PASS][80]
[79]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl9/igt@kms_async_flips@alternate-sync-async-flip.html
[80]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl9/igt@kms_async_flips@alternate-sync-async-flip.html
* igt@kms_cursor_crc@pipe-b-cursor-128x128-onscreen:
- shard-skl: [FAIL][81] ([i915#3444]) -> [PASS][82]
[81]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl2/igt@kms_cursor_crc@pipe-b-cursor-128x128-onscreen.html
[82]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl8/igt@kms_cursor_crc@pipe-b-cursor-128x128-onscreen.html
* igt@kms_cursor_legacy@basic-flip-before-cursor-varying-size:
- shard-skl: [FAIL][83] ([i915#2346]) -> [PASS][84]
[83]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl3/igt@kms_cursor_legacy@basic-flip-before-cursor-varying-size.html
[84]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl2/igt@kms_cursor_legacy@basic-flip-before-cursor-varying-size.html
* igt@kms_flip@2x-plain-flip-fb-recreate-interruptible@bc-hdmi-a1-hdmi-a2:
- shard-glk: [FAIL][85] ([i915#2122]) -> [PASS][86] +1 similar issue
[85]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-glk9/igt@kms_flip@2x-plain-flip-fb-recreate-interruptible@bc-hdmi-a1-hdmi-a2.html
[86]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-glk1/igt@kms_flip@2x-plain-flip-fb-recreate-interruptible@bc-hdmi-a1-hdmi-a2.html
* igt@kms_hdr@bpc-switch-dpms:
- shard-skl: [FAIL][87] ([i915#1188]) -> [PASS][88]
[87]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl2/igt@kms_hdr@bpc-switch-dpms.html
[88]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl9/igt@kms_hdr@bpc-switch-dpms.html
* igt@kms_psr@psr2_primary_mmap_gtt:
- shard-iclb: [SKIP][89] ([fdo#109441]) -> [PASS][90]
[89]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-iclb6/igt@kms_psr@psr2_primary_mmap_gtt.html
[90]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-iclb2/igt@kms_psr@psr2_primary_mmap_gtt.html
* igt@perf@polling-small-buf:
- shard-skl: [FAIL][91] ([i915#1722]) -> [PASS][92]
[91]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl1/igt@perf@polling-small-buf.html
[92]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl5/igt@perf@polling-small-buf.html
* igt@sysfs_preempt_timeout@timeout@rcs0:
- shard-skl: [FAIL][93] ([i915#2821]) -> [PASS][94]
[93]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl7/igt@sysfs_preempt_timeout@timeout@rcs0.html
[94]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl7/igt@sysfs_preempt_timeout@timeout@rcs0.html
#### Warnings ####
* igt@gem_exec_fair@basic-pace@vecs0:
- shard-kbl: [FAIL][95] ([i915#2842]) -> [SKIP][96] ([fdo#109271])
[95]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl3/igt@gem_exec_fair@basic-pace@vecs0.html
[96]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl1/igt@gem_exec_fair@basic-pace@vecs0.html
* igt@i915_selftest@live@execlists:
- shard-tglb: [DMESG-FAIL][97] ([i915#3462]) -> [INCOMPLETE][98] ([i915#3462])
[97]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-tglb7/igt@i915_selftest@live@execlists.html
[98]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-tglb5/igt@i915_selftest@live@execlists.html
* igt@i915_selftest@live@gt_pm:
- shard-skl: [DMESG-FAIL][99] ([i915#2291]) -> [DMESG-FAIL][100] ([i915#1886] / [i915#2291])
[99]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-skl1/igt@i915_selftest@live@gt_pm.html
[100]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-skl4/igt@i915_selftest@live@gt_pm.html
* igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-1:
- shard-iclb: [SKIP][101] ([i915#658]) -> [SKIP][102] ([i915#2920])
[101]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-iclb6/igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-1.html
[102]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-iclb2/igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-1.html
* igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-5:
- shard-iclb: [SKIP][103] ([i915#2920]) -> [SKIP][104] ([i915#658]) +2 similar issues
[103]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-iclb2/igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-5.html
[104]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-iclb7/igt@kms_psr2_sf@primary-plane-update-sf-dmg-area-5.html
* igt@runner@aborted:
- shard-kbl: ([FAIL][105], [FAIL][106], [FAIL][107], [FAIL][108], [FAIL][109], [FAIL][110], [FAIL][111], [FAIL][112], [FAIL][113], [FAIL][114], [FAIL][115]) ([i915#1436] / [i915#180] / [i915#1814] / [i915#3002] / [i915#3363] / [i915#602]) -> ([FAIL][116], [FAIL][117], [FAIL][118], [FAIL][119], [FAIL][120], [FAIL][121], [FAIL][122], [FAIL][123], [FAIL][124], [FAIL][125], [FAIL][126]) ([i915#1436] / [i915#180] / [i915#1814] / [i915#3002] / [i915#3363] / [i915#602] / [i915#92])
[105]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl4/igt@runner@aborted.html
[106]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl4/igt@runner@aborted.html
[107]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl3/igt@runner@aborted.html
[108]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl3/igt@runner@aborted.html
[109]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl7/igt@runner@aborted.html
[110]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl1/igt@runner@aborted.html
[111]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl3/igt@runner@aborted.html
[112]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl1/igt@runner@aborted.html
[113]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl1/igt@runner@aborted.html
[114]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl2/igt@runner@aborted.html
[115]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-kbl3/igt@runner@aborted.html
[116]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl4/igt@runner@aborted.html
[117]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl4/igt@runner@aborted.html
[118]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl3/igt@runner@aborted.html
[119]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl4/igt@runner@aborted.html
[120]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl7/igt@runner@aborted.html
[121]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl4/igt@runner@aborted.html
[122]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl2/igt@runner@aborted.html
[123]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl3/igt@runner@aborted.html
[124]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl1/igt@runner@aborted.html
[125]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl2/igt@runner@aborted.html
[126]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-kbl1/igt@runner@aborted.html
- shard-apl: ([FAIL][127], [FAIL][128]) ([fdo#109271] / [i915#3002] / [i915#3363]) -> ([FAIL][129], [FAIL][130], [FAIL][131], [FAIL][132]) ([fdo#109271] / [i915#180] / [i915#3002] / [i915#3363])
[127]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-apl7/igt@runner@aborted.html
[128]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-apl3/igt@runner@aborted.html
[129]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl6/igt@runner@aborted.html
[130]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl1/igt@runner@aborted.html
[131]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl7/igt@runner@aborted.html
[132]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-apl8/igt@runner@aborted.html
- shard-tglb: ([FAIL][133], [FAIL][134], [FAIL][135]) ([i915#1436] / [i915#2426] / [i915#2966] / [i915#3002]) -> ([FAIL][136], [FAIL][137], [FAIL][138]) ([i915#1436] / [i915#2966] / [i915#3002])
[133]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-tglb5/igt@runner@aborted.html
[134]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-tglb6/igt@runner@aborted.html
[135]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10168/shard-tglb7/igt@runner@aborted.html
[136]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-tglb5/igt@runner@aborted.html
[137]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-tglb3/igt@runner@aborted.html
[138]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/shard-tglb6/igt@runner@aborted.html
[fdo#108145]: https://bugs.freedesktop.org/show_bug.cgi?id=108145
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[i915#1099]: https://gitlab.freedesktop.org/drm/intel/issues/1099
[i915#1188]: https://gitlab.freedesktop.org/drm/intel/issues/1188
[i915#1436]: https://gitlab.freedesktop.org/drm/intel/issues/1436
[i915#155]: https://gitlab.freedesktop.org/drm/intel/issues/155
[i915#1722]: https://gitlab.freedesktop.org/drm/intel/issues/1722
[i915#180]: https://gitlab.freedesktop.org/drm/intel/issues/180
[i915#1814]: https://gitlab.freedesktop.org/drm/intel/issues/1814
[i915#1886]: https://gitlab.freedesktop.org/drm/intel/issues/1886
[i915#198]: https://gitlab.freedesktop.org/drm/intel/issues/198
[i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
[i915#2122]: https://gitlab.freedesktop.org/drm/intel/issues/2122
[i915#2291]: https://gitlab.freedesktop.org/drm/intel/issues/2291
[i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
[i915#2389]: https://gitlab.freedesktop.org/drm/intel/issues/2389
[i915#2410]: https://gitlab.freedesktop.org/drm/intel/issues/2410
[i915#2426]: https://gitlab.freedesktop.org/drm/intel/issues/2426
[i915#2428]: https://gitlab.freedesktop.org/drm/intel/issues/2428
[i915#2437]: https://gitlab.freedesktop.org/drm/intel/issues/2437
[i915#2521]: https://gitlab.freedesktop.org/drm/intel/issues/2521
[i915#265]: https://gitlab.freedesktop.org/drm/intel/issues/265
[i915#2672]: https://gitlab.freedesktop.org/drm/intel/issues/2672
[i915#2705]: https://gitlab.freedesktop.org/drm/intel/issues/2705
[i915#2724]: https://gitlab.freedesktop.org/drm/intel/issues/2724
[i915#2782]: https://gitlab.freedesktop.org/drm/intel/issues/2782
[i915#2821]: https://gitlab.freedesktop.org/drm/intel/issues/2821
[i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
[i915#2846]: https://gitlab.freedesktop.org/drm/intel/issues/2846
[i915#2849]: https://gitlab.freedesktop.org/drm/intel/issues/2849
[i915#2920]: https://gitlab.freedesktop.org/drm/intel/issues/2920
[i915#2966]: https://gitlab.freedesktop.org/drm/intel/issues/2966
[i915#2994]: https://gitlab.freedesktop.org/drm/intel/issues/2994
[i915#3002]: https://gitlab.freedesktop.org/drm/intel/issues/3002
[i915#307]: https://gitlab.freedesktop.org/drm/intel/issues/307
[i915#3160]: https://gitlab.freedesktop.org/drm/intel/issues/3160
[i915#3318]: https://gitlab.freedesktop.org/drm/intel/issues/3318
[i915#3363]: https://gitlab.freedesktop.org/drm/intel/issues/3363
[i915#3444]: https://gitlab.freedesktop.org/drm/intel/issues/3444
[i915#3462]: https://gitlab.freedesktop.org/drm/intel/issues/3462
[i915#454]: https://gitlab.freedesktop.org/drm/intel/issues/454
[i915#533]: https://gitlab.freedesktop.org/drm/intel/issues/533
[i915#602]: https://gitlab.freedesktop.org/drm/intel/issues/602
[i915#636]: https://gitlab.freedesktop.org/drm/intel/issues/636
[i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
[i915#716]: https://gitlab.freedesktop.org/drm/intel/issues/716
[i915#92]: https://gitlab.freedesktop.org/drm/intel/issues/92
Participating hosts (10 -> 10)
------------------------------
No changes in participating hosts
Build changes
-------------
* Linux: CI_DRM_10168 -> Patchwork_20279
CI-20190529: 20190529
CI_DRM_10168: ad6cb7b043355cf535e0027ff90c4a6aacb790d1 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6098: 1fbc1e7d602f96a7f4e2b95057eef994656b8e74 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
Patchwork_20279: 378f81235475bf26dcbe8c9daaafd01aea52fb13 @ git://anongit.freedesktop.org/gfx-ci/linux
piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20279/index.html
[-- Attachment #1.2: Type: text/html, Size: 37018 bytes --]
[-- Attachment #2: Type: text/plain, Size: 160 bytes --]
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 9/9] drm/i915/doc: Add kernel doc for i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 17:20 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 17:20 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> Documentation/gpu/i915.rst | 6 ++++
> drivers/gpu/drm/i915/i915_scheduler_types.h | 37 ++++++++++++++++++---
> 2 files changed, 38 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/gpu/i915.rst b/Documentation/gpu/i915.rst
> index 42ce0196930a..8f4f5471a05b 100644
> --- a/Documentation/gpu/i915.rst
> +++ b/Documentation/gpu/i915.rst
> @@ -425,6 +425,12 @@ User Batchbuffer Execution
> .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> :doc: User command execution
>
> +Scheduling
> +----------
> +.. kernel-doc:: drivers/gpu/drm/i915/i915_scheduler_types.h
> + :functions: i915_sched_engine
> +
> +
> Logical Rings, Logical Ring Contexts and Execlists
> --------------------------------------------------
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 9d79514450de..e3da7517853f 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> @@ -91,7 +91,21 @@ struct i915_dependency {
> &(rq__)->sched.signalers_list, \
> signal_link)
>
> +/**
> + * sturct i915_sched_engine - scheduler engine
> + *
> + * A schedule engine represents a submission queue with different priority
> + * bands. It contains all the common state (relative to the backend) to queue,
> + * track, and submit a request.
> + *
> + * This object at the moment is quite i915 specific but will transition into a
> + * container for the drm_gpu_scheduler plus a few other variables once the i915
> + * is integrated with the DRM scheduler.
> + */
Why is this its own patch? The above comment would be nice to have in
patch 1/9 and the kref could be documented there too.
> struct i915_sched_engine {
> + /**
> + * @ref: reference count of schedule engine object
> + */
> struct kref ref;
>
> /**
> @@ -100,11 +114,18 @@ struct i915_sched_engine {
> */
> spinlock_t lock;
>
> + /**
> + * @requests: list of requests inflight on this schedule engine
> + */
> struct list_head requests;
> - struct list_head hold; /* ready requests, but on hold */
>
> /**
> - * @tasklet: softirq tasklet for bottom handler
> + * @hold: list of requests on hold.
> + */
> + struct list_head hold;
> +
> + /**
> + * @tasklet: softirq tasklet for submission
> */
> struct tasklet_struct tasklet;
>
> @@ -137,14 +158,20 @@ struct i915_sched_engine {
> */
> bool no_priolist;
>
> - /* Back pointer to engine */
> + /**
> + * @engine: back pointer to engine
> + */
> struct intel_engine_cs *engine;
>
> - /* Kick backend */
> + /**
> + * @kick_backed: kick back after a request's priority has changed
> + */
> void (*kick_backend)(const struct i915_request *rq,
> int prio);
>
> - /*
> + /**
> + * @schedule: schedule function to adjust priority of request
> + *
> * Call when the priority on a request has changed and it and its
> * dependencies may need rescheduling. Note the request itself may
> * not be ready to run!
> --
> 2.28.0
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 9/9] drm/i915/doc: Add kernel doc for i915_sched_engine
@ 2021-06-04 17:20 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 17:20 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> Documentation/gpu/i915.rst | 6 ++++
> drivers/gpu/drm/i915/i915_scheduler_types.h | 37 ++++++++++++++++++---
> 2 files changed, 38 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/gpu/i915.rst b/Documentation/gpu/i915.rst
> index 42ce0196930a..8f4f5471a05b 100644
> --- a/Documentation/gpu/i915.rst
> +++ b/Documentation/gpu/i915.rst
> @@ -425,6 +425,12 @@ User Batchbuffer Execution
> .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> :doc: User command execution
>
> +Scheduling
> +----------
> +.. kernel-doc:: drivers/gpu/drm/i915/i915_scheduler_types.h
> + :functions: i915_sched_engine
> +
> +
> Logical Rings, Logical Ring Contexts and Execlists
> --------------------------------------------------
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 9d79514450de..e3da7517853f 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> @@ -91,7 +91,21 @@ struct i915_dependency {
> &(rq__)->sched.signalers_list, \
> signal_link)
>
> +/**
> + * sturct i915_sched_engine - scheduler engine
> + *
> + * A schedule engine represents a submission queue with different priority
> + * bands. It contains all the common state (relative to the backend) to queue,
> + * track, and submit a request.
> + *
> + * This object at the moment is quite i915 specific but will transition into a
> + * container for the drm_gpu_scheduler plus a few other variables once the i915
> + * is integrated with the DRM scheduler.
> + */
Why is this its own patch? The above comment would be nice to have in
patch 1/9 and the kref could be documented there too.
> struct i915_sched_engine {
> + /**
> + * @ref: reference count of schedule engine object
> + */
> struct kref ref;
>
> /**
> @@ -100,11 +114,18 @@ struct i915_sched_engine {
> */
> spinlock_t lock;
>
> + /**
> + * @requests: list of requests inflight on this schedule engine
> + */
> struct list_head requests;
> - struct list_head hold; /* ready requests, but on hold */
>
> /**
> - * @tasklet: softirq tasklet for bottom handler
> + * @hold: list of requests on hold.
> + */
> + struct list_head hold;
> +
> + /**
> + * @tasklet: softirq tasklet for submission
> */
> struct tasklet_struct tasklet;
>
> @@ -137,14 +158,20 @@ struct i915_sched_engine {
> */
> bool no_priolist;
>
> - /* Back pointer to engine */
> + /**
> + * @engine: back pointer to engine
> + */
> struct intel_engine_cs *engine;
>
> - /* Kick backend */
> + /**
> + * @kick_backed: kick back after a request's priority has changed
> + */
> void (*kick_backend)(const struct i915_request *rq,
> int prio);
>
> - /*
> + /**
> + * @schedule: schedule function to adjust priority of request
> + *
> * Call when the priority on a request has changed and it and its
> * dependencies may need rescheduling. Note the request itself may
> * not be ready to run!
> --
> 2.28.0
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 9/9] drm/i915/doc: Add kernel doc for i915_sched_engine
2021-06-04 17:20 ` Jason Ekstrand
@ 2021-06-04 17:26 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 17:26 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 12:20:36PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > Documentation/gpu/i915.rst | 6 ++++
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 37 ++++++++++++++++++---
> > 2 files changed, 38 insertions(+), 5 deletions(-)
> >
> > diff --git a/Documentation/gpu/i915.rst b/Documentation/gpu/i915.rst
> > index 42ce0196930a..8f4f5471a05b 100644
> > --- a/Documentation/gpu/i915.rst
> > +++ b/Documentation/gpu/i915.rst
> > @@ -425,6 +425,12 @@ User Batchbuffer Execution
> > .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> > :doc: User command execution
> >
> > +Scheduling
> > +----------
> > +.. kernel-doc:: drivers/gpu/drm/i915/i915_scheduler_types.h
> > + :functions: i915_sched_engine
> > +
> > +
> > Logical Rings, Logical Ring Contexts and Execlists
> > --------------------------------------------------
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 9d79514450de..e3da7517853f 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > @@ -91,7 +91,21 @@ struct i915_dependency {
> > &(rq__)->sched.signalers_list, \
> > signal_link)
> >
> > +/**
> > + * sturct i915_sched_engine - scheduler engine
> > + *
> > + * A schedule engine represents a submission queue with different priority
> > + * bands. It contains all the common state (relative to the backend) to queue,
> > + * track, and submit a request.
> > + *
> > + * This object at the moment is quite i915 specific but will transition into a
> > + * container for the drm_gpu_scheduler plus a few other variables once the i915
> > + * is integrated with the DRM scheduler.
> > + */
>
> Why is this its own patch? The above comment would be nice to have in
> patch 1/9 and the kref could be documented there too.
>
I thought it make the review easier? Either way, can squash this patch into the
previous 8 as needed in next rev.
Matt
> > struct i915_sched_engine {
> > + /**
> > + * @ref: reference count of schedule engine object
> > + */
> > struct kref ref;
> >
> > /**
> > @@ -100,11 +114,18 @@ struct i915_sched_engine {
> > */
> > spinlock_t lock;
> >
> > + /**
> > + * @requests: list of requests inflight on this schedule engine
> > + */
> > struct list_head requests;
> > - struct list_head hold; /* ready requests, but on hold */
> >
> > /**
> > - * @tasklet: softirq tasklet for bottom handler
> > + * @hold: list of requests on hold.
> > + */
> > + struct list_head hold;
> > +
> > + /**
> > + * @tasklet: softirq tasklet for submission
> > */
> > struct tasklet_struct tasklet;
> >
> > @@ -137,14 +158,20 @@ struct i915_sched_engine {
> > */
> > bool no_priolist;
> >
> > - /* Back pointer to engine */
> > + /**
> > + * @engine: back pointer to engine
> > + */
> > struct intel_engine_cs *engine;
> >
> > - /* Kick backend */
> > + /**
> > + * @kick_backed: kick back after a request's priority has changed
> > + */
> > void (*kick_backend)(const struct i915_request *rq,
> > int prio);
> >
> > - /*
> > + /**
> > + * @schedule: schedule function to adjust priority of request
> > + *
> > * Call when the priority on a request has changed and it and its
> > * dependencies may need rescheduling. Note the request itself may
> > * not be ready to run!
> > --
> > 2.28.0
> >
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 9/9] drm/i915/doc: Add kernel doc for i915_sched_engine
@ 2021-06-04 17:26 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 17:26 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 12:20:36PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > Documentation/gpu/i915.rst | 6 ++++
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 37 ++++++++++++++++++---
> > 2 files changed, 38 insertions(+), 5 deletions(-)
> >
> > diff --git a/Documentation/gpu/i915.rst b/Documentation/gpu/i915.rst
> > index 42ce0196930a..8f4f5471a05b 100644
> > --- a/Documentation/gpu/i915.rst
> > +++ b/Documentation/gpu/i915.rst
> > @@ -425,6 +425,12 @@ User Batchbuffer Execution
> > .. kernel-doc:: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> > :doc: User command execution
> >
> > +Scheduling
> > +----------
> > +.. kernel-doc:: drivers/gpu/drm/i915/i915_scheduler_types.h
> > + :functions: i915_sched_engine
> > +
> > +
> > Logical Rings, Logical Ring Contexts and Execlists
> > --------------------------------------------------
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 9d79514450de..e3da7517853f 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > @@ -91,7 +91,21 @@ struct i915_dependency {
> > &(rq__)->sched.signalers_list, \
> > signal_link)
> >
> > +/**
> > + * sturct i915_sched_engine - scheduler engine
> > + *
> > + * A schedule engine represents a submission queue with different priority
> > + * bands. It contains all the common state (relative to the backend) to queue,
> > + * track, and submit a request.
> > + *
> > + * This object at the moment is quite i915 specific but will transition into a
> > + * container for the drm_gpu_scheduler plus a few other variables once the i915
> > + * is integrated with the DRM scheduler.
> > + */
>
> Why is this its own patch? The above comment would be nice to have in
> patch 1/9 and the kref could be documented there too.
>
I thought it make the review easier? Either way, can squash this patch into the
previous 8 as needed in next rev.
Matt
> > struct i915_sched_engine {
> > + /**
> > + * @ref: reference count of schedule engine object
> > + */
> > struct kref ref;
> >
> > /**
> > @@ -100,11 +114,18 @@ struct i915_sched_engine {
> > */
> > spinlock_t lock;
> >
> > + /**
> > + * @requests: list of requests inflight on this schedule engine
> > + */
> > struct list_head requests;
> > - struct list_head hold; /* ready requests, but on hold */
> >
> > /**
> > - * @tasklet: softirq tasklet for bottom handler
> > + * @hold: list of requests on hold.
> > + */
> > + struct list_head hold;
> > +
> > + /**
> > + * @tasklet: softirq tasklet for submission
> > */
> > struct tasklet_struct tasklet;
> >
> > @@ -137,14 +158,20 @@ struct i915_sched_engine {
> > */
> > bool no_priolist;
> >
> > - /* Back pointer to engine */
> > + /**
> > + * @engine: back pointer to engine
> > + */
> > struct intel_engine_cs *engine;
> >
> > - /* Kick backend */
> > + /**
> > + * @kick_backed: kick back after a request's priority has changed
> > + */
> > void (*kick_backend)(const struct i915_request *rq,
> > int prio);
> >
> > - /*
> > + /**
> > + * @schedule: schedule function to adjust priority of request
> > + *
> > * Call when the priority on a request has changed and it and its
> > * dependencies may need rescheduling. Note the request itself may
> > * not be ready to run!
> > --
> > 2.28.0
> >
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
2021-06-04 17:38 ` [Intel-gfx] " Jason Ekstrand
@ 2021-06-04 17:35 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 17:35 UTC (permalink / raw)
To: Jason Ekstrand
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Fri, Jun 04, 2021 at 12:38:22PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Introduce i915_sched_engine object which is lower level data structure
> > that i915_scheduler / generic code can operate on without touching
> > execlist specific structures. This allows additional submission backends
> > to be added without breaking the layering.
> >
> > This is a bit of detour to integrating the i915 with the DRM scheduler
> > but this object will still exist when the DRM scheduler lands in the
> > i915. It will however look a bit different. It will encapsulate the
> > drm_gpu_scheduler object plus and common variables (to the backends)
> > related to scheduling. Regardless this is a step in the right direction.
> >
> > This patch starts the aforementioned transition by moving the the
> > priolist into the i915_sched_engine object.
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
> > drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
> > drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
> > .../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
> > drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
> > drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
> > drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
> > 9 files changed, 169 insertions(+), 90 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > index 3f9a811eb02b..d0f3814440f6 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > @@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
> > memset(execlists->pending, 0, sizeof(execlists->pending));
> > execlists->active =
> > memset(execlists->inflight, 0, sizeof(execlists->inflight));
> > -
> > - execlists->queue_priority_hint = INT_MIN;
> > - execlists->queue = RB_ROOT_CACHED;
> > }
> >
> > static void cleanup_status_page(struct intel_engine_cs *engine)
> > @@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > goto err_status;
> > }
> >
> > + engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
> > + if (!engine->sched_engine) {
> > + err = -ENOMEM;
> > + goto err_sched_engine;
> > + }
> > +
> > err = intel_engine_init_cmd_parser(engine);
> > if (err)
> > goto err_cmd_parser;
> > @@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > return 0;
> >
> > err_cmd_parser:
> > + i915_sched_engine_put(engine->sched_engine);
> > +err_sched_engine:
> > intel_breadcrumbs_free(engine->breadcrumbs);
> > err_status:
> > cleanup_status_page(engine);
> > @@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > GEM_BUG_ON(!list_empty(&engine->active.requests));
> > tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> >
> > + i915_sched_engine_put(engine->sched_engine);
> > intel_breadcrumbs_free(engine->breadcrumbs);
> >
> > intel_engine_fini_retire(engine);
> > @@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> > intel_engine_flush_submission(engine);
> >
> > /* ELSP is empty, but there are ready requests? E.g. after reset */
> > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
> > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> > return false;
> >
> > /* Ring stopped? */
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > index 47f4397095e5..b6a00dd72808 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > @@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
> > intel_breadcrumbs_park(engine->breadcrumbs);
> >
> > /* Must be reset upon idling, or we may miss the busy wakeup. */
> > - GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
> > + GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
> >
> > if (engine->park)
> > engine->park(engine);
> >
> > - engine->execlists.no_priolist = false;
> > + engine->sched_engine->no_priolist = false;
> >
> > /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> > intel_gt_pm_put_async(engine->gt);
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > index 9ef349cd5cea..86b41ddec373 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > @@ -59,6 +59,7 @@ struct drm_i915_reg_table;
> > struct i915_gem_context;
> > struct i915_request;
> > struct i915_sched_attr;
> > +struct i915_sched_engine;
> > struct intel_gt;
> > struct intel_ring;
> > struct intel_uncore;
> > @@ -152,11 +153,6 @@ struct intel_engine_execlists {
> > */
> > struct timer_list preempt;
> >
> > - /**
> > - * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > - */
> > - struct i915_priolist default_priolist;
> > -
> > /**
> > * @ccid: identifier for contexts submitted to this engine
> > */
> > @@ -191,11 +187,6 @@ struct intel_engine_execlists {
> > */
> > u32 reset_ccid;
> >
> > - /**
> > - * @no_priolist: priority lists disabled
> > - */
> > - bool no_priolist;
> > -
> > /**
> > * @submit_reg: gen-specific execlist submission register
> > * set to the ExecList Submission Port (elsp) register pre-Gen11 and to
> > @@ -238,23 +229,8 @@ struct intel_engine_execlists {
> > unsigned int port_mask;
> >
> > /**
> > - * @queue_priority_hint: Highest pending priority.
> > - *
> > - * When we add requests into the queue, or adjust the priority of
> > - * executing requests, we compute the maximum priority of those
> > - * pending requests. We can then use this value to determine if
> > - * we need to preempt the executing requests to service the queue.
> > - * However, since the we may have recorded the priority of an inflight
> > - * request we wanted to preempt but since completed, at the time of
> > - * dequeuing the priority hint may no longer may match the highest
> > - * available request priority.
> > + * @virtual: virtual of requests, in priority lists
>
> Having trouble parsing this. Did you mean "queue of virtual
> requests"? Or maybe "virtual queue of requests"?
>
Yep, this doesn't really make sense. How about:
'queue of requests on a virtual engine, in priority lists'
> > */
> > - int queue_priority_hint;
> > -
> > - /**
> > - * @queue: queue of requests, in priority lists
> > - */
> > - struct rb_root_cached queue;
> > struct rb_root_cached virtual;
> >
> > /**
> > @@ -332,6 +308,8 @@ struct intel_engine_cs {
> > struct list_head hold; /* ready requests, but on hold */
> > } active;
> >
> > + struct i915_sched_engine *sched_engine;
> > +
> > /* keep a request in reserve for a [pm] barrier under oom */
> > struct i915_request *request_pool;
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 8db200422950..d1dc1db3e378 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
> > return prio;
> > }
> >
> > -static int queue_prio(const struct intel_engine_execlists *execlists)
> > +static int queue_prio(const struct i915_sched_engine *sched_engine)
> > {
> > struct rb_node *rb;
> >
> > - rb = rb_first_cached(&execlists->queue);
> > + rb = rb_first_cached(&sched_engine->queue);
> > if (!rb)
> > return INT_MIN;
> >
> > @@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > * to preserve FIFO ordering of dependencies.
> > */
> > last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
> > - if (engine->execlists.queue_priority_hint <= last_prio)
> > + if (engine->sched_engine->queue_priority_hint <= last_prio)
> > return false;
> >
> > /*
> > @@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > * context, it's priority would not exceed ELSP[0] aka last_prio.
> > */
> > return max(virtual_prio(&engine->execlists),
> > - queue_prio(&engine->execlists)) > last_prio;
> > + queue_prio(engine->sched_engine)) > last_prio;
> > }
> >
> > __maybe_unused static bool
> > @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > prio = rq_prio(rq);
> > pl = i915_sched_lookup_priolist(engine, prio);
> > }
> > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> >
> > list_move(&rq->sched.link, pl);
> > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> > }
> >
> > /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
> > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> > ENGINE_TRACE(engine, "timeslice required for queue\n");
> > return true;
> > }
> > @@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
> > static void execlists_dequeue(struct intel_engine_cs *engine)
> > {
> > struct intel_engine_execlists * const execlists = &engine->execlists;
> > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_request **port = execlists->pending;
> > struct i915_request ** const last_port = port + execlists->port_mask;
> > struct i915_request *last, * const *active;
> > @@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > last->fence.context,
> > last->fence.seqno,
> > last->sched.attr.priority,
> > - execlists->queue_priority_hint);
> > + sched_engine->queue_priority_hint);
> > record_preemption(execlists);
> >
> > /*
> > @@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > yesno(timer_expired(&execlists->timer)),
> > last->fence.context, last->fence.seqno,
> > rq_prio(last),
> > - execlists->queue_priority_hint,
> > + sched_engine->queue_priority_hint,
> > yesno(timeslice_yield(execlists, last)));
> >
> > /*
> > @@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > GEM_BUG_ON(rq->engine != &ve->base);
> > GEM_BUG_ON(rq->context != &ve->context);
> >
> > - if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
> > + if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> > spin_unlock(&ve->base.active.lock);
> > break;
> > }
> > @@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > yesno(engine != ve->siblings[0]));
> >
> > WRITE_ONCE(ve->request, NULL);
> > - WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
> > + WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
> >
> > rb = &ve->nodes[engine->id].rb;
> > rb_erase_cached(rb, &execlists->virtual);
> > @@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > break;
> > }
> >
> > - while ((rb = rb_first_cached(&execlists->queue))) {
> > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > struct i915_priolist *p = to_priolist(rb);
> > struct i915_request *rq, *rn;
> >
> > @@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > }
> > }
> >
> > - rb_erase_cached(&p->node, &execlists->queue);
> > + rb_erase_cached(&p->node, &sched_engine->queue);
> > i915_priolist_free(p);
> > }
> > done:
> > @@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > * request triggering preemption on the next dequeue (or subsequent
> > * interrupt for secondary ports).
> > */
> > - execlists->queue_priority_hint = queue_prio(execlists);
> > + sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > spin_unlock(&engine->active.lock);
> >
> > /*
> > @@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> > */
> > __execlists_unhold(rq);
> >
> > - if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
> > - engine->execlists.queue_priority_hint = rq_prio(rq);
> > + if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> > + engine->sched_engine->queue_priority_hint = rq_prio(rq);
> > tasklet_hi_schedule(&engine->execlists.tasklet);
> > }
> >
> > @@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
> > static bool submit_queue(struct intel_engine_cs *engine,
> > const struct i915_request *rq)
> > {
> > - struct intel_engine_execlists *execlists = &engine->execlists;
> > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> >
> > - if (rq_prio(rq) <= execlists->queue_priority_hint)
> > + if (rq_prio(rq) <= sched_engine->queue_priority_hint)
> > return false;
> >
> > - execlists->queue_priority_hint = rq_prio(rq);
> > + sched_engine->queue_priority_hint = rq_prio(rq);
> > return true;
> > }
> >
> > @@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> > } else {
> > queue_request(engine, request);
> >
> > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > GEM_BUG_ON(list_empty(&request->sched.link));
> >
> > if (submit_queue(engine, request))
> > @@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
> > from_tasklet(engine, t, execlists.tasklet);
> >
> > /* The driver is wedged; don't process any more events. */
> > - WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
> > + WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> > }
> >
> > static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > {
> > struct intel_engine_execlists * const execlists = &engine->execlists;
> > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_request *rq, *rn;
> > struct rb_node *rb;
> > unsigned long flags;
> > @@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > intel_engine_signal_breadcrumbs(engine);
> >
> > /* Flush the queued requests to the timeline list (for retiring). */
> > - while ((rb = rb_first_cached(&execlists->queue))) {
> > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > struct i915_priolist *p = to_priolist(rb);
> >
> > priolist_for_each_request_consume(rq, rn, p) {
> > @@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > }
> > }
> >
> > - rb_erase_cached(&p->node, &execlists->queue);
> > + rb_erase_cached(&p->node, &sched_engine->queue);
> > i915_priolist_free(p);
> > }
> >
> > @@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > }
> > i915_request_put(rq);
> >
> > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > + ve->base.sched_engine->queue_priority_hint = INT_MIN;
> > }
> > spin_unlock(&ve->base.active.lock);
> > }
> >
> > /* Remaining _unready_ requests will be nop'ed when submitted */
> >
> > - execlists->queue_priority_hint = INT_MIN;
> > - execlists->queue = RB_ROOT_CACHED;
> > + sched_engine->queue_priority_hint = INT_MIN;
> > + sched_engine->queue = RB_ROOT_CACHED;
> >
> > GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > execlists->tasklet.callback = nop_submission_tasklet;
> > @@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> >
> > static struct list_head *virtual_queue(struct virtual_engine *ve)
> > {
> > - return &ve->base.execlists.default_priolist.requests;
> > + return &ve->base.sched_engine->default_priolist.requests;
> > }
> >
> > static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > @@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > lrc_fini(&ve->context);
> > intel_context_fini(&ve->context);
> >
> > - intel_breadcrumbs_free(ve->base.breadcrumbs);
> > + if (ve->base.breadcrumbs)
> > + intel_breadcrumbs_free(ve->base.breadcrumbs);
> > + if (ve->base.sched_engine)
> > + i915_sched_engine_put(ve->base.sched_engine);
>
> Will either of these ever actually be NULL? The first line or two
> seems to indicate that breadcrumbs never is.
>
> > intel_engine_free_request_pool(&ve->base);
> >
> > kfree(ve->bonds);
> > @@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
> >
> > ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
> > rq->fence.context, rq->fence.seqno,
> > - mask, ve->base.execlists.queue_priority_hint);
> > + mask, ve->base.sched_engine->queue_priority_hint);
> >
> > return mask;
> > }
> > @@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > {
> > struct virtual_engine * const ve =
> > from_tasklet(ve, t, base.execlists.tasklet);
> > - const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
> > + const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> > intel_engine_mask_t mask;
> > unsigned int n;
> >
> > @@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > submit_engine:
> > GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> > node->prio = prio;
> > - if (first && prio > sibling->execlists.queue_priority_hint)
> > + if (first && prio > sibling->sched_engine->queue_priority_hint)
> > tasklet_hi_schedule(&sibling->execlists.tasklet);
> >
> > unlock_engine:
> > @@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > i915_request_put(ve->request);
> > }
> >
> > - ve->base.execlists.queue_priority_hint = rq_prio(rq);
> > + ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
> > ve->request = i915_request_get(rq);
> >
> > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > @@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> > intel_engine_init_execlists(&ve->base);
> >
> > + ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> > + if (!ve->base.sched_engine) {
> > + err = -ENOMEM;
> > + goto err_put;
> > + }
> > +
> > ve->base.cops = &virtual_context_ops;
> > ve->base.request_alloc = execlists_request_alloc;
> >
> > @@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > ve->base.bond_execute = virtual_bond_execute;
> >
> > INIT_LIST_HEAD(virtual_queue(ve));
> > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> >
> > intel_context_init(&ve->context, &ve->base);
> > @@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > unsigned int max)
> > {
> > const struct intel_engine_execlists *execlists = &engine->execlists;
> > + const struct i915_sched_engine *sched_engine = engine->sched_engine;
> > struct i915_request *rq, *last;
> > unsigned long flags;
> > unsigned int count;
> > @@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > show_request(m, last, "\t\t", 0);
> > }
> >
> > - if (execlists->queue_priority_hint != INT_MIN)
> > + if (sched_engine->queue_priority_hint != INT_MIN)
> > drm_printf(m, "\t\tQueue priority hint: %d\n",
> > - READ_ONCE(execlists->queue_priority_hint));
> > + READ_ONCE(sched_engine->queue_priority_hint));
> >
> > last = NULL;
> > count = 0;
> > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
> >
> > priolist_for_each_request(rq, p) {
> > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > index 32589c6625e1..b1fdba13e900 100644
> > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > @@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
> >
> > GEM_BUG_ON(timer_pending(&mock->hw_delay));
> >
> > + i915_sched_engine_put(engine->sched_engine);
> > intel_breadcrumbs_free(engine->breadcrumbs);
> >
> > intel_context_unpin(engine->kernel_context);
> > @@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > {
> > struct intel_context *ce;
> >
> > + engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> > + if (!engine->sched_engine)
> > + return -ENOMEM;
> > +
> > intel_engine_init_active(engine, ENGINE_MOCK);
> > intel_engine_init_execlists(engine);
> > intel_engine_init__pm(engine);
> > @@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> >
> > engine->breadcrumbs = intel_breadcrumbs_create(NULL);
> > if (!engine->breadcrumbs)
> > - return -ENOMEM;
> > + goto err_schedule;
> >
> > ce = create_kernel_context(engine);
> > if (IS_ERR(ce))
> > @@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
> >
> > err_breadcrumbs:
> > intel_breadcrumbs_free(engine->breadcrumbs);
> > +err_schedule:
> > + i915_sched_engine_put(engine->sched_engine);
> > return -ENOMEM;
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 335719f17490..d42dea79ee64 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
> > static void __guc_dequeue(struct intel_engine_cs *engine)
> > {
> > struct intel_engine_execlists * const execlists = &engine->execlists;
> > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_request **first = execlists->inflight;
> > struct i915_request ** const last_port = first + execlists->port_mask;
> > struct i915_request *last = first[0];
> > @@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > * event.
> > */
> > port = first;
> > - while ((rb = rb_first_cached(&execlists->queue))) {
> > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > struct i915_priolist *p = to_priolist(rb);
> > struct i915_request *rq, *rn;
> >
> > @@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > last = rq;
> > }
> >
> > - rb_erase_cached(&p->node, &execlists->queue);
> > + rb_erase_cached(&p->node, &sched_engine->queue);
> > i915_priolist_free(p);
> > }
> > done:
> > - execlists->queue_priority_hint =
> > + sched_engine->queue_priority_hint =
> > rb ? to_priolist(rb)->priority : INT_MIN;
> > if (submit) {
> > *port = schedule_in(last, port - execlists->inflight);
> > @@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> >
> > static void guc_reset_cancel(struct intel_engine_cs *engine)
> > {
> > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_request *rq, *rn;
> > struct rb_node *rb;
> > unsigned long flags;
> > @@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > }
> >
> > /* Flush the queued requests to the timeline list (for retiring). */
> > - while ((rb = rb_first_cached(&execlists->queue))) {
> > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > struct i915_priolist *p = to_priolist(rb);
> >
> > priolist_for_each_request_consume(rq, rn, p) {
> > @@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > i915_request_mark_complete(rq);
> > }
> >
> > - rb_erase_cached(&p->node, &execlists->queue);
> > + rb_erase_cached(&p->node, &sched_engine->queue);
> > i915_priolist_free(p);
> > }
> >
> > /* Remaining _unready_ requests will be nop'ed when submitted */
> >
> > - execlists->queue_priority_hint = INT_MIN;
> > - execlists->queue = RB_ROOT_CACHED;
> > + sched_engine->queue_priority_hint = INT_MIN;
> > + sched_engine->queue = RB_ROOT_CACHED;
> >
> > spin_unlock_irqrestore(&engine->active.lock, flags);
> > }
> > @@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
> >
> > queue_request(engine, rq, rq_prio(rq));
> >
> > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > GEM_BUG_ON(list_empty(&rq->sched.link));
> >
> > tasklet_hi_schedule(&engine->execlists.tasklet);
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > index efa638c3acc7..4953874a9ba6 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > @@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> > return rb_entry(rb, struct i915_priolist, node);
> > }
> >
> > -static void assert_priolists(struct intel_engine_execlists * const execlists)
> > +static void assert_priolists(struct i915_sched_engine * const sched_engine)
> > {
> > struct rb_node *rb;
> > long last_prio;
> > @@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> > return;
> >
> > - GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
> > - rb_first(&execlists->queue.rb_root));
> > + GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
> > + rb_first(&sched_engine->queue.rb_root));
> >
> > last_prio = INT_MAX;
> > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > const struct i915_priolist *p = to_priolist(rb);
> >
> > GEM_BUG_ON(p->priority > last_prio);
> > @@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > struct list_head *
> > i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > {
> > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_priolist *p;
> > struct rb_node **parent, *rb;
> > bool first = true;
> >
> > lockdep_assert_held(&engine->active.lock);
> > - assert_priolists(execlists);
> > + assert_priolists(sched_engine);
> >
> > - if (unlikely(execlists->no_priolist))
> > + if (unlikely(sched_engine->no_priolist))
> > prio = I915_PRIORITY_NORMAL;
> >
> > find_priolist:
> > /* most positive priority is scheduled first, equal priorities fifo */
> > rb = NULL;
> > - parent = &execlists->queue.rb_root.rb_node;
> > + parent = &sched_engine->queue.rb_root.rb_node;
> > while (*parent) {
> > rb = *parent;
> > p = to_priolist(rb);
> > @@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > }
> >
> > if (prio == I915_PRIORITY_NORMAL) {
> > - p = &execlists->default_priolist;
> > + p = &sched_engine->default_priolist;
> > } else {
> > p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
> > /* Convert an allocation failure to a priority bump */
> > @@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > * requests, so if userspace lied about their
> > * dependencies that reordering may be visible.
> > */
> > - execlists->no_priolist = true;
> > + sched_engine->no_priolist = true;
> > goto find_priolist;
> > }
> > }
> > @@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > INIT_LIST_HEAD(&p->requests);
> >
> > rb_link_node(&p->node, rb, parent);
> > - rb_insert_color_cached(&p->node, &execlists->queue, first);
> > + rb_insert_color_cached(&p->node, &sched_engine->queue, first);
> >
> > return &p->requests;
> > }
> > @@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > * We only need to kick the tasklet once for the high priority
> > * new context we add into the queue.
> > */
> > - if (prio <= engine->execlists.queue_priority_hint)
> > + if (prio <= engine->sched_engine->queue_priority_hint)
> > return;
> >
> > rcu_read_lock();
> > @@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > inflight->fence.context, inflight->fence.seqno,
> > inflight->sched.attr.priority);
> >
> > - engine->execlists.queue_priority_hint = prio;
> > + engine->sched_engine->queue_priority_hint = prio;
> > if (need_preempt(prio, rq_prio(inflight)))
> > tasklet_hi_schedule(&engine->execlists.tasklet);
> >
> > @@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
> > rcu_read_unlock();
> > }
> >
> > +void i915_sched_engine_free(struct kref *kref)
> > +{
> > + struct i915_sched_engine *sched_engine =
> > + container_of(kref, typeof(*sched_engine), ref);
> > +
> > + kfree(sched_engine);
> > +}
> > +
> > +struct i915_sched_engine *
> > +i915_sched_engine_create(unsigned int subclass)
> > +{
> > + struct i915_sched_engine *sched_engine;
> > +
> > + sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
> > + if (!sched_engine)
> > + return NULL;
> > +
> > + kref_init(&sched_engine->ref);
> > +
> > + sched_engine->queue = RB_ROOT_CACHED;
> > + sched_engine->queue_priority_hint = INT_MIN;
> > +
> > + return sched_engine;
> > +}
> > +
> > static void i915_global_scheduler_shrink(void)
> > {
> > kmem_cache_shrink(global.slab_dependencies);
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > index 858a0938f47a..91a04e34cac5 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > @@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
> > __i915_priolist_free(p);
> > }
> >
> > +struct i915_sched_engine *
> > +i915_sched_engine_create(unsigned int subclass);
> > +
> > +void i915_sched_engine_free(struct kref *kref);
> > +
> > +static inline struct i915_sched_engine *
> > +i915_sched_engine_get(struct i915_sched_engine *sched_engine)
> > +{
> > + kref_get(&sched_engine->ref);
> > + return sched_engine;
> > +}
> > +
> > +static inline void
> > +i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> > +{
> > + kref_put(&sched_engine->ref, i915_sched_engine_free);
> > +}
> > +
> > void i915_request_show_with_schedule(struct drm_printer *m,
> > const struct i915_request *rq,
> > const char *prefix,
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 343ed44d5ed4..16a054e67760 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
>
> Dumb question for which I'm not really in the position of having an
> opinion: Should this go in its own file?
>
I think this is in the right file.
Matt
> Patch seems mostly mechanically correct. Not sure where this is going
> yet, though. I'll keep reading. :-)
>
> --Jason
>
>
> > @@ -91,4 +91,37 @@ struct i915_dependency {
> > &(rq__)->sched.signalers_list, \
> > signal_link)
> >
> > +struct i915_sched_engine {
> > + struct kref ref;
> > +
> > + /**
> > + * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > + */
> > + struct i915_priolist default_priolist;
> > +
> > + /**
> > + * @queue_priority_hint: Highest pending priority.
> > + *
> > + * When we add requests into the queue, or adjust the priority of
> > + * executing requests, we compute the maximum priority of those
> > + * pending requests. We can then use this value to determine if
> > + * we need to preempt the executing requests to service the queue.
> > + * However, since the we may have recorded the priority of an inflight
> > + * request we wanted to preempt but since completed, at the time of
> > + * dequeuing the priority hint may no longer may match the highest
> > + * available request priority.
> > + */
> > + int queue_priority_hint;
> > +
> > + /**
> > + * @queue: queue of requests, in priority lists
> > + */
> > + struct rb_root_cached queue;
> > +
> > + /**
> > + * @no_priolist: priority lists disabled
> > + */
> > + bool no_priolist;
> > +};
> > +
> > #endif /* _I915_SCHEDULER_TYPES_H_ */
> > --
> > 2.28.0
> >
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
@ 2021-06-04 17:35 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 17:35 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 12:38:22PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Introduce i915_sched_engine object which is lower level data structure
> > that i915_scheduler / generic code can operate on without touching
> > execlist specific structures. This allows additional submission backends
> > to be added without breaking the layering.
> >
> > This is a bit of detour to integrating the i915 with the DRM scheduler
> > but this object will still exist when the DRM scheduler lands in the
> > i915. It will however look a bit different. It will encapsulate the
> > drm_gpu_scheduler object plus and common variables (to the backends)
> > related to scheduling. Regardless this is a step in the right direction.
> >
> > This patch starts the aforementioned transition by moving the the
> > priolist into the i915_sched_engine object.
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
> > drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
> > drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
> > .../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
> > drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
> > drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
> > drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
> > 9 files changed, 169 insertions(+), 90 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > index 3f9a811eb02b..d0f3814440f6 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > @@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
> > memset(execlists->pending, 0, sizeof(execlists->pending));
> > execlists->active =
> > memset(execlists->inflight, 0, sizeof(execlists->inflight));
> > -
> > - execlists->queue_priority_hint = INT_MIN;
> > - execlists->queue = RB_ROOT_CACHED;
> > }
> >
> > static void cleanup_status_page(struct intel_engine_cs *engine)
> > @@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > goto err_status;
> > }
> >
> > + engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
> > + if (!engine->sched_engine) {
> > + err = -ENOMEM;
> > + goto err_sched_engine;
> > + }
> > +
> > err = intel_engine_init_cmd_parser(engine);
> > if (err)
> > goto err_cmd_parser;
> > @@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > return 0;
> >
> > err_cmd_parser:
> > + i915_sched_engine_put(engine->sched_engine);
> > +err_sched_engine:
> > intel_breadcrumbs_free(engine->breadcrumbs);
> > err_status:
> > cleanup_status_page(engine);
> > @@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > GEM_BUG_ON(!list_empty(&engine->active.requests));
> > tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> >
> > + i915_sched_engine_put(engine->sched_engine);
> > intel_breadcrumbs_free(engine->breadcrumbs);
> >
> > intel_engine_fini_retire(engine);
> > @@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> > intel_engine_flush_submission(engine);
> >
> > /* ELSP is empty, but there are ready requests? E.g. after reset */
> > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
> > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> > return false;
> >
> > /* Ring stopped? */
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > index 47f4397095e5..b6a00dd72808 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > @@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
> > intel_breadcrumbs_park(engine->breadcrumbs);
> >
> > /* Must be reset upon idling, or we may miss the busy wakeup. */
> > - GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
> > + GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
> >
> > if (engine->park)
> > engine->park(engine);
> >
> > - engine->execlists.no_priolist = false;
> > + engine->sched_engine->no_priolist = false;
> >
> > /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> > intel_gt_pm_put_async(engine->gt);
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > index 9ef349cd5cea..86b41ddec373 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > @@ -59,6 +59,7 @@ struct drm_i915_reg_table;
> > struct i915_gem_context;
> > struct i915_request;
> > struct i915_sched_attr;
> > +struct i915_sched_engine;
> > struct intel_gt;
> > struct intel_ring;
> > struct intel_uncore;
> > @@ -152,11 +153,6 @@ struct intel_engine_execlists {
> > */
> > struct timer_list preempt;
> >
> > - /**
> > - * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > - */
> > - struct i915_priolist default_priolist;
> > -
> > /**
> > * @ccid: identifier for contexts submitted to this engine
> > */
> > @@ -191,11 +187,6 @@ struct intel_engine_execlists {
> > */
> > u32 reset_ccid;
> >
> > - /**
> > - * @no_priolist: priority lists disabled
> > - */
> > - bool no_priolist;
> > -
> > /**
> > * @submit_reg: gen-specific execlist submission register
> > * set to the ExecList Submission Port (elsp) register pre-Gen11 and to
> > @@ -238,23 +229,8 @@ struct intel_engine_execlists {
> > unsigned int port_mask;
> >
> > /**
> > - * @queue_priority_hint: Highest pending priority.
> > - *
> > - * When we add requests into the queue, or adjust the priority of
> > - * executing requests, we compute the maximum priority of those
> > - * pending requests. We can then use this value to determine if
> > - * we need to preempt the executing requests to service the queue.
> > - * However, since the we may have recorded the priority of an inflight
> > - * request we wanted to preempt but since completed, at the time of
> > - * dequeuing the priority hint may no longer may match the highest
> > - * available request priority.
> > + * @virtual: virtual of requests, in priority lists
>
> Having trouble parsing this. Did you mean "queue of virtual
> requests"? Or maybe "virtual queue of requests"?
>
Yep, this doesn't really make sense. How about:
'queue of requests on a virtual engine, in priority lists'
> > */
> > - int queue_priority_hint;
> > -
> > - /**
> > - * @queue: queue of requests, in priority lists
> > - */
> > - struct rb_root_cached queue;
> > struct rb_root_cached virtual;
> >
> > /**
> > @@ -332,6 +308,8 @@ struct intel_engine_cs {
> > struct list_head hold; /* ready requests, but on hold */
> > } active;
> >
> > + struct i915_sched_engine *sched_engine;
> > +
> > /* keep a request in reserve for a [pm] barrier under oom */
> > struct i915_request *request_pool;
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 8db200422950..d1dc1db3e378 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
> > return prio;
> > }
> >
> > -static int queue_prio(const struct intel_engine_execlists *execlists)
> > +static int queue_prio(const struct i915_sched_engine *sched_engine)
> > {
> > struct rb_node *rb;
> >
> > - rb = rb_first_cached(&execlists->queue);
> > + rb = rb_first_cached(&sched_engine->queue);
> > if (!rb)
> > return INT_MIN;
> >
> > @@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > * to preserve FIFO ordering of dependencies.
> > */
> > last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
> > - if (engine->execlists.queue_priority_hint <= last_prio)
> > + if (engine->sched_engine->queue_priority_hint <= last_prio)
> > return false;
> >
> > /*
> > @@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > * context, it's priority would not exceed ELSP[0] aka last_prio.
> > */
> > return max(virtual_prio(&engine->execlists),
> > - queue_prio(&engine->execlists)) > last_prio;
> > + queue_prio(engine->sched_engine)) > last_prio;
> > }
> >
> > __maybe_unused static bool
> > @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > prio = rq_prio(rq);
> > pl = i915_sched_lookup_priolist(engine, prio);
> > }
> > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> >
> > list_move(&rq->sched.link, pl);
> > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> > }
> >
> > /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
> > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> > ENGINE_TRACE(engine, "timeslice required for queue\n");
> > return true;
> > }
> > @@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
> > static void execlists_dequeue(struct intel_engine_cs *engine)
> > {
> > struct intel_engine_execlists * const execlists = &engine->execlists;
> > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_request **port = execlists->pending;
> > struct i915_request ** const last_port = port + execlists->port_mask;
> > struct i915_request *last, * const *active;
> > @@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > last->fence.context,
> > last->fence.seqno,
> > last->sched.attr.priority,
> > - execlists->queue_priority_hint);
> > + sched_engine->queue_priority_hint);
> > record_preemption(execlists);
> >
> > /*
> > @@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > yesno(timer_expired(&execlists->timer)),
> > last->fence.context, last->fence.seqno,
> > rq_prio(last),
> > - execlists->queue_priority_hint,
> > + sched_engine->queue_priority_hint,
> > yesno(timeslice_yield(execlists, last)));
> >
> > /*
> > @@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > GEM_BUG_ON(rq->engine != &ve->base);
> > GEM_BUG_ON(rq->context != &ve->context);
> >
> > - if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
> > + if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> > spin_unlock(&ve->base.active.lock);
> > break;
> > }
> > @@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > yesno(engine != ve->siblings[0]));
> >
> > WRITE_ONCE(ve->request, NULL);
> > - WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
> > + WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
> >
> > rb = &ve->nodes[engine->id].rb;
> > rb_erase_cached(rb, &execlists->virtual);
> > @@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > break;
> > }
> >
> > - while ((rb = rb_first_cached(&execlists->queue))) {
> > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > struct i915_priolist *p = to_priolist(rb);
> > struct i915_request *rq, *rn;
> >
> > @@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > }
> > }
> >
> > - rb_erase_cached(&p->node, &execlists->queue);
> > + rb_erase_cached(&p->node, &sched_engine->queue);
> > i915_priolist_free(p);
> > }
> > done:
> > @@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > * request triggering preemption on the next dequeue (or subsequent
> > * interrupt for secondary ports).
> > */
> > - execlists->queue_priority_hint = queue_prio(execlists);
> > + sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > spin_unlock(&engine->active.lock);
> >
> > /*
> > @@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> > */
> > __execlists_unhold(rq);
> >
> > - if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
> > - engine->execlists.queue_priority_hint = rq_prio(rq);
> > + if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> > + engine->sched_engine->queue_priority_hint = rq_prio(rq);
> > tasklet_hi_schedule(&engine->execlists.tasklet);
> > }
> >
> > @@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
> > static bool submit_queue(struct intel_engine_cs *engine,
> > const struct i915_request *rq)
> > {
> > - struct intel_engine_execlists *execlists = &engine->execlists;
> > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> >
> > - if (rq_prio(rq) <= execlists->queue_priority_hint)
> > + if (rq_prio(rq) <= sched_engine->queue_priority_hint)
> > return false;
> >
> > - execlists->queue_priority_hint = rq_prio(rq);
> > + sched_engine->queue_priority_hint = rq_prio(rq);
> > return true;
> > }
> >
> > @@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> > } else {
> > queue_request(engine, request);
> >
> > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > GEM_BUG_ON(list_empty(&request->sched.link));
> >
> > if (submit_queue(engine, request))
> > @@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
> > from_tasklet(engine, t, execlists.tasklet);
> >
> > /* The driver is wedged; don't process any more events. */
> > - WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
> > + WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> > }
> >
> > static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > {
> > struct intel_engine_execlists * const execlists = &engine->execlists;
> > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_request *rq, *rn;
> > struct rb_node *rb;
> > unsigned long flags;
> > @@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > intel_engine_signal_breadcrumbs(engine);
> >
> > /* Flush the queued requests to the timeline list (for retiring). */
> > - while ((rb = rb_first_cached(&execlists->queue))) {
> > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > struct i915_priolist *p = to_priolist(rb);
> >
> > priolist_for_each_request_consume(rq, rn, p) {
> > @@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > }
> > }
> >
> > - rb_erase_cached(&p->node, &execlists->queue);
> > + rb_erase_cached(&p->node, &sched_engine->queue);
> > i915_priolist_free(p);
> > }
> >
> > @@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > }
> > i915_request_put(rq);
> >
> > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > + ve->base.sched_engine->queue_priority_hint = INT_MIN;
> > }
> > spin_unlock(&ve->base.active.lock);
> > }
> >
> > /* Remaining _unready_ requests will be nop'ed when submitted */
> >
> > - execlists->queue_priority_hint = INT_MIN;
> > - execlists->queue = RB_ROOT_CACHED;
> > + sched_engine->queue_priority_hint = INT_MIN;
> > + sched_engine->queue = RB_ROOT_CACHED;
> >
> > GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > execlists->tasklet.callback = nop_submission_tasklet;
> > @@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> >
> > static struct list_head *virtual_queue(struct virtual_engine *ve)
> > {
> > - return &ve->base.execlists.default_priolist.requests;
> > + return &ve->base.sched_engine->default_priolist.requests;
> > }
> >
> > static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > @@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > lrc_fini(&ve->context);
> > intel_context_fini(&ve->context);
> >
> > - intel_breadcrumbs_free(ve->base.breadcrumbs);
> > + if (ve->base.breadcrumbs)
> > + intel_breadcrumbs_free(ve->base.breadcrumbs);
> > + if (ve->base.sched_engine)
> > + i915_sched_engine_put(ve->base.sched_engine);
>
> Will either of these ever actually be NULL? The first line or two
> seems to indicate that breadcrumbs never is.
>
> > intel_engine_free_request_pool(&ve->base);
> >
> > kfree(ve->bonds);
> > @@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
> >
> > ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
> > rq->fence.context, rq->fence.seqno,
> > - mask, ve->base.execlists.queue_priority_hint);
> > + mask, ve->base.sched_engine->queue_priority_hint);
> >
> > return mask;
> > }
> > @@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > {
> > struct virtual_engine * const ve =
> > from_tasklet(ve, t, base.execlists.tasklet);
> > - const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
> > + const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> > intel_engine_mask_t mask;
> > unsigned int n;
> >
> > @@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > submit_engine:
> > GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> > node->prio = prio;
> > - if (first && prio > sibling->execlists.queue_priority_hint)
> > + if (first && prio > sibling->sched_engine->queue_priority_hint)
> > tasklet_hi_schedule(&sibling->execlists.tasklet);
> >
> > unlock_engine:
> > @@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > i915_request_put(ve->request);
> > }
> >
> > - ve->base.execlists.queue_priority_hint = rq_prio(rq);
> > + ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
> > ve->request = i915_request_get(rq);
> >
> > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > @@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> > intel_engine_init_execlists(&ve->base);
> >
> > + ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> > + if (!ve->base.sched_engine) {
> > + err = -ENOMEM;
> > + goto err_put;
> > + }
> > +
> > ve->base.cops = &virtual_context_ops;
> > ve->base.request_alloc = execlists_request_alloc;
> >
> > @@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > ve->base.bond_execute = virtual_bond_execute;
> >
> > INIT_LIST_HEAD(virtual_queue(ve));
> > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> >
> > intel_context_init(&ve->context, &ve->base);
> > @@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > unsigned int max)
> > {
> > const struct intel_engine_execlists *execlists = &engine->execlists;
> > + const struct i915_sched_engine *sched_engine = engine->sched_engine;
> > struct i915_request *rq, *last;
> > unsigned long flags;
> > unsigned int count;
> > @@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > show_request(m, last, "\t\t", 0);
> > }
> >
> > - if (execlists->queue_priority_hint != INT_MIN)
> > + if (sched_engine->queue_priority_hint != INT_MIN)
> > drm_printf(m, "\t\tQueue priority hint: %d\n",
> > - READ_ONCE(execlists->queue_priority_hint));
> > + READ_ONCE(sched_engine->queue_priority_hint));
> >
> > last = NULL;
> > count = 0;
> > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
> >
> > priolist_for_each_request(rq, p) {
> > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > index 32589c6625e1..b1fdba13e900 100644
> > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > @@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
> >
> > GEM_BUG_ON(timer_pending(&mock->hw_delay));
> >
> > + i915_sched_engine_put(engine->sched_engine);
> > intel_breadcrumbs_free(engine->breadcrumbs);
> >
> > intel_context_unpin(engine->kernel_context);
> > @@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > {
> > struct intel_context *ce;
> >
> > + engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> > + if (!engine->sched_engine)
> > + return -ENOMEM;
> > +
> > intel_engine_init_active(engine, ENGINE_MOCK);
> > intel_engine_init_execlists(engine);
> > intel_engine_init__pm(engine);
> > @@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> >
> > engine->breadcrumbs = intel_breadcrumbs_create(NULL);
> > if (!engine->breadcrumbs)
> > - return -ENOMEM;
> > + goto err_schedule;
> >
> > ce = create_kernel_context(engine);
> > if (IS_ERR(ce))
> > @@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
> >
> > err_breadcrumbs:
> > intel_breadcrumbs_free(engine->breadcrumbs);
> > +err_schedule:
> > + i915_sched_engine_put(engine->sched_engine);
> > return -ENOMEM;
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 335719f17490..d42dea79ee64 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
> > static void __guc_dequeue(struct intel_engine_cs *engine)
> > {
> > struct intel_engine_execlists * const execlists = &engine->execlists;
> > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_request **first = execlists->inflight;
> > struct i915_request ** const last_port = first + execlists->port_mask;
> > struct i915_request *last = first[0];
> > @@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > * event.
> > */
> > port = first;
> > - while ((rb = rb_first_cached(&execlists->queue))) {
> > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > struct i915_priolist *p = to_priolist(rb);
> > struct i915_request *rq, *rn;
> >
> > @@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > last = rq;
> > }
> >
> > - rb_erase_cached(&p->node, &execlists->queue);
> > + rb_erase_cached(&p->node, &sched_engine->queue);
> > i915_priolist_free(p);
> > }
> > done:
> > - execlists->queue_priority_hint =
> > + sched_engine->queue_priority_hint =
> > rb ? to_priolist(rb)->priority : INT_MIN;
> > if (submit) {
> > *port = schedule_in(last, port - execlists->inflight);
> > @@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> >
> > static void guc_reset_cancel(struct intel_engine_cs *engine)
> > {
> > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_request *rq, *rn;
> > struct rb_node *rb;
> > unsigned long flags;
> > @@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > }
> >
> > /* Flush the queued requests to the timeline list (for retiring). */
> > - while ((rb = rb_first_cached(&execlists->queue))) {
> > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > struct i915_priolist *p = to_priolist(rb);
> >
> > priolist_for_each_request_consume(rq, rn, p) {
> > @@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > i915_request_mark_complete(rq);
> > }
> >
> > - rb_erase_cached(&p->node, &execlists->queue);
> > + rb_erase_cached(&p->node, &sched_engine->queue);
> > i915_priolist_free(p);
> > }
> >
> > /* Remaining _unready_ requests will be nop'ed when submitted */
> >
> > - execlists->queue_priority_hint = INT_MIN;
> > - execlists->queue = RB_ROOT_CACHED;
> > + sched_engine->queue_priority_hint = INT_MIN;
> > + sched_engine->queue = RB_ROOT_CACHED;
> >
> > spin_unlock_irqrestore(&engine->active.lock, flags);
> > }
> > @@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
> >
> > queue_request(engine, rq, rq_prio(rq));
> >
> > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > GEM_BUG_ON(list_empty(&rq->sched.link));
> >
> > tasklet_hi_schedule(&engine->execlists.tasklet);
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > index efa638c3acc7..4953874a9ba6 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > @@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> > return rb_entry(rb, struct i915_priolist, node);
> > }
> >
> > -static void assert_priolists(struct intel_engine_execlists * const execlists)
> > +static void assert_priolists(struct i915_sched_engine * const sched_engine)
> > {
> > struct rb_node *rb;
> > long last_prio;
> > @@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> > return;
> >
> > - GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
> > - rb_first(&execlists->queue.rb_root));
> > + GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
> > + rb_first(&sched_engine->queue.rb_root));
> >
> > last_prio = INT_MAX;
> > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > const struct i915_priolist *p = to_priolist(rb);
> >
> > GEM_BUG_ON(p->priority > last_prio);
> > @@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > struct list_head *
> > i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > {
> > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_priolist *p;
> > struct rb_node **parent, *rb;
> > bool first = true;
> >
> > lockdep_assert_held(&engine->active.lock);
> > - assert_priolists(execlists);
> > + assert_priolists(sched_engine);
> >
> > - if (unlikely(execlists->no_priolist))
> > + if (unlikely(sched_engine->no_priolist))
> > prio = I915_PRIORITY_NORMAL;
> >
> > find_priolist:
> > /* most positive priority is scheduled first, equal priorities fifo */
> > rb = NULL;
> > - parent = &execlists->queue.rb_root.rb_node;
> > + parent = &sched_engine->queue.rb_root.rb_node;
> > while (*parent) {
> > rb = *parent;
> > p = to_priolist(rb);
> > @@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > }
> >
> > if (prio == I915_PRIORITY_NORMAL) {
> > - p = &execlists->default_priolist;
> > + p = &sched_engine->default_priolist;
> > } else {
> > p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
> > /* Convert an allocation failure to a priority bump */
> > @@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > * requests, so if userspace lied about their
> > * dependencies that reordering may be visible.
> > */
> > - execlists->no_priolist = true;
> > + sched_engine->no_priolist = true;
> > goto find_priolist;
> > }
> > }
> > @@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > INIT_LIST_HEAD(&p->requests);
> >
> > rb_link_node(&p->node, rb, parent);
> > - rb_insert_color_cached(&p->node, &execlists->queue, first);
> > + rb_insert_color_cached(&p->node, &sched_engine->queue, first);
> >
> > return &p->requests;
> > }
> > @@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > * We only need to kick the tasklet once for the high priority
> > * new context we add into the queue.
> > */
> > - if (prio <= engine->execlists.queue_priority_hint)
> > + if (prio <= engine->sched_engine->queue_priority_hint)
> > return;
> >
> > rcu_read_lock();
> > @@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > inflight->fence.context, inflight->fence.seqno,
> > inflight->sched.attr.priority);
> >
> > - engine->execlists.queue_priority_hint = prio;
> > + engine->sched_engine->queue_priority_hint = prio;
> > if (need_preempt(prio, rq_prio(inflight)))
> > tasklet_hi_schedule(&engine->execlists.tasklet);
> >
> > @@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
> > rcu_read_unlock();
> > }
> >
> > +void i915_sched_engine_free(struct kref *kref)
> > +{
> > + struct i915_sched_engine *sched_engine =
> > + container_of(kref, typeof(*sched_engine), ref);
> > +
> > + kfree(sched_engine);
> > +}
> > +
> > +struct i915_sched_engine *
> > +i915_sched_engine_create(unsigned int subclass)
> > +{
> > + struct i915_sched_engine *sched_engine;
> > +
> > + sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
> > + if (!sched_engine)
> > + return NULL;
> > +
> > + kref_init(&sched_engine->ref);
> > +
> > + sched_engine->queue = RB_ROOT_CACHED;
> > + sched_engine->queue_priority_hint = INT_MIN;
> > +
> > + return sched_engine;
> > +}
> > +
> > static void i915_global_scheduler_shrink(void)
> > {
> > kmem_cache_shrink(global.slab_dependencies);
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > index 858a0938f47a..91a04e34cac5 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > @@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
> > __i915_priolist_free(p);
> > }
> >
> > +struct i915_sched_engine *
> > +i915_sched_engine_create(unsigned int subclass);
> > +
> > +void i915_sched_engine_free(struct kref *kref);
> > +
> > +static inline struct i915_sched_engine *
> > +i915_sched_engine_get(struct i915_sched_engine *sched_engine)
> > +{
> > + kref_get(&sched_engine->ref);
> > + return sched_engine;
> > +}
> > +
> > +static inline void
> > +i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> > +{
> > + kref_put(&sched_engine->ref, i915_sched_engine_free);
> > +}
> > +
> > void i915_request_show_with_schedule(struct drm_printer *m,
> > const struct i915_request *rq,
> > const char *prefix,
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 343ed44d5ed4..16a054e67760 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
>
> Dumb question for which I'm not really in the position of having an
> opinion: Should this go in its own file?
>
I think this is in the right file.
Matt
> Patch seems mostly mechanically correct. Not sure where this is going
> yet, though. I'll keep reading. :-)
>
> --Jason
>
>
> > @@ -91,4 +91,37 @@ struct i915_dependency {
> > &(rq__)->sched.signalers_list, \
> > signal_link)
> >
> > +struct i915_sched_engine {
> > + struct kref ref;
> > +
> > + /**
> > + * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > + */
> > + struct i915_priolist default_priolist;
> > +
> > + /**
> > + * @queue_priority_hint: Highest pending priority.
> > + *
> > + * When we add requests into the queue, or adjust the priority of
> > + * executing requests, we compute the maximum priority of those
> > + * pending requests. We can then use this value to determine if
> > + * we need to preempt the executing requests to service the queue.
> > + * However, since the we may have recorded the priority of an inflight
> > + * request we wanted to preempt but since completed, at the time of
> > + * dequeuing the priority hint may no longer may match the highest
> > + * available request priority.
> > + */
> > + int queue_priority_hint;
> > +
> > + /**
> > + * @queue: queue of requests, in priority lists
> > + */
> > + struct rb_root_cached queue;
> > +
> > + /**
> > + * @no_priolist: priority lists disabled
> > + */
> > + bool no_priolist;
> > +};
> > +
> > #endif /* _I915_SCHEDULER_TYPES_H_ */
> > --
> > 2.28.0
> >
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 17:38 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 17:38 UTC (permalink / raw)
To: Matthew Brost
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Introduce i915_sched_engine object which is lower level data structure
> that i915_scheduler / generic code can operate on without touching
> execlist specific structures. This allows additional submission backends
> to be added without breaking the layering.
>
> This is a bit of detour to integrating the i915 with the DRM scheduler
> but this object will still exist when the DRM scheduler lands in the
> i915. It will however look a bit different. It will encapsulate the
> drm_gpu_scheduler object plus and common variables (to the backends)
> related to scheduling. Regardless this is a step in the right direction.
>
> This patch starts the aforementioned transition by moving the the
> priolist into the i915_sched_engine object.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
> drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
> drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
> .../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
> drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
> drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
> drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
> drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
> 9 files changed, 169 insertions(+), 90 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index 3f9a811eb02b..d0f3814440f6 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
> memset(execlists->pending, 0, sizeof(execlists->pending));
> execlists->active =
> memset(execlists->inflight, 0, sizeof(execlists->inflight));
> -
> - execlists->queue_priority_hint = INT_MIN;
> - execlists->queue = RB_ROOT_CACHED;
> }
>
> static void cleanup_status_page(struct intel_engine_cs *engine)
> @@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> goto err_status;
> }
>
> + engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
> + if (!engine->sched_engine) {
> + err = -ENOMEM;
> + goto err_sched_engine;
> + }
> +
> err = intel_engine_init_cmd_parser(engine);
> if (err)
> goto err_cmd_parser;
> @@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> return 0;
>
> err_cmd_parser:
> + i915_sched_engine_put(engine->sched_engine);
> +err_sched_engine:
> intel_breadcrumbs_free(engine->breadcrumbs);
> err_status:
> cleanup_status_page(engine);
> @@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> GEM_BUG_ON(!list_empty(&engine->active.requests));
> tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
>
> + i915_sched_engine_put(engine->sched_engine);
> intel_breadcrumbs_free(engine->breadcrumbs);
>
> intel_engine_fini_retire(engine);
> @@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> intel_engine_flush_submission(engine);
>
> /* ELSP is empty, but there are ready requests? E.g. after reset */
> - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
> + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> return false;
>
> /* Ring stopped? */
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> index 47f4397095e5..b6a00dd72808 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> @@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
> intel_breadcrumbs_park(engine->breadcrumbs);
>
> /* Must be reset upon idling, or we may miss the busy wakeup. */
> - GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
> + GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
>
> if (engine->park)
> engine->park(engine);
>
> - engine->execlists.no_priolist = false;
> + engine->sched_engine->no_priolist = false;
>
> /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> intel_gt_pm_put_async(engine->gt);
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> index 9ef349cd5cea..86b41ddec373 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> @@ -59,6 +59,7 @@ struct drm_i915_reg_table;
> struct i915_gem_context;
> struct i915_request;
> struct i915_sched_attr;
> +struct i915_sched_engine;
> struct intel_gt;
> struct intel_ring;
> struct intel_uncore;
> @@ -152,11 +153,6 @@ struct intel_engine_execlists {
> */
> struct timer_list preempt;
>
> - /**
> - * @default_priolist: priority list for I915_PRIORITY_NORMAL
> - */
> - struct i915_priolist default_priolist;
> -
> /**
> * @ccid: identifier for contexts submitted to this engine
> */
> @@ -191,11 +187,6 @@ struct intel_engine_execlists {
> */
> u32 reset_ccid;
>
> - /**
> - * @no_priolist: priority lists disabled
> - */
> - bool no_priolist;
> -
> /**
> * @submit_reg: gen-specific execlist submission register
> * set to the ExecList Submission Port (elsp) register pre-Gen11 and to
> @@ -238,23 +229,8 @@ struct intel_engine_execlists {
> unsigned int port_mask;
>
> /**
> - * @queue_priority_hint: Highest pending priority.
> - *
> - * When we add requests into the queue, or adjust the priority of
> - * executing requests, we compute the maximum priority of those
> - * pending requests. We can then use this value to determine if
> - * we need to preempt the executing requests to service the queue.
> - * However, since the we may have recorded the priority of an inflight
> - * request we wanted to preempt but since completed, at the time of
> - * dequeuing the priority hint may no longer may match the highest
> - * available request priority.
> + * @virtual: virtual of requests, in priority lists
Having trouble parsing this. Did you mean "queue of virtual
requests"? Or maybe "virtual queue of requests"?
> */
> - int queue_priority_hint;
> -
> - /**
> - * @queue: queue of requests, in priority lists
> - */
> - struct rb_root_cached queue;
> struct rb_root_cached virtual;
>
> /**
> @@ -332,6 +308,8 @@ struct intel_engine_cs {
> struct list_head hold; /* ready requests, but on hold */
> } active;
>
> + struct i915_sched_engine *sched_engine;
> +
> /* keep a request in reserve for a [pm] barrier under oom */
> struct i915_request *request_pool;
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 8db200422950..d1dc1db3e378 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
> return prio;
> }
>
> -static int queue_prio(const struct intel_engine_execlists *execlists)
> +static int queue_prio(const struct i915_sched_engine *sched_engine)
> {
> struct rb_node *rb;
>
> - rb = rb_first_cached(&execlists->queue);
> + rb = rb_first_cached(&sched_engine->queue);
> if (!rb)
> return INT_MIN;
>
> @@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> * to preserve FIFO ordering of dependencies.
> */
> last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
> - if (engine->execlists.queue_priority_hint <= last_prio)
> + if (engine->sched_engine->queue_priority_hint <= last_prio)
> return false;
>
> /*
> @@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> * context, it's priority would not exceed ELSP[0] aka last_prio.
> */
> return max(virtual_prio(&engine->execlists),
> - queue_prio(&engine->execlists)) > last_prio;
> + queue_prio(engine->sched_engine)) > last_prio;
> }
>
> __maybe_unused static bool
> @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> prio = rq_prio(rq);
> pl = i915_sched_lookup_priolist(engine, prio);
> }
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
>
> list_move(&rq->sched.link, pl);
> set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> }
>
> /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
> + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> ENGINE_TRACE(engine, "timeslice required for queue\n");
> return true;
> }
> @@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
> static void execlists_dequeue(struct intel_engine_cs *engine)
> {
> struct intel_engine_execlists * const execlists = &engine->execlists;
> + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_request **port = execlists->pending;
> struct i915_request ** const last_port = port + execlists->port_mask;
> struct i915_request *last, * const *active;
> @@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> last->fence.context,
> last->fence.seqno,
> last->sched.attr.priority,
> - execlists->queue_priority_hint);
> + sched_engine->queue_priority_hint);
> record_preemption(execlists);
>
> /*
> @@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> yesno(timer_expired(&execlists->timer)),
> last->fence.context, last->fence.seqno,
> rq_prio(last),
> - execlists->queue_priority_hint,
> + sched_engine->queue_priority_hint,
> yesno(timeslice_yield(execlists, last)));
>
> /*
> @@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> GEM_BUG_ON(rq->engine != &ve->base);
> GEM_BUG_ON(rq->context != &ve->context);
>
> - if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
> + if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> spin_unlock(&ve->base.active.lock);
> break;
> }
> @@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> yesno(engine != ve->siblings[0]));
>
> WRITE_ONCE(ve->request, NULL);
> - WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
> + WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
>
> rb = &ve->nodes[engine->id].rb;
> rb_erase_cached(rb, &execlists->virtual);
> @@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> break;
> }
>
> - while ((rb = rb_first_cached(&execlists->queue))) {
> + while ((rb = rb_first_cached(&sched_engine->queue))) {
> struct i915_priolist *p = to_priolist(rb);
> struct i915_request *rq, *rn;
>
> @@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> }
> }
>
> - rb_erase_cached(&p->node, &execlists->queue);
> + rb_erase_cached(&p->node, &sched_engine->queue);
> i915_priolist_free(p);
> }
> done:
> @@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> * request triggering preemption on the next dequeue (or subsequent
> * interrupt for secondary ports).
> */
> - execlists->queue_priority_hint = queue_prio(execlists);
> + sched_engine->queue_priority_hint = queue_prio(sched_engine);
> spin_unlock(&engine->active.lock);
>
> /*
> @@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> */
> __execlists_unhold(rq);
>
> - if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
> - engine->execlists.queue_priority_hint = rq_prio(rq);
> + if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> + engine->sched_engine->queue_priority_hint = rq_prio(rq);
> tasklet_hi_schedule(&engine->execlists.tasklet);
> }
>
> @@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
> static bool submit_queue(struct intel_engine_cs *engine,
> const struct i915_request *rq)
> {
> - struct intel_engine_execlists *execlists = &engine->execlists;
> + struct i915_sched_engine *sched_engine = engine->sched_engine;
>
> - if (rq_prio(rq) <= execlists->queue_priority_hint)
> + if (rq_prio(rq) <= sched_engine->queue_priority_hint)
> return false;
>
> - execlists->queue_priority_hint = rq_prio(rq);
> + sched_engine->queue_priority_hint = rq_prio(rq);
> return true;
> }
>
> @@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> } else {
> queue_request(engine, request);
>
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> GEM_BUG_ON(list_empty(&request->sched.link));
>
> if (submit_queue(engine, request))
> @@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
> from_tasklet(engine, t, execlists.tasklet);
>
> /* The driver is wedged; don't process any more events. */
> - WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
> + WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> }
>
> static void execlists_reset_cancel(struct intel_engine_cs *engine)
> {
> struct intel_engine_execlists * const execlists = &engine->execlists;
> + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_request *rq, *rn;
> struct rb_node *rb;
> unsigned long flags;
> @@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> intel_engine_signal_breadcrumbs(engine);
>
> /* Flush the queued requests to the timeline list (for retiring). */
> - while ((rb = rb_first_cached(&execlists->queue))) {
> + while ((rb = rb_first_cached(&sched_engine->queue))) {
> struct i915_priolist *p = to_priolist(rb);
>
> priolist_for_each_request_consume(rq, rn, p) {
> @@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> }
> }
>
> - rb_erase_cached(&p->node, &execlists->queue);
> + rb_erase_cached(&p->node, &sched_engine->queue);
> i915_priolist_free(p);
> }
>
> @@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> }
> i915_request_put(rq);
>
> - ve->base.execlists.queue_priority_hint = INT_MIN;
> + ve->base.sched_engine->queue_priority_hint = INT_MIN;
> }
> spin_unlock(&ve->base.active.lock);
> }
>
> /* Remaining _unready_ requests will be nop'ed when submitted */
>
> - execlists->queue_priority_hint = INT_MIN;
> - execlists->queue = RB_ROOT_CACHED;
> + sched_engine->queue_priority_hint = INT_MIN;
> + sched_engine->queue = RB_ROOT_CACHED;
>
> GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> execlists->tasklet.callback = nop_submission_tasklet;
> @@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
>
> static struct list_head *virtual_queue(struct virtual_engine *ve)
> {
> - return &ve->base.execlists.default_priolist.requests;
> + return &ve->base.sched_engine->default_priolist.requests;
> }
>
> static void rcu_virtual_context_destroy(struct work_struct *wrk)
> @@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> lrc_fini(&ve->context);
> intel_context_fini(&ve->context);
>
> - intel_breadcrumbs_free(ve->base.breadcrumbs);
> + if (ve->base.breadcrumbs)
> + intel_breadcrumbs_free(ve->base.breadcrumbs);
> + if (ve->base.sched_engine)
> + i915_sched_engine_put(ve->base.sched_engine);
Will either of these ever actually be NULL? The first line or two
seems to indicate that breadcrumbs never is.
> intel_engine_free_request_pool(&ve->base);
>
> kfree(ve->bonds);
> @@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
>
> ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
> rq->fence.context, rq->fence.seqno,
> - mask, ve->base.execlists.queue_priority_hint);
> + mask, ve->base.sched_engine->queue_priority_hint);
>
> return mask;
> }
> @@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> {
> struct virtual_engine * const ve =
> from_tasklet(ve, t, base.execlists.tasklet);
> - const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
> + const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> intel_engine_mask_t mask;
> unsigned int n;
>
> @@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> submit_engine:
> GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> node->prio = prio;
> - if (first && prio > sibling->execlists.queue_priority_hint)
> + if (first && prio > sibling->sched_engine->queue_priority_hint)
> tasklet_hi_schedule(&sibling->execlists.tasklet);
>
> unlock_engine:
> @@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
> i915_request_put(ve->request);
> }
>
> - ve->base.execlists.queue_priority_hint = rq_prio(rq);
> + ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
> ve->request = i915_request_get(rq);
>
> GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> @@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> intel_engine_init_execlists(&ve->base);
>
> + ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> + if (!ve->base.sched_engine) {
> + err = -ENOMEM;
> + goto err_put;
> + }
> +
> ve->base.cops = &virtual_context_ops;
> ve->base.request_alloc = execlists_request_alloc;
>
> @@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> ve->base.bond_execute = virtual_bond_execute;
>
> INIT_LIST_HEAD(virtual_queue(ve));
> - ve->base.execlists.queue_priority_hint = INT_MIN;
> tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
>
> intel_context_init(&ve->context, &ve->base);
> @@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> unsigned int max)
> {
> const struct intel_engine_execlists *execlists = &engine->execlists;
> + const struct i915_sched_engine *sched_engine = engine->sched_engine;
> struct i915_request *rq, *last;
> unsigned long flags;
> unsigned int count;
> @@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> show_request(m, last, "\t\t", 0);
> }
>
> - if (execlists->queue_priority_hint != INT_MIN)
> + if (sched_engine->queue_priority_hint != INT_MIN)
> drm_printf(m, "\t\tQueue priority hint: %d\n",
> - READ_ONCE(execlists->queue_priority_hint));
> + READ_ONCE(sched_engine->queue_priority_hint));
>
> last = NULL;
> count = 0;
> - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
>
> priolist_for_each_request(rq, p) {
> diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> index 32589c6625e1..b1fdba13e900 100644
> --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> @@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
>
> GEM_BUG_ON(timer_pending(&mock->hw_delay));
>
> + i915_sched_engine_put(engine->sched_engine);
> intel_breadcrumbs_free(engine->breadcrumbs);
>
> intel_context_unpin(engine->kernel_context);
> @@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
> {
> struct intel_context *ce;
>
> + engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> + if (!engine->sched_engine)
> + return -ENOMEM;
> +
> intel_engine_init_active(engine, ENGINE_MOCK);
> intel_engine_init_execlists(engine);
> intel_engine_init__pm(engine);
> @@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
>
> engine->breadcrumbs = intel_breadcrumbs_create(NULL);
> if (!engine->breadcrumbs)
> - return -ENOMEM;
> + goto err_schedule;
>
> ce = create_kernel_context(engine);
> if (IS_ERR(ce))
> @@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
>
> err_breadcrumbs:
> intel_breadcrumbs_free(engine->breadcrumbs);
> +err_schedule:
> + i915_sched_engine_put(engine->sched_engine);
> return -ENOMEM;
> }
>
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 335719f17490..d42dea79ee64 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
> static void __guc_dequeue(struct intel_engine_cs *engine)
> {
> struct intel_engine_execlists * const execlists = &engine->execlists;
> + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_request **first = execlists->inflight;
> struct i915_request ** const last_port = first + execlists->port_mask;
> struct i915_request *last = first[0];
> @@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> * event.
> */
> port = first;
> - while ((rb = rb_first_cached(&execlists->queue))) {
> + while ((rb = rb_first_cached(&sched_engine->queue))) {
> struct i915_priolist *p = to_priolist(rb);
> struct i915_request *rq, *rn;
>
> @@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> last = rq;
> }
>
> - rb_erase_cached(&p->node, &execlists->queue);
> + rb_erase_cached(&p->node, &sched_engine->queue);
> i915_priolist_free(p);
> }
> done:
> - execlists->queue_priority_hint =
> + sched_engine->queue_priority_hint =
> rb ? to_priolist(rb)->priority : INT_MIN;
> if (submit) {
> *port = schedule_in(last, port - execlists->inflight);
> @@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
>
> static void guc_reset_cancel(struct intel_engine_cs *engine)
> {
> - struct intel_engine_execlists * const execlists = &engine->execlists;
> + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_request *rq, *rn;
> struct rb_node *rb;
> unsigned long flags;
> @@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> }
>
> /* Flush the queued requests to the timeline list (for retiring). */
> - while ((rb = rb_first_cached(&execlists->queue))) {
> + while ((rb = rb_first_cached(&sched_engine->queue))) {
> struct i915_priolist *p = to_priolist(rb);
>
> priolist_for_each_request_consume(rq, rn, p) {
> @@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> i915_request_mark_complete(rq);
> }
>
> - rb_erase_cached(&p->node, &execlists->queue);
> + rb_erase_cached(&p->node, &sched_engine->queue);
> i915_priolist_free(p);
> }
>
> /* Remaining _unready_ requests will be nop'ed when submitted */
>
> - execlists->queue_priority_hint = INT_MIN;
> - execlists->queue = RB_ROOT_CACHED;
> + sched_engine->queue_priority_hint = INT_MIN;
> + sched_engine->queue = RB_ROOT_CACHED;
>
> spin_unlock_irqrestore(&engine->active.lock, flags);
> }
> @@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
>
> queue_request(engine, rq, rq_prio(rq));
>
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> GEM_BUG_ON(list_empty(&rq->sched.link));
>
> tasklet_hi_schedule(&engine->execlists.tasklet);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index efa638c3acc7..4953874a9ba6 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> return rb_entry(rb, struct i915_priolist, node);
> }
>
> -static void assert_priolists(struct intel_engine_execlists * const execlists)
> +static void assert_priolists(struct i915_sched_engine * const sched_engine)
> {
> struct rb_node *rb;
> long last_prio;
> @@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> return;
>
> - GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
> - rb_first(&execlists->queue.rb_root));
> + GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
> + rb_first(&sched_engine->queue.rb_root));
>
> last_prio = INT_MAX;
> - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> const struct i915_priolist *p = to_priolist(rb);
>
> GEM_BUG_ON(p->priority > last_prio);
> @@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> struct list_head *
> i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> {
> - struct intel_engine_execlists * const execlists = &engine->execlists;
> + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_priolist *p;
> struct rb_node **parent, *rb;
> bool first = true;
>
> lockdep_assert_held(&engine->active.lock);
> - assert_priolists(execlists);
> + assert_priolists(sched_engine);
>
> - if (unlikely(execlists->no_priolist))
> + if (unlikely(sched_engine->no_priolist))
> prio = I915_PRIORITY_NORMAL;
>
> find_priolist:
> /* most positive priority is scheduled first, equal priorities fifo */
> rb = NULL;
> - parent = &execlists->queue.rb_root.rb_node;
> + parent = &sched_engine->queue.rb_root.rb_node;
> while (*parent) {
> rb = *parent;
> p = to_priolist(rb);
> @@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> }
>
> if (prio == I915_PRIORITY_NORMAL) {
> - p = &execlists->default_priolist;
> + p = &sched_engine->default_priolist;
> } else {
> p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
> /* Convert an allocation failure to a priority bump */
> @@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> * requests, so if userspace lied about their
> * dependencies that reordering may be visible.
> */
> - execlists->no_priolist = true;
> + sched_engine->no_priolist = true;
> goto find_priolist;
> }
> }
> @@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> INIT_LIST_HEAD(&p->requests);
>
> rb_link_node(&p->node, rb, parent);
> - rb_insert_color_cached(&p->node, &execlists->queue, first);
> + rb_insert_color_cached(&p->node, &sched_engine->queue, first);
>
> return &p->requests;
> }
> @@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> * We only need to kick the tasklet once for the high priority
> * new context we add into the queue.
> */
> - if (prio <= engine->execlists.queue_priority_hint)
> + if (prio <= engine->sched_engine->queue_priority_hint)
> return;
>
> rcu_read_lock();
> @@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> inflight->fence.context, inflight->fence.seqno,
> inflight->sched.attr.priority);
>
> - engine->execlists.queue_priority_hint = prio;
> + engine->sched_engine->queue_priority_hint = prio;
> if (need_preempt(prio, rq_prio(inflight)))
> tasklet_hi_schedule(&engine->execlists.tasklet);
>
> @@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
> rcu_read_unlock();
> }
>
> +void i915_sched_engine_free(struct kref *kref)
> +{
> + struct i915_sched_engine *sched_engine =
> + container_of(kref, typeof(*sched_engine), ref);
> +
> + kfree(sched_engine);
> +}
> +
> +struct i915_sched_engine *
> +i915_sched_engine_create(unsigned int subclass)
> +{
> + struct i915_sched_engine *sched_engine;
> +
> + sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
> + if (!sched_engine)
> + return NULL;
> +
> + kref_init(&sched_engine->ref);
> +
> + sched_engine->queue = RB_ROOT_CACHED;
> + sched_engine->queue_priority_hint = INT_MIN;
> +
> + return sched_engine;
> +}
> +
> static void i915_global_scheduler_shrink(void)
> {
> kmem_cache_shrink(global.slab_dependencies);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 858a0938f47a..91a04e34cac5 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
> __i915_priolist_free(p);
> }
>
> +struct i915_sched_engine *
> +i915_sched_engine_create(unsigned int subclass);
> +
> +void i915_sched_engine_free(struct kref *kref);
> +
> +static inline struct i915_sched_engine *
> +i915_sched_engine_get(struct i915_sched_engine *sched_engine)
> +{
> + kref_get(&sched_engine->ref);
> + return sched_engine;
> +}
> +
> +static inline void
> +i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> +{
> + kref_put(&sched_engine->ref, i915_sched_engine_free);
> +}
> +
> void i915_request_show_with_schedule(struct drm_printer *m,
> const struct i915_request *rq,
> const char *prefix,
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 343ed44d5ed4..16a054e67760 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
Dumb question for which I'm not really in the position of having an
opinion: Should this go in its own file?
Patch seems mostly mechanically correct. Not sure where this is going
yet, though. I'll keep reading. :-)
--Jason
> @@ -91,4 +91,37 @@ struct i915_dependency {
> &(rq__)->sched.signalers_list, \
> signal_link)
>
> +struct i915_sched_engine {
> + struct kref ref;
> +
> + /**
> + * @default_priolist: priority list for I915_PRIORITY_NORMAL
> + */
> + struct i915_priolist default_priolist;
> +
> + /**
> + * @queue_priority_hint: Highest pending priority.
> + *
> + * When we add requests into the queue, or adjust the priority of
> + * executing requests, we compute the maximum priority of those
> + * pending requests. We can then use this value to determine if
> + * we need to preempt the executing requests to service the queue.
> + * However, since the we may have recorded the priority of an inflight
> + * request we wanted to preempt but since completed, at the time of
> + * dequeuing the priority hint may no longer may match the highest
> + * available request priority.
> + */
> + int queue_priority_hint;
> +
> + /**
> + * @queue: queue of requests, in priority lists
> + */
> + struct rb_root_cached queue;
> +
> + /**
> + * @no_priolist: priority lists disabled
> + */
> + bool no_priolist;
> +};
> +
> #endif /* _I915_SCHEDULER_TYPES_H_ */
> --
> 2.28.0
>
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
@ 2021-06-04 17:38 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 17:38 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Introduce i915_sched_engine object which is lower level data structure
> that i915_scheduler / generic code can operate on without touching
> execlist specific structures. This allows additional submission backends
> to be added without breaking the layering.
>
> This is a bit of detour to integrating the i915 with the DRM scheduler
> but this object will still exist when the DRM scheduler lands in the
> i915. It will however look a bit different. It will encapsulate the
> drm_gpu_scheduler object plus and common variables (to the backends)
> related to scheduling. Regardless this is a step in the right direction.
>
> This patch starts the aforementioned transition by moving the the
> priolist into the i915_sched_engine object.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
> drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
> drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
> .../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
> drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
> drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
> drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
> drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
> 9 files changed, 169 insertions(+), 90 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index 3f9a811eb02b..d0f3814440f6 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
> memset(execlists->pending, 0, sizeof(execlists->pending));
> execlists->active =
> memset(execlists->inflight, 0, sizeof(execlists->inflight));
> -
> - execlists->queue_priority_hint = INT_MIN;
> - execlists->queue = RB_ROOT_CACHED;
> }
>
> static void cleanup_status_page(struct intel_engine_cs *engine)
> @@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> goto err_status;
> }
>
> + engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
> + if (!engine->sched_engine) {
> + err = -ENOMEM;
> + goto err_sched_engine;
> + }
> +
> err = intel_engine_init_cmd_parser(engine);
> if (err)
> goto err_cmd_parser;
> @@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> return 0;
>
> err_cmd_parser:
> + i915_sched_engine_put(engine->sched_engine);
> +err_sched_engine:
> intel_breadcrumbs_free(engine->breadcrumbs);
> err_status:
> cleanup_status_page(engine);
> @@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> GEM_BUG_ON(!list_empty(&engine->active.requests));
> tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
>
> + i915_sched_engine_put(engine->sched_engine);
> intel_breadcrumbs_free(engine->breadcrumbs);
>
> intel_engine_fini_retire(engine);
> @@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> intel_engine_flush_submission(engine);
>
> /* ELSP is empty, but there are ready requests? E.g. after reset */
> - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
> + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> return false;
>
> /* Ring stopped? */
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> index 47f4397095e5..b6a00dd72808 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> @@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
> intel_breadcrumbs_park(engine->breadcrumbs);
>
> /* Must be reset upon idling, or we may miss the busy wakeup. */
> - GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
> + GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
>
> if (engine->park)
> engine->park(engine);
>
> - engine->execlists.no_priolist = false;
> + engine->sched_engine->no_priolist = false;
>
> /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> intel_gt_pm_put_async(engine->gt);
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> index 9ef349cd5cea..86b41ddec373 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> @@ -59,6 +59,7 @@ struct drm_i915_reg_table;
> struct i915_gem_context;
> struct i915_request;
> struct i915_sched_attr;
> +struct i915_sched_engine;
> struct intel_gt;
> struct intel_ring;
> struct intel_uncore;
> @@ -152,11 +153,6 @@ struct intel_engine_execlists {
> */
> struct timer_list preempt;
>
> - /**
> - * @default_priolist: priority list for I915_PRIORITY_NORMAL
> - */
> - struct i915_priolist default_priolist;
> -
> /**
> * @ccid: identifier for contexts submitted to this engine
> */
> @@ -191,11 +187,6 @@ struct intel_engine_execlists {
> */
> u32 reset_ccid;
>
> - /**
> - * @no_priolist: priority lists disabled
> - */
> - bool no_priolist;
> -
> /**
> * @submit_reg: gen-specific execlist submission register
> * set to the ExecList Submission Port (elsp) register pre-Gen11 and to
> @@ -238,23 +229,8 @@ struct intel_engine_execlists {
> unsigned int port_mask;
>
> /**
> - * @queue_priority_hint: Highest pending priority.
> - *
> - * When we add requests into the queue, or adjust the priority of
> - * executing requests, we compute the maximum priority of those
> - * pending requests. We can then use this value to determine if
> - * we need to preempt the executing requests to service the queue.
> - * However, since the we may have recorded the priority of an inflight
> - * request we wanted to preempt but since completed, at the time of
> - * dequeuing the priority hint may no longer may match the highest
> - * available request priority.
> + * @virtual: virtual of requests, in priority lists
Having trouble parsing this. Did you mean "queue of virtual
requests"? Or maybe "virtual queue of requests"?
> */
> - int queue_priority_hint;
> -
> - /**
> - * @queue: queue of requests, in priority lists
> - */
> - struct rb_root_cached queue;
> struct rb_root_cached virtual;
>
> /**
> @@ -332,6 +308,8 @@ struct intel_engine_cs {
> struct list_head hold; /* ready requests, but on hold */
> } active;
>
> + struct i915_sched_engine *sched_engine;
> +
> /* keep a request in reserve for a [pm] barrier under oom */
> struct i915_request *request_pool;
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 8db200422950..d1dc1db3e378 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
> return prio;
> }
>
> -static int queue_prio(const struct intel_engine_execlists *execlists)
> +static int queue_prio(const struct i915_sched_engine *sched_engine)
> {
> struct rb_node *rb;
>
> - rb = rb_first_cached(&execlists->queue);
> + rb = rb_first_cached(&sched_engine->queue);
> if (!rb)
> return INT_MIN;
>
> @@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> * to preserve FIFO ordering of dependencies.
> */
> last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
> - if (engine->execlists.queue_priority_hint <= last_prio)
> + if (engine->sched_engine->queue_priority_hint <= last_prio)
> return false;
>
> /*
> @@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> * context, it's priority would not exceed ELSP[0] aka last_prio.
> */
> return max(virtual_prio(&engine->execlists),
> - queue_prio(&engine->execlists)) > last_prio;
> + queue_prio(engine->sched_engine)) > last_prio;
> }
>
> __maybe_unused static bool
> @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> prio = rq_prio(rq);
> pl = i915_sched_lookup_priolist(engine, prio);
> }
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
>
> list_move(&rq->sched.link, pl);
> set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> }
>
> /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
> + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> ENGINE_TRACE(engine, "timeslice required for queue\n");
> return true;
> }
> @@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
> static void execlists_dequeue(struct intel_engine_cs *engine)
> {
> struct intel_engine_execlists * const execlists = &engine->execlists;
> + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_request **port = execlists->pending;
> struct i915_request ** const last_port = port + execlists->port_mask;
> struct i915_request *last, * const *active;
> @@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> last->fence.context,
> last->fence.seqno,
> last->sched.attr.priority,
> - execlists->queue_priority_hint);
> + sched_engine->queue_priority_hint);
> record_preemption(execlists);
>
> /*
> @@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> yesno(timer_expired(&execlists->timer)),
> last->fence.context, last->fence.seqno,
> rq_prio(last),
> - execlists->queue_priority_hint,
> + sched_engine->queue_priority_hint,
> yesno(timeslice_yield(execlists, last)));
>
> /*
> @@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> GEM_BUG_ON(rq->engine != &ve->base);
> GEM_BUG_ON(rq->context != &ve->context);
>
> - if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
> + if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> spin_unlock(&ve->base.active.lock);
> break;
> }
> @@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> yesno(engine != ve->siblings[0]));
>
> WRITE_ONCE(ve->request, NULL);
> - WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
> + WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
>
> rb = &ve->nodes[engine->id].rb;
> rb_erase_cached(rb, &execlists->virtual);
> @@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> break;
> }
>
> - while ((rb = rb_first_cached(&execlists->queue))) {
> + while ((rb = rb_first_cached(&sched_engine->queue))) {
> struct i915_priolist *p = to_priolist(rb);
> struct i915_request *rq, *rn;
>
> @@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> }
> }
>
> - rb_erase_cached(&p->node, &execlists->queue);
> + rb_erase_cached(&p->node, &sched_engine->queue);
> i915_priolist_free(p);
> }
> done:
> @@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> * request triggering preemption on the next dequeue (or subsequent
> * interrupt for secondary ports).
> */
> - execlists->queue_priority_hint = queue_prio(execlists);
> + sched_engine->queue_priority_hint = queue_prio(sched_engine);
> spin_unlock(&engine->active.lock);
>
> /*
> @@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> */
> __execlists_unhold(rq);
>
> - if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
> - engine->execlists.queue_priority_hint = rq_prio(rq);
> + if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> + engine->sched_engine->queue_priority_hint = rq_prio(rq);
> tasklet_hi_schedule(&engine->execlists.tasklet);
> }
>
> @@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
> static bool submit_queue(struct intel_engine_cs *engine,
> const struct i915_request *rq)
> {
> - struct intel_engine_execlists *execlists = &engine->execlists;
> + struct i915_sched_engine *sched_engine = engine->sched_engine;
>
> - if (rq_prio(rq) <= execlists->queue_priority_hint)
> + if (rq_prio(rq) <= sched_engine->queue_priority_hint)
> return false;
>
> - execlists->queue_priority_hint = rq_prio(rq);
> + sched_engine->queue_priority_hint = rq_prio(rq);
> return true;
> }
>
> @@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> } else {
> queue_request(engine, request);
>
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> GEM_BUG_ON(list_empty(&request->sched.link));
>
> if (submit_queue(engine, request))
> @@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
> from_tasklet(engine, t, execlists.tasklet);
>
> /* The driver is wedged; don't process any more events. */
> - WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
> + WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> }
>
> static void execlists_reset_cancel(struct intel_engine_cs *engine)
> {
> struct intel_engine_execlists * const execlists = &engine->execlists;
> + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_request *rq, *rn;
> struct rb_node *rb;
> unsigned long flags;
> @@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> intel_engine_signal_breadcrumbs(engine);
>
> /* Flush the queued requests to the timeline list (for retiring). */
> - while ((rb = rb_first_cached(&execlists->queue))) {
> + while ((rb = rb_first_cached(&sched_engine->queue))) {
> struct i915_priolist *p = to_priolist(rb);
>
> priolist_for_each_request_consume(rq, rn, p) {
> @@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> }
> }
>
> - rb_erase_cached(&p->node, &execlists->queue);
> + rb_erase_cached(&p->node, &sched_engine->queue);
> i915_priolist_free(p);
> }
>
> @@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> }
> i915_request_put(rq);
>
> - ve->base.execlists.queue_priority_hint = INT_MIN;
> + ve->base.sched_engine->queue_priority_hint = INT_MIN;
> }
> spin_unlock(&ve->base.active.lock);
> }
>
> /* Remaining _unready_ requests will be nop'ed when submitted */
>
> - execlists->queue_priority_hint = INT_MIN;
> - execlists->queue = RB_ROOT_CACHED;
> + sched_engine->queue_priority_hint = INT_MIN;
> + sched_engine->queue = RB_ROOT_CACHED;
>
> GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> execlists->tasklet.callback = nop_submission_tasklet;
> @@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
>
> static struct list_head *virtual_queue(struct virtual_engine *ve)
> {
> - return &ve->base.execlists.default_priolist.requests;
> + return &ve->base.sched_engine->default_priolist.requests;
> }
>
> static void rcu_virtual_context_destroy(struct work_struct *wrk)
> @@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> lrc_fini(&ve->context);
> intel_context_fini(&ve->context);
>
> - intel_breadcrumbs_free(ve->base.breadcrumbs);
> + if (ve->base.breadcrumbs)
> + intel_breadcrumbs_free(ve->base.breadcrumbs);
> + if (ve->base.sched_engine)
> + i915_sched_engine_put(ve->base.sched_engine);
Will either of these ever actually be NULL? The first line or two
seems to indicate that breadcrumbs never is.
> intel_engine_free_request_pool(&ve->base);
>
> kfree(ve->bonds);
> @@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
>
> ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
> rq->fence.context, rq->fence.seqno,
> - mask, ve->base.execlists.queue_priority_hint);
> + mask, ve->base.sched_engine->queue_priority_hint);
>
> return mask;
> }
> @@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> {
> struct virtual_engine * const ve =
> from_tasklet(ve, t, base.execlists.tasklet);
> - const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
> + const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> intel_engine_mask_t mask;
> unsigned int n;
>
> @@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> submit_engine:
> GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> node->prio = prio;
> - if (first && prio > sibling->execlists.queue_priority_hint)
> + if (first && prio > sibling->sched_engine->queue_priority_hint)
> tasklet_hi_schedule(&sibling->execlists.tasklet);
>
> unlock_engine:
> @@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
> i915_request_put(ve->request);
> }
>
> - ve->base.execlists.queue_priority_hint = rq_prio(rq);
> + ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
> ve->request = i915_request_get(rq);
>
> GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> @@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> intel_engine_init_execlists(&ve->base);
>
> + ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> + if (!ve->base.sched_engine) {
> + err = -ENOMEM;
> + goto err_put;
> + }
> +
> ve->base.cops = &virtual_context_ops;
> ve->base.request_alloc = execlists_request_alloc;
>
> @@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> ve->base.bond_execute = virtual_bond_execute;
>
> INIT_LIST_HEAD(virtual_queue(ve));
> - ve->base.execlists.queue_priority_hint = INT_MIN;
> tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
>
> intel_context_init(&ve->context, &ve->base);
> @@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> unsigned int max)
> {
> const struct intel_engine_execlists *execlists = &engine->execlists;
> + const struct i915_sched_engine *sched_engine = engine->sched_engine;
> struct i915_request *rq, *last;
> unsigned long flags;
> unsigned int count;
> @@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> show_request(m, last, "\t\t", 0);
> }
>
> - if (execlists->queue_priority_hint != INT_MIN)
> + if (sched_engine->queue_priority_hint != INT_MIN)
> drm_printf(m, "\t\tQueue priority hint: %d\n",
> - READ_ONCE(execlists->queue_priority_hint));
> + READ_ONCE(sched_engine->queue_priority_hint));
>
> last = NULL;
> count = 0;
> - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
>
> priolist_for_each_request(rq, p) {
> diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> index 32589c6625e1..b1fdba13e900 100644
> --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> @@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
>
> GEM_BUG_ON(timer_pending(&mock->hw_delay));
>
> + i915_sched_engine_put(engine->sched_engine);
> intel_breadcrumbs_free(engine->breadcrumbs);
>
> intel_context_unpin(engine->kernel_context);
> @@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
> {
> struct intel_context *ce;
>
> + engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> + if (!engine->sched_engine)
> + return -ENOMEM;
> +
> intel_engine_init_active(engine, ENGINE_MOCK);
> intel_engine_init_execlists(engine);
> intel_engine_init__pm(engine);
> @@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
>
> engine->breadcrumbs = intel_breadcrumbs_create(NULL);
> if (!engine->breadcrumbs)
> - return -ENOMEM;
> + goto err_schedule;
>
> ce = create_kernel_context(engine);
> if (IS_ERR(ce))
> @@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
>
> err_breadcrumbs:
> intel_breadcrumbs_free(engine->breadcrumbs);
> +err_schedule:
> + i915_sched_engine_put(engine->sched_engine);
> return -ENOMEM;
> }
>
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 335719f17490..d42dea79ee64 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
> static void __guc_dequeue(struct intel_engine_cs *engine)
> {
> struct intel_engine_execlists * const execlists = &engine->execlists;
> + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_request **first = execlists->inflight;
> struct i915_request ** const last_port = first + execlists->port_mask;
> struct i915_request *last = first[0];
> @@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> * event.
> */
> port = first;
> - while ((rb = rb_first_cached(&execlists->queue))) {
> + while ((rb = rb_first_cached(&sched_engine->queue))) {
> struct i915_priolist *p = to_priolist(rb);
> struct i915_request *rq, *rn;
>
> @@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> last = rq;
> }
>
> - rb_erase_cached(&p->node, &execlists->queue);
> + rb_erase_cached(&p->node, &sched_engine->queue);
> i915_priolist_free(p);
> }
> done:
> - execlists->queue_priority_hint =
> + sched_engine->queue_priority_hint =
> rb ? to_priolist(rb)->priority : INT_MIN;
> if (submit) {
> *port = schedule_in(last, port - execlists->inflight);
> @@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
>
> static void guc_reset_cancel(struct intel_engine_cs *engine)
> {
> - struct intel_engine_execlists * const execlists = &engine->execlists;
> + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_request *rq, *rn;
> struct rb_node *rb;
> unsigned long flags;
> @@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> }
>
> /* Flush the queued requests to the timeline list (for retiring). */
> - while ((rb = rb_first_cached(&execlists->queue))) {
> + while ((rb = rb_first_cached(&sched_engine->queue))) {
> struct i915_priolist *p = to_priolist(rb);
>
> priolist_for_each_request_consume(rq, rn, p) {
> @@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> i915_request_mark_complete(rq);
> }
>
> - rb_erase_cached(&p->node, &execlists->queue);
> + rb_erase_cached(&p->node, &sched_engine->queue);
> i915_priolist_free(p);
> }
>
> /* Remaining _unready_ requests will be nop'ed when submitted */
>
> - execlists->queue_priority_hint = INT_MIN;
> - execlists->queue = RB_ROOT_CACHED;
> + sched_engine->queue_priority_hint = INT_MIN;
> + sched_engine->queue = RB_ROOT_CACHED;
>
> spin_unlock_irqrestore(&engine->active.lock, flags);
> }
> @@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
>
> queue_request(engine, rq, rq_prio(rq));
>
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> GEM_BUG_ON(list_empty(&rq->sched.link));
>
> tasklet_hi_schedule(&engine->execlists.tasklet);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index efa638c3acc7..4953874a9ba6 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> return rb_entry(rb, struct i915_priolist, node);
> }
>
> -static void assert_priolists(struct intel_engine_execlists * const execlists)
> +static void assert_priolists(struct i915_sched_engine * const sched_engine)
> {
> struct rb_node *rb;
> long last_prio;
> @@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> return;
>
> - GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
> - rb_first(&execlists->queue.rb_root));
> + GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
> + rb_first(&sched_engine->queue.rb_root));
>
> last_prio = INT_MAX;
> - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> const struct i915_priolist *p = to_priolist(rb);
>
> GEM_BUG_ON(p->priority > last_prio);
> @@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> struct list_head *
> i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> {
> - struct intel_engine_execlists * const execlists = &engine->execlists;
> + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_priolist *p;
> struct rb_node **parent, *rb;
> bool first = true;
>
> lockdep_assert_held(&engine->active.lock);
> - assert_priolists(execlists);
> + assert_priolists(sched_engine);
>
> - if (unlikely(execlists->no_priolist))
> + if (unlikely(sched_engine->no_priolist))
> prio = I915_PRIORITY_NORMAL;
>
> find_priolist:
> /* most positive priority is scheduled first, equal priorities fifo */
> rb = NULL;
> - parent = &execlists->queue.rb_root.rb_node;
> + parent = &sched_engine->queue.rb_root.rb_node;
> while (*parent) {
> rb = *parent;
> p = to_priolist(rb);
> @@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> }
>
> if (prio == I915_PRIORITY_NORMAL) {
> - p = &execlists->default_priolist;
> + p = &sched_engine->default_priolist;
> } else {
> p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
> /* Convert an allocation failure to a priority bump */
> @@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> * requests, so if userspace lied about their
> * dependencies that reordering may be visible.
> */
> - execlists->no_priolist = true;
> + sched_engine->no_priolist = true;
> goto find_priolist;
> }
> }
> @@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> INIT_LIST_HEAD(&p->requests);
>
> rb_link_node(&p->node, rb, parent);
> - rb_insert_color_cached(&p->node, &execlists->queue, first);
> + rb_insert_color_cached(&p->node, &sched_engine->queue, first);
>
> return &p->requests;
> }
> @@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> * We only need to kick the tasklet once for the high priority
> * new context we add into the queue.
> */
> - if (prio <= engine->execlists.queue_priority_hint)
> + if (prio <= engine->sched_engine->queue_priority_hint)
> return;
>
> rcu_read_lock();
> @@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> inflight->fence.context, inflight->fence.seqno,
> inflight->sched.attr.priority);
>
> - engine->execlists.queue_priority_hint = prio;
> + engine->sched_engine->queue_priority_hint = prio;
> if (need_preempt(prio, rq_prio(inflight)))
> tasklet_hi_schedule(&engine->execlists.tasklet);
>
> @@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
> rcu_read_unlock();
> }
>
> +void i915_sched_engine_free(struct kref *kref)
> +{
> + struct i915_sched_engine *sched_engine =
> + container_of(kref, typeof(*sched_engine), ref);
> +
> + kfree(sched_engine);
> +}
> +
> +struct i915_sched_engine *
> +i915_sched_engine_create(unsigned int subclass)
> +{
> + struct i915_sched_engine *sched_engine;
> +
> + sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
> + if (!sched_engine)
> + return NULL;
> +
> + kref_init(&sched_engine->ref);
> +
> + sched_engine->queue = RB_ROOT_CACHED;
> + sched_engine->queue_priority_hint = INT_MIN;
> +
> + return sched_engine;
> +}
> +
> static void i915_global_scheduler_shrink(void)
> {
> kmem_cache_shrink(global.slab_dependencies);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 858a0938f47a..91a04e34cac5 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
> __i915_priolist_free(p);
> }
>
> +struct i915_sched_engine *
> +i915_sched_engine_create(unsigned int subclass);
> +
> +void i915_sched_engine_free(struct kref *kref);
> +
> +static inline struct i915_sched_engine *
> +i915_sched_engine_get(struct i915_sched_engine *sched_engine)
> +{
> + kref_get(&sched_engine->ref);
> + return sched_engine;
> +}
> +
> +static inline void
> +i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> +{
> + kref_put(&sched_engine->ref, i915_sched_engine_free);
> +}
> +
> void i915_request_show_with_schedule(struct drm_printer *m,
> const struct i915_request *rq,
> const char *prefix,
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 343ed44d5ed4..16a054e67760 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
Dumb question for which I'm not really in the position of having an
opinion: Should this go in its own file?
Patch seems mostly mechanically correct. Not sure where this is going
yet, though. I'll keep reading. :-)
--Jason
> @@ -91,4 +91,37 @@ struct i915_dependency {
> &(rq__)->sched.signalers_list, \
> signal_link)
>
> +struct i915_sched_engine {
> + struct kref ref;
> +
> + /**
> + * @default_priolist: priority list for I915_PRIORITY_NORMAL
> + */
> + struct i915_priolist default_priolist;
> +
> + /**
> + * @queue_priority_hint: Highest pending priority.
> + *
> + * When we add requests into the queue, or adjust the priority of
> + * executing requests, we compute the maximum priority of those
> + * pending requests. We can then use this value to determine if
> + * we need to preempt the executing requests to service the queue.
> + * However, since the we may have recorded the priority of an inflight
> + * request we wanted to preempt but since completed, at the time of
> + * dequeuing the priority hint may no longer may match the highest
> + * available request priority.
> + */
> + int queue_priority_hint;
> +
> + /**
> + * @queue: queue of requests, in priority lists
> + */
> + struct rb_root_cached queue;
> +
> + /**
> + * @no_priolist: priority lists disabled
> + */
> + bool no_priolist;
> +};
> +
> #endif /* _I915_SCHEDULER_TYPES_H_ */
> --
> 2.28.0
>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 2/9] drm/i915: Add i915_sched_engine_is_empty function
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 17:50 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 17:50 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Add wrapper function around RB tree to determine if i915_sched_engine is
> empty.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +-
> drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 6 +++---
> drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
> drivers/gpu/drm/i915/i915_scheduler.h | 6 ++++++
> 4 files changed, 11 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index d0f3814440f6..85f2effe9dc6 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -1287,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> intel_engine_flush_submission(engine);
>
> /* ELSP is empty, but there are ready requests? E.g. after reset */
> - if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> + if (!i915_sched_engine_is_empty(engine->sched_engine))
> return false;
>
> /* Ring stopped? */
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index d1dc1db3e378..2326a73af6d3 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> prio = rq_prio(rq);
> pl = i915_sched_lookup_priolist(engine, prio);
> }
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> + GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
>
> list_move(&rq->sched.link, pl);
> set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> }
>
> /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> - if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> + if (!i915_sched_engine_is_empty(engine->sched_engine)) {
> ENGINE_TRACE(engine, "timeslice required for queue\n");
> return true;
> }
> @@ -2487,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> } else {
> queue_request(engine, request);
>
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> + GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
> GEM_BUG_ON(list_empty(&request->sched.link));
>
> if (submit_queue(engine, request))
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index d42dea79ee64..5d00f2e3c1de 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -541,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
>
> queue_request(engine, rq, rq_prio(rq));
>
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> + GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
> GEM_BUG_ON(list_empty(&rq->sched.link));
>
> tasklet_hi_schedule(&engine->execlists.tasklet);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 91a04e34cac5..5bec7b3b8456 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -66,6 +66,12 @@ i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> kref_put(&sched_engine->ref, i915_sched_engine_free);
> }
>
> +static inline bool
> +i915_sched_engine_is_empty(struct i915_sched_engine *sched_engine)
> +{
> + return RB_EMPTY_ROOT(&sched_engine->queue.rb_root);
> +}
> +
> void i915_request_show_with_schedule(struct drm_printer *m,
> const struct i915_request *rq,
> const char *prefix,
> --
> 2.28.0
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 2/9] drm/i915: Add i915_sched_engine_is_empty function
@ 2021-06-04 17:50 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 17:50 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Add wrapper function around RB tree to determine if i915_sched_engine is
> empty.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +-
> drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 6 +++---
> drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
> drivers/gpu/drm/i915/i915_scheduler.h | 6 ++++++
> 4 files changed, 11 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index d0f3814440f6..85f2effe9dc6 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -1287,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> intel_engine_flush_submission(engine);
>
> /* ELSP is empty, but there are ready requests? E.g. after reset */
> - if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> + if (!i915_sched_engine_is_empty(engine->sched_engine))
> return false;
>
> /* Ring stopped? */
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index d1dc1db3e378..2326a73af6d3 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> prio = rq_prio(rq);
> pl = i915_sched_lookup_priolist(engine, prio);
> }
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> + GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
>
> list_move(&rq->sched.link, pl);
> set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> }
>
> /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> - if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> + if (!i915_sched_engine_is_empty(engine->sched_engine)) {
> ENGINE_TRACE(engine, "timeslice required for queue\n");
> return true;
> }
> @@ -2487,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> } else {
> queue_request(engine, request);
>
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> + GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
> GEM_BUG_ON(list_empty(&request->sched.link));
>
> if (submit_queue(engine, request))
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index d42dea79ee64..5d00f2e3c1de 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -541,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
>
> queue_request(engine, rq, rq_prio(rq));
>
> - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> + GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
> GEM_BUG_ON(list_empty(&rq->sched.link));
>
> tasklet_hi_schedule(&engine->execlists.tasklet);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 91a04e34cac5..5bec7b3b8456 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -66,6 +66,12 @@ i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> kref_put(&sched_engine->ref, i915_sched_engine_free);
> }
>
> +static inline bool
> +i915_sched_engine_is_empty(struct i915_sched_engine *sched_engine)
> +{
> + return RB_EMPTY_ROOT(&sched_engine->queue.rb_root);
> +}
> +
> void i915_request_show_with_schedule(struct drm_printer *m,
> const struct i915_request *rq,
> const char *prefix,
> --
> 2.28.0
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
2021-06-04 17:35 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 17:51 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 17:51 UTC (permalink / raw)
To: Matthew Brost
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Fri, Jun 4, 2021 at 12:42 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> On Fri, Jun 04, 2021 at 12:38:22PM -0500, Jason Ekstrand wrote:
> > On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> > >
> > > Introduce i915_sched_engine object which is lower level data structure
> > > that i915_scheduler / generic code can operate on without touching
> > > execlist specific structures. This allows additional submission backends
> > > to be added without breaking the layering.
> > >
> > > This is a bit of detour to integrating the i915 with the DRM scheduler
> > > but this object will still exist when the DRM scheduler lands in the
> > > i915. It will however look a bit different. It will encapsulate the
> > > drm_gpu_scheduler object plus and common variables (to the backends)
> > > related to scheduling. Regardless this is a step in the right direction.
> > >
> > > This patch starts the aforementioned transition by moving the the
> > > priolist into the i915_sched_engine object.
> > >
> > > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > > ---
> > > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
> > > drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
> > > drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
> > > .../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
> > > drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
> > > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
> > > drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
> > > drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
> > > drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
> > > 9 files changed, 169 insertions(+), 90 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > index 3f9a811eb02b..d0f3814440f6 100644
> > > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > @@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
> > > memset(execlists->pending, 0, sizeof(execlists->pending));
> > > execlists->active =
> > > memset(execlists->inflight, 0, sizeof(execlists->inflight));
> > > -
> > > - execlists->queue_priority_hint = INT_MIN;
> > > - execlists->queue = RB_ROOT_CACHED;
> > > }
> > >
> > > static void cleanup_status_page(struct intel_engine_cs *engine)
> > > @@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > goto err_status;
> > > }
> > >
> > > + engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
> > > + if (!engine->sched_engine) {
> > > + err = -ENOMEM;
> > > + goto err_sched_engine;
> > > + }
> > > +
> > > err = intel_engine_init_cmd_parser(engine);
> > > if (err)
> > > goto err_cmd_parser;
> > > @@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > return 0;
> > >
> > > err_cmd_parser:
> > > + i915_sched_engine_put(engine->sched_engine);
> > > +err_sched_engine:
> > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > err_status:
> > > cleanup_status_page(engine);
> > > @@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > > GEM_BUG_ON(!list_empty(&engine->active.requests));
> > > tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> > >
> > > + i915_sched_engine_put(engine->sched_engine);
> > > intel_breadcrumbs_free(engine->breadcrumbs);
> > >
> > > intel_engine_fini_retire(engine);
> > > @@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> > > intel_engine_flush_submission(engine);
> > >
> > > /* ELSP is empty, but there are ready requests? E.g. after reset */
> > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
> > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> > > return false;
> > >
> > > /* Ring stopped? */
> > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > index 47f4397095e5..b6a00dd72808 100644
> > > --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > @@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
> > > intel_breadcrumbs_park(engine->breadcrumbs);
> > >
> > > /* Must be reset upon idling, or we may miss the busy wakeup. */
> > > - GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
> > > + GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
> > >
> > > if (engine->park)
> > > engine->park(engine);
> > >
> > > - engine->execlists.no_priolist = false;
> > > + engine->sched_engine->no_priolist = false;
> > >
> > > /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> > > intel_gt_pm_put_async(engine->gt);
> > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > index 9ef349cd5cea..86b41ddec373 100644
> > > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > @@ -59,6 +59,7 @@ struct drm_i915_reg_table;
> > > struct i915_gem_context;
> > > struct i915_request;
> > > struct i915_sched_attr;
> > > +struct i915_sched_engine;
> > > struct intel_gt;
> > > struct intel_ring;
> > > struct intel_uncore;
> > > @@ -152,11 +153,6 @@ struct intel_engine_execlists {
> > > */
> > > struct timer_list preempt;
> > >
> > > - /**
> > > - * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > - */
> > > - struct i915_priolist default_priolist;
> > > -
> > > /**
> > > * @ccid: identifier for contexts submitted to this engine
> > > */
> > > @@ -191,11 +187,6 @@ struct intel_engine_execlists {
> > > */
> > > u32 reset_ccid;
> > >
> > > - /**
> > > - * @no_priolist: priority lists disabled
> > > - */
> > > - bool no_priolist;
> > > -
> > > /**
> > > * @submit_reg: gen-specific execlist submission register
> > > * set to the ExecList Submission Port (elsp) register pre-Gen11 and to
> > > @@ -238,23 +229,8 @@ struct intel_engine_execlists {
> > > unsigned int port_mask;
> > >
> > > /**
> > > - * @queue_priority_hint: Highest pending priority.
> > > - *
> > > - * When we add requests into the queue, or adjust the priority of
> > > - * executing requests, we compute the maximum priority of those
> > > - * pending requests. We can then use this value to determine if
> > > - * we need to preempt the executing requests to service the queue.
> > > - * However, since the we may have recorded the priority of an inflight
> > > - * request we wanted to preempt but since completed, at the time of
> > > - * dequeuing the priority hint may no longer may match the highest
> > > - * available request priority.
> > > + * @virtual: virtual of requests, in priority lists
> >
> > Having trouble parsing this. Did you mean "queue of virtual
> > requests"? Or maybe "virtual queue of requests"?
> >
>
> Yep, this doesn't really make sense. How about:
> 'queue of requests on a virtual engine, in priority lists'
I'm not sure what "in priority lists" means.
--Jason
> > > */
> > > - int queue_priority_hint;
> > > -
> > > - /**
> > > - * @queue: queue of requests, in priority lists
> > > - */
> > > - struct rb_root_cached queue;
> > > struct rb_root_cached virtual;
> > >
> > > /**
> > > @@ -332,6 +308,8 @@ struct intel_engine_cs {
> > > struct list_head hold; /* ready requests, but on hold */
> > > } active;
> > >
> > > + struct i915_sched_engine *sched_engine;
> > > +
> > > /* keep a request in reserve for a [pm] barrier under oom */
> > > struct i915_request *request_pool;
> > >
> > > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > index 8db200422950..d1dc1db3e378 100644
> > > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > @@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
> > > return prio;
> > > }
> > >
> > > -static int queue_prio(const struct intel_engine_execlists *execlists)
> > > +static int queue_prio(const struct i915_sched_engine *sched_engine)
> > > {
> > > struct rb_node *rb;
> > >
> > > - rb = rb_first_cached(&execlists->queue);
> > > + rb = rb_first_cached(&sched_engine->queue);
> > > if (!rb)
> > > return INT_MIN;
> > >
> > > @@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > * to preserve FIFO ordering of dependencies.
> > > */
> > > last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
> > > - if (engine->execlists.queue_priority_hint <= last_prio)
> > > + if (engine->sched_engine->queue_priority_hint <= last_prio)
> > > return false;
> > >
> > > /*
> > > @@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > * context, it's priority would not exceed ELSP[0] aka last_prio.
> > > */
> > > return max(virtual_prio(&engine->execlists),
> > > - queue_prio(&engine->execlists)) > last_prio;
> > > + queue_prio(engine->sched_engine)) > last_prio;
> > > }
> > >
> > > __maybe_unused static bool
> > > @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > > prio = rq_prio(rq);
> > > pl = i915_sched_lookup_priolist(engine, prio);
> > > }
> > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > >
> > > list_move(&rq->sched.link, pl);
> > > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > > @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> > > }
> > >
> > > /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
> > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> > > ENGINE_TRACE(engine, "timeslice required for queue\n");
> > > return true;
> > > }
> > > @@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
> > > static void execlists_dequeue(struct intel_engine_cs *engine)
> > > {
> > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > struct i915_request **port = execlists->pending;
> > > struct i915_request ** const last_port = port + execlists->port_mask;
> > > struct i915_request *last, * const *active;
> > > @@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > last->fence.context,
> > > last->fence.seqno,
> > > last->sched.attr.priority,
> > > - execlists->queue_priority_hint);
> > > + sched_engine->queue_priority_hint);
> > > record_preemption(execlists);
> > >
> > > /*
> > > @@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > yesno(timer_expired(&execlists->timer)),
> > > last->fence.context, last->fence.seqno,
> > > rq_prio(last),
> > > - execlists->queue_priority_hint,
> > > + sched_engine->queue_priority_hint,
> > > yesno(timeslice_yield(execlists, last)));
> > >
> > > /*
> > > @@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > GEM_BUG_ON(rq->engine != &ve->base);
> > > GEM_BUG_ON(rq->context != &ve->context);
> > >
> > > - if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
> > > + if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> > > spin_unlock(&ve->base.active.lock);
> > > break;
> > > }
> > > @@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > yesno(engine != ve->siblings[0]));
> > >
> > > WRITE_ONCE(ve->request, NULL);
> > > - WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
> > > + WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
> > >
> > > rb = &ve->nodes[engine->id].rb;
> > > rb_erase_cached(rb, &execlists->virtual);
> > > @@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > break;
> > > }
> > >
> > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > struct i915_priolist *p = to_priolist(rb);
> > > struct i915_request *rq, *rn;
> > >
> > > @@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > }
> > > }
> > >
> > > - rb_erase_cached(&p->node, &execlists->queue);
> > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > i915_priolist_free(p);
> > > }
> > > done:
> > > @@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > * request triggering preemption on the next dequeue (or subsequent
> > > * interrupt for secondary ports).
> > > */
> > > - execlists->queue_priority_hint = queue_prio(execlists);
> > > + sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > > spin_unlock(&engine->active.lock);
> > >
> > > /*
> > > @@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> > > */
> > > __execlists_unhold(rq);
> > >
> > > - if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
> > > - engine->execlists.queue_priority_hint = rq_prio(rq);
> > > + if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> > > + engine->sched_engine->queue_priority_hint = rq_prio(rq);
> > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > }
> > >
> > > @@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
> > > static bool submit_queue(struct intel_engine_cs *engine,
> > > const struct i915_request *rq)
> > > {
> > > - struct intel_engine_execlists *execlists = &engine->execlists;
> > > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> > >
> > > - if (rq_prio(rq) <= execlists->queue_priority_hint)
> > > + if (rq_prio(rq) <= sched_engine->queue_priority_hint)
> > > return false;
> > >
> > > - execlists->queue_priority_hint = rq_prio(rq);
> > > + sched_engine->queue_priority_hint = rq_prio(rq);
> > > return true;
> > > }
> > >
> > > @@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> > > } else {
> > > queue_request(engine, request);
> > >
> > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > GEM_BUG_ON(list_empty(&request->sched.link));
> > >
> > > if (submit_queue(engine, request))
> > > @@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
> > > from_tasklet(engine, t, execlists.tasklet);
> > >
> > > /* The driver is wedged; don't process any more events. */
> > > - WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
> > > + WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> > > }
> > >
> > > static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > {
> > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > struct i915_request *rq, *rn;
> > > struct rb_node *rb;
> > > unsigned long flags;
> > > @@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > intel_engine_signal_breadcrumbs(engine);
> > >
> > > /* Flush the queued requests to the timeline list (for retiring). */
> > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > struct i915_priolist *p = to_priolist(rb);
> > >
> > > priolist_for_each_request_consume(rq, rn, p) {
> > > @@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > }
> > > }
> > >
> > > - rb_erase_cached(&p->node, &execlists->queue);
> > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > i915_priolist_free(p);
> > > }
> > >
> > > @@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > }
> > > i915_request_put(rq);
> > >
> > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > + ve->base.sched_engine->queue_priority_hint = INT_MIN;
> > > }
> > > spin_unlock(&ve->base.active.lock);
> > > }
> > >
> > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > >
> > > - execlists->queue_priority_hint = INT_MIN;
> > > - execlists->queue = RB_ROOT_CACHED;
> > > + sched_engine->queue_priority_hint = INT_MIN;
> > > + sched_engine->queue = RB_ROOT_CACHED;
> > >
> > > GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > > execlists->tasklet.callback = nop_submission_tasklet;
> > > @@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> > >
> > > static struct list_head *virtual_queue(struct virtual_engine *ve)
> > > {
> > > - return &ve->base.execlists.default_priolist.requests;
> > > + return &ve->base.sched_engine->default_priolist.requests;
> > > }
> > >
> > > static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > @@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > lrc_fini(&ve->context);
> > > intel_context_fini(&ve->context);
> > >
> > > - intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > + if (ve->base.breadcrumbs)
> > > + intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > + if (ve->base.sched_engine)
> > > + i915_sched_engine_put(ve->base.sched_engine);
> >
> > Will either of these ever actually be NULL? The first line or two
> > seems to indicate that breadcrumbs never is.
> >
> > > intel_engine_free_request_pool(&ve->base);
> > >
> > > kfree(ve->bonds);
> > > @@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
> > >
> > > ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
> > > rq->fence.context, rq->fence.seqno,
> > > - mask, ve->base.execlists.queue_priority_hint);
> > > + mask, ve->base.sched_engine->queue_priority_hint);
> > >
> > > return mask;
> > > }
> > > @@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > {
> > > struct virtual_engine * const ve =
> > > from_tasklet(ve, t, base.execlists.tasklet);
> > > - const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
> > > + const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> > > intel_engine_mask_t mask;
> > > unsigned int n;
> > >
> > > @@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > submit_engine:
> > > GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> > > node->prio = prio;
> > > - if (first && prio > sibling->execlists.queue_priority_hint)
> > > + if (first && prio > sibling->sched_engine->queue_priority_hint)
> > > tasklet_hi_schedule(&sibling->execlists.tasklet);
> > >
> > > unlock_engine:
> > > @@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > > i915_request_put(ve->request);
> > > }
> > >
> > > - ve->base.execlists.queue_priority_hint = rq_prio(rq);
> > > + ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
> > > ve->request = i915_request_get(rq);
> > >
> > > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > > @@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> > > intel_engine_init_execlists(&ve->base);
> > >
> > > + ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> > > + if (!ve->base.sched_engine) {
> > > + err = -ENOMEM;
> > > + goto err_put;
> > > + }
> > > +
> > > ve->base.cops = &virtual_context_ops;
> > > ve->base.request_alloc = execlists_request_alloc;
> > >
> > > @@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > ve->base.bond_execute = virtual_bond_execute;
> > >
> > > INIT_LIST_HEAD(virtual_queue(ve));
> > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> > >
> > > intel_context_init(&ve->context, &ve->base);
> > > @@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > unsigned int max)
> > > {
> > > const struct intel_engine_execlists *execlists = &engine->execlists;
> > > + const struct i915_sched_engine *sched_engine = engine->sched_engine;
> > > struct i915_request *rq, *last;
> > > unsigned long flags;
> > > unsigned int count;
> > > @@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > show_request(m, last, "\t\t", 0);
> > > }
> > >
> > > - if (execlists->queue_priority_hint != INT_MIN)
> > > + if (sched_engine->queue_priority_hint != INT_MIN)
> > > drm_printf(m, "\t\tQueue priority hint: %d\n",
> > > - READ_ONCE(execlists->queue_priority_hint));
> > > + READ_ONCE(sched_engine->queue_priority_hint));
> > >
> > > last = NULL;
> > > count = 0;
> > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
> > >
> > > priolist_for_each_request(rq, p) {
> > > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > index 32589c6625e1..b1fdba13e900 100644
> > > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > @@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
> > >
> > > GEM_BUG_ON(timer_pending(&mock->hw_delay));
> > >
> > > + i915_sched_engine_put(engine->sched_engine);
> > > intel_breadcrumbs_free(engine->breadcrumbs);
> > >
> > > intel_context_unpin(engine->kernel_context);
> > > @@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > {
> > > struct intel_context *ce;
> > >
> > > + engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> > > + if (!engine->sched_engine)
> > > + return -ENOMEM;
> > > +
> > > intel_engine_init_active(engine, ENGINE_MOCK);
> > > intel_engine_init_execlists(engine);
> > > intel_engine_init__pm(engine);
> > > @@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > >
> > > engine->breadcrumbs = intel_breadcrumbs_create(NULL);
> > > if (!engine->breadcrumbs)
> > > - return -ENOMEM;
> > > + goto err_schedule;
> > >
> > > ce = create_kernel_context(engine);
> > > if (IS_ERR(ce))
> > > @@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > >
> > > err_breadcrumbs:
> > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > +err_schedule:
> > > + i915_sched_engine_put(engine->sched_engine);
> > > return -ENOMEM;
> > > }
> > >
> > > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > index 335719f17490..d42dea79ee64 100644
> > > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > @@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
> > > static void __guc_dequeue(struct intel_engine_cs *engine)
> > > {
> > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > struct i915_request **first = execlists->inflight;
> > > struct i915_request ** const last_port = first + execlists->port_mask;
> > > struct i915_request *last = first[0];
> > > @@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > * event.
> > > */
> > > port = first;
> > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > struct i915_priolist *p = to_priolist(rb);
> > > struct i915_request *rq, *rn;
> > >
> > > @@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > last = rq;
> > > }
> > >
> > > - rb_erase_cached(&p->node, &execlists->queue);
> > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > i915_priolist_free(p);
> > > }
> > > done:
> > > - execlists->queue_priority_hint =
> > > + sched_engine->queue_priority_hint =
> > > rb ? to_priolist(rb)->priority : INT_MIN;
> > > if (submit) {
> > > *port = schedule_in(last, port - execlists->inflight);
> > > @@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > >
> > > static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > {
> > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > struct i915_request *rq, *rn;
> > > struct rb_node *rb;
> > > unsigned long flags;
> > > @@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > }
> > >
> > > /* Flush the queued requests to the timeline list (for retiring). */
> > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > struct i915_priolist *p = to_priolist(rb);
> > >
> > > priolist_for_each_request_consume(rq, rn, p) {
> > > @@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > i915_request_mark_complete(rq);
> > > }
> > >
> > > - rb_erase_cached(&p->node, &execlists->queue);
> > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > i915_priolist_free(p);
> > > }
> > >
> > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > >
> > > - execlists->queue_priority_hint = INT_MIN;
> > > - execlists->queue = RB_ROOT_CACHED;
> > > + sched_engine->queue_priority_hint = INT_MIN;
> > > + sched_engine->queue = RB_ROOT_CACHED;
> > >
> > > spin_unlock_irqrestore(&engine->active.lock, flags);
> > > }
> > > @@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
> > >
> > > queue_request(engine, rq, rq_prio(rq));
> > >
> > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > GEM_BUG_ON(list_empty(&rq->sched.link));
> > >
> > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > > index efa638c3acc7..4953874a9ba6 100644
> > > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > > @@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> > > return rb_entry(rb, struct i915_priolist, node);
> > > }
> > >
> > > -static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > +static void assert_priolists(struct i915_sched_engine * const sched_engine)
> > > {
> > > struct rb_node *rb;
> > > long last_prio;
> > > @@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> > > return;
> > >
> > > - GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
> > > - rb_first(&execlists->queue.rb_root));
> > > + GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
> > > + rb_first(&sched_engine->queue.rb_root));
> > >
> > > last_prio = INT_MAX;
> > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > const struct i915_priolist *p = to_priolist(rb);
> > >
> > > GEM_BUG_ON(p->priority > last_prio);
> > > @@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > struct list_head *
> > > i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > {
> > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > struct i915_priolist *p;
> > > struct rb_node **parent, *rb;
> > > bool first = true;
> > >
> > > lockdep_assert_held(&engine->active.lock);
> > > - assert_priolists(execlists);
> > > + assert_priolists(sched_engine);
> > >
> > > - if (unlikely(execlists->no_priolist))
> > > + if (unlikely(sched_engine->no_priolist))
> > > prio = I915_PRIORITY_NORMAL;
> > >
> > > find_priolist:
> > > /* most positive priority is scheduled first, equal priorities fifo */
> > > rb = NULL;
> > > - parent = &execlists->queue.rb_root.rb_node;
> > > + parent = &sched_engine->queue.rb_root.rb_node;
> > > while (*parent) {
> > > rb = *parent;
> > > p = to_priolist(rb);
> > > @@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > }
> > >
> > > if (prio == I915_PRIORITY_NORMAL) {
> > > - p = &execlists->default_priolist;
> > > + p = &sched_engine->default_priolist;
> > > } else {
> > > p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
> > > /* Convert an allocation failure to a priority bump */
> > > @@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > * requests, so if userspace lied about their
> > > * dependencies that reordering may be visible.
> > > */
> > > - execlists->no_priolist = true;
> > > + sched_engine->no_priolist = true;
> > > goto find_priolist;
> > > }
> > > }
> > > @@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > INIT_LIST_HEAD(&p->requests);
> > >
> > > rb_link_node(&p->node, rb, parent);
> > > - rb_insert_color_cached(&p->node, &execlists->queue, first);
> > > + rb_insert_color_cached(&p->node, &sched_engine->queue, first);
> > >
> > > return &p->requests;
> > > }
> > > @@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > * We only need to kick the tasklet once for the high priority
> > > * new context we add into the queue.
> > > */
> > > - if (prio <= engine->execlists.queue_priority_hint)
> > > + if (prio <= engine->sched_engine->queue_priority_hint)
> > > return;
> > >
> > > rcu_read_lock();
> > > @@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > inflight->fence.context, inflight->fence.seqno,
> > > inflight->sched.attr.priority);
> > >
> > > - engine->execlists.queue_priority_hint = prio;
> > > + engine->sched_engine->queue_priority_hint = prio;
> > > if (need_preempt(prio, rq_prio(inflight)))
> > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > >
> > > @@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
> > > rcu_read_unlock();
> > > }
> > >
> > > +void i915_sched_engine_free(struct kref *kref)
> > > +{
> > > + struct i915_sched_engine *sched_engine =
> > > + container_of(kref, typeof(*sched_engine), ref);
> > > +
> > > + kfree(sched_engine);
> > > +}
> > > +
> > > +struct i915_sched_engine *
> > > +i915_sched_engine_create(unsigned int subclass)
> > > +{
> > > + struct i915_sched_engine *sched_engine;
> > > +
> > > + sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
> > > + if (!sched_engine)
> > > + return NULL;
> > > +
> > > + kref_init(&sched_engine->ref);
> > > +
> > > + sched_engine->queue = RB_ROOT_CACHED;
> > > + sched_engine->queue_priority_hint = INT_MIN;
> > > +
> > > + return sched_engine;
> > > +}
> > > +
> > > static void i915_global_scheduler_shrink(void)
> > > {
> > > kmem_cache_shrink(global.slab_dependencies);
> > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > > index 858a0938f47a..91a04e34cac5 100644
> > > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > > @@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
> > > __i915_priolist_free(p);
> > > }
> > >
> > > +struct i915_sched_engine *
> > > +i915_sched_engine_create(unsigned int subclass);
> > > +
> > > +void i915_sched_engine_free(struct kref *kref);
> > > +
> > > +static inline struct i915_sched_engine *
> > > +i915_sched_engine_get(struct i915_sched_engine *sched_engine)
> > > +{
> > > + kref_get(&sched_engine->ref);
> > > + return sched_engine;
> > > +}
> > > +
> > > +static inline void
> > > +i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> > > +{
> > > + kref_put(&sched_engine->ref, i915_sched_engine_free);
> > > +}
> > > +
> > > void i915_request_show_with_schedule(struct drm_printer *m,
> > > const struct i915_request *rq,
> > > const char *prefix,
> > > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > index 343ed44d5ed4..16a054e67760 100644
> > > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> >
> > Dumb question for which I'm not really in the position of having an
> > opinion: Should this go in its own file?
> >
>
> I think this is in the right file.
>
> Matt
>
> > Patch seems mostly mechanically correct. Not sure where this is going
> > yet, though. I'll keep reading. :-)
> >
> > --Jason
> >
> >
> > > @@ -91,4 +91,37 @@ struct i915_dependency {
> > > &(rq__)->sched.signalers_list, \
> > > signal_link)
> > >
> > > +struct i915_sched_engine {
> > > + struct kref ref;
> > > +
> > > + /**
> > > + * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > + */
> > > + struct i915_priolist default_priolist;
> > > +
> > > + /**
> > > + * @queue_priority_hint: Highest pending priority.
> > > + *
> > > + * When we add requests into the queue, or adjust the priority of
> > > + * executing requests, we compute the maximum priority of those
> > > + * pending requests. We can then use this value to determine if
> > > + * we need to preempt the executing requests to service the queue.
> > > + * However, since the we may have recorded the priority of an inflight
> > > + * request we wanted to preempt but since completed, at the time of
> > > + * dequeuing the priority hint may no longer may match the highest
> > > + * available request priority.
> > > + */
> > > + int queue_priority_hint;
> > > +
> > > + /**
> > > + * @queue: queue of requests, in priority lists
> > > + */
> > > + struct rb_root_cached queue;
> > > +
> > > + /**
> > > + * @no_priolist: priority lists disabled
> > > + */
> > > + bool no_priolist;
> > > +};
> > > +
> > > #endif /* _I915_SCHEDULER_TYPES_H_ */
> > > --
> > > 2.28.0
> > >
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
@ 2021-06-04 17:51 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 17:51 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 4, 2021 at 12:42 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> On Fri, Jun 04, 2021 at 12:38:22PM -0500, Jason Ekstrand wrote:
> > On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> > >
> > > Introduce i915_sched_engine object which is lower level data structure
> > > that i915_scheduler / generic code can operate on without touching
> > > execlist specific structures. This allows additional submission backends
> > > to be added without breaking the layering.
> > >
> > > This is a bit of detour to integrating the i915 with the DRM scheduler
> > > but this object will still exist when the DRM scheduler lands in the
> > > i915. It will however look a bit different. It will encapsulate the
> > > drm_gpu_scheduler object plus and common variables (to the backends)
> > > related to scheduling. Regardless this is a step in the right direction.
> > >
> > > This patch starts the aforementioned transition by moving the the
> > > priolist into the i915_sched_engine object.
> > >
> > > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > > ---
> > > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
> > > drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
> > > drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
> > > .../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
> > > drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
> > > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
> > > drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
> > > drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
> > > drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
> > > 9 files changed, 169 insertions(+), 90 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > index 3f9a811eb02b..d0f3814440f6 100644
> > > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > @@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
> > > memset(execlists->pending, 0, sizeof(execlists->pending));
> > > execlists->active =
> > > memset(execlists->inflight, 0, sizeof(execlists->inflight));
> > > -
> > > - execlists->queue_priority_hint = INT_MIN;
> > > - execlists->queue = RB_ROOT_CACHED;
> > > }
> > >
> > > static void cleanup_status_page(struct intel_engine_cs *engine)
> > > @@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > goto err_status;
> > > }
> > >
> > > + engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
> > > + if (!engine->sched_engine) {
> > > + err = -ENOMEM;
> > > + goto err_sched_engine;
> > > + }
> > > +
> > > err = intel_engine_init_cmd_parser(engine);
> > > if (err)
> > > goto err_cmd_parser;
> > > @@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > return 0;
> > >
> > > err_cmd_parser:
> > > + i915_sched_engine_put(engine->sched_engine);
> > > +err_sched_engine:
> > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > err_status:
> > > cleanup_status_page(engine);
> > > @@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > > GEM_BUG_ON(!list_empty(&engine->active.requests));
> > > tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> > >
> > > + i915_sched_engine_put(engine->sched_engine);
> > > intel_breadcrumbs_free(engine->breadcrumbs);
> > >
> > > intel_engine_fini_retire(engine);
> > > @@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> > > intel_engine_flush_submission(engine);
> > >
> > > /* ELSP is empty, but there are ready requests? E.g. after reset */
> > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
> > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> > > return false;
> > >
> > > /* Ring stopped? */
> > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > index 47f4397095e5..b6a00dd72808 100644
> > > --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > @@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
> > > intel_breadcrumbs_park(engine->breadcrumbs);
> > >
> > > /* Must be reset upon idling, or we may miss the busy wakeup. */
> > > - GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
> > > + GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
> > >
> > > if (engine->park)
> > > engine->park(engine);
> > >
> > > - engine->execlists.no_priolist = false;
> > > + engine->sched_engine->no_priolist = false;
> > >
> > > /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> > > intel_gt_pm_put_async(engine->gt);
> > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > index 9ef349cd5cea..86b41ddec373 100644
> > > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > @@ -59,6 +59,7 @@ struct drm_i915_reg_table;
> > > struct i915_gem_context;
> > > struct i915_request;
> > > struct i915_sched_attr;
> > > +struct i915_sched_engine;
> > > struct intel_gt;
> > > struct intel_ring;
> > > struct intel_uncore;
> > > @@ -152,11 +153,6 @@ struct intel_engine_execlists {
> > > */
> > > struct timer_list preempt;
> > >
> > > - /**
> > > - * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > - */
> > > - struct i915_priolist default_priolist;
> > > -
> > > /**
> > > * @ccid: identifier for contexts submitted to this engine
> > > */
> > > @@ -191,11 +187,6 @@ struct intel_engine_execlists {
> > > */
> > > u32 reset_ccid;
> > >
> > > - /**
> > > - * @no_priolist: priority lists disabled
> > > - */
> > > - bool no_priolist;
> > > -
> > > /**
> > > * @submit_reg: gen-specific execlist submission register
> > > * set to the ExecList Submission Port (elsp) register pre-Gen11 and to
> > > @@ -238,23 +229,8 @@ struct intel_engine_execlists {
> > > unsigned int port_mask;
> > >
> > > /**
> > > - * @queue_priority_hint: Highest pending priority.
> > > - *
> > > - * When we add requests into the queue, or adjust the priority of
> > > - * executing requests, we compute the maximum priority of those
> > > - * pending requests. We can then use this value to determine if
> > > - * we need to preempt the executing requests to service the queue.
> > > - * However, since the we may have recorded the priority of an inflight
> > > - * request we wanted to preempt but since completed, at the time of
> > > - * dequeuing the priority hint may no longer may match the highest
> > > - * available request priority.
> > > + * @virtual: virtual of requests, in priority lists
> >
> > Having trouble parsing this. Did you mean "queue of virtual
> > requests"? Or maybe "virtual queue of requests"?
> >
>
> Yep, this doesn't really make sense. How about:
> 'queue of requests on a virtual engine, in priority lists'
I'm not sure what "in priority lists" means.
--Jason
> > > */
> > > - int queue_priority_hint;
> > > -
> > > - /**
> > > - * @queue: queue of requests, in priority lists
> > > - */
> > > - struct rb_root_cached queue;
> > > struct rb_root_cached virtual;
> > >
> > > /**
> > > @@ -332,6 +308,8 @@ struct intel_engine_cs {
> > > struct list_head hold; /* ready requests, but on hold */
> > > } active;
> > >
> > > + struct i915_sched_engine *sched_engine;
> > > +
> > > /* keep a request in reserve for a [pm] barrier under oom */
> > > struct i915_request *request_pool;
> > >
> > > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > index 8db200422950..d1dc1db3e378 100644
> > > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > @@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
> > > return prio;
> > > }
> > >
> > > -static int queue_prio(const struct intel_engine_execlists *execlists)
> > > +static int queue_prio(const struct i915_sched_engine *sched_engine)
> > > {
> > > struct rb_node *rb;
> > >
> > > - rb = rb_first_cached(&execlists->queue);
> > > + rb = rb_first_cached(&sched_engine->queue);
> > > if (!rb)
> > > return INT_MIN;
> > >
> > > @@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > * to preserve FIFO ordering of dependencies.
> > > */
> > > last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
> > > - if (engine->execlists.queue_priority_hint <= last_prio)
> > > + if (engine->sched_engine->queue_priority_hint <= last_prio)
> > > return false;
> > >
> > > /*
> > > @@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > * context, it's priority would not exceed ELSP[0] aka last_prio.
> > > */
> > > return max(virtual_prio(&engine->execlists),
> > > - queue_prio(&engine->execlists)) > last_prio;
> > > + queue_prio(engine->sched_engine)) > last_prio;
> > > }
> > >
> > > __maybe_unused static bool
> > > @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > > prio = rq_prio(rq);
> > > pl = i915_sched_lookup_priolist(engine, prio);
> > > }
> > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > >
> > > list_move(&rq->sched.link, pl);
> > > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > > @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> > > }
> > >
> > > /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
> > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> > > ENGINE_TRACE(engine, "timeslice required for queue\n");
> > > return true;
> > > }
> > > @@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
> > > static void execlists_dequeue(struct intel_engine_cs *engine)
> > > {
> > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > struct i915_request **port = execlists->pending;
> > > struct i915_request ** const last_port = port + execlists->port_mask;
> > > struct i915_request *last, * const *active;
> > > @@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > last->fence.context,
> > > last->fence.seqno,
> > > last->sched.attr.priority,
> > > - execlists->queue_priority_hint);
> > > + sched_engine->queue_priority_hint);
> > > record_preemption(execlists);
> > >
> > > /*
> > > @@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > yesno(timer_expired(&execlists->timer)),
> > > last->fence.context, last->fence.seqno,
> > > rq_prio(last),
> > > - execlists->queue_priority_hint,
> > > + sched_engine->queue_priority_hint,
> > > yesno(timeslice_yield(execlists, last)));
> > >
> > > /*
> > > @@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > GEM_BUG_ON(rq->engine != &ve->base);
> > > GEM_BUG_ON(rq->context != &ve->context);
> > >
> > > - if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
> > > + if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> > > spin_unlock(&ve->base.active.lock);
> > > break;
> > > }
> > > @@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > yesno(engine != ve->siblings[0]));
> > >
> > > WRITE_ONCE(ve->request, NULL);
> > > - WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
> > > + WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
> > >
> > > rb = &ve->nodes[engine->id].rb;
> > > rb_erase_cached(rb, &execlists->virtual);
> > > @@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > break;
> > > }
> > >
> > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > struct i915_priolist *p = to_priolist(rb);
> > > struct i915_request *rq, *rn;
> > >
> > > @@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > }
> > > }
> > >
> > > - rb_erase_cached(&p->node, &execlists->queue);
> > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > i915_priolist_free(p);
> > > }
> > > done:
> > > @@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > * request triggering preemption on the next dequeue (or subsequent
> > > * interrupt for secondary ports).
> > > */
> > > - execlists->queue_priority_hint = queue_prio(execlists);
> > > + sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > > spin_unlock(&engine->active.lock);
> > >
> > > /*
> > > @@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> > > */
> > > __execlists_unhold(rq);
> > >
> > > - if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
> > > - engine->execlists.queue_priority_hint = rq_prio(rq);
> > > + if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> > > + engine->sched_engine->queue_priority_hint = rq_prio(rq);
> > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > }
> > >
> > > @@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
> > > static bool submit_queue(struct intel_engine_cs *engine,
> > > const struct i915_request *rq)
> > > {
> > > - struct intel_engine_execlists *execlists = &engine->execlists;
> > > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> > >
> > > - if (rq_prio(rq) <= execlists->queue_priority_hint)
> > > + if (rq_prio(rq) <= sched_engine->queue_priority_hint)
> > > return false;
> > >
> > > - execlists->queue_priority_hint = rq_prio(rq);
> > > + sched_engine->queue_priority_hint = rq_prio(rq);
> > > return true;
> > > }
> > >
> > > @@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> > > } else {
> > > queue_request(engine, request);
> > >
> > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > GEM_BUG_ON(list_empty(&request->sched.link));
> > >
> > > if (submit_queue(engine, request))
> > > @@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
> > > from_tasklet(engine, t, execlists.tasklet);
> > >
> > > /* The driver is wedged; don't process any more events. */
> > > - WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
> > > + WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> > > }
> > >
> > > static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > {
> > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > struct i915_request *rq, *rn;
> > > struct rb_node *rb;
> > > unsigned long flags;
> > > @@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > intel_engine_signal_breadcrumbs(engine);
> > >
> > > /* Flush the queued requests to the timeline list (for retiring). */
> > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > struct i915_priolist *p = to_priolist(rb);
> > >
> > > priolist_for_each_request_consume(rq, rn, p) {
> > > @@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > }
> > > }
> > >
> > > - rb_erase_cached(&p->node, &execlists->queue);
> > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > i915_priolist_free(p);
> > > }
> > >
> > > @@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > }
> > > i915_request_put(rq);
> > >
> > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > + ve->base.sched_engine->queue_priority_hint = INT_MIN;
> > > }
> > > spin_unlock(&ve->base.active.lock);
> > > }
> > >
> > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > >
> > > - execlists->queue_priority_hint = INT_MIN;
> > > - execlists->queue = RB_ROOT_CACHED;
> > > + sched_engine->queue_priority_hint = INT_MIN;
> > > + sched_engine->queue = RB_ROOT_CACHED;
> > >
> > > GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > > execlists->tasklet.callback = nop_submission_tasklet;
> > > @@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> > >
> > > static struct list_head *virtual_queue(struct virtual_engine *ve)
> > > {
> > > - return &ve->base.execlists.default_priolist.requests;
> > > + return &ve->base.sched_engine->default_priolist.requests;
> > > }
> > >
> > > static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > @@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > lrc_fini(&ve->context);
> > > intel_context_fini(&ve->context);
> > >
> > > - intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > + if (ve->base.breadcrumbs)
> > > + intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > + if (ve->base.sched_engine)
> > > + i915_sched_engine_put(ve->base.sched_engine);
> >
> > Will either of these ever actually be NULL? The first line or two
> > seems to indicate that breadcrumbs never is.
> >
> > > intel_engine_free_request_pool(&ve->base);
> > >
> > > kfree(ve->bonds);
> > > @@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
> > >
> > > ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
> > > rq->fence.context, rq->fence.seqno,
> > > - mask, ve->base.execlists.queue_priority_hint);
> > > + mask, ve->base.sched_engine->queue_priority_hint);
> > >
> > > return mask;
> > > }
> > > @@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > {
> > > struct virtual_engine * const ve =
> > > from_tasklet(ve, t, base.execlists.tasklet);
> > > - const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
> > > + const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> > > intel_engine_mask_t mask;
> > > unsigned int n;
> > >
> > > @@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > submit_engine:
> > > GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> > > node->prio = prio;
> > > - if (first && prio > sibling->execlists.queue_priority_hint)
> > > + if (first && prio > sibling->sched_engine->queue_priority_hint)
> > > tasklet_hi_schedule(&sibling->execlists.tasklet);
> > >
> > > unlock_engine:
> > > @@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > > i915_request_put(ve->request);
> > > }
> > >
> > > - ve->base.execlists.queue_priority_hint = rq_prio(rq);
> > > + ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
> > > ve->request = i915_request_get(rq);
> > >
> > > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > > @@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> > > intel_engine_init_execlists(&ve->base);
> > >
> > > + ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> > > + if (!ve->base.sched_engine) {
> > > + err = -ENOMEM;
> > > + goto err_put;
> > > + }
> > > +
> > > ve->base.cops = &virtual_context_ops;
> > > ve->base.request_alloc = execlists_request_alloc;
> > >
> > > @@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > ve->base.bond_execute = virtual_bond_execute;
> > >
> > > INIT_LIST_HEAD(virtual_queue(ve));
> > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> > >
> > > intel_context_init(&ve->context, &ve->base);
> > > @@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > unsigned int max)
> > > {
> > > const struct intel_engine_execlists *execlists = &engine->execlists;
> > > + const struct i915_sched_engine *sched_engine = engine->sched_engine;
> > > struct i915_request *rq, *last;
> > > unsigned long flags;
> > > unsigned int count;
> > > @@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > show_request(m, last, "\t\t", 0);
> > > }
> > >
> > > - if (execlists->queue_priority_hint != INT_MIN)
> > > + if (sched_engine->queue_priority_hint != INT_MIN)
> > > drm_printf(m, "\t\tQueue priority hint: %d\n",
> > > - READ_ONCE(execlists->queue_priority_hint));
> > > + READ_ONCE(sched_engine->queue_priority_hint));
> > >
> > > last = NULL;
> > > count = 0;
> > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
> > >
> > > priolist_for_each_request(rq, p) {
> > > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > index 32589c6625e1..b1fdba13e900 100644
> > > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > @@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
> > >
> > > GEM_BUG_ON(timer_pending(&mock->hw_delay));
> > >
> > > + i915_sched_engine_put(engine->sched_engine);
> > > intel_breadcrumbs_free(engine->breadcrumbs);
> > >
> > > intel_context_unpin(engine->kernel_context);
> > > @@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > {
> > > struct intel_context *ce;
> > >
> > > + engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> > > + if (!engine->sched_engine)
> > > + return -ENOMEM;
> > > +
> > > intel_engine_init_active(engine, ENGINE_MOCK);
> > > intel_engine_init_execlists(engine);
> > > intel_engine_init__pm(engine);
> > > @@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > >
> > > engine->breadcrumbs = intel_breadcrumbs_create(NULL);
> > > if (!engine->breadcrumbs)
> > > - return -ENOMEM;
> > > + goto err_schedule;
> > >
> > > ce = create_kernel_context(engine);
> > > if (IS_ERR(ce))
> > > @@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > >
> > > err_breadcrumbs:
> > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > +err_schedule:
> > > + i915_sched_engine_put(engine->sched_engine);
> > > return -ENOMEM;
> > > }
> > >
> > > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > index 335719f17490..d42dea79ee64 100644
> > > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > @@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
> > > static void __guc_dequeue(struct intel_engine_cs *engine)
> > > {
> > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > struct i915_request **first = execlists->inflight;
> > > struct i915_request ** const last_port = first + execlists->port_mask;
> > > struct i915_request *last = first[0];
> > > @@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > * event.
> > > */
> > > port = first;
> > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > struct i915_priolist *p = to_priolist(rb);
> > > struct i915_request *rq, *rn;
> > >
> > > @@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > last = rq;
> > > }
> > >
> > > - rb_erase_cached(&p->node, &execlists->queue);
> > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > i915_priolist_free(p);
> > > }
> > > done:
> > > - execlists->queue_priority_hint =
> > > + sched_engine->queue_priority_hint =
> > > rb ? to_priolist(rb)->priority : INT_MIN;
> > > if (submit) {
> > > *port = schedule_in(last, port - execlists->inflight);
> > > @@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > >
> > > static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > {
> > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > struct i915_request *rq, *rn;
> > > struct rb_node *rb;
> > > unsigned long flags;
> > > @@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > }
> > >
> > > /* Flush the queued requests to the timeline list (for retiring). */
> > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > struct i915_priolist *p = to_priolist(rb);
> > >
> > > priolist_for_each_request_consume(rq, rn, p) {
> > > @@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > i915_request_mark_complete(rq);
> > > }
> > >
> > > - rb_erase_cached(&p->node, &execlists->queue);
> > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > i915_priolist_free(p);
> > > }
> > >
> > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > >
> > > - execlists->queue_priority_hint = INT_MIN;
> > > - execlists->queue = RB_ROOT_CACHED;
> > > + sched_engine->queue_priority_hint = INT_MIN;
> > > + sched_engine->queue = RB_ROOT_CACHED;
> > >
> > > spin_unlock_irqrestore(&engine->active.lock, flags);
> > > }
> > > @@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
> > >
> > > queue_request(engine, rq, rq_prio(rq));
> > >
> > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > GEM_BUG_ON(list_empty(&rq->sched.link));
> > >
> > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > > index efa638c3acc7..4953874a9ba6 100644
> > > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > > @@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> > > return rb_entry(rb, struct i915_priolist, node);
> > > }
> > >
> > > -static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > +static void assert_priolists(struct i915_sched_engine * const sched_engine)
> > > {
> > > struct rb_node *rb;
> > > long last_prio;
> > > @@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> > > return;
> > >
> > > - GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
> > > - rb_first(&execlists->queue.rb_root));
> > > + GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
> > > + rb_first(&sched_engine->queue.rb_root));
> > >
> > > last_prio = INT_MAX;
> > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > const struct i915_priolist *p = to_priolist(rb);
> > >
> > > GEM_BUG_ON(p->priority > last_prio);
> > > @@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > struct list_head *
> > > i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > {
> > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > struct i915_priolist *p;
> > > struct rb_node **parent, *rb;
> > > bool first = true;
> > >
> > > lockdep_assert_held(&engine->active.lock);
> > > - assert_priolists(execlists);
> > > + assert_priolists(sched_engine);
> > >
> > > - if (unlikely(execlists->no_priolist))
> > > + if (unlikely(sched_engine->no_priolist))
> > > prio = I915_PRIORITY_NORMAL;
> > >
> > > find_priolist:
> > > /* most positive priority is scheduled first, equal priorities fifo */
> > > rb = NULL;
> > > - parent = &execlists->queue.rb_root.rb_node;
> > > + parent = &sched_engine->queue.rb_root.rb_node;
> > > while (*parent) {
> > > rb = *parent;
> > > p = to_priolist(rb);
> > > @@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > }
> > >
> > > if (prio == I915_PRIORITY_NORMAL) {
> > > - p = &execlists->default_priolist;
> > > + p = &sched_engine->default_priolist;
> > > } else {
> > > p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
> > > /* Convert an allocation failure to a priority bump */
> > > @@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > * requests, so if userspace lied about their
> > > * dependencies that reordering may be visible.
> > > */
> > > - execlists->no_priolist = true;
> > > + sched_engine->no_priolist = true;
> > > goto find_priolist;
> > > }
> > > }
> > > @@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > INIT_LIST_HEAD(&p->requests);
> > >
> > > rb_link_node(&p->node, rb, parent);
> > > - rb_insert_color_cached(&p->node, &execlists->queue, first);
> > > + rb_insert_color_cached(&p->node, &sched_engine->queue, first);
> > >
> > > return &p->requests;
> > > }
> > > @@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > * We only need to kick the tasklet once for the high priority
> > > * new context we add into the queue.
> > > */
> > > - if (prio <= engine->execlists.queue_priority_hint)
> > > + if (prio <= engine->sched_engine->queue_priority_hint)
> > > return;
> > >
> > > rcu_read_lock();
> > > @@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > inflight->fence.context, inflight->fence.seqno,
> > > inflight->sched.attr.priority);
> > >
> > > - engine->execlists.queue_priority_hint = prio;
> > > + engine->sched_engine->queue_priority_hint = prio;
> > > if (need_preempt(prio, rq_prio(inflight)))
> > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > >
> > > @@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
> > > rcu_read_unlock();
> > > }
> > >
> > > +void i915_sched_engine_free(struct kref *kref)
> > > +{
> > > + struct i915_sched_engine *sched_engine =
> > > + container_of(kref, typeof(*sched_engine), ref);
> > > +
> > > + kfree(sched_engine);
> > > +}
> > > +
> > > +struct i915_sched_engine *
> > > +i915_sched_engine_create(unsigned int subclass)
> > > +{
> > > + struct i915_sched_engine *sched_engine;
> > > +
> > > + sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
> > > + if (!sched_engine)
> > > + return NULL;
> > > +
> > > + kref_init(&sched_engine->ref);
> > > +
> > > + sched_engine->queue = RB_ROOT_CACHED;
> > > + sched_engine->queue_priority_hint = INT_MIN;
> > > +
> > > + return sched_engine;
> > > +}
> > > +
> > > static void i915_global_scheduler_shrink(void)
> > > {
> > > kmem_cache_shrink(global.slab_dependencies);
> > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > > index 858a0938f47a..91a04e34cac5 100644
> > > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > > @@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
> > > __i915_priolist_free(p);
> > > }
> > >
> > > +struct i915_sched_engine *
> > > +i915_sched_engine_create(unsigned int subclass);
> > > +
> > > +void i915_sched_engine_free(struct kref *kref);
> > > +
> > > +static inline struct i915_sched_engine *
> > > +i915_sched_engine_get(struct i915_sched_engine *sched_engine)
> > > +{
> > > + kref_get(&sched_engine->ref);
> > > + return sched_engine;
> > > +}
> > > +
> > > +static inline void
> > > +i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> > > +{
> > > + kref_put(&sched_engine->ref, i915_sched_engine_free);
> > > +}
> > > +
> > > void i915_request_show_with_schedule(struct drm_printer *m,
> > > const struct i915_request *rq,
> > > const char *prefix,
> > > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > index 343ed44d5ed4..16a054e67760 100644
> > > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> >
> > Dumb question for which I'm not really in the position of having an
> > opinion: Should this go in its own file?
> >
>
> I think this is in the right file.
>
> Matt
>
> > Patch seems mostly mechanically correct. Not sure where this is going
> > yet, though. I'll keep reading. :-)
> >
> > --Jason
> >
> >
> > > @@ -91,4 +91,37 @@ struct i915_dependency {
> > > &(rq__)->sched.signalers_list, \
> > > signal_link)
> > >
> > > +struct i915_sched_engine {
> > > + struct kref ref;
> > > +
> > > + /**
> > > + * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > + */
> > > + struct i915_priolist default_priolist;
> > > +
> > > + /**
> > > + * @queue_priority_hint: Highest pending priority.
> > > + *
> > > + * When we add requests into the queue, or adjust the priority of
> > > + * executing requests, we compute the maximum priority of those
> > > + * pending requests. We can then use this value to determine if
> > > + * we need to preempt the executing requests to service the queue.
> > > + * However, since the we may have recorded the priority of an inflight
> > > + * request we wanted to preempt but since completed, at the time of
> > > + * dequeuing the priority hint may no longer may match the highest
> > > + * available request priority.
> > > + */
> > > + int queue_priority_hint;
> > > +
> > > + /**
> > > + * @queue: queue of requests, in priority lists
> > > + */
> > > + struct rb_root_cached queue;
> > > +
> > > + /**
> > > + * @no_priolist: priority lists disabled
> > > + */
> > > + bool no_priolist;
> > > +};
> > > +
> > > #endif /* _I915_SCHEDULER_TYPES_H_ */
> > > --
> > > 2.28.0
> > >
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
2021-06-04 17:51 ` [Intel-gfx] " Jason Ekstrand
@ 2021-06-04 17:51 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 17:51 UTC (permalink / raw)
To: Jason Ekstrand
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Fri, Jun 04, 2021 at 12:51:43PM -0500, Jason Ekstrand wrote:
> On Fri, Jun 4, 2021 at 12:42 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > On Fri, Jun 04, 2021 at 12:38:22PM -0500, Jason Ekstrand wrote:
> > > On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> > > >
> > > > Introduce i915_sched_engine object which is lower level data structure
> > > > that i915_scheduler / generic code can operate on without touching
> > > > execlist specific structures. This allows additional submission backends
> > > > to be added without breaking the layering.
> > > >
> > > > This is a bit of detour to integrating the i915 with the DRM scheduler
> > > > but this object will still exist when the DRM scheduler lands in the
> > > > i915. It will however look a bit different. It will encapsulate the
> > > > drm_gpu_scheduler object plus and common variables (to the backends)
> > > > related to scheduling. Regardless this is a step in the right direction.
> > > >
> > > > This patch starts the aforementioned transition by moving the the
> > > > priolist into the i915_sched_engine object.
> > > >
> > > > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > > > ---
> > > > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
> > > > drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
> > > > drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
> > > > .../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
> > > > drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
> > > > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
> > > > drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
> > > > drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
> > > > drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
> > > > 9 files changed, 169 insertions(+), 90 deletions(-)
> > > >
> > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > index 3f9a811eb02b..d0f3814440f6 100644
> > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > @@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
> > > > memset(execlists->pending, 0, sizeof(execlists->pending));
> > > > execlists->active =
> > > > memset(execlists->inflight, 0, sizeof(execlists->inflight));
> > > > -
> > > > - execlists->queue_priority_hint = INT_MIN;
> > > > - execlists->queue = RB_ROOT_CACHED;
> > > > }
> > > >
> > > > static void cleanup_status_page(struct intel_engine_cs *engine)
> > > > @@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > > goto err_status;
> > > > }
> > > >
> > > > + engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
> > > > + if (!engine->sched_engine) {
> > > > + err = -ENOMEM;
> > > > + goto err_sched_engine;
> > > > + }
> > > > +
> > > > err = intel_engine_init_cmd_parser(engine);
> > > > if (err)
> > > > goto err_cmd_parser;
> > > > @@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > > return 0;
> > > >
> > > > err_cmd_parser:
> > > > + i915_sched_engine_put(engine->sched_engine);
> > > > +err_sched_engine:
> > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > err_status:
> > > > cleanup_status_page(engine);
> > > > @@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > > > GEM_BUG_ON(!list_empty(&engine->active.requests));
> > > > tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> > > >
> > > > + i915_sched_engine_put(engine->sched_engine);
> > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > >
> > > > intel_engine_fini_retire(engine);
> > > > @@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> > > > intel_engine_flush_submission(engine);
> > > >
> > > > /* ELSP is empty, but there are ready requests? E.g. after reset */
> > > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
> > > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> > > > return false;
> > > >
> > > > /* Ring stopped? */
> > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > index 47f4397095e5..b6a00dd72808 100644
> > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > @@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
> > > > intel_breadcrumbs_park(engine->breadcrumbs);
> > > >
> > > > /* Must be reset upon idling, or we may miss the busy wakeup. */
> > > > - GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
> > > > + GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
> > > >
> > > > if (engine->park)
> > > > engine->park(engine);
> > > >
> > > > - engine->execlists.no_priolist = false;
> > > > + engine->sched_engine->no_priolist = false;
> > > >
> > > > /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> > > > intel_gt_pm_put_async(engine->gt);
> > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > index 9ef349cd5cea..86b41ddec373 100644
> > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > @@ -59,6 +59,7 @@ struct drm_i915_reg_table;
> > > > struct i915_gem_context;
> > > > struct i915_request;
> > > > struct i915_sched_attr;
> > > > +struct i915_sched_engine;
> > > > struct intel_gt;
> > > > struct intel_ring;
> > > > struct intel_uncore;
> > > > @@ -152,11 +153,6 @@ struct intel_engine_execlists {
> > > > */
> > > > struct timer_list preempt;
> > > >
> > > > - /**
> > > > - * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > > - */
> > > > - struct i915_priolist default_priolist;
> > > > -
> > > > /**
> > > > * @ccid: identifier for contexts submitted to this engine
> > > > */
> > > > @@ -191,11 +187,6 @@ struct intel_engine_execlists {
> > > > */
> > > > u32 reset_ccid;
> > > >
> > > > - /**
> > > > - * @no_priolist: priority lists disabled
> > > > - */
> > > > - bool no_priolist;
> > > > -
> > > > /**
> > > > * @submit_reg: gen-specific execlist submission register
> > > > * set to the ExecList Submission Port (elsp) register pre-Gen11 and to
> > > > @@ -238,23 +229,8 @@ struct intel_engine_execlists {
> > > > unsigned int port_mask;
> > > >
> > > > /**
> > > > - * @queue_priority_hint: Highest pending priority.
> > > > - *
> > > > - * When we add requests into the queue, or adjust the priority of
> > > > - * executing requests, we compute the maximum priority of those
> > > > - * pending requests. We can then use this value to determine if
> > > > - * we need to preempt the executing requests to service the queue.
> > > > - * However, since the we may have recorded the priority of an inflight
> > > > - * request we wanted to preempt but since completed, at the time of
> > > > - * dequeuing the priority hint may no longer may match the highest
> > > > - * available request priority.
> > > > + * @virtual: virtual of requests, in priority lists
> > >
> > > Having trouble parsing this. Did you mean "queue of virtual
> > > requests"? Or maybe "virtual queue of requests"?
> > >
> >
> > Yep, this doesn't really make sense. How about:
> > 'queue of requests on a virtual engine, in priority lists'
>
> I'm not sure what "in priority lists" means.
>
Ah, this is confusing to explain. Let me try to this again.
'Queue of requets on a virtual engine, sorted by priority. Each RB entry
is a i915_priolist containing a list of requests of the same priority.'
Matt
> --Jason
>
> > > > */
> > > > - int queue_priority_hint;
> > > > -
> > > > - /**
> > > > - * @queue: queue of requests, in priority lists
> > > > - */
> > > > - struct rb_root_cached queue;
> > > > struct rb_root_cached virtual;
> > > >
> > > > /**
> > > > @@ -332,6 +308,8 @@ struct intel_engine_cs {
> > > > struct list_head hold; /* ready requests, but on hold */
> > > > } active;
> > > >
> > > > + struct i915_sched_engine *sched_engine;
> > > > +
> > > > /* keep a request in reserve for a [pm] barrier under oom */
> > > > struct i915_request *request_pool;
> > > >
> > > > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > index 8db200422950..d1dc1db3e378 100644
> > > > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > @@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
> > > > return prio;
> > > > }
> > > >
> > > > -static int queue_prio(const struct intel_engine_execlists *execlists)
> > > > +static int queue_prio(const struct i915_sched_engine *sched_engine)
> > > > {
> > > > struct rb_node *rb;
> > > >
> > > > - rb = rb_first_cached(&execlists->queue);
> > > > + rb = rb_first_cached(&sched_engine->queue);
> > > > if (!rb)
> > > > return INT_MIN;
> > > >
> > > > @@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > > * to preserve FIFO ordering of dependencies.
> > > > */
> > > > last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
> > > > - if (engine->execlists.queue_priority_hint <= last_prio)
> > > > + if (engine->sched_engine->queue_priority_hint <= last_prio)
> > > > return false;
> > > >
> > > > /*
> > > > @@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > > * context, it's priority would not exceed ELSP[0] aka last_prio.
> > > > */
> > > > return max(virtual_prio(&engine->execlists),
> > > > - queue_prio(&engine->execlists)) > last_prio;
> > > > + queue_prio(engine->sched_engine)) > last_prio;
> > > > }
> > > >
> > > > __maybe_unused static bool
> > > > @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > > > prio = rq_prio(rq);
> > > > pl = i915_sched_lookup_priolist(engine, prio);
> > > > }
> > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > >
> > > > list_move(&rq->sched.link, pl);
> > > > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > > > @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> > > > }
> > > >
> > > > /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> > > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
> > > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> > > > ENGINE_TRACE(engine, "timeslice required for queue\n");
> > > > return true;
> > > > }
> > > > @@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
> > > > static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > {
> > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > struct i915_request **port = execlists->pending;
> > > > struct i915_request ** const last_port = port + execlists->port_mask;
> > > > struct i915_request *last, * const *active;
> > > > @@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > last->fence.context,
> > > > last->fence.seqno,
> > > > last->sched.attr.priority,
> > > > - execlists->queue_priority_hint);
> > > > + sched_engine->queue_priority_hint);
> > > > record_preemption(execlists);
> > > >
> > > > /*
> > > > @@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > yesno(timer_expired(&execlists->timer)),
> > > > last->fence.context, last->fence.seqno,
> > > > rq_prio(last),
> > > > - execlists->queue_priority_hint,
> > > > + sched_engine->queue_priority_hint,
> > > > yesno(timeslice_yield(execlists, last)));
> > > >
> > > > /*
> > > > @@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > GEM_BUG_ON(rq->engine != &ve->base);
> > > > GEM_BUG_ON(rq->context != &ve->context);
> > > >
> > > > - if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
> > > > + if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> > > > spin_unlock(&ve->base.active.lock);
> > > > break;
> > > > }
> > > > @@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > yesno(engine != ve->siblings[0]));
> > > >
> > > > WRITE_ONCE(ve->request, NULL);
> > > > - WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
> > > > + WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
> > > >
> > > > rb = &ve->nodes[engine->id].rb;
> > > > rb_erase_cached(rb, &execlists->virtual);
> > > > @@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > break;
> > > > }
> > > >
> > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > struct i915_priolist *p = to_priolist(rb);
> > > > struct i915_request *rq, *rn;
> > > >
> > > > @@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > }
> > > > }
> > > >
> > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > i915_priolist_free(p);
> > > > }
> > > > done:
> > > > @@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > * request triggering preemption on the next dequeue (or subsequent
> > > > * interrupt for secondary ports).
> > > > */
> > > > - execlists->queue_priority_hint = queue_prio(execlists);
> > > > + sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > > > spin_unlock(&engine->active.lock);
> > > >
> > > > /*
> > > > @@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> > > > */
> > > > __execlists_unhold(rq);
> > > >
> > > > - if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
> > > > - engine->execlists.queue_priority_hint = rq_prio(rq);
> > > > + if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> > > > + engine->sched_engine->queue_priority_hint = rq_prio(rq);
> > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > > }
> > > >
> > > > @@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
> > > > static bool submit_queue(struct intel_engine_cs *engine,
> > > > const struct i915_request *rq)
> > > > {
> > > > - struct intel_engine_execlists *execlists = &engine->execlists;
> > > > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> > > >
> > > > - if (rq_prio(rq) <= execlists->queue_priority_hint)
> > > > + if (rq_prio(rq) <= sched_engine->queue_priority_hint)
> > > > return false;
> > > >
> > > > - execlists->queue_priority_hint = rq_prio(rq);
> > > > + sched_engine->queue_priority_hint = rq_prio(rq);
> > > > return true;
> > > > }
> > > >
> > > > @@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> > > > } else {
> > > > queue_request(engine, request);
> > > >
> > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > > GEM_BUG_ON(list_empty(&request->sched.link));
> > > >
> > > > if (submit_queue(engine, request))
> > > > @@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
> > > > from_tasklet(engine, t, execlists.tasklet);
> > > >
> > > > /* The driver is wedged; don't process any more events. */
> > > > - WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
> > > > + WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> > > > }
> > > >
> > > > static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > {
> > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > struct i915_request *rq, *rn;
> > > > struct rb_node *rb;
> > > > unsigned long flags;
> > > > @@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > intel_engine_signal_breadcrumbs(engine);
> > > >
> > > > /* Flush the queued requests to the timeline list (for retiring). */
> > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > struct i915_priolist *p = to_priolist(rb);
> > > >
> > > > priolist_for_each_request_consume(rq, rn, p) {
> > > > @@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > }
> > > > }
> > > >
> > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > i915_priolist_free(p);
> > > > }
> > > >
> > > > @@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > }
> > > > i915_request_put(rq);
> > > >
> > > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > > + ve->base.sched_engine->queue_priority_hint = INT_MIN;
> > > > }
> > > > spin_unlock(&ve->base.active.lock);
> > > > }
> > > >
> > > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > > >
> > > > - execlists->queue_priority_hint = INT_MIN;
> > > > - execlists->queue = RB_ROOT_CACHED;
> > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > >
> > > > GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > > > execlists->tasklet.callback = nop_submission_tasklet;
> > > > @@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> > > >
> > > > static struct list_head *virtual_queue(struct virtual_engine *ve)
> > > > {
> > > > - return &ve->base.execlists.default_priolist.requests;
> > > > + return &ve->base.sched_engine->default_priolist.requests;
> > > > }
> > > >
> > > > static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > > @@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > > lrc_fini(&ve->context);
> > > > intel_context_fini(&ve->context);
> > > >
> > > > - intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > > + if (ve->base.breadcrumbs)
> > > > + intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > > + if (ve->base.sched_engine)
> > > > + i915_sched_engine_put(ve->base.sched_engine);
> > >
> > > Will either of these ever actually be NULL? The first line or two
> > > seems to indicate that breadcrumbs never is.
> > >
> > > > intel_engine_free_request_pool(&ve->base);
> > > >
> > > > kfree(ve->bonds);
> > > > @@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
> > > >
> > > > ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
> > > > rq->fence.context, rq->fence.seqno,
> > > > - mask, ve->base.execlists.queue_priority_hint);
> > > > + mask, ve->base.sched_engine->queue_priority_hint);
> > > >
> > > > return mask;
> > > > }
> > > > @@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > > {
> > > > struct virtual_engine * const ve =
> > > > from_tasklet(ve, t, base.execlists.tasklet);
> > > > - const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
> > > > + const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> > > > intel_engine_mask_t mask;
> > > > unsigned int n;
> > > >
> > > > @@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > > submit_engine:
> > > > GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> > > > node->prio = prio;
> > > > - if (first && prio > sibling->execlists.queue_priority_hint)
> > > > + if (first && prio > sibling->sched_engine->queue_priority_hint)
> > > > tasklet_hi_schedule(&sibling->execlists.tasklet);
> > > >
> > > > unlock_engine:
> > > > @@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > > > i915_request_put(ve->request);
> > > > }
> > > >
> > > > - ve->base.execlists.queue_priority_hint = rq_prio(rq);
> > > > + ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
> > > > ve->request = i915_request_get(rq);
> > > >
> > > > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > > > @@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > > intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> > > > intel_engine_init_execlists(&ve->base);
> > > >
> > > > + ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> > > > + if (!ve->base.sched_engine) {
> > > > + err = -ENOMEM;
> > > > + goto err_put;
> > > > + }
> > > > +
> > > > ve->base.cops = &virtual_context_ops;
> > > > ve->base.request_alloc = execlists_request_alloc;
> > > >
> > > > @@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > > ve->base.bond_execute = virtual_bond_execute;
> > > >
> > > > INIT_LIST_HEAD(virtual_queue(ve));
> > > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > > tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> > > >
> > > > intel_context_init(&ve->context, &ve->base);
> > > > @@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > > unsigned int max)
> > > > {
> > > > const struct intel_engine_execlists *execlists = &engine->execlists;
> > > > + const struct i915_sched_engine *sched_engine = engine->sched_engine;
> > > > struct i915_request *rq, *last;
> > > > unsigned long flags;
> > > > unsigned int count;
> > > > @@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > > show_request(m, last, "\t\t", 0);
> > > > }
> > > >
> > > > - if (execlists->queue_priority_hint != INT_MIN)
> > > > + if (sched_engine->queue_priority_hint != INT_MIN)
> > > > drm_printf(m, "\t\tQueue priority hint: %d\n",
> > > > - READ_ONCE(execlists->queue_priority_hint));
> > > > + READ_ONCE(sched_engine->queue_priority_hint));
> > > >
> > > > last = NULL;
> > > > count = 0;
> > > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > > struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
> > > >
> > > > priolist_for_each_request(rq, p) {
> > > > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > index 32589c6625e1..b1fdba13e900 100644
> > > > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > @@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
> > > >
> > > > GEM_BUG_ON(timer_pending(&mock->hw_delay));
> > > >
> > > > + i915_sched_engine_put(engine->sched_engine);
> > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > >
> > > > intel_context_unpin(engine->kernel_context);
> > > > @@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > > {
> > > > struct intel_context *ce;
> > > >
> > > > + engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> > > > + if (!engine->sched_engine)
> > > > + return -ENOMEM;
> > > > +
> > > > intel_engine_init_active(engine, ENGINE_MOCK);
> > > > intel_engine_init_execlists(engine);
> > > > intel_engine_init__pm(engine);
> > > > @@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > >
> > > > engine->breadcrumbs = intel_breadcrumbs_create(NULL);
> > > > if (!engine->breadcrumbs)
> > > > - return -ENOMEM;
> > > > + goto err_schedule;
> > > >
> > > > ce = create_kernel_context(engine);
> > > > if (IS_ERR(ce))
> > > > @@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > >
> > > > err_breadcrumbs:
> > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > +err_schedule:
> > > > + i915_sched_engine_put(engine->sched_engine);
> > > > return -ENOMEM;
> > > > }
> > > >
> > > > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > index 335719f17490..d42dea79ee64 100644
> > > > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > @@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
> > > > static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > {
> > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > struct i915_request **first = execlists->inflight;
> > > > struct i915_request ** const last_port = first + execlists->port_mask;
> > > > struct i915_request *last = first[0];
> > > > @@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > * event.
> > > > */
> > > > port = first;
> > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > struct i915_priolist *p = to_priolist(rb);
> > > > struct i915_request *rq, *rn;
> > > >
> > > > @@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > last = rq;
> > > > }
> > > >
> > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > i915_priolist_free(p);
> > > > }
> > > > done:
> > > > - execlists->queue_priority_hint =
> > > > + sched_engine->queue_priority_hint =
> > > > rb ? to_priolist(rb)->priority : INT_MIN;
> > > > if (submit) {
> > > > *port = schedule_in(last, port - execlists->inflight);
> > > > @@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > > >
> > > > static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > {
> > > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > struct i915_request *rq, *rn;
> > > > struct rb_node *rb;
> > > > unsigned long flags;
> > > > @@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > }
> > > >
> > > > /* Flush the queued requests to the timeline list (for retiring). */
> > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > struct i915_priolist *p = to_priolist(rb);
> > > >
> > > > priolist_for_each_request_consume(rq, rn, p) {
> > > > @@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > i915_request_mark_complete(rq);
> > > > }
> > > >
> > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > i915_priolist_free(p);
> > > > }
> > > >
> > > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > > >
> > > > - execlists->queue_priority_hint = INT_MIN;
> > > > - execlists->queue = RB_ROOT_CACHED;
> > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > >
> > > > spin_unlock_irqrestore(&engine->active.lock, flags);
> > > > }
> > > > @@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
> > > >
> > > > queue_request(engine, rq, rq_prio(rq));
> > > >
> > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > > GEM_BUG_ON(list_empty(&rq->sched.link));
> > > >
> > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > > > index efa638c3acc7..4953874a9ba6 100644
> > > > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > > > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > > > @@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> > > > return rb_entry(rb, struct i915_priolist, node);
> > > > }
> > > >
> > > > -static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > +static void assert_priolists(struct i915_sched_engine * const sched_engine)
> > > > {
> > > > struct rb_node *rb;
> > > > long last_prio;
> > > > @@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> > > > return;
> > > >
> > > > - GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
> > > > - rb_first(&execlists->queue.rb_root));
> > > > + GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
> > > > + rb_first(&sched_engine->queue.rb_root));
> > > >
> > > > last_prio = INT_MAX;
> > > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > > const struct i915_priolist *p = to_priolist(rb);
> > > >
> > > > GEM_BUG_ON(p->priority > last_prio);
> > > > @@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > struct list_head *
> > > > i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > {
> > > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > struct i915_priolist *p;
> > > > struct rb_node **parent, *rb;
> > > > bool first = true;
> > > >
> > > > lockdep_assert_held(&engine->active.lock);
> > > > - assert_priolists(execlists);
> > > > + assert_priolists(sched_engine);
> > > >
> > > > - if (unlikely(execlists->no_priolist))
> > > > + if (unlikely(sched_engine->no_priolist))
> > > > prio = I915_PRIORITY_NORMAL;
> > > >
> > > > find_priolist:
> > > > /* most positive priority is scheduled first, equal priorities fifo */
> > > > rb = NULL;
> > > > - parent = &execlists->queue.rb_root.rb_node;
> > > > + parent = &sched_engine->queue.rb_root.rb_node;
> > > > while (*parent) {
> > > > rb = *parent;
> > > > p = to_priolist(rb);
> > > > @@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > }
> > > >
> > > > if (prio == I915_PRIORITY_NORMAL) {
> > > > - p = &execlists->default_priolist;
> > > > + p = &sched_engine->default_priolist;
> > > > } else {
> > > > p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
> > > > /* Convert an allocation failure to a priority bump */
> > > > @@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > * requests, so if userspace lied about their
> > > > * dependencies that reordering may be visible.
> > > > */
> > > > - execlists->no_priolist = true;
> > > > + sched_engine->no_priolist = true;
> > > > goto find_priolist;
> > > > }
> > > > }
> > > > @@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > INIT_LIST_HEAD(&p->requests);
> > > >
> > > > rb_link_node(&p->node, rb, parent);
> > > > - rb_insert_color_cached(&p->node, &execlists->queue, first);
> > > > + rb_insert_color_cached(&p->node, &sched_engine->queue, first);
> > > >
> > > > return &p->requests;
> > > > }
> > > > @@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > > * We only need to kick the tasklet once for the high priority
> > > > * new context we add into the queue.
> > > > */
> > > > - if (prio <= engine->execlists.queue_priority_hint)
> > > > + if (prio <= engine->sched_engine->queue_priority_hint)
> > > > return;
> > > >
> > > > rcu_read_lock();
> > > > @@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > > inflight->fence.context, inflight->fence.seqno,
> > > > inflight->sched.attr.priority);
> > > >
> > > > - engine->execlists.queue_priority_hint = prio;
> > > > + engine->sched_engine->queue_priority_hint = prio;
> > > > if (need_preempt(prio, rq_prio(inflight)))
> > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > >
> > > > @@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
> > > > rcu_read_unlock();
> > > > }
> > > >
> > > > +void i915_sched_engine_free(struct kref *kref)
> > > > +{
> > > > + struct i915_sched_engine *sched_engine =
> > > > + container_of(kref, typeof(*sched_engine), ref);
> > > > +
> > > > + kfree(sched_engine);
> > > > +}
> > > > +
> > > > +struct i915_sched_engine *
> > > > +i915_sched_engine_create(unsigned int subclass)
> > > > +{
> > > > + struct i915_sched_engine *sched_engine;
> > > > +
> > > > + sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
> > > > + if (!sched_engine)
> > > > + return NULL;
> > > > +
> > > > + kref_init(&sched_engine->ref);
> > > > +
> > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > +
> > > > + return sched_engine;
> > > > +}
> > > > +
> > > > static void i915_global_scheduler_shrink(void)
> > > > {
> > > > kmem_cache_shrink(global.slab_dependencies);
> > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > > > index 858a0938f47a..91a04e34cac5 100644
> > > > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > > > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > > > @@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
> > > > __i915_priolist_free(p);
> > > > }
> > > >
> > > > +struct i915_sched_engine *
> > > > +i915_sched_engine_create(unsigned int subclass);
> > > > +
> > > > +void i915_sched_engine_free(struct kref *kref);
> > > > +
> > > > +static inline struct i915_sched_engine *
> > > > +i915_sched_engine_get(struct i915_sched_engine *sched_engine)
> > > > +{
> > > > + kref_get(&sched_engine->ref);
> > > > + return sched_engine;
> > > > +}
> > > > +
> > > > +static inline void
> > > > +i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> > > > +{
> > > > + kref_put(&sched_engine->ref, i915_sched_engine_free);
> > > > +}
> > > > +
> > > > void i915_request_show_with_schedule(struct drm_printer *m,
> > > > const struct i915_request *rq,
> > > > const char *prefix,
> > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > > index 343ed44d5ed4..16a054e67760 100644
> > > > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > >
> > > Dumb question for which I'm not really in the position of having an
> > > opinion: Should this go in its own file?
> > >
> >
> > I think this is in the right file.
> >
> > Matt
> >
> > > Patch seems mostly mechanically correct. Not sure where this is going
> > > yet, though. I'll keep reading. :-)
> > >
> > > --Jason
> > >
> > >
> > > > @@ -91,4 +91,37 @@ struct i915_dependency {
> > > > &(rq__)->sched.signalers_list, \
> > > > signal_link)
> > > >
> > > > +struct i915_sched_engine {
> > > > + struct kref ref;
> > > > +
> > > > + /**
> > > > + * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > > + */
> > > > + struct i915_priolist default_priolist;
> > > > +
> > > > + /**
> > > > + * @queue_priority_hint: Highest pending priority.
> > > > + *
> > > > + * When we add requests into the queue, or adjust the priority of
> > > > + * executing requests, we compute the maximum priority of those
> > > > + * pending requests. We can then use this value to determine if
> > > > + * we need to preempt the executing requests to service the queue.
> > > > + * However, since the we may have recorded the priority of an inflight
> > > > + * request we wanted to preempt but since completed, at the time of
> > > > + * dequeuing the priority hint may no longer may match the highest
> > > > + * available request priority.
> > > > + */
> > > > + int queue_priority_hint;
> > > > +
> > > > + /**
> > > > + * @queue: queue of requests, in priority lists
> > > > + */
> > > > + struct rb_root_cached queue;
> > > > +
> > > > + /**
> > > > + * @no_priolist: priority lists disabled
> > > > + */
> > > > + bool no_priolist;
> > > > +};
> > > > +
> > > > #endif /* _I915_SCHEDULER_TYPES_H_ */
> > > > --
> > > > 2.28.0
> > > >
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
@ 2021-06-04 17:51 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 17:51 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 12:51:43PM -0500, Jason Ekstrand wrote:
> On Fri, Jun 4, 2021 at 12:42 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > On Fri, Jun 04, 2021 at 12:38:22PM -0500, Jason Ekstrand wrote:
> > > On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> > > >
> > > > Introduce i915_sched_engine object which is lower level data structure
> > > > that i915_scheduler / generic code can operate on without touching
> > > > execlist specific structures. This allows additional submission backends
> > > > to be added without breaking the layering.
> > > >
> > > > This is a bit of detour to integrating the i915 with the DRM scheduler
> > > > but this object will still exist when the DRM scheduler lands in the
> > > > i915. It will however look a bit different. It will encapsulate the
> > > > drm_gpu_scheduler object plus and common variables (to the backends)
> > > > related to scheduling. Regardless this is a step in the right direction.
> > > >
> > > > This patch starts the aforementioned transition by moving the the
> > > > priolist into the i915_sched_engine object.
> > > >
> > > > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > > > ---
> > > > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
> > > > drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
> > > > drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
> > > > .../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
> > > > drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
> > > > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
> > > > drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
> > > > drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
> > > > drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
> > > > 9 files changed, 169 insertions(+), 90 deletions(-)
> > > >
> > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > index 3f9a811eb02b..d0f3814440f6 100644
> > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > @@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
> > > > memset(execlists->pending, 0, sizeof(execlists->pending));
> > > > execlists->active =
> > > > memset(execlists->inflight, 0, sizeof(execlists->inflight));
> > > > -
> > > > - execlists->queue_priority_hint = INT_MIN;
> > > > - execlists->queue = RB_ROOT_CACHED;
> > > > }
> > > >
> > > > static void cleanup_status_page(struct intel_engine_cs *engine)
> > > > @@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > > goto err_status;
> > > > }
> > > >
> > > > + engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
> > > > + if (!engine->sched_engine) {
> > > > + err = -ENOMEM;
> > > > + goto err_sched_engine;
> > > > + }
> > > > +
> > > > err = intel_engine_init_cmd_parser(engine);
> > > > if (err)
> > > > goto err_cmd_parser;
> > > > @@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > > return 0;
> > > >
> > > > err_cmd_parser:
> > > > + i915_sched_engine_put(engine->sched_engine);
> > > > +err_sched_engine:
> > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > err_status:
> > > > cleanup_status_page(engine);
> > > > @@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > > > GEM_BUG_ON(!list_empty(&engine->active.requests));
> > > > tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> > > >
> > > > + i915_sched_engine_put(engine->sched_engine);
> > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > >
> > > > intel_engine_fini_retire(engine);
> > > > @@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> > > > intel_engine_flush_submission(engine);
> > > >
> > > > /* ELSP is empty, but there are ready requests? E.g. after reset */
> > > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
> > > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> > > > return false;
> > > >
> > > > /* Ring stopped? */
> > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > index 47f4397095e5..b6a00dd72808 100644
> > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > @@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
> > > > intel_breadcrumbs_park(engine->breadcrumbs);
> > > >
> > > > /* Must be reset upon idling, or we may miss the busy wakeup. */
> > > > - GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
> > > > + GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
> > > >
> > > > if (engine->park)
> > > > engine->park(engine);
> > > >
> > > > - engine->execlists.no_priolist = false;
> > > > + engine->sched_engine->no_priolist = false;
> > > >
> > > > /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> > > > intel_gt_pm_put_async(engine->gt);
> > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > index 9ef349cd5cea..86b41ddec373 100644
> > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > @@ -59,6 +59,7 @@ struct drm_i915_reg_table;
> > > > struct i915_gem_context;
> > > > struct i915_request;
> > > > struct i915_sched_attr;
> > > > +struct i915_sched_engine;
> > > > struct intel_gt;
> > > > struct intel_ring;
> > > > struct intel_uncore;
> > > > @@ -152,11 +153,6 @@ struct intel_engine_execlists {
> > > > */
> > > > struct timer_list preempt;
> > > >
> > > > - /**
> > > > - * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > > - */
> > > > - struct i915_priolist default_priolist;
> > > > -
> > > > /**
> > > > * @ccid: identifier for contexts submitted to this engine
> > > > */
> > > > @@ -191,11 +187,6 @@ struct intel_engine_execlists {
> > > > */
> > > > u32 reset_ccid;
> > > >
> > > > - /**
> > > > - * @no_priolist: priority lists disabled
> > > > - */
> > > > - bool no_priolist;
> > > > -
> > > > /**
> > > > * @submit_reg: gen-specific execlist submission register
> > > > * set to the ExecList Submission Port (elsp) register pre-Gen11 and to
> > > > @@ -238,23 +229,8 @@ struct intel_engine_execlists {
> > > > unsigned int port_mask;
> > > >
> > > > /**
> > > > - * @queue_priority_hint: Highest pending priority.
> > > > - *
> > > > - * When we add requests into the queue, or adjust the priority of
> > > > - * executing requests, we compute the maximum priority of those
> > > > - * pending requests. We can then use this value to determine if
> > > > - * we need to preempt the executing requests to service the queue.
> > > > - * However, since the we may have recorded the priority of an inflight
> > > > - * request we wanted to preempt but since completed, at the time of
> > > > - * dequeuing the priority hint may no longer may match the highest
> > > > - * available request priority.
> > > > + * @virtual: virtual of requests, in priority lists
> > >
> > > Having trouble parsing this. Did you mean "queue of virtual
> > > requests"? Or maybe "virtual queue of requests"?
> > >
> >
> > Yep, this doesn't really make sense. How about:
> > 'queue of requests on a virtual engine, in priority lists'
>
> I'm not sure what "in priority lists" means.
>
Ah, this is confusing to explain. Let me try to this again.
'Queue of requets on a virtual engine, sorted by priority. Each RB entry
is a i915_priolist containing a list of requests of the same priority.'
Matt
> --Jason
>
> > > > */
> > > > - int queue_priority_hint;
> > > > -
> > > > - /**
> > > > - * @queue: queue of requests, in priority lists
> > > > - */
> > > > - struct rb_root_cached queue;
> > > > struct rb_root_cached virtual;
> > > >
> > > > /**
> > > > @@ -332,6 +308,8 @@ struct intel_engine_cs {
> > > > struct list_head hold; /* ready requests, but on hold */
> > > > } active;
> > > >
> > > > + struct i915_sched_engine *sched_engine;
> > > > +
> > > > /* keep a request in reserve for a [pm] barrier under oom */
> > > > struct i915_request *request_pool;
> > > >
> > > > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > index 8db200422950..d1dc1db3e378 100644
> > > > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > @@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
> > > > return prio;
> > > > }
> > > >
> > > > -static int queue_prio(const struct intel_engine_execlists *execlists)
> > > > +static int queue_prio(const struct i915_sched_engine *sched_engine)
> > > > {
> > > > struct rb_node *rb;
> > > >
> > > > - rb = rb_first_cached(&execlists->queue);
> > > > + rb = rb_first_cached(&sched_engine->queue);
> > > > if (!rb)
> > > > return INT_MIN;
> > > >
> > > > @@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > > * to preserve FIFO ordering of dependencies.
> > > > */
> > > > last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
> > > > - if (engine->execlists.queue_priority_hint <= last_prio)
> > > > + if (engine->sched_engine->queue_priority_hint <= last_prio)
> > > > return false;
> > > >
> > > > /*
> > > > @@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > > * context, it's priority would not exceed ELSP[0] aka last_prio.
> > > > */
> > > > return max(virtual_prio(&engine->execlists),
> > > > - queue_prio(&engine->execlists)) > last_prio;
> > > > + queue_prio(engine->sched_engine)) > last_prio;
> > > > }
> > > >
> > > > __maybe_unused static bool
> > > > @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > > > prio = rq_prio(rq);
> > > > pl = i915_sched_lookup_priolist(engine, prio);
> > > > }
> > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > >
> > > > list_move(&rq->sched.link, pl);
> > > > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > > > @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> > > > }
> > > >
> > > > /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> > > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
> > > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> > > > ENGINE_TRACE(engine, "timeslice required for queue\n");
> > > > return true;
> > > > }
> > > > @@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
> > > > static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > {
> > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > struct i915_request **port = execlists->pending;
> > > > struct i915_request ** const last_port = port + execlists->port_mask;
> > > > struct i915_request *last, * const *active;
> > > > @@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > last->fence.context,
> > > > last->fence.seqno,
> > > > last->sched.attr.priority,
> > > > - execlists->queue_priority_hint);
> > > > + sched_engine->queue_priority_hint);
> > > > record_preemption(execlists);
> > > >
> > > > /*
> > > > @@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > yesno(timer_expired(&execlists->timer)),
> > > > last->fence.context, last->fence.seqno,
> > > > rq_prio(last),
> > > > - execlists->queue_priority_hint,
> > > > + sched_engine->queue_priority_hint,
> > > > yesno(timeslice_yield(execlists, last)));
> > > >
> > > > /*
> > > > @@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > GEM_BUG_ON(rq->engine != &ve->base);
> > > > GEM_BUG_ON(rq->context != &ve->context);
> > > >
> > > > - if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
> > > > + if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> > > > spin_unlock(&ve->base.active.lock);
> > > > break;
> > > > }
> > > > @@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > yesno(engine != ve->siblings[0]));
> > > >
> > > > WRITE_ONCE(ve->request, NULL);
> > > > - WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
> > > > + WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
> > > >
> > > > rb = &ve->nodes[engine->id].rb;
> > > > rb_erase_cached(rb, &execlists->virtual);
> > > > @@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > break;
> > > > }
> > > >
> > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > struct i915_priolist *p = to_priolist(rb);
> > > > struct i915_request *rq, *rn;
> > > >
> > > > @@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > }
> > > > }
> > > >
> > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > i915_priolist_free(p);
> > > > }
> > > > done:
> > > > @@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > * request triggering preemption on the next dequeue (or subsequent
> > > > * interrupt for secondary ports).
> > > > */
> > > > - execlists->queue_priority_hint = queue_prio(execlists);
> > > > + sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > > > spin_unlock(&engine->active.lock);
> > > >
> > > > /*
> > > > @@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> > > > */
> > > > __execlists_unhold(rq);
> > > >
> > > > - if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
> > > > - engine->execlists.queue_priority_hint = rq_prio(rq);
> > > > + if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> > > > + engine->sched_engine->queue_priority_hint = rq_prio(rq);
> > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > > }
> > > >
> > > > @@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
> > > > static bool submit_queue(struct intel_engine_cs *engine,
> > > > const struct i915_request *rq)
> > > > {
> > > > - struct intel_engine_execlists *execlists = &engine->execlists;
> > > > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> > > >
> > > > - if (rq_prio(rq) <= execlists->queue_priority_hint)
> > > > + if (rq_prio(rq) <= sched_engine->queue_priority_hint)
> > > > return false;
> > > >
> > > > - execlists->queue_priority_hint = rq_prio(rq);
> > > > + sched_engine->queue_priority_hint = rq_prio(rq);
> > > > return true;
> > > > }
> > > >
> > > > @@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> > > > } else {
> > > > queue_request(engine, request);
> > > >
> > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > > GEM_BUG_ON(list_empty(&request->sched.link));
> > > >
> > > > if (submit_queue(engine, request))
> > > > @@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
> > > > from_tasklet(engine, t, execlists.tasklet);
> > > >
> > > > /* The driver is wedged; don't process any more events. */
> > > > - WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
> > > > + WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> > > > }
> > > >
> > > > static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > {
> > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > struct i915_request *rq, *rn;
> > > > struct rb_node *rb;
> > > > unsigned long flags;
> > > > @@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > intel_engine_signal_breadcrumbs(engine);
> > > >
> > > > /* Flush the queued requests to the timeline list (for retiring). */
> > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > struct i915_priolist *p = to_priolist(rb);
> > > >
> > > > priolist_for_each_request_consume(rq, rn, p) {
> > > > @@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > }
> > > > }
> > > >
> > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > i915_priolist_free(p);
> > > > }
> > > >
> > > > @@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > }
> > > > i915_request_put(rq);
> > > >
> > > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > > + ve->base.sched_engine->queue_priority_hint = INT_MIN;
> > > > }
> > > > spin_unlock(&ve->base.active.lock);
> > > > }
> > > >
> > > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > > >
> > > > - execlists->queue_priority_hint = INT_MIN;
> > > > - execlists->queue = RB_ROOT_CACHED;
> > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > >
> > > > GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > > > execlists->tasklet.callback = nop_submission_tasklet;
> > > > @@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> > > >
> > > > static struct list_head *virtual_queue(struct virtual_engine *ve)
> > > > {
> > > > - return &ve->base.execlists.default_priolist.requests;
> > > > + return &ve->base.sched_engine->default_priolist.requests;
> > > > }
> > > >
> > > > static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > > @@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > > lrc_fini(&ve->context);
> > > > intel_context_fini(&ve->context);
> > > >
> > > > - intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > > + if (ve->base.breadcrumbs)
> > > > + intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > > + if (ve->base.sched_engine)
> > > > + i915_sched_engine_put(ve->base.sched_engine);
> > >
> > > Will either of these ever actually be NULL? The first line or two
> > > seems to indicate that breadcrumbs never is.
> > >
> > > > intel_engine_free_request_pool(&ve->base);
> > > >
> > > > kfree(ve->bonds);
> > > > @@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
> > > >
> > > > ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
> > > > rq->fence.context, rq->fence.seqno,
> > > > - mask, ve->base.execlists.queue_priority_hint);
> > > > + mask, ve->base.sched_engine->queue_priority_hint);
> > > >
> > > > return mask;
> > > > }
> > > > @@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > > {
> > > > struct virtual_engine * const ve =
> > > > from_tasklet(ve, t, base.execlists.tasklet);
> > > > - const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
> > > > + const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> > > > intel_engine_mask_t mask;
> > > > unsigned int n;
> > > >
> > > > @@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > > submit_engine:
> > > > GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> > > > node->prio = prio;
> > > > - if (first && prio > sibling->execlists.queue_priority_hint)
> > > > + if (first && prio > sibling->sched_engine->queue_priority_hint)
> > > > tasklet_hi_schedule(&sibling->execlists.tasklet);
> > > >
> > > > unlock_engine:
> > > > @@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > > > i915_request_put(ve->request);
> > > > }
> > > >
> > > > - ve->base.execlists.queue_priority_hint = rq_prio(rq);
> > > > + ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
> > > > ve->request = i915_request_get(rq);
> > > >
> > > > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > > > @@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > > intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> > > > intel_engine_init_execlists(&ve->base);
> > > >
> > > > + ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> > > > + if (!ve->base.sched_engine) {
> > > > + err = -ENOMEM;
> > > > + goto err_put;
> > > > + }
> > > > +
> > > > ve->base.cops = &virtual_context_ops;
> > > > ve->base.request_alloc = execlists_request_alloc;
> > > >
> > > > @@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > > ve->base.bond_execute = virtual_bond_execute;
> > > >
> > > > INIT_LIST_HEAD(virtual_queue(ve));
> > > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > > tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> > > >
> > > > intel_context_init(&ve->context, &ve->base);
> > > > @@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > > unsigned int max)
> > > > {
> > > > const struct intel_engine_execlists *execlists = &engine->execlists;
> > > > + const struct i915_sched_engine *sched_engine = engine->sched_engine;
> > > > struct i915_request *rq, *last;
> > > > unsigned long flags;
> > > > unsigned int count;
> > > > @@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > > show_request(m, last, "\t\t", 0);
> > > > }
> > > >
> > > > - if (execlists->queue_priority_hint != INT_MIN)
> > > > + if (sched_engine->queue_priority_hint != INT_MIN)
> > > > drm_printf(m, "\t\tQueue priority hint: %d\n",
> > > > - READ_ONCE(execlists->queue_priority_hint));
> > > > + READ_ONCE(sched_engine->queue_priority_hint));
> > > >
> > > > last = NULL;
> > > > count = 0;
> > > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > > struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
> > > >
> > > > priolist_for_each_request(rq, p) {
> > > > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > index 32589c6625e1..b1fdba13e900 100644
> > > > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > @@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
> > > >
> > > > GEM_BUG_ON(timer_pending(&mock->hw_delay));
> > > >
> > > > + i915_sched_engine_put(engine->sched_engine);
> > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > >
> > > > intel_context_unpin(engine->kernel_context);
> > > > @@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > > {
> > > > struct intel_context *ce;
> > > >
> > > > + engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> > > > + if (!engine->sched_engine)
> > > > + return -ENOMEM;
> > > > +
> > > > intel_engine_init_active(engine, ENGINE_MOCK);
> > > > intel_engine_init_execlists(engine);
> > > > intel_engine_init__pm(engine);
> > > > @@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > >
> > > > engine->breadcrumbs = intel_breadcrumbs_create(NULL);
> > > > if (!engine->breadcrumbs)
> > > > - return -ENOMEM;
> > > > + goto err_schedule;
> > > >
> > > > ce = create_kernel_context(engine);
> > > > if (IS_ERR(ce))
> > > > @@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > >
> > > > err_breadcrumbs:
> > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > +err_schedule:
> > > > + i915_sched_engine_put(engine->sched_engine);
> > > > return -ENOMEM;
> > > > }
> > > >
> > > > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > index 335719f17490..d42dea79ee64 100644
> > > > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > @@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
> > > > static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > {
> > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > struct i915_request **first = execlists->inflight;
> > > > struct i915_request ** const last_port = first + execlists->port_mask;
> > > > struct i915_request *last = first[0];
> > > > @@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > * event.
> > > > */
> > > > port = first;
> > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > struct i915_priolist *p = to_priolist(rb);
> > > > struct i915_request *rq, *rn;
> > > >
> > > > @@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > last = rq;
> > > > }
> > > >
> > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > i915_priolist_free(p);
> > > > }
> > > > done:
> > > > - execlists->queue_priority_hint =
> > > > + sched_engine->queue_priority_hint =
> > > > rb ? to_priolist(rb)->priority : INT_MIN;
> > > > if (submit) {
> > > > *port = schedule_in(last, port - execlists->inflight);
> > > > @@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > > >
> > > > static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > {
> > > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > struct i915_request *rq, *rn;
> > > > struct rb_node *rb;
> > > > unsigned long flags;
> > > > @@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > }
> > > >
> > > > /* Flush the queued requests to the timeline list (for retiring). */
> > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > struct i915_priolist *p = to_priolist(rb);
> > > >
> > > > priolist_for_each_request_consume(rq, rn, p) {
> > > > @@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > i915_request_mark_complete(rq);
> > > > }
> > > >
> > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > i915_priolist_free(p);
> > > > }
> > > >
> > > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > > >
> > > > - execlists->queue_priority_hint = INT_MIN;
> > > > - execlists->queue = RB_ROOT_CACHED;
> > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > >
> > > > spin_unlock_irqrestore(&engine->active.lock, flags);
> > > > }
> > > > @@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
> > > >
> > > > queue_request(engine, rq, rq_prio(rq));
> > > >
> > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > > GEM_BUG_ON(list_empty(&rq->sched.link));
> > > >
> > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > > > index efa638c3acc7..4953874a9ba6 100644
> > > > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > > > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > > > @@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> > > > return rb_entry(rb, struct i915_priolist, node);
> > > > }
> > > >
> > > > -static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > +static void assert_priolists(struct i915_sched_engine * const sched_engine)
> > > > {
> > > > struct rb_node *rb;
> > > > long last_prio;
> > > > @@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> > > > return;
> > > >
> > > > - GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
> > > > - rb_first(&execlists->queue.rb_root));
> > > > + GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
> > > > + rb_first(&sched_engine->queue.rb_root));
> > > >
> > > > last_prio = INT_MAX;
> > > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > > const struct i915_priolist *p = to_priolist(rb);
> > > >
> > > > GEM_BUG_ON(p->priority > last_prio);
> > > > @@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > struct list_head *
> > > > i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > {
> > > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > struct i915_priolist *p;
> > > > struct rb_node **parent, *rb;
> > > > bool first = true;
> > > >
> > > > lockdep_assert_held(&engine->active.lock);
> > > > - assert_priolists(execlists);
> > > > + assert_priolists(sched_engine);
> > > >
> > > > - if (unlikely(execlists->no_priolist))
> > > > + if (unlikely(sched_engine->no_priolist))
> > > > prio = I915_PRIORITY_NORMAL;
> > > >
> > > > find_priolist:
> > > > /* most positive priority is scheduled first, equal priorities fifo */
> > > > rb = NULL;
> > > > - parent = &execlists->queue.rb_root.rb_node;
> > > > + parent = &sched_engine->queue.rb_root.rb_node;
> > > > while (*parent) {
> > > > rb = *parent;
> > > > p = to_priolist(rb);
> > > > @@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > }
> > > >
> > > > if (prio == I915_PRIORITY_NORMAL) {
> > > > - p = &execlists->default_priolist;
> > > > + p = &sched_engine->default_priolist;
> > > > } else {
> > > > p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
> > > > /* Convert an allocation failure to a priority bump */
> > > > @@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > * requests, so if userspace lied about their
> > > > * dependencies that reordering may be visible.
> > > > */
> > > > - execlists->no_priolist = true;
> > > > + sched_engine->no_priolist = true;
> > > > goto find_priolist;
> > > > }
> > > > }
> > > > @@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > INIT_LIST_HEAD(&p->requests);
> > > >
> > > > rb_link_node(&p->node, rb, parent);
> > > > - rb_insert_color_cached(&p->node, &execlists->queue, first);
> > > > + rb_insert_color_cached(&p->node, &sched_engine->queue, first);
> > > >
> > > > return &p->requests;
> > > > }
> > > > @@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > > * We only need to kick the tasklet once for the high priority
> > > > * new context we add into the queue.
> > > > */
> > > > - if (prio <= engine->execlists.queue_priority_hint)
> > > > + if (prio <= engine->sched_engine->queue_priority_hint)
> > > > return;
> > > >
> > > > rcu_read_lock();
> > > > @@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > > inflight->fence.context, inflight->fence.seqno,
> > > > inflight->sched.attr.priority);
> > > >
> > > > - engine->execlists.queue_priority_hint = prio;
> > > > + engine->sched_engine->queue_priority_hint = prio;
> > > > if (need_preempt(prio, rq_prio(inflight)))
> > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > >
> > > > @@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
> > > > rcu_read_unlock();
> > > > }
> > > >
> > > > +void i915_sched_engine_free(struct kref *kref)
> > > > +{
> > > > + struct i915_sched_engine *sched_engine =
> > > > + container_of(kref, typeof(*sched_engine), ref);
> > > > +
> > > > + kfree(sched_engine);
> > > > +}
> > > > +
> > > > +struct i915_sched_engine *
> > > > +i915_sched_engine_create(unsigned int subclass)
> > > > +{
> > > > + struct i915_sched_engine *sched_engine;
> > > > +
> > > > + sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
> > > > + if (!sched_engine)
> > > > + return NULL;
> > > > +
> > > > + kref_init(&sched_engine->ref);
> > > > +
> > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > +
> > > > + return sched_engine;
> > > > +}
> > > > +
> > > > static void i915_global_scheduler_shrink(void)
> > > > {
> > > > kmem_cache_shrink(global.slab_dependencies);
> > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > > > index 858a0938f47a..91a04e34cac5 100644
> > > > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > > > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > > > @@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
> > > > __i915_priolist_free(p);
> > > > }
> > > >
> > > > +struct i915_sched_engine *
> > > > +i915_sched_engine_create(unsigned int subclass);
> > > > +
> > > > +void i915_sched_engine_free(struct kref *kref);
> > > > +
> > > > +static inline struct i915_sched_engine *
> > > > +i915_sched_engine_get(struct i915_sched_engine *sched_engine)
> > > > +{
> > > > + kref_get(&sched_engine->ref);
> > > > + return sched_engine;
> > > > +}
> > > > +
> > > > +static inline void
> > > > +i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> > > > +{
> > > > + kref_put(&sched_engine->ref, i915_sched_engine_free);
> > > > +}
> > > > +
> > > > void i915_request_show_with_schedule(struct drm_printer *m,
> > > > const struct i915_request *rq,
> > > > const char *prefix,
> > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > > index 343ed44d5ed4..16a054e67760 100644
> > > > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > >
> > > Dumb question for which I'm not really in the position of having an
> > > opinion: Should this go in its own file?
> > >
> >
> > I think this is in the right file.
> >
> > Matt
> >
> > > Patch seems mostly mechanically correct. Not sure where this is going
> > > yet, though. I'll keep reading. :-)
> > >
> > > --Jason
> > >
> > >
> > > > @@ -91,4 +91,37 @@ struct i915_dependency {
> > > > &(rq__)->sched.signalers_list, \
> > > > signal_link)
> > > >
> > > > +struct i915_sched_engine {
> > > > + struct kref ref;
> > > > +
> > > > + /**
> > > > + * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > > + */
> > > > + struct i915_priolist default_priolist;
> > > > +
> > > > + /**
> > > > + * @queue_priority_hint: Highest pending priority.
> > > > + *
> > > > + * When we add requests into the queue, or adjust the priority of
> > > > + * executing requests, we compute the maximum priority of those
> > > > + * pending requests. We can then use this value to determine if
> > > > + * we need to preempt the executing requests to service the queue.
> > > > + * However, since the we may have recorded the priority of an inflight
> > > > + * request we wanted to preempt but since completed, at the time of
> > > > + * dequeuing the priority hint may no longer may match the highest
> > > > + * available request priority.
> > > > + */
> > > > + int queue_priority_hint;
> > > > +
> > > > + /**
> > > > + * @queue: queue of requests, in priority lists
> > > > + */
> > > > + struct rb_root_cached queue;
> > > > +
> > > > + /**
> > > > + * @no_priolist: priority lists disabled
> > > > + */
> > > > + bool no_priolist;
> > > > +};
> > > > +
> > > > #endif /* _I915_SCHEDULER_TYPES_H_ */
> > > > --
> > > > 2.28.0
> > > >
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
2021-06-04 17:51 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 18:14 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 18:14 UTC (permalink / raw)
To: Matthew Brost
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Fri, Jun 4, 2021 at 12:59 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> On Fri, Jun 04, 2021 at 12:51:43PM -0500, Jason Ekstrand wrote:
> > On Fri, Jun 4, 2021 at 12:42 PM Matthew Brost <matthew.brost@intel.com> wrote:
> > >
> > > On Fri, Jun 04, 2021 at 12:38:22PM -0500, Jason Ekstrand wrote:
> > > > On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> > > > >
> > > > > Introduce i915_sched_engine object which is lower level data structure
> > > > > that i915_scheduler / generic code can operate on without touching
> > > > > execlist specific structures. This allows additional submission backends
> > > > > to be added without breaking the layering.
> > > > >
> > > > > This is a bit of detour to integrating the i915 with the DRM scheduler
> > > > > but this object will still exist when the DRM scheduler lands in the
> > > > > i915. It will however look a bit different. It will encapsulate the
> > > > > drm_gpu_scheduler object plus and common variables (to the backends)
> > > > > related to scheduling. Regardless this is a step in the right direction.
> > > > >
> > > > > This patch starts the aforementioned transition by moving the the
> > > > > priolist into the i915_sched_engine object.
> > > > >
> > > > > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > > > > ---
> > > > > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
> > > > > drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
> > > > > drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
> > > > > .../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
> > > > > drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
> > > > > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
> > > > > drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
> > > > > drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
> > > > > drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
> > > > > 9 files changed, 169 insertions(+), 90 deletions(-)
> > > > >
> > > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > > index 3f9a811eb02b..d0f3814440f6 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > > @@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
> > > > > memset(execlists->pending, 0, sizeof(execlists->pending));
> > > > > execlists->active =
> > > > > memset(execlists->inflight, 0, sizeof(execlists->inflight));
> > > > > -
> > > > > - execlists->queue_priority_hint = INT_MIN;
> > > > > - execlists->queue = RB_ROOT_CACHED;
> > > > > }
> > > > >
> > > > > static void cleanup_status_page(struct intel_engine_cs *engine)
> > > > > @@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > > > goto err_status;
> > > > > }
> > > > >
> > > > > + engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
> > > > > + if (!engine->sched_engine) {
> > > > > + err = -ENOMEM;
> > > > > + goto err_sched_engine;
> > > > > + }
> > > > > +
> > > > > err = intel_engine_init_cmd_parser(engine);
> > > > > if (err)
> > > > > goto err_cmd_parser;
> > > > > @@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > > > return 0;
> > > > >
> > > > > err_cmd_parser:
> > > > > + i915_sched_engine_put(engine->sched_engine);
> > > > > +err_sched_engine:
> > > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > > err_status:
> > > > > cleanup_status_page(engine);
> > > > > @@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > > > > GEM_BUG_ON(!list_empty(&engine->active.requests));
> > > > > tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> > > > >
> > > > > + i915_sched_engine_put(engine->sched_engine);
> > > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > >
> > > > > intel_engine_fini_retire(engine);
> > > > > @@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> > > > > intel_engine_flush_submission(engine);
> > > > >
> > > > > /* ELSP is empty, but there are ready requests? E.g. after reset */
> > > > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
> > > > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> > > > > return false;
> > > > >
> > > > > /* Ring stopped? */
> > > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > > index 47f4397095e5..b6a00dd72808 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > > @@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
> > > > > intel_breadcrumbs_park(engine->breadcrumbs);
> > > > >
> > > > > /* Must be reset upon idling, or we may miss the busy wakeup. */
> > > > > - GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
> > > > > + GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
> > > > >
> > > > > if (engine->park)
> > > > > engine->park(engine);
> > > > >
> > > > > - engine->execlists.no_priolist = false;
> > > > > + engine->sched_engine->no_priolist = false;
> > > > >
> > > > > /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> > > > > intel_gt_pm_put_async(engine->gt);
> > > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > > index 9ef349cd5cea..86b41ddec373 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > > @@ -59,6 +59,7 @@ struct drm_i915_reg_table;
> > > > > struct i915_gem_context;
> > > > > struct i915_request;
> > > > > struct i915_sched_attr;
> > > > > +struct i915_sched_engine;
> > > > > struct intel_gt;
> > > > > struct intel_ring;
> > > > > struct intel_uncore;
> > > > > @@ -152,11 +153,6 @@ struct intel_engine_execlists {
> > > > > */
> > > > > struct timer_list preempt;
> > > > >
> > > > > - /**
> > > > > - * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > > > - */
> > > > > - struct i915_priolist default_priolist;
> > > > > -
> > > > > /**
> > > > > * @ccid: identifier for contexts submitted to this engine
> > > > > */
> > > > > @@ -191,11 +187,6 @@ struct intel_engine_execlists {
> > > > > */
> > > > > u32 reset_ccid;
> > > > >
> > > > > - /**
> > > > > - * @no_priolist: priority lists disabled
> > > > > - */
> > > > > - bool no_priolist;
> > > > > -
> > > > > /**
> > > > > * @submit_reg: gen-specific execlist submission register
> > > > > * set to the ExecList Submission Port (elsp) register pre-Gen11 and to
> > > > > @@ -238,23 +229,8 @@ struct intel_engine_execlists {
> > > > > unsigned int port_mask;
> > > > >
> > > > > /**
> > > > > - * @queue_priority_hint: Highest pending priority.
> > > > > - *
> > > > > - * When we add requests into the queue, or adjust the priority of
> > > > > - * executing requests, we compute the maximum priority of those
> > > > > - * pending requests. We can then use this value to determine if
> > > > > - * we need to preempt the executing requests to service the queue.
> > > > > - * However, since the we may have recorded the priority of an inflight
> > > > > - * request we wanted to preempt but since completed, at the time of
> > > > > - * dequeuing the priority hint may no longer may match the highest
> > > > > - * available request priority.
> > > > > + * @virtual: virtual of requests, in priority lists
> > > >
> > > > Having trouble parsing this. Did you mean "queue of virtual
> > > > requests"? Or maybe "virtual queue of requests"?
> > > >
> > >
> > > Yep, this doesn't really make sense. How about:
> > > 'queue of requests on a virtual engine, in priority lists'
> >
> > I'm not sure what "in priority lists" means.
> >
>
> Ah, this is confusing to explain. Let me try to this again.
>
> 'Queue of requets on a virtual engine, sorted by priority. Each RB entry
> is a i915_priolist containing a list of requests of the same priority.'
Way better! Maybe throw a "struct" in front of "i915_priolist" so
that kerneldoc will hyperlink properly.
--Jason
>
> Matt
>
>
> > --Jason
> >
> > > > > */
> > > > > - int queue_priority_hint;
> > > > > -
> > > > > - /**
> > > > > - * @queue: queue of requests, in priority lists
> > > > > - */
> > > > > - struct rb_root_cached queue;
> > > > > struct rb_root_cached virtual;
> > > > >
> > > > > /**
> > > > > @@ -332,6 +308,8 @@ struct intel_engine_cs {
> > > > > struct list_head hold; /* ready requests, but on hold */
> > > > > } active;
> > > > >
> > > > > + struct i915_sched_engine *sched_engine;
> > > > > +
> > > > > /* keep a request in reserve for a [pm] barrier under oom */
> > > > > struct i915_request *request_pool;
> > > > >
> > > > > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > > index 8db200422950..d1dc1db3e378 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > > @@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
> > > > > return prio;
> > > > > }
> > > > >
> > > > > -static int queue_prio(const struct intel_engine_execlists *execlists)
> > > > > +static int queue_prio(const struct i915_sched_engine *sched_engine)
> > > > > {
> > > > > struct rb_node *rb;
> > > > >
> > > > > - rb = rb_first_cached(&execlists->queue);
> > > > > + rb = rb_first_cached(&sched_engine->queue);
> > > > > if (!rb)
> > > > > return INT_MIN;
> > > > >
> > > > > @@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > > > * to preserve FIFO ordering of dependencies.
> > > > > */
> > > > > last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
> > > > > - if (engine->execlists.queue_priority_hint <= last_prio)
> > > > > + if (engine->sched_engine->queue_priority_hint <= last_prio)
> > > > > return false;
> > > > >
> > > > > /*
> > > > > @@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > > > * context, it's priority would not exceed ELSP[0] aka last_prio.
> > > > > */
> > > > > return max(virtual_prio(&engine->execlists),
> > > > > - queue_prio(&engine->execlists)) > last_prio;
> > > > > + queue_prio(engine->sched_engine)) > last_prio;
> > > > > }
> > > > >
> > > > > __maybe_unused static bool
> > > > > @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > > > > prio = rq_prio(rq);
> > > > > pl = i915_sched_lookup_priolist(engine, prio);
> > > > > }
> > > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > > >
> > > > > list_move(&rq->sched.link, pl);
> > > > > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > > > > @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> > > > > }
> > > > >
> > > > > /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> > > > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
> > > > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> > > > > ENGINE_TRACE(engine, "timeslice required for queue\n");
> > > > > return true;
> > > > > }
> > > > > @@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
> > > > > static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > {
> > > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > > struct i915_request **port = execlists->pending;
> > > > > struct i915_request ** const last_port = port + execlists->port_mask;
> > > > > struct i915_request *last, * const *active;
> > > > > @@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > last->fence.context,
> > > > > last->fence.seqno,
> > > > > last->sched.attr.priority,
> > > > > - execlists->queue_priority_hint);
> > > > > + sched_engine->queue_priority_hint);
> > > > > record_preemption(execlists);
> > > > >
> > > > > /*
> > > > > @@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > yesno(timer_expired(&execlists->timer)),
> > > > > last->fence.context, last->fence.seqno,
> > > > > rq_prio(last),
> > > > > - execlists->queue_priority_hint,
> > > > > + sched_engine->queue_priority_hint,
> > > > > yesno(timeslice_yield(execlists, last)));
> > > > >
> > > > > /*
> > > > > @@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > GEM_BUG_ON(rq->engine != &ve->base);
> > > > > GEM_BUG_ON(rq->context != &ve->context);
> > > > >
> > > > > - if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
> > > > > + if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> > > > > spin_unlock(&ve->base.active.lock);
> > > > > break;
> > > > > }
> > > > > @@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > yesno(engine != ve->siblings[0]));
> > > > >
> > > > > WRITE_ONCE(ve->request, NULL);
> > > > > - WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
> > > > > + WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
> > > > >
> > > > > rb = &ve->nodes[engine->id].rb;
> > > > > rb_erase_cached(rb, &execlists->virtual);
> > > > > @@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > break;
> > > > > }
> > > > >
> > > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > > struct i915_priolist *p = to_priolist(rb);
> > > > > struct i915_request *rq, *rn;
> > > > >
> > > > > @@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > }
> > > > > }
> > > > >
> > > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > > i915_priolist_free(p);
> > > > > }
> > > > > done:
> > > > > @@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > * request triggering preemption on the next dequeue (or subsequent
> > > > > * interrupt for secondary ports).
> > > > > */
> > > > > - execlists->queue_priority_hint = queue_prio(execlists);
> > > > > + sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > > > > spin_unlock(&engine->active.lock);
> > > > >
> > > > > /*
> > > > > @@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> > > > > */
> > > > > __execlists_unhold(rq);
> > > > >
> > > > > - if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
> > > > > - engine->execlists.queue_priority_hint = rq_prio(rq);
> > > > > + if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> > > > > + engine->sched_engine->queue_priority_hint = rq_prio(rq);
> > > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > > > }
> > > > >
> > > > > @@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
> > > > > static bool submit_queue(struct intel_engine_cs *engine,
> > > > > const struct i915_request *rq)
> > > > > {
> > > > > - struct intel_engine_execlists *execlists = &engine->execlists;
> > > > > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> > > > >
> > > > > - if (rq_prio(rq) <= execlists->queue_priority_hint)
> > > > > + if (rq_prio(rq) <= sched_engine->queue_priority_hint)
> > > > > return false;
> > > > >
> > > > > - execlists->queue_priority_hint = rq_prio(rq);
> > > > > + sched_engine->queue_priority_hint = rq_prio(rq);
> > > > > return true;
> > > > > }
> > > > >
> > > > > @@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> > > > > } else {
> > > > > queue_request(engine, request);
> > > > >
> > > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > > > GEM_BUG_ON(list_empty(&request->sched.link));
> > > > >
> > > > > if (submit_queue(engine, request))
> > > > > @@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
> > > > > from_tasklet(engine, t, execlists.tasklet);
> > > > >
> > > > > /* The driver is wedged; don't process any more events. */
> > > > > - WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
> > > > > + WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> > > > > }
> > > > >
> > > > > static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > > {
> > > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > > struct i915_request *rq, *rn;
> > > > > struct rb_node *rb;
> > > > > unsigned long flags;
> > > > > @@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > > intel_engine_signal_breadcrumbs(engine);
> > > > >
> > > > > /* Flush the queued requests to the timeline list (for retiring). */
> > > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > > struct i915_priolist *p = to_priolist(rb);
> > > > >
> > > > > priolist_for_each_request_consume(rq, rn, p) {
> > > > > @@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > > }
> > > > > }
> > > > >
> > > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > > i915_priolist_free(p);
> > > > > }
> > > > >
> > > > > @@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > > }
> > > > > i915_request_put(rq);
> > > > >
> > > > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > > > + ve->base.sched_engine->queue_priority_hint = INT_MIN;
> > > > > }
> > > > > spin_unlock(&ve->base.active.lock);
> > > > > }
> > > > >
> > > > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > > > >
> > > > > - execlists->queue_priority_hint = INT_MIN;
> > > > > - execlists->queue = RB_ROOT_CACHED;
> > > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > > >
> > > > > GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > > > > execlists->tasklet.callback = nop_submission_tasklet;
> > > > > @@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> > > > >
> > > > > static struct list_head *virtual_queue(struct virtual_engine *ve)
> > > > > {
> > > > > - return &ve->base.execlists.default_priolist.requests;
> > > > > + return &ve->base.sched_engine->default_priolist.requests;
> > > > > }
> > > > >
> > > > > static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > > > @@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > > > lrc_fini(&ve->context);
> > > > > intel_context_fini(&ve->context);
> > > > >
> > > > > - intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > > > + if (ve->base.breadcrumbs)
> > > > > + intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > > > + if (ve->base.sched_engine)
> > > > > + i915_sched_engine_put(ve->base.sched_engine);
> > > >
> > > > Will either of these ever actually be NULL? The first line or two
> > > > seems to indicate that breadcrumbs never is.
> > > >
> > > > > intel_engine_free_request_pool(&ve->base);
> > > > >
> > > > > kfree(ve->bonds);
> > > > > @@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
> > > > >
> > > > > ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
> > > > > rq->fence.context, rq->fence.seqno,
> > > > > - mask, ve->base.execlists.queue_priority_hint);
> > > > > + mask, ve->base.sched_engine->queue_priority_hint);
> > > > >
> > > > > return mask;
> > > > > }
> > > > > @@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > > > {
> > > > > struct virtual_engine * const ve =
> > > > > from_tasklet(ve, t, base.execlists.tasklet);
> > > > > - const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
> > > > > + const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> > > > > intel_engine_mask_t mask;
> > > > > unsigned int n;
> > > > >
> > > > > @@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > > > submit_engine:
> > > > > GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> > > > > node->prio = prio;
> > > > > - if (first && prio > sibling->execlists.queue_priority_hint)
> > > > > + if (first && prio > sibling->sched_engine->queue_priority_hint)
> > > > > tasklet_hi_schedule(&sibling->execlists.tasklet);
> > > > >
> > > > > unlock_engine:
> > > > > @@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > > > > i915_request_put(ve->request);
> > > > > }
> > > > >
> > > > > - ve->base.execlists.queue_priority_hint = rq_prio(rq);
> > > > > + ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
> > > > > ve->request = i915_request_get(rq);
> > > > >
> > > > > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > > > > @@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > > > intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> > > > > intel_engine_init_execlists(&ve->base);
> > > > >
> > > > > + ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> > > > > + if (!ve->base.sched_engine) {
> > > > > + err = -ENOMEM;
> > > > > + goto err_put;
> > > > > + }
> > > > > +
> > > > > ve->base.cops = &virtual_context_ops;
> > > > > ve->base.request_alloc = execlists_request_alloc;
> > > > >
> > > > > @@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > > > ve->base.bond_execute = virtual_bond_execute;
> > > > >
> > > > > INIT_LIST_HEAD(virtual_queue(ve));
> > > > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > > > tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> > > > >
> > > > > intel_context_init(&ve->context, &ve->base);
> > > > > @@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > > > unsigned int max)
> > > > > {
> > > > > const struct intel_engine_execlists *execlists = &engine->execlists;
> > > > > + const struct i915_sched_engine *sched_engine = engine->sched_engine;
> > > > > struct i915_request *rq, *last;
> > > > > unsigned long flags;
> > > > > unsigned int count;
> > > > > @@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > > > show_request(m, last, "\t\t", 0);
> > > > > }
> > > > >
> > > > > - if (execlists->queue_priority_hint != INT_MIN)
> > > > > + if (sched_engine->queue_priority_hint != INT_MIN)
> > > > > drm_printf(m, "\t\tQueue priority hint: %d\n",
> > > > > - READ_ONCE(execlists->queue_priority_hint));
> > > > > + READ_ONCE(sched_engine->queue_priority_hint));
> > > > >
> > > > > last = NULL;
> > > > > count = 0;
> > > > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > > > struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
> > > > >
> > > > > priolist_for_each_request(rq, p) {
> > > > > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > > index 32589c6625e1..b1fdba13e900 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > > @@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
> > > > >
> > > > > GEM_BUG_ON(timer_pending(&mock->hw_delay));
> > > > >
> > > > > + i915_sched_engine_put(engine->sched_engine);
> > > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > >
> > > > > intel_context_unpin(engine->kernel_context);
> > > > > @@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > > > {
> > > > > struct intel_context *ce;
> > > > >
> > > > > + engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> > > > > + if (!engine->sched_engine)
> > > > > + return -ENOMEM;
> > > > > +
> > > > > intel_engine_init_active(engine, ENGINE_MOCK);
> > > > > intel_engine_init_execlists(engine);
> > > > > intel_engine_init__pm(engine);
> > > > > @@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > > >
> > > > > engine->breadcrumbs = intel_breadcrumbs_create(NULL);
> > > > > if (!engine->breadcrumbs)
> > > > > - return -ENOMEM;
> > > > > + goto err_schedule;
> > > > >
> > > > > ce = create_kernel_context(engine);
> > > > > if (IS_ERR(ce))
> > > > > @@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > > >
> > > > > err_breadcrumbs:
> > > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > > +err_schedule:
> > > > > + i915_sched_engine_put(engine->sched_engine);
> > > > > return -ENOMEM;
> > > > > }
> > > > >
> > > > > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > > index 335719f17490..d42dea79ee64 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > > @@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
> > > > > static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > > {
> > > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > > struct i915_request **first = execlists->inflight;
> > > > > struct i915_request ** const last_port = first + execlists->port_mask;
> > > > > struct i915_request *last = first[0];
> > > > > @@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > > * event.
> > > > > */
> > > > > port = first;
> > > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > > struct i915_priolist *p = to_priolist(rb);
> > > > > struct i915_request *rq, *rn;
> > > > >
> > > > > @@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > > last = rq;
> > > > > }
> > > > >
> > > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > > i915_priolist_free(p);
> > > > > }
> > > > > done:
> > > > > - execlists->queue_priority_hint =
> > > > > + sched_engine->queue_priority_hint =
> > > > > rb ? to_priolist(rb)->priority : INT_MIN;
> > > > > if (submit) {
> > > > > *port = schedule_in(last, port - execlists->inflight);
> > > > > @@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > > > >
> > > > > static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > > {
> > > > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > > struct i915_request *rq, *rn;
> > > > > struct rb_node *rb;
> > > > > unsigned long flags;
> > > > > @@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > > }
> > > > >
> > > > > /* Flush the queued requests to the timeline list (for retiring). */
> > > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > > struct i915_priolist *p = to_priolist(rb);
> > > > >
> > > > > priolist_for_each_request_consume(rq, rn, p) {
> > > > > @@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > > i915_request_mark_complete(rq);
> > > > > }
> > > > >
> > > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > > i915_priolist_free(p);
> > > > > }
> > > > >
> > > > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > > > >
> > > > > - execlists->queue_priority_hint = INT_MIN;
> > > > > - execlists->queue = RB_ROOT_CACHED;
> > > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > > >
> > > > > spin_unlock_irqrestore(&engine->active.lock, flags);
> > > > > }
> > > > > @@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
> > > > >
> > > > > queue_request(engine, rq, rq_prio(rq));
> > > > >
> > > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > > > GEM_BUG_ON(list_empty(&rq->sched.link));
> > > > >
> > > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > > > > index efa638c3acc7..4953874a9ba6 100644
> > > > > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > > > > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > > > > @@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> > > > > return rb_entry(rb, struct i915_priolist, node);
> > > > > }
> > > > >
> > > > > -static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > > +static void assert_priolists(struct i915_sched_engine * const sched_engine)
> > > > > {
> > > > > struct rb_node *rb;
> > > > > long last_prio;
> > > > > @@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > > if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> > > > > return;
> > > > >
> > > > > - GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
> > > > > - rb_first(&execlists->queue.rb_root));
> > > > > + GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
> > > > > + rb_first(&sched_engine->queue.rb_root));
> > > > >
> > > > > last_prio = INT_MAX;
> > > > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > > > const struct i915_priolist *p = to_priolist(rb);
> > > > >
> > > > > GEM_BUG_ON(p->priority > last_prio);
> > > > > @@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > > struct list_head *
> > > > > i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > > {
> > > > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > > struct i915_priolist *p;
> > > > > struct rb_node **parent, *rb;
> > > > > bool first = true;
> > > > >
> > > > > lockdep_assert_held(&engine->active.lock);
> > > > > - assert_priolists(execlists);
> > > > > + assert_priolists(sched_engine);
> > > > >
> > > > > - if (unlikely(execlists->no_priolist))
> > > > > + if (unlikely(sched_engine->no_priolist))
> > > > > prio = I915_PRIORITY_NORMAL;
> > > > >
> > > > > find_priolist:
> > > > > /* most positive priority is scheduled first, equal priorities fifo */
> > > > > rb = NULL;
> > > > > - parent = &execlists->queue.rb_root.rb_node;
> > > > > + parent = &sched_engine->queue.rb_root.rb_node;
> > > > > while (*parent) {
> > > > > rb = *parent;
> > > > > p = to_priolist(rb);
> > > > > @@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > > }
> > > > >
> > > > > if (prio == I915_PRIORITY_NORMAL) {
> > > > > - p = &execlists->default_priolist;
> > > > > + p = &sched_engine->default_priolist;
> > > > > } else {
> > > > > p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
> > > > > /* Convert an allocation failure to a priority bump */
> > > > > @@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > > * requests, so if userspace lied about their
> > > > > * dependencies that reordering may be visible.
> > > > > */
> > > > > - execlists->no_priolist = true;
> > > > > + sched_engine->no_priolist = true;
> > > > > goto find_priolist;
> > > > > }
> > > > > }
> > > > > @@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > > INIT_LIST_HEAD(&p->requests);
> > > > >
> > > > > rb_link_node(&p->node, rb, parent);
> > > > > - rb_insert_color_cached(&p->node, &execlists->queue, first);
> > > > > + rb_insert_color_cached(&p->node, &sched_engine->queue, first);
> > > > >
> > > > > return &p->requests;
> > > > > }
> > > > > @@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > > > * We only need to kick the tasklet once for the high priority
> > > > > * new context we add into the queue.
> > > > > */
> > > > > - if (prio <= engine->execlists.queue_priority_hint)
> > > > > + if (prio <= engine->sched_engine->queue_priority_hint)
> > > > > return;
> > > > >
> > > > > rcu_read_lock();
> > > > > @@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > > > inflight->fence.context, inflight->fence.seqno,
> > > > > inflight->sched.attr.priority);
> > > > >
> > > > > - engine->execlists.queue_priority_hint = prio;
> > > > > + engine->sched_engine->queue_priority_hint = prio;
> > > > > if (need_preempt(prio, rq_prio(inflight)))
> > > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > > >
> > > > > @@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
> > > > > rcu_read_unlock();
> > > > > }
> > > > >
> > > > > +void i915_sched_engine_free(struct kref *kref)
> > > > > +{
> > > > > + struct i915_sched_engine *sched_engine =
> > > > > + container_of(kref, typeof(*sched_engine), ref);
> > > > > +
> > > > > + kfree(sched_engine);
> > > > > +}
> > > > > +
> > > > > +struct i915_sched_engine *
> > > > > +i915_sched_engine_create(unsigned int subclass)
> > > > > +{
> > > > > + struct i915_sched_engine *sched_engine;
> > > > > +
> > > > > + sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
> > > > > + if (!sched_engine)
> > > > > + return NULL;
> > > > > +
> > > > > + kref_init(&sched_engine->ref);
> > > > > +
> > > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > > +
> > > > > + return sched_engine;
> > > > > +}
> > > > > +
> > > > > static void i915_global_scheduler_shrink(void)
> > > > > {
> > > > > kmem_cache_shrink(global.slab_dependencies);
> > > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > > > > index 858a0938f47a..91a04e34cac5 100644
> > > > > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > > > > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > > > > @@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
> > > > > __i915_priolist_free(p);
> > > > > }
> > > > >
> > > > > +struct i915_sched_engine *
> > > > > +i915_sched_engine_create(unsigned int subclass);
> > > > > +
> > > > > +void i915_sched_engine_free(struct kref *kref);
> > > > > +
> > > > > +static inline struct i915_sched_engine *
> > > > > +i915_sched_engine_get(struct i915_sched_engine *sched_engine)
> > > > > +{
> > > > > + kref_get(&sched_engine->ref);
> > > > > + return sched_engine;
> > > > > +}
> > > > > +
> > > > > +static inline void
> > > > > +i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> > > > > +{
> > > > > + kref_put(&sched_engine->ref, i915_sched_engine_free);
> > > > > +}
> > > > > +
> > > > > void i915_request_show_with_schedule(struct drm_printer *m,
> > > > > const struct i915_request *rq,
> > > > > const char *prefix,
> > > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > > > index 343ed44d5ed4..16a054e67760 100644
> > > > > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > > > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > >
> > > > Dumb question for which I'm not really in the position of having an
> > > > opinion: Should this go in its own file?
> > > >
> > >
> > > I think this is in the right file.
> > >
> > > Matt
> > >
> > > > Patch seems mostly mechanically correct. Not sure where this is going
> > > > yet, though. I'll keep reading. :-)
> > > >
> > > > --Jason
> > > >
> > > >
> > > > > @@ -91,4 +91,37 @@ struct i915_dependency {
> > > > > &(rq__)->sched.signalers_list, \
> > > > > signal_link)
> > > > >
> > > > > +struct i915_sched_engine {
> > > > > + struct kref ref;
> > > > > +
> > > > > + /**
> > > > > + * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > > > + */
> > > > > + struct i915_priolist default_priolist;
> > > > > +
> > > > > + /**
> > > > > + * @queue_priority_hint: Highest pending priority.
> > > > > + *
> > > > > + * When we add requests into the queue, or adjust the priority of
> > > > > + * executing requests, we compute the maximum priority of those
> > > > > + * pending requests. We can then use this value to determine if
> > > > > + * we need to preempt the executing requests to service the queue.
> > > > > + * However, since the we may have recorded the priority of an inflight
> > > > > + * request we wanted to preempt but since completed, at the time of
> > > > > + * dequeuing the priority hint may no longer may match the highest
> > > > > + * available request priority.
> > > > > + */
> > > > > + int queue_priority_hint;
> > > > > +
> > > > > + /**
> > > > > + * @queue: queue of requests, in priority lists
> > > > > + */
> > > > > + struct rb_root_cached queue;
> > > > > +
> > > > > + /**
> > > > > + * @no_priolist: priority lists disabled
> > > > > + */
> > > > > + bool no_priolist;
> > > > > +};
> > > > > +
> > > > > #endif /* _I915_SCHEDULER_TYPES_H_ */
> > > > > --
> > > > > 2.28.0
> > > > >
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 1/9] drm/i915: Move priolist to new i915_sched_engine object
@ 2021-06-04 18:14 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 18:14 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 4, 2021 at 12:59 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> On Fri, Jun 04, 2021 at 12:51:43PM -0500, Jason Ekstrand wrote:
> > On Fri, Jun 4, 2021 at 12:42 PM Matthew Brost <matthew.brost@intel.com> wrote:
> > >
> > > On Fri, Jun 04, 2021 at 12:38:22PM -0500, Jason Ekstrand wrote:
> > > > On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> > > > >
> > > > > Introduce i915_sched_engine object which is lower level data structure
> > > > > that i915_scheduler / generic code can operate on without touching
> > > > > execlist specific structures. This allows additional submission backends
> > > > > to be added without breaking the layering.
> > > > >
> > > > > This is a bit of detour to integrating the i915 with the DRM scheduler
> > > > > but this object will still exist when the DRM scheduler lands in the
> > > > > i915. It will however look a bit different. It will encapsulate the
> > > > > drm_gpu_scheduler object plus and common variables (to the backends)
> > > > > related to scheduling. Regardless this is a step in the right direction.
> > > > >
> > > > > This patch starts the aforementioned transition by moving the the
> > > > > priolist into the i915_sched_engine object.
> > > > >
> > > > > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > > > > ---
> > > > > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 14 +++-
> > > > > drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 +-
> > > > > drivers/gpu/drm/i915/gt/intel_engine_types.h | 30 +------
> > > > > .../drm/i915/gt/intel_execlists_submission.c | 81 +++++++++++--------
> > > > > drivers/gpu/drm/i915/gt/mock_engine.c | 9 ++-
> > > > > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 ++---
> > > > > drivers/gpu/drm/i915/i915_scheduler.c | 51 +++++++++---
> > > > > drivers/gpu/drm/i915/i915_scheduler.h | 18 +++++
> > > > > drivers/gpu/drm/i915/i915_scheduler_types.h | 33 ++++++++
> > > > > 9 files changed, 169 insertions(+), 90 deletions(-)
> > > > >
> > > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > > index 3f9a811eb02b..d0f3814440f6 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > > > > @@ -583,9 +583,6 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine)
> > > > > memset(execlists->pending, 0, sizeof(execlists->pending));
> > > > > execlists->active =
> > > > > memset(execlists->inflight, 0, sizeof(execlists->inflight));
> > > > > -
> > > > > - execlists->queue_priority_hint = INT_MIN;
> > > > > - execlists->queue = RB_ROOT_CACHED;
> > > > > }
> > > > >
> > > > > static void cleanup_status_page(struct intel_engine_cs *engine)
> > > > > @@ -712,6 +709,12 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > > > goto err_status;
> > > > > }
> > > > >
> > > > > + engine->sched_engine = i915_sched_engine_create(ENGINE_PHYSICAL);
> > > > > + if (!engine->sched_engine) {
> > > > > + err = -ENOMEM;
> > > > > + goto err_sched_engine;
> > > > > + }
> > > > > +
> > > > > err = intel_engine_init_cmd_parser(engine);
> > > > > if (err)
> > > > > goto err_cmd_parser;
> > > > > @@ -735,6 +738,8 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > > > > return 0;
> > > > >
> > > > > err_cmd_parser:
> > > > > + i915_sched_engine_put(engine->sched_engine);
> > > > > +err_sched_engine:
> > > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > > err_status:
> > > > > cleanup_status_page(engine);
> > > > > @@ -958,6 +963,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > > > > GEM_BUG_ON(!list_empty(&engine->active.requests));
> > > > > tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> > > > >
> > > > > + i915_sched_engine_put(engine->sched_engine);
> > > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > >
> > > > > intel_engine_fini_retire(engine);
> > > > > @@ -1281,7 +1287,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
> > > > > intel_engine_flush_submission(engine);
> > > > >
> > > > > /* ELSP is empty, but there are ready requests? E.g. after reset */
> > > > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root))
> > > > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root))
> > > > > return false;
> > > > >
> > > > > /* Ring stopped? */
> > > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > > index 47f4397095e5..b6a00dd72808 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > > > > @@ -275,12 +275,12 @@ static int __engine_park(struct intel_wakeref *wf)
> > > > > intel_breadcrumbs_park(engine->breadcrumbs);
> > > > >
> > > > > /* Must be reset upon idling, or we may miss the busy wakeup. */
> > > > > - GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
> > > > > + GEM_BUG_ON(engine->sched_engine->queue_priority_hint != INT_MIN);
> > > > >
> > > > > if (engine->park)
> > > > > engine->park(engine);
> > > > >
> > > > > - engine->execlists.no_priolist = false;
> > > > > + engine->sched_engine->no_priolist = false;
> > > > >
> > > > > /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> > > > > intel_gt_pm_put_async(engine->gt);
> > > > > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > > index 9ef349cd5cea..86b41ddec373 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > > > > @@ -59,6 +59,7 @@ struct drm_i915_reg_table;
> > > > > struct i915_gem_context;
> > > > > struct i915_request;
> > > > > struct i915_sched_attr;
> > > > > +struct i915_sched_engine;
> > > > > struct intel_gt;
> > > > > struct intel_ring;
> > > > > struct intel_uncore;
> > > > > @@ -152,11 +153,6 @@ struct intel_engine_execlists {
> > > > > */
> > > > > struct timer_list preempt;
> > > > >
> > > > > - /**
> > > > > - * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > > > - */
> > > > > - struct i915_priolist default_priolist;
> > > > > -
> > > > > /**
> > > > > * @ccid: identifier for contexts submitted to this engine
> > > > > */
> > > > > @@ -191,11 +187,6 @@ struct intel_engine_execlists {
> > > > > */
> > > > > u32 reset_ccid;
> > > > >
> > > > > - /**
> > > > > - * @no_priolist: priority lists disabled
> > > > > - */
> > > > > - bool no_priolist;
> > > > > -
> > > > > /**
> > > > > * @submit_reg: gen-specific execlist submission register
> > > > > * set to the ExecList Submission Port (elsp) register pre-Gen11 and to
> > > > > @@ -238,23 +229,8 @@ struct intel_engine_execlists {
> > > > > unsigned int port_mask;
> > > > >
> > > > > /**
> > > > > - * @queue_priority_hint: Highest pending priority.
> > > > > - *
> > > > > - * When we add requests into the queue, or adjust the priority of
> > > > > - * executing requests, we compute the maximum priority of those
> > > > > - * pending requests. We can then use this value to determine if
> > > > > - * we need to preempt the executing requests to service the queue.
> > > > > - * However, since the we may have recorded the priority of an inflight
> > > > > - * request we wanted to preempt but since completed, at the time of
> > > > > - * dequeuing the priority hint may no longer may match the highest
> > > > > - * available request priority.
> > > > > + * @virtual: virtual of requests, in priority lists
> > > >
> > > > Having trouble parsing this. Did you mean "queue of virtual
> > > > requests"? Or maybe "virtual queue of requests"?
> > > >
> > >
> > > Yep, this doesn't really make sense. How about:
> > > 'queue of requests on a virtual engine, in priority lists'
> >
> > I'm not sure what "in priority lists" means.
> >
>
> Ah, this is confusing to explain. Let me try to this again.
>
> 'Queue of requets on a virtual engine, sorted by priority. Each RB entry
> is a i915_priolist containing a list of requests of the same priority.'
Way better! Maybe throw a "struct" in front of "i915_priolist" so
that kerneldoc will hyperlink properly.
--Jason
>
> Matt
>
>
> > --Jason
> >
> > > > > */
> > > > > - int queue_priority_hint;
> > > > > -
> > > > > - /**
> > > > > - * @queue: queue of requests, in priority lists
> > > > > - */
> > > > > - struct rb_root_cached queue;
> > > > > struct rb_root_cached virtual;
> > > > >
> > > > > /**
> > > > > @@ -332,6 +308,8 @@ struct intel_engine_cs {
> > > > > struct list_head hold; /* ready requests, but on hold */
> > > > > } active;
> > > > >
> > > > > + struct i915_sched_engine *sched_engine;
> > > > > +
> > > > > /* keep a request in reserve for a [pm] barrier under oom */
> > > > > struct i915_request *request_pool;
> > > > >
> > > > > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > > index 8db200422950..d1dc1db3e378 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > > > > @@ -273,11 +273,11 @@ static int effective_prio(const struct i915_request *rq)
> > > > > return prio;
> > > > > }
> > > > >
> > > > > -static int queue_prio(const struct intel_engine_execlists *execlists)
> > > > > +static int queue_prio(const struct i915_sched_engine *sched_engine)
> > > > > {
> > > > > struct rb_node *rb;
> > > > >
> > > > > - rb = rb_first_cached(&execlists->queue);
> > > > > + rb = rb_first_cached(&sched_engine->queue);
> > > > > if (!rb)
> > > > > return INT_MIN;
> > > > >
> > > > > @@ -318,7 +318,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > > > * to preserve FIFO ordering of dependencies.
> > > > > */
> > > > > last_prio = max(effective_prio(rq), I915_PRIORITY_NORMAL - 1);
> > > > > - if (engine->execlists.queue_priority_hint <= last_prio)
> > > > > + if (engine->sched_engine->queue_priority_hint <= last_prio)
> > > > > return false;
> > > > >
> > > > > /*
> > > > > @@ -340,7 +340,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > > > > * context, it's priority would not exceed ELSP[0] aka last_prio.
> > > > > */
> > > > > return max(virtual_prio(&engine->execlists),
> > > > > - queue_prio(&engine->execlists)) > last_prio;
> > > > > + queue_prio(engine->sched_engine)) > last_prio;
> > > > > }
> > > > >
> > > > > __maybe_unused static bool
> > > > > @@ -384,7 +384,7 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > > > > prio = rq_prio(rq);
> > > > > pl = i915_sched_lookup_priolist(engine, prio);
> > > > > }
> > > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > > >
> > > > > list_move(&rq->sched.link, pl);
> > > > > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > > > > @@ -1139,7 +1139,7 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> > > > > }
> > > > >
> > > > > /* Otherwise, ELSP[0] is by itself, but may be waiting in the queue */
> > > > > - if (!RB_EMPTY_ROOT(&engine->execlists.queue.rb_root)) {
> > > > > + if (!RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root)) {
> > > > > ENGINE_TRACE(engine, "timeslice required for queue\n");
> > > > > return true;
> > > > > }
> > > > > @@ -1236,6 +1236,7 @@ static bool completed(const struct i915_request *rq)
> > > > > static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > {
> > > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > > struct i915_request **port = execlists->pending;
> > > > > struct i915_request ** const last_port = port + execlists->port_mask;
> > > > > struct i915_request *last, * const *active;
> > > > > @@ -1287,7 +1288,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > last->fence.context,
> > > > > last->fence.seqno,
> > > > > last->sched.attr.priority,
> > > > > - execlists->queue_priority_hint);
> > > > > + sched_engine->queue_priority_hint);
> > > > > record_preemption(execlists);
> > > > >
> > > > > /*
> > > > > @@ -1313,7 +1314,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > yesno(timer_expired(&execlists->timer)),
> > > > > last->fence.context, last->fence.seqno,
> > > > > rq_prio(last),
> > > > > - execlists->queue_priority_hint,
> > > > > + sched_engine->queue_priority_hint,
> > > > > yesno(timeslice_yield(execlists, last)));
> > > > >
> > > > > /*
> > > > > @@ -1384,7 +1385,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > GEM_BUG_ON(rq->engine != &ve->base);
> > > > > GEM_BUG_ON(rq->context != &ve->context);
> > > > >
> > > > > - if (unlikely(rq_prio(rq) < queue_prio(execlists))) {
> > > > > + if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> > > > > spin_unlock(&ve->base.active.lock);
> > > > > break;
> > > > > }
> > > > > @@ -1405,7 +1406,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > yesno(engine != ve->siblings[0]));
> > > > >
> > > > > WRITE_ONCE(ve->request, NULL);
> > > > > - WRITE_ONCE(ve->base.execlists.queue_priority_hint, INT_MIN);
> > > > > + WRITE_ONCE(ve->base.sched_engine->queue_priority_hint, INT_MIN);
> > > > >
> > > > > rb = &ve->nodes[engine->id].rb;
> > > > > rb_erase_cached(rb, &execlists->virtual);
> > > > > @@ -1450,7 +1451,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > break;
> > > > > }
> > > > >
> > > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > > struct i915_priolist *p = to_priolist(rb);
> > > > > struct i915_request *rq, *rn;
> > > > >
> > > > > @@ -1529,7 +1530,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > }
> > > > > }
> > > > >
> > > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > > i915_priolist_free(p);
> > > > > }
> > > > > done:
> > > > > @@ -1551,7 +1552,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > > > > * request triggering preemption on the next dequeue (or subsequent
> > > > > * interrupt for secondary ports).
> > > > > */
> > > > > - execlists->queue_priority_hint = queue_prio(execlists);
> > > > > + sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > > > > spin_unlock(&engine->active.lock);
> > > > >
> > > > > /*
> > > > > @@ -2123,8 +2124,8 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> > > > > */
> > > > > __execlists_unhold(rq);
> > > > >
> > > > > - if (rq_prio(rq) > engine->execlists.queue_priority_hint) {
> > > > > - engine->execlists.queue_priority_hint = rq_prio(rq);
> > > > > + if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> > > > > + engine->sched_engine->queue_priority_hint = rq_prio(rq);
> > > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > > > }
> > > > >
> > > > > @@ -2455,12 +2456,12 @@ static void queue_request(struct intel_engine_cs *engine,
> > > > > static bool submit_queue(struct intel_engine_cs *engine,
> > > > > const struct i915_request *rq)
> > > > > {
> > > > > - struct intel_engine_execlists *execlists = &engine->execlists;
> > > > > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> > > > >
> > > > > - if (rq_prio(rq) <= execlists->queue_priority_hint)
> > > > > + if (rq_prio(rq) <= sched_engine->queue_priority_hint)
> > > > > return false;
> > > > >
> > > > > - execlists->queue_priority_hint = rq_prio(rq);
> > > > > + sched_engine->queue_priority_hint = rq_prio(rq);
> > > > > return true;
> > > > > }
> > > > >
> > > > > @@ -2486,7 +2487,7 @@ static void execlists_submit_request(struct i915_request *request)
> > > > > } else {
> > > > > queue_request(engine, request);
> > > > >
> > > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > > > GEM_BUG_ON(list_empty(&request->sched.link));
> > > > >
> > > > > if (submit_queue(engine, request))
> > > > > @@ -2969,12 +2970,13 @@ static void nop_submission_tasklet(struct tasklet_struct *t)
> > > > > from_tasklet(engine, t, execlists.tasklet);
> > > > >
> > > > > /* The driver is wedged; don't process any more events. */
> > > > > - WRITE_ONCE(engine->execlists.queue_priority_hint, INT_MIN);
> > > > > + WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> > > > > }
> > > > >
> > > > > static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > > {
> > > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > > struct i915_request *rq, *rn;
> > > > > struct rb_node *rb;
> > > > > unsigned long flags;
> > > > > @@ -3006,7 +3008,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > > intel_engine_signal_breadcrumbs(engine);
> > > > >
> > > > > /* Flush the queued requests to the timeline list (for retiring). */
> > > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > > struct i915_priolist *p = to_priolist(rb);
> > > > >
> > > > > priolist_for_each_request_consume(rq, rn, p) {
> > > > > @@ -3016,7 +3018,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > > }
> > > > > }
> > > > >
> > > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > > i915_priolist_free(p);
> > > > > }
> > > > >
> > > > > @@ -3042,15 +3044,15 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > > > > }
> > > > > i915_request_put(rq);
> > > > >
> > > > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > > > + ve->base.sched_engine->queue_priority_hint = INT_MIN;
> > > > > }
> > > > > spin_unlock(&ve->base.active.lock);
> > > > > }
> > > > >
> > > > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > > > >
> > > > > - execlists->queue_priority_hint = INT_MIN;
> > > > > - execlists->queue = RB_ROOT_CACHED;
> > > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > > >
> > > > > GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > > > > execlists->tasklet.callback = nop_submission_tasklet;
> > > > > @@ -3286,7 +3288,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> > > > >
> > > > > static struct list_head *virtual_queue(struct virtual_engine *ve)
> > > > > {
> > > > > - return &ve->base.execlists.default_priolist.requests;
> > > > > + return &ve->base.sched_engine->default_priolist.requests;
> > > > > }
> > > > >
> > > > > static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > > > @@ -3344,7 +3346,10 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > > > > lrc_fini(&ve->context);
> > > > > intel_context_fini(&ve->context);
> > > > >
> > > > > - intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > > > + if (ve->base.breadcrumbs)
> > > > > + intel_breadcrumbs_free(ve->base.breadcrumbs);
> > > > > + if (ve->base.sched_engine)
> > > > > + i915_sched_engine_put(ve->base.sched_engine);
> > > >
> > > > Will either of these ever actually be NULL? The first line or two
> > > > seems to indicate that breadcrumbs never is.
> > > >
> > > > > intel_engine_free_request_pool(&ve->base);
> > > > >
> > > > > kfree(ve->bonds);
> > > > > @@ -3475,7 +3480,7 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
> > > > >
> > > > > ENGINE_TRACE(&ve->base, "rq=%llx:%lld, mask=%x, prio=%d\n",
> > > > > rq->fence.context, rq->fence.seqno,
> > > > > - mask, ve->base.execlists.queue_priority_hint);
> > > > > + mask, ve->base.sched_engine->queue_priority_hint);
> > > > >
> > > > > return mask;
> > > > > }
> > > > > @@ -3484,7 +3489,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > > > {
> > > > > struct virtual_engine * const ve =
> > > > > from_tasklet(ve, t, base.execlists.tasklet);
> > > > > - const int prio = READ_ONCE(ve->base.execlists.queue_priority_hint);
> > > > > + const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> > > > > intel_engine_mask_t mask;
> > > > > unsigned int n;
> > > > >
> > > > > @@ -3552,7 +3557,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > > > > submit_engine:
> > > > > GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> > > > > node->prio = prio;
> > > > > - if (first && prio > sibling->execlists.queue_priority_hint)
> > > > > + if (first && prio > sibling->sched_engine->queue_priority_hint)
> > > > > tasklet_hi_schedule(&sibling->execlists.tasklet);
> > > > >
> > > > > unlock_engine:
> > > > > @@ -3588,7 +3593,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > > > > i915_request_put(ve->request);
> > > > > }
> > > > >
> > > > > - ve->base.execlists.queue_priority_hint = rq_prio(rq);
> > > > > + ve->base.sched_engine->queue_priority_hint = rq_prio(rq);
> > > > > ve->request = i915_request_get(rq);
> > > > >
> > > > > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > > > > @@ -3684,6 +3689,12 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > > > intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> > > > > intel_engine_init_execlists(&ve->base);
> > > > >
> > > > > + ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> > > > > + if (!ve->base.sched_engine) {
> > > > > + err = -ENOMEM;
> > > > > + goto err_put;
> > > > > + }
> > > > > +
> > > > > ve->base.cops = &virtual_context_ops;
> > > > > ve->base.request_alloc = execlists_request_alloc;
> > > > >
> > > > > @@ -3692,7 +3703,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > > > > ve->base.bond_execute = virtual_bond_execute;
> > > > >
> > > > > INIT_LIST_HEAD(virtual_queue(ve));
> > > > > - ve->base.execlists.queue_priority_hint = INT_MIN;
> > > > > tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> > > > >
> > > > > intel_context_init(&ve->context, &ve->base);
> > > > > @@ -3849,6 +3859,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > > > unsigned int max)
> > > > > {
> > > > > const struct intel_engine_execlists *execlists = &engine->execlists;
> > > > > + const struct i915_sched_engine *sched_engine = engine->sched_engine;
> > > > > struct i915_request *rq, *last;
> > > > > unsigned long flags;
> > > > > unsigned int count;
> > > > > @@ -3873,13 +3884,13 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > > > > show_request(m, last, "\t\t", 0);
> > > > > }
> > > > >
> > > > > - if (execlists->queue_priority_hint != INT_MIN)
> > > > > + if (sched_engine->queue_priority_hint != INT_MIN)
> > > > > drm_printf(m, "\t\tQueue priority hint: %d\n",
> > > > > - READ_ONCE(execlists->queue_priority_hint));
> > > > > + READ_ONCE(sched_engine->queue_priority_hint));
> > > > >
> > > > > last = NULL;
> > > > > count = 0;
> > > > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > > > struct i915_priolist *p = rb_entry(rb, typeof(*p), node);
> > > > >
> > > > > priolist_for_each_request(rq, p) {
> > > > > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > > index 32589c6625e1..b1fdba13e900 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > > > > @@ -283,6 +283,7 @@ static void mock_engine_release(struct intel_engine_cs *engine)
> > > > >
> > > > > GEM_BUG_ON(timer_pending(&mock->hw_delay));
> > > > >
> > > > > + i915_sched_engine_put(engine->sched_engine);
> > > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > >
> > > > > intel_context_unpin(engine->kernel_context);
> > > > > @@ -345,6 +346,10 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > > > {
> > > > > struct intel_context *ce;
> > > > >
> > > > > + engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> > > > > + if (!engine->sched_engine)
> > > > > + return -ENOMEM;
> > > > > +
> > > > > intel_engine_init_active(engine, ENGINE_MOCK);
> > > > > intel_engine_init_execlists(engine);
> > > > > intel_engine_init__pm(engine);
> > > > > @@ -352,7 +357,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > > >
> > > > > engine->breadcrumbs = intel_breadcrumbs_create(NULL);
> > > > > if (!engine->breadcrumbs)
> > > > > - return -ENOMEM;
> > > > > + goto err_schedule;
> > > > >
> > > > > ce = create_kernel_context(engine);
> > > > > if (IS_ERR(ce))
> > > > > @@ -366,6 +371,8 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > > > >
> > > > > err_breadcrumbs:
> > > > > intel_breadcrumbs_free(engine->breadcrumbs);
> > > > > +err_schedule:
> > > > > + i915_sched_engine_put(engine->sched_engine);
> > > > > return -ENOMEM;
> > > > > }
> > > > >
> > > > > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > > index 335719f17490..d42dea79ee64 100644
> > > > > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > > > > @@ -182,6 +182,7 @@ static void schedule_out(struct i915_request *rq)
> > > > > static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > > {
> > > > > struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > > struct i915_request **first = execlists->inflight;
> > > > > struct i915_request ** const last_port = first + execlists->port_mask;
> > > > > struct i915_request *last = first[0];
> > > > > @@ -204,7 +205,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > > * event.
> > > > > */
> > > > > port = first;
> > > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > > struct i915_priolist *p = to_priolist(rb);
> > > > > struct i915_request *rq, *rn;
> > > > >
> > > > > @@ -224,11 +225,11 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > > > > last = rq;
> > > > > }
> > > > >
> > > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > > i915_priolist_free(p);
> > > > > }
> > > > > done:
> > > > > - execlists->queue_priority_hint =
> > > > > + sched_engine->queue_priority_hint =
> > > > > rb ? to_priolist(rb)->priority : INT_MIN;
> > > > > if (submit) {
> > > > > *port = schedule_in(last, port - execlists->inflight);
> > > > > @@ -338,7 +339,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > > > >
> > > > > static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > > {
> > > > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > > struct i915_request *rq, *rn;
> > > > > struct rb_node *rb;
> > > > > unsigned long flags;
> > > > > @@ -368,7 +369,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > > }
> > > > >
> > > > > /* Flush the queued requests to the timeline list (for retiring). */
> > > > > - while ((rb = rb_first_cached(&execlists->queue))) {
> > > > > + while ((rb = rb_first_cached(&sched_engine->queue))) {
> > > > > struct i915_priolist *p = to_priolist(rb);
> > > > >
> > > > > priolist_for_each_request_consume(rq, rn, p) {
> > > > > @@ -378,14 +379,14 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > > > > i915_request_mark_complete(rq);
> > > > > }
> > > > >
> > > > > - rb_erase_cached(&p->node, &execlists->queue);
> > > > > + rb_erase_cached(&p->node, &sched_engine->queue);
> > > > > i915_priolist_free(p);
> > > > > }
> > > > >
> > > > > /* Remaining _unready_ requests will be nop'ed when submitted */
> > > > >
> > > > > - execlists->queue_priority_hint = INT_MIN;
> > > > > - execlists->queue = RB_ROOT_CACHED;
> > > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > > >
> > > > > spin_unlock_irqrestore(&engine->active.lock, flags);
> > > > > }
> > > > > @@ -540,7 +541,7 @@ static void guc_submit_request(struct i915_request *rq)
> > > > >
> > > > > queue_request(engine, rq, rq_prio(rq));
> > > > >
> > > > > - GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
> > > > > + GEM_BUG_ON(RB_EMPTY_ROOT(&engine->sched_engine->queue.rb_root));
> > > > > GEM_BUG_ON(list_empty(&rq->sched.link));
> > > > >
> > > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > > > > index efa638c3acc7..4953874a9ba6 100644
> > > > > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > > > > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > > > > @@ -40,7 +40,7 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> > > > > return rb_entry(rb, struct i915_priolist, node);
> > > > > }
> > > > >
> > > > > -static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > > +static void assert_priolists(struct i915_sched_engine * const sched_engine)
> > > > > {
> > > > > struct rb_node *rb;
> > > > > long last_prio;
> > > > > @@ -48,11 +48,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > > if (!IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> > > > > return;
> > > > >
> > > > > - GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
> > > > > - rb_first(&execlists->queue.rb_root));
> > > > > + GEM_BUG_ON(rb_first_cached(&sched_engine->queue) !=
> > > > > + rb_first(&sched_engine->queue.rb_root));
> > > > >
> > > > > last_prio = INT_MAX;
> > > > > - for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
> > > > > + for (rb = rb_first_cached(&sched_engine->queue); rb; rb = rb_next(rb)) {
> > > > > const struct i915_priolist *p = to_priolist(rb);
> > > > >
> > > > > GEM_BUG_ON(p->priority > last_prio);
> > > > > @@ -63,21 +63,21 @@ static void assert_priolists(struct intel_engine_execlists * const execlists)
> > > > > struct list_head *
> > > > > i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > > {
> > > > > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > > > > + struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > > > > struct i915_priolist *p;
> > > > > struct rb_node **parent, *rb;
> > > > > bool first = true;
> > > > >
> > > > > lockdep_assert_held(&engine->active.lock);
> > > > > - assert_priolists(execlists);
> > > > > + assert_priolists(sched_engine);
> > > > >
> > > > > - if (unlikely(execlists->no_priolist))
> > > > > + if (unlikely(sched_engine->no_priolist))
> > > > > prio = I915_PRIORITY_NORMAL;
> > > > >
> > > > > find_priolist:
> > > > > /* most positive priority is scheduled first, equal priorities fifo */
> > > > > rb = NULL;
> > > > > - parent = &execlists->queue.rb_root.rb_node;
> > > > > + parent = &sched_engine->queue.rb_root.rb_node;
> > > > > while (*parent) {
> > > > > rb = *parent;
> > > > > p = to_priolist(rb);
> > > > > @@ -92,7 +92,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > > }
> > > > >
> > > > > if (prio == I915_PRIORITY_NORMAL) {
> > > > > - p = &execlists->default_priolist;
> > > > > + p = &sched_engine->default_priolist;
> > > > > } else {
> > > > > p = kmem_cache_alloc(global.slab_priorities, GFP_ATOMIC);
> > > > > /* Convert an allocation failure to a priority bump */
> > > > > @@ -107,7 +107,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > > * requests, so if userspace lied about their
> > > > > * dependencies that reordering may be visible.
> > > > > */
> > > > > - execlists->no_priolist = true;
> > > > > + sched_engine->no_priolist = true;
> > > > > goto find_priolist;
> > > > > }
> > > > > }
> > > > > @@ -116,7 +116,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > > > > INIT_LIST_HEAD(&p->requests);
> > > > >
> > > > > rb_link_node(&p->node, rb, parent);
> > > > > - rb_insert_color_cached(&p->node, &execlists->queue, first);
> > > > > + rb_insert_color_cached(&p->node, &sched_engine->queue, first);
> > > > >
> > > > > return &p->requests;
> > > > > }
> > > > > @@ -184,7 +184,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > > > * We only need to kick the tasklet once for the high priority
> > > > > * new context we add into the queue.
> > > > > */
> > > > > - if (prio <= engine->execlists.queue_priority_hint)
> > > > > + if (prio <= engine->sched_engine->queue_priority_hint)
> > > > > return;
> > > > >
> > > > > rcu_read_lock();
> > > > > @@ -208,7 +208,7 @@ static void kick_submission(struct intel_engine_cs *engine,
> > > > > inflight->fence.context, inflight->fence.seqno,
> > > > > inflight->sched.attr.priority);
> > > > >
> > > > > - engine->execlists.queue_priority_hint = prio;
> > > > > + engine->sched_engine->queue_priority_hint = prio;
> > > > > if (need_preempt(prio, rq_prio(inflight)))
> > > > > tasklet_hi_schedule(&engine->execlists.tasklet);
> > > > >
> > > > > @@ -489,6 +489,31 @@ void i915_request_show_with_schedule(struct drm_printer *m,
> > > > > rcu_read_unlock();
> > > > > }
> > > > >
> > > > > +void i915_sched_engine_free(struct kref *kref)
> > > > > +{
> > > > > + struct i915_sched_engine *sched_engine =
> > > > > + container_of(kref, typeof(*sched_engine), ref);
> > > > > +
> > > > > + kfree(sched_engine);
> > > > > +}
> > > > > +
> > > > > +struct i915_sched_engine *
> > > > > +i915_sched_engine_create(unsigned int subclass)
> > > > > +{
> > > > > + struct i915_sched_engine *sched_engine;
> > > > > +
> > > > > + sched_engine = kzalloc(sizeof(*sched_engine), GFP_KERNEL);
> > > > > + if (!sched_engine)
> > > > > + return NULL;
> > > > > +
> > > > > + kref_init(&sched_engine->ref);
> > > > > +
> > > > > + sched_engine->queue = RB_ROOT_CACHED;
> > > > > + sched_engine->queue_priority_hint = INT_MIN;
> > > > > +
> > > > > + return sched_engine;
> > > > > +}
> > > > > +
> > > > > static void i915_global_scheduler_shrink(void)
> > > > > {
> > > > > kmem_cache_shrink(global.slab_dependencies);
> > > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > > > > index 858a0938f47a..91a04e34cac5 100644
> > > > > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > > > > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > > > > @@ -48,6 +48,24 @@ static inline void i915_priolist_free(struct i915_priolist *p)
> > > > > __i915_priolist_free(p);
> > > > > }
> > > > >
> > > > > +struct i915_sched_engine *
> > > > > +i915_sched_engine_create(unsigned int subclass);
> > > > > +
> > > > > +void i915_sched_engine_free(struct kref *kref);
> > > > > +
> > > > > +static inline struct i915_sched_engine *
> > > > > +i915_sched_engine_get(struct i915_sched_engine *sched_engine)
> > > > > +{
> > > > > + kref_get(&sched_engine->ref);
> > > > > + return sched_engine;
> > > > > +}
> > > > > +
> > > > > +static inline void
> > > > > +i915_sched_engine_put(struct i915_sched_engine *sched_engine)
> > > > > +{
> > > > > + kref_put(&sched_engine->ref, i915_sched_engine_free);
> > > > > +}
> > > > > +
> > > > > void i915_request_show_with_schedule(struct drm_printer *m,
> > > > > const struct i915_request *rq,
> > > > > const char *prefix,
> > > > > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > > > index 343ed44d5ed4..16a054e67760 100644
> > > > > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > > > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > > >
> > > > Dumb question for which I'm not really in the position of having an
> > > > opinion: Should this go in its own file?
> > > >
> > >
> > > I think this is in the right file.
> > >
> > > Matt
> > >
> > > > Patch seems mostly mechanically correct. Not sure where this is going
> > > > yet, though. I'll keep reading. :-)
> > > >
> > > > --Jason
> > > >
> > > >
> > > > > @@ -91,4 +91,37 @@ struct i915_dependency {
> > > > > &(rq__)->sched.signalers_list, \
> > > > > signal_link)
> > > > >
> > > > > +struct i915_sched_engine {
> > > > > + struct kref ref;
> > > > > +
> > > > > + /**
> > > > > + * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > > > > + */
> > > > > + struct i915_priolist default_priolist;
> > > > > +
> > > > > + /**
> > > > > + * @queue_priority_hint: Highest pending priority.
> > > > > + *
> > > > > + * When we add requests into the queue, or adjust the priority of
> > > > > + * executing requests, we compute the maximum priority of those
> > > > > + * pending requests. We can then use this value to determine if
> > > > > + * we need to preempt the executing requests to service the queue.
> > > > > + * However, since the we may have recorded the priority of an inflight
> > > > > + * request we wanted to preempt but since completed, at the time of
> > > > > + * dequeuing the priority hint may no longer may match the highest
> > > > > + * available request priority.
> > > > > + */
> > > > > + int queue_priority_hint;
> > > > > +
> > > > > + /**
> > > > > + * @queue: queue of requests, in priority lists
> > > > > + */
> > > > > + struct rb_root_cached queue;
> > > > > +
> > > > > + /**
> > > > > + * @no_priolist: priority lists disabled
> > > > > + */
> > > > > + bool no_priolist;
> > > > > +};
> > > > > +
> > > > > #endif /* _I915_SCHEDULER_TYPES_H_ */
> > > > > --
> > > > > 2.28.0
> > > > >
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 3/9] drm/i915: Add i915_sched_engine_reset_on_empty function
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 18:31 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 18:31 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Rather than touching schedule state in the generic PM code, reset the
> priolist allocation when empty in the submission code. Add a wrapper
> function to do this and update the backends to call it in the correct
> place.
Seems reasonable, I think. I'm by no means an expert but
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
anyway. My one suggestion would be to tweak the commit message to
talk about the functional change rather than the helper. Something
like
drm/i915: Reset sched_engine.no_priolist immediately after dequeue
Typically patches which say "add a helper function" don't come with a
non-trivial functional change.
--Jason
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_engine_pm.c | 2 --
> drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 1 +
> drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 ++
> drivers/gpu/drm/i915/i915_scheduler.h | 7 +++++++
> 4 files changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> index b6a00dd72808..1f07ac4e0672 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> @@ -280,8 +280,6 @@ static int __engine_park(struct intel_wakeref *wf)
> if (engine->park)
> engine->park(engine);
>
> - engine->sched_engine->no_priolist = false;
> -
> /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> intel_gt_pm_put_async(engine->gt);
> return 0;
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 2326a73af6d3..609753b5401a 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -1553,6 +1553,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> * interrupt for secondary ports).
> */
> sched_engine->queue_priority_hint = queue_prio(sched_engine);
> + i915_sched_engine_reset_on_empty(sched_engine);
> spin_unlock(&engine->active.lock);
>
> /*
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 5d00f2e3c1de..f4a6fbfaf82e 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -263,6 +263,8 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
>
> __guc_dequeue(engine);
>
> + i915_sched_engine_reset_on_empty(engine->sched_engine);
> +
> spin_unlock_irqrestore(&engine->active.lock, flags);
> }
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 5bec7b3b8456..713c38c99de9 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -72,6 +72,13 @@ i915_sched_engine_is_empty(struct i915_sched_engine *sched_engine)
> return RB_EMPTY_ROOT(&sched_engine->queue.rb_root);
> }
>
> +static inline void
> +i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
> +{
> + if (i915_sched_engine_is_empty(sched_engine))
> + sched_engine->no_priolist = false;
> +}
> +
> void i915_request_show_with_schedule(struct drm_printer *m,
> const struct i915_request *rq,
> const char *prefix,
> --
> 2.28.0
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 3/9] drm/i915: Add i915_sched_engine_reset_on_empty function
@ 2021-06-04 18:31 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 18:31 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Rather than touching schedule state in the generic PM code, reset the
> priolist allocation when empty in the submission code. Add a wrapper
> function to do this and update the backends to call it in the correct
> place.
Seems reasonable, I think. I'm by no means an expert but
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
anyway. My one suggestion would be to tweak the commit message to
talk about the functional change rather than the helper. Something
like
drm/i915: Reset sched_engine.no_priolist immediately after dequeue
Typically patches which say "add a helper function" don't come with a
non-trivial functional change.
--Jason
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_engine_pm.c | 2 --
> drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 1 +
> drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 ++
> drivers/gpu/drm/i915/i915_scheduler.h | 7 +++++++
> 4 files changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> index b6a00dd72808..1f07ac4e0672 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> @@ -280,8 +280,6 @@ static int __engine_park(struct intel_wakeref *wf)
> if (engine->park)
> engine->park(engine);
>
> - engine->sched_engine->no_priolist = false;
> -
> /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> intel_gt_pm_put_async(engine->gt);
> return 0;
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 2326a73af6d3..609753b5401a 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -1553,6 +1553,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> * interrupt for secondary ports).
> */
> sched_engine->queue_priority_hint = queue_prio(sched_engine);
> + i915_sched_engine_reset_on_empty(sched_engine);
> spin_unlock(&engine->active.lock);
>
> /*
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 5d00f2e3c1de..f4a6fbfaf82e 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -263,6 +263,8 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
>
> __guc_dequeue(engine);
>
> + i915_sched_engine_reset_on_empty(engine->sched_engine);
> +
> spin_unlock_irqrestore(&engine->active.lock, flags);
> }
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 5bec7b3b8456..713c38c99de9 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -72,6 +72,13 @@ i915_sched_engine_is_empty(struct i915_sched_engine *sched_engine)
> return RB_EMPTY_ROOT(&sched_engine->queue.rb_root);
> }
>
> +static inline void
> +i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
> +{
> + if (i915_sched_engine_is_empty(sched_engine))
> + sched_engine->no_priolist = false;
> +}
> +
> void i915_request_show_with_schedule(struct drm_printer *m,
> const struct i915_request *rq,
> const char *prefix,
> --
> 2.28.0
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 3/9] drm/i915: Add i915_sched_engine_reset_on_empty function
2021-06-04 18:31 ` Jason Ekstrand
@ 2021-06-04 18:49 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 18:49 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 01:31:42PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Rather than touching schedule state in the generic PM code, reset the
> > priolist allocation when empty in the submission code. Add a wrapper
> > function to do this and update the backends to call it in the correct
> > place.
>
> Seems reasonable, I think. I'm by no means an expert but
>
> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
>
> anyway. My one suggestion would be to tweak the commit message to
> talk about the functional change rather than the helper. Something
> like
>
> drm/i915: Reset sched_engine.no_priolist immediately after dequeue
>
> Typically patches which say "add a helper function" don't come with a
> non-trivial functional change.
>
Agree.
Thanks - Matt
> --Jason
>
>
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gt/intel_engine_pm.c | 2 --
> > drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 1 +
> > drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 ++
> > drivers/gpu/drm/i915/i915_scheduler.h | 7 +++++++
> > 4 files changed, 10 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > index b6a00dd72808..1f07ac4e0672 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > @@ -280,8 +280,6 @@ static int __engine_park(struct intel_wakeref *wf)
> > if (engine->park)
> > engine->park(engine);
> >
> > - engine->sched_engine->no_priolist = false;
> > -
> > /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> > intel_gt_pm_put_async(engine->gt);
> > return 0;
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 2326a73af6d3..609753b5401a 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -1553,6 +1553,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > * interrupt for secondary ports).
> > */
> > sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > + i915_sched_engine_reset_on_empty(sched_engine);
> > spin_unlock(&engine->active.lock);
> >
> > /*
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 5d00f2e3c1de..f4a6fbfaf82e 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -263,6 +263,8 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
> >
> > __guc_dequeue(engine);
> >
> > + i915_sched_engine_reset_on_empty(engine->sched_engine);
> > +
> > spin_unlock_irqrestore(&engine->active.lock, flags);
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > index 5bec7b3b8456..713c38c99de9 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > @@ -72,6 +72,13 @@ i915_sched_engine_is_empty(struct i915_sched_engine *sched_engine)
> > return RB_EMPTY_ROOT(&sched_engine->queue.rb_root);
> > }
> >
> > +static inline void
> > +i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
> > +{
> > + if (i915_sched_engine_is_empty(sched_engine))
> > + sched_engine->no_priolist = false;
> > +}
> > +
> > void i915_request_show_with_schedule(struct drm_printer *m,
> > const struct i915_request *rq,
> > const char *prefix,
> > --
> > 2.28.0
> >
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 3/9] drm/i915: Add i915_sched_engine_reset_on_empty function
@ 2021-06-04 18:49 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 18:49 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 01:31:42PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Rather than touching schedule state in the generic PM code, reset the
> > priolist allocation when empty in the submission code. Add a wrapper
> > function to do this and update the backends to call it in the correct
> > place.
>
> Seems reasonable, I think. I'm by no means an expert but
>
> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
>
> anyway. My one suggestion would be to tweak the commit message to
> talk about the functional change rather than the helper. Something
> like
>
> drm/i915: Reset sched_engine.no_priolist immediately after dequeue
>
> Typically patches which say "add a helper function" don't come with a
> non-trivial functional change.
>
Agree.
Thanks - Matt
> --Jason
>
>
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gt/intel_engine_pm.c | 2 --
> > drivers/gpu/drm/i915/gt/intel_execlists_submission.c | 1 +
> > drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 ++
> > drivers/gpu/drm/i915/i915_scheduler.h | 7 +++++++
> > 4 files changed, 10 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > index b6a00dd72808..1f07ac4e0672 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
> > @@ -280,8 +280,6 @@ static int __engine_park(struct intel_wakeref *wf)
> > if (engine->park)
> > engine->park(engine);
> >
> > - engine->sched_engine->no_priolist = false;
> > -
> > /* While gt calls i915_vma_parked(), we have to break the lock cycle */
> > intel_gt_pm_put_async(engine->gt);
> > return 0;
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 2326a73af6d3..609753b5401a 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -1553,6 +1553,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > * interrupt for secondary ports).
> > */
> > sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > + i915_sched_engine_reset_on_empty(sched_engine);
> > spin_unlock(&engine->active.lock);
> >
> > /*
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 5d00f2e3c1de..f4a6fbfaf82e 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -263,6 +263,8 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
> >
> > __guc_dequeue(engine);
> >
> > + i915_sched_engine_reset_on_empty(engine->sched_engine);
> > +
> > spin_unlock_irqrestore(&engine->active.lock, flags);
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > index 5bec7b3b8456..713c38c99de9 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > @@ -72,6 +72,13 @@ i915_sched_engine_is_empty(struct i915_sched_engine *sched_engine)
> > return RB_EMPTY_ROOT(&sched_engine->queue.rb_root);
> > }
> >
> > +static inline void
> > +i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
> > +{
> > + if (i915_sched_engine_is_empty(sched_engine))
> > + sched_engine->no_priolist = false;
> > +}
> > +
> > void i915_request_show_with_schedule(struct drm_printer *m,
> > const struct i915_request *rq,
> > const char *prefix,
> > --
> > 2.28.0
> >
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 4/9] drm/i915: Move active tracking to i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 19:00 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 19:00 UTC (permalink / raw)
To: Matthew Brost
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Move active request tracking and its lock to i915_sched_engine. This
> lock is also the submission lock so having it in the i915_sched_engine
> is the correct place.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_engine.h | 2 -
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 43 +++-----
> drivers/gpu/drm/i915/gt/intel_engine_types.h | 6 --
> .../drm/i915/gt/intel_execlists_submission.c | 98 ++++++++++---------
> .../gpu/drm/i915/gt/intel_ring_submission.c | 12 +--
> drivers/gpu/drm/i915/gt/mock_engine.c | 7 +-
> drivers/gpu/drm/i915/gt/selftest_execlists.c | 4 +-
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 20 ++--
> drivers/gpu/drm/i915/i915_gpu_error.c | 4 +-
> drivers/gpu/drm/i915/i915_request.c | 32 +++---
> drivers/gpu/drm/i915/i915_request.h | 2 +-
> drivers/gpu/drm/i915/i915_scheduler.c | 30 ++++--
> drivers/gpu/drm/i915/i915_scheduler_types.h | 9 ++
> 13 files changed, 134 insertions(+), 135 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
> index 8d9184920c51..a8b2174b4395 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
> @@ -257,8 +257,6 @@ intel_engine_find_active_request(struct intel_engine_cs *engine);
>
> u32 intel_engine_context_size(struct intel_gt *gt, u8 class);
>
> -void intel_engine_init_active(struct intel_engine_cs *engine,
> - unsigned int subclass);
> #define ENGINE_PHYSICAL 0
> #define ENGINE_MOCK 1
> #define ENGINE_VIRTUAL 2
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index 85f2effe9dc6..33d879137908 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -719,7 +719,6 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> if (err)
> goto err_cmd_parser;
>
> - intel_engine_init_active(engine, ENGINE_PHYSICAL);
> intel_engine_init_execlists(engine);
> intel_engine_init__pm(engine);
> intel_engine_init_retire(engine);
> @@ -778,11 +777,11 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
> frame->rq.ring = &frame->ring;
>
> mutex_lock(&ce->timeline->mutex);
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
>
> dw = engine->emit_fini_breadcrumb(&frame->rq, frame->cs) - frame->cs;
>
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> mutex_unlock(&ce->timeline->mutex);
>
> GEM_BUG_ON(dw & 1); /* RING_TAIL must be qword aligned */
> @@ -791,28 +790,6 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
> return dw;
> }
>
> -void
> -intel_engine_init_active(struct intel_engine_cs *engine, unsigned int subclass)
> -{
> - INIT_LIST_HEAD(&engine->active.requests);
> - INIT_LIST_HEAD(&engine->active.hold);
> -
> - spin_lock_init(&engine->active.lock);
> - lockdep_set_subclass(&engine->active.lock, subclass);
> -
> - /*
> - * Due to an interesting quirk in lockdep's internal debug tracking,
> - * after setting a subclass we must ensure the lock is used. Otherwise,
> - * nr_unused_locks is incremented once too often.
> - */
> -#ifdef CONFIG_DEBUG_LOCK_ALLOC
> - local_irq_disable();
> - lock_map_acquire(&engine->active.lock.dep_map);
> - lock_map_release(&engine->active.lock.dep_map);
> - local_irq_enable();
> -#endif
> -}
> -
> static struct intel_context *
> create_pinned_context(struct intel_engine_cs *engine,
> unsigned int hwsp,
> @@ -960,7 +937,7 @@ int intel_engines_init(struct intel_gt *gt)
> */
> void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> {
> - GEM_BUG_ON(!list_empty(&engine->active.requests));
> + GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
> tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
>
> i915_sched_engine_put(engine->sched_engine);
> @@ -1353,7 +1330,7 @@ static struct intel_timeline *get_timeline(struct i915_request *rq)
> struct intel_timeline *tl;
>
> /*
> - * Even though we are holding the engine->active.lock here, there
> + * Even though we are holding the engine->sched_engine->lock here, there
> * is no control over the submission queue per-se and we are
> * inspecting the active state at a random point in time, with an
> * unknown queue. Play safe and make sure the timeline remains valid.
> @@ -1700,7 +1677,7 @@ void intel_engine_dump(struct intel_engine_cs *engine,
>
> drm_printf(m, "\tRequests:\n");
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> rq = intel_engine_find_active_request(engine);
> if (rq) {
> struct intel_timeline *tl = get_timeline(rq);
> @@ -1731,8 +1708,9 @@ void intel_engine_dump(struct intel_engine_cs *engine,
> hexdump(m, rq->context->lrc_reg_state, PAGE_SIZE);
> }
> }
> - drm_printf(m, "\tOn hold?: %lu\n", list_count(&engine->active.hold));
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + drm_printf(m, "\tOn hold?: %lu\n",
> + list_count(&engine->sched_engine->hold));
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
>
> drm_printf(m, "\tMMIO base: 0x%08x\n", engine->mmio_base);
> wakeref = intel_runtime_pm_get_if_in_use(engine->uncore->rpm);
> @@ -1812,7 +1790,7 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
> * At all other times, we must assume the GPU is still running, but
> * we only care about the snapshot of this moment.
> */
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
>
> rcu_read_lock();
> request = execlists_active(&engine->execlists);
> @@ -1830,7 +1808,8 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
> if (active)
> return active;
>
> - list_for_each_entry(request, &engine->active.requests, sched.link) {
> + list_for_each_entry(request, &engine->sched_engine->requests,
> + sched.link) {
> if (__i915_request_is_complete(request))
> continue;
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> index 86b41ddec373..7197b9fa5e35 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> @@ -302,12 +302,6 @@ struct intel_engine_cs {
>
> struct intel_sseu sseu;
>
> - struct {
> - spinlock_t lock;
> - struct list_head requests;
> - struct list_head hold; /* ready requests, but on hold */
> - } active;
> -
> struct i915_sched_engine *sched_engine;
>
> /* keep a request in reserve for a [pm] barrier under oom */
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 609753b5401a..0413ba87cc1e 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -325,7 +325,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> * Check against the first request in ELSP[1], it will, thanks to the
> * power of PI, be the highest priority of that context.
> */
> - if (!list_is_last(&rq->sched.link, &engine->active.requests) &&
> + if (!list_is_last(&rq->sched.link, &engine->sched_engine->requests) &&
> rq_prio(list_next_entry(rq, sched.link)) > last_prio)
> return true;
>
> @@ -367,10 +367,10 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> struct list_head *pl;
> int prio = I915_PRIORITY_INVALID;
>
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
>
> list_for_each_entry_safe_reverse(rq, rn,
> - &engine->active.requests,
> + &engine->sched_engine->requests,
> sched.link) {
> if (__i915_request_is_complete(rq)) {
> list_del_init(&rq->sched.link);
> @@ -534,13 +534,13 @@ resubmit_virtual_request(struct i915_request *rq, struct virtual_engine *ve)
> {
> struct intel_engine_cs *engine = rq->engine;
>
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
>
> clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> WRITE_ONCE(rq->engine, &ve->base);
> ve->base.submit_request(rq);
>
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> }
>
> static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
> @@ -579,7 +579,7 @@ static void __execlists_schedule_out(struct i915_request * const rq,
> unsigned int ccid;
>
> /*
> - * NB process_csb() is not under the engine->active.lock and hence
> + * NB process_csb() is not under the engine->sched_engine->lock and hence
> * schedule_out can race with schedule_in meaning that we should
> * refrain from doing non-trivial work here.
> */
> @@ -1133,7 +1133,8 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> return false;
>
> /* If ELSP[1] is occupied, always check to see if worth slicing */
> - if (!list_is_last_rcu(&rq->sched.link, &engine->active.requests)) {
> + if (!list_is_last_rcu(&rq->sched.link,
> + &engine->sched_engine->requests)) {
> ENGINE_TRACE(engine, "timeslice required for second inflight context\n");
> return true;
> }
> @@ -1266,7 +1267,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> * and context switches) submission.
> */
>
> - spin_lock(&engine->active.lock);
> + spin_lock(&sched_engine->lock);
>
> /*
> * If the queue is higher priority than the last
> @@ -1366,7 +1367,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> * Even if ELSP[1] is occupied and not worthy
> * of timeslices, our queue might be.
> */
> - spin_unlock(&engine->active.lock);
> + spin_unlock(&sched_engine->lock);
> return;
> }
> }
> @@ -1376,7 +1377,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> while ((ve = first_virtual_engine(engine))) {
> struct i915_request *rq;
>
> - spin_lock(&ve->base.active.lock);
> + spin_lock(&ve->base.sched_engine->lock);
>
> rq = ve->request;
> if (unlikely(!virtual_matches(ve, rq, engine)))
> @@ -1386,13 +1387,13 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> GEM_BUG_ON(rq->context != &ve->context);
>
> if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> - spin_unlock(&ve->base.active.lock);
> + spin_unlock(&ve->base.sched_engine->lock);
> break;
> }
>
> if (last && !can_merge_rq(last, rq)) {
> - spin_unlock(&ve->base.active.lock);
> - spin_unlock(&engine->active.lock);
> + spin_unlock(&ve->base.sched_engine->lock);
> + spin_unlock(&engine->sched_engine->lock);
> return; /* leave this for another sibling */
> }
>
> @@ -1438,7 +1439,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
>
> i915_request_put(rq);
> unlock:
> - spin_unlock(&ve->base.active.lock);
> + spin_unlock(&ve->base.sched_engine->lock);
>
> /*
> * Hmm, we have a bunch of virtual engine requests,
> @@ -1554,7 +1555,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> */
> sched_engine->queue_priority_hint = queue_prio(sched_engine);
> i915_sched_engine_reset_on_empty(sched_engine);
> - spin_unlock(&engine->active.lock);
> + spin_unlock(&sched_engine->lock);
>
> /*
> * We can skip poking the HW if we ended up with exactly the same set
> @@ -1981,7 +1982,8 @@ static void __execlists_hold(struct i915_request *rq)
> __i915_request_unsubmit(rq);
>
> clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> - list_move_tail(&rq->sched.link, &rq->engine->active.hold);
> + list_move_tail(&rq->sched.link,
> + &rq->engine->sched_engine->hold);
> i915_request_set_hold(rq);
> RQ_TRACE(rq, "on hold\n");
>
> @@ -2018,7 +2020,7 @@ static bool execlists_hold(struct intel_engine_cs *engine,
> if (i915_request_on_hold(rq))
> return false;
>
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
>
> if (__i915_request_is_complete(rq)) { /* too late! */
> rq = NULL;
> @@ -2034,10 +2036,10 @@ static bool execlists_hold(struct intel_engine_cs *engine,
> GEM_BUG_ON(i915_request_on_hold(rq));
> GEM_BUG_ON(rq->engine != engine);
> __execlists_hold(rq);
> - GEM_BUG_ON(list_empty(&engine->active.hold));
> + GEM_BUG_ON(list_empty(&engine->sched_engine->hold));
>
> unlock:
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> return rq;
> }
>
> @@ -2117,7 +2119,7 @@ static void __execlists_unhold(struct i915_request *rq)
> static void execlists_unhold(struct intel_engine_cs *engine,
> struct i915_request *rq)
> {
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
>
> /*
> * Move this request back to the priority queue, and all of its
> @@ -2130,7 +2132,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> tasklet_hi_schedule(&engine->execlists.tasklet);
> }
>
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> }
>
> struct execlists_capture {
> @@ -2260,13 +2262,13 @@ static void execlists_capture(struct intel_engine_cs *engine)
> if (!cap)
> return;
>
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
> cap->rq = active_context(engine, active_ccid(engine));
> if (cap->rq) {
> cap->rq = active_request(cap->rq->context->timeline, cap->rq);
> cap->rq = i915_request_get_rcu(cap->rq);
> }
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> if (!cap->rq)
> goto err_free;
>
> @@ -2470,7 +2472,7 @@ static bool ancestor_on_hold(const struct intel_engine_cs *engine,
> const struct i915_request *rq)
> {
> GEM_BUG_ON(i915_request_on_hold(rq));
> - return !list_empty(&engine->active.hold) && hold_request(rq);
> + return !list_empty(&engine->sched_engine->hold) && hold_request(rq);
> }
>
> static void execlists_submit_request(struct i915_request *request)
> @@ -2479,11 +2481,12 @@ static void execlists_submit_request(struct i915_request *request)
> unsigned long flags;
>
> /* Will be called from irq-context when using foreign fences. */
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> if (unlikely(ancestor_on_hold(engine, request))) {
> RQ_TRACE(request, "ancestor on hold\n");
> - list_add_tail(&request->sched.link, &engine->active.hold);
> + list_add_tail(&request->sched.link,
> + &engine->sched_engine->hold);
> i915_request_set_hold(request);
> } else {
> queue_request(engine, request);
> @@ -2495,7 +2498,7 @@ static void execlists_submit_request(struct i915_request *request)
> __execlists_kick(&engine->execlists);
> }
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static int
> @@ -2959,9 +2962,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
>
> /* Push back any incomplete requests for replay after the reset. */
> rcu_read_lock();
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> __unwind_incomplete_requests(engine);
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> rcu_read_unlock();
> }
>
> @@ -3001,10 +3004,10 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> execlists_reset_csb(engine, true);
>
> rcu_read_lock();
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> /* Mark all executing requests as skipped. */
> - list_for_each_entry(rq, &engine->active.requests, sched.link)
> + list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
> i915_request_put(i915_request_mark_eio(rq));
> intel_engine_signal_breadcrumbs(engine);
>
> @@ -3024,7 +3027,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> }
>
> /* On-hold requests will be flushed to timeline upon their release */
> - list_for_each_entry(rq, &engine->active.hold, sched.link)
> + list_for_each_entry(rq, &sched_engine->hold, sched.link)
> i915_request_put(i915_request_mark_eio(rq));
>
> /* Cancel all attached virtual engines */
> @@ -3035,7 +3038,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> rb_erase_cached(rb, &execlists->virtual);
> RB_CLEAR_NODE(rb);
>
> - spin_lock(&ve->base.active.lock);
> + spin_lock(&ve->base.sched_engine->lock);
> rq = fetch_and_zero(&ve->request);
> if (rq) {
> if (i915_request_mark_eio(rq)) {
> @@ -3047,7 +3050,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
>
> ve->base.sched_engine->queue_priority_hint = INT_MIN;
> }
> - spin_unlock(&ve->base.active.lock);
> + spin_unlock(&ve->base.sched_engine->lock);
> }
>
> /* Remaining _unready_ requests will be nop'ed when submitted */
> @@ -3058,7 +3061,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> execlists->tasklet.callback = nop_submission_tasklet;
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> rcu_read_unlock();
> }
>
> @@ -3304,7 +3307,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> if (unlikely(ve->request)) {
> struct i915_request *old;
>
> - spin_lock_irq(&ve->base.active.lock);
> + spin_lock_irq(&ve->base.sched_engine->lock);
>
> old = fetch_and_zero(&ve->request);
> if (old) {
> @@ -3313,7 +3316,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> i915_request_put(old);
> }
>
> - spin_unlock_irq(&ve->base.active.lock);
> + spin_unlock_irq(&ve->base.sched_engine->lock);
> }
>
> /*
> @@ -3333,13 +3336,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> if (RB_EMPTY_NODE(node))
> continue;
>
> - spin_lock_irq(&sibling->active.lock);
> + spin_lock_irq(&sibling->sched_engine->lock);
>
> /* Detachment is lazily performed in the execlists tasklet */
> if (!RB_EMPTY_NODE(node))
> rb_erase_cached(node, &sibling->execlists.virtual);
>
> - spin_unlock_irq(&sibling->active.lock);
> + spin_unlock_irq(&sibling->sched_engine->lock);
> }
> GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
> GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> @@ -3509,7 +3512,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> if (!READ_ONCE(ve->request))
> break; /* already handled by a sibling's tasklet */
>
> - spin_lock_irq(&sibling->active.lock);
> + spin_lock_irq(&sibling->sched_engine->lock);
>
> if (unlikely(!(mask & sibling->mask))) {
> if (!RB_EMPTY_NODE(&node->rb)) {
> @@ -3562,7 +3565,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> tasklet_hi_schedule(&sibling->execlists.tasklet);
>
> unlock_engine:
> - spin_unlock_irq(&sibling->active.lock);
> + spin_unlock_irq(&sibling->sched_engine->lock);
>
> if (intel_context_inflight(&ve->context))
> break;
> @@ -3580,7 +3583,7 @@ static void virtual_submit_request(struct i915_request *rq)
>
> GEM_BUG_ON(ve->base.submit_request != virtual_submit_request);
>
> - spin_lock_irqsave(&ve->base.active.lock, flags);
> + spin_lock_irqsave(&ve->base.sched_engine->lock, flags);
>
> /* By the time we resubmit a request, it may be completed */
> if (__i915_request_is_complete(rq)) {
> @@ -3603,7 +3606,7 @@ static void virtual_submit_request(struct i915_request *rq)
> tasklet_hi_schedule(&ve->base.execlists.tasklet);
>
> unlock:
> - spin_unlock_irqrestore(&ve->base.active.lock, flags);
> + spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
> }
>
> static struct ve_bond *
> @@ -3687,7 +3690,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
>
> snprintf(ve->base.name, sizeof(ve->base.name), "virtual");
>
> - intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> intel_engine_init_execlists(&ve->base);
>
> ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> @@ -3860,17 +3862,17 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> unsigned int max)
> {
> const struct intel_engine_execlists *execlists = &engine->execlists;
> - const struct i915_sched_engine *sched_engine = engine->sched_engine;
> + struct i915_sched_engine *sched_engine = engine->sched_engine;
> struct i915_request *rq, *last;
> unsigned long flags;
> unsigned int count;
> struct rb_node *rb;
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&sched_engine->lock, flags);
>
> last = NULL;
> count = 0;
> - list_for_each_entry(rq, &engine->active.requests, sched.link) {
> + list_for_each_entry(rq, &sched_engine->requests, sched.link) {
> if (count++ < max - 1)
> show_request(m, rq, "\t\t", 0);
> else
> @@ -3933,7 +3935,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> show_request(m, last, "\t\t", 0);
> }
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&sched_engine->lock, flags);
> }
>
> #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
> diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> index 2b6dffcc2262..c98ae0f58790 100644
> --- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> @@ -339,9 +339,9 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
> u32 head;
>
> rq = NULL;
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> rcu_read_lock();
> - list_for_each_entry(pos, &engine->active.requests, sched.link) {
> + list_for_each_entry(pos, &engine->sched_engine->requests, sched.link) {
> if (!__i915_request_is_complete(pos)) {
> rq = pos;
> break;
> @@ -396,7 +396,7 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
> }
> engine->legacy.ring->head = intel_ring_wrap(engine->legacy.ring, head);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void reset_finish(struct intel_engine_cs *engine)
> @@ -408,16 +408,16 @@ static void reset_cancel(struct intel_engine_cs *engine)
> struct i915_request *request;
> unsigned long flags;
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> /* Mark all submitted requests as skipped. */
> - list_for_each_entry(request, &engine->active.requests, sched.link)
> + list_for_each_entry(request, &engine->sched_engine->requests, sched.link)
> i915_request_put(i915_request_mark_eio(request));
> intel_engine_signal_breadcrumbs(engine);
>
> /* Remaining _unready_ requests will be nop'ed when submitted */
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void i9xx_submit_request(struct i915_request *request)
> diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> index b1fdba13e900..a49fd3039f13 100644
> --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> @@ -253,10 +253,10 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
>
> del_timer_sync(&mock->hw_delay);
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> /* Mark all submitted requests as skipped. */
> - list_for_each_entry(rq, &engine->active.requests, sched.link)
> + list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
> i915_request_put(i915_request_mark_eio(rq));
> intel_engine_signal_breadcrumbs(engine);
>
> @@ -269,7 +269,7 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
> }
> INIT_LIST_HEAD(&mock->hw_queue);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void mock_reset_finish(struct intel_engine_cs *engine)
> @@ -350,7 +350,6 @@ int mock_engine_init(struct intel_engine_cs *engine)
> if (!engine->sched_engine)
> return -ENOMEM;
>
> - intel_engine_init_active(engine, ENGINE_MOCK);
> intel_engine_init_execlists(engine);
> intel_engine_init__pm(engine);
> intel_engine_init_retire(engine);
> diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> index 1f93591a8c69..fd824fa329c2 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> @@ -4599,9 +4599,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
> GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
>
> /* Fake a preemption event; failed of course */
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
> __unwind_incomplete_requests(engine);
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> GEM_BUG_ON(rq->engine != engine);
>
> /* Reset the engine while keeping our active request on hold */
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index f4a6fbfaf82e..2551019ee217 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -190,7 +190,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> bool submit = false;
> struct rb_node *rb;
>
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&sched_engine->lock);
>
> if (last) {
> if (*++first)
> @@ -247,7 +247,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
> struct i915_request **port, *rq;
> unsigned long flags;
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> for (port = execlists->inflight; (rq = *port); port++) {
> if (!i915_request_completed(rq))
> @@ -265,7 +265,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
>
> i915_sched_engine_reset_on_empty(engine->sched_engine);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
> @@ -322,7 +322,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> struct i915_request *rq;
> unsigned long flags;
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> /* Push back any incomplete requests for replay after the reset. */
> rq = execlists_unwind_incomplete_requests(execlists);
> @@ -336,7 +336,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> guc_reset_state(rq->context, engine, rq->head, stalled);
>
> out_unlock:
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void guc_reset_cancel(struct intel_engine_cs *engine)
> @@ -362,10 +362,10 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> * submission's irq state, we also wish to remind ourselves that
> * it is irq state.)
> */
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&sched_engine->lock, flags);
>
> /* Mark all executing requests as skipped. */
> - list_for_each_entry(rq, &engine->active.requests, sched.link) {
> + list_for_each_entry(rq, &sched_engine->requests, sched.link) {
> i915_request_set_error_once(rq, -EIO);
> i915_request_mark_complete(rq);
> }
> @@ -390,7 +390,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> sched_engine->queue_priority_hint = INT_MIN;
> sched_engine->queue = RB_ROOT_CACHED;
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&sched_engine->lock, flags);
> }
>
> static void guc_reset_finish(struct intel_engine_cs *engine)
> @@ -539,7 +539,7 @@ static void guc_submit_request(struct i915_request *rq)
> unsigned long flags;
>
> /* Will be called from irq-context when using foreign fences. */
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> queue_request(engine, rq, rq_prio(rq));
>
> @@ -548,7 +548,7 @@ static void guc_submit_request(struct i915_request *rq)
>
> tasklet_hi_schedule(&engine->execlists.tasklet);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void sanitize_hwsp(struct intel_engine_cs *engine)
> diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
> index 833d3e8b7631..cd005f0a203c 100644
> --- a/drivers/gpu/drm/i915/i915_gpu_error.c
> +++ b/drivers/gpu/drm/i915/i915_gpu_error.c
> @@ -1436,12 +1436,12 @@ capture_engine(struct intel_engine_cs *engine,
> if (!ee)
> return NULL;
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> rq = intel_engine_find_active_request(engine);
> if (rq)
> capture = intel_engine_coredump_add_request(ee, rq,
> ATOMIC_MAYFAIL);
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> if (!capture) {
> kfree(ee);
> return NULL;
> diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
> index 970d8f4986bb..95e859c319ca 100644
> --- a/drivers/gpu/drm/i915/i915_request.c
> +++ b/drivers/gpu/drm/i915/i915_request.c
> @@ -272,11 +272,11 @@ i915_request_active_engine(struct i915_request *rq,
> * check that we have acquired the lock on the final engine.
> */
> locked = READ_ONCE(rq->engine);
> - spin_lock_irq(&locked->active.lock);
> + spin_lock_irq(&locked->sched_engine->lock);
> while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
> - spin_unlock(&locked->active.lock);
> + spin_unlock(&locked->sched_engine->lock);
> locked = engine;
> - spin_lock(&locked->active.lock);
> + spin_lock(&locked->sched_engine->lock);
> }
>
> if (i915_request_is_active(rq)) {
> @@ -285,7 +285,7 @@ i915_request_active_engine(struct i915_request *rq,
> ret = true;
> }
>
> - spin_unlock_irq(&locked->active.lock);
> + spin_unlock_irq(&locked->sched_engine->lock);
>
> return ret;
> }
> @@ -302,10 +302,10 @@ static void remove_from_engine(struct i915_request *rq)
> * check that the rq still belongs to the newly locked engine.
> */
> locked = READ_ONCE(rq->engine);
> - spin_lock_irq(&locked->active.lock);
> + spin_lock_irq(&locked->sched_engine->lock);
> while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
> - spin_unlock(&locked->active.lock);
> - spin_lock(&engine->active.lock);
> + spin_unlock(&locked->sched_engine->lock);
> + spin_lock(&engine->sched_engine->lock);
> locked = engine;
> }
> list_del_init(&rq->sched.link);
> @@ -316,7 +316,7 @@ static void remove_from_engine(struct i915_request *rq)
> /* Prevent further __await_execution() registering a cb, then flush */
> set_bit(I915_FENCE_FLAG_ACTIVE, &rq->fence.flags);
>
> - spin_unlock_irq(&locked->active.lock);
> + spin_unlock_irq(&locked->sched_engine->lock);
>
> __notify_execute_cb_imm(rq);
> }
> @@ -637,7 +637,7 @@ bool __i915_request_submit(struct i915_request *request)
> RQ_TRACE(request, "\n");
>
> GEM_BUG_ON(!irqs_disabled());
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
>
> /*
> * With the advent of preempt-to-busy, we frequently encounter
> @@ -649,7 +649,7 @@ bool __i915_request_submit(struct i915_request *request)
> *
> * We must remove the request from the caller's priority queue,
> * and the caller must only call us when the request is in their
> - * priority queue, under the active.lock. This ensures that the
> + * priority queue, under the sched_engine->lock. This ensures that the
> * request has *not* yet been retired and we can safely move
> * the request into the engine->active.list where it will be
> * dropped upon retiring. (Otherwise if resubmit a *retired*
> @@ -694,7 +694,7 @@ bool __i915_request_submit(struct i915_request *request)
> result = true;
>
> GEM_BUG_ON(test_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags));
> - list_move_tail(&request->sched.link, &engine->active.requests);
> + list_move_tail(&request->sched.link, &engine->sched_engine->requests);
> active:
> clear_bit(I915_FENCE_FLAG_PQUEUE, &request->fence.flags);
> set_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags);
> @@ -724,11 +724,11 @@ void i915_request_submit(struct i915_request *request)
> unsigned long flags;
>
> /* Will be called from irq-context when using foreign fences. */
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> __i915_request_submit(request);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> void __i915_request_unsubmit(struct i915_request *request)
> @@ -742,7 +742,7 @@ void __i915_request_unsubmit(struct i915_request *request)
> RQ_TRACE(request, "\n");
>
> GEM_BUG_ON(!irqs_disabled());
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
>
> /*
> * Before we remove this breadcrumb from the signal list, we have
> @@ -775,11 +775,11 @@ void i915_request_unsubmit(struct i915_request *request)
> unsigned long flags;
>
> /* Will be called from irq-context when using foreign fences. */
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> __i915_request_unsubmit(request);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void __cancel_request(struct i915_request *rq)
> diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
> index 270f6cd37650..239964bec1fa 100644
> --- a/drivers/gpu/drm/i915/i915_request.h
> +++ b/drivers/gpu/drm/i915/i915_request.h
> @@ -613,7 +613,7 @@ i915_request_active_timeline(const struct i915_request *rq)
> * this submission.
> */
> return rcu_dereference_protected(rq->timeline,
> - lockdep_is_held(&rq->engine->active.lock));
> + lockdep_is_held(&rq->engine->sched_engine->lock));
> }
>
> static inline u32
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 4953874a9ba6..4bc6969f6a97 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -68,7 +68,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> struct rb_node **parent, *rb;
> bool first = true;
>
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
> assert_priolists(sched_engine);
>
> if (unlikely(sched_engine->no_priolist))
> @@ -147,9 +147,9 @@ sched_lock_engine(const struct i915_sched_node *node,
> * check that the rq still belongs to the newly locked engine.
> */
> while (locked != (engine = READ_ONCE(rq->engine))) {
> - spin_unlock(&locked->active.lock);
> + spin_unlock(&locked->sched_engine->lock);
> memset(cache, 0, sizeof(*cache));
> - spin_lock(&engine->active.lock);
> + spin_lock(&engine->sched_engine->lock);
> locked = engine;
> }
>
> @@ -296,7 +296,7 @@ static void __i915_schedule(struct i915_sched_node *node,
>
> memset(&cache, 0, sizeof(cache));
> engine = node_to_request(node)->engine;
> - spin_lock(&engine->active.lock);
> + spin_lock(&engine->sched_engine->lock);
>
> /* Fifo and depth-first replacement ensure our deps execute before us */
> engine = sched_lock_engine(node, engine, &cache);
> @@ -305,7 +305,7 @@ static void __i915_schedule(struct i915_sched_node *node,
>
> node = dep->signaler;
> engine = sched_lock_engine(node, engine, &cache);
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
>
> /* Recheck after acquiring the engine->timeline.lock */
> if (prio <= node->attr.priority || node_signaled(node))
> @@ -338,7 +338,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> kick_submission(engine, node_to_request(node), prio);
> }
>
> - spin_unlock(&engine->active.lock);
> + spin_unlock(&engine->sched_engine->lock);
> }
>
> void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
> @@ -511,6 +511,24 @@ i915_sched_engine_create(unsigned int subclass)
> sched_engine->queue = RB_ROOT_CACHED;
> sched_engine->queue_priority_hint = INT_MIN;
>
> + INIT_LIST_HEAD(&sched_engine->requests);
> + INIT_LIST_HEAD(&sched_engine->hold);
> +
> + spin_lock_init(&sched_engine->lock);
> + lockdep_set_subclass(&sched_engine->lock, subclass);
> +
> + /*
> + * Due to an interesting quirk in lockdep's internal debug tracking,
> + * after setting a subclass we must ensure the lock is used. Otherwise,
> + * nr_unused_locks is incremented once too often.
> + */
> +#ifdef CONFIG_DEBUG_LOCK_ALLOC
> + local_irq_disable();
> + lock_map_acquire(&sched_engine->lock.dep_map);
> + lock_map_release(&sched_engine->lock.dep_map);
> + local_irq_enable();
> +#endif
> +
> return sched_engine;
> }
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 16a054e67760..5cb0b1c98d6b 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> @@ -94,6 +94,15 @@ struct i915_dependency {
> struct i915_sched_engine {
> struct kref ref;
>
> + /**
> + * @lock: protects requests in priority lists, requests, hold and
> + * tasklet while running
To be clear, this protects other stuff in this struct, right? Not
just requests and hold?
Otherwise, this looks mechanically correct. Still not getting the big
picture yet.
--Jason
> + */
> + spinlock_t lock;
> +
> + struct list_head requests;
> + struct list_head hold; /* ready requests, but on hold */
> +
> /**
> * @default_priolist: priority list for I915_PRIORITY_NORMAL
> */
> --
> 2.28.0
>
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 4/9] drm/i915: Move active tracking to i915_sched_engine
@ 2021-06-04 19:00 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 19:00 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Move active request tracking and its lock to i915_sched_engine. This
> lock is also the submission lock so having it in the i915_sched_engine
> is the correct place.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_engine.h | 2 -
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 43 +++-----
> drivers/gpu/drm/i915/gt/intel_engine_types.h | 6 --
> .../drm/i915/gt/intel_execlists_submission.c | 98 ++++++++++---------
> .../gpu/drm/i915/gt/intel_ring_submission.c | 12 +--
> drivers/gpu/drm/i915/gt/mock_engine.c | 7 +-
> drivers/gpu/drm/i915/gt/selftest_execlists.c | 4 +-
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 20 ++--
> drivers/gpu/drm/i915/i915_gpu_error.c | 4 +-
> drivers/gpu/drm/i915/i915_request.c | 32 +++---
> drivers/gpu/drm/i915/i915_request.h | 2 +-
> drivers/gpu/drm/i915/i915_scheduler.c | 30 ++++--
> drivers/gpu/drm/i915/i915_scheduler_types.h | 9 ++
> 13 files changed, 134 insertions(+), 135 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
> index 8d9184920c51..a8b2174b4395 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
> @@ -257,8 +257,6 @@ intel_engine_find_active_request(struct intel_engine_cs *engine);
>
> u32 intel_engine_context_size(struct intel_gt *gt, u8 class);
>
> -void intel_engine_init_active(struct intel_engine_cs *engine,
> - unsigned int subclass);
> #define ENGINE_PHYSICAL 0
> #define ENGINE_MOCK 1
> #define ENGINE_VIRTUAL 2
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index 85f2effe9dc6..33d879137908 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -719,7 +719,6 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> if (err)
> goto err_cmd_parser;
>
> - intel_engine_init_active(engine, ENGINE_PHYSICAL);
> intel_engine_init_execlists(engine);
> intel_engine_init__pm(engine);
> intel_engine_init_retire(engine);
> @@ -778,11 +777,11 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
> frame->rq.ring = &frame->ring;
>
> mutex_lock(&ce->timeline->mutex);
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
>
> dw = engine->emit_fini_breadcrumb(&frame->rq, frame->cs) - frame->cs;
>
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> mutex_unlock(&ce->timeline->mutex);
>
> GEM_BUG_ON(dw & 1); /* RING_TAIL must be qword aligned */
> @@ -791,28 +790,6 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
> return dw;
> }
>
> -void
> -intel_engine_init_active(struct intel_engine_cs *engine, unsigned int subclass)
> -{
> - INIT_LIST_HEAD(&engine->active.requests);
> - INIT_LIST_HEAD(&engine->active.hold);
> -
> - spin_lock_init(&engine->active.lock);
> - lockdep_set_subclass(&engine->active.lock, subclass);
> -
> - /*
> - * Due to an interesting quirk in lockdep's internal debug tracking,
> - * after setting a subclass we must ensure the lock is used. Otherwise,
> - * nr_unused_locks is incremented once too often.
> - */
> -#ifdef CONFIG_DEBUG_LOCK_ALLOC
> - local_irq_disable();
> - lock_map_acquire(&engine->active.lock.dep_map);
> - lock_map_release(&engine->active.lock.dep_map);
> - local_irq_enable();
> -#endif
> -}
> -
> static struct intel_context *
> create_pinned_context(struct intel_engine_cs *engine,
> unsigned int hwsp,
> @@ -960,7 +937,7 @@ int intel_engines_init(struct intel_gt *gt)
> */
> void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> {
> - GEM_BUG_ON(!list_empty(&engine->active.requests));
> + GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
> tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
>
> i915_sched_engine_put(engine->sched_engine);
> @@ -1353,7 +1330,7 @@ static struct intel_timeline *get_timeline(struct i915_request *rq)
> struct intel_timeline *tl;
>
> /*
> - * Even though we are holding the engine->active.lock here, there
> + * Even though we are holding the engine->sched_engine->lock here, there
> * is no control over the submission queue per-se and we are
> * inspecting the active state at a random point in time, with an
> * unknown queue. Play safe and make sure the timeline remains valid.
> @@ -1700,7 +1677,7 @@ void intel_engine_dump(struct intel_engine_cs *engine,
>
> drm_printf(m, "\tRequests:\n");
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> rq = intel_engine_find_active_request(engine);
> if (rq) {
> struct intel_timeline *tl = get_timeline(rq);
> @@ -1731,8 +1708,9 @@ void intel_engine_dump(struct intel_engine_cs *engine,
> hexdump(m, rq->context->lrc_reg_state, PAGE_SIZE);
> }
> }
> - drm_printf(m, "\tOn hold?: %lu\n", list_count(&engine->active.hold));
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + drm_printf(m, "\tOn hold?: %lu\n",
> + list_count(&engine->sched_engine->hold));
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
>
> drm_printf(m, "\tMMIO base: 0x%08x\n", engine->mmio_base);
> wakeref = intel_runtime_pm_get_if_in_use(engine->uncore->rpm);
> @@ -1812,7 +1790,7 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
> * At all other times, we must assume the GPU is still running, but
> * we only care about the snapshot of this moment.
> */
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
>
> rcu_read_lock();
> request = execlists_active(&engine->execlists);
> @@ -1830,7 +1808,8 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
> if (active)
> return active;
>
> - list_for_each_entry(request, &engine->active.requests, sched.link) {
> + list_for_each_entry(request, &engine->sched_engine->requests,
> + sched.link) {
> if (__i915_request_is_complete(request))
> continue;
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> index 86b41ddec373..7197b9fa5e35 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> @@ -302,12 +302,6 @@ struct intel_engine_cs {
>
> struct intel_sseu sseu;
>
> - struct {
> - spinlock_t lock;
> - struct list_head requests;
> - struct list_head hold; /* ready requests, but on hold */
> - } active;
> -
> struct i915_sched_engine *sched_engine;
>
> /* keep a request in reserve for a [pm] barrier under oom */
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 609753b5401a..0413ba87cc1e 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -325,7 +325,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> * Check against the first request in ELSP[1], it will, thanks to the
> * power of PI, be the highest priority of that context.
> */
> - if (!list_is_last(&rq->sched.link, &engine->active.requests) &&
> + if (!list_is_last(&rq->sched.link, &engine->sched_engine->requests) &&
> rq_prio(list_next_entry(rq, sched.link)) > last_prio)
> return true;
>
> @@ -367,10 +367,10 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> struct list_head *pl;
> int prio = I915_PRIORITY_INVALID;
>
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
>
> list_for_each_entry_safe_reverse(rq, rn,
> - &engine->active.requests,
> + &engine->sched_engine->requests,
> sched.link) {
> if (__i915_request_is_complete(rq)) {
> list_del_init(&rq->sched.link);
> @@ -534,13 +534,13 @@ resubmit_virtual_request(struct i915_request *rq, struct virtual_engine *ve)
> {
> struct intel_engine_cs *engine = rq->engine;
>
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
>
> clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> WRITE_ONCE(rq->engine, &ve->base);
> ve->base.submit_request(rq);
>
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> }
>
> static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
> @@ -579,7 +579,7 @@ static void __execlists_schedule_out(struct i915_request * const rq,
> unsigned int ccid;
>
> /*
> - * NB process_csb() is not under the engine->active.lock and hence
> + * NB process_csb() is not under the engine->sched_engine->lock and hence
> * schedule_out can race with schedule_in meaning that we should
> * refrain from doing non-trivial work here.
> */
> @@ -1133,7 +1133,8 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> return false;
>
> /* If ELSP[1] is occupied, always check to see if worth slicing */
> - if (!list_is_last_rcu(&rq->sched.link, &engine->active.requests)) {
> + if (!list_is_last_rcu(&rq->sched.link,
> + &engine->sched_engine->requests)) {
> ENGINE_TRACE(engine, "timeslice required for second inflight context\n");
> return true;
> }
> @@ -1266,7 +1267,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> * and context switches) submission.
> */
>
> - spin_lock(&engine->active.lock);
> + spin_lock(&sched_engine->lock);
>
> /*
> * If the queue is higher priority than the last
> @@ -1366,7 +1367,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> * Even if ELSP[1] is occupied and not worthy
> * of timeslices, our queue might be.
> */
> - spin_unlock(&engine->active.lock);
> + spin_unlock(&sched_engine->lock);
> return;
> }
> }
> @@ -1376,7 +1377,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> while ((ve = first_virtual_engine(engine))) {
> struct i915_request *rq;
>
> - spin_lock(&ve->base.active.lock);
> + spin_lock(&ve->base.sched_engine->lock);
>
> rq = ve->request;
> if (unlikely(!virtual_matches(ve, rq, engine)))
> @@ -1386,13 +1387,13 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> GEM_BUG_ON(rq->context != &ve->context);
>
> if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> - spin_unlock(&ve->base.active.lock);
> + spin_unlock(&ve->base.sched_engine->lock);
> break;
> }
>
> if (last && !can_merge_rq(last, rq)) {
> - spin_unlock(&ve->base.active.lock);
> - spin_unlock(&engine->active.lock);
> + spin_unlock(&ve->base.sched_engine->lock);
> + spin_unlock(&engine->sched_engine->lock);
> return; /* leave this for another sibling */
> }
>
> @@ -1438,7 +1439,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
>
> i915_request_put(rq);
> unlock:
> - spin_unlock(&ve->base.active.lock);
> + spin_unlock(&ve->base.sched_engine->lock);
>
> /*
> * Hmm, we have a bunch of virtual engine requests,
> @@ -1554,7 +1555,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> */
> sched_engine->queue_priority_hint = queue_prio(sched_engine);
> i915_sched_engine_reset_on_empty(sched_engine);
> - spin_unlock(&engine->active.lock);
> + spin_unlock(&sched_engine->lock);
>
> /*
> * We can skip poking the HW if we ended up with exactly the same set
> @@ -1981,7 +1982,8 @@ static void __execlists_hold(struct i915_request *rq)
> __i915_request_unsubmit(rq);
>
> clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> - list_move_tail(&rq->sched.link, &rq->engine->active.hold);
> + list_move_tail(&rq->sched.link,
> + &rq->engine->sched_engine->hold);
> i915_request_set_hold(rq);
> RQ_TRACE(rq, "on hold\n");
>
> @@ -2018,7 +2020,7 @@ static bool execlists_hold(struct intel_engine_cs *engine,
> if (i915_request_on_hold(rq))
> return false;
>
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
>
> if (__i915_request_is_complete(rq)) { /* too late! */
> rq = NULL;
> @@ -2034,10 +2036,10 @@ static bool execlists_hold(struct intel_engine_cs *engine,
> GEM_BUG_ON(i915_request_on_hold(rq));
> GEM_BUG_ON(rq->engine != engine);
> __execlists_hold(rq);
> - GEM_BUG_ON(list_empty(&engine->active.hold));
> + GEM_BUG_ON(list_empty(&engine->sched_engine->hold));
>
> unlock:
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> return rq;
> }
>
> @@ -2117,7 +2119,7 @@ static void __execlists_unhold(struct i915_request *rq)
> static void execlists_unhold(struct intel_engine_cs *engine,
> struct i915_request *rq)
> {
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
>
> /*
> * Move this request back to the priority queue, and all of its
> @@ -2130,7 +2132,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> tasklet_hi_schedule(&engine->execlists.tasklet);
> }
>
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> }
>
> struct execlists_capture {
> @@ -2260,13 +2262,13 @@ static void execlists_capture(struct intel_engine_cs *engine)
> if (!cap)
> return;
>
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
> cap->rq = active_context(engine, active_ccid(engine));
> if (cap->rq) {
> cap->rq = active_request(cap->rq->context->timeline, cap->rq);
> cap->rq = i915_request_get_rcu(cap->rq);
> }
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> if (!cap->rq)
> goto err_free;
>
> @@ -2470,7 +2472,7 @@ static bool ancestor_on_hold(const struct intel_engine_cs *engine,
> const struct i915_request *rq)
> {
> GEM_BUG_ON(i915_request_on_hold(rq));
> - return !list_empty(&engine->active.hold) && hold_request(rq);
> + return !list_empty(&engine->sched_engine->hold) && hold_request(rq);
> }
>
> static void execlists_submit_request(struct i915_request *request)
> @@ -2479,11 +2481,12 @@ static void execlists_submit_request(struct i915_request *request)
> unsigned long flags;
>
> /* Will be called from irq-context when using foreign fences. */
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> if (unlikely(ancestor_on_hold(engine, request))) {
> RQ_TRACE(request, "ancestor on hold\n");
> - list_add_tail(&request->sched.link, &engine->active.hold);
> + list_add_tail(&request->sched.link,
> + &engine->sched_engine->hold);
> i915_request_set_hold(request);
> } else {
> queue_request(engine, request);
> @@ -2495,7 +2498,7 @@ static void execlists_submit_request(struct i915_request *request)
> __execlists_kick(&engine->execlists);
> }
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static int
> @@ -2959,9 +2962,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
>
> /* Push back any incomplete requests for replay after the reset. */
> rcu_read_lock();
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> __unwind_incomplete_requests(engine);
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> rcu_read_unlock();
> }
>
> @@ -3001,10 +3004,10 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> execlists_reset_csb(engine, true);
>
> rcu_read_lock();
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> /* Mark all executing requests as skipped. */
> - list_for_each_entry(rq, &engine->active.requests, sched.link)
> + list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
> i915_request_put(i915_request_mark_eio(rq));
> intel_engine_signal_breadcrumbs(engine);
>
> @@ -3024,7 +3027,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> }
>
> /* On-hold requests will be flushed to timeline upon their release */
> - list_for_each_entry(rq, &engine->active.hold, sched.link)
> + list_for_each_entry(rq, &sched_engine->hold, sched.link)
> i915_request_put(i915_request_mark_eio(rq));
>
> /* Cancel all attached virtual engines */
> @@ -3035,7 +3038,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> rb_erase_cached(rb, &execlists->virtual);
> RB_CLEAR_NODE(rb);
>
> - spin_lock(&ve->base.active.lock);
> + spin_lock(&ve->base.sched_engine->lock);
> rq = fetch_and_zero(&ve->request);
> if (rq) {
> if (i915_request_mark_eio(rq)) {
> @@ -3047,7 +3050,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
>
> ve->base.sched_engine->queue_priority_hint = INT_MIN;
> }
> - spin_unlock(&ve->base.active.lock);
> + spin_unlock(&ve->base.sched_engine->lock);
> }
>
> /* Remaining _unready_ requests will be nop'ed when submitted */
> @@ -3058,7 +3061,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> execlists->tasklet.callback = nop_submission_tasklet;
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> rcu_read_unlock();
> }
>
> @@ -3304,7 +3307,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> if (unlikely(ve->request)) {
> struct i915_request *old;
>
> - spin_lock_irq(&ve->base.active.lock);
> + spin_lock_irq(&ve->base.sched_engine->lock);
>
> old = fetch_and_zero(&ve->request);
> if (old) {
> @@ -3313,7 +3316,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> i915_request_put(old);
> }
>
> - spin_unlock_irq(&ve->base.active.lock);
> + spin_unlock_irq(&ve->base.sched_engine->lock);
> }
>
> /*
> @@ -3333,13 +3336,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> if (RB_EMPTY_NODE(node))
> continue;
>
> - spin_lock_irq(&sibling->active.lock);
> + spin_lock_irq(&sibling->sched_engine->lock);
>
> /* Detachment is lazily performed in the execlists tasklet */
> if (!RB_EMPTY_NODE(node))
> rb_erase_cached(node, &sibling->execlists.virtual);
>
> - spin_unlock_irq(&sibling->active.lock);
> + spin_unlock_irq(&sibling->sched_engine->lock);
> }
> GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
> GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> @@ -3509,7 +3512,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> if (!READ_ONCE(ve->request))
> break; /* already handled by a sibling's tasklet */
>
> - spin_lock_irq(&sibling->active.lock);
> + spin_lock_irq(&sibling->sched_engine->lock);
>
> if (unlikely(!(mask & sibling->mask))) {
> if (!RB_EMPTY_NODE(&node->rb)) {
> @@ -3562,7 +3565,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> tasklet_hi_schedule(&sibling->execlists.tasklet);
>
> unlock_engine:
> - spin_unlock_irq(&sibling->active.lock);
> + spin_unlock_irq(&sibling->sched_engine->lock);
>
> if (intel_context_inflight(&ve->context))
> break;
> @@ -3580,7 +3583,7 @@ static void virtual_submit_request(struct i915_request *rq)
>
> GEM_BUG_ON(ve->base.submit_request != virtual_submit_request);
>
> - spin_lock_irqsave(&ve->base.active.lock, flags);
> + spin_lock_irqsave(&ve->base.sched_engine->lock, flags);
>
> /* By the time we resubmit a request, it may be completed */
> if (__i915_request_is_complete(rq)) {
> @@ -3603,7 +3606,7 @@ static void virtual_submit_request(struct i915_request *rq)
> tasklet_hi_schedule(&ve->base.execlists.tasklet);
>
> unlock:
> - spin_unlock_irqrestore(&ve->base.active.lock, flags);
> + spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
> }
>
> static struct ve_bond *
> @@ -3687,7 +3690,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
>
> snprintf(ve->base.name, sizeof(ve->base.name), "virtual");
>
> - intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> intel_engine_init_execlists(&ve->base);
>
> ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> @@ -3860,17 +3862,17 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> unsigned int max)
> {
> const struct intel_engine_execlists *execlists = &engine->execlists;
> - const struct i915_sched_engine *sched_engine = engine->sched_engine;
> + struct i915_sched_engine *sched_engine = engine->sched_engine;
> struct i915_request *rq, *last;
> unsigned long flags;
> unsigned int count;
> struct rb_node *rb;
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&sched_engine->lock, flags);
>
> last = NULL;
> count = 0;
> - list_for_each_entry(rq, &engine->active.requests, sched.link) {
> + list_for_each_entry(rq, &sched_engine->requests, sched.link) {
> if (count++ < max - 1)
> show_request(m, rq, "\t\t", 0);
> else
> @@ -3933,7 +3935,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> show_request(m, last, "\t\t", 0);
> }
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&sched_engine->lock, flags);
> }
>
> #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
> diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> index 2b6dffcc2262..c98ae0f58790 100644
> --- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> @@ -339,9 +339,9 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
> u32 head;
>
> rq = NULL;
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> rcu_read_lock();
> - list_for_each_entry(pos, &engine->active.requests, sched.link) {
> + list_for_each_entry(pos, &engine->sched_engine->requests, sched.link) {
> if (!__i915_request_is_complete(pos)) {
> rq = pos;
> break;
> @@ -396,7 +396,7 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
> }
> engine->legacy.ring->head = intel_ring_wrap(engine->legacy.ring, head);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void reset_finish(struct intel_engine_cs *engine)
> @@ -408,16 +408,16 @@ static void reset_cancel(struct intel_engine_cs *engine)
> struct i915_request *request;
> unsigned long flags;
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> /* Mark all submitted requests as skipped. */
> - list_for_each_entry(request, &engine->active.requests, sched.link)
> + list_for_each_entry(request, &engine->sched_engine->requests, sched.link)
> i915_request_put(i915_request_mark_eio(request));
> intel_engine_signal_breadcrumbs(engine);
>
> /* Remaining _unready_ requests will be nop'ed when submitted */
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void i9xx_submit_request(struct i915_request *request)
> diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> index b1fdba13e900..a49fd3039f13 100644
> --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> @@ -253,10 +253,10 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
>
> del_timer_sync(&mock->hw_delay);
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> /* Mark all submitted requests as skipped. */
> - list_for_each_entry(rq, &engine->active.requests, sched.link)
> + list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
> i915_request_put(i915_request_mark_eio(rq));
> intel_engine_signal_breadcrumbs(engine);
>
> @@ -269,7 +269,7 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
> }
> INIT_LIST_HEAD(&mock->hw_queue);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void mock_reset_finish(struct intel_engine_cs *engine)
> @@ -350,7 +350,6 @@ int mock_engine_init(struct intel_engine_cs *engine)
> if (!engine->sched_engine)
> return -ENOMEM;
>
> - intel_engine_init_active(engine, ENGINE_MOCK);
> intel_engine_init_execlists(engine);
> intel_engine_init__pm(engine);
> intel_engine_init_retire(engine);
> diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> index 1f93591a8c69..fd824fa329c2 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> @@ -4599,9 +4599,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
> GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
>
> /* Fake a preemption event; failed of course */
> - spin_lock_irq(&engine->active.lock);
> + spin_lock_irq(&engine->sched_engine->lock);
> __unwind_incomplete_requests(engine);
> - spin_unlock_irq(&engine->active.lock);
> + spin_unlock_irq(&engine->sched_engine->lock);
> GEM_BUG_ON(rq->engine != engine);
>
> /* Reset the engine while keeping our active request on hold */
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index f4a6fbfaf82e..2551019ee217 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -190,7 +190,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> bool submit = false;
> struct rb_node *rb;
>
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&sched_engine->lock);
>
> if (last) {
> if (*++first)
> @@ -247,7 +247,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
> struct i915_request **port, *rq;
> unsigned long flags;
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> for (port = execlists->inflight; (rq = *port); port++) {
> if (!i915_request_completed(rq))
> @@ -265,7 +265,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
>
> i915_sched_engine_reset_on_empty(engine->sched_engine);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
> @@ -322,7 +322,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> struct i915_request *rq;
> unsigned long flags;
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> /* Push back any incomplete requests for replay after the reset. */
> rq = execlists_unwind_incomplete_requests(execlists);
> @@ -336,7 +336,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> guc_reset_state(rq->context, engine, rq->head, stalled);
>
> out_unlock:
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void guc_reset_cancel(struct intel_engine_cs *engine)
> @@ -362,10 +362,10 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> * submission's irq state, we also wish to remind ourselves that
> * it is irq state.)
> */
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&sched_engine->lock, flags);
>
> /* Mark all executing requests as skipped. */
> - list_for_each_entry(rq, &engine->active.requests, sched.link) {
> + list_for_each_entry(rq, &sched_engine->requests, sched.link) {
> i915_request_set_error_once(rq, -EIO);
> i915_request_mark_complete(rq);
> }
> @@ -390,7 +390,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> sched_engine->queue_priority_hint = INT_MIN;
> sched_engine->queue = RB_ROOT_CACHED;
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&sched_engine->lock, flags);
> }
>
> static void guc_reset_finish(struct intel_engine_cs *engine)
> @@ -539,7 +539,7 @@ static void guc_submit_request(struct i915_request *rq)
> unsigned long flags;
>
> /* Will be called from irq-context when using foreign fences. */
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> queue_request(engine, rq, rq_prio(rq));
>
> @@ -548,7 +548,7 @@ static void guc_submit_request(struct i915_request *rq)
>
> tasklet_hi_schedule(&engine->execlists.tasklet);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void sanitize_hwsp(struct intel_engine_cs *engine)
> diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
> index 833d3e8b7631..cd005f0a203c 100644
> --- a/drivers/gpu/drm/i915/i915_gpu_error.c
> +++ b/drivers/gpu/drm/i915/i915_gpu_error.c
> @@ -1436,12 +1436,12 @@ capture_engine(struct intel_engine_cs *engine,
> if (!ee)
> return NULL;
>
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> rq = intel_engine_find_active_request(engine);
> if (rq)
> capture = intel_engine_coredump_add_request(ee, rq,
> ATOMIC_MAYFAIL);
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> if (!capture) {
> kfree(ee);
> return NULL;
> diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
> index 970d8f4986bb..95e859c319ca 100644
> --- a/drivers/gpu/drm/i915/i915_request.c
> +++ b/drivers/gpu/drm/i915/i915_request.c
> @@ -272,11 +272,11 @@ i915_request_active_engine(struct i915_request *rq,
> * check that we have acquired the lock on the final engine.
> */
> locked = READ_ONCE(rq->engine);
> - spin_lock_irq(&locked->active.lock);
> + spin_lock_irq(&locked->sched_engine->lock);
> while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
> - spin_unlock(&locked->active.lock);
> + spin_unlock(&locked->sched_engine->lock);
> locked = engine;
> - spin_lock(&locked->active.lock);
> + spin_lock(&locked->sched_engine->lock);
> }
>
> if (i915_request_is_active(rq)) {
> @@ -285,7 +285,7 @@ i915_request_active_engine(struct i915_request *rq,
> ret = true;
> }
>
> - spin_unlock_irq(&locked->active.lock);
> + spin_unlock_irq(&locked->sched_engine->lock);
>
> return ret;
> }
> @@ -302,10 +302,10 @@ static void remove_from_engine(struct i915_request *rq)
> * check that the rq still belongs to the newly locked engine.
> */
> locked = READ_ONCE(rq->engine);
> - spin_lock_irq(&locked->active.lock);
> + spin_lock_irq(&locked->sched_engine->lock);
> while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
> - spin_unlock(&locked->active.lock);
> - spin_lock(&engine->active.lock);
> + spin_unlock(&locked->sched_engine->lock);
> + spin_lock(&engine->sched_engine->lock);
> locked = engine;
> }
> list_del_init(&rq->sched.link);
> @@ -316,7 +316,7 @@ static void remove_from_engine(struct i915_request *rq)
> /* Prevent further __await_execution() registering a cb, then flush */
> set_bit(I915_FENCE_FLAG_ACTIVE, &rq->fence.flags);
>
> - spin_unlock_irq(&locked->active.lock);
> + spin_unlock_irq(&locked->sched_engine->lock);
>
> __notify_execute_cb_imm(rq);
> }
> @@ -637,7 +637,7 @@ bool __i915_request_submit(struct i915_request *request)
> RQ_TRACE(request, "\n");
>
> GEM_BUG_ON(!irqs_disabled());
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
>
> /*
> * With the advent of preempt-to-busy, we frequently encounter
> @@ -649,7 +649,7 @@ bool __i915_request_submit(struct i915_request *request)
> *
> * We must remove the request from the caller's priority queue,
> * and the caller must only call us when the request is in their
> - * priority queue, under the active.lock. This ensures that the
> + * priority queue, under the sched_engine->lock. This ensures that the
> * request has *not* yet been retired and we can safely move
> * the request into the engine->active.list where it will be
> * dropped upon retiring. (Otherwise if resubmit a *retired*
> @@ -694,7 +694,7 @@ bool __i915_request_submit(struct i915_request *request)
> result = true;
>
> GEM_BUG_ON(test_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags));
> - list_move_tail(&request->sched.link, &engine->active.requests);
> + list_move_tail(&request->sched.link, &engine->sched_engine->requests);
> active:
> clear_bit(I915_FENCE_FLAG_PQUEUE, &request->fence.flags);
> set_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags);
> @@ -724,11 +724,11 @@ void i915_request_submit(struct i915_request *request)
> unsigned long flags;
>
> /* Will be called from irq-context when using foreign fences. */
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> __i915_request_submit(request);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> void __i915_request_unsubmit(struct i915_request *request)
> @@ -742,7 +742,7 @@ void __i915_request_unsubmit(struct i915_request *request)
> RQ_TRACE(request, "\n");
>
> GEM_BUG_ON(!irqs_disabled());
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
>
> /*
> * Before we remove this breadcrumb from the signal list, we have
> @@ -775,11 +775,11 @@ void i915_request_unsubmit(struct i915_request *request)
> unsigned long flags;
>
> /* Will be called from irq-context when using foreign fences. */
> - spin_lock_irqsave(&engine->active.lock, flags);
> + spin_lock_irqsave(&engine->sched_engine->lock, flags);
>
> __i915_request_unsubmit(request);
>
> - spin_unlock_irqrestore(&engine->active.lock, flags);
> + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
>
> static void __cancel_request(struct i915_request *rq)
> diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
> index 270f6cd37650..239964bec1fa 100644
> --- a/drivers/gpu/drm/i915/i915_request.h
> +++ b/drivers/gpu/drm/i915/i915_request.h
> @@ -613,7 +613,7 @@ i915_request_active_timeline(const struct i915_request *rq)
> * this submission.
> */
> return rcu_dereference_protected(rq->timeline,
> - lockdep_is_held(&rq->engine->active.lock));
> + lockdep_is_held(&rq->engine->sched_engine->lock));
> }
>
> static inline u32
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 4953874a9ba6..4bc6969f6a97 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -68,7 +68,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> struct rb_node **parent, *rb;
> bool first = true;
>
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
> assert_priolists(sched_engine);
>
> if (unlikely(sched_engine->no_priolist))
> @@ -147,9 +147,9 @@ sched_lock_engine(const struct i915_sched_node *node,
> * check that the rq still belongs to the newly locked engine.
> */
> while (locked != (engine = READ_ONCE(rq->engine))) {
> - spin_unlock(&locked->active.lock);
> + spin_unlock(&locked->sched_engine->lock);
> memset(cache, 0, sizeof(*cache));
> - spin_lock(&engine->active.lock);
> + spin_lock(&engine->sched_engine->lock);
> locked = engine;
> }
>
> @@ -296,7 +296,7 @@ static void __i915_schedule(struct i915_sched_node *node,
>
> memset(&cache, 0, sizeof(cache));
> engine = node_to_request(node)->engine;
> - spin_lock(&engine->active.lock);
> + spin_lock(&engine->sched_engine->lock);
>
> /* Fifo and depth-first replacement ensure our deps execute before us */
> engine = sched_lock_engine(node, engine, &cache);
> @@ -305,7 +305,7 @@ static void __i915_schedule(struct i915_sched_node *node,
>
> node = dep->signaler;
> engine = sched_lock_engine(node, engine, &cache);
> - lockdep_assert_held(&engine->active.lock);
> + lockdep_assert_held(&engine->sched_engine->lock);
>
> /* Recheck after acquiring the engine->timeline.lock */
> if (prio <= node->attr.priority || node_signaled(node))
> @@ -338,7 +338,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> kick_submission(engine, node_to_request(node), prio);
> }
>
> - spin_unlock(&engine->active.lock);
> + spin_unlock(&engine->sched_engine->lock);
> }
>
> void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
> @@ -511,6 +511,24 @@ i915_sched_engine_create(unsigned int subclass)
> sched_engine->queue = RB_ROOT_CACHED;
> sched_engine->queue_priority_hint = INT_MIN;
>
> + INIT_LIST_HEAD(&sched_engine->requests);
> + INIT_LIST_HEAD(&sched_engine->hold);
> +
> + spin_lock_init(&sched_engine->lock);
> + lockdep_set_subclass(&sched_engine->lock, subclass);
> +
> + /*
> + * Due to an interesting quirk in lockdep's internal debug tracking,
> + * after setting a subclass we must ensure the lock is used. Otherwise,
> + * nr_unused_locks is incremented once too often.
> + */
> +#ifdef CONFIG_DEBUG_LOCK_ALLOC
> + local_irq_disable();
> + lock_map_acquire(&sched_engine->lock.dep_map);
> + lock_map_release(&sched_engine->lock.dep_map);
> + local_irq_enable();
> +#endif
> +
> return sched_engine;
> }
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 16a054e67760..5cb0b1c98d6b 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> @@ -94,6 +94,15 @@ struct i915_dependency {
> struct i915_sched_engine {
> struct kref ref;
>
> + /**
> + * @lock: protects requests in priority lists, requests, hold and
> + * tasklet while running
To be clear, this protects other stuff in this struct, right? Not
just requests and hold?
Otherwise, this looks mechanically correct. Still not getting the big
picture yet.
--Jason
> + */
> + spinlock_t lock;
> +
> + struct list_head requests;
> + struct list_head hold; /* ready requests, but on hold */
> +
> /**
> * @default_priolist: priority list for I915_PRIORITY_NORMAL
> */
> --
> 2.28.0
>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 5/9] drm/i915: Move engine->schedule to i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 19:03 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 19:03 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> The schedule function should be in the schedule object.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gem/i915_gem_wait.c | 4 ++--
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 3 ---
> drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++--
> drivers/gpu/drm/i915/gt/intel_engine_types.h | 8 --------
> drivers/gpu/drm/i915/gt/intel_engine_user.c | 2 +-
> .../gpu/drm/i915/gt/intel_execlists_submission.c | 4 ++--
> drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++++++++--------
> drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 4 ++--
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
> drivers/gpu/drm/i915/i915_request.c | 10 +++++-----
> drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++++++++
> 11 files changed, 31 insertions(+), 34 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_wait.c b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> index 4b9856d5ba14..af1fbf8e2a9a 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> @@ -104,8 +104,8 @@ static void fence_set_priority(struct dma_fence *fence,
> engine = rq->engine;
>
> rcu_read_lock(); /* RCU serialisation for set-wedged protection */
> - if (engine->schedule)
> - engine->schedule(rq, attr);
> + if (engine->sched_engine->schedule)
> + engine->sched_engine->schedule(rq, attr);
> rcu_read_unlock();
> }
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index 33d879137908..b480fcb1aad1 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -326,9 +326,6 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
> if (engine->context_size)
> DRIVER_CAPS(i915)->has_logical_contexts = true;
>
> - /* Nothing to do here, execute in order of dependencies */
> - engine->schedule = NULL;
> -
> ewma__engine_latency_init(&engine->latency);
> seqcount_init(&engine->stats.lock);
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> index b99ac41695f3..b6a305e6a974 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> @@ -121,7 +121,7 @@ static void heartbeat(struct work_struct *wrk)
> * but all other contexts, including the kernel
> * context are stuck waiting for the signal.
> */
> - } else if (engine->schedule &&
> + } else if (engine->sched_engine->schedule &&
> rq->sched.attr.priority < I915_PRIORITY_BARRIER) {
> /*
> * Gradually raise the priority of the heartbeat to
> @@ -136,7 +136,7 @@ static void heartbeat(struct work_struct *wrk)
> attr.priority = I915_PRIORITY_BARRIER;
>
> local_bh_disable();
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
> local_bh_enable();
> } else {
> if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> index 7197b9fa5e35..f1b14aff5118 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> @@ -426,14 +426,6 @@ struct intel_engine_cs {
> void (*bond_execute)(struct i915_request *rq,
> struct dma_fence *signal);
>
> - /*
> - * Call when the priority on a request has changed and it and its
> - * dependencies may need rescheduling. Note the request itself may
> - * not be ready to run!
> - */
> - void (*schedule)(struct i915_request *request,
> - const struct i915_sched_attr *attr);
> -
> void (*release)(struct intel_engine_cs *engine);
>
> struct intel_engine_execlists execlists;
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c
> index 3cca7ea2d6ea..84142127ebd8 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_user.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c
> @@ -108,7 +108,7 @@ static void set_scheduler_caps(struct drm_i915_private *i915)
> for_each_uabi_engine(engine, i915) { /* all engines must agree! */
> int i;
>
> - if (engine->schedule)
> + if (engine->sched_engine->schedule)
> enabled |= (I915_SCHEDULER_CAP_ENABLED |
> I915_SCHEDULER_CAP_PRIORITY);
> else
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 0413ba87cc1e..23fd03815ad0 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -3119,7 +3119,7 @@ static bool can_preempt(struct intel_engine_cs *engine)
> static void execlists_set_default_submission(struct intel_engine_cs *engine)
> {
> engine->submit_request = execlists_submit_request;
> - engine->schedule = i915_schedule;
> + engine->sched_engine->schedule = i915_schedule;
Dumb question: Why is this a function pointer? It's always set to
i915_schedule.
--Jason
> engine->execlists.tasklet.callback = execlists_submission_tasklet;
> }
>
> @@ -3701,7 +3701,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> ve->base.cops = &virtual_context_ops;
> ve->base.request_alloc = execlists_request_alloc;
>
> - ve->base.schedule = i915_schedule;
> + ve->base.sched_engine->schedule = i915_schedule;
> ve->base.submit_request = virtual_submit_request;
> ve->base.bond_execute = virtual_bond_execute;
>
> diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> index fd824fa329c2..5cc7648d1e5a 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> @@ -273,7 +273,7 @@ static int live_unlite_restore(struct intel_gt *gt, int prio)
> };
>
> /* Alternatively preempt the spinner with ce[1] */
> - engine->schedule(rq[1], &attr);
> + engine->sched_engine->schedule(rq[1], &attr);
> }
>
> /* And switch back to ce[0] for good measure */
> @@ -900,7 +900,7 @@ release_queue(struct intel_engine_cs *engine,
> i915_request_add(rq);
>
> local_bh_disable();
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
> local_bh_enable(); /* kick tasklet */
>
> i915_request_put(rq);
> @@ -1325,7 +1325,7 @@ static int live_timeslice_queue(void *arg)
> err = PTR_ERR(rq);
> goto err_heartbeat;
> }
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
> err = wait_for_submit(engine, rq, HZ / 2);
> if (err) {
> pr_err("%s: Timed out trying to submit semaphores\n",
> @@ -1867,7 +1867,7 @@ static int live_late_preempt(void *arg)
> }
>
> attr.priority = I915_PRIORITY_MAX;
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
>
> if (!igt_wait_for_spinner(&spin_hi, rq)) {
> pr_err("High priority context failed to preempt the low priority context\n");
> @@ -2480,7 +2480,7 @@ static int live_suppress_self_preempt(void *arg)
> i915_request_add(rq_b);
>
> GEM_BUG_ON(i915_request_completed(rq_a));
> - engine->schedule(rq_a, &attr);
> + engine->sched_engine->schedule(rq_a, &attr);
> igt_spinner_end(&a.spin);
>
> if (!igt_wait_for_spinner(&b.spin, rq_b)) {
> @@ -2612,7 +2612,7 @@ static int live_chain_preempt(void *arg)
>
> i915_request_get(rq);
> i915_request_add(rq);
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
>
> igt_spinner_end(&hi.spin);
> if (i915_request_wait(rq, 0, HZ / 5) < 0) {
> @@ -2971,7 +2971,7 @@ static int live_preempt_gang(void *arg)
> break;
>
> /* Submit each spinner at increasing priority */
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
> } while (prio <= I915_PRIORITY_MAX &&
> !__igt_timeout(end_time, NULL));
> pr_debug("%s: Preempt chain of %d requests\n",
> @@ -3219,7 +3219,7 @@ static int preempt_user(struct intel_engine_cs *engine,
> i915_request_get(rq);
> i915_request_add(rq);
>
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
>
> if (i915_request_wait(rq, 0, HZ / 2) < 0)
> err = -ETIME;
> diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> index 5b63d4df8c93..e57dc900ae8d 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> @@ -858,12 +858,12 @@ static int active_engine(void *data)
> rq[idx] = i915_request_get(new);
> i915_request_add(new);
>
> - if (engine->schedule && arg->flags & TEST_PRIORITY) {
> + if (engine->sched_engine->schedule && arg->flags & TEST_PRIORITY) {
> struct i915_sched_attr attr = {
> .priority =
> i915_prandom_u32_max_state(512, &prng),
> };
> - engine->schedule(rq[idx], &attr);
> + engine->sched_engine->schedule(rq[idx], &attr);
> }
>
> err = active_request_put(old);
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 2551019ee217..4c5bbec0775d 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -641,7 +641,7 @@ static void guc_default_vfuncs(struct intel_engine_cs *engine)
> engine->cops = &guc_context_ops;
> engine->request_alloc = guc_request_alloc;
>
> - engine->schedule = i915_schedule;
> + engine->sched_engine->schedule = i915_schedule;
>
> engine->reset.prepare = guc_reset_prepare;
> engine->reset.rewind = guc_reset_rewind;
> diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
> index 95e859c319ca..f0cb8820fb97 100644
> --- a/drivers/gpu/drm/i915/i915_request.c
> +++ b/drivers/gpu/drm/i915/i915_request.c
> @@ -1343,7 +1343,7 @@ __i915_request_await_execution(struct i915_request *to,
> }
>
> /* Couple the dependency tree for PI on this exposed to->fence */
> - if (to->engine->schedule) {
> + if (to->engine->sched_engine->schedule) {
> err = i915_sched_node_add_dependency(&to->sched,
> &from->sched,
> I915_DEPENDENCY_WEAK);
> @@ -1484,7 +1484,7 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from)
> return 0;
> }
>
> - if (to->engine->schedule) {
> + if (to->engine->sched_engine->schedule) {
> ret = i915_sched_node_add_dependency(&to->sched,
> &from->sched,
> I915_DEPENDENCY_EXTERNAL);
> @@ -1671,7 +1671,7 @@ __i915_request_add_to_timeline(struct i915_request *rq)
> __i915_sw_fence_await_dma_fence(&rq->submit,
> &prev->fence,
> &rq->dmaq);
> - if (rq->engine->schedule)
> + if (rq->engine->sched_engine->schedule)
> __i915_sched_node_add_dependency(&rq->sched,
> &prev->sched,
> &rq->dep,
> @@ -1743,8 +1743,8 @@ void __i915_request_queue(struct i915_request *rq,
> * decide whether to preempt the entire chain so that it is ready to
> * run at the earliest possible convenience.
> */
> - if (attr && rq->engine->schedule)
> - rq->engine->schedule(rq, attr);
> + if (attr && rq->engine->sched_engine->schedule)
> + rq->engine->sched_engine->schedule(rq, attr);
>
> local_bh_disable();
> __i915_request_queue_bh(rq);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 5cb0b1c98d6b..3f462f8b06f2 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> @@ -131,6 +131,14 @@ struct i915_sched_engine {
> * @no_priolist: priority lists disabled
> */
> bool no_priolist;
> +
> + /*
> + * Call when the priority on a request has changed and it and its
> + * dependencies may need rescheduling. Note the request itself may
> + * not be ready to run!
> + */
> + void (*schedule)(struct i915_request *request,
> + const struct i915_sched_attr *attr);
> };
>
> #endif /* _I915_SCHEDULER_TYPES_H_ */
> --
> 2.28.0
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 5/9] drm/i915: Move engine->schedule to i915_sched_engine
@ 2021-06-04 19:03 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 19:03 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> The schedule function should be in the schedule object.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gem/i915_gem_wait.c | 4 ++--
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 3 ---
> drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++--
> drivers/gpu/drm/i915/gt/intel_engine_types.h | 8 --------
> drivers/gpu/drm/i915/gt/intel_engine_user.c | 2 +-
> .../gpu/drm/i915/gt/intel_execlists_submission.c | 4 ++--
> drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++++++++--------
> drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 4 ++--
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
> drivers/gpu/drm/i915/i915_request.c | 10 +++++-----
> drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++++++++
> 11 files changed, 31 insertions(+), 34 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_wait.c b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> index 4b9856d5ba14..af1fbf8e2a9a 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> @@ -104,8 +104,8 @@ static void fence_set_priority(struct dma_fence *fence,
> engine = rq->engine;
>
> rcu_read_lock(); /* RCU serialisation for set-wedged protection */
> - if (engine->schedule)
> - engine->schedule(rq, attr);
> + if (engine->sched_engine->schedule)
> + engine->sched_engine->schedule(rq, attr);
> rcu_read_unlock();
> }
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index 33d879137908..b480fcb1aad1 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -326,9 +326,6 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
> if (engine->context_size)
> DRIVER_CAPS(i915)->has_logical_contexts = true;
>
> - /* Nothing to do here, execute in order of dependencies */
> - engine->schedule = NULL;
> -
> ewma__engine_latency_init(&engine->latency);
> seqcount_init(&engine->stats.lock);
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> index b99ac41695f3..b6a305e6a974 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> @@ -121,7 +121,7 @@ static void heartbeat(struct work_struct *wrk)
> * but all other contexts, including the kernel
> * context are stuck waiting for the signal.
> */
> - } else if (engine->schedule &&
> + } else if (engine->sched_engine->schedule &&
> rq->sched.attr.priority < I915_PRIORITY_BARRIER) {
> /*
> * Gradually raise the priority of the heartbeat to
> @@ -136,7 +136,7 @@ static void heartbeat(struct work_struct *wrk)
> attr.priority = I915_PRIORITY_BARRIER;
>
> local_bh_disable();
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
> local_bh_enable();
> } else {
> if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> index 7197b9fa5e35..f1b14aff5118 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> @@ -426,14 +426,6 @@ struct intel_engine_cs {
> void (*bond_execute)(struct i915_request *rq,
> struct dma_fence *signal);
>
> - /*
> - * Call when the priority on a request has changed and it and its
> - * dependencies may need rescheduling. Note the request itself may
> - * not be ready to run!
> - */
> - void (*schedule)(struct i915_request *request,
> - const struct i915_sched_attr *attr);
> -
> void (*release)(struct intel_engine_cs *engine);
>
> struct intel_engine_execlists execlists;
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c
> index 3cca7ea2d6ea..84142127ebd8 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_user.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c
> @@ -108,7 +108,7 @@ static void set_scheduler_caps(struct drm_i915_private *i915)
> for_each_uabi_engine(engine, i915) { /* all engines must agree! */
> int i;
>
> - if (engine->schedule)
> + if (engine->sched_engine->schedule)
> enabled |= (I915_SCHEDULER_CAP_ENABLED |
> I915_SCHEDULER_CAP_PRIORITY);
> else
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 0413ba87cc1e..23fd03815ad0 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -3119,7 +3119,7 @@ static bool can_preempt(struct intel_engine_cs *engine)
> static void execlists_set_default_submission(struct intel_engine_cs *engine)
> {
> engine->submit_request = execlists_submit_request;
> - engine->schedule = i915_schedule;
> + engine->sched_engine->schedule = i915_schedule;
Dumb question: Why is this a function pointer? It's always set to
i915_schedule.
--Jason
> engine->execlists.tasklet.callback = execlists_submission_tasklet;
> }
>
> @@ -3701,7 +3701,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> ve->base.cops = &virtual_context_ops;
> ve->base.request_alloc = execlists_request_alloc;
>
> - ve->base.schedule = i915_schedule;
> + ve->base.sched_engine->schedule = i915_schedule;
> ve->base.submit_request = virtual_submit_request;
> ve->base.bond_execute = virtual_bond_execute;
>
> diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> index fd824fa329c2..5cc7648d1e5a 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> @@ -273,7 +273,7 @@ static int live_unlite_restore(struct intel_gt *gt, int prio)
> };
>
> /* Alternatively preempt the spinner with ce[1] */
> - engine->schedule(rq[1], &attr);
> + engine->sched_engine->schedule(rq[1], &attr);
> }
>
> /* And switch back to ce[0] for good measure */
> @@ -900,7 +900,7 @@ release_queue(struct intel_engine_cs *engine,
> i915_request_add(rq);
>
> local_bh_disable();
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
> local_bh_enable(); /* kick tasklet */
>
> i915_request_put(rq);
> @@ -1325,7 +1325,7 @@ static int live_timeslice_queue(void *arg)
> err = PTR_ERR(rq);
> goto err_heartbeat;
> }
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
> err = wait_for_submit(engine, rq, HZ / 2);
> if (err) {
> pr_err("%s: Timed out trying to submit semaphores\n",
> @@ -1867,7 +1867,7 @@ static int live_late_preempt(void *arg)
> }
>
> attr.priority = I915_PRIORITY_MAX;
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
>
> if (!igt_wait_for_spinner(&spin_hi, rq)) {
> pr_err("High priority context failed to preempt the low priority context\n");
> @@ -2480,7 +2480,7 @@ static int live_suppress_self_preempt(void *arg)
> i915_request_add(rq_b);
>
> GEM_BUG_ON(i915_request_completed(rq_a));
> - engine->schedule(rq_a, &attr);
> + engine->sched_engine->schedule(rq_a, &attr);
> igt_spinner_end(&a.spin);
>
> if (!igt_wait_for_spinner(&b.spin, rq_b)) {
> @@ -2612,7 +2612,7 @@ static int live_chain_preempt(void *arg)
>
> i915_request_get(rq);
> i915_request_add(rq);
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
>
> igt_spinner_end(&hi.spin);
> if (i915_request_wait(rq, 0, HZ / 5) < 0) {
> @@ -2971,7 +2971,7 @@ static int live_preempt_gang(void *arg)
> break;
>
> /* Submit each spinner at increasing priority */
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
> } while (prio <= I915_PRIORITY_MAX &&
> !__igt_timeout(end_time, NULL));
> pr_debug("%s: Preempt chain of %d requests\n",
> @@ -3219,7 +3219,7 @@ static int preempt_user(struct intel_engine_cs *engine,
> i915_request_get(rq);
> i915_request_add(rq);
>
> - engine->schedule(rq, &attr);
> + engine->sched_engine->schedule(rq, &attr);
>
> if (i915_request_wait(rq, 0, HZ / 2) < 0)
> err = -ETIME;
> diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> index 5b63d4df8c93..e57dc900ae8d 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> @@ -858,12 +858,12 @@ static int active_engine(void *data)
> rq[idx] = i915_request_get(new);
> i915_request_add(new);
>
> - if (engine->schedule && arg->flags & TEST_PRIORITY) {
> + if (engine->sched_engine->schedule && arg->flags & TEST_PRIORITY) {
> struct i915_sched_attr attr = {
> .priority =
> i915_prandom_u32_max_state(512, &prng),
> };
> - engine->schedule(rq[idx], &attr);
> + engine->sched_engine->schedule(rq[idx], &attr);
> }
>
> err = active_request_put(old);
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 2551019ee217..4c5bbec0775d 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -641,7 +641,7 @@ static void guc_default_vfuncs(struct intel_engine_cs *engine)
> engine->cops = &guc_context_ops;
> engine->request_alloc = guc_request_alloc;
>
> - engine->schedule = i915_schedule;
> + engine->sched_engine->schedule = i915_schedule;
>
> engine->reset.prepare = guc_reset_prepare;
> engine->reset.rewind = guc_reset_rewind;
> diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
> index 95e859c319ca..f0cb8820fb97 100644
> --- a/drivers/gpu/drm/i915/i915_request.c
> +++ b/drivers/gpu/drm/i915/i915_request.c
> @@ -1343,7 +1343,7 @@ __i915_request_await_execution(struct i915_request *to,
> }
>
> /* Couple the dependency tree for PI on this exposed to->fence */
> - if (to->engine->schedule) {
> + if (to->engine->sched_engine->schedule) {
> err = i915_sched_node_add_dependency(&to->sched,
> &from->sched,
> I915_DEPENDENCY_WEAK);
> @@ -1484,7 +1484,7 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from)
> return 0;
> }
>
> - if (to->engine->schedule) {
> + if (to->engine->sched_engine->schedule) {
> ret = i915_sched_node_add_dependency(&to->sched,
> &from->sched,
> I915_DEPENDENCY_EXTERNAL);
> @@ -1671,7 +1671,7 @@ __i915_request_add_to_timeline(struct i915_request *rq)
> __i915_sw_fence_await_dma_fence(&rq->submit,
> &prev->fence,
> &rq->dmaq);
> - if (rq->engine->schedule)
> + if (rq->engine->sched_engine->schedule)
> __i915_sched_node_add_dependency(&rq->sched,
> &prev->sched,
> &rq->dep,
> @@ -1743,8 +1743,8 @@ void __i915_request_queue(struct i915_request *rq,
> * decide whether to preempt the entire chain so that it is ready to
> * run at the earliest possible convenience.
> */
> - if (attr && rq->engine->schedule)
> - rq->engine->schedule(rq, attr);
> + if (attr && rq->engine->sched_engine->schedule)
> + rq->engine->sched_engine->schedule(rq, attr);
>
> local_bh_disable();
> __i915_request_queue_bh(rq);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 5cb0b1c98d6b..3f462f8b06f2 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> @@ -131,6 +131,14 @@ struct i915_sched_engine {
> * @no_priolist: priority lists disabled
> */
> bool no_priolist;
> +
> + /*
> + * Call when the priority on a request has changed and it and its
> + * dependencies may need rescheduling. Note the request itself may
> + * not be ready to run!
> + */
> + void (*schedule)(struct i915_request *request,
> + const struct i915_sched_attr *attr);
> };
>
> #endif /* _I915_SCHEDULER_TYPES_H_ */
> --
> 2.28.0
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 6/9] drm/i915: Add kick_backend function to i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 19:09 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 19:09 UTC (permalink / raw)
To: Matthew Brost
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Rather than touching execlist specific structures in the generic
> scheduling code, add a callback function in the backend.
Seems reasonable but why does the function that's there today do
nothing for the ringbuffer and current GuC back-ends? I'm sure
there's some reason but it's not immediately obvious to me.
--Jason
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> .../drm/i915/gt/intel_execlists_submission.c | 52 ++++++++++++++++
> drivers/gpu/drm/i915/i915_scheduler.c | 62 +------------------
> drivers/gpu/drm/i915/i915_scheduler_types.h | 4 ++
> 3 files changed, 58 insertions(+), 60 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 23fd03815ad0..3fac17103837 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -3116,10 +3116,61 @@ static bool can_preempt(struct intel_engine_cs *engine)
> return engine->class != RENDER_CLASS;
> }
>
> +static void kick_execlists(const struct i915_request *rq, int prio)
> +{
> + struct intel_engine_cs *engine = rq->engine;
> + struct i915_sched_engine *sched_engine = engine->sched_engine;
> + const struct i915_request *inflight;
> +
> + /*
> + * We only need to kick the tasklet once for the high priority
> + * new context we add into the queue.
> + */
> + if (prio <= sched_engine->queue_priority_hint)
> + return;
> +
> + rcu_read_lock();
> +
> + /* Nothing currently active? We're overdue for a submission! */
> + inflight = execlists_active(&engine->execlists);
> + if (!inflight)
> + goto unlock;
> +
> + /*
> + * If we are already the currently executing context, don't
> + * bother evaluating if we should preempt ourselves.
> + */
> + if (inflight->context == rq->context)
> + goto unlock;
> +
> + ENGINE_TRACE(engine,
> + "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
> + prio,
> + rq->fence.context, rq->fence.seqno,
> + inflight->fence.context, inflight->fence.seqno,
> + inflight->sched.attr.priority);
> +
> + sched_engine->queue_priority_hint = prio;
> +
> + /*
> + * Allow preemption of low -> normal -> high, but we do
> + * not allow low priority tasks to preempt other low priority
> + * tasks under the impression that latency for low priority
> + * tasks does not matter (as much as background throughput),
> + * so kiss.
> + */
> + if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
> + tasklet_hi_schedule(&engine->execlists.tasklet);
> +
> +unlock:
> + rcu_read_unlock();
> +}
> +
> static void execlists_set_default_submission(struct intel_engine_cs *engine)
> {
> engine->submit_request = execlists_submit_request;
> engine->sched_engine->schedule = i915_schedule;
> + engine->sched_engine->kick_backend = kick_execlists;
> engine->execlists.tasklet.callback = execlists_submission_tasklet;
> }
>
> @@ -3702,6 +3753,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> ve->base.request_alloc = execlists_request_alloc;
>
> ve->base.sched_engine->schedule = i915_schedule;
> + ve->base.sched_engine->kick_backend = kick_execlists;
> ve->base.submit_request = virtual_submit_request;
> ve->base.bond_execute = virtual_bond_execute;
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 4bc6969f6a97..035b88f2e4aa 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -157,65 +157,6 @@ sched_lock_engine(const struct i915_sched_node *node,
> return locked;
> }
>
> -static inline int rq_prio(const struct i915_request *rq)
> -{
> - return rq->sched.attr.priority;
> -}
> -
> -static inline bool need_preempt(int prio, int active)
> -{
> - /*
> - * Allow preemption of low -> normal -> high, but we do
> - * not allow low priority tasks to preempt other low priority
> - * tasks under the impression that latency for low priority
> - * tasks does not matter (as much as background throughput),
> - * so kiss.
> - */
> - return prio >= max(I915_PRIORITY_NORMAL, active);
> -}
> -
> -static void kick_submission(struct intel_engine_cs *engine,
> - const struct i915_request *rq,
> - int prio)
> -{
> - const struct i915_request *inflight;
> -
> - /*
> - * We only need to kick the tasklet once for the high priority
> - * new context we add into the queue.
> - */
> - if (prio <= engine->sched_engine->queue_priority_hint)
> - return;
> -
> - rcu_read_lock();
> -
> - /* Nothing currently active? We're overdue for a submission! */
> - inflight = execlists_active(&engine->execlists);
> - if (!inflight)
> - goto unlock;
> -
> - /*
> - * If we are already the currently executing context, don't
> - * bother evaluating if we should preempt ourselves.
> - */
> - if (inflight->context == rq->context)
> - goto unlock;
> -
> - ENGINE_TRACE(engine,
> - "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
> - prio,
> - rq->fence.context, rq->fence.seqno,
> - inflight->fence.context, inflight->fence.seqno,
> - inflight->sched.attr.priority);
> -
> - engine->sched_engine->queue_priority_hint = prio;
> - if (need_preempt(prio, rq_prio(inflight)))
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> -
> -unlock:
> - rcu_read_unlock();
> -}
> -
> static void __i915_schedule(struct i915_sched_node *node,
> const struct i915_sched_attr *attr)
> {
> @@ -335,7 +276,8 @@ static void __i915_schedule(struct i915_sched_node *node,
> }
>
> /* Defer (tasklet) submission until after all of our updates. */
> - kick_submission(engine, node_to_request(node), prio);
> + if (engine->sched_engine->kick_backend)
> + engine->sched_engine->kick_backend(node_to_request(node), prio);
> }
>
> spin_unlock(&engine->sched_engine->lock);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 3f462f8b06f2..8b3af536e6cf 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> @@ -132,6 +132,10 @@ struct i915_sched_engine {
> */
> bool no_priolist;
>
> + /* Kick backend */
> + void (*kick_backend)(const struct i915_request *rq,
> + int prio);
> +
> /*
> * Call when the priority on a request has changed and it and its
> * dependencies may need rescheduling. Note the request itself may
> --
> 2.28.0
>
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 6/9] drm/i915: Add kick_backend function to i915_sched_engine
@ 2021-06-04 19:09 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 19:09 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Rather than touching execlist specific structures in the generic
> scheduling code, add a callback function in the backend.
Seems reasonable but why does the function that's there today do
nothing for the ringbuffer and current GuC back-ends? I'm sure
there's some reason but it's not immediately obvious to me.
--Jason
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> .../drm/i915/gt/intel_execlists_submission.c | 52 ++++++++++++++++
> drivers/gpu/drm/i915/i915_scheduler.c | 62 +------------------
> drivers/gpu/drm/i915/i915_scheduler_types.h | 4 ++
> 3 files changed, 58 insertions(+), 60 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 23fd03815ad0..3fac17103837 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -3116,10 +3116,61 @@ static bool can_preempt(struct intel_engine_cs *engine)
> return engine->class != RENDER_CLASS;
> }
>
> +static void kick_execlists(const struct i915_request *rq, int prio)
> +{
> + struct intel_engine_cs *engine = rq->engine;
> + struct i915_sched_engine *sched_engine = engine->sched_engine;
> + const struct i915_request *inflight;
> +
> + /*
> + * We only need to kick the tasklet once for the high priority
> + * new context we add into the queue.
> + */
> + if (prio <= sched_engine->queue_priority_hint)
> + return;
> +
> + rcu_read_lock();
> +
> + /* Nothing currently active? We're overdue for a submission! */
> + inflight = execlists_active(&engine->execlists);
> + if (!inflight)
> + goto unlock;
> +
> + /*
> + * If we are already the currently executing context, don't
> + * bother evaluating if we should preempt ourselves.
> + */
> + if (inflight->context == rq->context)
> + goto unlock;
> +
> + ENGINE_TRACE(engine,
> + "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
> + prio,
> + rq->fence.context, rq->fence.seqno,
> + inflight->fence.context, inflight->fence.seqno,
> + inflight->sched.attr.priority);
> +
> + sched_engine->queue_priority_hint = prio;
> +
> + /*
> + * Allow preemption of low -> normal -> high, but we do
> + * not allow low priority tasks to preempt other low priority
> + * tasks under the impression that latency for low priority
> + * tasks does not matter (as much as background throughput),
> + * so kiss.
> + */
> + if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
> + tasklet_hi_schedule(&engine->execlists.tasklet);
> +
> +unlock:
> + rcu_read_unlock();
> +}
> +
> static void execlists_set_default_submission(struct intel_engine_cs *engine)
> {
> engine->submit_request = execlists_submit_request;
> engine->sched_engine->schedule = i915_schedule;
> + engine->sched_engine->kick_backend = kick_execlists;
> engine->execlists.tasklet.callback = execlists_submission_tasklet;
> }
>
> @@ -3702,6 +3753,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> ve->base.request_alloc = execlists_request_alloc;
>
> ve->base.sched_engine->schedule = i915_schedule;
> + ve->base.sched_engine->kick_backend = kick_execlists;
> ve->base.submit_request = virtual_submit_request;
> ve->base.bond_execute = virtual_bond_execute;
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 4bc6969f6a97..035b88f2e4aa 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -157,65 +157,6 @@ sched_lock_engine(const struct i915_sched_node *node,
> return locked;
> }
>
> -static inline int rq_prio(const struct i915_request *rq)
> -{
> - return rq->sched.attr.priority;
> -}
> -
> -static inline bool need_preempt(int prio, int active)
> -{
> - /*
> - * Allow preemption of low -> normal -> high, but we do
> - * not allow low priority tasks to preempt other low priority
> - * tasks under the impression that latency for low priority
> - * tasks does not matter (as much as background throughput),
> - * so kiss.
> - */
> - return prio >= max(I915_PRIORITY_NORMAL, active);
> -}
> -
> -static void kick_submission(struct intel_engine_cs *engine,
> - const struct i915_request *rq,
> - int prio)
> -{
> - const struct i915_request *inflight;
> -
> - /*
> - * We only need to kick the tasklet once for the high priority
> - * new context we add into the queue.
> - */
> - if (prio <= engine->sched_engine->queue_priority_hint)
> - return;
> -
> - rcu_read_lock();
> -
> - /* Nothing currently active? We're overdue for a submission! */
> - inflight = execlists_active(&engine->execlists);
> - if (!inflight)
> - goto unlock;
> -
> - /*
> - * If we are already the currently executing context, don't
> - * bother evaluating if we should preempt ourselves.
> - */
> - if (inflight->context == rq->context)
> - goto unlock;
> -
> - ENGINE_TRACE(engine,
> - "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
> - prio,
> - rq->fence.context, rq->fence.seqno,
> - inflight->fence.context, inflight->fence.seqno,
> - inflight->sched.attr.priority);
> -
> - engine->sched_engine->queue_priority_hint = prio;
> - if (need_preempt(prio, rq_prio(inflight)))
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> -
> -unlock:
> - rcu_read_unlock();
> -}
> -
> static void __i915_schedule(struct i915_sched_node *node,
> const struct i915_sched_attr *attr)
> {
> @@ -335,7 +276,8 @@ static void __i915_schedule(struct i915_sched_node *node,
> }
>
> /* Defer (tasklet) submission until after all of our updates. */
> - kick_submission(engine, node_to_request(node), prio);
> + if (engine->sched_engine->kick_backend)
> + engine->sched_engine->kick_backend(node_to_request(node), prio);
> }
>
> spin_unlock(&engine->sched_engine->lock);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 3f462f8b06f2..8b3af536e6cf 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> @@ -132,6 +132,10 @@ struct i915_sched_engine {
> */
> bool no_priolist;
>
> + /* Kick backend */
> + void (*kick_backend)(const struct i915_request *rq,
> + int prio);
> +
> /*
> * Call when the priority on a request has changed and it and its
> * dependencies may need rescheduling. Note the request itself may
> --
> 2.28.0
>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 4/9] drm/i915: Move active tracking to i915_sched_engine
2021-06-04 19:00 ` [Intel-gfx] " Jason Ekstrand
@ 2021-06-04 19:12 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 19:12 UTC (permalink / raw)
To: Jason Ekstrand
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Fri, Jun 04, 2021 at 02:00:33PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Move active request tracking and its lock to i915_sched_engine. This
> > lock is also the submission lock so having it in the i915_sched_engine
> > is the correct place.
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gt/intel_engine.h | 2 -
> > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 43 +++-----
> > drivers/gpu/drm/i915/gt/intel_engine_types.h | 6 --
> > .../drm/i915/gt/intel_execlists_submission.c | 98 ++++++++++---------
> > .../gpu/drm/i915/gt/intel_ring_submission.c | 12 +--
> > drivers/gpu/drm/i915/gt/mock_engine.c | 7 +-
> > drivers/gpu/drm/i915/gt/selftest_execlists.c | 4 +-
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 20 ++--
> > drivers/gpu/drm/i915/i915_gpu_error.c | 4 +-
> > drivers/gpu/drm/i915/i915_request.c | 32 +++---
> > drivers/gpu/drm/i915/i915_request.h | 2 +-
> > drivers/gpu/drm/i915/i915_scheduler.c | 30 ++++--
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 9 ++
> > 13 files changed, 134 insertions(+), 135 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
> > index 8d9184920c51..a8b2174b4395 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
> > @@ -257,8 +257,6 @@ intel_engine_find_active_request(struct intel_engine_cs *engine);
> >
> > u32 intel_engine_context_size(struct intel_gt *gt, u8 class);
> >
> > -void intel_engine_init_active(struct intel_engine_cs *engine,
> > - unsigned int subclass);
> > #define ENGINE_PHYSICAL 0
> > #define ENGINE_MOCK 1
> > #define ENGINE_VIRTUAL 2
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > index 85f2effe9dc6..33d879137908 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > @@ -719,7 +719,6 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > if (err)
> > goto err_cmd_parser;
> >
> > - intel_engine_init_active(engine, ENGINE_PHYSICAL);
> > intel_engine_init_execlists(engine);
> > intel_engine_init__pm(engine);
> > intel_engine_init_retire(engine);
> > @@ -778,11 +777,11 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
> > frame->rq.ring = &frame->ring;
> >
> > mutex_lock(&ce->timeline->mutex);
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> >
> > dw = engine->emit_fini_breadcrumb(&frame->rq, frame->cs) - frame->cs;
> >
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > mutex_unlock(&ce->timeline->mutex);
> >
> > GEM_BUG_ON(dw & 1); /* RING_TAIL must be qword aligned */
> > @@ -791,28 +790,6 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
> > return dw;
> > }
> >
> > -void
> > -intel_engine_init_active(struct intel_engine_cs *engine, unsigned int subclass)
> > -{
> > - INIT_LIST_HEAD(&engine->active.requests);
> > - INIT_LIST_HEAD(&engine->active.hold);
> > -
> > - spin_lock_init(&engine->active.lock);
> > - lockdep_set_subclass(&engine->active.lock, subclass);
> > -
> > - /*
> > - * Due to an interesting quirk in lockdep's internal debug tracking,
> > - * after setting a subclass we must ensure the lock is used. Otherwise,
> > - * nr_unused_locks is incremented once too often.
> > - */
> > -#ifdef CONFIG_DEBUG_LOCK_ALLOC
> > - local_irq_disable();
> > - lock_map_acquire(&engine->active.lock.dep_map);
> > - lock_map_release(&engine->active.lock.dep_map);
> > - local_irq_enable();
> > -#endif
> > -}
> > -
> > static struct intel_context *
> > create_pinned_context(struct intel_engine_cs *engine,
> > unsigned int hwsp,
> > @@ -960,7 +937,7 @@ int intel_engines_init(struct intel_gt *gt)
> > */
> > void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > {
> > - GEM_BUG_ON(!list_empty(&engine->active.requests));
> > + GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
> > tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> >
> > i915_sched_engine_put(engine->sched_engine);
> > @@ -1353,7 +1330,7 @@ static struct intel_timeline *get_timeline(struct i915_request *rq)
> > struct intel_timeline *tl;
> >
> > /*
> > - * Even though we are holding the engine->active.lock here, there
> > + * Even though we are holding the engine->sched_engine->lock here, there
> > * is no control over the submission queue per-se and we are
> > * inspecting the active state at a random point in time, with an
> > * unknown queue. Play safe and make sure the timeline remains valid.
> > @@ -1700,7 +1677,7 @@ void intel_engine_dump(struct intel_engine_cs *engine,
> >
> > drm_printf(m, "\tRequests:\n");
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> > rq = intel_engine_find_active_request(engine);
> > if (rq) {
> > struct intel_timeline *tl = get_timeline(rq);
> > @@ -1731,8 +1708,9 @@ void intel_engine_dump(struct intel_engine_cs *engine,
> > hexdump(m, rq->context->lrc_reg_state, PAGE_SIZE);
> > }
> > }
> > - drm_printf(m, "\tOn hold?: %lu\n", list_count(&engine->active.hold));
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + drm_printf(m, "\tOn hold?: %lu\n",
> > + list_count(&engine->sched_engine->hold));
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> >
> > drm_printf(m, "\tMMIO base: 0x%08x\n", engine->mmio_base);
> > wakeref = intel_runtime_pm_get_if_in_use(engine->uncore->rpm);
> > @@ -1812,7 +1790,7 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
> > * At all other times, we must assume the GPU is still running, but
> > * we only care about the snapshot of this moment.
> > */
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> >
> > rcu_read_lock();
> > request = execlists_active(&engine->execlists);
> > @@ -1830,7 +1808,8 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
> > if (active)
> > return active;
> >
> > - list_for_each_entry(request, &engine->active.requests, sched.link) {
> > + list_for_each_entry(request, &engine->sched_engine->requests,
> > + sched.link) {
> > if (__i915_request_is_complete(request))
> > continue;
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > index 86b41ddec373..7197b9fa5e35 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > @@ -302,12 +302,6 @@ struct intel_engine_cs {
> >
> > struct intel_sseu sseu;
> >
> > - struct {
> > - spinlock_t lock;
> > - struct list_head requests;
> > - struct list_head hold; /* ready requests, but on hold */
> > - } active;
> > -
> > struct i915_sched_engine *sched_engine;
> >
> > /* keep a request in reserve for a [pm] barrier under oom */
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 609753b5401a..0413ba87cc1e 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -325,7 +325,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > * Check against the first request in ELSP[1], it will, thanks to the
> > * power of PI, be the highest priority of that context.
> > */
> > - if (!list_is_last(&rq->sched.link, &engine->active.requests) &&
> > + if (!list_is_last(&rq->sched.link, &engine->sched_engine->requests) &&
> > rq_prio(list_next_entry(rq, sched.link)) > last_prio)
> > return true;
> >
> > @@ -367,10 +367,10 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > struct list_head *pl;
> > int prio = I915_PRIORITY_INVALID;
> >
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> >
> > list_for_each_entry_safe_reverse(rq, rn,
> > - &engine->active.requests,
> > + &engine->sched_engine->requests,
> > sched.link) {
> > if (__i915_request_is_complete(rq)) {
> > list_del_init(&rq->sched.link);
> > @@ -534,13 +534,13 @@ resubmit_virtual_request(struct i915_request *rq, struct virtual_engine *ve)
> > {
> > struct intel_engine_cs *engine = rq->engine;
> >
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> >
> > clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > WRITE_ONCE(rq->engine, &ve->base);
> > ve->base.submit_request(rq);
> >
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > }
> >
> > static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
> > @@ -579,7 +579,7 @@ static void __execlists_schedule_out(struct i915_request * const rq,
> > unsigned int ccid;
> >
> > /*
> > - * NB process_csb() is not under the engine->active.lock and hence
> > + * NB process_csb() is not under the engine->sched_engine->lock and hence
> > * schedule_out can race with schedule_in meaning that we should
> > * refrain from doing non-trivial work here.
> > */
> > @@ -1133,7 +1133,8 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> > return false;
> >
> > /* If ELSP[1] is occupied, always check to see if worth slicing */
> > - if (!list_is_last_rcu(&rq->sched.link, &engine->active.requests)) {
> > + if (!list_is_last_rcu(&rq->sched.link,
> > + &engine->sched_engine->requests)) {
> > ENGINE_TRACE(engine, "timeslice required for second inflight context\n");
> > return true;
> > }
> > @@ -1266,7 +1267,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > * and context switches) submission.
> > */
> >
> > - spin_lock(&engine->active.lock);
> > + spin_lock(&sched_engine->lock);
> >
> > /*
> > * If the queue is higher priority than the last
> > @@ -1366,7 +1367,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > * Even if ELSP[1] is occupied and not worthy
> > * of timeslices, our queue might be.
> > */
> > - spin_unlock(&engine->active.lock);
> > + spin_unlock(&sched_engine->lock);
> > return;
> > }
> > }
> > @@ -1376,7 +1377,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > while ((ve = first_virtual_engine(engine))) {
> > struct i915_request *rq;
> >
> > - spin_lock(&ve->base.active.lock);
> > + spin_lock(&ve->base.sched_engine->lock);
> >
> > rq = ve->request;
> > if (unlikely(!virtual_matches(ve, rq, engine)))
> > @@ -1386,13 +1387,13 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > GEM_BUG_ON(rq->context != &ve->context);
> >
> > if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> > - spin_unlock(&ve->base.active.lock);
> > + spin_unlock(&ve->base.sched_engine->lock);
> > break;
> > }
> >
> > if (last && !can_merge_rq(last, rq)) {
> > - spin_unlock(&ve->base.active.lock);
> > - spin_unlock(&engine->active.lock);
> > + spin_unlock(&ve->base.sched_engine->lock);
> > + spin_unlock(&engine->sched_engine->lock);
> > return; /* leave this for another sibling */
> > }
> >
> > @@ -1438,7 +1439,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> >
> > i915_request_put(rq);
> > unlock:
> > - spin_unlock(&ve->base.active.lock);
> > + spin_unlock(&ve->base.sched_engine->lock);
> >
> > /*
> > * Hmm, we have a bunch of virtual engine requests,
> > @@ -1554,7 +1555,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > */
> > sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > i915_sched_engine_reset_on_empty(sched_engine);
> > - spin_unlock(&engine->active.lock);
> > + spin_unlock(&sched_engine->lock);
> >
> > /*
> > * We can skip poking the HW if we ended up with exactly the same set
> > @@ -1981,7 +1982,8 @@ static void __execlists_hold(struct i915_request *rq)
> > __i915_request_unsubmit(rq);
> >
> > clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > - list_move_tail(&rq->sched.link, &rq->engine->active.hold);
> > + list_move_tail(&rq->sched.link,
> > + &rq->engine->sched_engine->hold);
> > i915_request_set_hold(rq);
> > RQ_TRACE(rq, "on hold\n");
> >
> > @@ -2018,7 +2020,7 @@ static bool execlists_hold(struct intel_engine_cs *engine,
> > if (i915_request_on_hold(rq))
> > return false;
> >
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> >
> > if (__i915_request_is_complete(rq)) { /* too late! */
> > rq = NULL;
> > @@ -2034,10 +2036,10 @@ static bool execlists_hold(struct intel_engine_cs *engine,
> > GEM_BUG_ON(i915_request_on_hold(rq));
> > GEM_BUG_ON(rq->engine != engine);
> > __execlists_hold(rq);
> > - GEM_BUG_ON(list_empty(&engine->active.hold));
> > + GEM_BUG_ON(list_empty(&engine->sched_engine->hold));
> >
> > unlock:
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > return rq;
> > }
> >
> > @@ -2117,7 +2119,7 @@ static void __execlists_unhold(struct i915_request *rq)
> > static void execlists_unhold(struct intel_engine_cs *engine,
> > struct i915_request *rq)
> > {
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> >
> > /*
> > * Move this request back to the priority queue, and all of its
> > @@ -2130,7 +2132,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> > tasklet_hi_schedule(&engine->execlists.tasklet);
> > }
> >
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > }
> >
> > struct execlists_capture {
> > @@ -2260,13 +2262,13 @@ static void execlists_capture(struct intel_engine_cs *engine)
> > if (!cap)
> > return;
> >
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> > cap->rq = active_context(engine, active_ccid(engine));
> > if (cap->rq) {
> > cap->rq = active_request(cap->rq->context->timeline, cap->rq);
> > cap->rq = i915_request_get_rcu(cap->rq);
> > }
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > if (!cap->rq)
> > goto err_free;
> >
> > @@ -2470,7 +2472,7 @@ static bool ancestor_on_hold(const struct intel_engine_cs *engine,
> > const struct i915_request *rq)
> > {
> > GEM_BUG_ON(i915_request_on_hold(rq));
> > - return !list_empty(&engine->active.hold) && hold_request(rq);
> > + return !list_empty(&engine->sched_engine->hold) && hold_request(rq);
> > }
> >
> > static void execlists_submit_request(struct i915_request *request)
> > @@ -2479,11 +2481,12 @@ static void execlists_submit_request(struct i915_request *request)
> > unsigned long flags;
> >
> > /* Will be called from irq-context when using foreign fences. */
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > if (unlikely(ancestor_on_hold(engine, request))) {
> > RQ_TRACE(request, "ancestor on hold\n");
> > - list_add_tail(&request->sched.link, &engine->active.hold);
> > + list_add_tail(&request->sched.link,
> > + &engine->sched_engine->hold);
> > i915_request_set_hold(request);
> > } else {
> > queue_request(engine, request);
> > @@ -2495,7 +2498,7 @@ static void execlists_submit_request(struct i915_request *request)
> > __execlists_kick(&engine->execlists);
> > }
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static int
> > @@ -2959,9 +2962,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> >
> > /* Push back any incomplete requests for replay after the reset. */
> > rcu_read_lock();
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> > __unwind_incomplete_requests(engine);
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > rcu_read_unlock();
> > }
> >
> > @@ -3001,10 +3004,10 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > execlists_reset_csb(engine, true);
> >
> > rcu_read_lock();
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > /* Mark all executing requests as skipped. */
> > - list_for_each_entry(rq, &engine->active.requests, sched.link)
> > + list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
> > i915_request_put(i915_request_mark_eio(rq));
> > intel_engine_signal_breadcrumbs(engine);
> >
> > @@ -3024,7 +3027,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > }
> >
> > /* On-hold requests will be flushed to timeline upon their release */
> > - list_for_each_entry(rq, &engine->active.hold, sched.link)
> > + list_for_each_entry(rq, &sched_engine->hold, sched.link)
> > i915_request_put(i915_request_mark_eio(rq));
> >
> > /* Cancel all attached virtual engines */
> > @@ -3035,7 +3038,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > rb_erase_cached(rb, &execlists->virtual);
> > RB_CLEAR_NODE(rb);
> >
> > - spin_lock(&ve->base.active.lock);
> > + spin_lock(&ve->base.sched_engine->lock);
> > rq = fetch_and_zero(&ve->request);
> > if (rq) {
> > if (i915_request_mark_eio(rq)) {
> > @@ -3047,7 +3050,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> >
> > ve->base.sched_engine->queue_priority_hint = INT_MIN;
> > }
> > - spin_unlock(&ve->base.active.lock);
> > + spin_unlock(&ve->base.sched_engine->lock);
> > }
> >
> > /* Remaining _unready_ requests will be nop'ed when submitted */
> > @@ -3058,7 +3061,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > execlists->tasklet.callback = nop_submission_tasklet;
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > rcu_read_unlock();
> > }
> >
> > @@ -3304,7 +3307,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > if (unlikely(ve->request)) {
> > struct i915_request *old;
> >
> > - spin_lock_irq(&ve->base.active.lock);
> > + spin_lock_irq(&ve->base.sched_engine->lock);
> >
> > old = fetch_and_zero(&ve->request);
> > if (old) {
> > @@ -3313,7 +3316,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > i915_request_put(old);
> > }
> >
> > - spin_unlock_irq(&ve->base.active.lock);
> > + spin_unlock_irq(&ve->base.sched_engine->lock);
> > }
> >
> > /*
> > @@ -3333,13 +3336,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > if (RB_EMPTY_NODE(node))
> > continue;
> >
> > - spin_lock_irq(&sibling->active.lock);
> > + spin_lock_irq(&sibling->sched_engine->lock);
> >
> > /* Detachment is lazily performed in the execlists tasklet */
> > if (!RB_EMPTY_NODE(node))
> > rb_erase_cached(node, &sibling->execlists.virtual);
> >
> > - spin_unlock_irq(&sibling->active.lock);
> > + spin_unlock_irq(&sibling->sched_engine->lock);
> > }
> > GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
> > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > @@ -3509,7 +3512,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > if (!READ_ONCE(ve->request))
> > break; /* already handled by a sibling's tasklet */
> >
> > - spin_lock_irq(&sibling->active.lock);
> > + spin_lock_irq(&sibling->sched_engine->lock);
> >
> > if (unlikely(!(mask & sibling->mask))) {
> > if (!RB_EMPTY_NODE(&node->rb)) {
> > @@ -3562,7 +3565,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > tasklet_hi_schedule(&sibling->execlists.tasklet);
> >
> > unlock_engine:
> > - spin_unlock_irq(&sibling->active.lock);
> > + spin_unlock_irq(&sibling->sched_engine->lock);
> >
> > if (intel_context_inflight(&ve->context))
> > break;
> > @@ -3580,7 +3583,7 @@ static void virtual_submit_request(struct i915_request *rq)
> >
> > GEM_BUG_ON(ve->base.submit_request != virtual_submit_request);
> >
> > - spin_lock_irqsave(&ve->base.active.lock, flags);
> > + spin_lock_irqsave(&ve->base.sched_engine->lock, flags);
> >
> > /* By the time we resubmit a request, it may be completed */
> > if (__i915_request_is_complete(rq)) {
> > @@ -3603,7 +3606,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > tasklet_hi_schedule(&ve->base.execlists.tasklet);
> >
> > unlock:
> > - spin_unlock_irqrestore(&ve->base.active.lock, flags);
> > + spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
> > }
> >
> > static struct ve_bond *
> > @@ -3687,7 +3690,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> >
> > snprintf(ve->base.name, sizeof(ve->base.name), "virtual");
> >
> > - intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> > intel_engine_init_execlists(&ve->base);
> >
> > ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> > @@ -3860,17 +3862,17 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > unsigned int max)
> > {
> > const struct intel_engine_execlists *execlists = &engine->execlists;
> > - const struct i915_sched_engine *sched_engine = engine->sched_engine;
> > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> > struct i915_request *rq, *last;
> > unsigned long flags;
> > unsigned int count;
> > struct rb_node *rb;
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&sched_engine->lock, flags);
> >
> > last = NULL;
> > count = 0;
> > - list_for_each_entry(rq, &engine->active.requests, sched.link) {
> > + list_for_each_entry(rq, &sched_engine->requests, sched.link) {
> > if (count++ < max - 1)
> > show_request(m, rq, "\t\t", 0);
> > else
> > @@ -3933,7 +3935,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > show_request(m, last, "\t\t", 0);
> > }
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&sched_engine->lock, flags);
> > }
> >
> > #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
> > diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> > index 2b6dffcc2262..c98ae0f58790 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> > @@ -339,9 +339,9 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > u32 head;
> >
> > rq = NULL;
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> > rcu_read_lock();
> > - list_for_each_entry(pos, &engine->active.requests, sched.link) {
> > + list_for_each_entry(pos, &engine->sched_engine->requests, sched.link) {
> > if (!__i915_request_is_complete(pos)) {
> > rq = pos;
> > break;
> > @@ -396,7 +396,7 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > }
> > engine->legacy.ring->head = intel_ring_wrap(engine->legacy.ring, head);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void reset_finish(struct intel_engine_cs *engine)
> > @@ -408,16 +408,16 @@ static void reset_cancel(struct intel_engine_cs *engine)
> > struct i915_request *request;
> > unsigned long flags;
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > /* Mark all submitted requests as skipped. */
> > - list_for_each_entry(request, &engine->active.requests, sched.link)
> > + list_for_each_entry(request, &engine->sched_engine->requests, sched.link)
> > i915_request_put(i915_request_mark_eio(request));
> > intel_engine_signal_breadcrumbs(engine);
> >
> > /* Remaining _unready_ requests will be nop'ed when submitted */
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void i9xx_submit_request(struct i915_request *request)
> > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > index b1fdba13e900..a49fd3039f13 100644
> > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > @@ -253,10 +253,10 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
> >
> > del_timer_sync(&mock->hw_delay);
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > /* Mark all submitted requests as skipped. */
> > - list_for_each_entry(rq, &engine->active.requests, sched.link)
> > + list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
> > i915_request_put(i915_request_mark_eio(rq));
> > intel_engine_signal_breadcrumbs(engine);
> >
> > @@ -269,7 +269,7 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
> > }
> > INIT_LIST_HEAD(&mock->hw_queue);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void mock_reset_finish(struct intel_engine_cs *engine)
> > @@ -350,7 +350,6 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > if (!engine->sched_engine)
> > return -ENOMEM;
> >
> > - intel_engine_init_active(engine, ENGINE_MOCK);
> > intel_engine_init_execlists(engine);
> > intel_engine_init__pm(engine);
> > intel_engine_init_retire(engine);
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > index 1f93591a8c69..fd824fa329c2 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > @@ -4599,9 +4599,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
> > GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
> >
> > /* Fake a preemption event; failed of course */
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> > __unwind_incomplete_requests(engine);
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > GEM_BUG_ON(rq->engine != engine);
> >
> > /* Reset the engine while keeping our active request on hold */
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index f4a6fbfaf82e..2551019ee217 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -190,7 +190,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > bool submit = false;
> > struct rb_node *rb;
> >
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&sched_engine->lock);
> >
> > if (last) {
> > if (*++first)
> > @@ -247,7 +247,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
> > struct i915_request **port, *rq;
> > unsigned long flags;
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > for (port = execlists->inflight; (rq = *port); port++) {
> > if (!i915_request_completed(rq))
> > @@ -265,7 +265,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
> >
> > i915_sched_engine_reset_on_empty(engine->sched_engine);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
> > @@ -322,7 +322,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > struct i915_request *rq;
> > unsigned long flags;
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > /* Push back any incomplete requests for replay after the reset. */
> > rq = execlists_unwind_incomplete_requests(execlists);
> > @@ -336,7 +336,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > guc_reset_state(rq->context, engine, rq->head, stalled);
> >
> > out_unlock:
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void guc_reset_cancel(struct intel_engine_cs *engine)
> > @@ -362,10 +362,10 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > * submission's irq state, we also wish to remind ourselves that
> > * it is irq state.)
> > */
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&sched_engine->lock, flags);
> >
> > /* Mark all executing requests as skipped. */
> > - list_for_each_entry(rq, &engine->active.requests, sched.link) {
> > + list_for_each_entry(rq, &sched_engine->requests, sched.link) {
> > i915_request_set_error_once(rq, -EIO);
> > i915_request_mark_complete(rq);
> > }
> > @@ -390,7 +390,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > sched_engine->queue_priority_hint = INT_MIN;
> > sched_engine->queue = RB_ROOT_CACHED;
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&sched_engine->lock, flags);
> > }
> >
> > static void guc_reset_finish(struct intel_engine_cs *engine)
> > @@ -539,7 +539,7 @@ static void guc_submit_request(struct i915_request *rq)
> > unsigned long flags;
> >
> > /* Will be called from irq-context when using foreign fences. */
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > queue_request(engine, rq, rq_prio(rq));
> >
> > @@ -548,7 +548,7 @@ static void guc_submit_request(struct i915_request *rq)
> >
> > tasklet_hi_schedule(&engine->execlists.tasklet);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void sanitize_hwsp(struct intel_engine_cs *engine)
> > diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
> > index 833d3e8b7631..cd005f0a203c 100644
> > --- a/drivers/gpu/drm/i915/i915_gpu_error.c
> > +++ b/drivers/gpu/drm/i915/i915_gpu_error.c
> > @@ -1436,12 +1436,12 @@ capture_engine(struct intel_engine_cs *engine,
> > if (!ee)
> > return NULL;
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> > rq = intel_engine_find_active_request(engine);
> > if (rq)
> > capture = intel_engine_coredump_add_request(ee, rq,
> > ATOMIC_MAYFAIL);
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > if (!capture) {
> > kfree(ee);
> > return NULL;
> > diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
> > index 970d8f4986bb..95e859c319ca 100644
> > --- a/drivers/gpu/drm/i915/i915_request.c
> > +++ b/drivers/gpu/drm/i915/i915_request.c
> > @@ -272,11 +272,11 @@ i915_request_active_engine(struct i915_request *rq,
> > * check that we have acquired the lock on the final engine.
> > */
> > locked = READ_ONCE(rq->engine);
> > - spin_lock_irq(&locked->active.lock);
> > + spin_lock_irq(&locked->sched_engine->lock);
> > while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
> > - spin_unlock(&locked->active.lock);
> > + spin_unlock(&locked->sched_engine->lock);
> > locked = engine;
> > - spin_lock(&locked->active.lock);
> > + spin_lock(&locked->sched_engine->lock);
> > }
> >
> > if (i915_request_is_active(rq)) {
> > @@ -285,7 +285,7 @@ i915_request_active_engine(struct i915_request *rq,
> > ret = true;
> > }
> >
> > - spin_unlock_irq(&locked->active.lock);
> > + spin_unlock_irq(&locked->sched_engine->lock);
> >
> > return ret;
> > }
> > @@ -302,10 +302,10 @@ static void remove_from_engine(struct i915_request *rq)
> > * check that the rq still belongs to the newly locked engine.
> > */
> > locked = READ_ONCE(rq->engine);
> > - spin_lock_irq(&locked->active.lock);
> > + spin_lock_irq(&locked->sched_engine->lock);
> > while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
> > - spin_unlock(&locked->active.lock);
> > - spin_lock(&engine->active.lock);
> > + spin_unlock(&locked->sched_engine->lock);
> > + spin_lock(&engine->sched_engine->lock);
> > locked = engine;
> > }
> > list_del_init(&rq->sched.link);
> > @@ -316,7 +316,7 @@ static void remove_from_engine(struct i915_request *rq)
> > /* Prevent further __await_execution() registering a cb, then flush */
> > set_bit(I915_FENCE_FLAG_ACTIVE, &rq->fence.flags);
> >
> > - spin_unlock_irq(&locked->active.lock);
> > + spin_unlock_irq(&locked->sched_engine->lock);
> >
> > __notify_execute_cb_imm(rq);
> > }
> > @@ -637,7 +637,7 @@ bool __i915_request_submit(struct i915_request *request)
> > RQ_TRACE(request, "\n");
> >
> > GEM_BUG_ON(!irqs_disabled());
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> >
> > /*
> > * With the advent of preempt-to-busy, we frequently encounter
> > @@ -649,7 +649,7 @@ bool __i915_request_submit(struct i915_request *request)
> > *
> > * We must remove the request from the caller's priority queue,
> > * and the caller must only call us when the request is in their
> > - * priority queue, under the active.lock. This ensures that the
> > + * priority queue, under the sched_engine->lock. This ensures that the
> > * request has *not* yet been retired and we can safely move
> > * the request into the engine->active.list where it will be
> > * dropped upon retiring. (Otherwise if resubmit a *retired*
> > @@ -694,7 +694,7 @@ bool __i915_request_submit(struct i915_request *request)
> > result = true;
> >
> > GEM_BUG_ON(test_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags));
> > - list_move_tail(&request->sched.link, &engine->active.requests);
> > + list_move_tail(&request->sched.link, &engine->sched_engine->requests);
> > active:
> > clear_bit(I915_FENCE_FLAG_PQUEUE, &request->fence.flags);
> > set_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags);
> > @@ -724,11 +724,11 @@ void i915_request_submit(struct i915_request *request)
> > unsigned long flags;
> >
> > /* Will be called from irq-context when using foreign fences. */
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > __i915_request_submit(request);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > void __i915_request_unsubmit(struct i915_request *request)
> > @@ -742,7 +742,7 @@ void __i915_request_unsubmit(struct i915_request *request)
> > RQ_TRACE(request, "\n");
> >
> > GEM_BUG_ON(!irqs_disabled());
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> >
> > /*
> > * Before we remove this breadcrumb from the signal list, we have
> > @@ -775,11 +775,11 @@ void i915_request_unsubmit(struct i915_request *request)
> > unsigned long flags;
> >
> > /* Will be called from irq-context when using foreign fences. */
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > __i915_request_unsubmit(request);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void __cancel_request(struct i915_request *rq)
> > diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
> > index 270f6cd37650..239964bec1fa 100644
> > --- a/drivers/gpu/drm/i915/i915_request.h
> > +++ b/drivers/gpu/drm/i915/i915_request.h
> > @@ -613,7 +613,7 @@ i915_request_active_timeline(const struct i915_request *rq)
> > * this submission.
> > */
> > return rcu_dereference_protected(rq->timeline,
> > - lockdep_is_held(&rq->engine->active.lock));
> > + lockdep_is_held(&rq->engine->sched_engine->lock));
> > }
> >
> > static inline u32
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > index 4953874a9ba6..4bc6969f6a97 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > @@ -68,7 +68,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > struct rb_node **parent, *rb;
> > bool first = true;
> >
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> > assert_priolists(sched_engine);
> >
> > if (unlikely(sched_engine->no_priolist))
> > @@ -147,9 +147,9 @@ sched_lock_engine(const struct i915_sched_node *node,
> > * check that the rq still belongs to the newly locked engine.
> > */
> > while (locked != (engine = READ_ONCE(rq->engine))) {
> > - spin_unlock(&locked->active.lock);
> > + spin_unlock(&locked->sched_engine->lock);
> > memset(cache, 0, sizeof(*cache));
> > - spin_lock(&engine->active.lock);
> > + spin_lock(&engine->sched_engine->lock);
> > locked = engine;
> > }
> >
> > @@ -296,7 +296,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> >
> > memset(&cache, 0, sizeof(cache));
> > engine = node_to_request(node)->engine;
> > - spin_lock(&engine->active.lock);
> > + spin_lock(&engine->sched_engine->lock);
> >
> > /* Fifo and depth-first replacement ensure our deps execute before us */
> > engine = sched_lock_engine(node, engine, &cache);
> > @@ -305,7 +305,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> >
> > node = dep->signaler;
> > engine = sched_lock_engine(node, engine, &cache);
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> >
> > /* Recheck after acquiring the engine->timeline.lock */
> > if (prio <= node->attr.priority || node_signaled(node))
> > @@ -338,7 +338,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> > kick_submission(engine, node_to_request(node), prio);
> > }
> >
> > - spin_unlock(&engine->active.lock);
> > + spin_unlock(&engine->sched_engine->lock);
> > }
> >
> > void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
> > @@ -511,6 +511,24 @@ i915_sched_engine_create(unsigned int subclass)
> > sched_engine->queue = RB_ROOT_CACHED;
> > sched_engine->queue_priority_hint = INT_MIN;
> >
> > + INIT_LIST_HEAD(&sched_engine->requests);
> > + INIT_LIST_HEAD(&sched_engine->hold);
> > +
> > + spin_lock_init(&sched_engine->lock);
> > + lockdep_set_subclass(&sched_engine->lock, subclass);
> > +
> > + /*
> > + * Due to an interesting quirk in lockdep's internal debug tracking,
> > + * after setting a subclass we must ensure the lock is used. Otherwise,
> > + * nr_unused_locks is incremented once too often.
> > + */
> > +#ifdef CONFIG_DEBUG_LOCK_ALLOC
> > + local_irq_disable();
> > + lock_map_acquire(&sched_engine->lock.dep_map);
> > + lock_map_release(&sched_engine->lock.dep_map);
> > + local_irq_enable();
> > +#endif
> > +
> > return sched_engine;
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 16a054e67760..5cb0b1c98d6b 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > @@ -94,6 +94,15 @@ struct i915_dependency {
> > struct i915_sched_engine {
> > struct kref ref;
> >
> > + /**
> > + * @lock: protects requests in priority lists, requests, hold and
> > + * tasklet while running
>
> To be clear, this protects other stuff in this struct, right? Not
> just requests and hold?
>
Yea, this lock is kinda mess. It protects the request when it is in the
priority list (struct i915_priolist) and these lists. It also is the
submission lock, e.g. you have to hold this lock when submitting /
unsubmitting the request. It also is used for some timeline RCU thing I
don't fully understand.
In the GuC submission code this lock is used as a submission lock for
consistency with the execlist code.
Once we go the DRM scheduler these lists go away, at least with GuC
submission. Also a submission lock really isn't needed as the backend
will have a single entry point rather and only 1 request is allowed in
the backend at time. Hoping we can push this lock + all of these
structures in an execlist specific structure + only use it in
intel_execlists_submission.c and be done with it.
> Otherwise, this looks mechanically correct. Still not getting the big
Anyways, there is not a change in behavior here.
Matt
> picture yet.
>
> --Jason
>
>
> > + */
> > + spinlock_t lock;
> > +
> > + struct list_head requests;
> > + struct list_head hold; /* ready requests, but on hold */
> > +
> > /**
> > * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > */
> > --
> > 2.28.0
> >
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 4/9] drm/i915: Move active tracking to i915_sched_engine
@ 2021-06-04 19:12 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 19:12 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 02:00:33PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Move active request tracking and its lock to i915_sched_engine. This
> > lock is also the submission lock so having it in the i915_sched_engine
> > is the correct place.
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gt/intel_engine.h | 2 -
> > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 43 +++-----
> > drivers/gpu/drm/i915/gt/intel_engine_types.h | 6 --
> > .../drm/i915/gt/intel_execlists_submission.c | 98 ++++++++++---------
> > .../gpu/drm/i915/gt/intel_ring_submission.c | 12 +--
> > drivers/gpu/drm/i915/gt/mock_engine.c | 7 +-
> > drivers/gpu/drm/i915/gt/selftest_execlists.c | 4 +-
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 20 ++--
> > drivers/gpu/drm/i915/i915_gpu_error.c | 4 +-
> > drivers/gpu/drm/i915/i915_request.c | 32 +++---
> > drivers/gpu/drm/i915/i915_request.h | 2 +-
> > drivers/gpu/drm/i915/i915_scheduler.c | 30 ++++--
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 9 ++
> > 13 files changed, 134 insertions(+), 135 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
> > index 8d9184920c51..a8b2174b4395 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
> > @@ -257,8 +257,6 @@ intel_engine_find_active_request(struct intel_engine_cs *engine);
> >
> > u32 intel_engine_context_size(struct intel_gt *gt, u8 class);
> >
> > -void intel_engine_init_active(struct intel_engine_cs *engine,
> > - unsigned int subclass);
> > #define ENGINE_PHYSICAL 0
> > #define ENGINE_MOCK 1
> > #define ENGINE_VIRTUAL 2
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > index 85f2effe9dc6..33d879137908 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > @@ -719,7 +719,6 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > if (err)
> > goto err_cmd_parser;
> >
> > - intel_engine_init_active(engine, ENGINE_PHYSICAL);
> > intel_engine_init_execlists(engine);
> > intel_engine_init__pm(engine);
> > intel_engine_init_retire(engine);
> > @@ -778,11 +777,11 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
> > frame->rq.ring = &frame->ring;
> >
> > mutex_lock(&ce->timeline->mutex);
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> >
> > dw = engine->emit_fini_breadcrumb(&frame->rq, frame->cs) - frame->cs;
> >
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > mutex_unlock(&ce->timeline->mutex);
> >
> > GEM_BUG_ON(dw & 1); /* RING_TAIL must be qword aligned */
> > @@ -791,28 +790,6 @@ static int measure_breadcrumb_dw(struct intel_context *ce)
> > return dw;
> > }
> >
> > -void
> > -intel_engine_init_active(struct intel_engine_cs *engine, unsigned int subclass)
> > -{
> > - INIT_LIST_HEAD(&engine->active.requests);
> > - INIT_LIST_HEAD(&engine->active.hold);
> > -
> > - spin_lock_init(&engine->active.lock);
> > - lockdep_set_subclass(&engine->active.lock, subclass);
> > -
> > - /*
> > - * Due to an interesting quirk in lockdep's internal debug tracking,
> > - * after setting a subclass we must ensure the lock is used. Otherwise,
> > - * nr_unused_locks is incremented once too often.
> > - */
> > -#ifdef CONFIG_DEBUG_LOCK_ALLOC
> > - local_irq_disable();
> > - lock_map_acquire(&engine->active.lock.dep_map);
> > - lock_map_release(&engine->active.lock.dep_map);
> > - local_irq_enable();
> > -#endif
> > -}
> > -
> > static struct intel_context *
> > create_pinned_context(struct intel_engine_cs *engine,
> > unsigned int hwsp,
> > @@ -960,7 +937,7 @@ int intel_engines_init(struct intel_gt *gt)
> > */
> > void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > {
> > - GEM_BUG_ON(!list_empty(&engine->active.requests));
> > + GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
> > tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> >
> > i915_sched_engine_put(engine->sched_engine);
> > @@ -1353,7 +1330,7 @@ static struct intel_timeline *get_timeline(struct i915_request *rq)
> > struct intel_timeline *tl;
> >
> > /*
> > - * Even though we are holding the engine->active.lock here, there
> > + * Even though we are holding the engine->sched_engine->lock here, there
> > * is no control over the submission queue per-se and we are
> > * inspecting the active state at a random point in time, with an
> > * unknown queue. Play safe and make sure the timeline remains valid.
> > @@ -1700,7 +1677,7 @@ void intel_engine_dump(struct intel_engine_cs *engine,
> >
> > drm_printf(m, "\tRequests:\n");
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> > rq = intel_engine_find_active_request(engine);
> > if (rq) {
> > struct intel_timeline *tl = get_timeline(rq);
> > @@ -1731,8 +1708,9 @@ void intel_engine_dump(struct intel_engine_cs *engine,
> > hexdump(m, rq->context->lrc_reg_state, PAGE_SIZE);
> > }
> > }
> > - drm_printf(m, "\tOn hold?: %lu\n", list_count(&engine->active.hold));
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + drm_printf(m, "\tOn hold?: %lu\n",
> > + list_count(&engine->sched_engine->hold));
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> >
> > drm_printf(m, "\tMMIO base: 0x%08x\n", engine->mmio_base);
> > wakeref = intel_runtime_pm_get_if_in_use(engine->uncore->rpm);
> > @@ -1812,7 +1790,7 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
> > * At all other times, we must assume the GPU is still running, but
> > * we only care about the snapshot of this moment.
> > */
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> >
> > rcu_read_lock();
> > request = execlists_active(&engine->execlists);
> > @@ -1830,7 +1808,8 @@ intel_engine_find_active_request(struct intel_engine_cs *engine)
> > if (active)
> > return active;
> >
> > - list_for_each_entry(request, &engine->active.requests, sched.link) {
> > + list_for_each_entry(request, &engine->sched_engine->requests,
> > + sched.link) {
> > if (__i915_request_is_complete(request))
> > continue;
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > index 86b41ddec373..7197b9fa5e35 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > @@ -302,12 +302,6 @@ struct intel_engine_cs {
> >
> > struct intel_sseu sseu;
> >
> > - struct {
> > - spinlock_t lock;
> > - struct list_head requests;
> > - struct list_head hold; /* ready requests, but on hold */
> > - } active;
> > -
> > struct i915_sched_engine *sched_engine;
> >
> > /* keep a request in reserve for a [pm] barrier under oom */
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 609753b5401a..0413ba87cc1e 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -325,7 +325,7 @@ static bool need_preempt(const struct intel_engine_cs *engine,
> > * Check against the first request in ELSP[1], it will, thanks to the
> > * power of PI, be the highest priority of that context.
> > */
> > - if (!list_is_last(&rq->sched.link, &engine->active.requests) &&
> > + if (!list_is_last(&rq->sched.link, &engine->sched_engine->requests) &&
> > rq_prio(list_next_entry(rq, sched.link)) > last_prio)
> > return true;
> >
> > @@ -367,10 +367,10 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > struct list_head *pl;
> > int prio = I915_PRIORITY_INVALID;
> >
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> >
> > list_for_each_entry_safe_reverse(rq, rn,
> > - &engine->active.requests,
> > + &engine->sched_engine->requests,
> > sched.link) {
> > if (__i915_request_is_complete(rq)) {
> > list_del_init(&rq->sched.link);
> > @@ -534,13 +534,13 @@ resubmit_virtual_request(struct i915_request *rq, struct virtual_engine *ve)
> > {
> > struct intel_engine_cs *engine = rq->engine;
> >
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> >
> > clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > WRITE_ONCE(rq->engine, &ve->base);
> > ve->base.submit_request(rq);
> >
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > }
> >
> > static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
> > @@ -579,7 +579,7 @@ static void __execlists_schedule_out(struct i915_request * const rq,
> > unsigned int ccid;
> >
> > /*
> > - * NB process_csb() is not under the engine->active.lock and hence
> > + * NB process_csb() is not under the engine->sched_engine->lock and hence
> > * schedule_out can race with schedule_in meaning that we should
> > * refrain from doing non-trivial work here.
> > */
> > @@ -1133,7 +1133,8 @@ static bool needs_timeslice(const struct intel_engine_cs *engine,
> > return false;
> >
> > /* If ELSP[1] is occupied, always check to see if worth slicing */
> > - if (!list_is_last_rcu(&rq->sched.link, &engine->active.requests)) {
> > + if (!list_is_last_rcu(&rq->sched.link,
> > + &engine->sched_engine->requests)) {
> > ENGINE_TRACE(engine, "timeslice required for second inflight context\n");
> > return true;
> > }
> > @@ -1266,7 +1267,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > * and context switches) submission.
> > */
> >
> > - spin_lock(&engine->active.lock);
> > + spin_lock(&sched_engine->lock);
> >
> > /*
> > * If the queue is higher priority than the last
> > @@ -1366,7 +1367,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > * Even if ELSP[1] is occupied and not worthy
> > * of timeslices, our queue might be.
> > */
> > - spin_unlock(&engine->active.lock);
> > + spin_unlock(&sched_engine->lock);
> > return;
> > }
> > }
> > @@ -1376,7 +1377,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > while ((ve = first_virtual_engine(engine))) {
> > struct i915_request *rq;
> >
> > - spin_lock(&ve->base.active.lock);
> > + spin_lock(&ve->base.sched_engine->lock);
> >
> > rq = ve->request;
> > if (unlikely(!virtual_matches(ve, rq, engine)))
> > @@ -1386,13 +1387,13 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > GEM_BUG_ON(rq->context != &ve->context);
> >
> > if (unlikely(rq_prio(rq) < queue_prio(sched_engine))) {
> > - spin_unlock(&ve->base.active.lock);
> > + spin_unlock(&ve->base.sched_engine->lock);
> > break;
> > }
> >
> > if (last && !can_merge_rq(last, rq)) {
> > - spin_unlock(&ve->base.active.lock);
> > - spin_unlock(&engine->active.lock);
> > + spin_unlock(&ve->base.sched_engine->lock);
> > + spin_unlock(&engine->sched_engine->lock);
> > return; /* leave this for another sibling */
> > }
> >
> > @@ -1438,7 +1439,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> >
> > i915_request_put(rq);
> > unlock:
> > - spin_unlock(&ve->base.active.lock);
> > + spin_unlock(&ve->base.sched_engine->lock);
> >
> > /*
> > * Hmm, we have a bunch of virtual engine requests,
> > @@ -1554,7 +1555,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
> > */
> > sched_engine->queue_priority_hint = queue_prio(sched_engine);
> > i915_sched_engine_reset_on_empty(sched_engine);
> > - spin_unlock(&engine->active.lock);
> > + spin_unlock(&sched_engine->lock);
> >
> > /*
> > * We can skip poking the HW if we ended up with exactly the same set
> > @@ -1981,7 +1982,8 @@ static void __execlists_hold(struct i915_request *rq)
> > __i915_request_unsubmit(rq);
> >
> > clear_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > - list_move_tail(&rq->sched.link, &rq->engine->active.hold);
> > + list_move_tail(&rq->sched.link,
> > + &rq->engine->sched_engine->hold);
> > i915_request_set_hold(rq);
> > RQ_TRACE(rq, "on hold\n");
> >
> > @@ -2018,7 +2020,7 @@ static bool execlists_hold(struct intel_engine_cs *engine,
> > if (i915_request_on_hold(rq))
> > return false;
> >
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> >
> > if (__i915_request_is_complete(rq)) { /* too late! */
> > rq = NULL;
> > @@ -2034,10 +2036,10 @@ static bool execlists_hold(struct intel_engine_cs *engine,
> > GEM_BUG_ON(i915_request_on_hold(rq));
> > GEM_BUG_ON(rq->engine != engine);
> > __execlists_hold(rq);
> > - GEM_BUG_ON(list_empty(&engine->active.hold));
> > + GEM_BUG_ON(list_empty(&engine->sched_engine->hold));
> >
> > unlock:
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > return rq;
> > }
> >
> > @@ -2117,7 +2119,7 @@ static void __execlists_unhold(struct i915_request *rq)
> > static void execlists_unhold(struct intel_engine_cs *engine,
> > struct i915_request *rq)
> > {
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> >
> > /*
> > * Move this request back to the priority queue, and all of its
> > @@ -2130,7 +2132,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> > tasklet_hi_schedule(&engine->execlists.tasklet);
> > }
> >
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > }
> >
> > struct execlists_capture {
> > @@ -2260,13 +2262,13 @@ static void execlists_capture(struct intel_engine_cs *engine)
> > if (!cap)
> > return;
> >
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> > cap->rq = active_context(engine, active_ccid(engine));
> > if (cap->rq) {
> > cap->rq = active_request(cap->rq->context->timeline, cap->rq);
> > cap->rq = i915_request_get_rcu(cap->rq);
> > }
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > if (!cap->rq)
> > goto err_free;
> >
> > @@ -2470,7 +2472,7 @@ static bool ancestor_on_hold(const struct intel_engine_cs *engine,
> > const struct i915_request *rq)
> > {
> > GEM_BUG_ON(i915_request_on_hold(rq));
> > - return !list_empty(&engine->active.hold) && hold_request(rq);
> > + return !list_empty(&engine->sched_engine->hold) && hold_request(rq);
> > }
> >
> > static void execlists_submit_request(struct i915_request *request)
> > @@ -2479,11 +2481,12 @@ static void execlists_submit_request(struct i915_request *request)
> > unsigned long flags;
> >
> > /* Will be called from irq-context when using foreign fences. */
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > if (unlikely(ancestor_on_hold(engine, request))) {
> > RQ_TRACE(request, "ancestor on hold\n");
> > - list_add_tail(&request->sched.link, &engine->active.hold);
> > + list_add_tail(&request->sched.link,
> > + &engine->sched_engine->hold);
> > i915_request_set_hold(request);
> > } else {
> > queue_request(engine, request);
> > @@ -2495,7 +2498,7 @@ static void execlists_submit_request(struct i915_request *request)
> > __execlists_kick(&engine->execlists);
> > }
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static int
> > @@ -2959,9 +2962,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> >
> > /* Push back any incomplete requests for replay after the reset. */
> > rcu_read_lock();
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> > __unwind_incomplete_requests(engine);
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > rcu_read_unlock();
> > }
> >
> > @@ -3001,10 +3004,10 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > execlists_reset_csb(engine, true);
> >
> > rcu_read_lock();
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > /* Mark all executing requests as skipped. */
> > - list_for_each_entry(rq, &engine->active.requests, sched.link)
> > + list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
> > i915_request_put(i915_request_mark_eio(rq));
> > intel_engine_signal_breadcrumbs(engine);
> >
> > @@ -3024,7 +3027,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > }
> >
> > /* On-hold requests will be flushed to timeline upon their release */
> > - list_for_each_entry(rq, &engine->active.hold, sched.link)
> > + list_for_each_entry(rq, &sched_engine->hold, sched.link)
> > i915_request_put(i915_request_mark_eio(rq));
> >
> > /* Cancel all attached virtual engines */
> > @@ -3035,7 +3038,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > rb_erase_cached(rb, &execlists->virtual);
> > RB_CLEAR_NODE(rb);
> >
> > - spin_lock(&ve->base.active.lock);
> > + spin_lock(&ve->base.sched_engine->lock);
> > rq = fetch_and_zero(&ve->request);
> > if (rq) {
> > if (i915_request_mark_eio(rq)) {
> > @@ -3047,7 +3050,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> >
> > ve->base.sched_engine->queue_priority_hint = INT_MIN;
> > }
> > - spin_unlock(&ve->base.active.lock);
> > + spin_unlock(&ve->base.sched_engine->lock);
> > }
> >
> > /* Remaining _unready_ requests will be nop'ed when submitted */
> > @@ -3058,7 +3061,7 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > execlists->tasklet.callback = nop_submission_tasklet;
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > rcu_read_unlock();
> > }
> >
> > @@ -3304,7 +3307,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > if (unlikely(ve->request)) {
> > struct i915_request *old;
> >
> > - spin_lock_irq(&ve->base.active.lock);
> > + spin_lock_irq(&ve->base.sched_engine->lock);
> >
> > old = fetch_and_zero(&ve->request);
> > if (old) {
> > @@ -3313,7 +3316,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > i915_request_put(old);
> > }
> >
> > - spin_unlock_irq(&ve->base.active.lock);
> > + spin_unlock_irq(&ve->base.sched_engine->lock);
> > }
> >
> > /*
> > @@ -3333,13 +3336,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > if (RB_EMPTY_NODE(node))
> > continue;
> >
> > - spin_lock_irq(&sibling->active.lock);
> > + spin_lock_irq(&sibling->sched_engine->lock);
> >
> > /* Detachment is lazily performed in the execlists tasklet */
> > if (!RB_EMPTY_NODE(node))
> > rb_erase_cached(node, &sibling->execlists.virtual);
> >
> > - spin_unlock_irq(&sibling->active.lock);
> > + spin_unlock_irq(&sibling->sched_engine->lock);
> > }
> > GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
> > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > @@ -3509,7 +3512,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > if (!READ_ONCE(ve->request))
> > break; /* already handled by a sibling's tasklet */
> >
> > - spin_lock_irq(&sibling->active.lock);
> > + spin_lock_irq(&sibling->sched_engine->lock);
> >
> > if (unlikely(!(mask & sibling->mask))) {
> > if (!RB_EMPTY_NODE(&node->rb)) {
> > @@ -3562,7 +3565,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > tasklet_hi_schedule(&sibling->execlists.tasklet);
> >
> > unlock_engine:
> > - spin_unlock_irq(&sibling->active.lock);
> > + spin_unlock_irq(&sibling->sched_engine->lock);
> >
> > if (intel_context_inflight(&ve->context))
> > break;
> > @@ -3580,7 +3583,7 @@ static void virtual_submit_request(struct i915_request *rq)
> >
> > GEM_BUG_ON(ve->base.submit_request != virtual_submit_request);
> >
> > - spin_lock_irqsave(&ve->base.active.lock, flags);
> > + spin_lock_irqsave(&ve->base.sched_engine->lock, flags);
> >
> > /* By the time we resubmit a request, it may be completed */
> > if (__i915_request_is_complete(rq)) {
> > @@ -3603,7 +3606,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > tasklet_hi_schedule(&ve->base.execlists.tasklet);
> >
> > unlock:
> > - spin_unlock_irqrestore(&ve->base.active.lock, flags);
> > + spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
> > }
> >
> > static struct ve_bond *
> > @@ -3687,7 +3690,6 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> >
> > snprintf(ve->base.name, sizeof(ve->base.name), "virtual");
> >
> > - intel_engine_init_active(&ve->base, ENGINE_VIRTUAL);
> > intel_engine_init_execlists(&ve->base);
> >
> > ve->base.sched_engine = i915_sched_engine_create(ENGINE_VIRTUAL);
> > @@ -3860,17 +3862,17 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > unsigned int max)
> > {
> > const struct intel_engine_execlists *execlists = &engine->execlists;
> > - const struct i915_sched_engine *sched_engine = engine->sched_engine;
> > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> > struct i915_request *rq, *last;
> > unsigned long flags;
> > unsigned int count;
> > struct rb_node *rb;
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&sched_engine->lock, flags);
> >
> > last = NULL;
> > count = 0;
> > - list_for_each_entry(rq, &engine->active.requests, sched.link) {
> > + list_for_each_entry(rq, &sched_engine->requests, sched.link) {
> > if (count++ < max - 1)
> > show_request(m, rq, "\t\t", 0);
> > else
> > @@ -3933,7 +3935,7 @@ void intel_execlists_show_requests(struct intel_engine_cs *engine,
> > show_request(m, last, "\t\t", 0);
> > }
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&sched_engine->lock, flags);
> > }
> >
> > #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
> > diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> > index 2b6dffcc2262..c98ae0f58790 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
> > @@ -339,9 +339,9 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > u32 head;
> >
> > rq = NULL;
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> > rcu_read_lock();
> > - list_for_each_entry(pos, &engine->active.requests, sched.link) {
> > + list_for_each_entry(pos, &engine->sched_engine->requests, sched.link) {
> > if (!__i915_request_is_complete(pos)) {
> > rq = pos;
> > break;
> > @@ -396,7 +396,7 @@ static void reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > }
> > engine->legacy.ring->head = intel_ring_wrap(engine->legacy.ring, head);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void reset_finish(struct intel_engine_cs *engine)
> > @@ -408,16 +408,16 @@ static void reset_cancel(struct intel_engine_cs *engine)
> > struct i915_request *request;
> > unsigned long flags;
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > /* Mark all submitted requests as skipped. */
> > - list_for_each_entry(request, &engine->active.requests, sched.link)
> > + list_for_each_entry(request, &engine->sched_engine->requests, sched.link)
> > i915_request_put(i915_request_mark_eio(request));
> > intel_engine_signal_breadcrumbs(engine);
> >
> > /* Remaining _unready_ requests will be nop'ed when submitted */
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void i9xx_submit_request(struct i915_request *request)
> > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > index b1fdba13e900..a49fd3039f13 100644
> > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > @@ -253,10 +253,10 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
> >
> > del_timer_sync(&mock->hw_delay);
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > /* Mark all submitted requests as skipped. */
> > - list_for_each_entry(rq, &engine->active.requests, sched.link)
> > + list_for_each_entry(rq, &engine->sched_engine->requests, sched.link)
> > i915_request_put(i915_request_mark_eio(rq));
> > intel_engine_signal_breadcrumbs(engine);
> >
> > @@ -269,7 +269,7 @@ static void mock_reset_cancel(struct intel_engine_cs *engine)
> > }
> > INIT_LIST_HEAD(&mock->hw_queue);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void mock_reset_finish(struct intel_engine_cs *engine)
> > @@ -350,7 +350,6 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > if (!engine->sched_engine)
> > return -ENOMEM;
> >
> > - intel_engine_init_active(engine, ENGINE_MOCK);
> > intel_engine_init_execlists(engine);
> > intel_engine_init__pm(engine);
> > intel_engine_init_retire(engine);
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > index 1f93591a8c69..fd824fa329c2 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > @@ -4599,9 +4599,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
> > GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
> >
> > /* Fake a preemption event; failed of course */
> > - spin_lock_irq(&engine->active.lock);
> > + spin_lock_irq(&engine->sched_engine->lock);
> > __unwind_incomplete_requests(engine);
> > - spin_unlock_irq(&engine->active.lock);
> > + spin_unlock_irq(&engine->sched_engine->lock);
> > GEM_BUG_ON(rq->engine != engine);
> >
> > /* Reset the engine while keeping our active request on hold */
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index f4a6fbfaf82e..2551019ee217 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -190,7 +190,7 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> > bool submit = false;
> > struct rb_node *rb;
> >
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&sched_engine->lock);
> >
> > if (last) {
> > if (*++first)
> > @@ -247,7 +247,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
> > struct i915_request **port, *rq;
> > unsigned long flags;
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > for (port = execlists->inflight; (rq = *port); port++) {
> > if (!i915_request_completed(rq))
> > @@ -265,7 +265,7 @@ static void guc_submission_tasklet(struct tasklet_struct *t)
> >
> > i915_sched_engine_reset_on_empty(engine->sched_engine);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
> > @@ -322,7 +322,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > struct i915_request *rq;
> > unsigned long flags;
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > /* Push back any incomplete requests for replay after the reset. */
> > rq = execlists_unwind_incomplete_requests(execlists);
> > @@ -336,7 +336,7 @@ static void guc_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> > guc_reset_state(rq->context, engine, rq->head, stalled);
> >
> > out_unlock:
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void guc_reset_cancel(struct intel_engine_cs *engine)
> > @@ -362,10 +362,10 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > * submission's irq state, we also wish to remind ourselves that
> > * it is irq state.)
> > */
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&sched_engine->lock, flags);
> >
> > /* Mark all executing requests as skipped. */
> > - list_for_each_entry(rq, &engine->active.requests, sched.link) {
> > + list_for_each_entry(rq, &sched_engine->requests, sched.link) {
> > i915_request_set_error_once(rq, -EIO);
> > i915_request_mark_complete(rq);
> > }
> > @@ -390,7 +390,7 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> > sched_engine->queue_priority_hint = INT_MIN;
> > sched_engine->queue = RB_ROOT_CACHED;
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&sched_engine->lock, flags);
> > }
> >
> > static void guc_reset_finish(struct intel_engine_cs *engine)
> > @@ -539,7 +539,7 @@ static void guc_submit_request(struct i915_request *rq)
> > unsigned long flags;
> >
> > /* Will be called from irq-context when using foreign fences. */
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > queue_request(engine, rq, rq_prio(rq));
> >
> > @@ -548,7 +548,7 @@ static void guc_submit_request(struct i915_request *rq)
> >
> > tasklet_hi_schedule(&engine->execlists.tasklet);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void sanitize_hwsp(struct intel_engine_cs *engine)
> > diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
> > index 833d3e8b7631..cd005f0a203c 100644
> > --- a/drivers/gpu/drm/i915/i915_gpu_error.c
> > +++ b/drivers/gpu/drm/i915/i915_gpu_error.c
> > @@ -1436,12 +1436,12 @@ capture_engine(struct intel_engine_cs *engine,
> > if (!ee)
> > return NULL;
> >
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> > rq = intel_engine_find_active_request(engine);
> > if (rq)
> > capture = intel_engine_coredump_add_request(ee, rq,
> > ATOMIC_MAYFAIL);
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > if (!capture) {
> > kfree(ee);
> > return NULL;
> > diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
> > index 970d8f4986bb..95e859c319ca 100644
> > --- a/drivers/gpu/drm/i915/i915_request.c
> > +++ b/drivers/gpu/drm/i915/i915_request.c
> > @@ -272,11 +272,11 @@ i915_request_active_engine(struct i915_request *rq,
> > * check that we have acquired the lock on the final engine.
> > */
> > locked = READ_ONCE(rq->engine);
> > - spin_lock_irq(&locked->active.lock);
> > + spin_lock_irq(&locked->sched_engine->lock);
> > while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
> > - spin_unlock(&locked->active.lock);
> > + spin_unlock(&locked->sched_engine->lock);
> > locked = engine;
> > - spin_lock(&locked->active.lock);
> > + spin_lock(&locked->sched_engine->lock);
> > }
> >
> > if (i915_request_is_active(rq)) {
> > @@ -285,7 +285,7 @@ i915_request_active_engine(struct i915_request *rq,
> > ret = true;
> > }
> >
> > - spin_unlock_irq(&locked->active.lock);
> > + spin_unlock_irq(&locked->sched_engine->lock);
> >
> > return ret;
> > }
> > @@ -302,10 +302,10 @@ static void remove_from_engine(struct i915_request *rq)
> > * check that the rq still belongs to the newly locked engine.
> > */
> > locked = READ_ONCE(rq->engine);
> > - spin_lock_irq(&locked->active.lock);
> > + spin_lock_irq(&locked->sched_engine->lock);
> > while (unlikely(locked != (engine = READ_ONCE(rq->engine)))) {
> > - spin_unlock(&locked->active.lock);
> > - spin_lock(&engine->active.lock);
> > + spin_unlock(&locked->sched_engine->lock);
> > + spin_lock(&engine->sched_engine->lock);
> > locked = engine;
> > }
> > list_del_init(&rq->sched.link);
> > @@ -316,7 +316,7 @@ static void remove_from_engine(struct i915_request *rq)
> > /* Prevent further __await_execution() registering a cb, then flush */
> > set_bit(I915_FENCE_FLAG_ACTIVE, &rq->fence.flags);
> >
> > - spin_unlock_irq(&locked->active.lock);
> > + spin_unlock_irq(&locked->sched_engine->lock);
> >
> > __notify_execute_cb_imm(rq);
> > }
> > @@ -637,7 +637,7 @@ bool __i915_request_submit(struct i915_request *request)
> > RQ_TRACE(request, "\n");
> >
> > GEM_BUG_ON(!irqs_disabled());
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> >
> > /*
> > * With the advent of preempt-to-busy, we frequently encounter
> > @@ -649,7 +649,7 @@ bool __i915_request_submit(struct i915_request *request)
> > *
> > * We must remove the request from the caller's priority queue,
> > * and the caller must only call us when the request is in their
> > - * priority queue, under the active.lock. This ensures that the
> > + * priority queue, under the sched_engine->lock. This ensures that the
> > * request has *not* yet been retired and we can safely move
> > * the request into the engine->active.list where it will be
> > * dropped upon retiring. (Otherwise if resubmit a *retired*
> > @@ -694,7 +694,7 @@ bool __i915_request_submit(struct i915_request *request)
> > result = true;
> >
> > GEM_BUG_ON(test_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags));
> > - list_move_tail(&request->sched.link, &engine->active.requests);
> > + list_move_tail(&request->sched.link, &engine->sched_engine->requests);
> > active:
> > clear_bit(I915_FENCE_FLAG_PQUEUE, &request->fence.flags);
> > set_bit(I915_FENCE_FLAG_ACTIVE, &request->fence.flags);
> > @@ -724,11 +724,11 @@ void i915_request_submit(struct i915_request *request)
> > unsigned long flags;
> >
> > /* Will be called from irq-context when using foreign fences. */
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > __i915_request_submit(request);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > void __i915_request_unsubmit(struct i915_request *request)
> > @@ -742,7 +742,7 @@ void __i915_request_unsubmit(struct i915_request *request)
> > RQ_TRACE(request, "\n");
> >
> > GEM_BUG_ON(!irqs_disabled());
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> >
> > /*
> > * Before we remove this breadcrumb from the signal list, we have
> > @@ -775,11 +775,11 @@ void i915_request_unsubmit(struct i915_request *request)
> > unsigned long flags;
> >
> > /* Will be called from irq-context when using foreign fences. */
> > - spin_lock_irqsave(&engine->active.lock, flags);
> > + spin_lock_irqsave(&engine->sched_engine->lock, flags);
> >
> > __i915_request_unsubmit(request);
> >
> > - spin_unlock_irqrestore(&engine->active.lock, flags);
> > + spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> >
> > static void __cancel_request(struct i915_request *rq)
> > diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
> > index 270f6cd37650..239964bec1fa 100644
> > --- a/drivers/gpu/drm/i915/i915_request.h
> > +++ b/drivers/gpu/drm/i915/i915_request.h
> > @@ -613,7 +613,7 @@ i915_request_active_timeline(const struct i915_request *rq)
> > * this submission.
> > */
> > return rcu_dereference_protected(rq->timeline,
> > - lockdep_is_held(&rq->engine->active.lock));
> > + lockdep_is_held(&rq->engine->sched_engine->lock));
> > }
> >
> > static inline u32
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > index 4953874a9ba6..4bc6969f6a97 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > @@ -68,7 +68,7 @@ i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > struct rb_node **parent, *rb;
> > bool first = true;
> >
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> > assert_priolists(sched_engine);
> >
> > if (unlikely(sched_engine->no_priolist))
> > @@ -147,9 +147,9 @@ sched_lock_engine(const struct i915_sched_node *node,
> > * check that the rq still belongs to the newly locked engine.
> > */
> > while (locked != (engine = READ_ONCE(rq->engine))) {
> > - spin_unlock(&locked->active.lock);
> > + spin_unlock(&locked->sched_engine->lock);
> > memset(cache, 0, sizeof(*cache));
> > - spin_lock(&engine->active.lock);
> > + spin_lock(&engine->sched_engine->lock);
> > locked = engine;
> > }
> >
> > @@ -296,7 +296,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> >
> > memset(&cache, 0, sizeof(cache));
> > engine = node_to_request(node)->engine;
> > - spin_lock(&engine->active.lock);
> > + spin_lock(&engine->sched_engine->lock);
> >
> > /* Fifo and depth-first replacement ensure our deps execute before us */
> > engine = sched_lock_engine(node, engine, &cache);
> > @@ -305,7 +305,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> >
> > node = dep->signaler;
> > engine = sched_lock_engine(node, engine, &cache);
> > - lockdep_assert_held(&engine->active.lock);
> > + lockdep_assert_held(&engine->sched_engine->lock);
> >
> > /* Recheck after acquiring the engine->timeline.lock */
> > if (prio <= node->attr.priority || node_signaled(node))
> > @@ -338,7 +338,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> > kick_submission(engine, node_to_request(node), prio);
> > }
> >
> > - spin_unlock(&engine->active.lock);
> > + spin_unlock(&engine->sched_engine->lock);
> > }
> >
> > void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
> > @@ -511,6 +511,24 @@ i915_sched_engine_create(unsigned int subclass)
> > sched_engine->queue = RB_ROOT_CACHED;
> > sched_engine->queue_priority_hint = INT_MIN;
> >
> > + INIT_LIST_HEAD(&sched_engine->requests);
> > + INIT_LIST_HEAD(&sched_engine->hold);
> > +
> > + spin_lock_init(&sched_engine->lock);
> > + lockdep_set_subclass(&sched_engine->lock, subclass);
> > +
> > + /*
> > + * Due to an interesting quirk in lockdep's internal debug tracking,
> > + * after setting a subclass we must ensure the lock is used. Otherwise,
> > + * nr_unused_locks is incremented once too often.
> > + */
> > +#ifdef CONFIG_DEBUG_LOCK_ALLOC
> > + local_irq_disable();
> > + lock_map_acquire(&sched_engine->lock.dep_map);
> > + lock_map_release(&sched_engine->lock.dep_map);
> > + local_irq_enable();
> > +#endif
> > +
> > return sched_engine;
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 16a054e67760..5cb0b1c98d6b 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > @@ -94,6 +94,15 @@ struct i915_dependency {
> > struct i915_sched_engine {
> > struct kref ref;
> >
> > + /**
> > + * @lock: protects requests in priority lists, requests, hold and
> > + * tasklet while running
>
> To be clear, this protects other stuff in this struct, right? Not
> just requests and hold?
>
Yea, this lock is kinda mess. It protects the request when it is in the
priority list (struct i915_priolist) and these lists. It also is the
submission lock, e.g. you have to hold this lock when submitting /
unsubmitting the request. It also is used for some timeline RCU thing I
don't fully understand.
In the GuC submission code this lock is used as a submission lock for
consistency with the execlist code.
Once we go the DRM scheduler these lists go away, at least with GuC
submission. Also a submission lock really isn't needed as the backend
will have a single entry point rather and only 1 request is allowed in
the backend at time. Hoping we can push this lock + all of these
structures in an execlist specific structure + only use it in
intel_execlists_submission.c and be done with it.
> Otherwise, this looks mechanically correct. Still not getting the big
Anyways, there is not a change in behavior here.
Matt
> picture yet.
>
> --Jason
>
>
> > + */
> > + spinlock_t lock;
> > +
> > + struct list_head requests;
> > + struct list_head hold; /* ready requests, but on hold */
> > +
> > /**
> > * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > */
> > --
> > 2.28.0
> >
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 7/9] drm/i915: Update i915_scheduler to operate on i915_sched_engine
2021-06-04 19:17 ` Jason Ekstrand
@ 2021-06-04 19:14 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 19:14 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 02:17:58PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Rather passing around an intel_engine_cs in the scheduling code, pass
> > around a i915_sched_engine.
>
> 👍
>
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > .../drm/i915/gt/intel_execlists_submission.c | 11 +++--
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
> > drivers/gpu/drm/i915/i915_scheduler.c | 46 +++++++++----------
> > drivers/gpu/drm/i915/i915_scheduler.h | 2 +-
> > 4 files changed, 32 insertions(+), 29 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 3fac17103837..7240c153be35 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -382,7 +382,8 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > GEM_BUG_ON(rq_prio(rq) == I915_PRIORITY_INVALID);
> > if (rq_prio(rq) != prio) {
> > prio = rq_prio(rq);
> > - pl = i915_sched_lookup_priolist(engine, prio);
> > + pl = i915_sched_lookup_priolist(engine->sched_engine,
> > + prio);
> > }
> > GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
> >
> > @@ -1096,7 +1097,8 @@ static void defer_active(struct intel_engine_cs *engine)
> > if (!rq)
> > return;
> >
> > - defer_request(rq, i915_sched_lookup_priolist(engine, rq_prio(rq)));
> > + defer_request(rq, i915_sched_lookup_priolist(engine->sched_engine,
> > + rq_prio(rq)));
> > }
> >
> > static bool
> > @@ -2083,7 +2085,7 @@ static void __execlists_unhold(struct i915_request *rq)
> >
> > i915_request_clear_hold(rq);
> > list_move_tail(&rq->sched.link,
> > - i915_sched_lookup_priolist(rq->engine,
> > + i915_sched_lookup_priolist(rq->engine->sched_engine,
> > rq_prio(rq)));
> > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> >
> > @@ -2452,7 +2454,8 @@ static void queue_request(struct intel_engine_cs *engine,
> > {
> > GEM_BUG_ON(!list_empty(&rq->sched.link));
> > list_add_tail(&rq->sched.link,
> > - i915_sched_lookup_priolist(engine, rq_prio(rq)));
> > + i915_sched_lookup_priolist(engine->sched_engine,
> > + rq_prio(rq)));
> > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 4c5bbec0775d..7ed21670ef14 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -529,7 +529,7 @@ static inline void queue_request(struct intel_engine_cs *engine,
> > {
> > GEM_BUG_ON(!list_empty(&rq->sched.link));
> > list_add_tail(&rq->sched.link,
> > - i915_sched_lookup_priolist(engine, prio));
> > + i915_sched_lookup_priolist(engine->sched_engine, prio));
> > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > index 035b88f2e4aa..3d36e4447b5d 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > @@ -61,14 +61,13 @@ static void assert_priolists(struct i915_sched_engine * const sched_engine)
> > }
> >
> > struct list_head *
> > -i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > +i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio)
> > {
> > - struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_priolist *p;
> > struct rb_node **parent, *rb;
> > bool first = true;
> >
> > - lockdep_assert_held(&engine->sched_engine->lock);
> > + lockdep_assert_held(&sched_engine->lock);
> > assert_priolists(sched_engine);
> >
> > if (unlikely(sched_engine->no_priolist))
> > @@ -130,13 +129,13 @@ struct sched_cache {
> > struct list_head *priolist;
> > };
> >
> > -static struct intel_engine_cs *
> > -sched_lock_engine(const struct i915_sched_node *node,
> > - struct intel_engine_cs *locked,
> > +static struct i915_sched_engine *
> > +lock_sched_engine(struct i915_sched_node *node,
> > + struct i915_sched_engine *locked,
> > struct sched_cache *cache)
> > {
> > const struct i915_request *rq = node_to_request(node);
> > - struct intel_engine_cs *engine;
> > + struct i915_sched_engine *sched_engine;
> >
> > GEM_BUG_ON(!locked);
> >
> > @@ -146,14 +145,14 @@ sched_lock_engine(const struct i915_sched_node *node,
> > * engine lock. The simple ploy we use is to take the lock then
> > * check that the rq still belongs to the newly locked engine.
> > */
> > - while (locked != (engine = READ_ONCE(rq->engine))) {
> > - spin_unlock(&locked->sched_engine->lock);
> > + while (locked != (sched_engine = rq->engine->sched_engine)) {
>
> You lost the READ_ONCE here. Based on the comment above, that may
> matter. I guess it depends on what all barriers spin_lock() implies.
>
Ah, good catch. I think it should be:
READ_ONCE(rq->engine)->sched_engine
As the rq->engine is the unstable pointer.
Matt
> Assuming that's all good,
>
> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
>
> > + spin_unlock(&locked->lock);
> > memset(cache, 0, sizeof(*cache));
> > - spin_lock(&engine->sched_engine->lock);
> > - locked = engine;
> > + spin_lock(&sched_engine->lock);
> > + locked = sched_engine;
> > }
> >
> > - GEM_BUG_ON(locked != engine);
> > + GEM_BUG_ON(locked != sched_engine);
> > return locked;
> > }
> >
> > @@ -161,7 +160,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> > const struct i915_sched_attr *attr)
> > {
> > const int prio = max(attr->priority, node->attr.priority);
> > - struct intel_engine_cs *engine;
> > + struct i915_sched_engine *sched_engine;
> > struct i915_dependency *dep, *p;
> > struct i915_dependency stack;
> > struct sched_cache cache;
> > @@ -236,23 +235,24 @@ static void __i915_schedule(struct i915_sched_node *node,
> > }
> >
> > memset(&cache, 0, sizeof(cache));
> > - engine = node_to_request(node)->engine;
> > - spin_lock(&engine->sched_engine->lock);
> > + sched_engine = node_to_request(node)->engine->sched_engine;
> > + spin_lock(&sched_engine->lock);
> >
> > /* Fifo and depth-first replacement ensure our deps execute before us */
> > - engine = sched_lock_engine(node, engine, &cache);
> > + sched_engine = lock_sched_engine(node, sched_engine, &cache);
> > list_for_each_entry_safe_reverse(dep, p, &dfs, dfs_link) {
> > INIT_LIST_HEAD(&dep->dfs_link);
> >
> > node = dep->signaler;
> > - engine = sched_lock_engine(node, engine, &cache);
> > - lockdep_assert_held(&engine->sched_engine->lock);
> > + sched_engine = lock_sched_engine(node, sched_engine, &cache);
> > + lockdep_assert_held(&sched_engine->lock);
> >
> > /* Recheck after acquiring the engine->timeline.lock */
> > if (prio <= node->attr.priority || node_signaled(node))
> > continue;
> >
> > - GEM_BUG_ON(node_to_request(node)->engine != engine);
> > + GEM_BUG_ON(node_to_request(node)->engine->sched_engine !=
> > + sched_engine);
> >
> > WRITE_ONCE(node->attr.priority, prio);
> >
> > @@ -270,17 +270,17 @@ static void __i915_schedule(struct i915_sched_node *node,
> > if (i915_request_in_priority_queue(node_to_request(node))) {
> > if (!cache.priolist)
> > cache.priolist =
> > - i915_sched_lookup_priolist(engine,
> > + i915_sched_lookup_priolist(sched_engine,
> > prio);
> > list_move_tail(&node->link, cache.priolist);
> > }
> >
> > /* Defer (tasklet) submission until after all of our updates. */
> > - if (engine->sched_engine->kick_backend)
> > - engine->sched_engine->kick_backend(node_to_request(node), prio);
> > + if (sched_engine->kick_backend)
> > + sched_engine->kick_backend(node_to_request(node), prio);
> > }
> >
> > - spin_unlock(&engine->sched_engine->lock);
> > + spin_unlock(&sched_engine->lock);
> > }
> >
> > void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > index 713c38c99de9..0014745bda30 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > @@ -39,7 +39,7 @@ void i915_schedule(struct i915_request *request,
> > const struct i915_sched_attr *attr);
> >
> > struct list_head *
> > -i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio);
> > +i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio);
> >
> > void __i915_priolist_free(struct i915_priolist *p);
> > static inline void i915_priolist_free(struct i915_priolist *p)
> > --
> > 2.28.0
> >
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 7/9] drm/i915: Update i915_scheduler to operate on i915_sched_engine
@ 2021-06-04 19:14 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 19:14 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 02:17:58PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Rather passing around an intel_engine_cs in the scheduling code, pass
> > around a i915_sched_engine.
>
> 👍
>
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > .../drm/i915/gt/intel_execlists_submission.c | 11 +++--
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
> > drivers/gpu/drm/i915/i915_scheduler.c | 46 +++++++++----------
> > drivers/gpu/drm/i915/i915_scheduler.h | 2 +-
> > 4 files changed, 32 insertions(+), 29 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 3fac17103837..7240c153be35 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -382,7 +382,8 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> > GEM_BUG_ON(rq_prio(rq) == I915_PRIORITY_INVALID);
> > if (rq_prio(rq) != prio) {
> > prio = rq_prio(rq);
> > - pl = i915_sched_lookup_priolist(engine, prio);
> > + pl = i915_sched_lookup_priolist(engine->sched_engine,
> > + prio);
> > }
> > GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
> >
> > @@ -1096,7 +1097,8 @@ static void defer_active(struct intel_engine_cs *engine)
> > if (!rq)
> > return;
> >
> > - defer_request(rq, i915_sched_lookup_priolist(engine, rq_prio(rq)));
> > + defer_request(rq, i915_sched_lookup_priolist(engine->sched_engine,
> > + rq_prio(rq)));
> > }
> >
> > static bool
> > @@ -2083,7 +2085,7 @@ static void __execlists_unhold(struct i915_request *rq)
> >
> > i915_request_clear_hold(rq);
> > list_move_tail(&rq->sched.link,
> > - i915_sched_lookup_priolist(rq->engine,
> > + i915_sched_lookup_priolist(rq->engine->sched_engine,
> > rq_prio(rq)));
> > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> >
> > @@ -2452,7 +2454,8 @@ static void queue_request(struct intel_engine_cs *engine,
> > {
> > GEM_BUG_ON(!list_empty(&rq->sched.link));
> > list_add_tail(&rq->sched.link,
> > - i915_sched_lookup_priolist(engine, rq_prio(rq)));
> > + i915_sched_lookup_priolist(engine->sched_engine,
> > + rq_prio(rq)));
> > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 4c5bbec0775d..7ed21670ef14 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -529,7 +529,7 @@ static inline void queue_request(struct intel_engine_cs *engine,
> > {
> > GEM_BUG_ON(!list_empty(&rq->sched.link));
> > list_add_tail(&rq->sched.link,
> > - i915_sched_lookup_priolist(engine, prio));
> > + i915_sched_lookup_priolist(engine->sched_engine, prio));
> > set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > index 035b88f2e4aa..3d36e4447b5d 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > @@ -61,14 +61,13 @@ static void assert_priolists(struct i915_sched_engine * const sched_engine)
> > }
> >
> > struct list_head *
> > -i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> > +i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio)
> > {
> > - struct i915_sched_engine * const sched_engine = engine->sched_engine;
> > struct i915_priolist *p;
> > struct rb_node **parent, *rb;
> > bool first = true;
> >
> > - lockdep_assert_held(&engine->sched_engine->lock);
> > + lockdep_assert_held(&sched_engine->lock);
> > assert_priolists(sched_engine);
> >
> > if (unlikely(sched_engine->no_priolist))
> > @@ -130,13 +129,13 @@ struct sched_cache {
> > struct list_head *priolist;
> > };
> >
> > -static struct intel_engine_cs *
> > -sched_lock_engine(const struct i915_sched_node *node,
> > - struct intel_engine_cs *locked,
> > +static struct i915_sched_engine *
> > +lock_sched_engine(struct i915_sched_node *node,
> > + struct i915_sched_engine *locked,
> > struct sched_cache *cache)
> > {
> > const struct i915_request *rq = node_to_request(node);
> > - struct intel_engine_cs *engine;
> > + struct i915_sched_engine *sched_engine;
> >
> > GEM_BUG_ON(!locked);
> >
> > @@ -146,14 +145,14 @@ sched_lock_engine(const struct i915_sched_node *node,
> > * engine lock. The simple ploy we use is to take the lock then
> > * check that the rq still belongs to the newly locked engine.
> > */
> > - while (locked != (engine = READ_ONCE(rq->engine))) {
> > - spin_unlock(&locked->sched_engine->lock);
> > + while (locked != (sched_engine = rq->engine->sched_engine)) {
>
> You lost the READ_ONCE here. Based on the comment above, that may
> matter. I guess it depends on what all barriers spin_lock() implies.
>
Ah, good catch. I think it should be:
READ_ONCE(rq->engine)->sched_engine
As the rq->engine is the unstable pointer.
Matt
> Assuming that's all good,
>
> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
>
> > + spin_unlock(&locked->lock);
> > memset(cache, 0, sizeof(*cache));
> > - spin_lock(&engine->sched_engine->lock);
> > - locked = engine;
> > + spin_lock(&sched_engine->lock);
> > + locked = sched_engine;
> > }
> >
> > - GEM_BUG_ON(locked != engine);
> > + GEM_BUG_ON(locked != sched_engine);
> > return locked;
> > }
> >
> > @@ -161,7 +160,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> > const struct i915_sched_attr *attr)
> > {
> > const int prio = max(attr->priority, node->attr.priority);
> > - struct intel_engine_cs *engine;
> > + struct i915_sched_engine *sched_engine;
> > struct i915_dependency *dep, *p;
> > struct i915_dependency stack;
> > struct sched_cache cache;
> > @@ -236,23 +235,24 @@ static void __i915_schedule(struct i915_sched_node *node,
> > }
> >
> > memset(&cache, 0, sizeof(cache));
> > - engine = node_to_request(node)->engine;
> > - spin_lock(&engine->sched_engine->lock);
> > + sched_engine = node_to_request(node)->engine->sched_engine;
> > + spin_lock(&sched_engine->lock);
> >
> > /* Fifo and depth-first replacement ensure our deps execute before us */
> > - engine = sched_lock_engine(node, engine, &cache);
> > + sched_engine = lock_sched_engine(node, sched_engine, &cache);
> > list_for_each_entry_safe_reverse(dep, p, &dfs, dfs_link) {
> > INIT_LIST_HEAD(&dep->dfs_link);
> >
> > node = dep->signaler;
> > - engine = sched_lock_engine(node, engine, &cache);
> > - lockdep_assert_held(&engine->sched_engine->lock);
> > + sched_engine = lock_sched_engine(node, sched_engine, &cache);
> > + lockdep_assert_held(&sched_engine->lock);
> >
> > /* Recheck after acquiring the engine->timeline.lock */
> > if (prio <= node->attr.priority || node_signaled(node))
> > continue;
> >
> > - GEM_BUG_ON(node_to_request(node)->engine != engine);
> > + GEM_BUG_ON(node_to_request(node)->engine->sched_engine !=
> > + sched_engine);
> >
> > WRITE_ONCE(node->attr.priority, prio);
> >
> > @@ -270,17 +270,17 @@ static void __i915_schedule(struct i915_sched_node *node,
> > if (i915_request_in_priority_queue(node_to_request(node))) {
> > if (!cache.priolist)
> > cache.priolist =
> > - i915_sched_lookup_priolist(engine,
> > + i915_sched_lookup_priolist(sched_engine,
> > prio);
> > list_move_tail(&node->link, cache.priolist);
> > }
> >
> > /* Defer (tasklet) submission until after all of our updates. */
> > - if (engine->sched_engine->kick_backend)
> > - engine->sched_engine->kick_backend(node_to_request(node), prio);
> > + if (sched_engine->kick_backend)
> > + sched_engine->kick_backend(node_to_request(node), prio);
> > }
> >
> > - spin_unlock(&engine->sched_engine->lock);
> > + spin_unlock(&sched_engine->lock);
> > }
> >
> > void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > index 713c38c99de9..0014745bda30 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > @@ -39,7 +39,7 @@ void i915_schedule(struct i915_request *request,
> > const struct i915_sched_attr *attr);
> >
> > struct list_head *
> > -i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio);
> > +i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio);
> >
> > void __i915_priolist_free(struct i915_priolist *p);
> > static inline void i915_priolist_free(struct i915_priolist *p)
> > --
> > 2.28.0
> >
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 7/9] drm/i915: Update i915_scheduler to operate on i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 19:17 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 19:17 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Rather passing around an intel_engine_cs in the scheduling code, pass
> around a i915_sched_engine.
👍
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> .../drm/i915/gt/intel_execlists_submission.c | 11 +++--
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
> drivers/gpu/drm/i915/i915_scheduler.c | 46 +++++++++----------
> drivers/gpu/drm/i915/i915_scheduler.h | 2 +-
> 4 files changed, 32 insertions(+), 29 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 3fac17103837..7240c153be35 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -382,7 +382,8 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> GEM_BUG_ON(rq_prio(rq) == I915_PRIORITY_INVALID);
> if (rq_prio(rq) != prio) {
> prio = rq_prio(rq);
> - pl = i915_sched_lookup_priolist(engine, prio);
> + pl = i915_sched_lookup_priolist(engine->sched_engine,
> + prio);
> }
> GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
>
> @@ -1096,7 +1097,8 @@ static void defer_active(struct intel_engine_cs *engine)
> if (!rq)
> return;
>
> - defer_request(rq, i915_sched_lookup_priolist(engine, rq_prio(rq)));
> + defer_request(rq, i915_sched_lookup_priolist(engine->sched_engine,
> + rq_prio(rq)));
> }
>
> static bool
> @@ -2083,7 +2085,7 @@ static void __execlists_unhold(struct i915_request *rq)
>
> i915_request_clear_hold(rq);
> list_move_tail(&rq->sched.link,
> - i915_sched_lookup_priolist(rq->engine,
> + i915_sched_lookup_priolist(rq->engine->sched_engine,
> rq_prio(rq)));
> set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
>
> @@ -2452,7 +2454,8 @@ static void queue_request(struct intel_engine_cs *engine,
> {
> GEM_BUG_ON(!list_empty(&rq->sched.link));
> list_add_tail(&rq->sched.link,
> - i915_sched_lookup_priolist(engine, rq_prio(rq)));
> + i915_sched_lookup_priolist(engine->sched_engine,
> + rq_prio(rq)));
> set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> }
>
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 4c5bbec0775d..7ed21670ef14 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -529,7 +529,7 @@ static inline void queue_request(struct intel_engine_cs *engine,
> {
> GEM_BUG_ON(!list_empty(&rq->sched.link));
> list_add_tail(&rq->sched.link,
> - i915_sched_lookup_priolist(engine, prio));
> + i915_sched_lookup_priolist(engine->sched_engine, prio));
> set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> }
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 035b88f2e4aa..3d36e4447b5d 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -61,14 +61,13 @@ static void assert_priolists(struct i915_sched_engine * const sched_engine)
> }
>
> struct list_head *
> -i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> +i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio)
> {
> - struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_priolist *p;
> struct rb_node **parent, *rb;
> bool first = true;
>
> - lockdep_assert_held(&engine->sched_engine->lock);
> + lockdep_assert_held(&sched_engine->lock);
> assert_priolists(sched_engine);
>
> if (unlikely(sched_engine->no_priolist))
> @@ -130,13 +129,13 @@ struct sched_cache {
> struct list_head *priolist;
> };
>
> -static struct intel_engine_cs *
> -sched_lock_engine(const struct i915_sched_node *node,
> - struct intel_engine_cs *locked,
> +static struct i915_sched_engine *
> +lock_sched_engine(struct i915_sched_node *node,
> + struct i915_sched_engine *locked,
> struct sched_cache *cache)
> {
> const struct i915_request *rq = node_to_request(node);
> - struct intel_engine_cs *engine;
> + struct i915_sched_engine *sched_engine;
>
> GEM_BUG_ON(!locked);
>
> @@ -146,14 +145,14 @@ sched_lock_engine(const struct i915_sched_node *node,
> * engine lock. The simple ploy we use is to take the lock then
> * check that the rq still belongs to the newly locked engine.
> */
> - while (locked != (engine = READ_ONCE(rq->engine))) {
> - spin_unlock(&locked->sched_engine->lock);
> + while (locked != (sched_engine = rq->engine->sched_engine)) {
You lost the READ_ONCE here. Based on the comment above, that may
matter. I guess it depends on what all barriers spin_lock() implies.
Assuming that's all good,
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
> + spin_unlock(&locked->lock);
> memset(cache, 0, sizeof(*cache));
> - spin_lock(&engine->sched_engine->lock);
> - locked = engine;
> + spin_lock(&sched_engine->lock);
> + locked = sched_engine;
> }
>
> - GEM_BUG_ON(locked != engine);
> + GEM_BUG_ON(locked != sched_engine);
> return locked;
> }
>
> @@ -161,7 +160,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> const struct i915_sched_attr *attr)
> {
> const int prio = max(attr->priority, node->attr.priority);
> - struct intel_engine_cs *engine;
> + struct i915_sched_engine *sched_engine;
> struct i915_dependency *dep, *p;
> struct i915_dependency stack;
> struct sched_cache cache;
> @@ -236,23 +235,24 @@ static void __i915_schedule(struct i915_sched_node *node,
> }
>
> memset(&cache, 0, sizeof(cache));
> - engine = node_to_request(node)->engine;
> - spin_lock(&engine->sched_engine->lock);
> + sched_engine = node_to_request(node)->engine->sched_engine;
> + spin_lock(&sched_engine->lock);
>
> /* Fifo and depth-first replacement ensure our deps execute before us */
> - engine = sched_lock_engine(node, engine, &cache);
> + sched_engine = lock_sched_engine(node, sched_engine, &cache);
> list_for_each_entry_safe_reverse(dep, p, &dfs, dfs_link) {
> INIT_LIST_HEAD(&dep->dfs_link);
>
> node = dep->signaler;
> - engine = sched_lock_engine(node, engine, &cache);
> - lockdep_assert_held(&engine->sched_engine->lock);
> + sched_engine = lock_sched_engine(node, sched_engine, &cache);
> + lockdep_assert_held(&sched_engine->lock);
>
> /* Recheck after acquiring the engine->timeline.lock */
> if (prio <= node->attr.priority || node_signaled(node))
> continue;
>
> - GEM_BUG_ON(node_to_request(node)->engine != engine);
> + GEM_BUG_ON(node_to_request(node)->engine->sched_engine !=
> + sched_engine);
>
> WRITE_ONCE(node->attr.priority, prio);
>
> @@ -270,17 +270,17 @@ static void __i915_schedule(struct i915_sched_node *node,
> if (i915_request_in_priority_queue(node_to_request(node))) {
> if (!cache.priolist)
> cache.priolist =
> - i915_sched_lookup_priolist(engine,
> + i915_sched_lookup_priolist(sched_engine,
> prio);
> list_move_tail(&node->link, cache.priolist);
> }
>
> /* Defer (tasklet) submission until after all of our updates. */
> - if (engine->sched_engine->kick_backend)
> - engine->sched_engine->kick_backend(node_to_request(node), prio);
> + if (sched_engine->kick_backend)
> + sched_engine->kick_backend(node_to_request(node), prio);
> }
>
> - spin_unlock(&engine->sched_engine->lock);
> + spin_unlock(&sched_engine->lock);
> }
>
> void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 713c38c99de9..0014745bda30 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -39,7 +39,7 @@ void i915_schedule(struct i915_request *request,
> const struct i915_sched_attr *attr);
>
> struct list_head *
> -i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio);
> +i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio);
>
> void __i915_priolist_free(struct i915_priolist *p);
> static inline void i915_priolist_free(struct i915_priolist *p)
> --
> 2.28.0
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 7/9] drm/i915: Update i915_scheduler to operate on i915_sched_engine
@ 2021-06-04 19:17 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 19:17 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> Rather passing around an intel_engine_cs in the scheduling code, pass
> around a i915_sched_engine.
👍
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> .../drm/i915/gt/intel_execlists_submission.c | 11 +++--
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
> drivers/gpu/drm/i915/i915_scheduler.c | 46 +++++++++----------
> drivers/gpu/drm/i915/i915_scheduler.h | 2 +-
> 4 files changed, 32 insertions(+), 29 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 3fac17103837..7240c153be35 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -382,7 +382,8 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine)
> GEM_BUG_ON(rq_prio(rq) == I915_PRIORITY_INVALID);
> if (rq_prio(rq) != prio) {
> prio = rq_prio(rq);
> - pl = i915_sched_lookup_priolist(engine, prio);
> + pl = i915_sched_lookup_priolist(engine->sched_engine,
> + prio);
> }
> GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
>
> @@ -1096,7 +1097,8 @@ static void defer_active(struct intel_engine_cs *engine)
> if (!rq)
> return;
>
> - defer_request(rq, i915_sched_lookup_priolist(engine, rq_prio(rq)));
> + defer_request(rq, i915_sched_lookup_priolist(engine->sched_engine,
> + rq_prio(rq)));
> }
>
> static bool
> @@ -2083,7 +2085,7 @@ static void __execlists_unhold(struct i915_request *rq)
>
> i915_request_clear_hold(rq);
> list_move_tail(&rq->sched.link,
> - i915_sched_lookup_priolist(rq->engine,
> + i915_sched_lookup_priolist(rq->engine->sched_engine,
> rq_prio(rq)));
> set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
>
> @@ -2452,7 +2454,8 @@ static void queue_request(struct intel_engine_cs *engine,
> {
> GEM_BUG_ON(!list_empty(&rq->sched.link));
> list_add_tail(&rq->sched.link,
> - i915_sched_lookup_priolist(engine, rq_prio(rq)));
> + i915_sched_lookup_priolist(engine->sched_engine,
> + rq_prio(rq)));
> set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> }
>
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 4c5bbec0775d..7ed21670ef14 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -529,7 +529,7 @@ static inline void queue_request(struct intel_engine_cs *engine,
> {
> GEM_BUG_ON(!list_empty(&rq->sched.link));
> list_add_tail(&rq->sched.link,
> - i915_sched_lookup_priolist(engine, prio));
> + i915_sched_lookup_priolist(engine->sched_engine, prio));
> set_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
> }
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 035b88f2e4aa..3d36e4447b5d 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -61,14 +61,13 @@ static void assert_priolists(struct i915_sched_engine * const sched_engine)
> }
>
> struct list_head *
> -i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio)
> +i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio)
> {
> - struct i915_sched_engine * const sched_engine = engine->sched_engine;
> struct i915_priolist *p;
> struct rb_node **parent, *rb;
> bool first = true;
>
> - lockdep_assert_held(&engine->sched_engine->lock);
> + lockdep_assert_held(&sched_engine->lock);
> assert_priolists(sched_engine);
>
> if (unlikely(sched_engine->no_priolist))
> @@ -130,13 +129,13 @@ struct sched_cache {
> struct list_head *priolist;
> };
>
> -static struct intel_engine_cs *
> -sched_lock_engine(const struct i915_sched_node *node,
> - struct intel_engine_cs *locked,
> +static struct i915_sched_engine *
> +lock_sched_engine(struct i915_sched_node *node,
> + struct i915_sched_engine *locked,
> struct sched_cache *cache)
> {
> const struct i915_request *rq = node_to_request(node);
> - struct intel_engine_cs *engine;
> + struct i915_sched_engine *sched_engine;
>
> GEM_BUG_ON(!locked);
>
> @@ -146,14 +145,14 @@ sched_lock_engine(const struct i915_sched_node *node,
> * engine lock. The simple ploy we use is to take the lock then
> * check that the rq still belongs to the newly locked engine.
> */
> - while (locked != (engine = READ_ONCE(rq->engine))) {
> - spin_unlock(&locked->sched_engine->lock);
> + while (locked != (sched_engine = rq->engine->sched_engine)) {
You lost the READ_ONCE here. Based on the comment above, that may
matter. I guess it depends on what all barriers spin_lock() implies.
Assuming that's all good,
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
> + spin_unlock(&locked->lock);
> memset(cache, 0, sizeof(*cache));
> - spin_lock(&engine->sched_engine->lock);
> - locked = engine;
> + spin_lock(&sched_engine->lock);
> + locked = sched_engine;
> }
>
> - GEM_BUG_ON(locked != engine);
> + GEM_BUG_ON(locked != sched_engine);
> return locked;
> }
>
> @@ -161,7 +160,7 @@ static void __i915_schedule(struct i915_sched_node *node,
> const struct i915_sched_attr *attr)
> {
> const int prio = max(attr->priority, node->attr.priority);
> - struct intel_engine_cs *engine;
> + struct i915_sched_engine *sched_engine;
> struct i915_dependency *dep, *p;
> struct i915_dependency stack;
> struct sched_cache cache;
> @@ -236,23 +235,24 @@ static void __i915_schedule(struct i915_sched_node *node,
> }
>
> memset(&cache, 0, sizeof(cache));
> - engine = node_to_request(node)->engine;
> - spin_lock(&engine->sched_engine->lock);
> + sched_engine = node_to_request(node)->engine->sched_engine;
> + spin_lock(&sched_engine->lock);
>
> /* Fifo and depth-first replacement ensure our deps execute before us */
> - engine = sched_lock_engine(node, engine, &cache);
> + sched_engine = lock_sched_engine(node, sched_engine, &cache);
> list_for_each_entry_safe_reverse(dep, p, &dfs, dfs_link) {
> INIT_LIST_HEAD(&dep->dfs_link);
>
> node = dep->signaler;
> - engine = sched_lock_engine(node, engine, &cache);
> - lockdep_assert_held(&engine->sched_engine->lock);
> + sched_engine = lock_sched_engine(node, sched_engine, &cache);
> + lockdep_assert_held(&sched_engine->lock);
>
> /* Recheck after acquiring the engine->timeline.lock */
> if (prio <= node->attr.priority || node_signaled(node))
> continue;
>
> - GEM_BUG_ON(node_to_request(node)->engine != engine);
> + GEM_BUG_ON(node_to_request(node)->engine->sched_engine !=
> + sched_engine);
>
> WRITE_ONCE(node->attr.priority, prio);
>
> @@ -270,17 +270,17 @@ static void __i915_schedule(struct i915_sched_node *node,
> if (i915_request_in_priority_queue(node_to_request(node))) {
> if (!cache.priolist)
> cache.priolist =
> - i915_sched_lookup_priolist(engine,
> + i915_sched_lookup_priolist(sched_engine,
> prio);
> list_move_tail(&node->link, cache.priolist);
> }
>
> /* Defer (tasklet) submission until after all of our updates. */
> - if (engine->sched_engine->kick_backend)
> - engine->sched_engine->kick_backend(node_to_request(node), prio);
> + if (sched_engine->kick_backend)
> + sched_engine->kick_backend(node_to_request(node), prio);
> }
>
> - spin_unlock(&engine->sched_engine->lock);
> + spin_unlock(&sched_engine->lock);
> }
>
> void i915_schedule(struct i915_request *rq, const struct i915_sched_attr *attr)
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 713c38c99de9..0014745bda30 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -39,7 +39,7 @@ void i915_schedule(struct i915_request *request,
> const struct i915_sched_attr *attr);
>
> struct list_head *
> -i915_sched_lookup_priolist(struct intel_engine_cs *engine, int prio);
> +i915_sched_lookup_priolist(struct i915_sched_engine *sched_engine, int prio);
>
> void __i915_priolist_free(struct i915_priolist *p);
> static inline void i915_priolist_free(struct i915_priolist *p)
> --
> 2.28.0
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 6/9] drm/i915: Add kick_backend function to i915_sched_engine
2021-06-04 19:09 ` [Intel-gfx] " Jason Ekstrand
@ 2021-06-04 19:19 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 19:19 UTC (permalink / raw)
To: Jason Ekstrand
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Fri, Jun 04, 2021 at 02:09:46PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Rather than touching execlist specific structures in the generic
> > scheduling code, add a callback function in the backend.
>
> Seems reasonable but why does the function that's there today do
> nothing for the ringbuffer and current GuC back-ends? I'm sure
> there's some reason but it's not immediately obvious to me.
>
This code isn't used for ringbuffer submission and the current GuC
back-end is non-functional. The upcoming backend doesn't use these
structures nor does it is need to be kicked as it is always running if
requests are in the queue. So for the upcoming backend this function is
NULL.
Matt
> --Jason
>
>
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > .../drm/i915/gt/intel_execlists_submission.c | 52 ++++++++++++++++
> > drivers/gpu/drm/i915/i915_scheduler.c | 62 +------------------
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 4 ++
> > 3 files changed, 58 insertions(+), 60 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 23fd03815ad0..3fac17103837 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -3116,10 +3116,61 @@ static bool can_preempt(struct intel_engine_cs *engine)
> > return engine->class != RENDER_CLASS;
> > }
> >
> > +static void kick_execlists(const struct i915_request *rq, int prio)
> > +{
> > + struct intel_engine_cs *engine = rq->engine;
> > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> > + const struct i915_request *inflight;
> > +
> > + /*
> > + * We only need to kick the tasklet once for the high priority
> > + * new context we add into the queue.
> > + */
> > + if (prio <= sched_engine->queue_priority_hint)
> > + return;
> > +
> > + rcu_read_lock();
> > +
> > + /* Nothing currently active? We're overdue for a submission! */
> > + inflight = execlists_active(&engine->execlists);
> > + if (!inflight)
> > + goto unlock;
> > +
> > + /*
> > + * If we are already the currently executing context, don't
> > + * bother evaluating if we should preempt ourselves.
> > + */
> > + if (inflight->context == rq->context)
> > + goto unlock;
> > +
> > + ENGINE_TRACE(engine,
> > + "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
> > + prio,
> > + rq->fence.context, rq->fence.seqno,
> > + inflight->fence.context, inflight->fence.seqno,
> > + inflight->sched.attr.priority);
> > +
> > + sched_engine->queue_priority_hint = prio;
> > +
> > + /*
> > + * Allow preemption of low -> normal -> high, but we do
> > + * not allow low priority tasks to preempt other low priority
> > + * tasks under the impression that latency for low priority
> > + * tasks does not matter (as much as background throughput),
> > + * so kiss.
> > + */
> > + if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
> > + tasklet_hi_schedule(&engine->execlists.tasklet);
> > +
> > +unlock:
> > + rcu_read_unlock();
> > +}
> > +
> > static void execlists_set_default_submission(struct intel_engine_cs *engine)
> > {
> > engine->submit_request = execlists_submit_request;
> > engine->sched_engine->schedule = i915_schedule;
> > + engine->sched_engine->kick_backend = kick_execlists;
> > engine->execlists.tasklet.callback = execlists_submission_tasklet;
> > }
> >
> > @@ -3702,6 +3753,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > ve->base.request_alloc = execlists_request_alloc;
> >
> > ve->base.sched_engine->schedule = i915_schedule;
> > + ve->base.sched_engine->kick_backend = kick_execlists;
> > ve->base.submit_request = virtual_submit_request;
> > ve->base.bond_execute = virtual_bond_execute;
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > index 4bc6969f6a97..035b88f2e4aa 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > @@ -157,65 +157,6 @@ sched_lock_engine(const struct i915_sched_node *node,
> > return locked;
> > }
> >
> > -static inline int rq_prio(const struct i915_request *rq)
> > -{
> > - return rq->sched.attr.priority;
> > -}
> > -
> > -static inline bool need_preempt(int prio, int active)
> > -{
> > - /*
> > - * Allow preemption of low -> normal -> high, but we do
> > - * not allow low priority tasks to preempt other low priority
> > - * tasks under the impression that latency for low priority
> > - * tasks does not matter (as much as background throughput),
> > - * so kiss.
> > - */
> > - return prio >= max(I915_PRIORITY_NORMAL, active);
> > -}
> > -
> > -static void kick_submission(struct intel_engine_cs *engine,
> > - const struct i915_request *rq,
> > - int prio)
> > -{
> > - const struct i915_request *inflight;
> > -
> > - /*
> > - * We only need to kick the tasklet once for the high priority
> > - * new context we add into the queue.
> > - */
> > - if (prio <= engine->sched_engine->queue_priority_hint)
> > - return;
> > -
> > - rcu_read_lock();
> > -
> > - /* Nothing currently active? We're overdue for a submission! */
> > - inflight = execlists_active(&engine->execlists);
> > - if (!inflight)
> > - goto unlock;
> > -
> > - /*
> > - * If we are already the currently executing context, don't
> > - * bother evaluating if we should preempt ourselves.
> > - */
> > - if (inflight->context == rq->context)
> > - goto unlock;
> > -
> > - ENGINE_TRACE(engine,
> > - "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
> > - prio,
> > - rq->fence.context, rq->fence.seqno,
> > - inflight->fence.context, inflight->fence.seqno,
> > - inflight->sched.attr.priority);
> > -
> > - engine->sched_engine->queue_priority_hint = prio;
> > - if (need_preempt(prio, rq_prio(inflight)))
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > -
> > -unlock:
> > - rcu_read_unlock();
> > -}
> > -
> > static void __i915_schedule(struct i915_sched_node *node,
> > const struct i915_sched_attr *attr)
> > {
> > @@ -335,7 +276,8 @@ static void __i915_schedule(struct i915_sched_node *node,
> > }
> >
> > /* Defer (tasklet) submission until after all of our updates. */
> > - kick_submission(engine, node_to_request(node), prio);
> > + if (engine->sched_engine->kick_backend)
> > + engine->sched_engine->kick_backend(node_to_request(node), prio);
> > }
> >
> > spin_unlock(&engine->sched_engine->lock);
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 3f462f8b06f2..8b3af536e6cf 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > @@ -132,6 +132,10 @@ struct i915_sched_engine {
> > */
> > bool no_priolist;
> >
> > + /* Kick backend */
> > + void (*kick_backend)(const struct i915_request *rq,
> > + int prio);
> > +
> > /*
> > * Call when the priority on a request has changed and it and its
> > * dependencies may need rescheduling. Note the request itself may
> > --
> > 2.28.0
> >
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 6/9] drm/i915: Add kick_backend function to i915_sched_engine
@ 2021-06-04 19:19 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 19:19 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 02:09:46PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > Rather than touching execlist specific structures in the generic
> > scheduling code, add a callback function in the backend.
>
> Seems reasonable but why does the function that's there today do
> nothing for the ringbuffer and current GuC back-ends? I'm sure
> there's some reason but it's not immediately obvious to me.
>
This code isn't used for ringbuffer submission and the current GuC
back-end is non-functional. The upcoming backend doesn't use these
structures nor does it is need to be kicked as it is always running if
requests are in the queue. So for the upcoming backend this function is
NULL.
Matt
> --Jason
>
>
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > .../drm/i915/gt/intel_execlists_submission.c | 52 ++++++++++++++++
> > drivers/gpu/drm/i915/i915_scheduler.c | 62 +------------------
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 4 ++
> > 3 files changed, 58 insertions(+), 60 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 23fd03815ad0..3fac17103837 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -3116,10 +3116,61 @@ static bool can_preempt(struct intel_engine_cs *engine)
> > return engine->class != RENDER_CLASS;
> > }
> >
> > +static void kick_execlists(const struct i915_request *rq, int prio)
> > +{
> > + struct intel_engine_cs *engine = rq->engine;
> > + struct i915_sched_engine *sched_engine = engine->sched_engine;
> > + const struct i915_request *inflight;
> > +
> > + /*
> > + * We only need to kick the tasklet once for the high priority
> > + * new context we add into the queue.
> > + */
> > + if (prio <= sched_engine->queue_priority_hint)
> > + return;
> > +
> > + rcu_read_lock();
> > +
> > + /* Nothing currently active? We're overdue for a submission! */
> > + inflight = execlists_active(&engine->execlists);
> > + if (!inflight)
> > + goto unlock;
> > +
> > + /*
> > + * If we are already the currently executing context, don't
> > + * bother evaluating if we should preempt ourselves.
> > + */
> > + if (inflight->context == rq->context)
> > + goto unlock;
> > +
> > + ENGINE_TRACE(engine,
> > + "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
> > + prio,
> > + rq->fence.context, rq->fence.seqno,
> > + inflight->fence.context, inflight->fence.seqno,
> > + inflight->sched.attr.priority);
> > +
> > + sched_engine->queue_priority_hint = prio;
> > +
> > + /*
> > + * Allow preemption of low -> normal -> high, but we do
> > + * not allow low priority tasks to preempt other low priority
> > + * tasks under the impression that latency for low priority
> > + * tasks does not matter (as much as background throughput),
> > + * so kiss.
> > + */
> > + if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
> > + tasklet_hi_schedule(&engine->execlists.tasklet);
> > +
> > +unlock:
> > + rcu_read_unlock();
> > +}
> > +
> > static void execlists_set_default_submission(struct intel_engine_cs *engine)
> > {
> > engine->submit_request = execlists_submit_request;
> > engine->sched_engine->schedule = i915_schedule;
> > + engine->sched_engine->kick_backend = kick_execlists;
> > engine->execlists.tasklet.callback = execlists_submission_tasklet;
> > }
> >
> > @@ -3702,6 +3753,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > ve->base.request_alloc = execlists_request_alloc;
> >
> > ve->base.sched_engine->schedule = i915_schedule;
> > + ve->base.sched_engine->kick_backend = kick_execlists;
> > ve->base.submit_request = virtual_submit_request;
> > ve->base.bond_execute = virtual_bond_execute;
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > index 4bc6969f6a97..035b88f2e4aa 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > @@ -157,65 +157,6 @@ sched_lock_engine(const struct i915_sched_node *node,
> > return locked;
> > }
> >
> > -static inline int rq_prio(const struct i915_request *rq)
> > -{
> > - return rq->sched.attr.priority;
> > -}
> > -
> > -static inline bool need_preempt(int prio, int active)
> > -{
> > - /*
> > - * Allow preemption of low -> normal -> high, but we do
> > - * not allow low priority tasks to preempt other low priority
> > - * tasks under the impression that latency for low priority
> > - * tasks does not matter (as much as background throughput),
> > - * so kiss.
> > - */
> > - return prio >= max(I915_PRIORITY_NORMAL, active);
> > -}
> > -
> > -static void kick_submission(struct intel_engine_cs *engine,
> > - const struct i915_request *rq,
> > - int prio)
> > -{
> > - const struct i915_request *inflight;
> > -
> > - /*
> > - * We only need to kick the tasklet once for the high priority
> > - * new context we add into the queue.
> > - */
> > - if (prio <= engine->sched_engine->queue_priority_hint)
> > - return;
> > -
> > - rcu_read_lock();
> > -
> > - /* Nothing currently active? We're overdue for a submission! */
> > - inflight = execlists_active(&engine->execlists);
> > - if (!inflight)
> > - goto unlock;
> > -
> > - /*
> > - * If we are already the currently executing context, don't
> > - * bother evaluating if we should preempt ourselves.
> > - */
> > - if (inflight->context == rq->context)
> > - goto unlock;
> > -
> > - ENGINE_TRACE(engine,
> > - "bumping queue-priority-hint:%d for rq:%llx:%lld, inflight:%llx:%lld prio %d\n",
> > - prio,
> > - rq->fence.context, rq->fence.seqno,
> > - inflight->fence.context, inflight->fence.seqno,
> > - inflight->sched.attr.priority);
> > -
> > - engine->sched_engine->queue_priority_hint = prio;
> > - if (need_preempt(prio, rq_prio(inflight)))
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > -
> > -unlock:
> > - rcu_read_unlock();
> > -}
> > -
> > static void __i915_schedule(struct i915_sched_node *node,
> > const struct i915_sched_attr *attr)
> > {
> > @@ -335,7 +276,8 @@ static void __i915_schedule(struct i915_sched_node *node,
> > }
> >
> > /* Defer (tasklet) submission until after all of our updates. */
> > - kick_submission(engine, node_to_request(node), prio);
> > + if (engine->sched_engine->kick_backend)
> > + engine->sched_engine->kick_backend(node_to_request(node), prio);
> > }
> >
> > spin_unlock(&engine->sched_engine->lock);
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 3f462f8b06f2..8b3af536e6cf 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > @@ -132,6 +132,10 @@ struct i915_sched_engine {
> > */
> > bool no_priolist;
> >
> > + /* Kick backend */
> > + void (*kick_backend)(const struct i915_request *rq,
> > + int prio);
> > +
> > /*
> > * Call when the priority on a request has changed and it and its
> > * dependencies may need rescheduling. Note the request itself may
> > --
> > 2.28.0
> >
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 8/9] drm/i915: Move submission tasklet to i915_sched_engine
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
@ 2021-06-04 19:26 ` Jason Ekstrand
-1 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 19:26 UTC (permalink / raw)
To: Matthew Brost
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> The submission tasklet operates on i915_sched_engine, thus it is the
> correct place for it.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_engine.h | 14 ---
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 12 +--
> drivers/gpu/drm/i915/gt/intel_engine_types.h | 5 --
> .../drm/i915/gt/intel_execlists_submission.c | 86 ++++++++++---------
> drivers/gpu/drm/i915/gt/mock_engine.c | 1 +
> drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++--
> drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 2 +-
> drivers/gpu/drm/i915/gt/selftest_lrc.c | 6 +-
> drivers/gpu/drm/i915/gt/selftest_reset.c | 2 +-
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 25 +++---
> drivers/gpu/drm/i915/i915_scheduler.c | 1 +
> drivers/gpu/drm/i915/i915_scheduler.h | 14 +++
> drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++
> 13 files changed, 99 insertions(+), 93 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
> index a8b2174b4395..988d9688ae4d 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
> @@ -123,20 +123,6 @@ execlists_active(const struct intel_engine_execlists *execlists)
> return active;
> }
>
> -static inline void
> -execlists_active_lock_bh(struct intel_engine_execlists *execlists)
> -{
> - local_bh_disable(); /* prevent local softirq and lock recursion */
> - tasklet_lock(&execlists->tasklet);
> -}
> -
> -static inline void
> -execlists_active_unlock_bh(struct intel_engine_execlists *execlists)
> -{
> - tasklet_unlock(&execlists->tasklet);
> - local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
> -}
> -
> struct i915_request *
> execlists_unwind_incomplete_requests(struct intel_engine_execlists *execlists);
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index b480fcb1aad1..837d2132e31b 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -711,6 +711,7 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> err = -ENOMEM;
> goto err_sched_engine;
> }
> + engine->sched_engine->engine = engine;
This bothers me. If we're trying to separate things, why do we have a
back pointer? Yes, I know it's because we need to access our engine
back-end somehow. Makes me wonder if it should be some sort of a
void* private data pointer instead of the engine. That's probably
less safe but it seems more "right" in some sense? I don't know.
Also, dumb question but what's a tasklet?
--Jason
>
> err = intel_engine_init_cmd_parser(engine);
> if (err)
> @@ -935,7 +936,6 @@ int intel_engines_init(struct intel_gt *gt)
> void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> {
> GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
> - tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
>
> i915_sched_engine_put(engine->sched_engine);
> intel_breadcrumbs_free(engine->breadcrumbs);
> @@ -1221,7 +1221,7 @@ static bool ring_is_idle(struct intel_engine_cs *engine)
>
> void __intel_engine_flush_submission(struct intel_engine_cs *engine, bool sync)
> {
> - struct tasklet_struct *t = &engine->execlists.tasklet;
> + struct tasklet_struct *t = &engine->sched_engine->tasklet;
>
> if (!t->callback)
> return;
> @@ -1482,8 +1482,8 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
>
> drm_printf(m, "\tExeclist tasklet queued? %s (%s), preempt? %s, timeslice? %s\n",
> yesno(test_bit(TASKLET_STATE_SCHED,
> - &engine->execlists.tasklet.state)),
> - enableddisabled(!atomic_read(&engine->execlists.tasklet.count)),
> + &engine->sched_engine->tasklet.state)),
> + enableddisabled(!atomic_read(&engine->sched_engine->tasklet.count)),
> repr_timer(&engine->execlists.preempt),
> repr_timer(&engine->execlists.timer));
>
> @@ -1507,7 +1507,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
> idx, hws[idx * 2], hws[idx * 2 + 1]);
> }
>
> - execlists_active_lock_bh(execlists);
> + i915_sched_engine_active_lock_bh(engine->sched_engine);
> rcu_read_lock();
> for (port = execlists->active; (rq = *port); port++) {
> char hdr[160];
> @@ -1538,7 +1538,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
> i915_request_show(m, rq, hdr, 0);
> }
> rcu_read_unlock();
> - execlists_active_unlock_bh(execlists);
> + i915_sched_engine_active_unlock_bh(engine->sched_engine);
> } else if (INTEL_GEN(dev_priv) > 6) {
> drm_printf(m, "\tPP_DIR_BASE: 0x%08x\n",
> ENGINE_READ(engine, RING_PP_DIR_BASE));
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> index f1b14aff5118..6f2fd6f13ac4 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> @@ -138,11 +138,6 @@ struct st_preempt_hang {
> * driver and the hardware state for execlist mode of submission.
> */
> struct intel_engine_execlists {
> - /**
> - * @tasklet: softirq tasklet for bottom handler
> - */
> - struct tasklet_struct tasklet;
> -
> /**
> * @timer: kick the current context if its timeslice expires
> */
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 7240c153be35..7a93aec3f6a9 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -570,7 +570,7 @@ static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
> resubmit_virtual_request(rq, ve);
>
> if (READ_ONCE(ve->request))
> - tasklet_hi_schedule(&ve->base.execlists.tasklet);
> + tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
> }
>
> static void __execlists_schedule_out(struct i915_request * const rq,
> @@ -739,9 +739,9 @@ trace_ports(const struct intel_engine_execlists *execlists,
> }
>
> static bool
> -reset_in_progress(const struct intel_engine_execlists *execlists)
> +reset_in_progress(const struct intel_engine_cs *engine)
> {
> - return unlikely(!__tasklet_is_enabled(&execlists->tasklet));
> + return unlikely(!__tasklet_is_enabled(&engine->sched_engine->tasklet));
> }
>
> static __maybe_unused noinline bool
> @@ -757,7 +757,7 @@ assert_pending_valid(const struct intel_engine_execlists *execlists,
> trace_ports(execlists, msg, execlists->pending);
>
> /* We may be messing around with the lists during reset, lalala */
> - if (reset_in_progress(execlists))
> + if (reset_in_progress(engine))
> return true;
>
> if (!execlists->pending[0]) {
> @@ -1190,7 +1190,7 @@ static void start_timeslice(struct intel_engine_cs *engine)
> * its timeslice, so recheck.
> */
> if (!timer_pending(&el->timer))
> - tasklet_hi_schedule(&el->tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> return;
> }
>
> @@ -1772,8 +1772,8 @@ process_csb(struct intel_engine_cs *engine, struct i915_request **inactive)
> * access. Either we are inside the tasklet, or the tasklet is disabled
> * and we assume that is only inside the reset paths and so serialised.
> */
> - GEM_BUG_ON(!tasklet_is_locked(&execlists->tasklet) &&
> - !reset_in_progress(execlists));
> + GEM_BUG_ON(!tasklet_is_locked(&engine->sched_engine->tasklet) &&
> + !reset_in_progress(engine));
>
> /*
> * Note that csb_write, csb_status may be either in HWSP or mmio.
> @@ -2131,7 +2131,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
>
> if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> engine->sched_engine->queue_priority_hint = rq_prio(rq);
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> }
>
> spin_unlock_irq(&engine->sched_engine->lock);
> @@ -2322,13 +2322,13 @@ static void execlists_reset(struct intel_engine_cs *engine, const char *msg)
> ENGINE_TRACE(engine, "reset for %s\n", msg);
>
> /* Mark this tasklet as disabled to avoid waiting for it to complete */
> - tasklet_disable_nosync(&engine->execlists.tasklet);
> + tasklet_disable_nosync(&engine->sched_engine->tasklet);
>
> ring_set_paused(engine, 1); /* Freeze the current request in place */
> execlists_capture(engine);
> intel_engine_reset(engine, msg);
>
> - tasklet_enable(&engine->execlists.tasklet);
> + tasklet_enable(&engine->sched_engine->tasklet);
> clear_and_wake_up_bit(bit, lock);
> }
>
> @@ -2351,8 +2351,9 @@ static bool preempt_timeout(const struct intel_engine_cs *const engine)
> */
> static void execlists_submission_tasklet(struct tasklet_struct *t)
> {
> - struct intel_engine_cs * const engine =
> - from_tasklet(engine, t, execlists.tasklet);
> + struct i915_sched_engine *sched_engine =
> + from_tasklet(sched_engine, t, tasklet);
> + struct intel_engine_cs * const engine = sched_engine->engine;
> struct i915_request *post[2 * EXECLIST_MAX_PORTS];
> struct i915_request **inactive;
>
> @@ -2427,13 +2428,16 @@ static void execlists_irq_handler(struct intel_engine_cs *engine, u16 iir)
> intel_engine_signal_breadcrumbs(engine);
>
> if (tasklet)
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> }
>
> static void __execlists_kick(struct intel_engine_execlists *execlists)
> {
> + struct intel_engine_cs *engine =
> + container_of(execlists, typeof(*engine), execlists);
> +
> /* Kick the tasklet for some interrupt coalescing and reset handling */
> - tasklet_hi_schedule(&execlists->tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> }
>
> #define execlists_kick(t, member) \
> @@ -2808,10 +2812,8 @@ static int execlists_resume(struct intel_engine_cs *engine)
>
> static void execlists_reset_prepare(struct intel_engine_cs *engine)
> {
> - struct intel_engine_execlists * const execlists = &engine->execlists;
> -
> ENGINE_TRACE(engine, "depth<-%d\n",
> - atomic_read(&execlists->tasklet.count));
> + atomic_read(&engine->sched_engine->tasklet.count));
>
> /*
> * Prevent request submission to the hardware until we have
> @@ -2822,8 +2824,8 @@ static void execlists_reset_prepare(struct intel_engine_cs *engine)
> * Turning off the execlists->tasklet until the reset is over
> * prevents the race.
> */
> - __tasklet_disable_sync_once(&execlists->tasklet);
> - GEM_BUG_ON(!reset_in_progress(execlists));
> + __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
> + GEM_BUG_ON(!reset_in_progress(engine));
>
> /*
> * We stop engines, otherwise we might get failed reset and a
> @@ -2973,8 +2975,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
>
> static void nop_submission_tasklet(struct tasklet_struct *t)
> {
> - struct intel_engine_cs * const engine =
> - from_tasklet(engine, t, execlists.tasklet);
> + struct i915_sched_engine *sched_engine =
> + from_tasklet(sched_engine, t, tasklet);
> + struct intel_engine_cs * const engine = sched_engine->engine;
>
> /* The driver is wedged; don't process any more events. */
> WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> @@ -3061,8 +3064,8 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> sched_engine->queue_priority_hint = INT_MIN;
> sched_engine->queue = RB_ROOT_CACHED;
>
> - GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> - execlists->tasklet.callback = nop_submission_tasklet;
> + GEM_BUG_ON(__tasklet_is_enabled(&engine->sched_engine->tasklet));
> + engine->sched_engine->tasklet.callback = nop_submission_tasklet;
>
> spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> rcu_read_unlock();
> @@ -3082,14 +3085,14 @@ static void execlists_reset_finish(struct intel_engine_cs *engine)
> * reset as the next level of recovery, and as a final resort we
> * will declare the device wedged.
> */
> - GEM_BUG_ON(!reset_in_progress(execlists));
> + GEM_BUG_ON(!reset_in_progress(engine));
>
> /* And kick in case we missed a new request submission. */
> - if (__tasklet_enable(&execlists->tasklet))
> + if (__tasklet_enable(&engine->sched_engine->tasklet))
> __execlists_kick(execlists);
>
> ENGINE_TRACE(engine, "depth->%d\n",
> - atomic_read(&execlists->tasklet.count));
> + atomic_read(&engine->sched_engine->tasklet.count));
> }
>
> static void gen8_logical_ring_enable_irq(struct intel_engine_cs *engine)
> @@ -3153,8 +3156,6 @@ static void kick_execlists(const struct i915_request *rq, int prio)
> inflight->fence.context, inflight->fence.seqno,
> inflight->sched.attr.priority);
>
> - sched_engine->queue_priority_hint = prio;
> -
> /*
> * Allow preemption of low -> normal -> high, but we do
> * not allow low priority tasks to preempt other low priority
> @@ -3163,7 +3164,7 @@ static void kick_execlists(const struct i915_request *rq, int prio)
> * so kiss.
> */
> if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&sched_engine->tasklet);
>
> unlock:
> rcu_read_unlock();
> @@ -3174,7 +3175,7 @@ static void execlists_set_default_submission(struct intel_engine_cs *engine)
> engine->submit_request = execlists_submit_request;
> engine->sched_engine->schedule = i915_schedule;
> engine->sched_engine->kick_backend = kick_execlists;
> - engine->execlists.tasklet.callback = execlists_submission_tasklet;
> + engine->sched_engine->tasklet.callback = execlists_submission_tasklet;
> }
>
> static void execlists_shutdown(struct intel_engine_cs *engine)
> @@ -3182,7 +3183,7 @@ static void execlists_shutdown(struct intel_engine_cs *engine)
> /* Synchronise with residual timers and any softirq they raise */
> del_timer_sync(&engine->execlists.timer);
> del_timer_sync(&engine->execlists.preempt);
> - tasklet_kill(&engine->execlists.tasklet);
> + tasklet_kill(&engine->sched_engine->tasklet);
> }
>
> static void execlists_release(struct intel_engine_cs *engine)
> @@ -3298,7 +3299,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> struct intel_uncore *uncore = engine->uncore;
> u32 base = engine->mmio_base;
>
> - tasklet_setup(&engine->execlists.tasklet, execlists_submission_tasklet);
> + tasklet_setup(&engine->sched_engine->tasklet, execlists_submission_tasklet);
> timer_setup(&engine->execlists.timer, execlists_timeslice, 0);
> timer_setup(&engine->execlists.preempt, execlists_preempt, 0);
>
> @@ -3380,7 +3381,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> * rbtrees as in the case it is running in parallel, it may reinsert
> * the rb_node into a sibling.
> */
> - tasklet_kill(&ve->base.execlists.tasklet);
> + tasklet_kill(&ve->base.sched_engine->tasklet);
>
> /* Decouple ourselves from the siblings, no more access allowed. */
> for (n = 0; n < ve->num_siblings; n++) {
> @@ -3392,13 +3393,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
>
> spin_lock_irq(&sibling->sched_engine->lock);
>
> - /* Detachment is lazily performed in the execlists tasklet */
> + /* Detachment is lazily performed in the sched_engine->tasklet */
> if (!RB_EMPTY_NODE(node))
> rb_erase_cached(node, &sibling->execlists.virtual);
>
> spin_unlock_irq(&sibling->sched_engine->lock);
> }
> - GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
> + GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.sched_engine->tasklet));
> GEM_BUG_ON(!list_empty(virtual_queue(ve)));
>
> lrc_fini(&ve->context);
> @@ -3545,9 +3546,11 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
>
> static void virtual_submission_tasklet(struct tasklet_struct *t)
> {
> + struct i915_sched_engine *sched_engine =
> + from_tasklet(sched_engine, t, tasklet);
> struct virtual_engine * const ve =
> - from_tasklet(ve, t, base.execlists.tasklet);
> - const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> + (struct virtual_engine *)sched_engine->engine;
> + const int prio = READ_ONCE(sched_engine->queue_priority_hint);
> intel_engine_mask_t mask;
> unsigned int n;
>
> @@ -3616,7 +3619,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> node->prio = prio;
> if (first && prio > sibling->sched_engine->queue_priority_hint)
> - tasklet_hi_schedule(&sibling->execlists.tasklet);
> + tasklet_hi_schedule(&sibling->sched_engine->tasklet);
>
> unlock_engine:
> spin_unlock_irq(&sibling->sched_engine->lock);
> @@ -3657,7 +3660,7 @@ static void virtual_submit_request(struct i915_request *rq)
> GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> list_move_tail(&rq->sched.link, virtual_queue(ve));
>
> - tasklet_hi_schedule(&ve->base.execlists.tasklet);
> + tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
>
> unlock:
> spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
> @@ -3751,6 +3754,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> err = -ENOMEM;
> goto err_put;
> }
> + ve->base.sched_engine->engine = &ve->base;
>
> ve->base.cops = &virtual_context_ops;
> ve->base.request_alloc = execlists_request_alloc;
> @@ -3761,7 +3765,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> ve->base.bond_execute = virtual_bond_execute;
>
> INIT_LIST_HEAD(virtual_queue(ve));
> - tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> + tasklet_setup(&ve->base.sched_engine->tasklet, virtual_submission_tasklet);
>
> intel_context_init(&ve->context, &ve->base);
>
> @@ -3789,7 +3793,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> * layering if we handle cloning of the requests and
> * submitting a copy into each backend.
> */
> - if (sibling->execlists.tasklet.callback !=
> + if (sibling->sched_engine->tasklet.callback !=
> execlists_submission_tasklet) {
> err = -ENODEV;
> goto err_put;
> diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> index a49fd3039f13..bd005c1b6fd5 100644
> --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> @@ -349,6 +349,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> if (!engine->sched_engine)
> return -ENOMEM;
> + engine->sched_engine->engine = engine;
>
> intel_engine_init_execlists(engine);
> intel_engine_init__pm(engine);
> diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> index 5cc7648d1e5a..2554a2f343b4 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> @@ -43,7 +43,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
> unsigned long timeout)
> {
> /* Ignore our own attempts to suppress excess tasklets */
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
>
> timeout += jiffies;
> do {
> @@ -606,9 +606,9 @@ static int live_hold_reset(void *arg)
> err = -EBUSY;
> goto out;
> }
> - tasklet_disable(&engine->execlists.tasklet);
> + tasklet_disable(&engine->sched_engine->tasklet);
>
> - engine->execlists.tasklet.callback(&engine->execlists.tasklet);
> + engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
> GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
>
> i915_request_get(rq);
> @@ -618,7 +618,7 @@ static int live_hold_reset(void *arg)
> __intel_engine_reset_bh(engine, NULL);
> GEM_BUG_ON(rq->fence.error != -EIO);
>
> - tasklet_enable(&engine->execlists.tasklet);
> + tasklet_enable(&engine->sched_engine->tasklet);
> clear_and_wake_up_bit(I915_RESET_ENGINE + id,
> >->reset.flags);
> local_bh_enable();
> @@ -1183,7 +1183,7 @@ static int live_timeslice_rewind(void *arg)
> while (i915_request_is_active(rq[A2])) { /* semaphore yield! */
> /* Wait for the timeslice to kick in */
> del_timer(&engine->execlists.timer);
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> intel_engine_flush_submission(engine);
> }
> /* -> ELSP[] = { { A:rq1 }, { B:rq1 } } */
> @@ -4593,9 +4593,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
> err = -EBUSY;
> goto out_heartbeat;
> }
> - tasklet_disable(&engine->execlists.tasklet);
> + tasklet_disable(&engine->sched_engine->tasklet);
>
> - engine->execlists.tasklet.callback(&engine->execlists.tasklet);
> + engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
> GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
>
> /* Fake a preemption event; failed of course */
> @@ -4612,7 +4612,7 @@ static int reset_virtual_engine(struct intel_gt *gt,
> GEM_BUG_ON(rq->fence.error != -EIO);
>
> /* Release our grasp on the engine, letting CS flow again */
> - tasklet_enable(&engine->execlists.tasklet);
> + tasklet_enable(&engine->sched_engine->tasklet);
> clear_and_wake_up_bit(I915_RESET_ENGINE + engine->id, >->reset.flags);
> local_bh_enable();
>
> diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> index e57dc900ae8d..cbcb800e2ca0 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> @@ -1702,7 +1702,7 @@ static int __igt_atomic_reset_engine(struct intel_engine_cs *engine,
> const struct igt_atomic_section *p,
> const char *mode)
> {
> - struct tasklet_struct * const t = &engine->execlists.tasklet;
> + struct tasklet_struct * const t = &engine->sched_engine->tasklet;
> int err;
>
> GEM_TRACE("i915_reset_engine(%s:%s) under %s\n",
> diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c
> index d8f6623524e8..6a3a0d89dcd2 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_lrc.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c
> @@ -49,7 +49,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
> unsigned long timeout)
> {
> /* Ignore our own attempts to suppress excess tasklets */
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
>
> timeout += jiffies;
> do {
> @@ -1613,12 +1613,12 @@ static void garbage_reset(struct intel_engine_cs *engine,
>
> local_bh_disable();
> if (!test_and_set_bit(bit, lock)) {
> - tasklet_disable(&engine->execlists.tasklet);
> + tasklet_disable(&engine->sched_engine->tasklet);
>
> if (!rq->fence.error)
> __intel_engine_reset_bh(engine, NULL);
>
> - tasklet_enable(&engine->execlists.tasklet);
> + tasklet_enable(&engine->sched_engine->tasklet);
> clear_and_wake_up_bit(bit, lock);
> }
> local_bh_enable();
> diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c
> index 8784257ec808..7a50c9f4071b 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_reset.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c
> @@ -321,7 +321,7 @@ static int igt_atomic_engine_reset(void *arg)
> goto out_unlock;
>
> for_each_engine(engine, gt, id) {
> - struct tasklet_struct *t = &engine->execlists.tasklet;
> + struct tasklet_struct *t = &engine->sched_engine->tasklet;
>
> if (t->func)
> tasklet_disable(t);
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 7ed21670ef14..95c6f6af4047 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -241,8 +241,9 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
>
> static void guc_submission_tasklet(struct tasklet_struct *t)
> {
> - struct intel_engine_cs * const engine =
> - from_tasklet(engine, t, execlists.tasklet);
> + struct i915_sched_engine *sched_engine =
> + from_tasklet(sched_engine, t, tasklet);
> + struct intel_engine_cs * const engine = sched_engine->engine;
> struct intel_engine_execlists * const execlists = &engine->execlists;
> struct i915_request **port, *rq;
> unsigned long flags;
> @@ -272,14 +273,12 @@ static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
> {
> if (iir & GT_RENDER_USER_INTERRUPT) {
> intel_engine_signal_breadcrumbs(engine);
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> }
> }
>
> static void guc_reset_prepare(struct intel_engine_cs *engine)
> {
> - struct intel_engine_execlists * const execlists = &engine->execlists;
> -
> ENGINE_TRACE(engine, "\n");
>
> /*
> @@ -291,7 +290,7 @@ static void guc_reset_prepare(struct intel_engine_cs *engine)
> * Turning off the execlists->tasklet until the reset is over
> * prevents the race.
> */
> - __tasklet_disable_sync_once(&execlists->tasklet);
> + __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
> }
>
> static void guc_reset_state(struct intel_context *ce,
> @@ -395,14 +394,12 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
>
> static void guc_reset_finish(struct intel_engine_cs *engine)
> {
> - struct intel_engine_execlists * const execlists = &engine->execlists;
> -
> - if (__tasklet_enable(&execlists->tasklet))
> + if (__tasklet_enable(&engine->sched_engine->tasklet))
> /* And kick in case we missed a new request submission. */
> - tasklet_hi_schedule(&execlists->tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
>
> ENGINE_TRACE(engine, "depth->%d\n",
> - atomic_read(&execlists->tasklet.count));
> + atomic_read(&engine->sched_engine->tasklet.count));
> }
>
> /*
> @@ -546,7 +543,7 @@ static void guc_submit_request(struct i915_request *rq)
> GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
> GEM_BUG_ON(list_empty(&rq->sched.link));
>
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
>
> spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
> @@ -626,7 +623,7 @@ static void guc_release(struct intel_engine_cs *engine)
> {
> engine->sanitize = NULL; /* no longer in control, nothing to sanitize */
>
> - tasklet_kill(&engine->execlists.tasklet);
> + tasklet_kill(&engine->sched_engine->tasklet);
>
> intel_engine_cleanup_common(engine);
> lrc_fini_wa_ctx(engine);
> @@ -705,7 +702,7 @@ int intel_guc_submission_setup(struct intel_engine_cs *engine)
> */
> GEM_BUG_ON(INTEL_GEN(i915) < 11);
>
> - tasklet_setup(&engine->execlists.tasklet, guc_submission_tasklet);
> + tasklet_setup(&engine->sched_engine->tasklet, guc_submission_tasklet);
>
> guc_default_vfuncs(engine);
> guc_default_irqs(engine);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 3d36e4447b5d..6f082579ee9e 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -436,6 +436,7 @@ void i915_sched_engine_free(struct kref *kref)
> struct i915_sched_engine *sched_engine =
> container_of(kref, typeof(*sched_engine), ref);
>
> + tasklet_kill(&sched_engine->tasklet); /* flush the callback */
> kfree(sched_engine);
> }
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 0014745bda30..650ab8e0db9f 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -79,6 +79,20 @@ i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
> sched_engine->no_priolist = false;
> }
>
> +static inline void
> +i915_sched_engine_active_lock_bh(struct i915_sched_engine *sched_engine)
> +{
> + local_bh_disable(); /* prevent local softirq and lock recursion */
> + tasklet_lock(&sched_engine->tasklet);
> +}
> +
> +static inline void
> +i915_sched_engine_active_unlock_bh(struct i915_sched_engine *sched_engine)
> +{
> + tasklet_unlock(&sched_engine->tasklet);
> + local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
> +}
> +
> void i915_request_show_with_schedule(struct drm_printer *m,
> const struct i915_request *rq,
> const char *prefix,
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 8b3af536e6cf..9d79514450de 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> @@ -103,6 +103,11 @@ struct i915_sched_engine {
> struct list_head requests;
> struct list_head hold; /* ready requests, but on hold */
>
> + /**
> + * @tasklet: softirq tasklet for bottom handler
> + */
> + struct tasklet_struct tasklet;
> +
> /**
> * @default_priolist: priority list for I915_PRIORITY_NORMAL
> */
> @@ -132,6 +137,9 @@ struct i915_sched_engine {
> */
> bool no_priolist;
>
> + /* Back pointer to engine */
> + struct intel_engine_cs *engine;
> +
> /* Kick backend */
> void (*kick_backend)(const struct i915_request *rq,
> int prio);
> --
> 2.28.0
>
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 8/9] drm/i915: Move submission tasklet to i915_sched_engine
@ 2021-06-04 19:26 ` Jason Ekstrand
0 siblings, 0 replies; 64+ messages in thread
From: Jason Ekstrand @ 2021-06-04 19:26 UTC (permalink / raw)
To: Matthew Brost; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
>
> The submission tasklet operates on i915_sched_engine, thus it is the
> correct place for it.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_engine.h | 14 ---
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 12 +--
> drivers/gpu/drm/i915/gt/intel_engine_types.h | 5 --
> .../drm/i915/gt/intel_execlists_submission.c | 86 ++++++++++---------
> drivers/gpu/drm/i915/gt/mock_engine.c | 1 +
> drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++--
> drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 2 +-
> drivers/gpu/drm/i915/gt/selftest_lrc.c | 6 +-
> drivers/gpu/drm/i915/gt/selftest_reset.c | 2 +-
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 25 +++---
> drivers/gpu/drm/i915/i915_scheduler.c | 1 +
> drivers/gpu/drm/i915/i915_scheduler.h | 14 +++
> drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++
> 13 files changed, 99 insertions(+), 93 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
> index a8b2174b4395..988d9688ae4d 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
> @@ -123,20 +123,6 @@ execlists_active(const struct intel_engine_execlists *execlists)
> return active;
> }
>
> -static inline void
> -execlists_active_lock_bh(struct intel_engine_execlists *execlists)
> -{
> - local_bh_disable(); /* prevent local softirq and lock recursion */
> - tasklet_lock(&execlists->tasklet);
> -}
> -
> -static inline void
> -execlists_active_unlock_bh(struct intel_engine_execlists *execlists)
> -{
> - tasklet_unlock(&execlists->tasklet);
> - local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
> -}
> -
> struct i915_request *
> execlists_unwind_incomplete_requests(struct intel_engine_execlists *execlists);
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index b480fcb1aad1..837d2132e31b 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -711,6 +711,7 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> err = -ENOMEM;
> goto err_sched_engine;
> }
> + engine->sched_engine->engine = engine;
This bothers me. If we're trying to separate things, why do we have a
back pointer? Yes, I know it's because we need to access our engine
back-end somehow. Makes me wonder if it should be some sort of a
void* private data pointer instead of the engine. That's probably
less safe but it seems more "right" in some sense? I don't know.
Also, dumb question but what's a tasklet?
--Jason
>
> err = intel_engine_init_cmd_parser(engine);
> if (err)
> @@ -935,7 +936,6 @@ int intel_engines_init(struct intel_gt *gt)
> void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> {
> GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
> - tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
>
> i915_sched_engine_put(engine->sched_engine);
> intel_breadcrumbs_free(engine->breadcrumbs);
> @@ -1221,7 +1221,7 @@ static bool ring_is_idle(struct intel_engine_cs *engine)
>
> void __intel_engine_flush_submission(struct intel_engine_cs *engine, bool sync)
> {
> - struct tasklet_struct *t = &engine->execlists.tasklet;
> + struct tasklet_struct *t = &engine->sched_engine->tasklet;
>
> if (!t->callback)
> return;
> @@ -1482,8 +1482,8 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
>
> drm_printf(m, "\tExeclist tasklet queued? %s (%s), preempt? %s, timeslice? %s\n",
> yesno(test_bit(TASKLET_STATE_SCHED,
> - &engine->execlists.tasklet.state)),
> - enableddisabled(!atomic_read(&engine->execlists.tasklet.count)),
> + &engine->sched_engine->tasklet.state)),
> + enableddisabled(!atomic_read(&engine->sched_engine->tasklet.count)),
> repr_timer(&engine->execlists.preempt),
> repr_timer(&engine->execlists.timer));
>
> @@ -1507,7 +1507,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
> idx, hws[idx * 2], hws[idx * 2 + 1]);
> }
>
> - execlists_active_lock_bh(execlists);
> + i915_sched_engine_active_lock_bh(engine->sched_engine);
> rcu_read_lock();
> for (port = execlists->active; (rq = *port); port++) {
> char hdr[160];
> @@ -1538,7 +1538,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
> i915_request_show(m, rq, hdr, 0);
> }
> rcu_read_unlock();
> - execlists_active_unlock_bh(execlists);
> + i915_sched_engine_active_unlock_bh(engine->sched_engine);
> } else if (INTEL_GEN(dev_priv) > 6) {
> drm_printf(m, "\tPP_DIR_BASE: 0x%08x\n",
> ENGINE_READ(engine, RING_PP_DIR_BASE));
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> index f1b14aff5118..6f2fd6f13ac4 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> @@ -138,11 +138,6 @@ struct st_preempt_hang {
> * driver and the hardware state for execlist mode of submission.
> */
> struct intel_engine_execlists {
> - /**
> - * @tasklet: softirq tasklet for bottom handler
> - */
> - struct tasklet_struct tasklet;
> -
> /**
> * @timer: kick the current context if its timeslice expires
> */
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 7240c153be35..7a93aec3f6a9 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -570,7 +570,7 @@ static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
> resubmit_virtual_request(rq, ve);
>
> if (READ_ONCE(ve->request))
> - tasklet_hi_schedule(&ve->base.execlists.tasklet);
> + tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
> }
>
> static void __execlists_schedule_out(struct i915_request * const rq,
> @@ -739,9 +739,9 @@ trace_ports(const struct intel_engine_execlists *execlists,
> }
>
> static bool
> -reset_in_progress(const struct intel_engine_execlists *execlists)
> +reset_in_progress(const struct intel_engine_cs *engine)
> {
> - return unlikely(!__tasklet_is_enabled(&execlists->tasklet));
> + return unlikely(!__tasklet_is_enabled(&engine->sched_engine->tasklet));
> }
>
> static __maybe_unused noinline bool
> @@ -757,7 +757,7 @@ assert_pending_valid(const struct intel_engine_execlists *execlists,
> trace_ports(execlists, msg, execlists->pending);
>
> /* We may be messing around with the lists during reset, lalala */
> - if (reset_in_progress(execlists))
> + if (reset_in_progress(engine))
> return true;
>
> if (!execlists->pending[0]) {
> @@ -1190,7 +1190,7 @@ static void start_timeslice(struct intel_engine_cs *engine)
> * its timeslice, so recheck.
> */
> if (!timer_pending(&el->timer))
> - tasklet_hi_schedule(&el->tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> return;
> }
>
> @@ -1772,8 +1772,8 @@ process_csb(struct intel_engine_cs *engine, struct i915_request **inactive)
> * access. Either we are inside the tasklet, or the tasklet is disabled
> * and we assume that is only inside the reset paths and so serialised.
> */
> - GEM_BUG_ON(!tasklet_is_locked(&execlists->tasklet) &&
> - !reset_in_progress(execlists));
> + GEM_BUG_ON(!tasklet_is_locked(&engine->sched_engine->tasklet) &&
> + !reset_in_progress(engine));
>
> /*
> * Note that csb_write, csb_status may be either in HWSP or mmio.
> @@ -2131,7 +2131,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
>
> if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> engine->sched_engine->queue_priority_hint = rq_prio(rq);
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> }
>
> spin_unlock_irq(&engine->sched_engine->lock);
> @@ -2322,13 +2322,13 @@ static void execlists_reset(struct intel_engine_cs *engine, const char *msg)
> ENGINE_TRACE(engine, "reset for %s\n", msg);
>
> /* Mark this tasklet as disabled to avoid waiting for it to complete */
> - tasklet_disable_nosync(&engine->execlists.tasklet);
> + tasklet_disable_nosync(&engine->sched_engine->tasklet);
>
> ring_set_paused(engine, 1); /* Freeze the current request in place */
> execlists_capture(engine);
> intel_engine_reset(engine, msg);
>
> - tasklet_enable(&engine->execlists.tasklet);
> + tasklet_enable(&engine->sched_engine->tasklet);
> clear_and_wake_up_bit(bit, lock);
> }
>
> @@ -2351,8 +2351,9 @@ static bool preempt_timeout(const struct intel_engine_cs *const engine)
> */
> static void execlists_submission_tasklet(struct tasklet_struct *t)
> {
> - struct intel_engine_cs * const engine =
> - from_tasklet(engine, t, execlists.tasklet);
> + struct i915_sched_engine *sched_engine =
> + from_tasklet(sched_engine, t, tasklet);
> + struct intel_engine_cs * const engine = sched_engine->engine;
> struct i915_request *post[2 * EXECLIST_MAX_PORTS];
> struct i915_request **inactive;
>
> @@ -2427,13 +2428,16 @@ static void execlists_irq_handler(struct intel_engine_cs *engine, u16 iir)
> intel_engine_signal_breadcrumbs(engine);
>
> if (tasklet)
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> }
>
> static void __execlists_kick(struct intel_engine_execlists *execlists)
> {
> + struct intel_engine_cs *engine =
> + container_of(execlists, typeof(*engine), execlists);
> +
> /* Kick the tasklet for some interrupt coalescing and reset handling */
> - tasklet_hi_schedule(&execlists->tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> }
>
> #define execlists_kick(t, member) \
> @@ -2808,10 +2812,8 @@ static int execlists_resume(struct intel_engine_cs *engine)
>
> static void execlists_reset_prepare(struct intel_engine_cs *engine)
> {
> - struct intel_engine_execlists * const execlists = &engine->execlists;
> -
> ENGINE_TRACE(engine, "depth<-%d\n",
> - atomic_read(&execlists->tasklet.count));
> + atomic_read(&engine->sched_engine->tasklet.count));
>
> /*
> * Prevent request submission to the hardware until we have
> @@ -2822,8 +2824,8 @@ static void execlists_reset_prepare(struct intel_engine_cs *engine)
> * Turning off the execlists->tasklet until the reset is over
> * prevents the race.
> */
> - __tasklet_disable_sync_once(&execlists->tasklet);
> - GEM_BUG_ON(!reset_in_progress(execlists));
> + __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
> + GEM_BUG_ON(!reset_in_progress(engine));
>
> /*
> * We stop engines, otherwise we might get failed reset and a
> @@ -2973,8 +2975,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
>
> static void nop_submission_tasklet(struct tasklet_struct *t)
> {
> - struct intel_engine_cs * const engine =
> - from_tasklet(engine, t, execlists.tasklet);
> + struct i915_sched_engine *sched_engine =
> + from_tasklet(sched_engine, t, tasklet);
> + struct intel_engine_cs * const engine = sched_engine->engine;
>
> /* The driver is wedged; don't process any more events. */
> WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> @@ -3061,8 +3064,8 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> sched_engine->queue_priority_hint = INT_MIN;
> sched_engine->queue = RB_ROOT_CACHED;
>
> - GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> - execlists->tasklet.callback = nop_submission_tasklet;
> + GEM_BUG_ON(__tasklet_is_enabled(&engine->sched_engine->tasklet));
> + engine->sched_engine->tasklet.callback = nop_submission_tasklet;
>
> spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> rcu_read_unlock();
> @@ -3082,14 +3085,14 @@ static void execlists_reset_finish(struct intel_engine_cs *engine)
> * reset as the next level of recovery, and as a final resort we
> * will declare the device wedged.
> */
> - GEM_BUG_ON(!reset_in_progress(execlists));
> + GEM_BUG_ON(!reset_in_progress(engine));
>
> /* And kick in case we missed a new request submission. */
> - if (__tasklet_enable(&execlists->tasklet))
> + if (__tasklet_enable(&engine->sched_engine->tasklet))
> __execlists_kick(execlists);
>
> ENGINE_TRACE(engine, "depth->%d\n",
> - atomic_read(&execlists->tasklet.count));
> + atomic_read(&engine->sched_engine->tasklet.count));
> }
>
> static void gen8_logical_ring_enable_irq(struct intel_engine_cs *engine)
> @@ -3153,8 +3156,6 @@ static void kick_execlists(const struct i915_request *rq, int prio)
> inflight->fence.context, inflight->fence.seqno,
> inflight->sched.attr.priority);
>
> - sched_engine->queue_priority_hint = prio;
> -
> /*
> * Allow preemption of low -> normal -> high, but we do
> * not allow low priority tasks to preempt other low priority
> @@ -3163,7 +3164,7 @@ static void kick_execlists(const struct i915_request *rq, int prio)
> * so kiss.
> */
> if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&sched_engine->tasklet);
>
> unlock:
> rcu_read_unlock();
> @@ -3174,7 +3175,7 @@ static void execlists_set_default_submission(struct intel_engine_cs *engine)
> engine->submit_request = execlists_submit_request;
> engine->sched_engine->schedule = i915_schedule;
> engine->sched_engine->kick_backend = kick_execlists;
> - engine->execlists.tasklet.callback = execlists_submission_tasklet;
> + engine->sched_engine->tasklet.callback = execlists_submission_tasklet;
> }
>
> static void execlists_shutdown(struct intel_engine_cs *engine)
> @@ -3182,7 +3183,7 @@ static void execlists_shutdown(struct intel_engine_cs *engine)
> /* Synchronise with residual timers and any softirq they raise */
> del_timer_sync(&engine->execlists.timer);
> del_timer_sync(&engine->execlists.preempt);
> - tasklet_kill(&engine->execlists.tasklet);
> + tasklet_kill(&engine->sched_engine->tasklet);
> }
>
> static void execlists_release(struct intel_engine_cs *engine)
> @@ -3298,7 +3299,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> struct intel_uncore *uncore = engine->uncore;
> u32 base = engine->mmio_base;
>
> - tasklet_setup(&engine->execlists.tasklet, execlists_submission_tasklet);
> + tasklet_setup(&engine->sched_engine->tasklet, execlists_submission_tasklet);
> timer_setup(&engine->execlists.timer, execlists_timeslice, 0);
> timer_setup(&engine->execlists.preempt, execlists_preempt, 0);
>
> @@ -3380,7 +3381,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> * rbtrees as in the case it is running in parallel, it may reinsert
> * the rb_node into a sibling.
> */
> - tasklet_kill(&ve->base.execlists.tasklet);
> + tasklet_kill(&ve->base.sched_engine->tasklet);
>
> /* Decouple ourselves from the siblings, no more access allowed. */
> for (n = 0; n < ve->num_siblings; n++) {
> @@ -3392,13 +3393,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
>
> spin_lock_irq(&sibling->sched_engine->lock);
>
> - /* Detachment is lazily performed in the execlists tasklet */
> + /* Detachment is lazily performed in the sched_engine->tasklet */
> if (!RB_EMPTY_NODE(node))
> rb_erase_cached(node, &sibling->execlists.virtual);
>
> spin_unlock_irq(&sibling->sched_engine->lock);
> }
> - GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
> + GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.sched_engine->tasklet));
> GEM_BUG_ON(!list_empty(virtual_queue(ve)));
>
> lrc_fini(&ve->context);
> @@ -3545,9 +3546,11 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
>
> static void virtual_submission_tasklet(struct tasklet_struct *t)
> {
> + struct i915_sched_engine *sched_engine =
> + from_tasklet(sched_engine, t, tasklet);
> struct virtual_engine * const ve =
> - from_tasklet(ve, t, base.execlists.tasklet);
> - const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> + (struct virtual_engine *)sched_engine->engine;
> + const int prio = READ_ONCE(sched_engine->queue_priority_hint);
> intel_engine_mask_t mask;
> unsigned int n;
>
> @@ -3616,7 +3619,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> node->prio = prio;
> if (first && prio > sibling->sched_engine->queue_priority_hint)
> - tasklet_hi_schedule(&sibling->execlists.tasklet);
> + tasklet_hi_schedule(&sibling->sched_engine->tasklet);
>
> unlock_engine:
> spin_unlock_irq(&sibling->sched_engine->lock);
> @@ -3657,7 +3660,7 @@ static void virtual_submit_request(struct i915_request *rq)
> GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> list_move_tail(&rq->sched.link, virtual_queue(ve));
>
> - tasklet_hi_schedule(&ve->base.execlists.tasklet);
> + tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
>
> unlock:
> spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
> @@ -3751,6 +3754,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> err = -ENOMEM;
> goto err_put;
> }
> + ve->base.sched_engine->engine = &ve->base;
>
> ve->base.cops = &virtual_context_ops;
> ve->base.request_alloc = execlists_request_alloc;
> @@ -3761,7 +3765,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> ve->base.bond_execute = virtual_bond_execute;
>
> INIT_LIST_HEAD(virtual_queue(ve));
> - tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> + tasklet_setup(&ve->base.sched_engine->tasklet, virtual_submission_tasklet);
>
> intel_context_init(&ve->context, &ve->base);
>
> @@ -3789,7 +3793,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> * layering if we handle cloning of the requests and
> * submitting a copy into each backend.
> */
> - if (sibling->execlists.tasklet.callback !=
> + if (sibling->sched_engine->tasklet.callback !=
> execlists_submission_tasklet) {
> err = -ENODEV;
> goto err_put;
> diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> index a49fd3039f13..bd005c1b6fd5 100644
> --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> @@ -349,6 +349,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> if (!engine->sched_engine)
> return -ENOMEM;
> + engine->sched_engine->engine = engine;
>
> intel_engine_init_execlists(engine);
> intel_engine_init__pm(engine);
> diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> index 5cc7648d1e5a..2554a2f343b4 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> @@ -43,7 +43,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
> unsigned long timeout)
> {
> /* Ignore our own attempts to suppress excess tasklets */
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
>
> timeout += jiffies;
> do {
> @@ -606,9 +606,9 @@ static int live_hold_reset(void *arg)
> err = -EBUSY;
> goto out;
> }
> - tasklet_disable(&engine->execlists.tasklet);
> + tasklet_disable(&engine->sched_engine->tasklet);
>
> - engine->execlists.tasklet.callback(&engine->execlists.tasklet);
> + engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
> GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
>
> i915_request_get(rq);
> @@ -618,7 +618,7 @@ static int live_hold_reset(void *arg)
> __intel_engine_reset_bh(engine, NULL);
> GEM_BUG_ON(rq->fence.error != -EIO);
>
> - tasklet_enable(&engine->execlists.tasklet);
> + tasklet_enable(&engine->sched_engine->tasklet);
> clear_and_wake_up_bit(I915_RESET_ENGINE + id,
> >->reset.flags);
> local_bh_enable();
> @@ -1183,7 +1183,7 @@ static int live_timeslice_rewind(void *arg)
> while (i915_request_is_active(rq[A2])) { /* semaphore yield! */
> /* Wait for the timeslice to kick in */
> del_timer(&engine->execlists.timer);
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> intel_engine_flush_submission(engine);
> }
> /* -> ELSP[] = { { A:rq1 }, { B:rq1 } } */
> @@ -4593,9 +4593,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
> err = -EBUSY;
> goto out_heartbeat;
> }
> - tasklet_disable(&engine->execlists.tasklet);
> + tasklet_disable(&engine->sched_engine->tasklet);
>
> - engine->execlists.tasklet.callback(&engine->execlists.tasklet);
> + engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
> GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
>
> /* Fake a preemption event; failed of course */
> @@ -4612,7 +4612,7 @@ static int reset_virtual_engine(struct intel_gt *gt,
> GEM_BUG_ON(rq->fence.error != -EIO);
>
> /* Release our grasp on the engine, letting CS flow again */
> - tasklet_enable(&engine->execlists.tasklet);
> + tasklet_enable(&engine->sched_engine->tasklet);
> clear_and_wake_up_bit(I915_RESET_ENGINE + engine->id, >->reset.flags);
> local_bh_enable();
>
> diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> index e57dc900ae8d..cbcb800e2ca0 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> @@ -1702,7 +1702,7 @@ static int __igt_atomic_reset_engine(struct intel_engine_cs *engine,
> const struct igt_atomic_section *p,
> const char *mode)
> {
> - struct tasklet_struct * const t = &engine->execlists.tasklet;
> + struct tasklet_struct * const t = &engine->sched_engine->tasklet;
> int err;
>
> GEM_TRACE("i915_reset_engine(%s:%s) under %s\n",
> diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c
> index d8f6623524e8..6a3a0d89dcd2 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_lrc.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c
> @@ -49,7 +49,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
> unsigned long timeout)
> {
> /* Ignore our own attempts to suppress excess tasklets */
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
>
> timeout += jiffies;
> do {
> @@ -1613,12 +1613,12 @@ static void garbage_reset(struct intel_engine_cs *engine,
>
> local_bh_disable();
> if (!test_and_set_bit(bit, lock)) {
> - tasklet_disable(&engine->execlists.tasklet);
> + tasklet_disable(&engine->sched_engine->tasklet);
>
> if (!rq->fence.error)
> __intel_engine_reset_bh(engine, NULL);
>
> - tasklet_enable(&engine->execlists.tasklet);
> + tasklet_enable(&engine->sched_engine->tasklet);
> clear_and_wake_up_bit(bit, lock);
> }
> local_bh_enable();
> diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c
> index 8784257ec808..7a50c9f4071b 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_reset.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c
> @@ -321,7 +321,7 @@ static int igt_atomic_engine_reset(void *arg)
> goto out_unlock;
>
> for_each_engine(engine, gt, id) {
> - struct tasklet_struct *t = &engine->execlists.tasklet;
> + struct tasklet_struct *t = &engine->sched_engine->tasklet;
>
> if (t->func)
> tasklet_disable(t);
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 7ed21670ef14..95c6f6af4047 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -241,8 +241,9 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
>
> static void guc_submission_tasklet(struct tasklet_struct *t)
> {
> - struct intel_engine_cs * const engine =
> - from_tasklet(engine, t, execlists.tasklet);
> + struct i915_sched_engine *sched_engine =
> + from_tasklet(sched_engine, t, tasklet);
> + struct intel_engine_cs * const engine = sched_engine->engine;
> struct intel_engine_execlists * const execlists = &engine->execlists;
> struct i915_request **port, *rq;
> unsigned long flags;
> @@ -272,14 +273,12 @@ static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
> {
> if (iir & GT_RENDER_USER_INTERRUPT) {
> intel_engine_signal_breadcrumbs(engine);
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> }
> }
>
> static void guc_reset_prepare(struct intel_engine_cs *engine)
> {
> - struct intel_engine_execlists * const execlists = &engine->execlists;
> -
> ENGINE_TRACE(engine, "\n");
>
> /*
> @@ -291,7 +290,7 @@ static void guc_reset_prepare(struct intel_engine_cs *engine)
> * Turning off the execlists->tasklet until the reset is over
> * prevents the race.
> */
> - __tasklet_disable_sync_once(&execlists->tasklet);
> + __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
> }
>
> static void guc_reset_state(struct intel_context *ce,
> @@ -395,14 +394,12 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
>
> static void guc_reset_finish(struct intel_engine_cs *engine)
> {
> - struct intel_engine_execlists * const execlists = &engine->execlists;
> -
> - if (__tasklet_enable(&execlists->tasklet))
> + if (__tasklet_enable(&engine->sched_engine->tasklet))
> /* And kick in case we missed a new request submission. */
> - tasklet_hi_schedule(&execlists->tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
>
> ENGINE_TRACE(engine, "depth->%d\n",
> - atomic_read(&execlists->tasklet.count));
> + atomic_read(&engine->sched_engine->tasklet.count));
> }
>
> /*
> @@ -546,7 +543,7 @@ static void guc_submit_request(struct i915_request *rq)
> GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
> GEM_BUG_ON(list_empty(&rq->sched.link));
>
> - tasklet_hi_schedule(&engine->execlists.tasklet);
> + tasklet_hi_schedule(&engine->sched_engine->tasklet);
>
> spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> }
> @@ -626,7 +623,7 @@ static void guc_release(struct intel_engine_cs *engine)
> {
> engine->sanitize = NULL; /* no longer in control, nothing to sanitize */
>
> - tasklet_kill(&engine->execlists.tasklet);
> + tasklet_kill(&engine->sched_engine->tasklet);
>
> intel_engine_cleanup_common(engine);
> lrc_fini_wa_ctx(engine);
> @@ -705,7 +702,7 @@ int intel_guc_submission_setup(struct intel_engine_cs *engine)
> */
> GEM_BUG_ON(INTEL_GEN(i915) < 11);
>
> - tasklet_setup(&engine->execlists.tasklet, guc_submission_tasklet);
> + tasklet_setup(&engine->sched_engine->tasklet, guc_submission_tasklet);
>
> guc_default_vfuncs(engine);
> guc_default_irqs(engine);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 3d36e4447b5d..6f082579ee9e 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -436,6 +436,7 @@ void i915_sched_engine_free(struct kref *kref)
> struct i915_sched_engine *sched_engine =
> container_of(kref, typeof(*sched_engine), ref);
>
> + tasklet_kill(&sched_engine->tasklet); /* flush the callback */
> kfree(sched_engine);
> }
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 0014745bda30..650ab8e0db9f 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -79,6 +79,20 @@ i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
> sched_engine->no_priolist = false;
> }
>
> +static inline void
> +i915_sched_engine_active_lock_bh(struct i915_sched_engine *sched_engine)
> +{
> + local_bh_disable(); /* prevent local softirq and lock recursion */
> + tasklet_lock(&sched_engine->tasklet);
> +}
> +
> +static inline void
> +i915_sched_engine_active_unlock_bh(struct i915_sched_engine *sched_engine)
> +{
> + tasklet_unlock(&sched_engine->tasklet);
> + local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
> +}
> +
> void i915_request_show_with_schedule(struct drm_printer *m,
> const struct i915_request *rq,
> const char *prefix,
> diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> index 8b3af536e6cf..9d79514450de 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> @@ -103,6 +103,11 @@ struct i915_sched_engine {
> struct list_head requests;
> struct list_head hold; /* ready requests, but on hold */
>
> + /**
> + * @tasklet: softirq tasklet for bottom handler
> + */
> + struct tasklet_struct tasklet;
> +
> /**
> * @default_priolist: priority list for I915_PRIORITY_NORMAL
> */
> @@ -132,6 +137,9 @@ struct i915_sched_engine {
> */
> bool no_priolist;
>
> + /* Back pointer to engine */
> + struct intel_engine_cs *engine;
> +
> /* Kick backend */
> void (*kick_backend)(const struct i915_request *rq,
> int prio);
> --
> 2.28.0
>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 8/9] drm/i915: Move submission tasklet to i915_sched_engine
2021-06-04 19:26 ` [Intel-gfx] " Jason Ekstrand
@ 2021-06-04 19:55 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 19:55 UTC (permalink / raw)
To: Jason Ekstrand
Cc: Daniel Vetter, Intel GFX, Daniele Ceraolo Spurio,
Maling list - DRI developers
On Fri, Jun 04, 2021 at 02:26:38PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > The submission tasklet operates on i915_sched_engine, thus it is the
> > correct place for it.
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gt/intel_engine.h | 14 ---
> > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 12 +--
> > drivers/gpu/drm/i915/gt/intel_engine_types.h | 5 --
> > .../drm/i915/gt/intel_execlists_submission.c | 86 ++++++++++---------
> > drivers/gpu/drm/i915/gt/mock_engine.c | 1 +
> > drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++--
> > drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 2 +-
> > drivers/gpu/drm/i915/gt/selftest_lrc.c | 6 +-
> > drivers/gpu/drm/i915/gt/selftest_reset.c | 2 +-
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 25 +++---
> > drivers/gpu/drm/i915/i915_scheduler.c | 1 +
> > drivers/gpu/drm/i915/i915_scheduler.h | 14 +++
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++
> > 13 files changed, 99 insertions(+), 93 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
> > index a8b2174b4395..988d9688ae4d 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
> > @@ -123,20 +123,6 @@ execlists_active(const struct intel_engine_execlists *execlists)
> > return active;
> > }
> >
> > -static inline void
> > -execlists_active_lock_bh(struct intel_engine_execlists *execlists)
> > -{
> > - local_bh_disable(); /* prevent local softirq and lock recursion */
> > - tasklet_lock(&execlists->tasklet);
> > -}
> > -
> > -static inline void
> > -execlists_active_unlock_bh(struct intel_engine_execlists *execlists)
> > -{
> > - tasklet_unlock(&execlists->tasklet);
> > - local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
> > -}
> > -
> > struct i915_request *
> > execlists_unwind_incomplete_requests(struct intel_engine_execlists *execlists);
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > index b480fcb1aad1..837d2132e31b 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > @@ -711,6 +711,7 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > err = -ENOMEM;
> > goto err_sched_engine;
> > }
> > + engine->sched_engine->engine = engine;
>
> This bothers me. If we're trying to separate things, why do we have a
> back pointer? Yes, I know it's because we need to access our engine
> back-end somehow. Makes me wonder if it should be some sort of a
> void* private data pointer instead of the engine. That's probably
> less safe but it seems more "right" in some sense? I don't know.
>
This was new moving from internal to upstream because of the new tasklet
API that has landed. Previously the tasklet function was setup with u64
data which we used to point to the engine. Now it uses a callback with
the tasklet structure as the argument and can only use a container_of to
resolve the object which in this case a i915_sched_engine. In execlists
we still need the engine object to do the submit, hence the back
pointer.
Longterm we really should have an execlists_submit_engine which
encapsulates the i915_sched_engine and everything we need for execlists
submission (and with DRM scheduler the i915_sched_engine encapsulates
the drm_gpu_scheduler object and common variables between execlists and
GuC).
I already have an object like this for the GuC, guc_submit_eninge,
albeit slightly out of order in patch series.
Agree void* private is better as for the GuC code we likely need this to
be a backpointer to the GuC structure until the aforementioned
guc_submit_engine lands.
Are you ok with landing this with a void* and taking a note this needs
to be reworked?
> Also, dumb question but what's a tasklet?
>
All submissions in the current i915 go through a tasklet. The GuC
submission that will initially land uses this too. Once the DRM
scheduler lands this will go away, at least for GuC submission. If this
is still around for execlists we likely stick this in
execlists_submit_engine object and forget about it.
Matt
> --Jason
>
> >
> > err = intel_engine_init_cmd_parser(engine);
> > if (err)
> > @@ -935,7 +936,6 @@ int intel_engines_init(struct intel_gt *gt)
> > void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > {
> > GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
> > - tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> >
> > i915_sched_engine_put(engine->sched_engine);
> > intel_breadcrumbs_free(engine->breadcrumbs);
> > @@ -1221,7 +1221,7 @@ static bool ring_is_idle(struct intel_engine_cs *engine)
> >
> > void __intel_engine_flush_submission(struct intel_engine_cs *engine, bool sync)
> > {
> > - struct tasklet_struct *t = &engine->execlists.tasklet;
> > + struct tasklet_struct *t = &engine->sched_engine->tasklet;
> >
> > if (!t->callback)
> > return;
> > @@ -1482,8 +1482,8 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
> >
> > drm_printf(m, "\tExeclist tasklet queued? %s (%s), preempt? %s, timeslice? %s\n",
> > yesno(test_bit(TASKLET_STATE_SCHED,
> > - &engine->execlists.tasklet.state)),
> > - enableddisabled(!atomic_read(&engine->execlists.tasklet.count)),
> > + &engine->sched_engine->tasklet.state)),
> > + enableddisabled(!atomic_read(&engine->sched_engine->tasklet.count)),
> > repr_timer(&engine->execlists.preempt),
> > repr_timer(&engine->execlists.timer));
> >
> > @@ -1507,7 +1507,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
> > idx, hws[idx * 2], hws[idx * 2 + 1]);
> > }
> >
> > - execlists_active_lock_bh(execlists);
> > + i915_sched_engine_active_lock_bh(engine->sched_engine);
> > rcu_read_lock();
> > for (port = execlists->active; (rq = *port); port++) {
> > char hdr[160];
> > @@ -1538,7 +1538,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
> > i915_request_show(m, rq, hdr, 0);
> > }
> > rcu_read_unlock();
> > - execlists_active_unlock_bh(execlists);
> > + i915_sched_engine_active_unlock_bh(engine->sched_engine);
> > } else if (INTEL_GEN(dev_priv) > 6) {
> > drm_printf(m, "\tPP_DIR_BASE: 0x%08x\n",
> > ENGINE_READ(engine, RING_PP_DIR_BASE));
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > index f1b14aff5118..6f2fd6f13ac4 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > @@ -138,11 +138,6 @@ struct st_preempt_hang {
> > * driver and the hardware state for execlist mode of submission.
> > */
> > struct intel_engine_execlists {
> > - /**
> > - * @tasklet: softirq tasklet for bottom handler
> > - */
> > - struct tasklet_struct tasklet;
> > -
> > /**
> > * @timer: kick the current context if its timeslice expires
> > */
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 7240c153be35..7a93aec3f6a9 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -570,7 +570,7 @@ static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
> > resubmit_virtual_request(rq, ve);
> >
> > if (READ_ONCE(ve->request))
> > - tasklet_hi_schedule(&ve->base.execlists.tasklet);
> > + tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
> > }
> >
> > static void __execlists_schedule_out(struct i915_request * const rq,
> > @@ -739,9 +739,9 @@ trace_ports(const struct intel_engine_execlists *execlists,
> > }
> >
> > static bool
> > -reset_in_progress(const struct intel_engine_execlists *execlists)
> > +reset_in_progress(const struct intel_engine_cs *engine)
> > {
> > - return unlikely(!__tasklet_is_enabled(&execlists->tasklet));
> > + return unlikely(!__tasklet_is_enabled(&engine->sched_engine->tasklet));
> > }
> >
> > static __maybe_unused noinline bool
> > @@ -757,7 +757,7 @@ assert_pending_valid(const struct intel_engine_execlists *execlists,
> > trace_ports(execlists, msg, execlists->pending);
> >
> > /* We may be messing around with the lists during reset, lalala */
> > - if (reset_in_progress(execlists))
> > + if (reset_in_progress(engine))
> > return true;
> >
> > if (!execlists->pending[0]) {
> > @@ -1190,7 +1190,7 @@ static void start_timeslice(struct intel_engine_cs *engine)
> > * its timeslice, so recheck.
> > */
> > if (!timer_pending(&el->timer))
> > - tasklet_hi_schedule(&el->tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > return;
> > }
> >
> > @@ -1772,8 +1772,8 @@ process_csb(struct intel_engine_cs *engine, struct i915_request **inactive)
> > * access. Either we are inside the tasklet, or the tasklet is disabled
> > * and we assume that is only inside the reset paths and so serialised.
> > */
> > - GEM_BUG_ON(!tasklet_is_locked(&execlists->tasklet) &&
> > - !reset_in_progress(execlists));
> > + GEM_BUG_ON(!tasklet_is_locked(&engine->sched_engine->tasklet) &&
> > + !reset_in_progress(engine));
> >
> > /*
> > * Note that csb_write, csb_status may be either in HWSP or mmio.
> > @@ -2131,7 +2131,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> >
> > if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> > engine->sched_engine->queue_priority_hint = rq_prio(rq);
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > }
> >
> > spin_unlock_irq(&engine->sched_engine->lock);
> > @@ -2322,13 +2322,13 @@ static void execlists_reset(struct intel_engine_cs *engine, const char *msg)
> > ENGINE_TRACE(engine, "reset for %s\n", msg);
> >
> > /* Mark this tasklet as disabled to avoid waiting for it to complete */
> > - tasklet_disable_nosync(&engine->execlists.tasklet);
> > + tasklet_disable_nosync(&engine->sched_engine->tasklet);
> >
> > ring_set_paused(engine, 1); /* Freeze the current request in place */
> > execlists_capture(engine);
> > intel_engine_reset(engine, msg);
> >
> > - tasklet_enable(&engine->execlists.tasklet);
> > + tasklet_enable(&engine->sched_engine->tasklet);
> > clear_and_wake_up_bit(bit, lock);
> > }
> >
> > @@ -2351,8 +2351,9 @@ static bool preempt_timeout(const struct intel_engine_cs *const engine)
> > */
> > static void execlists_submission_tasklet(struct tasklet_struct *t)
> > {
> > - struct intel_engine_cs * const engine =
> > - from_tasklet(engine, t, execlists.tasklet);
> > + struct i915_sched_engine *sched_engine =
> > + from_tasklet(sched_engine, t, tasklet);
> > + struct intel_engine_cs * const engine = sched_engine->engine;
> > struct i915_request *post[2 * EXECLIST_MAX_PORTS];
> > struct i915_request **inactive;
> >
> > @@ -2427,13 +2428,16 @@ static void execlists_irq_handler(struct intel_engine_cs *engine, u16 iir)
> > intel_engine_signal_breadcrumbs(engine);
> >
> > if (tasklet)
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > }
> >
> > static void __execlists_kick(struct intel_engine_execlists *execlists)
> > {
> > + struct intel_engine_cs *engine =
> > + container_of(execlists, typeof(*engine), execlists);
> > +
> > /* Kick the tasklet for some interrupt coalescing and reset handling */
> > - tasklet_hi_schedule(&execlists->tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > }
> >
> > #define execlists_kick(t, member) \
> > @@ -2808,10 +2812,8 @@ static int execlists_resume(struct intel_engine_cs *engine)
> >
> > static void execlists_reset_prepare(struct intel_engine_cs *engine)
> > {
> > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > -
> > ENGINE_TRACE(engine, "depth<-%d\n",
> > - atomic_read(&execlists->tasklet.count));
> > + atomic_read(&engine->sched_engine->tasklet.count));
> >
> > /*
> > * Prevent request submission to the hardware until we have
> > @@ -2822,8 +2824,8 @@ static void execlists_reset_prepare(struct intel_engine_cs *engine)
> > * Turning off the execlists->tasklet until the reset is over
> > * prevents the race.
> > */
> > - __tasklet_disable_sync_once(&execlists->tasklet);
> > - GEM_BUG_ON(!reset_in_progress(execlists));
> > + __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
> > + GEM_BUG_ON(!reset_in_progress(engine));
> >
> > /*
> > * We stop engines, otherwise we might get failed reset and a
> > @@ -2973,8 +2975,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> >
> > static void nop_submission_tasklet(struct tasklet_struct *t)
> > {
> > - struct intel_engine_cs * const engine =
> > - from_tasklet(engine, t, execlists.tasklet);
> > + struct i915_sched_engine *sched_engine =
> > + from_tasklet(sched_engine, t, tasklet);
> > + struct intel_engine_cs * const engine = sched_engine->engine;
> >
> > /* The driver is wedged; don't process any more events. */
> > WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> > @@ -3061,8 +3064,8 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > sched_engine->queue_priority_hint = INT_MIN;
> > sched_engine->queue = RB_ROOT_CACHED;
> >
> > - GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > - execlists->tasklet.callback = nop_submission_tasklet;
> > + GEM_BUG_ON(__tasklet_is_enabled(&engine->sched_engine->tasklet));
> > + engine->sched_engine->tasklet.callback = nop_submission_tasklet;
> >
> > spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > rcu_read_unlock();
> > @@ -3082,14 +3085,14 @@ static void execlists_reset_finish(struct intel_engine_cs *engine)
> > * reset as the next level of recovery, and as a final resort we
> > * will declare the device wedged.
> > */
> > - GEM_BUG_ON(!reset_in_progress(execlists));
> > + GEM_BUG_ON(!reset_in_progress(engine));
> >
> > /* And kick in case we missed a new request submission. */
> > - if (__tasklet_enable(&execlists->tasklet))
> > + if (__tasklet_enable(&engine->sched_engine->tasklet))
> > __execlists_kick(execlists);
> >
> > ENGINE_TRACE(engine, "depth->%d\n",
> > - atomic_read(&execlists->tasklet.count));
> > + atomic_read(&engine->sched_engine->tasklet.count));
> > }
> >
> > static void gen8_logical_ring_enable_irq(struct intel_engine_cs *engine)
> > @@ -3153,8 +3156,6 @@ static void kick_execlists(const struct i915_request *rq, int prio)
> > inflight->fence.context, inflight->fence.seqno,
> > inflight->sched.attr.priority);
> >
> > - sched_engine->queue_priority_hint = prio;
> > -
> > /*
> > * Allow preemption of low -> normal -> high, but we do
> > * not allow low priority tasks to preempt other low priority
> > @@ -3163,7 +3164,7 @@ static void kick_execlists(const struct i915_request *rq, int prio)
> > * so kiss.
> > */
> > if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&sched_engine->tasklet);
> >
> > unlock:
> > rcu_read_unlock();
> > @@ -3174,7 +3175,7 @@ static void execlists_set_default_submission(struct intel_engine_cs *engine)
> > engine->submit_request = execlists_submit_request;
> > engine->sched_engine->schedule = i915_schedule;
> > engine->sched_engine->kick_backend = kick_execlists;
> > - engine->execlists.tasklet.callback = execlists_submission_tasklet;
> > + engine->sched_engine->tasklet.callback = execlists_submission_tasklet;
> > }
> >
> > static void execlists_shutdown(struct intel_engine_cs *engine)
> > @@ -3182,7 +3183,7 @@ static void execlists_shutdown(struct intel_engine_cs *engine)
> > /* Synchronise with residual timers and any softirq they raise */
> > del_timer_sync(&engine->execlists.timer);
> > del_timer_sync(&engine->execlists.preempt);
> > - tasklet_kill(&engine->execlists.tasklet);
> > + tasklet_kill(&engine->sched_engine->tasklet);
> > }
> >
> > static void execlists_release(struct intel_engine_cs *engine)
> > @@ -3298,7 +3299,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> > struct intel_uncore *uncore = engine->uncore;
> > u32 base = engine->mmio_base;
> >
> > - tasklet_setup(&engine->execlists.tasklet, execlists_submission_tasklet);
> > + tasklet_setup(&engine->sched_engine->tasklet, execlists_submission_tasklet);
> > timer_setup(&engine->execlists.timer, execlists_timeslice, 0);
> > timer_setup(&engine->execlists.preempt, execlists_preempt, 0);
> >
> > @@ -3380,7 +3381,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > * rbtrees as in the case it is running in parallel, it may reinsert
> > * the rb_node into a sibling.
> > */
> > - tasklet_kill(&ve->base.execlists.tasklet);
> > + tasklet_kill(&ve->base.sched_engine->tasklet);
> >
> > /* Decouple ourselves from the siblings, no more access allowed. */
> > for (n = 0; n < ve->num_siblings; n++) {
> > @@ -3392,13 +3393,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> >
> > spin_lock_irq(&sibling->sched_engine->lock);
> >
> > - /* Detachment is lazily performed in the execlists tasklet */
> > + /* Detachment is lazily performed in the sched_engine->tasklet */
> > if (!RB_EMPTY_NODE(node))
> > rb_erase_cached(node, &sibling->execlists.virtual);
> >
> > spin_unlock_irq(&sibling->sched_engine->lock);
> > }
> > - GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
> > + GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.sched_engine->tasklet));
> > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> >
> > lrc_fini(&ve->context);
> > @@ -3545,9 +3546,11 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
> >
> > static void virtual_submission_tasklet(struct tasklet_struct *t)
> > {
> > + struct i915_sched_engine *sched_engine =
> > + from_tasklet(sched_engine, t, tasklet);
> > struct virtual_engine * const ve =
> > - from_tasklet(ve, t, base.execlists.tasklet);
> > - const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> > + (struct virtual_engine *)sched_engine->engine;
> > + const int prio = READ_ONCE(sched_engine->queue_priority_hint);
> > intel_engine_mask_t mask;
> > unsigned int n;
> >
> > @@ -3616,7 +3619,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> > node->prio = prio;
> > if (first && prio > sibling->sched_engine->queue_priority_hint)
> > - tasklet_hi_schedule(&sibling->execlists.tasklet);
> > + tasklet_hi_schedule(&sibling->sched_engine->tasklet);
> >
> > unlock_engine:
> > spin_unlock_irq(&sibling->sched_engine->lock);
> > @@ -3657,7 +3660,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > list_move_tail(&rq->sched.link, virtual_queue(ve));
> >
> > - tasklet_hi_schedule(&ve->base.execlists.tasklet);
> > + tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
> >
> > unlock:
> > spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
> > @@ -3751,6 +3754,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > err = -ENOMEM;
> > goto err_put;
> > }
> > + ve->base.sched_engine->engine = &ve->base;
> >
> > ve->base.cops = &virtual_context_ops;
> > ve->base.request_alloc = execlists_request_alloc;
> > @@ -3761,7 +3765,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > ve->base.bond_execute = virtual_bond_execute;
> >
> > INIT_LIST_HEAD(virtual_queue(ve));
> > - tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> > + tasklet_setup(&ve->base.sched_engine->tasklet, virtual_submission_tasklet);
> >
> > intel_context_init(&ve->context, &ve->base);
> >
> > @@ -3789,7 +3793,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > * layering if we handle cloning of the requests and
> > * submitting a copy into each backend.
> > */
> > - if (sibling->execlists.tasklet.callback !=
> > + if (sibling->sched_engine->tasklet.callback !=
> > execlists_submission_tasklet) {
> > err = -ENODEV;
> > goto err_put;
> > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > index a49fd3039f13..bd005c1b6fd5 100644
> > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > @@ -349,6 +349,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> > if (!engine->sched_engine)
> > return -ENOMEM;
> > + engine->sched_engine->engine = engine;
> >
> > intel_engine_init_execlists(engine);
> > intel_engine_init__pm(engine);
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > index 5cc7648d1e5a..2554a2f343b4 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > @@ -43,7 +43,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
> > unsigned long timeout)
> > {
> > /* Ignore our own attempts to suppress excess tasklets */
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> >
> > timeout += jiffies;
> > do {
> > @@ -606,9 +606,9 @@ static int live_hold_reset(void *arg)
> > err = -EBUSY;
> > goto out;
> > }
> > - tasklet_disable(&engine->execlists.tasklet);
> > + tasklet_disable(&engine->sched_engine->tasklet);
> >
> > - engine->execlists.tasklet.callback(&engine->execlists.tasklet);
> > + engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
> > GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
> >
> > i915_request_get(rq);
> > @@ -618,7 +618,7 @@ static int live_hold_reset(void *arg)
> > __intel_engine_reset_bh(engine, NULL);
> > GEM_BUG_ON(rq->fence.error != -EIO);
> >
> > - tasklet_enable(&engine->execlists.tasklet);
> > + tasklet_enable(&engine->sched_engine->tasklet);
> > clear_and_wake_up_bit(I915_RESET_ENGINE + id,
> > >->reset.flags);
> > local_bh_enable();
> > @@ -1183,7 +1183,7 @@ static int live_timeslice_rewind(void *arg)
> > while (i915_request_is_active(rq[A2])) { /* semaphore yield! */
> > /* Wait for the timeslice to kick in */
> > del_timer(&engine->execlists.timer);
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > intel_engine_flush_submission(engine);
> > }
> > /* -> ELSP[] = { { A:rq1 }, { B:rq1 } } */
> > @@ -4593,9 +4593,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
> > err = -EBUSY;
> > goto out_heartbeat;
> > }
> > - tasklet_disable(&engine->execlists.tasklet);
> > + tasklet_disable(&engine->sched_engine->tasklet);
> >
> > - engine->execlists.tasklet.callback(&engine->execlists.tasklet);
> > + engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
> > GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
> >
> > /* Fake a preemption event; failed of course */
> > @@ -4612,7 +4612,7 @@ static int reset_virtual_engine(struct intel_gt *gt,
> > GEM_BUG_ON(rq->fence.error != -EIO);
> >
> > /* Release our grasp on the engine, letting CS flow again */
> > - tasklet_enable(&engine->execlists.tasklet);
> > + tasklet_enable(&engine->sched_engine->tasklet);
> > clear_and_wake_up_bit(I915_RESET_ENGINE + engine->id, >->reset.flags);
> > local_bh_enable();
> >
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > index e57dc900ae8d..cbcb800e2ca0 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > @@ -1702,7 +1702,7 @@ static int __igt_atomic_reset_engine(struct intel_engine_cs *engine,
> > const struct igt_atomic_section *p,
> > const char *mode)
> > {
> > - struct tasklet_struct * const t = &engine->execlists.tasklet;
> > + struct tasklet_struct * const t = &engine->sched_engine->tasklet;
> > int err;
> >
> > GEM_TRACE("i915_reset_engine(%s:%s) under %s\n",
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c
> > index d8f6623524e8..6a3a0d89dcd2 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_lrc.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c
> > @@ -49,7 +49,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
> > unsigned long timeout)
> > {
> > /* Ignore our own attempts to suppress excess tasklets */
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> >
> > timeout += jiffies;
> > do {
> > @@ -1613,12 +1613,12 @@ static void garbage_reset(struct intel_engine_cs *engine,
> >
> > local_bh_disable();
> > if (!test_and_set_bit(bit, lock)) {
> > - tasklet_disable(&engine->execlists.tasklet);
> > + tasklet_disable(&engine->sched_engine->tasklet);
> >
> > if (!rq->fence.error)
> > __intel_engine_reset_bh(engine, NULL);
> >
> > - tasklet_enable(&engine->execlists.tasklet);
> > + tasklet_enable(&engine->sched_engine->tasklet);
> > clear_and_wake_up_bit(bit, lock);
> > }
> > local_bh_enable();
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c
> > index 8784257ec808..7a50c9f4071b 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_reset.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c
> > @@ -321,7 +321,7 @@ static int igt_atomic_engine_reset(void *arg)
> > goto out_unlock;
> >
> > for_each_engine(engine, gt, id) {
> > - struct tasklet_struct *t = &engine->execlists.tasklet;
> > + struct tasklet_struct *t = &engine->sched_engine->tasklet;
> >
> > if (t->func)
> > tasklet_disable(t);
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 7ed21670ef14..95c6f6af4047 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -241,8 +241,9 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> >
> > static void guc_submission_tasklet(struct tasklet_struct *t)
> > {
> > - struct intel_engine_cs * const engine =
> > - from_tasklet(engine, t, execlists.tasklet);
> > + struct i915_sched_engine *sched_engine =
> > + from_tasklet(sched_engine, t, tasklet);
> > + struct intel_engine_cs * const engine = sched_engine->engine;
> > struct intel_engine_execlists * const execlists = &engine->execlists;
> > struct i915_request **port, *rq;
> > unsigned long flags;
> > @@ -272,14 +273,12 @@ static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
> > {
> > if (iir & GT_RENDER_USER_INTERRUPT) {
> > intel_engine_signal_breadcrumbs(engine);
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > }
> > }
> >
> > static void guc_reset_prepare(struct intel_engine_cs *engine)
> > {
> > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > -
> > ENGINE_TRACE(engine, "\n");
> >
> > /*
> > @@ -291,7 +290,7 @@ static void guc_reset_prepare(struct intel_engine_cs *engine)
> > * Turning off the execlists->tasklet until the reset is over
> > * prevents the race.
> > */
> > - __tasklet_disable_sync_once(&execlists->tasklet);
> > + __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
> > }
> >
> > static void guc_reset_state(struct intel_context *ce,
> > @@ -395,14 +394,12 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> >
> > static void guc_reset_finish(struct intel_engine_cs *engine)
> > {
> > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > -
> > - if (__tasklet_enable(&execlists->tasklet))
> > + if (__tasklet_enable(&engine->sched_engine->tasklet))
> > /* And kick in case we missed a new request submission. */
> > - tasklet_hi_schedule(&execlists->tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> >
> > ENGINE_TRACE(engine, "depth->%d\n",
> > - atomic_read(&execlists->tasklet.count));
> > + atomic_read(&engine->sched_engine->tasklet.count));
> > }
> >
> > /*
> > @@ -546,7 +543,7 @@ static void guc_submit_request(struct i915_request *rq)
> > GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
> > GEM_BUG_ON(list_empty(&rq->sched.link));
> >
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> >
> > spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> > @@ -626,7 +623,7 @@ static void guc_release(struct intel_engine_cs *engine)
> > {
> > engine->sanitize = NULL; /* no longer in control, nothing to sanitize */
> >
> > - tasklet_kill(&engine->execlists.tasklet);
> > + tasklet_kill(&engine->sched_engine->tasklet);
> >
> > intel_engine_cleanup_common(engine);
> > lrc_fini_wa_ctx(engine);
> > @@ -705,7 +702,7 @@ int intel_guc_submission_setup(struct intel_engine_cs *engine)
> > */
> > GEM_BUG_ON(INTEL_GEN(i915) < 11);
> >
> > - tasklet_setup(&engine->execlists.tasklet, guc_submission_tasklet);
> > + tasklet_setup(&engine->sched_engine->tasklet, guc_submission_tasklet);
> >
> > guc_default_vfuncs(engine);
> > guc_default_irqs(engine);
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > index 3d36e4447b5d..6f082579ee9e 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > @@ -436,6 +436,7 @@ void i915_sched_engine_free(struct kref *kref)
> > struct i915_sched_engine *sched_engine =
> > container_of(kref, typeof(*sched_engine), ref);
> >
> > + tasklet_kill(&sched_engine->tasklet); /* flush the callback */
> > kfree(sched_engine);
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > index 0014745bda30..650ab8e0db9f 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > @@ -79,6 +79,20 @@ i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
> > sched_engine->no_priolist = false;
> > }
> >
> > +static inline void
> > +i915_sched_engine_active_lock_bh(struct i915_sched_engine *sched_engine)
> > +{
> > + local_bh_disable(); /* prevent local softirq and lock recursion */
> > + tasklet_lock(&sched_engine->tasklet);
> > +}
> > +
> > +static inline void
> > +i915_sched_engine_active_unlock_bh(struct i915_sched_engine *sched_engine)
> > +{
> > + tasklet_unlock(&sched_engine->tasklet);
> > + local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
> > +}
> > +
> > void i915_request_show_with_schedule(struct drm_printer *m,
> > const struct i915_request *rq,
> > const char *prefix,
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 8b3af536e6cf..9d79514450de 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > @@ -103,6 +103,11 @@ struct i915_sched_engine {
> > struct list_head requests;
> > struct list_head hold; /* ready requests, but on hold */
> >
> > + /**
> > + * @tasklet: softirq tasklet for bottom handler
> > + */
> > + struct tasklet_struct tasklet;
> > +
> > /**
> > * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > */
> > @@ -132,6 +137,9 @@ struct i915_sched_engine {
> > */
> > bool no_priolist;
> >
> > + /* Back pointer to engine */
> > + struct intel_engine_cs *engine;
> > +
> > /* Kick backend */
> > void (*kick_backend)(const struct i915_request *rq,
> > int prio);
> > --
> > 2.28.0
> >
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 8/9] drm/i915: Move submission tasklet to i915_sched_engine
@ 2021-06-04 19:55 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 19:55 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 02:26:38PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > The submission tasklet operates on i915_sched_engine, thus it is the
> > correct place for it.
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gt/intel_engine.h | 14 ---
> > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 12 +--
> > drivers/gpu/drm/i915/gt/intel_engine_types.h | 5 --
> > .../drm/i915/gt/intel_execlists_submission.c | 86 ++++++++++---------
> > drivers/gpu/drm/i915/gt/mock_engine.c | 1 +
> > drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++--
> > drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 2 +-
> > drivers/gpu/drm/i915/gt/selftest_lrc.c | 6 +-
> > drivers/gpu/drm/i915/gt/selftest_reset.c | 2 +-
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 25 +++---
> > drivers/gpu/drm/i915/i915_scheduler.c | 1 +
> > drivers/gpu/drm/i915/i915_scheduler.h | 14 +++
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++
> > 13 files changed, 99 insertions(+), 93 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
> > index a8b2174b4395..988d9688ae4d 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
> > @@ -123,20 +123,6 @@ execlists_active(const struct intel_engine_execlists *execlists)
> > return active;
> > }
> >
> > -static inline void
> > -execlists_active_lock_bh(struct intel_engine_execlists *execlists)
> > -{
> > - local_bh_disable(); /* prevent local softirq and lock recursion */
> > - tasklet_lock(&execlists->tasklet);
> > -}
> > -
> > -static inline void
> > -execlists_active_unlock_bh(struct intel_engine_execlists *execlists)
> > -{
> > - tasklet_unlock(&execlists->tasklet);
> > - local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
> > -}
> > -
> > struct i915_request *
> > execlists_unwind_incomplete_requests(struct intel_engine_execlists *execlists);
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > index b480fcb1aad1..837d2132e31b 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > @@ -711,6 +711,7 @@ static int engine_setup_common(struct intel_engine_cs *engine)
> > err = -ENOMEM;
> > goto err_sched_engine;
> > }
> > + engine->sched_engine->engine = engine;
>
> This bothers me. If we're trying to separate things, why do we have a
> back pointer? Yes, I know it's because we need to access our engine
> back-end somehow. Makes me wonder if it should be some sort of a
> void* private data pointer instead of the engine. That's probably
> less safe but it seems more "right" in some sense? I don't know.
>
This was new moving from internal to upstream because of the new tasklet
API that has landed. Previously the tasklet function was setup with u64
data which we used to point to the engine. Now it uses a callback with
the tasklet structure as the argument and can only use a container_of to
resolve the object which in this case a i915_sched_engine. In execlists
we still need the engine object to do the submit, hence the back
pointer.
Longterm we really should have an execlists_submit_engine which
encapsulates the i915_sched_engine and everything we need for execlists
submission (and with DRM scheduler the i915_sched_engine encapsulates
the drm_gpu_scheduler object and common variables between execlists and
GuC).
I already have an object like this for the GuC, guc_submit_eninge,
albeit slightly out of order in patch series.
Agree void* private is better as for the GuC code we likely need this to
be a backpointer to the GuC structure until the aforementioned
guc_submit_engine lands.
Are you ok with landing this with a void* and taking a note this needs
to be reworked?
> Also, dumb question but what's a tasklet?
>
All submissions in the current i915 go through a tasklet. The GuC
submission that will initially land uses this too. Once the DRM
scheduler lands this will go away, at least for GuC submission. If this
is still around for execlists we likely stick this in
execlists_submit_engine object and forget about it.
Matt
> --Jason
>
> >
> > err = intel_engine_init_cmd_parser(engine);
> > if (err)
> > @@ -935,7 +936,6 @@ int intel_engines_init(struct intel_gt *gt)
> > void intel_engine_cleanup_common(struct intel_engine_cs *engine)
> > {
> > GEM_BUG_ON(!list_empty(&engine->sched_engine->requests));
> > - tasklet_kill(&engine->execlists.tasklet); /* flush the callback */
> >
> > i915_sched_engine_put(engine->sched_engine);
> > intel_breadcrumbs_free(engine->breadcrumbs);
> > @@ -1221,7 +1221,7 @@ static bool ring_is_idle(struct intel_engine_cs *engine)
> >
> > void __intel_engine_flush_submission(struct intel_engine_cs *engine, bool sync)
> > {
> > - struct tasklet_struct *t = &engine->execlists.tasklet;
> > + struct tasklet_struct *t = &engine->sched_engine->tasklet;
> >
> > if (!t->callback)
> > return;
> > @@ -1482,8 +1482,8 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
> >
> > drm_printf(m, "\tExeclist tasklet queued? %s (%s), preempt? %s, timeslice? %s\n",
> > yesno(test_bit(TASKLET_STATE_SCHED,
> > - &engine->execlists.tasklet.state)),
> > - enableddisabled(!atomic_read(&engine->execlists.tasklet.count)),
> > + &engine->sched_engine->tasklet.state)),
> > + enableddisabled(!atomic_read(&engine->sched_engine->tasklet.count)),
> > repr_timer(&engine->execlists.preempt),
> > repr_timer(&engine->execlists.timer));
> >
> > @@ -1507,7 +1507,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
> > idx, hws[idx * 2], hws[idx * 2 + 1]);
> > }
> >
> > - execlists_active_lock_bh(execlists);
> > + i915_sched_engine_active_lock_bh(engine->sched_engine);
> > rcu_read_lock();
> > for (port = execlists->active; (rq = *port); port++) {
> > char hdr[160];
> > @@ -1538,7 +1538,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine,
> > i915_request_show(m, rq, hdr, 0);
> > }
> > rcu_read_unlock();
> > - execlists_active_unlock_bh(execlists);
> > + i915_sched_engine_active_unlock_bh(engine->sched_engine);
> > } else if (INTEL_GEN(dev_priv) > 6) {
> > drm_printf(m, "\tPP_DIR_BASE: 0x%08x\n",
> > ENGINE_READ(engine, RING_PP_DIR_BASE));
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > index f1b14aff5118..6f2fd6f13ac4 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > @@ -138,11 +138,6 @@ struct st_preempt_hang {
> > * driver and the hardware state for execlist mode of submission.
> > */
> > struct intel_engine_execlists {
> > - /**
> > - * @tasklet: softirq tasklet for bottom handler
> > - */
> > - struct tasklet_struct tasklet;
> > -
> > /**
> > * @timer: kick the current context if its timeslice expires
> > */
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 7240c153be35..7a93aec3f6a9 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -570,7 +570,7 @@ static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
> > resubmit_virtual_request(rq, ve);
> >
> > if (READ_ONCE(ve->request))
> > - tasklet_hi_schedule(&ve->base.execlists.tasklet);
> > + tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
> > }
> >
> > static void __execlists_schedule_out(struct i915_request * const rq,
> > @@ -739,9 +739,9 @@ trace_ports(const struct intel_engine_execlists *execlists,
> > }
> >
> > static bool
> > -reset_in_progress(const struct intel_engine_execlists *execlists)
> > +reset_in_progress(const struct intel_engine_cs *engine)
> > {
> > - return unlikely(!__tasklet_is_enabled(&execlists->tasklet));
> > + return unlikely(!__tasklet_is_enabled(&engine->sched_engine->tasklet));
> > }
> >
> > static __maybe_unused noinline bool
> > @@ -757,7 +757,7 @@ assert_pending_valid(const struct intel_engine_execlists *execlists,
> > trace_ports(execlists, msg, execlists->pending);
> >
> > /* We may be messing around with the lists during reset, lalala */
> > - if (reset_in_progress(execlists))
> > + if (reset_in_progress(engine))
> > return true;
> >
> > if (!execlists->pending[0]) {
> > @@ -1190,7 +1190,7 @@ static void start_timeslice(struct intel_engine_cs *engine)
> > * its timeslice, so recheck.
> > */
> > if (!timer_pending(&el->timer))
> > - tasklet_hi_schedule(&el->tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > return;
> > }
> >
> > @@ -1772,8 +1772,8 @@ process_csb(struct intel_engine_cs *engine, struct i915_request **inactive)
> > * access. Either we are inside the tasklet, or the tasklet is disabled
> > * and we assume that is only inside the reset paths and so serialised.
> > */
> > - GEM_BUG_ON(!tasklet_is_locked(&execlists->tasklet) &&
> > - !reset_in_progress(execlists));
> > + GEM_BUG_ON(!tasklet_is_locked(&engine->sched_engine->tasklet) &&
> > + !reset_in_progress(engine));
> >
> > /*
> > * Note that csb_write, csb_status may be either in HWSP or mmio.
> > @@ -2131,7 +2131,7 @@ static void execlists_unhold(struct intel_engine_cs *engine,
> >
> > if (rq_prio(rq) > engine->sched_engine->queue_priority_hint) {
> > engine->sched_engine->queue_priority_hint = rq_prio(rq);
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > }
> >
> > spin_unlock_irq(&engine->sched_engine->lock);
> > @@ -2322,13 +2322,13 @@ static void execlists_reset(struct intel_engine_cs *engine, const char *msg)
> > ENGINE_TRACE(engine, "reset for %s\n", msg);
> >
> > /* Mark this tasklet as disabled to avoid waiting for it to complete */
> > - tasklet_disable_nosync(&engine->execlists.tasklet);
> > + tasklet_disable_nosync(&engine->sched_engine->tasklet);
> >
> > ring_set_paused(engine, 1); /* Freeze the current request in place */
> > execlists_capture(engine);
> > intel_engine_reset(engine, msg);
> >
> > - tasklet_enable(&engine->execlists.tasklet);
> > + tasklet_enable(&engine->sched_engine->tasklet);
> > clear_and_wake_up_bit(bit, lock);
> > }
> >
> > @@ -2351,8 +2351,9 @@ static bool preempt_timeout(const struct intel_engine_cs *const engine)
> > */
> > static void execlists_submission_tasklet(struct tasklet_struct *t)
> > {
> > - struct intel_engine_cs * const engine =
> > - from_tasklet(engine, t, execlists.tasklet);
> > + struct i915_sched_engine *sched_engine =
> > + from_tasklet(sched_engine, t, tasklet);
> > + struct intel_engine_cs * const engine = sched_engine->engine;
> > struct i915_request *post[2 * EXECLIST_MAX_PORTS];
> > struct i915_request **inactive;
> >
> > @@ -2427,13 +2428,16 @@ static void execlists_irq_handler(struct intel_engine_cs *engine, u16 iir)
> > intel_engine_signal_breadcrumbs(engine);
> >
> > if (tasklet)
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > }
> >
> > static void __execlists_kick(struct intel_engine_execlists *execlists)
> > {
> > + struct intel_engine_cs *engine =
> > + container_of(execlists, typeof(*engine), execlists);
> > +
> > /* Kick the tasklet for some interrupt coalescing and reset handling */
> > - tasklet_hi_schedule(&execlists->tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > }
> >
> > #define execlists_kick(t, member) \
> > @@ -2808,10 +2812,8 @@ static int execlists_resume(struct intel_engine_cs *engine)
> >
> > static void execlists_reset_prepare(struct intel_engine_cs *engine)
> > {
> > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > -
> > ENGINE_TRACE(engine, "depth<-%d\n",
> > - atomic_read(&execlists->tasklet.count));
> > + atomic_read(&engine->sched_engine->tasklet.count));
> >
> > /*
> > * Prevent request submission to the hardware until we have
> > @@ -2822,8 +2824,8 @@ static void execlists_reset_prepare(struct intel_engine_cs *engine)
> > * Turning off the execlists->tasklet until the reset is over
> > * prevents the race.
> > */
> > - __tasklet_disable_sync_once(&execlists->tasklet);
> > - GEM_BUG_ON(!reset_in_progress(execlists));
> > + __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
> > + GEM_BUG_ON(!reset_in_progress(engine));
> >
> > /*
> > * We stop engines, otherwise we might get failed reset and a
> > @@ -2973,8 +2975,9 @@ static void execlists_reset_rewind(struct intel_engine_cs *engine, bool stalled)
> >
> > static void nop_submission_tasklet(struct tasklet_struct *t)
> > {
> > - struct intel_engine_cs * const engine =
> > - from_tasklet(engine, t, execlists.tasklet);
> > + struct i915_sched_engine *sched_engine =
> > + from_tasklet(sched_engine, t, tasklet);
> > + struct intel_engine_cs * const engine = sched_engine->engine;
> >
> > /* The driver is wedged; don't process any more events. */
> > WRITE_ONCE(engine->sched_engine->queue_priority_hint, INT_MIN);
> > @@ -3061,8 +3064,8 @@ static void execlists_reset_cancel(struct intel_engine_cs *engine)
> > sched_engine->queue_priority_hint = INT_MIN;
> > sched_engine->queue = RB_ROOT_CACHED;
> >
> > - GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
> > - execlists->tasklet.callback = nop_submission_tasklet;
> > + GEM_BUG_ON(__tasklet_is_enabled(&engine->sched_engine->tasklet));
> > + engine->sched_engine->tasklet.callback = nop_submission_tasklet;
> >
> > spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > rcu_read_unlock();
> > @@ -3082,14 +3085,14 @@ static void execlists_reset_finish(struct intel_engine_cs *engine)
> > * reset as the next level of recovery, and as a final resort we
> > * will declare the device wedged.
> > */
> > - GEM_BUG_ON(!reset_in_progress(execlists));
> > + GEM_BUG_ON(!reset_in_progress(engine));
> >
> > /* And kick in case we missed a new request submission. */
> > - if (__tasklet_enable(&execlists->tasklet))
> > + if (__tasklet_enable(&engine->sched_engine->tasklet))
> > __execlists_kick(execlists);
> >
> > ENGINE_TRACE(engine, "depth->%d\n",
> > - atomic_read(&execlists->tasklet.count));
> > + atomic_read(&engine->sched_engine->tasklet.count));
> > }
> >
> > static void gen8_logical_ring_enable_irq(struct intel_engine_cs *engine)
> > @@ -3153,8 +3156,6 @@ static void kick_execlists(const struct i915_request *rq, int prio)
> > inflight->fence.context, inflight->fence.seqno,
> > inflight->sched.attr.priority);
> >
> > - sched_engine->queue_priority_hint = prio;
> > -
> > /*
> > * Allow preemption of low -> normal -> high, but we do
> > * not allow low priority tasks to preempt other low priority
> > @@ -3163,7 +3164,7 @@ static void kick_execlists(const struct i915_request *rq, int prio)
> > * so kiss.
> > */
> > if (prio >= max(I915_PRIORITY_NORMAL, rq_prio(inflight)))
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&sched_engine->tasklet);
> >
> > unlock:
> > rcu_read_unlock();
> > @@ -3174,7 +3175,7 @@ static void execlists_set_default_submission(struct intel_engine_cs *engine)
> > engine->submit_request = execlists_submit_request;
> > engine->sched_engine->schedule = i915_schedule;
> > engine->sched_engine->kick_backend = kick_execlists;
> > - engine->execlists.tasklet.callback = execlists_submission_tasklet;
> > + engine->sched_engine->tasklet.callback = execlists_submission_tasklet;
> > }
> >
> > static void execlists_shutdown(struct intel_engine_cs *engine)
> > @@ -3182,7 +3183,7 @@ static void execlists_shutdown(struct intel_engine_cs *engine)
> > /* Synchronise with residual timers and any softirq they raise */
> > del_timer_sync(&engine->execlists.timer);
> > del_timer_sync(&engine->execlists.preempt);
> > - tasklet_kill(&engine->execlists.tasklet);
> > + tasklet_kill(&engine->sched_engine->tasklet);
> > }
> >
> > static void execlists_release(struct intel_engine_cs *engine)
> > @@ -3298,7 +3299,7 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine)
> > struct intel_uncore *uncore = engine->uncore;
> > u32 base = engine->mmio_base;
> >
> > - tasklet_setup(&engine->execlists.tasklet, execlists_submission_tasklet);
> > + tasklet_setup(&engine->sched_engine->tasklet, execlists_submission_tasklet);
> > timer_setup(&engine->execlists.timer, execlists_timeslice, 0);
> > timer_setup(&engine->execlists.preempt, execlists_preempt, 0);
> >
> > @@ -3380,7 +3381,7 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> > * rbtrees as in the case it is running in parallel, it may reinsert
> > * the rb_node into a sibling.
> > */
> > - tasklet_kill(&ve->base.execlists.tasklet);
> > + tasklet_kill(&ve->base.sched_engine->tasklet);
> >
> > /* Decouple ourselves from the siblings, no more access allowed. */
> > for (n = 0; n < ve->num_siblings; n++) {
> > @@ -3392,13 +3393,13 @@ static void rcu_virtual_context_destroy(struct work_struct *wrk)
> >
> > spin_lock_irq(&sibling->sched_engine->lock);
> >
> > - /* Detachment is lazily performed in the execlists tasklet */
> > + /* Detachment is lazily performed in the sched_engine->tasklet */
> > if (!RB_EMPTY_NODE(node))
> > rb_erase_cached(node, &sibling->execlists.virtual);
> >
> > spin_unlock_irq(&sibling->sched_engine->lock);
> > }
> > - GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.execlists.tasklet));
> > + GEM_BUG_ON(__tasklet_is_scheduled(&ve->base.sched_engine->tasklet));
> > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> >
> > lrc_fini(&ve->context);
> > @@ -3545,9 +3546,11 @@ static intel_engine_mask_t virtual_submission_mask(struct virtual_engine *ve)
> >
> > static void virtual_submission_tasklet(struct tasklet_struct *t)
> > {
> > + struct i915_sched_engine *sched_engine =
> > + from_tasklet(sched_engine, t, tasklet);
> > struct virtual_engine * const ve =
> > - from_tasklet(ve, t, base.execlists.tasklet);
> > - const int prio = READ_ONCE(ve->base.sched_engine->queue_priority_hint);
> > + (struct virtual_engine *)sched_engine->engine;
> > + const int prio = READ_ONCE(sched_engine->queue_priority_hint);
> > intel_engine_mask_t mask;
> > unsigned int n;
> >
> > @@ -3616,7 +3619,7 @@ static void virtual_submission_tasklet(struct tasklet_struct *t)
> > GEM_BUG_ON(RB_EMPTY_NODE(&node->rb));
> > node->prio = prio;
> > if (first && prio > sibling->sched_engine->queue_priority_hint)
> > - tasklet_hi_schedule(&sibling->execlists.tasklet);
> > + tasklet_hi_schedule(&sibling->sched_engine->tasklet);
> >
> > unlock_engine:
> > spin_unlock_irq(&sibling->sched_engine->lock);
> > @@ -3657,7 +3660,7 @@ static void virtual_submit_request(struct i915_request *rq)
> > GEM_BUG_ON(!list_empty(virtual_queue(ve)));
> > list_move_tail(&rq->sched.link, virtual_queue(ve));
> >
> > - tasklet_hi_schedule(&ve->base.execlists.tasklet);
> > + tasklet_hi_schedule(&ve->base.sched_engine->tasklet);
> >
> > unlock:
> > spin_unlock_irqrestore(&ve->base.sched_engine->lock, flags);
> > @@ -3751,6 +3754,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > err = -ENOMEM;
> > goto err_put;
> > }
> > + ve->base.sched_engine->engine = &ve->base;
> >
> > ve->base.cops = &virtual_context_ops;
> > ve->base.request_alloc = execlists_request_alloc;
> > @@ -3761,7 +3765,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > ve->base.bond_execute = virtual_bond_execute;
> >
> > INIT_LIST_HEAD(virtual_queue(ve));
> > - tasklet_setup(&ve->base.execlists.tasklet, virtual_submission_tasklet);
> > + tasklet_setup(&ve->base.sched_engine->tasklet, virtual_submission_tasklet);
> >
> > intel_context_init(&ve->context, &ve->base);
> >
> > @@ -3789,7 +3793,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > * layering if we handle cloning of the requests and
> > * submitting a copy into each backend.
> > */
> > - if (sibling->execlists.tasklet.callback !=
> > + if (sibling->sched_engine->tasklet.callback !=
> > execlists_submission_tasklet) {
> > err = -ENODEV;
> > goto err_put;
> > diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
> > index a49fd3039f13..bd005c1b6fd5 100644
> > --- a/drivers/gpu/drm/i915/gt/mock_engine.c
> > +++ b/drivers/gpu/drm/i915/gt/mock_engine.c
> > @@ -349,6 +349,7 @@ int mock_engine_init(struct intel_engine_cs *engine)
> > engine->sched_engine = i915_sched_engine_create(ENGINE_MOCK);
> > if (!engine->sched_engine)
> > return -ENOMEM;
> > + engine->sched_engine->engine = engine;
> >
> > intel_engine_init_execlists(engine);
> > intel_engine_init__pm(engine);
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > index 5cc7648d1e5a..2554a2f343b4 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > @@ -43,7 +43,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
> > unsigned long timeout)
> > {
> > /* Ignore our own attempts to suppress excess tasklets */
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> >
> > timeout += jiffies;
> > do {
> > @@ -606,9 +606,9 @@ static int live_hold_reset(void *arg)
> > err = -EBUSY;
> > goto out;
> > }
> > - tasklet_disable(&engine->execlists.tasklet);
> > + tasklet_disable(&engine->sched_engine->tasklet);
> >
> > - engine->execlists.tasklet.callback(&engine->execlists.tasklet);
> > + engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
> > GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
> >
> > i915_request_get(rq);
> > @@ -618,7 +618,7 @@ static int live_hold_reset(void *arg)
> > __intel_engine_reset_bh(engine, NULL);
> > GEM_BUG_ON(rq->fence.error != -EIO);
> >
> > - tasklet_enable(&engine->execlists.tasklet);
> > + tasklet_enable(&engine->sched_engine->tasklet);
> > clear_and_wake_up_bit(I915_RESET_ENGINE + id,
> > >->reset.flags);
> > local_bh_enable();
> > @@ -1183,7 +1183,7 @@ static int live_timeslice_rewind(void *arg)
> > while (i915_request_is_active(rq[A2])) { /* semaphore yield! */
> > /* Wait for the timeslice to kick in */
> > del_timer(&engine->execlists.timer);
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > intel_engine_flush_submission(engine);
> > }
> > /* -> ELSP[] = { { A:rq1 }, { B:rq1 } } */
> > @@ -4593,9 +4593,9 @@ static int reset_virtual_engine(struct intel_gt *gt,
> > err = -EBUSY;
> > goto out_heartbeat;
> > }
> > - tasklet_disable(&engine->execlists.tasklet);
> > + tasklet_disable(&engine->sched_engine->tasklet);
> >
> > - engine->execlists.tasklet.callback(&engine->execlists.tasklet);
> > + engine->sched_engine->tasklet.callback(&engine->sched_engine->tasklet);
> > GEM_BUG_ON(execlists_active(&engine->execlists) != rq);
> >
> > /* Fake a preemption event; failed of course */
> > @@ -4612,7 +4612,7 @@ static int reset_virtual_engine(struct intel_gt *gt,
> > GEM_BUG_ON(rq->fence.error != -EIO);
> >
> > /* Release our grasp on the engine, letting CS flow again */
> > - tasklet_enable(&engine->execlists.tasklet);
> > + tasklet_enable(&engine->sched_engine->tasklet);
> > clear_and_wake_up_bit(I915_RESET_ENGINE + engine->id, >->reset.flags);
> > local_bh_enable();
> >
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > index e57dc900ae8d..cbcb800e2ca0 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > @@ -1702,7 +1702,7 @@ static int __igt_atomic_reset_engine(struct intel_engine_cs *engine,
> > const struct igt_atomic_section *p,
> > const char *mode)
> > {
> > - struct tasklet_struct * const t = &engine->execlists.tasklet;
> > + struct tasklet_struct * const t = &engine->sched_engine->tasklet;
> > int err;
> >
> > GEM_TRACE("i915_reset_engine(%s:%s) under %s\n",
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c
> > index d8f6623524e8..6a3a0d89dcd2 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_lrc.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c
> > @@ -49,7 +49,7 @@ static int wait_for_submit(struct intel_engine_cs *engine,
> > unsigned long timeout)
> > {
> > /* Ignore our own attempts to suppress excess tasklets */
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> >
> > timeout += jiffies;
> > do {
> > @@ -1613,12 +1613,12 @@ static void garbage_reset(struct intel_engine_cs *engine,
> >
> > local_bh_disable();
> > if (!test_and_set_bit(bit, lock)) {
> > - tasklet_disable(&engine->execlists.tasklet);
> > + tasklet_disable(&engine->sched_engine->tasklet);
> >
> > if (!rq->fence.error)
> > __intel_engine_reset_bh(engine, NULL);
> >
> > - tasklet_enable(&engine->execlists.tasklet);
> > + tasklet_enable(&engine->sched_engine->tasklet);
> > clear_and_wake_up_bit(bit, lock);
> > }
> > local_bh_enable();
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_reset.c b/drivers/gpu/drm/i915/gt/selftest_reset.c
> > index 8784257ec808..7a50c9f4071b 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_reset.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_reset.c
> > @@ -321,7 +321,7 @@ static int igt_atomic_engine_reset(void *arg)
> > goto out_unlock;
> >
> > for_each_engine(engine, gt, id) {
> > - struct tasklet_struct *t = &engine->execlists.tasklet;
> > + struct tasklet_struct *t = &engine->sched_engine->tasklet;
> >
> > if (t->func)
> > tasklet_disable(t);
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 7ed21670ef14..95c6f6af4047 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -241,8 +241,9 @@ static void __guc_dequeue(struct intel_engine_cs *engine)
> >
> > static void guc_submission_tasklet(struct tasklet_struct *t)
> > {
> > - struct intel_engine_cs * const engine =
> > - from_tasklet(engine, t, execlists.tasklet);
> > + struct i915_sched_engine *sched_engine =
> > + from_tasklet(sched_engine, t, tasklet);
> > + struct intel_engine_cs * const engine = sched_engine->engine;
> > struct intel_engine_execlists * const execlists = &engine->execlists;
> > struct i915_request **port, *rq;
> > unsigned long flags;
> > @@ -272,14 +273,12 @@ static void cs_irq_handler(struct intel_engine_cs *engine, u16 iir)
> > {
> > if (iir & GT_RENDER_USER_INTERRUPT) {
> > intel_engine_signal_breadcrumbs(engine);
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> > }
> > }
> >
> > static void guc_reset_prepare(struct intel_engine_cs *engine)
> > {
> > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > -
> > ENGINE_TRACE(engine, "\n");
> >
> > /*
> > @@ -291,7 +290,7 @@ static void guc_reset_prepare(struct intel_engine_cs *engine)
> > * Turning off the execlists->tasklet until the reset is over
> > * prevents the race.
> > */
> > - __tasklet_disable_sync_once(&execlists->tasklet);
> > + __tasklet_disable_sync_once(&engine->sched_engine->tasklet);
> > }
> >
> > static void guc_reset_state(struct intel_context *ce,
> > @@ -395,14 +394,12 @@ static void guc_reset_cancel(struct intel_engine_cs *engine)
> >
> > static void guc_reset_finish(struct intel_engine_cs *engine)
> > {
> > - struct intel_engine_execlists * const execlists = &engine->execlists;
> > -
> > - if (__tasklet_enable(&execlists->tasklet))
> > + if (__tasklet_enable(&engine->sched_engine->tasklet))
> > /* And kick in case we missed a new request submission. */
> > - tasklet_hi_schedule(&execlists->tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> >
> > ENGINE_TRACE(engine, "depth->%d\n",
> > - atomic_read(&execlists->tasklet.count));
> > + atomic_read(&engine->sched_engine->tasklet.count));
> > }
> >
> > /*
> > @@ -546,7 +543,7 @@ static void guc_submit_request(struct i915_request *rq)
> > GEM_BUG_ON(i915_sched_engine_is_empty(engine->sched_engine));
> > GEM_BUG_ON(list_empty(&rq->sched.link));
> >
> > - tasklet_hi_schedule(&engine->execlists.tasklet);
> > + tasklet_hi_schedule(&engine->sched_engine->tasklet);
> >
> > spin_unlock_irqrestore(&engine->sched_engine->lock, flags);
> > }
> > @@ -626,7 +623,7 @@ static void guc_release(struct intel_engine_cs *engine)
> > {
> > engine->sanitize = NULL; /* no longer in control, nothing to sanitize */
> >
> > - tasklet_kill(&engine->execlists.tasklet);
> > + tasklet_kill(&engine->sched_engine->tasklet);
> >
> > intel_engine_cleanup_common(engine);
> > lrc_fini_wa_ctx(engine);
> > @@ -705,7 +702,7 @@ int intel_guc_submission_setup(struct intel_engine_cs *engine)
> > */
> > GEM_BUG_ON(INTEL_GEN(i915) < 11);
> >
> > - tasklet_setup(&engine->execlists.tasklet, guc_submission_tasklet);
> > + tasklet_setup(&engine->sched_engine->tasklet, guc_submission_tasklet);
> >
> > guc_default_vfuncs(engine);
> > guc_default_irqs(engine);
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > index 3d36e4447b5d..6f082579ee9e 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > @@ -436,6 +436,7 @@ void i915_sched_engine_free(struct kref *kref)
> > struct i915_sched_engine *sched_engine =
> > container_of(kref, typeof(*sched_engine), ref);
> >
> > + tasklet_kill(&sched_engine->tasklet); /* flush the callback */
> > kfree(sched_engine);
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > index 0014745bda30..650ab8e0db9f 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > @@ -79,6 +79,20 @@ i915_sched_engine_reset_on_empty(struct i915_sched_engine *sched_engine)
> > sched_engine->no_priolist = false;
> > }
> >
> > +static inline void
> > +i915_sched_engine_active_lock_bh(struct i915_sched_engine *sched_engine)
> > +{
> > + local_bh_disable(); /* prevent local softirq and lock recursion */
> > + tasklet_lock(&sched_engine->tasklet);
> > +}
> > +
> > +static inline void
> > +i915_sched_engine_active_unlock_bh(struct i915_sched_engine *sched_engine)
> > +{
> > + tasklet_unlock(&sched_engine->tasklet);
> > + local_bh_enable(); /* restore softirq, and kick ksoftirqd! */
> > +}
> > +
> > void i915_request_show_with_schedule(struct drm_printer *m,
> > const struct i915_request *rq,
> > const char *prefix,
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 8b3af536e6cf..9d79514450de 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > @@ -103,6 +103,11 @@ struct i915_sched_engine {
> > struct list_head requests;
> > struct list_head hold; /* ready requests, but on hold */
> >
> > + /**
> > + * @tasklet: softirq tasklet for bottom handler
> > + */
> > + struct tasklet_struct tasklet;
> > +
> > /**
> > * @default_priolist: priority list for I915_PRIORITY_NORMAL
> > */
> > @@ -132,6 +137,9 @@ struct i915_sched_engine {
> > */
> > bool no_priolist;
> >
> > + /* Back pointer to engine */
> > + struct intel_engine_cs *engine;
> > +
> > /* Kick backend */
> > void (*kick_backend)(const struct i915_request *rq,
> > int prio);
> > --
> > 2.28.0
> >
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 5/9] drm/i915: Move engine->schedule to i915_sched_engine
2021-06-04 19:03 ` Jason Ekstrand
@ 2021-06-04 20:06 ` Matthew Brost
-1 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 20:06 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 02:03:46PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > The schedule function should be in the schedule object.
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gem/i915_gem_wait.c | 4 ++--
> > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 3 ---
> > drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++--
> > drivers/gpu/drm/i915/gt/intel_engine_types.h | 8 --------
> > drivers/gpu/drm/i915/gt/intel_engine_user.c | 2 +-
> > .../gpu/drm/i915/gt/intel_execlists_submission.c | 4 ++--
> > drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++++++++--------
> > drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 4 ++--
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
> > drivers/gpu/drm/i915/i915_request.c | 10 +++++-----
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++++++++
> > 11 files changed, 31 insertions(+), 34 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_wait.c b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> > index 4b9856d5ba14..af1fbf8e2a9a 100644
> > --- a/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> > +++ b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> > @@ -104,8 +104,8 @@ static void fence_set_priority(struct dma_fence *fence,
> > engine = rq->engine;
> >
> > rcu_read_lock(); /* RCU serialisation for set-wedged protection */
> > - if (engine->schedule)
> > - engine->schedule(rq, attr);
> > + if (engine->sched_engine->schedule)
> > + engine->sched_engine->schedule(rq, attr);
> > rcu_read_unlock();
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > index 33d879137908..b480fcb1aad1 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > @@ -326,9 +326,6 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
> > if (engine->context_size)
> > DRIVER_CAPS(i915)->has_logical_contexts = true;
> >
> > - /* Nothing to do here, execute in order of dependencies */
> > - engine->schedule = NULL;
> > -
> > ewma__engine_latency_init(&engine->latency);
> > seqcount_init(&engine->stats.lock);
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> > index b99ac41695f3..b6a305e6a974 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> > @@ -121,7 +121,7 @@ static void heartbeat(struct work_struct *wrk)
> > * but all other contexts, including the kernel
> > * context are stuck waiting for the signal.
> > */
> > - } else if (engine->schedule &&
> > + } else if (engine->sched_engine->schedule &&
> > rq->sched.attr.priority < I915_PRIORITY_BARRIER) {
> > /*
> > * Gradually raise the priority of the heartbeat to
> > @@ -136,7 +136,7 @@ static void heartbeat(struct work_struct *wrk)
> > attr.priority = I915_PRIORITY_BARRIER;
> >
> > local_bh_disable();
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> > local_bh_enable();
> > } else {
> > if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > index 7197b9fa5e35..f1b14aff5118 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > @@ -426,14 +426,6 @@ struct intel_engine_cs {
> > void (*bond_execute)(struct i915_request *rq,
> > struct dma_fence *signal);
> >
> > - /*
> > - * Call when the priority on a request has changed and it and its
> > - * dependencies may need rescheduling. Note the request itself may
> > - * not be ready to run!
> > - */
> > - void (*schedule)(struct i915_request *request,
> > - const struct i915_sched_attr *attr);
> > -
> > void (*release)(struct intel_engine_cs *engine);
> >
> > struct intel_engine_execlists execlists;
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c
> > index 3cca7ea2d6ea..84142127ebd8 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_user.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c
> > @@ -108,7 +108,7 @@ static void set_scheduler_caps(struct drm_i915_private *i915)
> > for_each_uabi_engine(engine, i915) { /* all engines must agree! */
> > int i;
> >
> > - if (engine->schedule)
> > + if (engine->sched_engine->schedule)
> > enabled |= (I915_SCHEDULER_CAP_ENABLED |
> > I915_SCHEDULER_CAP_PRIORITY);
> > else
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 0413ba87cc1e..23fd03815ad0 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -3119,7 +3119,7 @@ static bool can_preempt(struct intel_engine_cs *engine)
> > static void execlists_set_default_submission(struct intel_engine_cs *engine)
> > {
> > engine->submit_request = execlists_submit_request;
> > - engine->schedule = i915_schedule;
> > + engine->sched_engine->schedule = i915_schedule;
>
> Dumb question: Why is this a function pointer? It's always set to
> i915_schedule.
>
Ring submission doesn't set this and decisions are made whether it is
set or not.
Not trying to repeat myself but this is something that will away with
the DRM scheduler. Both the test if scheduler exists and the scheduler
function pointer being fixed functions within the DRM scheduler.
Matt
> --Jason
>
> > engine->execlists.tasklet.callback = execlists_submission_tasklet;
> > }
> >
> > @@ -3701,7 +3701,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > ve->base.cops = &virtual_context_ops;
> > ve->base.request_alloc = execlists_request_alloc;
> >
> > - ve->base.schedule = i915_schedule;
> > + ve->base.sched_engine->schedule = i915_schedule;
> > ve->base.submit_request = virtual_submit_request;
> > ve->base.bond_execute = virtual_bond_execute;
> >
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > index fd824fa329c2..5cc7648d1e5a 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > @@ -273,7 +273,7 @@ static int live_unlite_restore(struct intel_gt *gt, int prio)
> > };
> >
> > /* Alternatively preempt the spinner with ce[1] */
> > - engine->schedule(rq[1], &attr);
> > + engine->sched_engine->schedule(rq[1], &attr);
> > }
> >
> > /* And switch back to ce[0] for good measure */
> > @@ -900,7 +900,7 @@ release_queue(struct intel_engine_cs *engine,
> > i915_request_add(rq);
> >
> > local_bh_disable();
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> > local_bh_enable(); /* kick tasklet */
> >
> > i915_request_put(rq);
> > @@ -1325,7 +1325,7 @@ static int live_timeslice_queue(void *arg)
> > err = PTR_ERR(rq);
> > goto err_heartbeat;
> > }
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> > err = wait_for_submit(engine, rq, HZ / 2);
> > if (err) {
> > pr_err("%s: Timed out trying to submit semaphores\n",
> > @@ -1867,7 +1867,7 @@ static int live_late_preempt(void *arg)
> > }
> >
> > attr.priority = I915_PRIORITY_MAX;
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> >
> > if (!igt_wait_for_spinner(&spin_hi, rq)) {
> > pr_err("High priority context failed to preempt the low priority context\n");
> > @@ -2480,7 +2480,7 @@ static int live_suppress_self_preempt(void *arg)
> > i915_request_add(rq_b);
> >
> > GEM_BUG_ON(i915_request_completed(rq_a));
> > - engine->schedule(rq_a, &attr);
> > + engine->sched_engine->schedule(rq_a, &attr);
> > igt_spinner_end(&a.spin);
> >
> > if (!igt_wait_for_spinner(&b.spin, rq_b)) {
> > @@ -2612,7 +2612,7 @@ static int live_chain_preempt(void *arg)
> >
> > i915_request_get(rq);
> > i915_request_add(rq);
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> >
> > igt_spinner_end(&hi.spin);
> > if (i915_request_wait(rq, 0, HZ / 5) < 0) {
> > @@ -2971,7 +2971,7 @@ static int live_preempt_gang(void *arg)
> > break;
> >
> > /* Submit each spinner at increasing priority */
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> > } while (prio <= I915_PRIORITY_MAX &&
> > !__igt_timeout(end_time, NULL));
> > pr_debug("%s: Preempt chain of %d requests\n",
> > @@ -3219,7 +3219,7 @@ static int preempt_user(struct intel_engine_cs *engine,
> > i915_request_get(rq);
> > i915_request_add(rq);
> >
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> >
> > if (i915_request_wait(rq, 0, HZ / 2) < 0)
> > err = -ETIME;
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > index 5b63d4df8c93..e57dc900ae8d 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > @@ -858,12 +858,12 @@ static int active_engine(void *data)
> > rq[idx] = i915_request_get(new);
> > i915_request_add(new);
> >
> > - if (engine->schedule && arg->flags & TEST_PRIORITY) {
> > + if (engine->sched_engine->schedule && arg->flags & TEST_PRIORITY) {
> > struct i915_sched_attr attr = {
> > .priority =
> > i915_prandom_u32_max_state(512, &prng),
> > };
> > - engine->schedule(rq[idx], &attr);
> > + engine->sched_engine->schedule(rq[idx], &attr);
> > }
> >
> > err = active_request_put(old);
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 2551019ee217..4c5bbec0775d 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -641,7 +641,7 @@ static void guc_default_vfuncs(struct intel_engine_cs *engine)
> > engine->cops = &guc_context_ops;
> > engine->request_alloc = guc_request_alloc;
> >
> > - engine->schedule = i915_schedule;
> > + engine->sched_engine->schedule = i915_schedule;
> >
> > engine->reset.prepare = guc_reset_prepare;
> > engine->reset.rewind = guc_reset_rewind;
> > diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
> > index 95e859c319ca..f0cb8820fb97 100644
> > --- a/drivers/gpu/drm/i915/i915_request.c
> > +++ b/drivers/gpu/drm/i915/i915_request.c
> > @@ -1343,7 +1343,7 @@ __i915_request_await_execution(struct i915_request *to,
> > }
> >
> > /* Couple the dependency tree for PI on this exposed to->fence */
> > - if (to->engine->schedule) {
> > + if (to->engine->sched_engine->schedule) {
> > err = i915_sched_node_add_dependency(&to->sched,
> > &from->sched,
> > I915_DEPENDENCY_WEAK);
> > @@ -1484,7 +1484,7 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from)
> > return 0;
> > }
> >
> > - if (to->engine->schedule) {
> > + if (to->engine->sched_engine->schedule) {
> > ret = i915_sched_node_add_dependency(&to->sched,
> > &from->sched,
> > I915_DEPENDENCY_EXTERNAL);
> > @@ -1671,7 +1671,7 @@ __i915_request_add_to_timeline(struct i915_request *rq)
> > __i915_sw_fence_await_dma_fence(&rq->submit,
> > &prev->fence,
> > &rq->dmaq);
> > - if (rq->engine->schedule)
> > + if (rq->engine->sched_engine->schedule)
> > __i915_sched_node_add_dependency(&rq->sched,
> > &prev->sched,
> > &rq->dep,
> > @@ -1743,8 +1743,8 @@ void __i915_request_queue(struct i915_request *rq,
> > * decide whether to preempt the entire chain so that it is ready to
> > * run at the earliest possible convenience.
> > */
> > - if (attr && rq->engine->schedule)
> > - rq->engine->schedule(rq, attr);
> > + if (attr && rq->engine->sched_engine->schedule)
> > + rq->engine->sched_engine->schedule(rq, attr);
> >
> > local_bh_disable();
> > __i915_request_queue_bh(rq);
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 5cb0b1c98d6b..3f462f8b06f2 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > @@ -131,6 +131,14 @@ struct i915_sched_engine {
> > * @no_priolist: priority lists disabled
> > */
> > bool no_priolist;
> > +
> > + /*
> > + * Call when the priority on a request has changed and it and its
> > + * dependencies may need rescheduling. Note the request itself may
> > + * not be ready to run!
> > + */
> > + void (*schedule)(struct i915_request *request,
> > + const struct i915_sched_attr *attr);
> > };
> >
> > #endif /* _I915_SCHEDULER_TYPES_H_ */
> > --
> > 2.28.0
> >
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Intel-gfx] [PATCH 5/9] drm/i915: Move engine->schedule to i915_sched_engine
@ 2021-06-04 20:06 ` Matthew Brost
0 siblings, 0 replies; 64+ messages in thread
From: Matthew Brost @ 2021-06-04 20:06 UTC (permalink / raw)
To: Jason Ekstrand; +Cc: Daniel Vetter, Intel GFX, Maling list - DRI developers
On Fri, Jun 04, 2021 at 02:03:46PM -0500, Jason Ekstrand wrote:
> On Thu, Jun 3, 2021 at 4:09 PM Matthew Brost <matthew.brost@intel.com> wrote:
> >
> > The schedule function should be in the schedule object.
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gem/i915_gem_wait.c | 4 ++--
> > drivers/gpu/drm/i915/gt/intel_engine_cs.c | 3 ---
> > drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 ++--
> > drivers/gpu/drm/i915/gt/intel_engine_types.h | 8 --------
> > drivers/gpu/drm/i915/gt/intel_engine_user.c | 2 +-
> > .../gpu/drm/i915/gt/intel_execlists_submission.c | 4 ++--
> > drivers/gpu/drm/i915/gt/selftest_execlists.c | 16 ++++++++--------
> > drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 4 ++--
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +-
> > drivers/gpu/drm/i915/i915_request.c | 10 +++++-----
> > drivers/gpu/drm/i915/i915_scheduler_types.h | 8 ++++++++
> > 11 files changed, 31 insertions(+), 34 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_wait.c b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> > index 4b9856d5ba14..af1fbf8e2a9a 100644
> > --- a/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> > +++ b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
> > @@ -104,8 +104,8 @@ static void fence_set_priority(struct dma_fence *fence,
> > engine = rq->engine;
> >
> > rcu_read_lock(); /* RCU serialisation for set-wedged protection */
> > - if (engine->schedule)
> > - engine->schedule(rq, attr);
> > + if (engine->sched_engine->schedule)
> > + engine->sched_engine->schedule(rq, attr);
> > rcu_read_unlock();
> > }
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > index 33d879137908..b480fcb1aad1 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> > @@ -326,9 +326,6 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
> > if (engine->context_size)
> > DRIVER_CAPS(i915)->has_logical_contexts = true;
> >
> > - /* Nothing to do here, execute in order of dependencies */
> > - engine->schedule = NULL;
> > -
> > ewma__engine_latency_init(&engine->latency);
> > seqcount_init(&engine->stats.lock);
> >
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> > index b99ac41695f3..b6a305e6a974 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c
> > @@ -121,7 +121,7 @@ static void heartbeat(struct work_struct *wrk)
> > * but all other contexts, including the kernel
> > * context are stuck waiting for the signal.
> > */
> > - } else if (engine->schedule &&
> > + } else if (engine->sched_engine->schedule &&
> > rq->sched.attr.priority < I915_PRIORITY_BARRIER) {
> > /*
> > * Gradually raise the priority of the heartbeat to
> > @@ -136,7 +136,7 @@ static void heartbeat(struct work_struct *wrk)
> > attr.priority = I915_PRIORITY_BARRIER;
> >
> > local_bh_disable();
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> > local_bh_enable();
> > } else {
> > if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > index 7197b9fa5e35..f1b14aff5118 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
> > @@ -426,14 +426,6 @@ struct intel_engine_cs {
> > void (*bond_execute)(struct i915_request *rq,
> > struct dma_fence *signal);
> >
> > - /*
> > - * Call when the priority on a request has changed and it and its
> > - * dependencies may need rescheduling. Note the request itself may
> > - * not be ready to run!
> > - */
> > - void (*schedule)(struct i915_request *request,
> > - const struct i915_sched_attr *attr);
> > -
> > void (*release)(struct intel_engine_cs *engine);
> >
> > struct intel_engine_execlists execlists;
> > diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c
> > index 3cca7ea2d6ea..84142127ebd8 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_engine_user.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c
> > @@ -108,7 +108,7 @@ static void set_scheduler_caps(struct drm_i915_private *i915)
> > for_each_uabi_engine(engine, i915) { /* all engines must agree! */
> > int i;
> >
> > - if (engine->schedule)
> > + if (engine->sched_engine->schedule)
> > enabled |= (I915_SCHEDULER_CAP_ENABLED |
> > I915_SCHEDULER_CAP_PRIORITY);
> > else
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index 0413ba87cc1e..23fd03815ad0 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -3119,7 +3119,7 @@ static bool can_preempt(struct intel_engine_cs *engine)
> > static void execlists_set_default_submission(struct intel_engine_cs *engine)
> > {
> > engine->submit_request = execlists_submit_request;
> > - engine->schedule = i915_schedule;
> > + engine->sched_engine->schedule = i915_schedule;
>
> Dumb question: Why is this a function pointer? It's always set to
> i915_schedule.
>
Ring submission doesn't set this and decisions are made whether it is
set or not.
Not trying to repeat myself but this is something that will away with
the DRM scheduler. Both the test if scheduler exists and the scheduler
function pointer being fixed functions within the DRM scheduler.
Matt
> --Jason
>
> > engine->execlists.tasklet.callback = execlists_submission_tasklet;
> > }
> >
> > @@ -3701,7 +3701,7 @@ intel_execlists_create_virtual(struct intel_engine_cs **siblings,
> > ve->base.cops = &virtual_context_ops;
> > ve->base.request_alloc = execlists_request_alloc;
> >
> > - ve->base.schedule = i915_schedule;
> > + ve->base.sched_engine->schedule = i915_schedule;
> > ve->base.submit_request = virtual_submit_request;
> > ve->base.bond_execute = virtual_bond_execute;
> >
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > index fd824fa329c2..5cc7648d1e5a 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> > @@ -273,7 +273,7 @@ static int live_unlite_restore(struct intel_gt *gt, int prio)
> > };
> >
> > /* Alternatively preempt the spinner with ce[1] */
> > - engine->schedule(rq[1], &attr);
> > + engine->sched_engine->schedule(rq[1], &attr);
> > }
> >
> > /* And switch back to ce[0] for good measure */
> > @@ -900,7 +900,7 @@ release_queue(struct intel_engine_cs *engine,
> > i915_request_add(rq);
> >
> > local_bh_disable();
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> > local_bh_enable(); /* kick tasklet */
> >
> > i915_request_put(rq);
> > @@ -1325,7 +1325,7 @@ static int live_timeslice_queue(void *arg)
> > err = PTR_ERR(rq);
> > goto err_heartbeat;
> > }
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> > err = wait_for_submit(engine, rq, HZ / 2);
> > if (err) {
> > pr_err("%s: Timed out trying to submit semaphores\n",
> > @@ -1867,7 +1867,7 @@ static int live_late_preempt(void *arg)
> > }
> >
> > attr.priority = I915_PRIORITY_MAX;
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> >
> > if (!igt_wait_for_spinner(&spin_hi, rq)) {
> > pr_err("High priority context failed to preempt the low priority context\n");
> > @@ -2480,7 +2480,7 @@ static int live_suppress_self_preempt(void *arg)
> > i915_request_add(rq_b);
> >
> > GEM_BUG_ON(i915_request_completed(rq_a));
> > - engine->schedule(rq_a, &attr);
> > + engine->sched_engine->schedule(rq_a, &attr);
> > igt_spinner_end(&a.spin);
> >
> > if (!igt_wait_for_spinner(&b.spin, rq_b)) {
> > @@ -2612,7 +2612,7 @@ static int live_chain_preempt(void *arg)
> >
> > i915_request_get(rq);
> > i915_request_add(rq);
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> >
> > igt_spinner_end(&hi.spin);
> > if (i915_request_wait(rq, 0, HZ / 5) < 0) {
> > @@ -2971,7 +2971,7 @@ static int live_preempt_gang(void *arg)
> > break;
> >
> > /* Submit each spinner at increasing priority */
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> > } while (prio <= I915_PRIORITY_MAX &&
> > !__igt_timeout(end_time, NULL));
> > pr_debug("%s: Preempt chain of %d requests\n",
> > @@ -3219,7 +3219,7 @@ static int preempt_user(struct intel_engine_cs *engine,
> > i915_request_get(rq);
> > i915_request_add(rq);
> >
> > - engine->schedule(rq, &attr);
> > + engine->sched_engine->schedule(rq, &attr);
> >
> > if (i915_request_wait(rq, 0, HZ / 2) < 0)
> > err = -ETIME;
> > diff --git a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > index 5b63d4df8c93..e57dc900ae8d 100644
> > --- a/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > +++ b/drivers/gpu/drm/i915/gt/selftest_hangcheck.c
> > @@ -858,12 +858,12 @@ static int active_engine(void *data)
> > rq[idx] = i915_request_get(new);
> > i915_request_add(new);
> >
> > - if (engine->schedule && arg->flags & TEST_PRIORITY) {
> > + if (engine->sched_engine->schedule && arg->flags & TEST_PRIORITY) {
> > struct i915_sched_attr attr = {
> > .priority =
> > i915_prandom_u32_max_state(512, &prng),
> > };
> > - engine->schedule(rq[idx], &attr);
> > + engine->sched_engine->schedule(rq[idx], &attr);
> > }
> >
> > err = active_request_put(old);
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index 2551019ee217..4c5bbec0775d 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -641,7 +641,7 @@ static void guc_default_vfuncs(struct intel_engine_cs *engine)
> > engine->cops = &guc_context_ops;
> > engine->request_alloc = guc_request_alloc;
> >
> > - engine->schedule = i915_schedule;
> > + engine->sched_engine->schedule = i915_schedule;
> >
> > engine->reset.prepare = guc_reset_prepare;
> > engine->reset.rewind = guc_reset_rewind;
> > diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
> > index 95e859c319ca..f0cb8820fb97 100644
> > --- a/drivers/gpu/drm/i915/i915_request.c
> > +++ b/drivers/gpu/drm/i915/i915_request.c
> > @@ -1343,7 +1343,7 @@ __i915_request_await_execution(struct i915_request *to,
> > }
> >
> > /* Couple the dependency tree for PI on this exposed to->fence */
> > - if (to->engine->schedule) {
> > + if (to->engine->sched_engine->schedule) {
> > err = i915_sched_node_add_dependency(&to->sched,
> > &from->sched,
> > I915_DEPENDENCY_WEAK);
> > @@ -1484,7 +1484,7 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from)
> > return 0;
> > }
> >
> > - if (to->engine->schedule) {
> > + if (to->engine->sched_engine->schedule) {
> > ret = i915_sched_node_add_dependency(&to->sched,
> > &from->sched,
> > I915_DEPENDENCY_EXTERNAL);
> > @@ -1671,7 +1671,7 @@ __i915_request_add_to_timeline(struct i915_request *rq)
> > __i915_sw_fence_await_dma_fence(&rq->submit,
> > &prev->fence,
> > &rq->dmaq);
> > - if (rq->engine->schedule)
> > + if (rq->engine->sched_engine->schedule)
> > __i915_sched_node_add_dependency(&rq->sched,
> > &prev->sched,
> > &rq->dep,
> > @@ -1743,8 +1743,8 @@ void __i915_request_queue(struct i915_request *rq,
> > * decide whether to preempt the entire chain so that it is ready to
> > * run at the earliest possible convenience.
> > */
> > - if (attr && rq->engine->schedule)
> > - rq->engine->schedule(rq, attr);
> > + if (attr && rq->engine->sched_engine->schedule)
> > + rq->engine->sched_engine->schedule(rq, attr);
> >
> > local_bh_disable();
> > __i915_request_queue_bh(rq);
> > diff --git a/drivers/gpu/drm/i915/i915_scheduler_types.h b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > index 5cb0b1c98d6b..3f462f8b06f2 100644
> > --- a/drivers/gpu/drm/i915/i915_scheduler_types.h
> > +++ b/drivers/gpu/drm/i915/i915_scheduler_types.h
> > @@ -131,6 +131,14 @@ struct i915_sched_engine {
> > * @no_priolist: priority lists disabled
> > */
> > bool no_priolist;
> > +
> > + /*
> > + * Call when the priority on a request has changed and it and its
> > + * dependencies may need rescheduling. Note the request itself may
> > + * not be ready to run!
> > + */
> > + void (*schedule)(struct i915_request *request,
> > + const struct i915_sched_attr *attr);
> > };
> >
> > #endif /* _I915_SCHEDULER_TYPES_H_ */
> > --
> > 2.28.0
> >
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 64+ messages in thread
end of thread, other threads:[~2021-06-04 20:13 UTC | newest]
Thread overview: 64+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-03 21:27 [PATCH 0/9] Introduce i915_sched_engine object Matthew Brost
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
2021-06-03 21:27 ` [PATCH 1/9] drm/i915: Move priolist to new " Matthew Brost
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
2021-06-04 17:38 ` Jason Ekstrand
2021-06-04 17:38 ` [Intel-gfx] " Jason Ekstrand
2021-06-04 17:35 ` Matthew Brost
2021-06-04 17:35 ` [Intel-gfx] " Matthew Brost
2021-06-04 17:51 ` Jason Ekstrand
2021-06-04 17:51 ` [Intel-gfx] " Jason Ekstrand
2021-06-04 17:51 ` Matthew Brost
2021-06-04 17:51 ` [Intel-gfx] " Matthew Brost
2021-06-04 18:14 ` Jason Ekstrand
2021-06-04 18:14 ` [Intel-gfx] " Jason Ekstrand
2021-06-03 21:27 ` [PATCH 2/9] drm/i915: Add i915_sched_engine_is_empty function Matthew Brost
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
2021-06-04 17:50 ` Jason Ekstrand
2021-06-04 17:50 ` Jason Ekstrand
2021-06-03 21:27 ` [PATCH 3/9] drm/i915: Add i915_sched_engine_reset_on_empty function Matthew Brost
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
2021-06-04 18:31 ` Jason Ekstrand
2021-06-04 18:31 ` Jason Ekstrand
2021-06-04 18:49 ` Matthew Brost
2021-06-04 18:49 ` Matthew Brost
2021-06-03 21:27 ` [PATCH 4/9] drm/i915: Move active tracking to i915_sched_engine Matthew Brost
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
2021-06-04 19:00 ` Jason Ekstrand
2021-06-04 19:00 ` [Intel-gfx] " Jason Ekstrand
2021-06-04 19:12 ` Matthew Brost
2021-06-04 19:12 ` [Intel-gfx] " Matthew Brost
2021-06-03 21:27 ` [PATCH 5/9] drm/i915: Move engine->schedule " Matthew Brost
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
2021-06-04 19:03 ` Jason Ekstrand
2021-06-04 19:03 ` Jason Ekstrand
2021-06-04 20:06 ` Matthew Brost
2021-06-04 20:06 ` Matthew Brost
2021-06-03 21:27 ` [PATCH 6/9] drm/i915: Add kick_backend function " Matthew Brost
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
2021-06-04 19:09 ` Jason Ekstrand
2021-06-04 19:09 ` [Intel-gfx] " Jason Ekstrand
2021-06-04 19:19 ` Matthew Brost
2021-06-04 19:19 ` [Intel-gfx] " Matthew Brost
2021-06-03 21:27 ` [PATCH 7/9] drm/i915: Update i915_scheduler to operate on i915_sched_engine Matthew Brost
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
2021-06-04 19:17 ` Jason Ekstrand
2021-06-04 19:17 ` Jason Ekstrand
2021-06-04 19:14 ` Matthew Brost
2021-06-04 19:14 ` Matthew Brost
2021-06-03 21:27 ` [PATCH 8/9] drm/i915: Move submission tasklet to i915_sched_engine Matthew Brost
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
2021-06-04 19:26 ` Jason Ekstrand
2021-06-04 19:26 ` [Intel-gfx] " Jason Ekstrand
2021-06-04 19:55 ` Matthew Brost
2021-06-04 19:55 ` [Intel-gfx] " Matthew Brost
2021-06-03 21:27 ` [PATCH 9/9] drm/i915/doc: Add kernel doc for i915_sched_engine Matthew Brost
2021-06-03 21:27 ` [Intel-gfx] " Matthew Brost
2021-06-04 17:20 ` Jason Ekstrand
2021-06-04 17:20 ` Jason Ekstrand
2021-06-04 17:26 ` Matthew Brost
2021-06-04 17:26 ` Matthew Brost
2021-06-03 23:32 ` [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Introduce i915_sched_engine object (rev2) Patchwork
2021-06-03 23:37 ` [Intel-gfx] ✗ Fi.CI.DOCS: " Patchwork
2021-06-04 0:03 ` [Intel-gfx] ✓ Fi.CI.BAT: success " Patchwork
2021-06-04 2:03 ` [Intel-gfx] ✗ Fi.CI.IGT: failure " Patchwork
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.