* [PATCH v5 00/35] GPU scheduler for i915 driver
@ 2016-02-18 14:26 John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 01/35] drm/i915: Add total count to context status debugfs output John.C.Harrison
` (36 more replies)
0 siblings, 37 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Implemented a batch buffer submission scheduler for the i915 DRM driver.
The general theory of operation is that when batch buffers are
submitted to the driver, the execbuffer() code assigns a unique seqno
value and then packages up all the information required to execute the
batch buffer at a later time. This package is given over to the
scheduler which adds it to an internal node list. The scheduler also
scans the list of objects associated with the batch buffer and
compares them against the objects already in use by other buffers in
the node list. If matches are found then the new batch buffer node is
marked as being dependent upon the matching node. The same is done for
the context object. The scheduler also bumps up the priority of such
matching nodes on the grounds that the more dependencies a given batch
buffer has the more important it is likely to be.
The scheduler aims to have a given (tuneable) number of batch buffers
in flight on the hardware at any given time. If fewer than this are
currently executing when a new node is queued, then the node is passed
straight through to the submit function. Otherwise it is simply added
to the queue and the driver returns back to user land.
As each batch buffer completes, it raises an interrupt which wakes up
the scheduler. Note that it is possible for multiple buffers to
complete before the IRQ handler gets to run. Further, the seqno values
of the individual buffers are not necessary incrementing as the
scheduler may have re-ordered their submission. However, the scheduler
keeps the list of executing buffers in order of hardware submission.
Thus it can scan through the list until a matching seqno is found and
then mark all in flight nodes from that point on as completed.
A deferred work queue is also poked by the interrupt handler. When
this wakes up it can do more involved processing such as actually
removing completed nodes from the queue and freeing up the resources
associated with them (internal memory allocations, DRM object
references, context reference, etc.). The work handler also checks the
in flight count and calls the submission code if a new slot has
appeared.
When the scheduler's submit code is called, it scans the queued node
list for the highest priority node that has no unmet dependencies.
Note that the dependency calculation is complex as it must take
inter-ring dependencies and potential preemptions into account. Note
also that in the future this will be extended to include external
dependencies such as the Android Native Sync file descriptors and/or
the linux dma-buff synchronisation scheme.
If a suitable node is found then it is sent to execbuff_final() for
submission to the hardware. The in flight count is then re-checked and
a new node popped from the list if appropriate.
The scheduler also allows high priority batch buffers (e.g. from a
desktop compositor) to jump ahead of whatever is already running if
the underlying hardware supports pre-emption. In this situation, any
work that was pre-empted is returned to the queued list ready to be
resubmitted when no more high priority work is outstanding.
Various IGT tests are in progress to test the scheduler's operation
and will follow.
v2: Updated for changes in struct fence patch series and other changes
to underlying tree (e.g. removal of cliprects). Also changed priority
levels to be signed +/-1023 range and reduced mutex lock usage.
v3: More reuse of cached pointers rather than repeated dereferencing
(David Gordon).
Moved the dependency generation code out to a seperate function for
easier readability. Also added in support for the read-read
optimisation.
Major simplification of the DRM file close handler.
Fixed up an overzealous WARN.
Removed unnecessary flushing of the scheduler queue when waiting for a
request.
v4: Removed user land fence/sync integration as this is dependent upon
the de-staging of the Android sync code. That de-staging is now being
done by someone else. The sync support will be added back in to the
scheduler in a separate patch series which must wait for the
de-staging to be landed first.
Added support for killing batches from contexts that were banned after
the batches were submitted to the scheduler.
Changed various comments to fix typos, update to reflect changes to
the code, correct formatting and line wrapping, etc. Also wrapped
various long lines and twiddled white space to keep the style checker
happy.
Changed a bunch of BUG_ONs to WARN_ONs as apparently the latter are
preferred.
Used the correct array memory allocator function (kmalloc_array
instead of kmalloc).
Fixed a variable type (review comment by Joonas).
Fixed a WARN_ON firing incorrectly when removing killed nodes from the
scheduler's queue.
Dropped register definition update patch from this series. The changes
are all for pre-emption so it makes more sense for it to be part of
that series instead.
v5: Reverted power management changes as they apparently conflict with
mutex acquisition. Converted the override mask module parameter to a
set of boolean enable flags (just one in this patch set, but others
are added later for controlling pre-emption). [Chris Wilson]
Removed lots of whitespace from i915_scheduler.c and re-ordered it to
remove all forward declarations. Squashed down the i915_scheduler.c
sections of various patches into the initial 'start of scheduler'
patch. Thus the later patches simply hook in existing code into
various parts of the driver rather than adding the code as well. Added
documentation to various functions. Re-worked the submit function in
terms of mutex locking, error handling and exit paths. Split the
delayed work handler function in half. Made use of the kernel 'clamp'
macro. [Joonas Lahtinen]
Dropped the 'immediate submission' override option. This was a half
way house between full scheduler and direct submission and was only
really useful during early debug.
Added a re-install of the scheduler's interrupt hook around GPU reset.
[Zhipeng Gong]
Used lighter weight spinlocks.
[Patches against drm-intel-nightly tree fetched 19/01/2016 with struct
fence conversion patches applied]
Dave Gordon (2):
drm/i915: Cache request pointer in *_submission_final()
drm/i915: Add scheduling priority to per-context parameters
John Harrison (33):
drm/i915: Add total count to context status debugfs output
drm/i915: Prelude to splitting i915_gem_do_execbuffer in two
drm/i915: Split i915_dem_do_execbuffer() in half
drm/i915: Re-instate request->uniq because it is extremely useful
drm/i915: Start of GPU scheduler
drm/i915: Prepare retire_requests to handle out-of-order seqnos
drm/i915: Disable hardware semaphores when GPU scheduler is enabled
drm/i915: Force MMIO flips when scheduler enabled
drm/i915: Added scheduler hook when closing DRM file handles
drm/i915: Added scheduler hook into i915_gem_request_notify()
drm/i915: Added deferred work handler for scheduler
drm/i915: Redirect execbuffer_final() via scheduler
drm/i915: Keep the reserved space mechanism happy
drm/i915: Added tracking/locking of batch buffer objects
drm/i915: Hook scheduler node clean up into retire requests
drm/i915: Added scheduler support to __wait_request() calls
drm/i915: Added scheduler support to page fault handler
drm/i915: Added scheduler flush calls to ring throttle and idle functions
drm/i915: Add scheduler hook to GPU reset
drm/i915: Added a module parameter to allow the scheduler to be disabled
drm/i915: Support for 'unflushed' ring idle
drm/i915: Defer seqno allocation until actual hardware submission time
drm/i915: Added trace points to scheduler
drm/i915: Added scheduler queue throttling by DRM file handle
drm/i915: Added debugfs interface to scheduler tuning parameters
drm/i915: Added debug state dump facilities to scheduler
drm/i915: Add early exit to execbuff_final() if insufficient ring space
drm/i915: Added scheduler statistic reporting to debugfs
drm/i915: Add scheduler support functions for TDR
drm/i915: Scheduler state dump via debugfs
drm/i915: Enable GPU scheduler by default
drm/i915: Add support for retro-actively banning batch buffers
drm/i915: Allow scheduler to manage inter-ring object synchronisation
drivers/gpu/drm/i915/Makefile | 1 +
drivers/gpu/drm/i915/i915_debugfs.c | 280 ++++-
drivers/gpu/drm/i915/i915_dma.c | 6 +
drivers/gpu/drm/i915/i915_drv.c | 9 +
drivers/gpu/drm/i915/i915_drv.h | 54 +-
drivers/gpu/drm/i915/i915_gem.c | 179 ++-
drivers/gpu/drm/i915/i915_gem_context.c | 24 +
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 286 ++++-
drivers/gpu/drm/i915/i915_params.c | 4 +
drivers/gpu/drm/i915/i915_params.h | 1 +
drivers/gpu/drm/i915/i915_scheduler.c | 1663 ++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 159 +++
drivers/gpu/drm/i915/i915_trace.h | 228 +++-
drivers/gpu/drm/i915/intel_display.c | 10 +-
drivers/gpu/drm/i915/intel_lrc.c | 161 ++-
drivers/gpu/drm/i915/intel_lrc.h | 1 +
drivers/gpu/drm/i915/intel_ringbuffer.c | 49 +-
drivers/gpu/drm/i915/intel_ringbuffer.h | 3 +-
include/uapi/drm/i915_drm.h | 1 +
19 files changed, 2976 insertions(+), 143 deletions(-)
create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* [PATCH v5 01/35] drm/i915: Add total count to context status debugfs output
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 02/35] drm/i915: Prelude to splitting i915_gem_do_execbuffer in two John.C.Harrison
` (35 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
When there are lots and lots and even more lots of contexts (e.g. when
running with execlists) it is useful to be able to immediately see
what the total context count is.
v4: Re-typed a variable (review feedback from Joonas)
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_debugfs.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index b90d6ea..b923949 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -1971,7 +1971,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_engine_cs *ring;
struct intel_context *ctx;
- int ret, i;
+ int ret, i, count = 0;
ret = mutex_lock_interruptible(&dev->struct_mutex);
if (ret)
@@ -1984,6 +1984,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
seq_puts(m, "HW context ");
describe_ctx(m, ctx);
+ count++;
for_each_ring(ring, dev_priv, i) {
if (ring->default_context == ctx)
seq_printf(m, "(default context %s) ",
@@ -2012,6 +2013,8 @@ static int i915_context_status(struct seq_file *m, void *unused)
seq_putc(m, '\n');
}
+ seq_printf(m, "Total: %d contexts\n", count);
+
mutex_unlock(&dev->struct_mutex);
return 0;
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 02/35] drm/i915: Prelude to splitting i915_gem_do_execbuffer in two
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 01/35] drm/i915: Add total count to context status debugfs output John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 03/35] drm/i915: Split i915_dem_do_execbuffer() in half John.C.Harrison
` (34 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler decouples the submission of batch buffers to the driver
with their submission to the hardware. This basically means splitting
the execbuffer() function in half. This change rearranges some code
ready for the split to occur.
v5: Dropped runtime PM calls as they conflict with the mutex lock.
Instead of being done at the lowest submission level, they are now
left right at the top driver entry level. [feedback from Chris Wilson]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 41 ++++++++++++++++++------------
drivers/gpu/drm/i915/intel_lrc.c | 18 +++++++++----
2 files changed, 38 insertions(+), 21 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index e3e3666..d09130f 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -966,10 +966,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
if (flush_domains & I915_GEM_DOMAIN_GTT)
wmb();
- /* Unconditionally invalidate gpu caches and ensure that we do flush
- * any residual writes from the previous batch.
- */
- return intel_ring_invalidate_all_caches(req);
+ return 0;
}
static bool
@@ -1237,17 +1234,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
u32 instp_mask;
int ret;
- ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
- if (ret)
- return ret;
-
- ret = i915_switch_context(params->request);
- if (ret)
- return ret;
-
- WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
- "%s didn't clear reload\n", ring->name);
-
instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
instp_mask = I915_EXEC_CONSTANTS_MASK;
switch (instp_mode) {
@@ -1281,6 +1267,30 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
return -EINVAL;
}
+ ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
+ if (ret)
+ return ret;
+
+ i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+ /* To be split into two functions here... */
+
+ /*
+ * Unconditionally invalidate gpu caches and ensure that we do flush
+ * any residual writes from the previous batch.
+ */
+ ret = intel_ring_invalidate_all_caches(params->request);
+ if (ret)
+ return ret;
+
+ /* Switch to the correct context for the batch */
+ ret = i915_switch_context(params->request);
+ if (ret)
+ return ret;
+
+ WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
+ "%s didn't clear reload\n", ring->name);
+
if (ring == &dev_priv->ring[RCS] &&
instp_mode != dev_priv->relative_constants_mode) {
ret = intel_ring_begin(params->request, 4);
@@ -1317,7 +1327,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
- i915_gem_execbuffer_move_to_active(vmas, params->request);
i915_gem_execbuffer_retire_commands(params);
return 0;
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 5657924..53aeb55 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -678,10 +678,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
if (flush_domains & I915_GEM_DOMAIN_GTT)
wmb();
- /* Unconditionally invalidate gpu caches and ensure that we do flush
- * any residual writes from the previous batch.
- */
- return logical_ring_invalidate_all_caches(req);
+ return 0;
}
int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request)
@@ -957,6 +954,18 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
if (ret)
return ret;
+ i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+ /* To be split into two functions here... */
+
+ /*
+ * Unconditionally invalidate gpu caches and ensure that we do flush
+ * any residual writes from the previous batch.
+ */
+ ret = logical_ring_invalidate_all_caches(params->request);
+ if (ret)
+ return ret;
+
if (ring == &dev_priv->ring[RCS] &&
instp_mode != dev_priv->relative_constants_mode) {
ret = intel_logical_ring_begin(params->request, 4);
@@ -981,7 +990,6 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
- i915_gem_execbuffer_move_to_active(vmas, params->request);
i915_gem_execbuffer_retire_commands(params);
return 0;
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 03/35] drm/i915: Split i915_dem_do_execbuffer() in half
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 01/35] drm/i915: Add total count to context status debugfs output John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 02/35] drm/i915: Prelude to splitting i915_gem_do_execbuffer in two John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 04/35] drm/i915: Cache request pointer in *_submission_final() John.C.Harrison
` (33 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Split the execbuffer() function in half. The first half collects and
validates all the information required to process the batch buffer. It
also does all the object pinning, relocations, active list management,
etc - basically anything that must be done upfront before the IOCTL
returns and allows the user land side to start changing/freeing
things. The second half does the actual ring submission.
This change implements the split but leaves the back half being called
directly from the end of the front half.
v2: Updated due to changes in underlying tree - addition of sync fence
support and removal of cliprects.
v3: Moved local 'ringbuf' variable to make later patches in the
series a bit neater.
v4: Corrected a typo in the commit message and downgraded a BUG_ON to
a WARN_ON as the latter is preferred. Also removed all the external
sync/fence support as that will now be a separate patch series.
v5: Updated for runtime PM changes.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_drv.h | 11 +++
drivers/gpu/drm/i915/i915_gem.c | 2 +
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 103 ++++++++++++++++++++++-------
drivers/gpu/drm/i915/intel_lrc.c | 57 +++++++++++-----
drivers/gpu/drm/i915/intel_lrc.h | 1 +
5 files changed, 132 insertions(+), 42 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 0584846..8dd811e 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1687,10 +1687,18 @@ struct i915_execbuffer_params {
struct drm_device *dev;
struct drm_file *file;
uint32_t dispatch_flags;
+ uint32_t args_flags;
uint32_t args_batch_start_offset;
+ uint32_t args_batch_len;
+ uint32_t args_num_cliprects;
+ uint32_t args_DR1;
+ uint32_t args_DR4;
uint64_t batch_obj_vm_offset;
struct intel_engine_cs *ring;
struct drm_i915_gem_object *batch_obj;
+ struct drm_clip_rect *cliprects;
+ uint32_t instp_mask;
+ int instp_mode;
struct intel_context *ctx;
struct drm_i915_gem_request *request;
};
@@ -1965,6 +1973,7 @@ struct drm_i915_private {
int (*execbuf_submit)(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas);
+ int (*execbuf_final)(struct i915_execbuffer_params *params);
int (*init_rings)(struct drm_device *dev);
void (*cleanup_ring)(struct intel_engine_cs *ring);
void (*stop_ring)(struct intel_engine_cs *ring);
@@ -2835,9 +2844,11 @@ int i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,
void i915_gem_execbuffer_move_to_active(struct list_head *vmas,
struct drm_i915_gem_request *req);
void i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params);
+void i915_gem_execbuff_release_batch_obj(struct drm_i915_gem_object *batch_obj);
int i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas);
+int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params);
int i915_gem_execbuffer(struct drm_device *dev, void *data,
struct drm_file *file_priv);
int i915_gem_execbuffer2(struct drm_device *dev, void *data,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 72a37d6..bf39ca4 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -5293,11 +5293,13 @@ int i915_gem_init(struct drm_device *dev)
if (!i915.enable_execlists) {
dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission;
+ dev_priv->gt.execbuf_final = i915_gem_ringbuffer_submission_final;
dev_priv->gt.init_rings = i915_gem_init_rings;
dev_priv->gt.cleanup_ring = intel_cleanup_ring_buffer;
dev_priv->gt.stop_ring = intel_stop_ring_buffer;
} else {
dev_priv->gt.execbuf_submit = intel_execlists_submission;
+ dev_priv->gt.execbuf_final = intel_execlists_submission_final;
dev_priv->gt.init_rings = intel_logical_rings_init;
dev_priv->gt.cleanup_ring = intel_logical_ring_cleanup;
dev_priv->gt.stop_ring = intel_logical_ring_stop;
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index d09130f..3071a2b 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1229,41 +1229,38 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
struct drm_device *dev = params->dev;
struct intel_engine_cs *ring = params->ring;
struct drm_i915_private *dev_priv = dev->dev_private;
- u64 exec_start, exec_len;
- int instp_mode;
- u32 instp_mask;
int ret;
- instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
- instp_mask = I915_EXEC_CONSTANTS_MASK;
- switch (instp_mode) {
+ params->instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
+ params->instp_mask = I915_EXEC_CONSTANTS_MASK;
+ switch (params->instp_mode) {
case I915_EXEC_CONSTANTS_REL_GENERAL:
case I915_EXEC_CONSTANTS_ABSOLUTE:
case I915_EXEC_CONSTANTS_REL_SURFACE:
- if (instp_mode != 0 && ring != &dev_priv->ring[RCS]) {
+ if (params->instp_mode != 0 && ring != &dev_priv->ring[RCS]) {
DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
return -EINVAL;
}
- if (instp_mode != dev_priv->relative_constants_mode) {
+ if (params->instp_mode != dev_priv->relative_constants_mode) {
if (INTEL_INFO(dev)->gen < 4) {
DRM_DEBUG("no rel constants on pre-gen4\n");
return -EINVAL;
}
if (INTEL_INFO(dev)->gen > 5 &&
- instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
+ params->instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
DRM_DEBUG("rel surface constants mode invalid on gen5+\n");
return -EINVAL;
}
/* The HW changed the meaning on this bit on gen6 */
if (INTEL_INFO(dev)->gen >= 6)
- instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
+ params->instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
}
break;
default:
- DRM_DEBUG("execbuf with unknown constants: %d\n", instp_mode);
+ DRM_DEBUG("execbuf with unknown constants: %d\n", params->instp_mode);
return -EINVAL;
}
@@ -1273,7 +1270,33 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
- /* To be split into two functions here... */
+ ret = dev_priv->gt.execbuf_final(params);
+ if (ret)
+ return ret;
+
+ /*
+ * Free everything that was stored in the QE structure (until the
+ * scheduler arrives and does it instead):
+ */
+ if (params->dispatch_flags & I915_DISPATCH_SECURE)
+ i915_gem_execbuff_release_batch_obj(params->batch_obj);
+
+ return 0;
+}
+
+/*
+ * This is the main function for adding a batch to the ring.
+ * It is called from the scheduler, with the struct_mutex already held.
+ */
+int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
+{
+ struct drm_i915_private *dev_priv = params->dev->dev_private;
+ struct intel_engine_cs *ring = params->ring;
+ u64 exec_start, exec_len;
+ int ret;
+
+ /* The mutex must be acquired before calling this function */
+ WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
/*
* Unconditionally invalidate gpu caches and ensure that we do flush
@@ -1292,7 +1315,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
"%s didn't clear reload\n", ring->name);
if (ring == &dev_priv->ring[RCS] &&
- instp_mode != dev_priv->relative_constants_mode) {
+ params->instp_mode != dev_priv->relative_constants_mode) {
ret = intel_ring_begin(params->request, 4);
if (ret)
return ret;
@@ -1300,19 +1323,19 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
intel_ring_emit(ring, MI_NOOP);
intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
intel_ring_emit_reg(ring, INSTPM);
- intel_ring_emit(ring, instp_mask << 16 | instp_mode);
+ intel_ring_emit(ring, params->instp_mask << 16 | params->instp_mode);
intel_ring_advance(ring);
- dev_priv->relative_constants_mode = instp_mode;
+ dev_priv->relative_constants_mode = params->instp_mode;
}
- if (args->flags & I915_EXEC_GEN7_SOL_RESET) {
- ret = i915_reset_gen7_sol_offsets(dev, params->request);
+ if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
+ ret = i915_reset_gen7_sol_offsets(params->dev, params->request);
if (ret)
return ret;
}
- exec_len = args->batch_len;
+ exec_len = params->args_batch_len;
exec_start = params->batch_obj_vm_offset +
params->args_batch_start_offset;
@@ -1629,23 +1652,42 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
params->file = file;
params->ring = ring;
params->dispatch_flags = dispatch_flags;
+ params->args_flags = args->flags;
+ params->args_batch_len = args->batch_len;
+ params->args_num_cliprects = args->num_cliprects;
+ params->args_DR1 = args->DR1;
+ params->args_DR4 = args->DR4;
params->batch_obj = batch_obj;
params->ctx = ctx;
ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
+ if (ret)
+ goto err_batch_unpin;
+
+ /* the request owns the ref now */
+ i915_gem_context_unreference(ctx);
-err_batch_unpin:
/*
- * FIXME: We crucially rely upon the active tracking for the (ppgtt)
- * batch vma for correctness. For less ugly and less fragility this
- * needs to be adjusted to also track the ggtt batch vma properly as
- * active.
+ * The eb list is no longer required. The scheduler has extracted all
+ * the information than needs to persist.
+ */
+ eb_destroy(eb);
+
+ /*
+ * Don't clean up everything that is now saved away in the queue.
+ * Just unlock and return immediately.
*/
+ mutex_unlock(&dev->struct_mutex);
+
+ intel_runtime_pm_put(dev_priv);
+
+ return 0;
+
+err_batch_unpin:
if (dispatch_flags & I915_DISPATCH_SECURE)
- i915_gem_object_ggtt_unpin(batch_obj);
+ i915_gem_execbuff_release_batch_obj(batch_obj);
err:
- /* the request owns the ref now */
i915_gem_context_unreference(ctx);
eb_destroy(eb);
@@ -1666,6 +1708,17 @@ pre_mutex_err:
return ret;
}
+void i915_gem_execbuff_release_batch_obj(struct drm_i915_gem_object *batch_obj)
+{
+ /*
+ * FIXME: We crucially rely upon the active tracking for the (ppgtt)
+ * batch vma for correctness. For less ugly and less fragility this
+ * needs to be adjusted to also track the ggtt batch vma properly as
+ * active.
+ */
+ i915_gem_object_ggtt_unpin(batch_obj);
+}
+
/*
* Legacy execbuffer just creates an exec2 list from the original exec object
* list array and passes it to the real function.
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 53aeb55..2246a16 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -913,35 +913,31 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
struct drm_device *dev = params->dev;
struct intel_engine_cs *ring = params->ring;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_ringbuffer *ringbuf = params->ctx->engine[ring->id].ringbuf;
- u64 exec_start;
- int instp_mode;
- u32 instp_mask;
int ret;
- instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
- instp_mask = I915_EXEC_CONSTANTS_MASK;
- switch (instp_mode) {
+ params->instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
+ params->instp_mask = I915_EXEC_CONSTANTS_MASK;
+ switch (params->instp_mode) {
case I915_EXEC_CONSTANTS_REL_GENERAL:
case I915_EXEC_CONSTANTS_ABSOLUTE:
case I915_EXEC_CONSTANTS_REL_SURFACE:
- if (instp_mode != 0 && ring != &dev_priv->ring[RCS]) {
+ if (params->instp_mode != 0 && ring != &dev_priv->ring[RCS]) {
DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
return -EINVAL;
}
- if (instp_mode != dev_priv->relative_constants_mode) {
- if (instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
+ if (params->instp_mode != dev_priv->relative_constants_mode) {
+ if (params->instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
DRM_DEBUG("rel surface constants mode invalid on gen5+\n");
return -EINVAL;
}
/* The HW changed the meaning on this bit on gen6 */
- instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
+ params->instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
}
break;
default:
- DRM_DEBUG("execbuf with unknown constants: %d\n", instp_mode);
+ DRM_DEBUG("execbuf with unknown constants: %d\n", params->instp_mode);
return -EINVAL;
}
@@ -956,7 +952,34 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
- /* To be split into two functions here... */
+ ret = dev_priv->gt.execbuf_final(params);
+ if (ret)
+ return ret;
+
+ /*
+ * Free everything that was stored in the QE structure (until the
+ * scheduler arrives and does it instead):
+ */
+ if (params->dispatch_flags & I915_DISPATCH_SECURE)
+ i915_gem_execbuff_release_batch_obj(params->batch_obj);
+
+ return 0;
+}
+
+/*
+ * This is the main function for adding a batch to the ring.
+ * It is called from the scheduler, with the struct_mutex already held.
+ */
+int intel_execlists_submission_final(struct i915_execbuffer_params *params)
+{
+ struct drm_i915_private *dev_priv = params->dev->dev_private;
+ struct intel_ringbuffer *ringbuf = params->request->ringbuf;
+ struct intel_engine_cs *ring = params->ring;
+ u64 exec_start;
+ int ret;
+
+ /* The mutex must be acquired before calling this function */
+ WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
/*
* Unconditionally invalidate gpu caches and ensure that we do flush
@@ -967,7 +990,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
return ret;
if (ring == &dev_priv->ring[RCS] &&
- instp_mode != dev_priv->relative_constants_mode) {
+ params->instp_mode != dev_priv->relative_constants_mode) {
ret = intel_logical_ring_begin(params->request, 4);
if (ret)
return ret;
@@ -975,14 +998,14 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
intel_logical_ring_emit(ringbuf, MI_NOOP);
intel_logical_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(1));
intel_logical_ring_emit_reg(ringbuf, INSTPM);
- intel_logical_ring_emit(ringbuf, instp_mask << 16 | instp_mode);
+ intel_logical_ring_emit(ringbuf, params->instp_mask << 16 | params->instp_mode);
intel_logical_ring_advance(ringbuf);
- dev_priv->relative_constants_mode = instp_mode;
+ dev_priv->relative_constants_mode = params->instp_mode;
}
exec_start = params->batch_obj_vm_offset +
- args->batch_start_offset;
+ params->args_batch_start_offset;
ret = ring->emit_bb_start(params->request, exec_start, params->dispatch_flags);
if (ret)
diff --git a/drivers/gpu/drm/i915/intel_lrc.h b/drivers/gpu/drm/i915/intel_lrc.h
index 49af638..d2e387f 100644
--- a/drivers/gpu/drm/i915/intel_lrc.h
+++ b/drivers/gpu/drm/i915/intel_lrc.h
@@ -116,6 +116,7 @@ struct i915_execbuffer_params;
int intel_execlists_submission(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas);
+int intel_execlists_submission_final(struct i915_execbuffer_params *params);
void intel_lrc_irq_handler(struct intel_engine_cs *ring);
void intel_execlists_retire_requests(struct intel_engine_cs *ring);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 04/35] drm/i915: Cache request pointer in *_submission_final()
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (2 preceding siblings ...)
2016-02-18 14:26 ` [PATCH v5 03/35] drm/i915: Split i915_dem_do_execbuffer() in half John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 05/35] drm/i915: Re-instate request->uniq because it is extremely useful John.C.Harrison
` (32 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: Dave Gordon <david.s.gordon@intel.com>
Keep a local copy of the request pointer in the _final() functions
rather than dereferencing the params block repeatedly.
v3: New patch in series.
For: VIZ-1587
Signed-off-by: Dave Gordon <david.s.gordon@intel.com>
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 13 +++++++------
drivers/gpu/drm/i915/intel_lrc.c | 11 ++++++-----
2 files changed, 13 insertions(+), 11 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 3071a2b..7978dae 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1291,6 +1291,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
{
struct drm_i915_private *dev_priv = params->dev->dev_private;
+ struct drm_i915_gem_request *req = params->request;
struct intel_engine_cs *ring = params->ring;
u64 exec_start, exec_len;
int ret;
@@ -1302,12 +1303,12 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
- ret = intel_ring_invalidate_all_caches(params->request);
+ ret = intel_ring_invalidate_all_caches(req);
if (ret)
return ret;
/* Switch to the correct context for the batch */
- ret = i915_switch_context(params->request);
+ ret = i915_switch_context(req);
if (ret)
return ret;
@@ -1316,7 +1317,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
if (ring == &dev_priv->ring[RCS] &&
params->instp_mode != dev_priv->relative_constants_mode) {
- ret = intel_ring_begin(params->request, 4);
+ ret = intel_ring_begin(req, 4);
if (ret)
return ret;
@@ -1330,7 +1331,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
}
if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
- ret = i915_reset_gen7_sol_offsets(params->dev, params->request);
+ ret = i915_reset_gen7_sol_offsets(params->dev, req);
if (ret)
return ret;
}
@@ -1342,13 +1343,13 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
if (exec_len == 0)
exec_len = params->batch_obj->base.size;
- ret = ring->dispatch_execbuffer(params->request,
+ ret = ring->dispatch_execbuffer(req,
exec_start, exec_len,
params->dispatch_flags);
if (ret)
return ret;
- trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+ trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
i915_gem_execbuffer_retire_commands(params);
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 2246a16..12e8949 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -973,7 +973,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
int intel_execlists_submission_final(struct i915_execbuffer_params *params)
{
struct drm_i915_private *dev_priv = params->dev->dev_private;
- struct intel_ringbuffer *ringbuf = params->request->ringbuf;
+ struct drm_i915_gem_request *req = params->request;
+ struct intel_ringbuffer *ringbuf = req->ringbuf;
struct intel_engine_cs *ring = params->ring;
u64 exec_start;
int ret;
@@ -985,13 +986,13 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
- ret = logical_ring_invalidate_all_caches(params->request);
+ ret = logical_ring_invalidate_all_caches(req);
if (ret)
return ret;
if (ring == &dev_priv->ring[RCS] &&
params->instp_mode != dev_priv->relative_constants_mode) {
- ret = intel_logical_ring_begin(params->request, 4);
+ ret = intel_logical_ring_begin(req, 4);
if (ret)
return ret;
@@ -1007,11 +1008,11 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
exec_start = params->batch_obj_vm_offset +
params->args_batch_start_offset;
- ret = ring->emit_bb_start(params->request, exec_start, params->dispatch_flags);
+ ret = ring->emit_bb_start(req, exec_start, params->dispatch_flags);
if (ret)
return ret;
- trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+ trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
i915_gem_execbuffer_retire_commands(params);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 05/35] drm/i915: Re-instate request->uniq because it is extremely useful
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (3 preceding siblings ...)
2016-02-18 14:26 ` [PATCH v5 04/35] drm/i915: Cache request pointer in *_submission_final() John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 06/35] drm/i915: Start of GPU scheduler John.C.Harrison
` (31 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The seqno value cannot always be used when debugging issues via trace
points. This is because it can be reset back to start, especially
during TDR type tests. Also, when the scheduler arrives the seqno is
only valid while a given request is executing on the hardware. While
the request is simply queued waiting for submission, it's seqno value
will be zero (meaning invalid).
v4: Wrapped a long line to keep the style checker happy.
v5: Added uniq to the dispatch trace point [Svetlana Kukanova]
For: VIZ-5115
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Tomas Elf <tomas.elf@intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 5 +++++
drivers/gpu/drm/i915/i915_gem.c | 4 +++-
drivers/gpu/drm/i915/i915_trace.h | 32 ++++++++++++++++++++++----------
3 files changed, 30 insertions(+), 11 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 8dd811e..f4487b9 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1986,6 +1986,8 @@ struct drm_i915_private {
struct intel_encoder *dig_port_map[I915_MAX_PORTS];
+ uint32_t request_uniq;
+
/*
* NOTE: This is the dri1/ums dungeon, don't add stuff here. Your patch
* will be rejected. Instead look for a better place.
@@ -2242,6 +2244,9 @@ struct drm_i915_gem_request {
*/
u32 seqno;
+ /* Unique identifier which can be used for trace points & debug */
+ uint32_t uniq;
+
/** Position in the ringbuffer of the start of the request */
u32 head;
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index bf39ca4..dfe43ea 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2960,7 +2960,8 @@ static void i915_gem_request_fence_value_str(struct fence *req_fence,
req = container_of(req_fence, typeof(*req), fence);
- snprintf(str, size, "%d [%d]", req->fence.seqno, req->seqno);
+ snprintf(str, size, "%d [%d:%d]", req->fence.seqno, req->uniq,
+ req->seqno);
}
static const struct fence_ops i915_gem_request_fops = {
@@ -3036,6 +3037,7 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,
req->i915 = dev_priv;
req->ring = ring;
+ req->uniq = dev_priv->request_uniq++;
req->ctx = ctx;
i915_gem_context_reference(req->ctx);
diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
index cfe4f03..455c215 100644
--- a/drivers/gpu/drm/i915/i915_trace.h
+++ b/drivers/gpu/drm/i915/i915_trace.h
@@ -469,6 +469,7 @@ TRACE_EVENT(i915_gem_ring_sync_to,
__field(u32, dev)
__field(u32, sync_from)
__field(u32, sync_to)
+ __field(u32, uniq_to)
__field(u32, seqno)
),
@@ -476,13 +477,14 @@ TRACE_EVENT(i915_gem_ring_sync_to,
__entry->dev = from->dev->primary->index;
__entry->sync_from = from->id;
__entry->sync_to = to_req->ring->id;
+ __entry->uniq_to = to_req->uniq;
__entry->seqno = i915_gem_request_get_seqno(req);
),
- TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u",
+ TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u, to_uniq=%u",
__entry->dev,
__entry->sync_from, __entry->sync_to,
- __entry->seqno)
+ __entry->seqno, __entry->uniq_to)
);
TRACE_EVENT(i915_gem_ring_dispatch,
@@ -492,6 +494,7 @@ TRACE_EVENT(i915_gem_ring_dispatch,
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
+ __field(u32, uniq)
__field(u32, seqno)
__field(u32, flags)
),
@@ -501,13 +504,15 @@ TRACE_EVENT(i915_gem_ring_dispatch,
i915_gem_request_get_ring(req);
__entry->dev = ring->dev->primary->index;
__entry->ring = ring->id;
+ __entry->uniq = req->uniq;
__entry->seqno = i915_gem_request_get_seqno(req);
__entry->flags = flags;
i915_trace_irq_get(ring, req);
),
- TP_printk("dev=%u, ring=%u, seqno=%u, flags=%x",
- __entry->dev, __entry->ring, __entry->seqno, __entry->flags)
+ TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u, flags=%x",
+ __entry->dev, __entry->ring, __entry->uniq,
+ __entry->seqno, __entry->flags)
);
TRACE_EVENT(i915_gem_ring_flush,
@@ -517,6 +522,7 @@ TRACE_EVENT(i915_gem_ring_flush,
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
+ __field(u32, uniq)
__field(u32, invalidate)
__field(u32, flush)
),
@@ -524,12 +530,13 @@ TRACE_EVENT(i915_gem_ring_flush,
TP_fast_assign(
__entry->dev = req->ring->dev->primary->index;
__entry->ring = req->ring->id;
+ __entry->uniq = req->uniq;
__entry->invalidate = invalidate;
__entry->flush = flush;
),
- TP_printk("dev=%u, ring=%x, invalidate=%04x, flush=%04x",
- __entry->dev, __entry->ring,
+ TP_printk("dev=%u, ring=%x, request=%u, invalidate=%04x, flush=%04x",
+ __entry->dev, __entry->ring, __entry->uniq,
__entry->invalidate, __entry->flush)
);
@@ -540,6 +547,7 @@ DECLARE_EVENT_CLASS(i915_gem_request,
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
+ __field(u32, uniq)
__field(u32, seqno)
),
@@ -548,11 +556,13 @@ DECLARE_EVENT_CLASS(i915_gem_request,
i915_gem_request_get_ring(req);
__entry->dev = ring->dev->primary->index;
__entry->ring = ring->id;
+ __entry->uniq = req ? req->uniq : 0;
__entry->seqno = i915_gem_request_get_seqno(req);
),
- TP_printk("dev=%u, ring=%u, seqno=%u",
- __entry->dev, __entry->ring, __entry->seqno)
+ TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u",
+ __entry->dev, __entry->ring, __entry->uniq,
+ __entry->seqno)
);
DEFINE_EVENT(i915_gem_request, i915_gem_request_add,
@@ -601,6 +611,7 @@ TRACE_EVENT(i915_gem_request_wait_begin,
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
+ __field(u32, uniq)
__field(u32, seqno)
__field(bool, blocking)
),
@@ -616,13 +627,14 @@ TRACE_EVENT(i915_gem_request_wait_begin,
i915_gem_request_get_ring(req);
__entry->dev = ring->dev->primary->index;
__entry->ring = ring->id;
+ __entry->uniq = req ? req->uniq : 0;
__entry->seqno = i915_gem_request_get_seqno(req);
__entry->blocking =
mutex_is_locked(&ring->dev->struct_mutex);
),
- TP_printk("dev=%u, ring=%u, seqno=%u, blocking=%s",
- __entry->dev, __entry->ring,
+ TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u, blocking=%s",
+ __entry->dev, __entry->ring, __entry->uniq,
__entry->seqno, __entry->blocking ? "yes (NB)" : "no")
);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 06/35] drm/i915: Start of GPU scheduler
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (4 preceding siblings ...)
2016-02-18 14:26 ` [PATCH v5 05/35] drm/i915: Re-instate request->uniq because it is extremely useful John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-02-19 13:03 ` Joonas Lahtinen
2016-02-18 14:26 ` [PATCH v5 07/35] drm/i915: Prepare retire_requests to handle out-of-order seqnos John.C.Harrison
` (30 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Initial creation of scheduler source files. Note that this patch
implements most of the scheduler functionality but does not hook it in
to the driver yet. It also leaves the scheduler code in 'pass through'
mode so that even when it is hooked in, it will not actually do very
much. This allows the hooks to be added one at a time in bite size
chunks and only when the scheduler is finally enabled at the end does
anything start happening.
The general theory of operation is that when batch buffers are
submitted to the driver, the execbuffer() code packages up all the
information required to execute the batch buffer at a later time. This
package is given over to the scheduler which adds it to an internal
node list. The scheduler also scans the list of objects associated
with the batch buffer and compares them against the objects already in
use by other buffers in the node list. If matches are found then the
new batch buffer node is marked as being dependent upon the matching
node. The same is done for the context object. The scheduler also
bumps up the priority of such matching nodes on the grounds that the
more dependencies a given batch buffer has the more important it is
likely to be.
The scheduler aims to have a given (tuneable) number of batch buffers
in flight on the hardware at any given time. If fewer than this are
currently executing when a new node is queued, then the node is passed
straight through to the submit function. Otherwise it is simply added
to the queue and the driver returns back to user land.
The scheduler is notified when each batch buffer completes and updates
its internal tracking accordingly. At the end of the completion
interrupt processing, if any scheduler tracked batches were processed,
the scheduler's deferred worker thread is woken up. This can do more
involved processing such as actually removing completed nodes from the
queue and freeing up the resources associated with them (internal
memory allocations, DRM object references, context reference, etc.).
The work handler also checks the in flight count and calls the
submission code if a new slot has appeared.
When the scheduler's submit code is called, it scans the queued node
list for the highest priority node that has no unmet dependencies.
Note that the dependency calculation is complex as it must take
inter-ring dependencies and potential preemptions into account. Note
also that in the future this will be extended to include external
dependencies such as the Android Native Sync file descriptors and/or
the linux dma-buff synchronisation scheme.
If a suitable node is found then it is sent to execbuff_final() for
submission to the hardware. The in flight count is then re-checked and
a new node popped from the list if appropriate. All nodes that are not
submitted have their priority bumped. This ensures that low priority
tasks do not get starved out by busy higher priority ones - everything
will eventually get its turn to run.
Note that this patch does not implement pre-emptive scheduling. Only
basic scheduling by re-ordering batch buffer submission is currently
implemented. Pre-emption of actively executing batch buffers comes in
the next patch series.
v2: Changed priority levels to +/-1023 due to feedback from Chris
Wilson.
Removed redundant index from scheduler node.
Changed time stamps to use jiffies instead of raw monotonic. This
provides lower resolution but improved compatibility with other i915
code.
Major re-write of completion tracking code due to struct fence
conversion. The scheduler no longer has it's own private IRQ handler
but just lets the existing request code handle completion events.
Instead, the scheduler now hooks into the request notify code to be
told when a request has completed.
Reduced driver mutex locking scope. Removal of scheduler nodes no
longer grabs the mutex lock.
v3: Refactor of dependency generation to make the code more readable.
Also added in read-read optimisation support - i.e., don't treat a
shared read-only buffer as being a dependency.
Allowed the killing of queued nodes rather than only flying ones.
v4: Updated the commit message to better reflect the current state of
the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array
memory allocator function (kmalloc_array instead of kmalloc).
Corrected the format of some comments. Wrapped some lines differently
to keep the style checker happy.
Fixed a WARN_ON when killing nodes. The dependency removal code checks
that nodes being destroyed do not have any oustanding dependencies
(which would imply they should not have been executed yet). In the
case of nodes being destroyed, e.g. due to context banning, then this
might well be the case - they have not been executed and do indeed
have outstanding dependencies.
Re-instated the code to disble interrupts when not in use. The
underlying problem causing broken IRQ reference counts seems to have
been fixed now.
v5: Shuffled various functions around to remove forward declarations
as apparently these are frowned upon. Removed lots of white space as
apparently having easy to read code is also frowned upon. Split the
direct submission scheduler bypass code out into a separate function.
Squashed down the i915_scheduler.c sections of various patches into
this patch. Thus the later patches simply hook in existing code into
various parts of the driver rather than adding the code as well. Added
documentation to various functions. Re-worked the submit function in
terms of mutex locking, error handling and exit paths. Split the
delayed work handler function in half. Made use of the kernel 'clamp'
macro. [Joonas Lahtinen]
Added runtime PM calls as these must be done at the top level before
acquiring the driver mutex lock. [Chris Wilson]
Removed some obsolete debug code that had been forgotten about.
Moved more clean up code into the 'i915_gem_scheduler_clean_node()'
function rather than replicating it in mutliple places.
Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/Makefile | 1 +
drivers/gpu/drm/i915/i915_drv.h | 6 +
drivers/gpu/drm/i915/i915_gem.c | 5 +
drivers/gpu/drm/i915/i915_scheduler.c | 874 ++++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 95 ++++
5 files changed, 981 insertions(+)
create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index 15398c5..79cb38b 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -10,6 +10,7 @@ ccflags-y := -Werror
i915-y := i915_drv.o \
i915_irq.o \
i915_params.o \
+ i915_scheduler.o \
i915_suspend.o \
i915_sysfs.o \
intel_csr.o \
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index f4487b9..03add1a 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1703,6 +1703,8 @@ struct i915_execbuffer_params {
struct drm_i915_gem_request *request;
};
+struct i915_scheduler;
+
/* used in computing the new watermarks state */
struct intel_wm_config {
unsigned int num_pipes_active;
@@ -1968,6 +1970,8 @@ struct drm_i915_private {
struct i915_runtime_pm pm;
+ struct i915_scheduler *scheduler;
+
/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
struct {
int (*execbuf_submit)(struct i915_execbuffer_params *params,
@@ -2290,6 +2294,8 @@ struct drm_i915_gem_request {
/** process identifier submitting this request */
struct pid *pid;
+ struct i915_scheduler_queue_entry *scheduler_qe;
+
/**
* The ELSP only accepts two elements at a time, so we queue
* context/tail pairs on a given queue (ring->execlist_queue) until the
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index dfe43ea..7d9aa24 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -32,6 +32,7 @@
#include "i915_vgpu.h"
#include "i915_trace.h"
#include "intel_drv.h"
+#include "i915_scheduler.h"
#include <linux/shmem_fs.h>
#include <linux/slab.h>
#include <linux/swap.h>
@@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)
*/
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+ ret = i915_scheduler_init(dev);
+ if (ret)
+ goto out_unlock;
+
ret = i915_gem_init_userptr(dev);
if (ret)
goto out_unlock;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
new file mode 100644
index 0000000..fc23ee7
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -0,0 +1,874 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "i915_drv.h"
+#include "intel_drv.h"
+#include "i915_scheduler.h"
+
+/**
+ * i915_scheduler_is_enabled - Returns true if the scheduler is enabled.
+ * @dev: DRM device
+ */
+bool i915_scheduler_is_enabled(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+
+ return dev_priv->scheduler != NULL;
+}
+
+/**
+ * i915_scheduler_init - Initialise the scheduler.
+ * @dev: DRM device
+ * Returns zero on success or -ENOMEM if memory allocations fail.
+ */
+int i915_scheduler_init(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ int r;
+
+ if (scheduler)
+ return 0;
+
+ scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
+ if (!scheduler)
+ return -ENOMEM;
+
+ spin_lock_init(&scheduler->lock);
+
+ for (r = 0; r < I915_NUM_RINGS; r++)
+ INIT_LIST_HEAD(&scheduler->node_queue[r]);
+
+ /* Default tuning values: */
+ scheduler->priority_level_min = -1023;
+ scheduler->priority_level_max = 1023;
+ scheduler->priority_level_bump = 50;
+ scheduler->priority_level_preempt = 900;
+ scheduler->min_flying = 2;
+
+ dev_priv->scheduler = scheduler;
+
+ return 0;
+}
+
+/*
+ * Add a popped node back in to the queue. For example, because the ring was
+ * hung when execfinal() was called and thus the ring submission needs to be
+ * retried later.
+ */
+static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
+{
+ WARN_ON(!node);
+ WARN_ON(!I915_SQS_IS_FLYING(node));
+
+ /* Seqno will be reassigned on relaunch */
+ node->params.request->seqno = 0;
+ node->status = i915_sqs_queued;
+}
+
+/*
+ * Give up on a node completely. For example, because it is causing the
+ * ring to hang or is using some resource that no longer exists.
+ */
+static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
+{
+ WARN_ON(!node);
+ WARN_ON(I915_SQS_IS_COMPLETE(node));
+
+ node->status = i915_sqs_dead;
+}
+
+/* Mark a node as in flight on the hardware. */
+static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
+{
+ struct drm_i915_private *dev_priv = node->params.dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct intel_engine_cs *ring;
+
+ WARN_ON(!scheduler);
+ WARN_ON(!node);
+ WARN_ON(node->status != i915_sqs_popped);
+
+ ring = node->params.ring;
+
+ /*
+ * Add the node (which should currently be in state popped) to the
+ * front of the queue. This ensure that flying nodes are always held
+ * in hardware submission order.
+ */
+ list_add(&node->link, &scheduler->node_queue[ring->id]);
+
+ node->status = i915_sqs_flying;
+
+ if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
+ bool success = true;
+
+ success = ring->irq_get(ring);
+ if (success)
+ scheduler->flags[ring->id] |= i915_sf_interrupts_enabled;
+ else
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static uint32_t i915_scheduler_count_flying(struct i915_scheduler *scheduler,
+ struct intel_engine_cs *ring)
+{
+ struct i915_scheduler_queue_entry *node;
+ uint32_t flying = 0;
+
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link)
+ if (I915_SQS_IS_FLYING(node))
+ flying++;
+
+ return flying;
+}
+
+static void i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler)
+{
+ struct i915_scheduler_queue_entry *node;
+ int i;
+
+ /*
+ * Ensure circular dependencies don't cause problems and that a bump
+ * by object usage only bumps each using buffer once:
+ */
+ for (i = 0; i < I915_NUM_RINGS; i++) {
+ list_for_each_entry(node, &scheduler->node_queue[i], link)
+ node->bumped = false;
+ }
+}
+
+static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
+ struct i915_scheduler_queue_entry *target,
+ uint32_t bump)
+{
+ uint32_t new_priority;
+ int i, count;
+
+ if (target->priority >= scheduler->priority_level_max)
+ return 1;
+
+ if (target->bumped)
+ return 0;
+
+ new_priority = target->priority + bump;
+ if ((new_priority <= target->priority) ||
+ (new_priority > scheduler->priority_level_max))
+ target->priority = scheduler->priority_level_max;
+ else
+ target->priority = new_priority;
+
+ count = 1;
+ target->bumped = true;
+
+ for (i = 0; i < target->num_deps; i++) {
+ if (!target->dep_list[i])
+ continue;
+
+ if (target->dep_list[i]->bumped)
+ continue;
+
+ count += i915_scheduler_priority_bump(scheduler,
+ target->dep_list[i],
+ bump);
+ }
+
+ return count;
+}
+
+/*
+ * Nodes are considered valid dependencies if they are queued on any ring or
+ * if they are in flight on a different ring. In flight on the same ring is no
+ * longer interesting for non-premptive nodes as the ring serialises execution.
+ * For pre-empting nodes, all in flight dependencies are valid as they must not
+ * be jumped by the act of pre-empting.
+ *
+ * Anything that is neither queued nor flying is uninteresting.
+ */
+static inline bool i915_scheduler_is_dependency_valid(
+ struct i915_scheduler_queue_entry *node, uint32_t idx)
+{
+ struct i915_scheduler_queue_entry *dep;
+
+ dep = node->dep_list[idx];
+ if (!dep)
+ return false;
+
+ if (I915_SQS_IS_QUEUED(dep))
+ return true;
+
+ if (I915_SQS_IS_FLYING(dep)) {
+ if (node->params.ring != dep->params.ring)
+ return true;
+ }
+
+ return false;
+}
+
+static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
+ struct i915_scheduler_queue_entry **pop_node)
+{
+ struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *best = NULL;
+ struct i915_scheduler_queue_entry *node;
+ int ret;
+ int i;
+ bool any_queued = false;
+ bool has_local, has_remote, only_remote = false;
+
+ *pop_node = NULL;
+ ret = -ENODATA;
+
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (!I915_SQS_IS_QUEUED(node))
+ continue;
+ any_queued = true;
+
+ has_local = false;
+ has_remote = false;
+ for (i = 0; i < node->num_deps; i++) {
+ if (!i915_scheduler_is_dependency_valid(node, i))
+ continue;
+
+ if (node->dep_list[i]->params.ring == node->params.ring)
+ has_local = true;
+ else
+ has_remote = true;
+ }
+
+ if (has_remote && !has_local)
+ only_remote = true;
+
+ if (!has_local && !has_remote) {
+ if (!best ||
+ (node->priority > best->priority))
+ best = node;
+ }
+ }
+
+ if (best) {
+ list_del(&best->link);
+
+ INIT_LIST_HEAD(&best->link);
+ best->status = i915_sqs_popped;
+
+ ret = 0;
+ } else {
+ /* Can only get here if:
+ * (a) there are no buffers in the queue
+ * (b) all queued buffers are dependent on other buffers
+ * e.g. on a buffer that is in flight on a different ring
+ */
+ if (only_remote) {
+ /* The only dependent buffers are on another ring. */
+ ret = -EAGAIN;
+ } else if (any_queued) {
+ /* It seems that something has gone horribly wrong! */
+ DRM_ERROR("Broken dependency tracking on ring %d!\n",
+ (int) ring->id);
+ }
+ }
+
+ *pop_node = best;
+ return ret;
+}
+
+/*
+ * NB: The driver mutex lock must be held before calling this function. It is
+ * only really required during the actual back end submission call. However,
+ * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
+ * And releasing the one before acquiring the other leads to other code
+ * being run and interfering.
+ */
+static int i915_scheduler_submit(struct intel_engine_cs *ring)
+{
+ struct drm_device *dev = ring->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+ int ret, count = 0, flying;
+
+ WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+
+ spin_lock_irq(&scheduler->lock);
+
+ WARN_ON(scheduler->flags[ring->id] & i915_sf_submitting);
+ scheduler->flags[ring->id] |= i915_sf_submitting;
+
+ /* First time around, complain if anything unexpected occurs: */
+ ret = i915_scheduler_pop_from_queue_locked(ring, &node);
+ if (ret)
+ goto error;
+
+ do {
+ WARN_ON(!node);
+ WARN_ON(node->params.ring != ring);
+ WARN_ON(node->status != i915_sqs_popped);
+ count++;
+
+ /*
+ * The call to pop above will have removed the node from the
+ * list. So add it back in and mark it as in flight.
+ */
+ i915_scheduler_node_fly(node);
+
+ spin_unlock_irq(&scheduler->lock);
+ ret = dev_priv->gt.execbuf_final(&node->params);
+ spin_lock_irq(&scheduler->lock);
+
+ /*
+ * Handle failed submission but first check that the
+ * watchdog/reset code has not nuked the node while we
+ * weren't looking:
+ */
+ if (ret && (node->status != i915_sqs_dead)) {
+ bool requeue = true;
+
+ /*
+ * Oh dear! Either the node is broken or the ring is
+ * busy. So need to kill the node or requeue it and try
+ * again later as appropriate.
+ */
+
+ switch (-ret) {
+ case ENODEV:
+ case ENOENT:
+ /* Fatal errors. Kill the node. */
+ requeue = false;
+ i915_scheduler_node_kill(node);
+ break;
+
+ case EAGAIN:
+ case EBUSY:
+ case EIO:
+ case ENOMEM:
+ case ERESTARTSYS:
+ case EINTR:
+ /* Supposedly recoverable errors. */
+ break;
+
+ default:
+ /*
+ * Assume the error is recoverable and hope
+ * for the best.
+ */
+ MISSING_CASE(-ret);
+ break;
+ }
+
+ if (requeue) {
+ i915_scheduler_node_requeue(node);
+ /*
+ * No point spinning if the ring is currently
+ * unavailable so just give up and come back
+ * later.
+ */
+ break;
+ }
+ }
+
+ /* Keep launching until the sky is sufficiently full. */
+ flying = i915_scheduler_count_flying(scheduler, ring);
+ if (flying >= scheduler->min_flying)
+ break;
+
+ /* Grab another node and go round again... */
+ ret = i915_scheduler_pop_from_queue_locked(ring, &node);
+ } while (ret == 0);
+
+ /* Don't complain about not being able to submit extra entries */
+ if (ret == -ENODATA)
+ ret = 0;
+
+ /*
+ * Bump the priority of everything that was not submitted to prevent
+ * starvation of low priority tasks by a spamming high priority task.
+ */
+ i915_scheduler_priority_bump_clear(scheduler);
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (!I915_SQS_IS_QUEUED(node))
+ continue;
+
+ i915_scheduler_priority_bump(scheduler, node,
+ scheduler->priority_level_bump);
+ }
+
+ /* On success, return the number of buffers submitted. */
+ if (ret == 0)
+ ret = count;
+
+error:
+ scheduler->flags[ring->id] &= ~i915_sf_submitting;
+ spin_unlock_irq(&scheduler->lock);
+ return ret;
+}
+
+static void i915_generate_dependencies(struct i915_scheduler *scheduler,
+ struct i915_scheduler_queue_entry *node,
+ uint32_t ring)
+{
+ struct i915_scheduler_obj_entry *this, *that;
+ struct i915_scheduler_queue_entry *test;
+ int i, j;
+ bool found;
+
+ list_for_each_entry(test, &scheduler->node_queue[ring], link) {
+ if (I915_SQS_IS_COMPLETE(test))
+ continue;
+
+ /*
+ * Batches on the same ring for the same
+ * context must be kept in order.
+ */
+ found = (node->params.ctx == test->params.ctx) &&
+ (node->params.ring == test->params.ring);
+
+ /*
+ * Batches working on the same objects must
+ * be kept in order.
+ */
+ for (i = 0; (i < node->num_objs) && !found; i++) {
+ this = node->saved_objects + i;
+
+ for (j = 0; j < test->num_objs; j++) {
+ that = test->saved_objects + j;
+
+ if (this->obj != that->obj)
+ continue;
+
+ /* Only need to worry about writes */
+ if (this->read_only && that->read_only)
+ continue;
+
+ found = true;
+ break;
+ }
+ }
+
+ if (found) {
+ node->dep_list[node->num_deps] = test;
+ node->num_deps++;
+ }
+ }
+}
+
+static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_entry *qe)
+{
+ struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ int ret;
+
+ scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
+ ret = dev_priv->gt.execbuf_final(&qe->params);
+ scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
+
+ /*
+ * Don't do any clean up on failure because the caller will
+ * do it all anyway.
+ */
+ if (ret)
+ return ret;
+
+ /* Need to release any resources held by the node: */
+ i915_scheduler_clean_node(qe);
+
+ return 0;
+}
+
+/**
+ * i915_scheduler_queue_execbuffer - Submit a batch buffer request to the
+ * scheduler.
+ * @qe: The batch buffer request to be queued.
+ * The expectation is the qe passed in is a local stack variable. This
+ * function will copy its contents into a freshly allocated list node. The
+ * new node takes ownership of said contents so the original qe should simply
+ * be discarded and not cleaned up (i.e. don't free memory it points to or
+ * dereference objects it holds). The node is added to the scheduler's queue
+ * and the batch buffer will be submitted to the hardware at some future
+ * point in time (which may be immediately, before returning or may be quite
+ * a lot later).
+ */
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
+{
+ struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct intel_engine_cs *ring = qe->params.ring;
+ struct i915_scheduler_queue_entry *node;
+ struct i915_scheduler_queue_entry *test;
+ bool not_flying;
+ int i, r;
+ int incomplete = 0;
+
+ WARN_ON(!scheduler);
+
+ if (1/*!i915.enable_scheduler*/)
+ return i915_scheduler_queue_execbuffer_bypass(qe);
+
+ node = kmalloc(sizeof(*node), GFP_KERNEL);
+ if (!node)
+ return -ENOMEM;
+
+ *node = *qe;
+ INIT_LIST_HEAD(&node->link);
+ node->status = i915_sqs_queued;
+ node->stamp = jiffies;
+ i915_gem_request_reference(node->params.request);
+
+ WARN_ON(node->params.request->scheduler_qe);
+ node->params.request->scheduler_qe = node;
+
+ /* Need to determine the number of incomplete entries in the list as
+ * that will be the maximum size of the dependency list.
+ *
+ * Note that the allocation must not be made with the spinlock acquired
+ * as kmalloc can sleep. However, the unlock/relock is safe because no
+ * new entries can be queued up during the unlock as the i915 driver
+ * mutex is still held. Entries could be removed from the list but that
+ * just means the dep_list will be over-allocated which is fine.
+ */
+ spin_lock_irq(&scheduler->lock);
+ for (r = 0; r < I915_NUM_RINGS; r++) {
+ list_for_each_entry(test, &scheduler->node_queue[r], link) {
+ if (I915_SQS_IS_COMPLETE(test))
+ continue;
+
+ incomplete++;
+ }
+ }
+
+ /* Temporarily unlock to allocate memory: */
+ spin_unlock_irq(&scheduler->lock);
+ if (incomplete) {
+ node->dep_list = kmalloc_array(incomplete,
+ sizeof(*node->dep_list),
+ GFP_KERNEL);
+ if (!node->dep_list) {
+ kfree(node);
+ return -ENOMEM;
+ }
+ } else
+ node->dep_list = NULL;
+
+ spin_lock_irq(&scheduler->lock);
+ node->num_deps = 0;
+
+ if (node->dep_list) {
+ for (r = 0; r < I915_NUM_RINGS; r++)
+ i915_generate_dependencies(scheduler, node, r);
+
+ WARN_ON(node->num_deps > incomplete);
+ }
+
+ node->priority = clamp(node->priority,
+ scheduler->priority_level_min,
+ scheduler->priority_level_max);
+
+ if ((node->priority > 0) && node->num_deps) {
+ i915_scheduler_priority_bump_clear(scheduler);
+
+ for (i = 0; i < node->num_deps; i++)
+ i915_scheduler_priority_bump(scheduler,
+ node->dep_list[i], node->priority);
+ }
+
+ list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
+
+ not_flying = i915_scheduler_count_flying(scheduler, ring) <
+ scheduler->min_flying;
+
+ spin_unlock_irq(&scheduler->lock);
+
+ if (not_flying)
+ i915_scheduler_submit(ring);
+
+ return 0;
+}
+
+/**
+ * i915_scheduler_notify_request - Notify the scheduler that the given
+ * request has completed on the hardware.
+ * @req: Request structure which has completed
+ * @preempt: Did it complete pre-emptively?
+ * A sequence number has popped out of the hardware and the request handling
+ * code has mapped it back to a request and will mark that request complete.
+ * It also calls this function to notify the scheduler about the completion
+ * so the scheduler's node can be updated appropriately.
+ * Returns true if the request is scheduler managed, false if not. The return
+ * value is combined for all freshly completed requests and if any were true
+ * then i915_scheduler_wakeup() is called so the scheduler can do further
+ * processing (submit more work) at the end.
+ */
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
+{
+ struct drm_i915_private *dev_priv = to_i915(req->ring->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node = req->scheduler_qe;
+ unsigned long flags;
+
+ if (!node)
+ return false;
+
+ spin_lock_irqsave(&scheduler->lock, flags);
+
+ WARN_ON(!I915_SQS_IS_FLYING(node));
+
+ /* Node was in flight so mark it as complete. */
+ if (req->cancelled)
+ node->status = i915_sqs_dead;
+ else
+ node->status = i915_sqs_complete;
+
+ spin_unlock_irqrestore(&scheduler->lock, flags);
+
+ return true;
+}
+
+static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
+ struct i915_scheduler_queue_entry *remove)
+{
+ struct i915_scheduler_queue_entry *node;
+ int i, r;
+ int count = 0;
+
+ /*
+ * Ensure that a node is not being removed which is still dependent
+ * upon other (not completed) work. If that happens, it implies
+ * something has gone very wrong with the dependency tracking! Note
+ * that there is no need to worry if this node has been explicitly
+ * killed for some reason - it might be being killed before it got
+ * sent to the hardware.
+ */
+ if (remove->status != i915_sqs_dead) {
+ for (i = 0; i < remove->num_deps; i++)
+ if ((remove->dep_list[i]) &&
+ (!I915_SQS_IS_COMPLETE(remove->dep_list[i])))
+ count++;
+ WARN_ON(count);
+ }
+
+ /*
+ * Remove this node from the dependency lists of any other node which
+ * might be waiting on it.
+ */
+ for (r = 0; r < I915_NUM_RINGS; r++) {
+ list_for_each_entry(node, &scheduler->node_queue[r], link) {
+ for (i = 0; i < node->num_deps; i++) {
+ if (node->dep_list[i] != remove)
+ continue;
+
+ node->dep_list[i] = NULL;
+ }
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * i915_scheduler_wakeup - wake the scheduler's worker thread
+ * @dev: DRM device
+ * Called at the end of seqno interrupt processing if any request has
+ * completed that corresponds to a scheduler node.
+ */
+void i915_scheduler_wakeup(struct drm_device *dev)
+{
+ /* XXX: Need to call i915_scheduler_remove() via work handler. */
+}
+
+/**
+ * i915_scheduler_clean_node - free up any allocations/references
+ * associated with the given scheduler queue entry.
+ * @node: Queue entry structure which is complete
+ * After a give batch buffer completes on the hardware, all the information
+ * required to resubmit it is no longer required. However, the node entry
+ * itself might still be required for tracking purposes for a while longer.
+ * This function should be called as soon as the node is known to be complete
+ * so that these resources may be freed even though the node itself might
+ * hang around.
+ */
+void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
+{
+ if (!I915_SQS_IS_COMPLETE(node)) {
+ WARN(!node->params.request->cancelled,
+ "Cleaning active node: %d!\n", node->status);
+ return;
+ }
+
+ if (node->params.batch_obj) {
+ /*
+ * The batch buffer must be unpinned before it is unreferenced
+ * otherwise the unpin fails with a missing vma!?
+ */
+ if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
+ i915_gem_execbuff_release_batch_obj(node->params.batch_obj);
+
+ node->params.batch_obj = NULL;
+ }
+
+ /* And anything else owned by the node: */
+ if (node->params.cliprects) {
+ kfree(node->params.cliprects);
+ node->params.cliprects = NULL;
+ }
+}
+
+static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
+ struct intel_engine_cs *ring,
+ struct list_head *remove)
+{
+ struct i915_scheduler_queue_entry *node, *node_next;
+ int flying = 0, queued = 0;
+ bool do_submit;
+ uint32_t min_seqno;
+
+ spin_lock_irq(&scheduler->lock);
+
+ /*
+ * In the case where the system is idle, starting 'min_seqno' from a big
+ * number will cause all nodes to be removed as they are now back to
+ * being in-order. However, this will be a problem if the last one to
+ * complete was actually out-of-order as the ring seqno value will be
+ * lower than one or more completed buffers. Thus code looking for the
+ * completion of said buffers will wait forever.
+ * Instead, use the hardware seqno as the starting point. This means
+ * that some buffers might be kept around even in a completely idle
+ * system but it should guarantee that no-one ever gets confused when
+ * waiting for buffer completion.
+ */
+ min_seqno = ring->get_seqno(ring, true);
+
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (I915_SQS_IS_QUEUED(node))
+ queued++;
+ else if (I915_SQS_IS_FLYING(node))
+ flying++;
+ else if (I915_SQS_IS_COMPLETE(node))
+ continue;
+
+ if (node->params.request->seqno == 0)
+ continue;
+
+ if (!i915_seqno_passed(node->params.request->seqno, min_seqno))
+ min_seqno = node->params.request->seqno;
+ }
+
+ INIT_LIST_HEAD(remove);
+ list_for_each_entry_safe(node, node_next, &scheduler->node_queue[ring->id], link) {
+ /*
+ * Only remove completed nodes which have a lower seqno than
+ * all pending nodes. While there is the possibility of the
+ * ring's seqno counting backwards, all higher buffers must
+ * be remembered so that the 'i915_seqno_passed()' test can
+ * report that they have in fact passed.
+ *
+ * NB: This is not true for 'dead' nodes. The GPU reset causes
+ * the software seqno to restart from its initial value. Thus
+ * the dead nodes must be removed even though their seqno values
+ * are potentially vastly greater than the current ring seqno.
+ */
+ if (!I915_SQS_IS_COMPLETE(node))
+ continue;
+
+ if (node->status != i915_sqs_dead) {
+ if (i915_seqno_passed(node->params.request->seqno, min_seqno) &&
+ (node->params.request->seqno != min_seqno))
+ continue;
+ }
+
+ list_del(&node->link);
+ list_add(&node->link, remove);
+
+ /* Strip the dependency info while the mutex is still locked */
+ i915_scheduler_remove_dependent(scheduler, node);
+
+ continue;
+ }
+
+ /*
+ * Release the interrupt reference count if there are no longer any
+ * nodes to worry about.
+ */
+ if (!flying && !queued &&
+ (scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
+ ring->irq_put(ring);
+ scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;
+ }
+
+ /* Launch more packets now? */
+ do_submit = (queued > 0) && (flying < scheduler->min_flying);
+
+ spin_unlock_irq(&scheduler->lock);
+
+ return do_submit;
+}
+
+void i915_scheduler_process_work(struct intel_engine_cs *ring)
+{
+ struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+ bool do_submit;
+ struct list_head remove;
+
+ if (list_empty(&scheduler->node_queue[ring->id]))
+ return;
+
+ /* Remove completed nodes. */
+ do_submit = i915_scheduler_remove(scheduler, ring, &remove);
+
+ if (!do_submit && list_empty(&remove))
+ return;
+
+ /* Need to grab the pm lock outside of the mutex lock */
+ if (do_submit)
+ intel_runtime_pm_get(dev_priv);
+
+ mutex_lock(&ring->dev->struct_mutex);
+
+ if (do_submit)
+ i915_scheduler_submit(ring);
+
+ while (!list_empty(&remove)) {
+ node = list_first_entry(&remove, typeof(*node), link);
+ list_del(&node->link);
+
+ /* Free up all the DRM references */
+ i915_scheduler_clean_node(node);
+
+ /* And anything else owned by the node: */
+ node->params.request->scheduler_qe = NULL;
+ i915_gem_request_unreference(node->params.request);
+ kfree(node->dep_list);
+ kfree(node);
+ }
+
+ mutex_unlock(&ring->dev->struct_mutex);
+
+ if (do_submit)
+ intel_runtime_pm_put(dev_priv);
+}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
new file mode 100644
index 0000000..415fec8
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -0,0 +1,95 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef _I915_SCHEDULER_H_
+#define _I915_SCHEDULER_H_
+
+enum i915_scheduler_queue_status {
+ /* Limbo: */
+ i915_sqs_none = 0,
+ /* Not yet submitted to hardware: */
+ i915_sqs_queued,
+ /* Popped from queue, ready to fly: */
+ i915_sqs_popped,
+ /* Sent to hardware for processing: */
+ i915_sqs_flying,
+ /* Finished processing on the hardware: */
+ i915_sqs_complete,
+ /* Killed by watchdog or catastrophic submission failure: */
+ i915_sqs_dead,
+ /* Limit value for use with arrays/loops */
+ i915_sqs_MAX
+};
+
+#define I915_SQS_IS_QUEUED(node) (((node)->status == i915_sqs_queued))
+#define I915_SQS_IS_FLYING(node) (((node)->status == i915_sqs_flying))
+#define I915_SQS_IS_COMPLETE(node) (((node)->status == i915_sqs_complete) || \
+ ((node)->status == i915_sqs_dead))
+
+struct i915_scheduler_obj_entry {
+ struct drm_i915_gem_object *obj;
+ bool read_only;
+};
+
+struct i915_scheduler_queue_entry {
+ struct i915_execbuffer_params params;
+ /* -1023 = lowest priority, 0 = default, 1023 = highest */
+ int32_t priority;
+ struct i915_scheduler_obj_entry *saved_objects;
+ int num_objs;
+ bool bumped;
+ struct i915_scheduler_queue_entry **dep_list;
+ int num_deps;
+ enum i915_scheduler_queue_status status;
+ unsigned long stamp;
+ struct list_head link;
+};
+
+struct i915_scheduler {
+ struct list_head node_queue[I915_NUM_RINGS];
+ uint32_t flags[I915_NUM_RINGS];
+ spinlock_t lock;
+
+ /* Tuning parameters: */
+ int32_t priority_level_min;
+ int32_t priority_level_max;
+ int32_t priority_level_bump;
+ int32_t priority_level_preempt;
+ uint32_t min_flying;
+};
+
+/* Flag bits for i915_scheduler::flags */
+enum {
+ i915_sf_interrupts_enabled = (1 << 0),
+ i915_sf_submitting = (1 << 1),
+};
+
+bool i915_scheduler_is_enabled(struct drm_device *dev);
+int i915_scheduler_init(struct drm_device *dev);
+void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
+void i915_scheduler_wakeup(struct drm_device *dev);
+
+#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 07/35] drm/i915: Prepare retire_requests to handle out-of-order seqnos
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (5 preceding siblings ...)
2016-02-18 14:26 ` [PATCH v5 06/35] drm/i915: Start of GPU scheduler John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-02-19 19:23 ` Jesse Barnes
2016-02-18 14:26 ` [PATCH v5 08/35] drm/i915: Disable hardware semaphores when GPU scheduler is enabled John.C.Harrison
` (29 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
A major point of the GPU scheduler is that it re-orders batch buffers
after they have been submitted to the driver. This leads to requests
completing out of order. In turn, this means that the retire
processing can no longer assume that all completed entries are at the
front of the list. Rather than attempting to re-order the request list
on a regular basis, it is better to simply scan the entire list.
v2: Removed deferred free code as no longer necessary due to request
handling updates.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 31 +++++++++++++------------------
1 file changed, 13 insertions(+), 18 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 7d9aa24..0003cfc 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3233,6 +3233,7 @@ void i915_gem_reset(struct drm_device *dev)
void
i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
{
+ struct drm_i915_gem_object *obj, *obj_next;
struct drm_i915_gem_request *req, *req_next;
LIST_HEAD(list_head);
@@ -3245,37 +3246,31 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
*/
i915_gem_request_notify(ring, false);
+ /*
+ * Note that request entries might be out of order due to rescheduling
+ * and pre-emption. Thus both lists must be processed in their entirety
+ * rather than stopping at the first non-complete entry.
+ */
+
/* Retire requests first as we use it above for the early return.
* If we retire requests last, we may use a later seqno and so clear
* the requests lists without clearing the active list, leading to
* confusion.
*/
- while (!list_empty(&ring->request_list)) {
- struct drm_i915_gem_request *request;
-
- request = list_first_entry(&ring->request_list,
- struct drm_i915_gem_request,
- list);
-
- if (!i915_gem_request_completed(request))
- break;
+ list_for_each_entry_safe(req, req_next, &ring->request_list, list) {
+ if (!i915_gem_request_completed(req))
+ continue;
- i915_gem_request_retire(request);
+ i915_gem_request_retire(req);
}
/* Move any buffers on the active list that are no longer referenced
* by the ringbuffer to the flushing/inactive lists as appropriate,
* before we free the context associated with the requests.
*/
- while (!list_empty(&ring->active_list)) {
- struct drm_i915_gem_object *obj;
-
- obj = list_first_entry(&ring->active_list,
- struct drm_i915_gem_object,
- ring_list[ring->id]);
-
+ list_for_each_entry_safe(obj, obj_next, &ring->active_list, ring_list[ring->id]) {
if (!list_empty(&obj->last_read_req[ring->id]->list))
- break;
+ continue;
i915_gem_object_retire__read(obj, ring->id);
}
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 08/35] drm/i915: Disable hardware semaphores when GPU scheduler is enabled
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (6 preceding siblings ...)
2016-02-18 14:26 ` [PATCH v5 07/35] drm/i915: Prepare retire_requests to handle out-of-order seqnos John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-02-19 19:27 ` Jesse Barnes
2016-02-18 14:26 ` [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled John.C.Harrison
` (28 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Hardware sempahores require seqno values to be continuously
incrementing. However, the scheduler's reordering of batch buffers
means that the seqno values going through the hardware could be out of
order. Thus semaphores can not be used.
On the other hand, the scheduler superceeds the need for hardware
semaphores anyway. Having one ring stall waiting for something to
complete on another ring is inefficient if that ring could be working
on some other, independent task. This is what the scheduler is meant
to do - keep the hardware as busy as possible by reordering batch
buffers to avoid dependency stalls.
v4: Downgraded a BUG_ON to WARN_ON as the latter is preferred.
v5: Squashed the i915_scheduler.c portions down into the 'start of
scheduler' patch. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_drv.c | 9 +++++++++
drivers/gpu/drm/i915/intel_ringbuffer.c | 4 ++++
2 files changed, 13 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
index 975af35..5760a17 100644
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ b/drivers/gpu/drm/i915/i915_drv.c
@@ -34,6 +34,7 @@
#include "i915_drv.h"
#include "i915_trace.h"
#include "intel_drv.h"
+#include "i915_scheduler.h"
#include <linux/console.h>
#include <linux/module.h>
@@ -517,6 +518,14 @@ void intel_detect_pch(struct drm_device *dev)
bool i915_semaphore_is_enabled(struct drm_device *dev)
{
+ /* Hardware semaphores are not compatible with the scheduler due to the
+ * seqno values being potentially out of order. However, semaphores are
+ * also not required as the scheduler will handle interring dependencies
+ * and try do so in a way that does not cause dead time on the hardware.
+ */
+ if (i915_scheduler_is_enabled(dev))
+ return false;
+
if (INTEL_INFO(dev)->gen < 6)
return false;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 9d4f19d..ca7b8af 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -33,6 +33,7 @@
#include <drm/i915_drm.h>
#include "i915_trace.h"
#include "intel_drv.h"
+#include "i915_scheduler.h"
int __intel_ring_space(int head, int tail, int size)
{
@@ -1400,6 +1401,9 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,
u32 wait_mbox = signaller->semaphore.mbox.wait[waiter->id];
int ret;
+ /* Arithmetic on sequence numbers is unreliable with a scheduler. */
+ WARN_ON(i915_scheduler_is_enabled(signaller->dev));
+
/* Throughout all of the GEM code, seqno passed implies our current
* seqno is >= the last seqno executed. However for hardware the
* comparison is strictly greater than.
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (7 preceding siblings ...)
2016-02-18 14:26 ` [PATCH v5 08/35] drm/i915: Disable hardware semaphores when GPU scheduler is enabled John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-02-19 19:28 ` Jesse Barnes
2016-02-18 14:26 ` [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles John.C.Harrison
` (27 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
MMIO flips are the preferred mechanism now but more importantly, pipe
based flips cause issues for the scheduler. Specifically, submitting
work to the rings around the side of the scheduler could cause that
work to be lost if the scheduler generates a pre-emption event on that
ring.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/intel_display.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 6e12ed7..731d20a 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -46,6 +46,7 @@
#include <linux/dma_remapping.h>
#include <linux/reservation.h>
#include <linux/dma-buf.h>
+#include "i915_scheduler.h"
/* Primary plane formats for gen <= 3 */
static const uint32_t i8xx_primary_formats[] = {
@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
return true;
else if (i915.enable_execlists)
return true;
+ else if (i915_scheduler_is_enabled(ring->dev))
+ return true;
else if (obj->base.dma_buf &&
!reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
false))
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (8 preceding siblings ...)
2016-02-18 14:26 ` [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-03-01 8:59 ` Joonas Lahtinen
2016-02-18 14:26 ` [PATCH v5 11/35] drm/i915: Added scheduler hook into i915_gem_request_notify() John.C.Harrison
` (26 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler decouples the submission of batch buffers to the driver
with submission of batch buffers to the hardware. Thus it is possible
for an application to close its DRM file handle while there is still
work outstanding. That means the scheduler needs to know about file
close events so it can remove the file pointer from such orphaned
batch buffers and not attempt to dereference it later.
v3: Updated to not wait for outstanding work to complete but merely
remove the file handle reference. The wait was getting excessively
complicated with inter-ring dependencies, pre-emption, and other such
issues.
v4: Changed some white space to keep the style checker happy.
v5: Added function documentation and removed apparently objectionable
white space. [Joonas Lahtinen]
Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_dma.c | 3 +++
drivers/gpu/drm/i915/i915_scheduler.c | 48 +++++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 2 ++
3 files changed, 53 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
index a0f5659..678adc7 100644
--- a/drivers/gpu/drm/i915/i915_dma.c
+++ b/drivers/gpu/drm/i915/i915_dma.c
@@ -46,6 +46,7 @@
#include <linux/vga_switcheroo.h>
#include <linux/slab.h>
#include <acpi/video.h>
+#include "i915_scheduler.h"
#include <linux/pm.h>
#include <linux/pm_runtime.h>
#include <linux/oom.h>
@@ -1258,6 +1259,8 @@ void i915_driver_lastclose(struct drm_device *dev)
void i915_driver_preclose(struct drm_device *dev, struct drm_file *file)
{
+ i915_scheduler_closefile(dev, file);
+
mutex_lock(&dev->struct_mutex);
i915_gem_context_close(dev, file);
i915_gem_release(dev, file);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index fc23ee7..ab5007a 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -872,3 +872,51 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
if (do_submit)
intel_runtime_pm_put(dev_priv);
}
+
+/**
+ * i915_scheduler_closefile - notify the scheduler that a DRM file handle
+ * has been closed.
+ * @dev: DRM device
+ * @file: file being closed
+ *
+ * Goes through the scheduler's queues and removes all connections to the
+ * disappearing file handle that still exist. There is an argument to say
+ * that this should also flush such outstanding work through the hardware.
+ * However, with pre-emption, TDR and other such complications doing so
+ * becomes a locking nightmare. So instead, just warn with a debug message
+ * if the application is leaking uncompleted work and make sure a null
+ * pointer dereference will not follow.
+ */
+int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
+{
+ struct i915_scheduler_queue_entry *node;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct intel_engine_cs *ring;
+ int i;
+
+ if (!scheduler)
+ return 0;
+
+ spin_lock_irq(&scheduler->lock);
+
+ for_each_ring(ring, dev_priv, i) {
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (node->params.file != file)
+ continue;
+
+ if (!I915_SQS_IS_COMPLETE(node))
+ DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
+ node->params.request->uniq,
+ node->params.request->seqno,
+ node->status,
+ ring->name);
+
+ node->params.file = NULL;
+ }
+ }
+
+ spin_unlock_irq(&scheduler->lock);
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 415fec8..0e8b6a9 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -87,6 +87,8 @@ enum {
bool i915_scheduler_is_enabled(struct drm_device *dev);
int i915_scheduler_init(struct drm_device *dev);
+int i915_scheduler_closefile(struct drm_device *dev,
+ struct drm_file *file);
void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 11/35] drm/i915: Added scheduler hook into i915_gem_request_notify()
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (9 preceding siblings ...)
2016-02-18 14:26 ` [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles John.C.Harrison
@ 2016-02-18 14:26 ` John.C.Harrison
2016-03-01 9:10 ` Joonas Lahtinen
2016-02-18 14:27 ` [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler John.C.Harrison
` (25 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:26 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler needs to know when requests have completed so that it
can keep its own internal state up to date and can submit new requests
to the hardware from its queue.
v2: Updated due to changes in request handling. The operation is now
reversed from before. Rather than the scheduler being in control of
completion events, it is now the request code itself. The scheduler
merely receives a notification event. It can then optionally request
it's worker thread be woken up after all completion processing is
complete.
v4: Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
v5: Squashed the i915_scheduler.c portions down into the 'start of
scheduler' patch. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 0003cfc..c3b7def 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2872,6 +2872,7 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
{
struct drm_i915_gem_request *req, *req_next;
unsigned long flags;
+ bool wake_sched = false;
u32 seqno;
if (list_empty(&ring->fence_signal_list)) {
@@ -2908,6 +2909,14 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
*/
list_del_init(&req->signal_link);
+ /*
+ * NB: Must notify the scheduler before signalling
+ * the node. Otherwise the node can get retired first
+ * and call scheduler_clean() while the scheduler
+ * thinks it is still active.
+ */
+ wake_sched |= i915_scheduler_notify_request(req);
+
if (!req->cancelled) {
fence_signal_locked(&req->fence);
trace_i915_gem_request_complete(req);
@@ -2924,6 +2933,13 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
if (!fence_locked)
spin_unlock_irqrestore(&ring->fence_lock, flags);
+
+ /* Necessary? Or does the fence_signal() call do an implicit wakeup? */
+ wake_up_all(&ring->irq_queue);
+
+ /* Final scheduler processing after all individual updates are done. */
+ if (wake_sched)
+ i915_scheduler_wakeup(ring->dev);
}
static const char *i915_gem_request_get_driver_name(struct fence *req_fence)
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (10 preceding siblings ...)
2016-02-18 14:26 ` [PATCH v5 11/35] drm/i915: Added scheduler hook into i915_gem_request_notify() John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-03-01 9:16 ` Joonas Lahtinen
2016-02-18 14:27 ` [PATCH v5 13/35] drm/i915: Redirect execbuffer_final() via scheduler John.C.Harrison
` (24 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler needs to do interrupt triggered work that is too complex
to do in the interrupt handler. Thus it requires a deferred work
handler to process such tasks asynchronously.
v2: Updated to reduce mutex lock usage. The lock is now only held for
the minimum time within the remove function rather than for the whole
of the worker thread's operation.
v5: Removed objectionable white space and added some documentation.
[Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_dma.c | 3 +++
drivers/gpu/drm/i915/i915_drv.h | 10 ++++++++++
drivers/gpu/drm/i915/i915_gem.c | 2 ++
drivers/gpu/drm/i915/i915_scheduler.c | 29 +++++++++++++++++++++++++++--
drivers/gpu/drm/i915/i915_scheduler.h | 1 +
5 files changed, 43 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
index 678adc7..c3d382d 100644
--- a/drivers/gpu/drm/i915/i915_dma.c
+++ b/drivers/gpu/drm/i915/i915_dma.c
@@ -1158,6 +1158,9 @@ int i915_driver_unload(struct drm_device *dev)
WARN_ON(unregister_oom_notifier(&dev_priv->mm.oom_notifier));
unregister_shrinker(&dev_priv->mm.shrinker);
+ /* Cancel the scheduler work handler, which should be idle now. */
+ cancel_work_sync(&dev_priv->mm.scheduler_work);
+
io_mapping_free(dev_priv->gtt.mappable);
arch_phys_wc_del(dev_priv->gtt.mtrr);
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 03add1a..4d544f1 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1291,6 +1291,16 @@ struct i915_gem_mm {
struct delayed_work retire_work;
/**
+ * New scheme is to get an interrupt after every work packet
+ * in order to allow the low latency scheduling of pending
+ * packets. The idea behind adding new packets to a pending
+ * queue rather than directly into the hardware ring buffer
+ * is to allow high priority packets to over take low priority
+ * ones.
+ */
+ struct work_struct scheduler_work;
+
+ /**
* When we detect an idle GPU, we want to turn on
* powersaving features. So once we see that there
* are no more requests outstanding and no more
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index c3b7def..1ab7256 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -5427,6 +5427,8 @@ i915_gem_load(struct drm_device *dev)
i915_gem_retire_work_handler);
INIT_DELAYED_WORK(&dev_priv->mm.idle_work,
i915_gem_idle_work_handler);
+ INIT_WORK(&dev_priv->mm.scheduler_work,
+ i915_scheduler_work_handler);
init_waitqueue_head(&dev_priv->gpu_error.reset_queue);
dev_priv->relative_constants_mode = I915_EXEC_CONSTANTS_REL_GENERAL;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index ab5007a..3986890 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -697,7 +697,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
*/
void i915_scheduler_wakeup(struct drm_device *dev)
{
- /* XXX: Need to call i915_scheduler_remove() via work handler. */
+ struct drm_i915_private *dev_priv = to_i915(dev);
+
+ queue_work(dev_priv->wq, &dev_priv->mm.scheduler_work);
}
/**
@@ -827,7 +829,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
return do_submit;
}
-void i915_scheduler_process_work(struct intel_engine_cs *ring)
+static void i915_scheduler_process_work(struct intel_engine_cs *ring)
{
struct drm_i915_private *dev_priv = ring->dev->dev_private;
struct i915_scheduler *scheduler = dev_priv->scheduler;
@@ -874,6 +876,29 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
}
/**
+ * i915_scheduler_work_handler - scheduler's work handler callback.
+ * @work: Work structure
+ * A lot of the scheduler's work must be done asynchronously in response to
+ * an interrupt or other event. However, that work cannot be done at
+ * interrupt time or in the context of the event signaller (which might in
+ * fact be an interrupt). Thus a worker thread is required. This function
+ * will cause the thread to wake up and do its processing.
+ */
+void i915_scheduler_work_handler(struct work_struct *work)
+{
+ struct intel_engine_cs *ring;
+ struct drm_i915_private *dev_priv;
+ struct drm_device *dev;
+ int i;
+
+ dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work);
+ dev = dev_priv->dev;
+
+ for_each_ring(ring, dev_priv, i)
+ i915_scheduler_process_work(ring);
+}
+
+/**
* i915_scheduler_closefile - notify the scheduler that a DRM file handle
* has been closed.
* @dev: DRM device
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 0e8b6a9..180d75f 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -93,5 +93,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
void i915_scheduler_wakeup(struct drm_device *dev);
+void i915_scheduler_work_handler(struct work_struct *work);
#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 13/35] drm/i915: Redirect execbuffer_final() via scheduler
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (11 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-19 19:33 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 14/35] drm/i915: Keep the reserved space mechanism happy John.C.Harrison
` (23 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Updated the execbuffer() code to pass the packaged up batch buffer
information to the scheduler rather than calling execbuffer_final()
directly. The scheduler queue() code is currently a stub which simply
chains on to _final() immediately.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 18 +++++++-----------
drivers/gpu/drm/i915/intel_lrc.c | 12 ++++--------
2 files changed, 11 insertions(+), 19 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 7978dae..09c5ce9 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -33,6 +33,7 @@
#include "intel_drv.h"
#include <linux/dma_remapping.h>
#include <linux/uaccess.h>
+#include "i915_scheduler.h"
#define __EXEC_OBJECT_HAS_PIN (1<<31)
#define __EXEC_OBJECT_HAS_FENCE (1<<30)
@@ -1226,6 +1227,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas)
{
+ struct i915_scheduler_queue_entry *qe;
struct drm_device *dev = params->dev;
struct intel_engine_cs *ring = params->ring;
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -1270,17 +1272,11 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
- ret = dev_priv->gt.execbuf_final(params);
+ qe = container_of(params, typeof(*qe), params);
+ ret = i915_scheduler_queue_execbuffer(qe);
if (ret)
return ret;
- /*
- * Free everything that was stored in the QE structure (until the
- * scheduler arrives and does it instead):
- */
- if (params->dispatch_flags & I915_DISPATCH_SECURE)
- i915_gem_execbuff_release_batch_obj(params->batch_obj);
-
return 0;
}
@@ -1420,8 +1416,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
struct intel_engine_cs *ring;
struct intel_context *ctx;
struct i915_address_space *vm;
- struct i915_execbuffer_params params_master; /* XXX: will be removed later */
- struct i915_execbuffer_params *params = ¶ms_master;
+ struct i915_scheduler_queue_entry qe;
+ struct i915_execbuffer_params *params = &qe.params;
const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
u32 dispatch_flags;
int ret;
@@ -1529,7 +1525,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
else
vm = &dev_priv->gtt.base;
- memset(¶ms_master, 0x00, sizeof(params_master));
+ memset(&qe, 0x00, sizeof(qe));
eb = eb_create(args);
if (eb == NULL) {
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 12e8949..ff4565f 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -136,6 +136,7 @@
#include <drm/i915_drm.h>
#include "i915_drv.h"
#include "intel_mocs.h"
+#include "i915_scheduler.h"
#define GEN9_LR_CONTEXT_RENDER_SIZE (22 * PAGE_SIZE)
#define GEN8_LR_CONTEXT_RENDER_SIZE (20 * PAGE_SIZE)
@@ -910,6 +911,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas)
{
+ struct i915_scheduler_queue_entry *qe;
struct drm_device *dev = params->dev;
struct intel_engine_cs *ring = params->ring;
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -952,17 +954,11 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
- ret = dev_priv->gt.execbuf_final(params);
+ qe = container_of(params, typeof(*qe), params);
+ ret = i915_scheduler_queue_execbuffer(qe);
if (ret)
return ret;
- /*
- * Free everything that was stored in the QE structure (until the
- * scheduler arrives and does it instead):
- */
- if (params->dispatch_flags & I915_DISPATCH_SECURE)
- i915_gem_execbuff_release_batch_obj(params->batch_obj);
-
return 0;
}
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 14/35] drm/i915: Keep the reserved space mechanism happy
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (12 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 13/35] drm/i915: Redirect execbuffer_final() via scheduler John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-19 19:36 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 15/35] drm/i915: Added tracking/locking of batch buffer objects John.C.Harrison
` (22 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Ring space is reserved when constructing a request to ensure that the
subsequent 'add_request()' call cannot fail due to waiting for space
on a busy or broken GPU. However, the scheduler jumps in to the middle
of the execbuffer process between request creation and request
submission. Thus it needs to cancel the reserved space when the
request is simply added to the scheduler's queue and not yet
submitted. Similarly, it needs to re-reserve the space when it finally
does want to send the batch buffer to the hardware.
v3: Updated to use locally cached request pointer.
v5: Updated due to changes to earlier patches in series - for runtime
PM calls and splitting bypass mode into a separate function.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 20 ++++++++++++++------
drivers/gpu/drm/i915/i915_scheduler.c | 4 ++++
drivers/gpu/drm/i915/intel_lrc.c | 13 +++++++++++--
3 files changed, 29 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 09c5ce9..11bea8d 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1295,18 +1295,22 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ ret = intel_ring_reserve_space(req);
+ if (ret)
+ goto error;
+
/*
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
ret = intel_ring_invalidate_all_caches(req);
if (ret)
- return ret;
+ goto error;
/* Switch to the correct context for the batch */
ret = i915_switch_context(req);
if (ret)
- return ret;
+ goto error;
WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
"%s didn't clear reload\n", ring->name);
@@ -1315,7 +1319,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
params->instp_mode != dev_priv->relative_constants_mode) {
ret = intel_ring_begin(req, 4);
if (ret)
- return ret;
+ goto error;
intel_ring_emit(ring, MI_NOOP);
intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
@@ -1329,7 +1333,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
ret = i915_reset_gen7_sol_offsets(params->dev, req);
if (ret)
- return ret;
+ goto error;
}
exec_len = params->args_batch_len;
@@ -1343,13 +1347,17 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
exec_start, exec_len,
params->dispatch_flags);
if (ret)
- return ret;
+ goto error;
trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
i915_gem_execbuffer_retire_commands(params);
- return 0;
+error:
+ if (ret)
+ intel_ring_reserved_space_cancel(req->ringbuf);
+
+ return ret;
}
/**
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 3986890..a3ffd04 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -483,6 +483,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
struct i915_scheduler *scheduler = dev_priv->scheduler;
int ret;
+ intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
+
scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
ret = dev_priv->gt.execbuf_final(&qe->params);
scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
@@ -539,6 +541,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
node->stamp = jiffies;
i915_gem_request_reference(node->params.request);
+ intel_ring_reserved_space_cancel(node->params.request->ringbuf);
+
WARN_ON(node->params.request->scheduler_qe);
node->params.request->scheduler_qe = node;
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index ff4565f..f4bab82 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -978,13 +978,17 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ ret = intel_logical_ring_reserve_space(req);
+ if (ret)
+ goto err;
+
/*
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
ret = logical_ring_invalidate_all_caches(req);
if (ret)
- return ret;
+ goto err;
if (ring == &dev_priv->ring[RCS] &&
params->instp_mode != dev_priv->relative_constants_mode) {
@@ -1006,13 +1010,18 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
ret = ring->emit_bb_start(req, exec_start, params->dispatch_flags);
if (ret)
- return ret;
+ goto err;
trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
i915_gem_execbuffer_retire_commands(params);
return 0;
+
+err:
+ intel_ring_reserved_space_cancel(params->request->ringbuf);
+
+ return ret;
}
void intel_execlists_retire_requests(struct intel_engine_cs *ring)
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 15/35] drm/i915: Added tracking/locking of batch buffer objects
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (13 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 14/35] drm/i915: Keep the reserved space mechanism happy John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-19 19:42 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 16/35] drm/i915: Hook scheduler node clean up into retire requests John.C.Harrison
` (21 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler needs to track interdependencies between batch buffers.
These are calculated by analysing the object lists of the buffers and
looking for commonality. The scheduler also needs to keep those
buffers locked long after the initial IOCTL call has returned to user
land.
v3: Updated to support read-read optimisation.
v5: Updated due to changes to earlier patches in series for splitting
bypass mode into a separate function and consoliding the clean up code.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 48 ++++++++++++++++++++++++++++--
drivers/gpu/drm/i915/i915_scheduler.c | 15 ++++++++++
2 files changed, 61 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 11bea8d..f45f4dc 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1428,7 +1428,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
struct i915_execbuffer_params *params = &qe.params;
const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
u32 dispatch_flags;
- int ret;
+ int ret, i;
bool need_relocs;
if (!i915_gem_check_execbuffer(args))
@@ -1543,6 +1543,14 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
goto pre_mutex_err;
}
+ qe.saved_objects = kzalloc(
+ sizeof(*qe.saved_objects) * args->buffer_count,
+ GFP_KERNEL);
+ if (!qe.saved_objects) {
+ ret = -ENOMEM;
+ goto err;
+ }
+
/* Look up object handles */
ret = eb_lookup_vmas(eb, exec, args, vm, file);
if (ret)
@@ -1663,7 +1671,30 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
params->args_DR1 = args->DR1;
params->args_DR4 = args->DR4;
params->batch_obj = batch_obj;
- params->ctx = ctx;
+
+ /*
+ * Save away the list of objects used by this batch buffer for the
+ * purpose of tracking inter-buffer dependencies.
+ */
+ for (i = 0; i < args->buffer_count; i++) {
+ struct drm_i915_gem_object *obj;
+
+ /*
+ * NB: 'drm_gem_object_lookup()' increments the object's
+ * reference count and so must be matched by a
+ * 'drm_gem_object_unreference' call.
+ */
+ obj = to_intel_bo(drm_gem_object_lookup(dev, file,
+ exec[i].handle));
+ qe.saved_objects[i].obj = obj;
+ qe.saved_objects[i].read_only = obj->base.pending_write_domain == 0;
+
+ }
+ qe.num_objs = i;
+
+ /* Lock and save the context object as well. */
+ i915_gem_context_reference(ctx);
+ params->ctx = ctx;
ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
if (ret)
@@ -1696,6 +1727,19 @@ err:
i915_gem_context_unreference(ctx);
eb_destroy(eb);
+ /* Need to release the objects: */
+ if (qe.saved_objects) {
+ for (i = 0; i < qe.num_objs; i++)
+ drm_gem_object_unreference(
+ &qe.saved_objects[i].obj->base);
+
+ kfree(qe.saved_objects);
+ }
+
+ /* Context too */
+ if (params->ctx)
+ i915_gem_context_unreference(params->ctx);
+
/*
* If the request was created but not successfully submitted then it
* must be freed again. If it was submitted then it is being tracked
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index a3ffd04..60a59d3 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -719,6 +719,8 @@ void i915_scheduler_wakeup(struct drm_device *dev)
*/
void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
{
+ int i;
+
if (!I915_SQS_IS_COMPLETE(node)) {
WARN(!node->params.request->cancelled,
"Cleaning active node: %d!\n", node->status);
@@ -736,6 +738,19 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
node->params.batch_obj = NULL;
}
+ /* Release the locked buffers: */
+ for (i = 0; i < node->num_objs; i++)
+ drm_gem_object_unreference(&node->saved_objects[i].obj->base);
+ kfree(node->saved_objects);
+ node->saved_objects = NULL;
+ node->num_objs = 0;
+
+ /* Context too: */
+ if (node->params.ctx) {
+ i915_gem_context_unreference(node->params.ctx);
+ node->params.ctx = NULL;
+ }
+
/* And anything else owned by the node: */
if (node->params.cliprects) {
kfree(node->params.cliprects);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 16/35] drm/i915: Hook scheduler node clean up into retire requests
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (14 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 15/35] drm/i915: Added tracking/locking of batch buffer objects John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-19 19:44 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls John.C.Harrison
` (20 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler keeps its own lock on various DRM objects in order to
guarantee safe access long after the original execbuff IOCTL has
completed. This is especially important when pre-emption is enabled as
the batch buffer might need to be submitted to the hardware multiple
times. This patch hooks the clean up of these locks into the request
retire function. The request can only be retired after it has
completed on the hardware and thus is no longer eligible for
re-submission. Thus there is no point holding on to the locks beyond
that time.
v3: Updated to not WARN when cleaning a node that is being cancelled.
The clean will happen later so skipping it at the point of
cancellation is fine.
v5: Squashed the i915_scheduler.c portions down into the 'start of
scheduler' patch. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 1ab7256..2dd9b55 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1489,6 +1489,9 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
fence_signal_locked(&request->fence);
}
+ if (request->scheduler_qe)
+ i915_scheduler_clean_node(request->scheduler_qe);
+
i915_gem_request_unreference(request);
}
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (15 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 16/35] drm/i915: Hook scheduler node clean up into retire requests John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-03-01 10:02 ` Joonas Lahtinen
2016-02-18 14:27 ` [PATCH v5 18/35] drm/i915: Added scheduler support to page fault handler John.C.Harrison
` (19 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler can cause batch buffers, and hence requests, to be
submitted to the ring out of order and asynchronously to their
submission to the driver. Thus at the point of waiting for the
completion of a given request, it is not even guaranteed that the
request has actually been sent to the hardware yet. Even it is has
been sent, it is possible that it could be pre-empted and thus
'unsent'.
This means that it is necessary to be able to submit requests to the
hardware during the wait call itself. Unfortunately, while some
callers of __wait_request() release the mutex lock first, others do
not (and apparently can not). Hence there is the ability to deadlock
as the wait stalls for submission but the asynchronous submission is
stalled for the mutex lock.
This change hooks the scheduler in to the __wait_request() code to
ensure correct behaviour. That is, flush the target batch buffer
through to the hardware and do not deadlock waiting for something that
cannot currently be submitted. Instead, the wait call must return
EAGAIN at least as far back as necessary to release the mutex lock and
allow the scheduler's asynchronous processing to get in and handle the
pre-emption operation and eventually (re-)submit the work.
v3: Removed the explicit scheduler flush from i915_wait_request().
This is no longer necessary and was causing unintended changes to the
scheduler priority level which broke a validation team test.
v4: Corrected the format of some comments to keep the style checker
happy.
v5: Added function description. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 3 ++-
drivers/gpu/drm/i915/i915_gem.c | 37 ++++++++++++++++++++++++++-------
drivers/gpu/drm/i915/i915_scheduler.c | 31 +++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 2 ++
drivers/gpu/drm/i915/intel_display.c | 5 +++--
drivers/gpu/drm/i915/intel_ringbuffer.c | 2 +-
6 files changed, 69 insertions(+), 11 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 4d544f1..5eeeced 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3071,7 +3071,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
unsigned reset_counter,
bool interruptible,
s64 *timeout,
- struct intel_rps_client *rps);
+ struct intel_rps_client *rps,
+ bool is_locked);
int __must_check i915_wait_request(struct drm_i915_gem_request *req);
int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);
int __must_check
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 2dd9b55..17b44b3 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1258,7 +1258,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
unsigned reset_counter,
bool interruptible,
s64 *timeout,
- struct intel_rps_client *rps)
+ struct intel_rps_client *rps,
+ bool is_locked)
{
struct intel_engine_cs *ring = i915_gem_request_get_ring(req);
struct drm_device *dev = ring->dev;
@@ -1268,8 +1269,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
DEFINE_WAIT(wait);
unsigned long timeout_expire;
s64 before = 0; /* Only to silence a compiler warning. */
- int ret;
+ int ret = 0;
+ bool busy;
+ might_sleep();
WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");
if (i915_gem_request_completed(req))
@@ -1324,6 +1327,26 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
break;
}
+ if (is_locked) {
+ /*
+ * If this request is being processed by the scheduler
+ * then it is unsafe to sleep with the mutex lock held
+ * as the scheduler may require the lock in order to
+ * progress the request.
+ */
+ if (i915_scheduler_is_request_tracked(req, NULL, &busy)) {
+ if (busy) {
+ ret = -EAGAIN;
+ break;
+ }
+ }
+
+ /*
+ * If the request is not tracked by the scheduler
+ * then the regular test can be done.
+ */
+ }
+
if (i915_gem_request_completed(req)) {
ret = 0;
break;
@@ -1542,7 +1565,7 @@ i915_wait_request(struct drm_i915_gem_request *req)
ret = __i915_wait_request(req,
atomic_read(&dev_priv->gpu_error.reset_counter),
- interruptible, NULL, NULL);
+ interruptible, NULL, NULL, true);
if (ret)
return ret;
@@ -1655,7 +1678,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
mutex_unlock(&dev->struct_mutex);
for (i = 0; ret == 0 && i < n; i++)
ret = __i915_wait_request(requests[i], reset_counter, true,
- NULL, rps);
+ NULL, rps, false);
mutex_lock(&dev->struct_mutex);
for (i = 0; i < n; i++) {
@@ -3511,7 +3534,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
if (ret == 0)
ret = __i915_wait_request(req[i], reset_counter, true,
args->timeout_ns > 0 ? &args->timeout_ns : NULL,
- to_rps_client(file));
+ to_rps_client(file), false);
i915_gem_request_unreference(req[i]);
}
return ret;
@@ -3544,7 +3567,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
atomic_read(&i915->gpu_error.reset_counter),
i915->mm.interruptible,
NULL,
- &i915->rps.semaphores);
+ &i915->rps.semaphores, true);
if (ret)
return ret;
@@ -4523,7 +4546,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
if (target == NULL)
return 0;
- ret = __i915_wait_request(target, reset_counter, true, NULL, NULL);
+ ret = __i915_wait_request(target, reset_counter, true, NULL, NULL, false);
if (ret == 0)
queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 60a59d3..edab63d 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -918,6 +918,37 @@ void i915_scheduler_work_handler(struct work_struct *work)
}
/**
+ * i915_scheduler_is_request_tracked - return info to say what the scheduler's
+ * connection to this request is (if any).
+ * @req: request to be queried
+ * @compeleted: if non-null, set to completion status
+ * @busy: if non-null set to busy status
+ *
+ * Looks up the given request in the scheduler's internal queue and reports
+ * on whether the request has completed or is still pending.
+ * Returns true if the request was found or false if it was not.
+ */
+bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+ bool *completed, bool *busy)
+{
+ struct drm_i915_private *dev_priv = req->ring->dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (!scheduler)
+ return false;
+
+ if (req->scheduler_qe == NULL)
+ return false;
+
+ if (completed)
+ *completed = I915_SQS_IS_COMPLETE(req->scheduler_qe);
+ if (busy)
+ *busy = I915_SQS_IS_QUEUED(req->scheduler_qe);
+
+ return true;
+}
+
+/**
* i915_scheduler_closefile - notify the scheduler that a DRM file handle
* has been closed.
* @dev: DRM device
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 180d75f..a88adce 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -94,5 +94,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
void i915_scheduler_wakeup(struct drm_device *dev);
void i915_scheduler_work_handler(struct work_struct *work);
+bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+ bool *completed, bool *busy);
#endif /* _I915_SCHEDULER_H_ */
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 731d20a..5953590 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11458,7 +11458,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work)
WARN_ON(__i915_wait_request(mmio_flip->req,
mmio_flip->crtc->reset_counter,
false, NULL,
- &mmio_flip->i915->rps.mmioflips));
+ &mmio_flip->i915->rps.mmioflips,
+ false));
i915_gem_request_unreference(mmio_flip->req);
}
@@ -13523,7 +13524,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev,
ret = __i915_wait_request(intel_plane_state->wait_req,
reset_counter, true,
- NULL, NULL);
+ NULL, NULL, false);
/* Swallow -EIO errors to allow updates during hw lockup. */
if (ret == -EIO)
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index ca7b8af..a2093f5 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2304,7 +2304,7 @@ int intel_ring_idle(struct intel_engine_cs *ring)
return __i915_wait_request(req,
atomic_read(&to_i915(ring->dev)->gpu_error.reset_counter),
to_i915(ring->dev)->mm.interruptible,
- NULL, NULL);
+ NULL, NULL, true);
}
int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 18/35] drm/i915: Added scheduler support to page fault handler
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (16 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-19 19:45 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions John.C.Harrison
` (18 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
GPU page faults can now require scheduler operation in order to
complete. For example, in order to free up sufficient memory to handle
the fault the handler must wait for a batch buffer to complete that
has not even been sent to the hardware yet. Thus EAGAIN no longer
means a GPU hang, it can occur under normal operation.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 17b44b3..a47a495 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2003,10 +2003,15 @@ out:
}
case -EAGAIN:
/*
- * EAGAIN means the gpu is hung and we'll wait for the error
- * handler to reset everything when re-faulting in
+ * EAGAIN can mean the gpu is hung and we'll have to wait for
+ * the error handler to reset everything when re-faulting in
* i915_mutex_lock_interruptible.
+ *
+ * It can also indicate various other nonfatal errors for which
+ * the best response is to give other threads a chance to run,
+ * and then retry the failing operation in its entirety.
*/
+ /*FALLTHRU*/
case 0:
case -ERESTARTSYS:
case -EINTR:
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (17 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 18/35] drm/i915: Added scheduler support to page fault handler John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-03-07 11:31 ` Joonas Lahtinen
2016-02-18 14:27 ` [PATCH v5 20/35] drm/i915: Add scheduler hook to GPU reset John.C.Harrison
` (17 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
When requesting that all GPU work is completed, it is now necessary to
get the scheduler involved in order to flush out work that queued and
not yet submitted.
v2: Updated to add support for flushing the scheduler queue by time
stamp rather than just doing a blanket flush.
v3: Moved submit_max_priority() to this patch from an earlier patch
is it is no longer required in the other.
v4: Corrected the format of a comment to keep the style checker happy.
Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
v5: Shuffled functions around to remove forward prototypes, removed
similarly offensive white space and added documentation. Re-worked the
mutex locking around the submit function. [Joonas Lahtinen]
Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 24 ++++-
drivers/gpu/drm/i915/i915_scheduler.c | 178 ++++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 3 +
3 files changed, 204 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index a47a495..d946f53 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3786,6 +3786,10 @@ int i915_gpu_idle(struct drm_device *dev)
/* Flush everything onto the inactive list. */
for_each_ring(ring, dev_priv, i) {
+ ret = i915_scheduler_flush(ring, true);
+ if (ret < 0)
+ return ret;
+
if (!i915.enable_execlists) {
struct drm_i915_gem_request *req;
@@ -4519,7 +4523,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES;
struct drm_i915_gem_request *request, *target = NULL;
unsigned reset_counter;
- int ret;
+ int i, ret;
+ struct intel_engine_cs *ring;
ret = i915_gem_wait_for_error(&dev_priv->gpu_error);
if (ret)
@@ -4529,6 +4534,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
if (ret)
return ret;
+ for_each_ring(ring, dev_priv, i) {
+ /*
+ * Flush out scheduler entries that are getting 'stale'. Note
+ * that the following recent_enough test will only check
+ * against the time at which the request was submitted to the
+ * hardware (i.e. when it left the scheduler) not the time it
+ * was submitted to the driver.
+ *
+ * Also, there is not much point worring about busy return
+ * codes from the scheduler flush call. Even if more work
+ * cannot be submitted right now for whatever reason, we
+ * still want to throttle against stale work that has already
+ * been submitted.
+ */
+ i915_scheduler_flush_stamp(ring, recent_enough, false);
+ }
+
spin_lock(&file_priv->mm.lock);
list_for_each_entry(request, &file_priv->mm.request_list, client_list) {
if (time_after_eq(request->emitted_jiffies, recent_enough))
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index edab63d..8130a9c 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -304,6 +304,10 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
* attempting to acquire a mutex while holding a spin lock is a Bad Idea.
* And releasing the one before acquiring the other leads to other code
* being run and interfering.
+ *
+ * Hence any caller that does not already have the mutex lock for other
+ * reasons should call i915_scheduler_submit_unlocked() instead in order to
+ * obtain the lock first.
*/
static int i915_scheduler_submit(struct intel_engine_cs *ring)
{
@@ -428,6 +432,22 @@ error:
return ret;
}
+static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)
+{
+ struct drm_device *dev = ring->dev;
+ int ret;
+
+ ret = i915_mutex_lock_interruptible(dev);
+ if (ret)
+ return ret;
+
+ ret = i915_scheduler_submit(ring);
+
+ mutex_unlock(&dev->struct_mutex);
+
+ return ret;
+}
+
static void i915_generate_dependencies(struct i915_scheduler *scheduler,
struct i915_scheduler_queue_entry *node,
uint32_t ring)
@@ -917,6 +937,164 @@ void i915_scheduler_work_handler(struct work_struct *work)
i915_scheduler_process_work(ring);
}
+static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
+ bool is_locked)
+{
+ struct i915_scheduler_queue_entry *node;
+ struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ int ret, count = 0;
+ bool found;
+
+ do {
+ found = false;
+ spin_lock_irq(&scheduler->lock);
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (!I915_SQS_IS_QUEUED(node))
+ continue;
+
+ if (node->priority < scheduler->priority_level_max)
+ continue;
+
+ found = true;
+ break;
+ }
+ spin_unlock_irq(&scheduler->lock);
+
+ if (!found)
+ break;
+
+ if (is_locked)
+ ret = i915_scheduler_submit(ring);
+ else
+ ret = i915_scheduler_submit_unlocked(ring);
+ if (ret < 0)
+ return ret;
+
+ count += ret;
+ } while (found);
+
+ return count;
+}
+
+/**
+ * i915_scheduler_flush_stamp - force requests of a given age through the
+ * scheduler.
+ * @ring: Ring to be flushed
+ * @target: Jiffy based time stamp to flush up to
+ * @is_locked: Is the driver mutex lock held?
+ * DRM has a throttle by age of request facility. This requires waiting for
+ * outstanding work over a given age. This function helps that by forcing
+ * queued batch buffers over said age through the system.
+ * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
+ * for a pre-emption event to complete) but the mutex lock is held which
+ * would prevent the scheduler's asynchronous processing from completing.
+ */
+int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
+ unsigned long target,
+ bool is_locked)
+{
+ struct i915_scheduler_queue_entry *node;
+ struct drm_i915_private *dev_priv;
+ struct i915_scheduler *scheduler;
+ int flush_count = 0;
+
+ if (!ring)
+ return -EINVAL;
+
+ dev_priv = ring->dev->dev_private;
+ scheduler = dev_priv->scheduler;
+
+ if (!scheduler)
+ return 0;
+
+ if (is_locked && (scheduler->flags[ring->id] & i915_sf_submitting)) {
+ /*
+ * Scheduler is busy already submitting another batch,
+ * come back later rather than going recursive...
+ */
+ return -EAGAIN;
+ }
+
+ spin_lock_irq(&scheduler->lock);
+ i915_scheduler_priority_bump_clear(scheduler);
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (!I915_SQS_IS_QUEUED(node))
+ continue;
+
+ if (node->stamp > target)
+ continue;
+
+ flush_count = i915_scheduler_priority_bump(scheduler,
+ node, scheduler->priority_level_max);
+ }
+ spin_unlock_irq(&scheduler->lock);
+
+ if (flush_count) {
+ DRM_DEBUG_DRIVER("<%s> Bumped %d entries\n", ring->name, flush_count);
+ flush_count = i915_scheduler_submit_max_priority(ring, is_locked);
+ }
+
+ return flush_count;
+}
+
+/**
+ * i915_scheduler_flush - force all requests through the scheduler.
+ * @ring: Ring to be flushed
+ * @is_locked: Is the driver mutex lock held?
+ * For various reasons it is sometimes necessary to the scheduler out, e.g.
+ * due to ring reset.
+ * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
+ * for a pre-emption event to complete) but the mutex lock is held which
+ * would prevent the scheduler's asynchronous processing from completing.
+ */
+int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked)
+{
+ struct i915_scheduler_queue_entry *node;
+ struct drm_i915_private *dev_priv;
+ struct i915_scheduler *scheduler;
+ bool found;
+ int ret;
+ uint32_t count = 0;
+
+ if (!ring)
+ return -EINVAL;
+
+ dev_priv = ring->dev->dev_private;
+ scheduler = dev_priv->scheduler;
+
+ if (!scheduler)
+ return 0;
+
+ WARN_ON(is_locked && (scheduler->flags[ring->id] & i915_sf_submitting));
+
+ do {
+ found = false;
+ spin_lock_irq(&scheduler->lock);
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (!I915_SQS_IS_QUEUED(node))
+ continue;
+
+ found = true;
+ break;
+ }
+ spin_unlock_irq(&scheduler->lock);
+
+ if (found) {
+ if (is_locked)
+ ret = i915_scheduler_submit(ring);
+ else
+ ret = i915_scheduler_submit_unlocked(ring);
+ if (ret < 0)
+ return ret;
+
+ count += ret;
+ }
+ } while (found);
+
+ return count;
+}
+
/**
* i915_scheduler_is_request_tracked - return info to say what the scheduler's
* connection to this request is (if any).
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index a88adce..839b048 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -94,6 +94,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
void i915_scheduler_wakeup(struct drm_device *dev);
void i915_scheduler_work_handler(struct work_struct *work);
+int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
+int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
+ unsigned long stamp, bool is_locked);
bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
bool *completed, bool *busy);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 20/35] drm/i915: Add scheduler hook to GPU reset
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (18 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-23 20:27 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 21/35] drm/i915: Added a module parameter to allow the scheduler to be disabled John.C.Harrison
` (16 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
When the watchdog resets the GPU, all interrupts get disabled despite
the reference count remaining. As the scheduler probably had
interrupts enabled during the reset (it would have been waiting for
the bad batch to complete), it must be poked to tell it that the
interrupt has been disabled.
v5: New patch in series.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 2 ++
drivers/gpu/drm/i915/i915_scheduler.c | 11 +++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 1 +
3 files changed, 14 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index d946f53..d7f7f7a 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3248,6 +3248,8 @@ static void i915_gem_reset_ring_cleanup(struct drm_i915_private *dev_priv,
buffer->last_retired_head = buffer->tail;
intel_ring_update_space(buffer);
}
+
+ i915_scheduler_reset_cleanup(ring);
}
void i915_gem_reset(struct drm_device *dev)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 8130a9c..4f25bf2 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -778,6 +778,17 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
}
}
+void i915_scheduler_reset_cleanup(struct intel_engine_cs *ring)
+{
+ struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (scheduler->flags[ring->id] & i915_sf_interrupts_enabled) {
+ ring->irq_put(ring);
+ scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;
+ }
+}
+
static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
struct intel_engine_cs *ring,
struct list_head *remove)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 839b048..075befb 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -89,6 +89,7 @@ bool i915_scheduler_is_enabled(struct drm_device *dev);
int i915_scheduler_init(struct drm_device *dev);
int i915_scheduler_closefile(struct drm_device *dev,
struct drm_file *file);
+void i915_scheduler_reset_cleanup(struct intel_engine_cs *ring);
void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 21/35] drm/i915: Added a module parameter to allow the scheduler to be disabled
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (19 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 20/35] drm/i915: Add scheduler hook to GPU reset John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-23 20:29 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 22/35] drm/i915: Support for 'unflushed' ring idle John.C.Harrison
` (15 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
It can be useful to be able to disable the GPU scheduler via a module
parameter for debugging purposes.
v5: Converted from a multi-feature 'overrides' mask to a single
'enable' boolean. Further features (e.g. pre-emption) will now be
separate 'enable' booleans added later. [Chris Wilson]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_params.c | 4 ++++
drivers/gpu/drm/i915/i915_params.h | 1 +
drivers/gpu/drm/i915/i915_scheduler.c | 5 ++++-
3 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c
index d0eba58..0ef3159 100644
--- a/drivers/gpu/drm/i915/i915_params.c
+++ b/drivers/gpu/drm/i915/i915_params.c
@@ -57,6 +57,7 @@ struct i915_params i915 __read_mostly = {
.edp_vswing = 0,
.enable_guc_submission = true,
.guc_log_level = -1,
+ .enable_scheduler = 0,
};
module_param_named(modeset, i915.modeset, int, 0400);
@@ -203,3 +204,6 @@ MODULE_PARM_DESC(enable_guc_submission, "Enable GuC submission (default:false)")
module_param_named(guc_log_level, i915.guc_log_level, int, 0400);
MODULE_PARM_DESC(guc_log_level,
"GuC firmware logging level (-1:disabled (default), 0-3:enabled)");
+
+module_param_named_unsafe(enable_scheduler, i915.enable_scheduler, int, 0600);
+MODULE_PARM_DESC(enable_scheduler, "Enable scheduler (0 = disable [default], 1 = enable)");
diff --git a/drivers/gpu/drm/i915/i915_params.h b/drivers/gpu/drm/i915/i915_params.h
index 5299290..f855c86 100644
--- a/drivers/gpu/drm/i915/i915_params.h
+++ b/drivers/gpu/drm/i915/i915_params.h
@@ -60,6 +60,7 @@ struct i915_params {
bool enable_guc_submission;
bool verbose_state_checks;
bool nuclear_pageflip;
+ int enable_scheduler;
};
extern struct i915_params i915 __read_mostly;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 4f25bf2..47d7de4 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -34,6 +34,9 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
+ if (!i915.enable_scheduler)
+ return false;
+
return dev_priv->scheduler != NULL;
}
@@ -548,7 +551,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
WARN_ON(!scheduler);
- if (1/*!i915.enable_scheduler*/)
+ if (!i915.enable_scheduler)
return i915_scheduler_queue_execbuffer_bypass(qe);
node = kmalloc(sizeof(*node), GFP_KERNEL);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 22/35] drm/i915: Support for 'unflushed' ring idle
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (20 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 21/35] drm/i915: Added a module parameter to allow the scheduler to be disabled John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-23 20:35 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 23/35] drm/i915: Defer seqno allocation until actual hardware submission time John.C.Harrison
` (14 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
When the seqno wraps around zero, the entire GPU is forced to be idle
for some reason (possibly only to work around issues with hardware
semaphores but no-one seems too sure!). This causes a problem if the
force idle occurs at an inopportune moment such as in the middle of
submitting a batch buffer. Specifically, it would lead to recursive
submits - submitting work requires a new seqno, the new seqno requires
idling the ring, idling the ring requires submitting work, submitting
work requires a new seqno...
This change adds a 'flush' parameter to the idle function call which
specifies whether the scheduler queues should be flushed out. I.e. is
the call intended to just idle the ring as it is right now (no flush)
or is it intended to force all outstanding work out of the system
(with flush).
In the seqno wrap case, pending work is not an issue because the next
operation will be to submit it. However, in other cases, the intention
is to make sure everything that could be done has been done.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 4 ++--
drivers/gpu/drm/i915/intel_lrc.c | 2 +-
drivers/gpu/drm/i915/intel_ringbuffer.c | 17 +++++++++++++++--
drivers/gpu/drm/i915/intel_ringbuffer.h | 2 +-
4 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index d7f7f7a..a249e52 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2564,7 +2564,7 @@ i915_gem_init_seqno(struct drm_device *dev, u32 seqno)
/* Carefully retire all requests without writing to the rings */
for_each_ring(ring, dev_priv, i) {
- ret = intel_ring_idle(ring);
+ ret = intel_ring_idle(ring, false);
if (ret)
return ret;
}
@@ -3808,7 +3808,7 @@ int i915_gpu_idle(struct drm_device *dev)
i915_add_request_no_flush(req);
}
- ret = intel_ring_idle(ring);
+ ret = intel_ring_idle(ring, true);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index f4bab82..e056875 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1058,7 +1058,7 @@ void intel_logical_ring_stop(struct intel_engine_cs *ring)
if (!intel_ring_initialized(ring))
return;
- ret = intel_ring_idle(ring);
+ ret = intel_ring_idle(ring, true);
if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
ring->name, ret);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index a2093f5..70ef9f0 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2288,9 +2288,22 @@ static void __wrap_ring_buffer(struct intel_ringbuffer *ringbuf)
intel_ring_update_space(ringbuf);
}
-int intel_ring_idle(struct intel_engine_cs *ring)
+int intel_ring_idle(struct intel_engine_cs *ring, bool flush)
{
struct drm_i915_gem_request *req;
+ int ret;
+
+ /*
+ * NB: Must not flush the scheduler if this idle request is from
+ * within an execbuff submission (i.e. due to 'get_seqno' calling
+ * 'wrap_seqno' calling 'idle'). As that would lead to recursive
+ * flushes!
+ */
+ if (flush) {
+ ret = i915_scheduler_flush(ring, true);
+ if (ret)
+ return ret;
+ }
/* Wait upon the last request to be completed */
if (list_empty(&ring->request_list))
@@ -3095,7 +3108,7 @@ intel_stop_ring_buffer(struct intel_engine_cs *ring)
if (!intel_ring_initialized(ring))
return;
- ret = intel_ring_idle(ring);
+ ret = intel_ring_idle(ring, true);
if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
ring->name, ret);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index ada93a9..cca476f 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -478,7 +478,7 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf);
int intel_ring_space(struct intel_ringbuffer *ringbuf);
bool intel_ring_stopped(struct intel_engine_cs *ring);
-int __must_check intel_ring_idle(struct intel_engine_cs *ring);
+int __must_check intel_ring_idle(struct intel_engine_cs *ring, bool flush);
void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno);
int intel_ring_flush_all_caches(struct drm_i915_gem_request *req);
int intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 23/35] drm/i915: Defer seqno allocation until actual hardware submission time
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (21 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 22/35] drm/i915: Support for 'unflushed' ring idle John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-03-07 12:16 ` Joonas Lahtinen
2016-02-18 14:27 ` [PATCH v5 24/35] drm/i915: Added trace points to scheduler John.C.Harrison
` (13 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The seqno value is now only used for the final test for completion of
a request. It is no longer used to track the request through the
software stack. Thus it is no longer necessary to allocate the seqno
immediately with the request. Instead, it can be done lazily and left
until the request is actually sent to the hardware. This is particular
advantageous with a GPU scheduler as the requests can then be
re-ordered between their creation and their hardware submission
without having out of order seqnos.
v2: i915_add_request() can't fail!
Combine with 'drm/i915: Assign seqno at start of exec_final()'
Various bits of code during the execbuf code path need a seqno value
to be assigned to the request. This change makes this assignment
explicit at the start of submission_final() rather than relying on an
auto-generated seqno to have happened already. This is in preparation
for a future patch which changes seqno values to be assigned lazily
(during add_request).
v3: Updated to use locally cached request pointer.
v4: Changed some white space and comment formatting to keep the style
checker happy.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 1 +
drivers/gpu/drm/i915/i915_gem.c | 23 ++++++++++++++++++++++-
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 14 ++++++++++++++
drivers/gpu/drm/i915/intel_lrc.c | 14 ++++++++++++++
4 files changed, 51 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 5eeeced..071a27b 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2257,6 +2257,7 @@ struct drm_i915_gem_request {
* has finished processing this request.
*/
u32 seqno;
+ u32 reserved_seqno;
/* Unique identifier which can be used for trace points & debug */
uint32_t uniq;
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index a249e52..a2c136d 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2616,6 +2616,11 @@ i915_gem_get_seqno(struct drm_device *dev, u32 *seqno)
/* reserve 0 for non-seqno */
if (dev_priv->next_seqno == 0) {
+ /*
+ * Why is the full re-initialisation required? Is it only for
+ * hardware semaphores? If so, could skip it in the case where
+ * semaphores are disabled?
+ */
int ret = i915_gem_init_seqno(dev, 0);
if (ret)
return ret;
@@ -2673,6 +2678,12 @@ void __i915_add_request(struct drm_i915_gem_request *request,
WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
}
+ /* Make the request's seqno 'live': */
+ if (!request->seqno) {
+ request->seqno = request->reserved_seqno;
+ WARN_ON(request->seqno != dev_priv->last_seqno);
+ }
+
/* Record the position of the start of the request so that
* should we detect the updated seqno part-way through the
* GPU processing the request, we never over-estimate the
@@ -2930,6 +2941,9 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
list_for_each_entry_safe(req, req_next, &ring->fence_signal_list, signal_link) {
if (!req->cancelled) {
+ /* How can this happen? */
+ WARN_ON(req->seqno == 0);
+
if (!i915_seqno_passed(seqno, req->seqno))
break;
}
@@ -3079,7 +3093,14 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,
if (req == NULL)
return -ENOMEM;
- ret = i915_gem_get_seqno(ring->dev, &req->seqno);
+ /*
+ * Assign an identifier to track this request through the hardware
+ * but don't make it live yet. It could change in the future if this
+ * request gets overtaken. However, it still needs to be allocated
+ * in advance because the point of submission must not fail and seqno
+ * allocation can fail.
+ */
+ ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
if (ret)
goto err;
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index f45f4dc..b9ad0fd 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1295,6 +1295,20 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ /* Make sure the request's seqno is the latest and greatest: */
+ if (req->reserved_seqno != dev_priv->last_seqno) {
+ ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
+ if (ret)
+ return ret;
+ }
+ /*
+ * And make it live because some of the execbuff submission code
+ * requires the seqno to be available up front.
+ */
+ WARN_ON(req->seqno);
+ req->seqno = req->reserved_seqno;
+ WARN_ON(req->seqno != dev_priv->last_seqno);
+
ret = intel_ring_reserve_space(req);
if (ret)
goto error;
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index e056875..9c7a79a 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -978,6 +978,20 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ /* Make sure the request's seqno is the latest and greatest: */
+ if (req->reserved_seqno != dev_priv->last_seqno) {
+ ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
+ if (ret)
+ return ret;
+ }
+ /*
+ * And make it live because some of the execbuff submission code
+ * requires the seqno to be available up front.
+ */
+ WARN_ON(req->seqno);
+ req->seqno = req->reserved_seqno;
+ WARN_ON(req->seqno != dev_priv->last_seqno);
+
ret = intel_logical_ring_reserve_space(req);
if (ret)
goto err;
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 24/35] drm/i915: Added trace points to scheduler
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (22 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 23/35] drm/i915: Defer seqno allocation until actual hardware submission time John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-23 20:42 ` Jesse Barnes
2016-02-23 20:42 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle John.C.Harrison
` (12 subsequent siblings)
36 siblings, 2 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Added trace points to the scheduler to track all the various events,
node state transitions and other interesting things that occur.
v2: Updated for new request completion tracking implementation.
v3: Updated for changes to node kill code.
v4: Wrapped some long lines to keep the style checker happy.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 +
drivers/gpu/drm/i915/i915_scheduler.c | 26 ++++
drivers/gpu/drm/i915/i915_trace.h | 196 +++++++++++++++++++++++++++++
drivers/gpu/drm/i915/intel_lrc.c | 2 +
4 files changed, 226 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index b9ad0fd..d4de8c7 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
+ trace_i915_gem_ring_queue(ring, params);
+
qe = container_of(params, typeof(*qe), params);
ret = i915_scheduler_queue_execbuffer(qe);
if (ret)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 47d7de4..e56ce08 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
/* Seqno will be reassigned on relaunch */
node->params.request->seqno = 0;
node->status = i915_sqs_queued;
+ trace_i915_scheduler_unfly(node->params.ring, node);
+ trace_i915_scheduler_node_state_change(node->params.ring, node);
}
/*
@@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
WARN_ON(!node);
WARN_ON(I915_SQS_IS_COMPLETE(node));
+ if (I915_SQS_IS_FLYING(node))
+ trace_i915_scheduler_unfly(node->params.ring, node);
+
node->status = i915_sqs_dead;
+ trace_i915_scheduler_node_state_change(node->params.ring, node);
}
/* Mark a node as in flight on the hardware. */
@@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
node->status = i915_sqs_flying;
+ trace_i915_scheduler_fly(ring, node);
+ trace_i915_scheduler_node_state_change(ring, node);
+
if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
bool success = true;
@@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
INIT_LIST_HEAD(&best->link);
best->status = i915_sqs_popped;
+ trace_i915_scheduler_node_state_change(ring, best);
+
ret = 0;
} else {
/* Can only get here if:
@@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
}
}
+ trace_i915_scheduler_pop_from_queue(ring, best);
+
*pop_node = best;
return ret;
}
@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
struct i915_scheduler *scheduler = dev_priv->scheduler;
int ret;
+ trace_i915_scheduler_queue(qe->params.ring, qe);
+
intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
not_flying = i915_scheduler_count_flying(scheduler, ring) <
scheduler->min_flying;
+ trace_i915_scheduler_queue(ring, node);
+ trace_i915_scheduler_node_state_change(ring, node);
+
spin_unlock_irq(&scheduler->lock);
if (not_flying)
@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
struct i915_scheduler_queue_entry *node = req->scheduler_qe;
unsigned long flags;
+ trace_i915_scheduler_landing(req);
+
if (!node)
return false;
@@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
else
node->status = i915_sqs_complete;
+ trace_i915_scheduler_node_state_change(req->ring, node);
+
spin_unlock_irqrestore(&scheduler->lock, flags);
return true;
@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
/* Launch more packets now? */
do_submit = (queued > 0) && (flying < scheduler->min_flying);
+ trace_i915_scheduler_remove(ring, min_seqno, do_submit);
+
spin_unlock_irq(&scheduler->lock);
return do_submit;
@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring)
node = list_first_entry(&remove, typeof(*node), link);
list_del(&node->link);
+ trace_i915_scheduler_destroy(ring, node);
+
/* Free up all the DRM references */
i915_scheduler_clean_node(node);
diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
index 455c215..c3c4e58 100644
--- a/drivers/gpu/drm/i915/i915_trace.h
+++ b/drivers/gpu/drm/i915/i915_trace.h
@@ -9,6 +9,7 @@
#include "i915_drv.h"
#include "intel_drv.h"
#include "intel_ringbuffer.h"
+#include "i915_scheduler.h"
#undef TRACE_SYSTEM
#define TRACE_SYSTEM i915
@@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm,
__entry->dev, __entry->ring, __entry->to, __entry->vm)
);
+TRACE_EVENT(i915_scheduler_queue,
+ TP_PROTO(struct intel_engine_cs *ring,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(ring, node),
+
+ TP_STRUCT__entry(
+ __field(u32, ring)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->ring = ring->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node ? node->params.request->seqno : 0;
+ ),
+
+ TP_printk("ring=%d, uniq=%d, seqno=%d",
+ __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_fly,
+ TP_PROTO(struct intel_engine_cs *ring,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(ring, node),
+
+ TP_STRUCT__entry(
+ __field(u32, ring)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->ring = ring->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node ? node->params.request->seqno : 0;
+ ),
+
+ TP_printk("ring=%d, uniq=%d, seqno=%d",
+ __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_unfly,
+ TP_PROTO(struct intel_engine_cs *ring,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(ring, node),
+
+ TP_STRUCT__entry(
+ __field(u32, ring)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->ring = ring->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node ? node->params.request->seqno : 0;
+ ),
+
+ TP_printk("ring=%d, uniq=%d, seqno=%d",
+ __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_landing,
+ TP_PROTO(struct drm_i915_gem_request *req),
+ TP_ARGS(req),
+
+ TP_STRUCT__entry(
+ __field(u32, ring)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ __field(u32, status)
+ ),
+
+ TP_fast_assign(
+ __entry->ring = req->ring->id;
+ __entry->uniq = req->uniq;
+ __entry->seqno = req->seqno;
+ __entry->status = req->scheduler_qe ?
+ req->scheduler_qe->status : ~0U;
+ ),
+
+ TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
+ __entry->ring, __entry->uniq, __entry->seqno,
+ __entry->status)
+);
+
+TRACE_EVENT(i915_scheduler_remove,
+ TP_PROTO(struct intel_engine_cs *ring,
+ u32 min_seqno, bool do_submit),
+ TP_ARGS(ring, min_seqno, do_submit),
+
+ TP_STRUCT__entry(
+ __field(u32, ring)
+ __field(u32, min_seqno)
+ __field(bool, do_submit)
+ ),
+
+ TP_fast_assign(
+ __entry->ring = ring->id;
+ __entry->min_seqno = min_seqno;
+ __entry->do_submit = do_submit;
+ ),
+
+ TP_printk("ring=%d, min_seqno = %d, do_submit=%d",
+ __entry->ring, __entry->min_seqno, __entry->do_submit)
+);
+
+TRACE_EVENT(i915_scheduler_destroy,
+ TP_PROTO(struct intel_engine_cs *ring,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(ring, node),
+
+ TP_STRUCT__entry(
+ __field(u32, ring)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->ring = ring->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node ? node->params.request->seqno : 0;
+ ),
+
+ TP_printk("ring=%d, uniq=%d, seqno=%d",
+ __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_pop_from_queue,
+ TP_PROTO(struct intel_engine_cs *ring,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(ring, node),
+
+ TP_STRUCT__entry(
+ __field(u32, ring)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->ring = ring->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node ? node->params.request->seqno : 0;
+ ),
+
+ TP_printk("ring=%d, uniq=%d, seqno=%d",
+ __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_node_state_change,
+ TP_PROTO(struct intel_engine_cs *ring,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(ring, node),
+
+ TP_STRUCT__entry(
+ __field(u32, ring)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ __field(u32, status)
+ ),
+
+ TP_fast_assign(
+ __entry->ring = ring->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node->params.request->seqno;
+ __entry->status = node->status;
+ ),
+
+ TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
+ __entry->ring, __entry->uniq, __entry->seqno,
+ __entry->status)
+);
+
+TRACE_EVENT(i915_gem_ring_queue,
+ TP_PROTO(struct intel_engine_cs *ring,
+ struct i915_execbuffer_params *params),
+ TP_ARGS(ring, params),
+
+ TP_STRUCT__entry(
+ __field(u32, ring)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->ring = ring->id;
+ __entry->uniq = params->request->uniq;
+ __entry->seqno = params->request->seqno;
+ ),
+
+ TP_printk("ring=%d, uniq=%d, seqno=%d", __entry->ring,
+ __entry->uniq, __entry->seqno)
+);
+
#endif /* _I915_TRACE_H_ */
/* This part must be outside protection */
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 9c7a79a..2b9f49c 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
+ trace_i915_gem_ring_queue(ring, params);
+
qe = container_of(params, typeof(*qe), params);
ret = i915_scheduler_queue_execbuffer(qe);
if (ret)
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (23 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 24/35] drm/i915: Added trace points to scheduler John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-23 21:02 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters John.C.Harrison
` (11 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler decouples the submission of batch buffers to the driver
from their subsequent submission to the hardware. This means that an
application which is continuously submitting buffers as fast as it can
could potentialy flood the driver. To prevent this, the driver now
tracks how many buffers are in progress (queued in software or
executing in hardware) and limits this to a given (tunable) number. If
this number is exceeded then the queue to the driver will return
EAGAIN and thus prevent the scheduler's queue becoming arbitrarily
large.
v3: Added a missing decrement of the file queue counter.
v4: Updated a comment.
v5: Updated due to changes to earlier patches in series - removing
forward declarations and white space. Also added some documentation.
[Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 2 ++
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 8 +++++
drivers/gpu/drm/i915/i915_scheduler.c | 48 ++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 2 ++
4 files changed, 60 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 071a27b..3f4c4f0 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -336,6 +336,8 @@ struct drm_i915_file_private {
} rps;
struct intel_engine_cs *bsd_ring;
+
+ u32 scheduler_queue_length;
};
enum intel_dpll_id {
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index d4de8c7..dff120c 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1803,6 +1803,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
return -EINVAL;
}
+ /* Throttle batch requests per device file */
+ if (i915_scheduler_file_queue_is_full(file))
+ return -EAGAIN;
+
/* Copy in the exec list from userland */
exec_list = drm_malloc_ab(sizeof(*exec_list), args->buffer_count);
exec2_list = drm_malloc_ab(sizeof(*exec2_list), args->buffer_count);
@@ -1893,6 +1897,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
return -EINVAL;
}
+ /* Throttle batch requests per device file */
+ if (i915_scheduler_file_queue_is_full(file))
+ return -EAGAIN;
+
exec2_list = kmalloc(sizeof(*exec2_list)*args->buffer_count,
GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
if (exec2_list == NULL)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index e56ce08..f7f29d5 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -69,6 +69,7 @@ int i915_scheduler_init(struct drm_device *dev)
scheduler->priority_level_bump = 50;
scheduler->priority_level_preempt = 900;
scheduler->min_flying = 2;
+ scheduler->file_queue_max = 64;
dev_priv->scheduler = scheduler;
@@ -464,6 +465,44 @@ static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)
return ret;
}
+/**
+ * i915_scheduler_file_queue_is_full - Returns true if the queue is full.
+ * @file: File object to query.
+ * This allows throttling of applications by limiting the total number of
+ * outstanding requests to a specified level. Once that limit is reached,
+ * this call will return true and no more requests should be accepted.
+ */
+bool i915_scheduler_file_queue_is_full(struct drm_file *file)
+{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+ struct drm_i915_private *dev_priv = file_priv->dev_priv;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ return file_priv->scheduler_queue_length >= scheduler->file_queue_max;
+}
+
+/**
+ * i915_scheduler_file_queue_inc - Increment the file's request queue count.
+ * @file: File object to process.
+ */
+static void i915_scheduler_file_queue_inc(struct drm_file *file)
+{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+
+ file_priv->scheduler_queue_length++;
+}
+
+/**
+ * i915_scheduler_file_queue_dec - Decrement the file's request queue count.
+ * @file: File object to process.
+ */
+static void i915_scheduler_file_queue_dec(struct drm_file *file)
+{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+
+ file_priv->scheduler_queue_length--;
+}
+
static void i915_generate_dependencies(struct i915_scheduler *scheduler,
struct i915_scheduler_queue_entry *node,
uint32_t ring)
@@ -640,6 +679,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
+ i915_scheduler_file_queue_inc(node->params.file);
+
not_flying = i915_scheduler_count_flying(scheduler, ring) <
scheduler->min_flying;
@@ -883,6 +924,12 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
/* Strip the dependency info while the mutex is still locked */
i915_scheduler_remove_dependent(scheduler, node);
+ /* Likewise clean up the file pointer. */
+ if (node->params.file) {
+ i915_scheduler_file_queue_dec(node->params.file);
+ node->params.file = NULL;
+ }
+
continue;
}
@@ -1205,6 +1252,7 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
node->status,
ring->name);
+ i915_scheduler_file_queue_dec(node->params.file);
node->params.file = NULL;
}
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 075befb..b78de12 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -77,6 +77,7 @@ struct i915_scheduler {
int32_t priority_level_bump;
int32_t priority_level_preempt;
uint32_t min_flying;
+ uint32_t file_queue_max;
};
/* Flag bits for i915_scheduler::flags */
@@ -100,5 +101,6 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
unsigned long stamp, bool is_locked);
bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
bool *completed, bool *busy);
+bool i915_scheduler_file_queue_is_full(struct drm_file *file);
#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (24 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-23 21:06 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 27/35] drm/i915: Added debug state dump facilities to scheduler John.C.Harrison
` (10 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
There are various parameters within the scheduler which can be tuned
to improve performance, reduce memory footprint, etc. This change adds
support for altering these via debugfs.
v2: Updated for priorities now being signed values.
v5: Squashed priority bumping entries into this patch rather than a
separate patch all of their own.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_debugfs.c | 169 ++++++++++++++++++++++++++++++++++++
1 file changed, 169 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index b923949..7d01c07 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -39,6 +39,7 @@
#include "intel_ringbuffer.h"
#include <drm/i915_drm.h>
#include "i915_drv.h"
+#include "i915_scheduler.h"
enum {
ACTIVE_LIST,
@@ -1122,6 +1123,168 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops,
i915_next_seqno_get, i915_next_seqno_set,
"0x%llx\n");
+static int
+i915_scheduler_priority_min_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->priority_level_min;
+ return 0;
+}
+
+static int
+i915_scheduler_priority_min_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ scheduler->priority_level_min = (int32_t) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_min_fops,
+ i915_scheduler_priority_min_get,
+ i915_scheduler_priority_min_set,
+ "%lld\n");
+
+static int
+i915_scheduler_priority_max_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->priority_level_max;
+ return 0;
+}
+
+static int
+i915_scheduler_priority_max_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ scheduler->priority_level_max = (int32_t) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_max_fops,
+ i915_scheduler_priority_max_get,
+ i915_scheduler_priority_max_set,
+ "%lld\n");
+
+static int
+i915_scheduler_priority_bump_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->priority_level_bump;
+ return 0;
+}
+
+static int
+i915_scheduler_priority_bump_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ scheduler->priority_level_bump = (u32) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_bump_fops,
+ i915_scheduler_priority_bump_get,
+ i915_scheduler_priority_bump_set,
+ "%lld\n");
+
+static int
+i915_scheduler_priority_preempt_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->priority_level_preempt;
+ return 0;
+}
+
+static int
+i915_scheduler_priority_preempt_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ scheduler->priority_level_preempt = (u32) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_preempt_fops,
+ i915_scheduler_priority_preempt_get,
+ i915_scheduler_priority_preempt_set,
+ "%lld\n");
+
+static int
+i915_scheduler_min_flying_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->min_flying;
+ return 0;
+}
+
+static int
+i915_scheduler_min_flying_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ scheduler->min_flying = (u32) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_min_flying_fops,
+ i915_scheduler_min_flying_get,
+ i915_scheduler_min_flying_set,
+ "%llu\n");
+
+static int
+i915_scheduler_file_queue_max_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->file_queue_max;
+ return 0;
+}
+
+static int
+i915_scheduler_file_queue_max_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ scheduler->file_queue_max = (u32) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
+ i915_scheduler_file_queue_max_get,
+ i915_scheduler_file_queue_max_set,
+ "%llu\n");
+
static int i915_frequency_info(struct seq_file *m, void *unused)
{
struct drm_info_node *node = m->private;
@@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files {
{"i915_gem_drop_caches", &i915_drop_caches_fops},
{"i915_error_state", &i915_error_state_fops},
{"i915_next_seqno", &i915_next_seqno_fops},
+ {"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
+ {"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
+ {"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops},
+ {"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
+ {"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
+ {"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 27/35] drm/i915: Added debug state dump facilities to scheduler
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (25 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-03-07 12:31 ` Joonas Lahtinen
2016-02-18 14:27 ` [PATCH v5 28/35] drm/i915: Add early exit to execbuff_final() if insufficient ring space John.C.Harrison
` (9 subsequent siblings)
36 siblings, 1 reply; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
When debugging batch buffer submission issues, it is useful to be able
to see what the current state of the scheduler is. This change adds
functions for decoding the internal scheduler state and reporting it.
v3: Updated a debug message with the new state_str() function.
v4: Wrapped some long lines to keep the style checker happy. Removed
the fence/sync code as that will now be part of a separate patch series.
v5: Removed forward declarations and white space. Added documentation.
[Joonas Lahtinen]
Also squashed in later patch to add seqno information from the start.
It was only being added in a separate patch due to historical reasons
which have since gone away.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_scheduler.c | 302 +++++++++++++++++++++++++++++++++-
drivers/gpu/drm/i915/i915_scheduler.h | 15 ++
2 files changed, 315 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index f7f29d5..d0eed52 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -40,6 +40,117 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
return dev_priv->scheduler != NULL;
}
+const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node)
+{
+ static char str[50];
+ char *ptr = str;
+
+ *(ptr++) = node->bumped ? 'B' : '-',
+ *(ptr++) = i915_gem_request_completed(node->params.request) ? 'C' : '-';
+
+ *ptr = 0;
+
+ return str;
+}
+
+char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status)
+{
+ switch (status) {
+ case i915_sqs_none:
+ return 'N';
+
+ case i915_sqs_queued:
+ return 'Q';
+
+ case i915_sqs_popped:
+ return 'X';
+
+ case i915_sqs_flying:
+ return 'F';
+
+ case i915_sqs_complete:
+ return 'C';
+
+ case i915_sqs_dead:
+ return 'D';
+
+ default:
+ break;
+ }
+
+ return '?';
+}
+
+const char *i915_scheduler_queue_status_str(
+ enum i915_scheduler_queue_status status)
+{
+ static char str[50];
+
+ switch (status) {
+ case i915_sqs_none:
+ return "None";
+
+ case i915_sqs_queued:
+ return "Queued";
+
+ case i915_sqs_popped:
+ return "Popped";
+
+ case i915_sqs_flying:
+ return "Flying";
+
+ case i915_sqs_complete:
+ return "Complete";
+
+ case i915_sqs_dead:
+ return "Dead";
+
+ default:
+ break;
+ }
+
+ sprintf(str, "[Unknown_%d!]", status);
+ return str;
+}
+
+const char *i915_scheduler_flag_str(uint32_t flags)
+{
+ static char str[100];
+ char *ptr = str;
+
+ *ptr = 0;
+
+#define TEST_FLAG(flag, msg) \
+ do { \
+ if (flags & (flag)) { \
+ strcpy(ptr, msg); \
+ ptr += strlen(ptr); \
+ flags &= ~(flag); \
+ } \
+ } while (0)
+
+ TEST_FLAG(i915_sf_interrupts_enabled, "IntOn|");
+ TEST_FLAG(i915_sf_submitting, "Submitting|");
+ TEST_FLAG(i915_sf_dump_force, "DumpForce|");
+ TEST_FLAG(i915_sf_dump_details, "DumpDetails|");
+ TEST_FLAG(i915_sf_dump_dependencies, "DumpDeps|");
+ TEST_FLAG(i915_sf_dump_seqno, "DumpSeqno|");
+
+#undef TEST_FLAG
+
+ if (flags) {
+ sprintf(ptr, "Unknown_0x%X!", flags);
+ ptr += strlen(ptr);
+ }
+
+ if (ptr == str)
+ strcpy(str, "-");
+ else
+ ptr[-1] = 0;
+
+ return str;
+};
+
/**
* i915_scheduler_init - Initialise the scheduler.
* @dev: DRM device
@@ -1024,6 +1135,193 @@ void i915_scheduler_work_handler(struct work_struct *work)
i915_scheduler_process_work(ring);
}
+static int i915_scheduler_dump_locked(struct intel_engine_cs *ring,
+ const char *msg)
+{
+ struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+ int flying = 0, queued = 0, complete = 0, other = 0;
+ static int old_flying = -1, old_queued = -1, old_complete = -1;
+ bool b_dump;
+ char brkt[2] = { '<', '>' };
+
+ if (!ring)
+ return -EINVAL;
+
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (I915_SQS_IS_QUEUED(node))
+ queued++;
+ else if (I915_SQS_IS_FLYING(node))
+ flying++;
+ else if (I915_SQS_IS_COMPLETE(node))
+ complete++;
+ else
+ other++;
+ }
+
+ b_dump = (flying != old_flying) ||
+ (queued != old_queued) ||
+ (complete != old_complete);
+ if (scheduler->flags[ring->id] & i915_sf_dump_force) {
+ if (!b_dump) {
+ b_dump = true;
+ brkt[0] = '{';
+ brkt[1] = '}';
+ }
+
+ scheduler->flags[ring->id] &= ~i915_sf_dump_force;
+ }
+
+ if (b_dump) {
+ old_flying = flying;
+ old_queued = queued;
+ old_complete = complete;
+ DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d, "
+ "Flags = %s, Next = %d:%d %c%s%c\n",
+ ring->name, queued, flying, complete, other,
+ i915_scheduler_flag_str(scheduler->flags[ring->id]),
+ dev_priv->request_uniq, dev_priv->next_seqno,
+ brkt[0], msg, brkt[1]);
+ } else {
+ /*DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d"
+ ", Flags = %s, Next = %d:%d [%s]\n",
+ ring->name,
+ queued, flying, complete, other,
+ i915_scheduler_flag_str(scheduler->flags[ring->id]),
+ dev_priv->request_uniq, dev_priv->next_seqno, msg); */
+
+ return 0;
+ }
+
+ if (scheduler->flags[ring->id] & i915_sf_dump_seqno) {
+ uint32_t seqno;
+
+ seqno = ring->get_seqno(ring, true);
+
+ DRM_DEBUG_DRIVER("<%s> Seqno = %d\n", ring->name, seqno);
+ }
+
+ if (scheduler->flags[ring->id] & i915_sf_dump_details) {
+ int i, deps;
+ uint32_t count, counts[i915_sqs_MAX];
+
+ memset(counts, 0x00, sizeof(counts));
+
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (node->status < i915_sqs_MAX) {
+ count = counts[node->status]++;
+ } else {
+ DRM_DEBUG_DRIVER("<%s> Unknown status: %d!\n",
+ ring->name, node->status);
+ count = -1;
+ }
+
+ deps = 0;
+ for (i = 0; i < node->num_deps; i++)
+ if (i915_scheduler_is_dependency_valid(node, i))
+ deps++;
+
+ DRM_DEBUG_DRIVER("<%s> %c:%02d> uniq = %d, seqno"
+ " = %d/%s, deps = %d / %d, %s [pri = "
+ "%4d]\n", ring->name,
+ i915_scheduler_queue_status_chr(node->status),
+ count,
+ node->params.request->uniq,
+ node->params.request->seqno,
+ node->params.ring->name,
+ deps, node->num_deps,
+ i915_qe_state_str(node),
+ node->priority);
+
+ if ((scheduler->flags[ring->id] & i915_sf_dump_dependencies)
+ == 0)
+ continue;
+
+ for (i = 0; i < node->num_deps; i++)
+ if (node->dep_list[i])
+ DRM_DEBUG_DRIVER("<%s> |-%c:"
+ "%02d%c uniq = %d, seqno = %d/%s, %s [pri = %4d]\n",
+ ring->name,
+ i915_scheduler_queue_status_chr(node->dep_list[i]->status),
+ i,
+ i915_scheduler_is_dependency_valid(node, i)
+ ? '>' : '#',
+ node->dep_list[i]->params.request->uniq,
+ node->dep_list[i]->params.request->seqno,
+ node->dep_list[i]->params.ring->name,
+ i915_qe_state_str(node->dep_list[i]),
+ node->dep_list[i]->priority);
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * i915_scheduler_dump - dump the scheduler's internal state to the debug log.
+ * @ring: Ring to dump info for
+ * @msg: A reason why it is being dumped
+ * For debugging purposes, it can be very useful to see the internal state of
+ * the scheduler for a given ring.
+ */
+int i915_scheduler_dump(struct intel_engine_cs *ring, const char *msg)
+{
+ struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&scheduler->lock, flags);
+ ret = i915_scheduler_dump_locked(ring, msg);
+ spin_unlock_irqrestore(&scheduler->lock, flags);
+
+ return ret;
+}
+
+static int i915_scheduler_dump_all_locked(struct drm_device *dev,
+ const char *msg)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct intel_engine_cs *ring;
+ int i, r, ret = 0;
+
+ for_each_ring(ring, dev_priv, i) {
+ scheduler->flags[ring->id] |= i915_sf_dump_force |
+ i915_sf_dump_details |
+ i915_sf_dump_seqno |
+ i915_sf_dump_dependencies;
+ r = i915_scheduler_dump_locked(ring, msg);
+ if (ret == 0)
+ ret = r;
+ }
+
+ return ret;
+}
+
+/**
+ * i915_scheduler_dump_all - dump the scheduler's internal state to the debug
+ * log.
+ * @dev: DRM device
+ * @msg: A reason why it is being dumped
+ * For debugging purposes, it can be very useful to see the internal state of
+ * the scheduler.
+ */
+int i915_scheduler_dump_all(struct drm_device *dev, const char *msg)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&scheduler->lock, flags);
+ ret = i915_scheduler_dump_all_locked(dev, msg);
+ spin_unlock_irqrestore(&scheduler->lock, flags);
+
+ return ret;
+}
+
static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
bool is_locked)
{
@@ -1246,10 +1544,10 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
continue;
if (!I915_SQS_IS_COMPLETE(node))
- DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
+ DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%s on %s\n",
node->params.request->uniq,
node->params.request->seqno,
- node->status,
+ i915_qe_state_str(node),
ring->name);
i915_scheduler_file_queue_dec(node->params.file);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index b78de12..a071ebb 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -41,6 +41,9 @@ enum i915_scheduler_queue_status {
/* Limit value for use with arrays/loops */
i915_sqs_MAX
};
+char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status);
+const char *i915_scheduler_queue_status_str(
+ enum i915_scheduler_queue_status status);
#define I915_SQS_IS_QUEUED(node) (((node)->status == i915_sqs_queued))
#define I915_SQS_IS_FLYING(node) (((node)->status == i915_sqs_flying))
@@ -65,6 +68,7 @@ struct i915_scheduler_queue_entry {
unsigned long stamp;
struct list_head link;
};
+const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
struct i915_scheduler {
struct list_head node_queue[I915_NUM_RINGS];
@@ -82,9 +86,17 @@ struct i915_scheduler {
/* Flag bits for i915_scheduler::flags */
enum {
+ /* Internal state */
i915_sf_interrupts_enabled = (1 << 0),
i915_sf_submitting = (1 << 1),
+
+ /* Dump/debug flags */
+ i915_sf_dump_force = (1 << 8),
+ i915_sf_dump_details = (1 << 9),
+ i915_sf_dump_dependencies = (1 << 10),
+ i915_sf_dump_seqno = (1 << 11),
};
+const char *i915_scheduler_flag_str(uint32_t flags);
bool i915_scheduler_is_enabled(struct drm_device *dev);
int i915_scheduler_init(struct drm_device *dev);
@@ -99,6 +111,9 @@ void i915_scheduler_work_handler(struct work_struct *work);
int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
unsigned long stamp, bool is_locked);
+int i915_scheduler_dump(struct intel_engine_cs *ring,
+ const char *msg);
+int i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
bool *completed, bool *busy);
bool i915_scheduler_file_queue_is_full(struct drm_file *file);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 28/35] drm/i915: Add early exit to execbuff_final() if insufficient ring space
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (26 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 27/35] drm/i915: Added debug state dump facilities to scheduler John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 29/35] drm/i915: Added scheduler statistic reporting to debugfs John.C.Harrison
` (8 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
One of the major purposes of the GPU scheduler is to avoid stalling
the CPU when the GPU is busy and unable to accept more work. This
change adds support to the ring submission code to allow a ring space
check to be performed before attempting to submit a batch buffer to
the hardware. If insufficient space is available then the scheduler
can go away and come back later, letting the CPU get on with other
work, rather than stalling and waiting for the hardware to catch up.
v3: Updated to use locally cached request pointer.
v4: Line wrapped some comments differently to keep the style checker
happy. Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
Removed some obsolete, commented out code.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 41 +++++++++++++++++------
drivers/gpu/drm/i915/intel_lrc.c | 54 +++++++++++++++++++++++++++---
drivers/gpu/drm/i915/intel_ringbuffer.c | 26 ++++++++++++++
drivers/gpu/drm/i915/intel_ringbuffer.h | 1 +
4 files changed, 107 insertions(+), 15 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index dff120c..83ce94d 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1147,25 +1147,19 @@ i915_reset_gen7_sol_offsets(struct drm_device *dev,
{
struct intel_engine_cs *ring = req->ring;
struct drm_i915_private *dev_priv = dev->dev_private;
- int ret, i;
+ int i;
if (!IS_GEN7(dev) || ring != &dev_priv->ring[RCS]) {
DRM_DEBUG("sol reset is gen7/rcs only\n");
return -EINVAL;
}
- ret = intel_ring_begin(req, 4 * 3);
- if (ret)
- return ret;
-
for (i = 0; i < 4; i++) {
intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
intel_ring_emit_reg(ring, GEN7_SO_WRITE_OFFSET(i));
intel_ring_emit(ring, 0);
}
- intel_ring_advance(ring);
-
return 0;
}
@@ -1293,6 +1287,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
struct intel_engine_cs *ring = params->ring;
u64 exec_start, exec_len;
int ret;
+ uint32_t min_space;
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
@@ -1316,6 +1311,34 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
goto error;
/*
+ * It would be a bad idea to run out of space while writing commands
+ * to the ring. One of the major aims of the scheduler is to not
+ * stall at any point for any reason. However, doing an early exit
+ * half way through submission could result in a partial sequence
+ * being written which would leave the engine in an unknown state.
+ * Therefore, check in advance that there will be enough space for
+ * the entire submission whether emitted by the code below OR by any
+ * other functions that may be executed before the end of final().
+ *
+ * NB: This test deliberately overestimates, because that's easier
+ * than tracing every potential path that could be taken!
+ *
+ * Current measurements suggest that we may need to emit up to 186
+ * dwords, so this is rounded up to 256 here. Then double that to get
+ * the free space requirement, because the block is not allowed to
+ * span the transition from the end to the beginning of the ring.
+ */
+#define I915_BATCH_EXEC_MAX_LEN 256 /* max dwords emitted here */
+ min_space = I915_BATCH_EXEC_MAX_LEN * 2 * sizeof(uint32_t);
+ ret = intel_ring_test_space(req->ringbuf, min_space);
+ if (ret)
+ goto error;
+
+ ret = intel_ring_begin(req, I915_BATCH_EXEC_MAX_LEN);
+ if (ret)
+ goto error;
+
+ /*
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
@@ -1333,10 +1356,6 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
if (ring == &dev_priv->ring[RCS] &&
params->instp_mode != dev_priv->relative_constants_mode) {
- ret = intel_ring_begin(req, 4);
- if (ret)
- goto error;
-
intel_ring_emit(ring, MI_NOOP);
intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
intel_ring_emit_reg(ring, INSTPM);
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 2b9f49c..e124443 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -231,6 +231,27 @@ static void lrc_setup_hardware_status_page(struct intel_engine_cs *ring,
struct drm_i915_gem_object *default_ctx_obj);
+/*
+ * Test to see if the ring has sufficient space to submit a given piece
+ * of work without causing a stall
+ */
+static int logical_ring_test_space(struct intel_ringbuffer *ringbuf,
+ int min_space)
+{
+ if (ringbuf->space < min_space) {
+ /* Need to update the actual ring space. Otherwise, the system
+ * hangs forever testing a software copy of the space value that
+ * never changes!
+ */
+ intel_ring_update_space(ringbuf);
+
+ if (ringbuf->space < min_space)
+ return -EAGAIN;
+ }
+
+ return 0;
+}
+
/**
* intel_sanitize_enable_execlists() - sanitize i915.enable_execlists
* @dev: DRM device.
@@ -976,6 +997,7 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
struct intel_engine_cs *ring = params->ring;
u64 exec_start;
int ret;
+ uint32_t min_space;
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
@@ -999,6 +1021,34 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
goto err;
/*
+ * It would be a bad idea to run out of space while writing commands
+ * to the ring. One of the major aims of the scheduler is to not
+ * stall at any point for any reason. However, doing an early exit
+ * half way through submission could result in a partial sequence
+ * being written which would leave the engine in an unknown state.
+ * Therefore, check in advance that there will be enough space for
+ * the entire submission whether emitted by the code below OR by any
+ * other functions that may be executed before the end of final().
+ *
+ * NB: This test deliberately overestimates, because that's easier
+ * than tracing every potential path that could be taken!
+ *
+ * Current measurements suggest that we may need to emit up to 186
+ * dwords, so this is rounded up to 256 here. Then double that to get
+ * the free space requirement, because the block is not allowed to
+ * span the transition from the end to the beginning of the ring.
+ */
+#define I915_BATCH_EXEC_MAX_LEN 256 /* max dwords emitted here */
+ min_space = I915_BATCH_EXEC_MAX_LEN * 2 * sizeof(uint32_t);
+ ret = logical_ring_test_space(ringbuf, min_space);
+ if (ret)
+ goto err;
+
+ ret = intel_logical_ring_begin(req, I915_BATCH_EXEC_MAX_LEN);
+ if (ret)
+ goto err;
+
+ /*
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
@@ -1008,10 +1058,6 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
if (ring == &dev_priv->ring[RCS] &&
params->instp_mode != dev_priv->relative_constants_mode) {
- ret = intel_logical_ring_begin(req, 4);
- if (ret)
- return ret;
-
intel_logical_ring_emit(ringbuf, MI_NOOP);
intel_logical_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(1));
intel_logical_ring_emit_reg(ringbuf, INSTPM);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 70ef9f0..5b9df9d 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2482,6 +2482,32 @@ int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
return 0;
}
+/*
+ * Test to see if the ring has sufficient space to submit a given piece
+ * of work without causing a stall
+ */
+int intel_ring_test_space(struct intel_ringbuffer *ringbuf, int min_space)
+{
+ struct drm_i915_private *dev_priv = ringbuf->ring->dev->dev_private;
+
+ /* There is a separate LRC version of this code. */
+ WARN_ON(i915.enable_execlists);
+
+ if (ringbuf->space < min_space) {
+ /* Need to update the actual ring space. Otherwise, the system
+ * hangs forever testing a software copy of the space value that
+ * never changes!
+ */
+ ringbuf->head = I915_READ_HEAD(ringbuf->ring);
+ ringbuf->space = intel_ring_space(ringbuf);
+
+ if (ringbuf->space < min_space)
+ return -EAGAIN;
+ }
+
+ return 0;
+}
+
void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno)
{
struct drm_device *dev = ring->dev;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index cca476f..de66845 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -454,6 +454,7 @@ void intel_cleanup_ring_buffer(struct intel_engine_cs *ring);
int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request);
+int intel_ring_test_space(struct intel_ringbuffer *ringbuf, int min_space);
int __must_check intel_ring_begin(struct drm_i915_gem_request *req, int n);
int __must_check intel_ring_cacheline_align(struct drm_i915_gem_request *req);
static inline void intel_ring_emit(struct intel_engine_cs *ring,
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 29/35] drm/i915: Added scheduler statistic reporting to debugfs
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (27 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 28/35] drm/i915: Add early exit to execbuff_final() if insufficient ring space John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 30/35] drm/i915: Add scheduler support functions for TDR John.C.Harrison
` (7 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
It is useful for know what the scheduler is doing for both debugging
and performance analysis purposes. This change adds a bunch of
counters and such that keep track of various scheduler operations
(batches submitted, completed, flush requests, etc.). The data can
then be read in userland via the debugfs mechanism.
v2: Updated to match changes to scheduler implementation.
v3: Updated for changes to kill code and flush code.
v4: Removed the fence/sync code as that will be part of a separate
patch series. Wrapped a long line to keep the style checker happy.
v5: Updated to remove forward declarations and white space. Added
documentation. [Joonas Lahtinen]
Used lighter weight spinlocks.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_debugfs.c | 73 ++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 3 ++
drivers/gpu/drm/i915/i915_scheduler.c | 78 ++++++++++++++++++++++++++++--
drivers/gpu/drm/i915/i915_scheduler.h | 31 ++++++++++++
4 files changed, 180 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 7d01c07..2c8b00f 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -3595,6 +3595,78 @@ static int i915_drrs_status(struct seq_file *m, void *unused)
return 0;
}
+static int i915_scheduler_info(struct seq_file *m, void *unused)
+{
+ struct drm_info_node *node = (struct drm_info_node *) m->private;
+ struct drm_device *dev = node->minor->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_stats *stats = scheduler->stats;
+ struct i915_scheduler_stats_nodes node_stats[I915_NUM_RINGS];
+ struct intel_engine_cs *ring;
+ char str[50 * (I915_NUM_RINGS + 1)], name[50], *ptr;
+ int ret, i, r;
+
+ ret = mutex_lock_interruptible(&dev->mode_config.mutex);
+ if (ret)
+ return ret;
+
+#define PRINT_VAR(name, fmt, var) \
+ do { \
+ sprintf(str, "%-22s", name); \
+ ptr = str + strlen(str); \
+ for_each_ring(ring, dev_priv, r) { \
+ sprintf(ptr, " %10" fmt, var); \
+ ptr += strlen(ptr); \
+ } \
+ seq_printf(m, "%s\n", str); \
+ } while (0)
+
+ PRINT_VAR("Ring name:", "s", dev_priv->ring[r].name);
+ PRINT_VAR(" Ring seqno", "d", ring->get_seqno(ring, false));
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Batch submissions:\n");
+ PRINT_VAR(" Queued", "u", stats[r].queued);
+ PRINT_VAR(" Submitted", "u", stats[r].submitted);
+ PRINT_VAR(" Completed", "u", stats[r].completed);
+ PRINT_VAR(" Expired", "u", stats[r].expired);
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Flush counts:\n");
+ PRINT_VAR(" By object", "u", stats[r].flush_obj);
+ PRINT_VAR(" By request", "u", stats[r].flush_req);
+ PRINT_VAR(" By stamp", "u", stats[r].flush_stamp);
+ PRINT_VAR(" Blanket", "u", stats[r].flush_all);
+ PRINT_VAR(" Entries bumped", "u", stats[r].flush_bump);
+ PRINT_VAR(" Entries submitted", "u", stats[r].flush_submit);
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Miscellaneous:\n");
+ PRINT_VAR(" ExecEarly retry", "u", stats[r].exec_early);
+ PRINT_VAR(" ExecFinal requeue", "u", stats[r].exec_again);
+ PRINT_VAR(" ExecFinal killed", "u", stats[r].exec_dead);
+ PRINT_VAR(" Hung flying", "u", stats[r].kill_flying);
+ PRINT_VAR(" Hung queued", "u", stats[r].kill_queued);
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Queue contents:\n");
+ for_each_ring(ring, dev_priv, i)
+ i915_scheduler_query_stats(ring, node_stats + ring->id);
+
+ for (i = 0; i < (i915_sqs_MAX + 1); i++) {
+ sprintf(name, " %s", i915_scheduler_queue_status_str(i));
+ PRINT_VAR(name, "d", node_stats[r].counts[i]);
+ }
+ seq_putc(m, '\n');
+
+#undef PRINT_VAR
+
+ mutex_unlock(&dev->mode_config.mutex);
+
+ return 0;
+}
+
struct pipe_crc_info {
const char *name;
struct drm_device *dev;
@@ -5565,6 +5637,7 @@ static const struct drm_info_list i915_debugfs_list[] = {
{"i915_semaphore_status", i915_semaphore_status, 0},
{"i915_shared_dplls_info", i915_shared_dplls_info, 0},
{"i915_dp_mst_info", i915_dp_mst_info, 0},
+ {"i915_scheduler_info", i915_scheduler_info, 0},
{"i915_wa_registers", i915_wa_registers, 0},
{"i915_ddb_info", i915_ddb_info, 0},
{"i915_sseu_status", i915_sseu_status, 0},
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 83ce94d..a42a13e 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1789,6 +1789,9 @@ pre_mutex_err:
/* intel_gpu_busy should also get a ref, so it will free when the device
* is really idle. */
intel_runtime_pm_put(dev_priv);
+
+ dev_priv->scheduler->stats[ring->id].exec_early++;
+
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index d0eed52..0068d03 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -105,6 +105,9 @@ const char *i915_scheduler_queue_status_str(
case i915_sqs_dead:
return "Dead";
+ case i915_sqs_MAX:
+ return "Invalid";
+
default:
break;
}
@@ -208,13 +211,17 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
* Give up on a node completely. For example, because it is causing the
* ring to hang or is using some resource that no longer exists.
*/
-static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
+static void i915_scheduler_node_kill(struct i915_scheduler *scheduler,
+ struct i915_scheduler_queue_entry *node)
{
WARN_ON(!node);
WARN_ON(I915_SQS_IS_COMPLETE(node));
- if (I915_SQS_IS_FLYING(node))
+ if (I915_SQS_IS_FLYING(node)) {
trace_i915_scheduler_unfly(node->params.ring, node);
+ scheduler->stats[node->params.ring->id].kill_flying++;
+ } else
+ scheduler->stats[node->params.ring->id].kill_queued++;
node->status = i915_sqs_dead;
trace_i915_scheduler_node_state_change(node->params.ring, node);
@@ -469,6 +476,8 @@ static int i915_scheduler_submit(struct intel_engine_cs *ring)
*/
i915_scheduler_node_fly(node);
+ scheduler->stats[ring->id].submitted++;
+
spin_unlock_irq(&scheduler->lock);
ret = dev_priv->gt.execbuf_final(&node->params);
spin_lock_irq(&scheduler->lock);
@@ -492,7 +501,8 @@ static int i915_scheduler_submit(struct intel_engine_cs *ring)
case ENOENT:
/* Fatal errors. Kill the node. */
requeue = false;
- i915_scheduler_node_kill(node);
+ scheduler->stats[ring->id].exec_dead++;
+ i915_scheduler_node_kill(scheduler, node);
break;
case EAGAIN:
@@ -502,6 +512,7 @@ static int i915_scheduler_submit(struct intel_engine_cs *ring)
case ERESTARTSYS:
case EINTR:
/* Supposedly recoverable errors. */
+ scheduler->stats[ring->id].exec_again++;
break;
default:
@@ -510,6 +521,7 @@ static int i915_scheduler_submit(struct intel_engine_cs *ring)
* for the best.
*/
MISSING_CASE(-ret);
+ scheduler->stats[ring->id].exec_again++;
break;
}
@@ -669,12 +681,15 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
struct i915_scheduler *scheduler = dev_priv->scheduler;
int ret;
+ scheduler->stats[qe->params.ring->id].queued++;
+
trace_i915_scheduler_queue(qe->params.ring, qe);
intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
ret = dev_priv->gt.execbuf_final(&qe->params);
+ scheduler->stats[qe->params.ring->id].submitted++;
scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
/*
@@ -687,6 +702,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
/* Need to release any resources held by the node: */
i915_scheduler_clean_node(qe);
+ scheduler->stats[qe->params.ring->id].expired++;
+
return 0;
}
@@ -795,6 +812,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
not_flying = i915_scheduler_count_flying(scheduler, ring) <
scheduler->min_flying;
+ scheduler->stats[ring->id].queued++;
+
trace_i915_scheduler_queue(ring, node);
trace_i915_scheduler_node_state_change(ring, node);
@@ -837,10 +856,13 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
WARN_ON(!I915_SQS_IS_FLYING(node));
/* Node was in flight so mark it as complete. */
- if (req->cancelled)
+ if (req->cancelled) {
node->status = i915_sqs_dead;
- else
+ scheduler->stats[req->ring->id].kill_flying++;
+ } else {
node->status = i915_sqs_complete;
+ scheduler->stats[req->ring->id].completed++;
+ }
trace_i915_scheduler_node_state_change(req->ring, node);
@@ -1031,6 +1053,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
list_del(&node->link);
list_add(&node->link, remove);
+ scheduler->stats[ring->id].expired++;
/* Strip the dependency info while the mutex is still locked */
i915_scheduler_remove_dependent(scheduler, node);
@@ -1322,6 +1345,44 @@ int i915_scheduler_dump_all(struct drm_device *dev, const char *msg)
return ret;
}
+/**
+ * i915_scheduler_query_stats - return various scheduler statistics
+ * @ring: Ring to report on
+ * @stats: Stats structure to be filled in
+ * For various reasons (debugging, performance analysis, curiosity) it is
+ * useful to see statistics about what the scheduler is doing. This function
+ * returns the stats that have been gathered in a data structure. The
+ * expectation is that this will be returned to the user via debugfs.
+ */
+int i915_scheduler_query_stats(struct intel_engine_cs *ring,
+ struct i915_scheduler_stats_nodes *stats)
+{
+ struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+
+ memset(stats, 0x00, sizeof(*stats));
+
+ spin_lock_irq(&scheduler->lock);
+
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (node->status >= i915_sqs_MAX) {
+ DRM_DEBUG_DRIVER("Invalid node state: %d! [uniq = %d, seqno = %d]\n",
+ node->status, node->params.request->uniq,
+ node->params.request->seqno);
+
+ stats->counts[i915_sqs_MAX]++;
+ continue;
+ }
+
+ stats->counts[node->status]++;
+ }
+
+ spin_unlock_irq(&scheduler->lock);
+
+ return 0;
+}
+
static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
bool is_locked)
{
@@ -1402,6 +1463,7 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
}
spin_lock_irq(&scheduler->lock);
+ scheduler->stats[ring->id].flush_stamp++;
i915_scheduler_priority_bump_clear(scheduler);
list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
if (!I915_SQS_IS_QUEUED(node))
@@ -1412,12 +1474,15 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
flush_count = i915_scheduler_priority_bump(scheduler,
node, scheduler->priority_level_max);
+ scheduler->stats[ring->id].flush_bump += flush_count;
}
spin_unlock_irq(&scheduler->lock);
if (flush_count) {
DRM_DEBUG_DRIVER("<%s> Bumped %d entries\n", ring->name, flush_count);
flush_count = i915_scheduler_submit_max_priority(ring, is_locked);
+ if (flush_count > 0)
+ scheduler->stats[ring->id].flush_submit += flush_count;
}
return flush_count;
@@ -1453,6 +1518,8 @@ int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked)
WARN_ON(is_locked && (scheduler->flags[ring->id] & i915_sf_submitting));
+ scheduler->stats[ring->id].flush_all++;
+
do {
found = false;
spin_lock_irq(&scheduler->lock);
@@ -1470,6 +1537,7 @@ int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked)
ret = i915_scheduler_submit(ring);
else
ret = i915_scheduler_submit_unlocked(ring);
+ scheduler->stats[ring->id].flush_submit++;
if (ret < 0)
return ret;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index a071ebb..065f2a3 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -70,6 +70,32 @@ struct i915_scheduler_queue_entry {
};
const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
+struct i915_scheduler_stats_nodes {
+ uint32_t counts[i915_sqs_MAX + 1];
+};
+
+struct i915_scheduler_stats {
+ /* Batch buffer counts: */
+ uint32_t queued;
+ uint32_t submitted;
+ uint32_t completed;
+ uint32_t expired;
+
+ /* Other stuff: */
+ uint32_t flush_obj;
+ uint32_t flush_req;
+ uint32_t flush_stamp;
+ uint32_t flush_all;
+ uint32_t flush_bump;
+ uint32_t flush_submit;
+
+ uint32_t exec_early;
+ uint32_t exec_again;
+ uint32_t exec_dead;
+ uint32_t kill_flying;
+ uint32_t kill_queued;
+};
+
struct i915_scheduler {
struct list_head node_queue[I915_NUM_RINGS];
uint32_t flags[I915_NUM_RINGS];
@@ -82,6 +108,9 @@ struct i915_scheduler {
int32_t priority_level_preempt;
uint32_t min_flying;
uint32_t file_queue_max;
+
+ /* Statistics: */
+ struct i915_scheduler_stats stats[I915_NUM_RINGS];
};
/* Flag bits for i915_scheduler::flags */
@@ -116,6 +145,8 @@ int i915_scheduler_dump(struct intel_engine_cs *ring,
int i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
bool *completed, bool *busy);
+int i915_scheduler_query_stats(struct intel_engine_cs *ring,
+ struct i915_scheduler_stats_nodes *stats);
bool i915_scheduler_file_queue_is_full(struct drm_file *file);
#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 30/35] drm/i915: Add scheduler support functions for TDR
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (28 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 29/35] drm/i915: Added scheduler statistic reporting to debugfs John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 31/35] drm/i915: Scheduler state dump via debugfs John.C.Harrison
` (6 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The TDR code needs to know what the scheduler is up to in order to
work out whether a ring is really hung or not.
v4: Removed some unnecessary braces to keep the style checker happy.
v5: Removed white space and added documentation. [Joonas Lahtinen]
Also updated for new module parameter.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_scheduler.c | 33 +++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 1 +
2 files changed, 34 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 0068d03..c69e2b8 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -1627,3 +1627,36 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
return 0;
}
+
+/**
+ * i915_scheduler_is_ring_flying - does the given ring have in flight batches?
+ * @ring: Ring to query
+ * Used by TDR to distinguish hung rings (not moving but with work to do)
+ * from idle rings (not moving because there is nothing to do). Returns true
+ * if the given ring has batches currently executing on the hardware.
+ */
+bool i915_scheduler_is_ring_flying(struct intel_engine_cs *ring)
+{
+ struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+ unsigned long flags;
+ bool found = false;
+
+ /* With the scheduler in bypass mode, no information can be returned. */
+ if (!i915.enable_scheduler)
+ return true;
+
+ spin_lock_irqsave(&scheduler->lock, flags);
+
+ list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+ if (I915_SQS_IS_FLYING(node)) {
+ found = true;
+ break;
+ }
+ }
+
+ spin_unlock_irqrestore(&scheduler->lock, flags);
+
+ return found;
+}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 065f2a3..dcf1f05 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -136,6 +136,7 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
void i915_scheduler_wakeup(struct drm_device *dev);
+bool i915_scheduler_is_ring_flying(struct intel_engine_cs *ring);
void i915_scheduler_work_handler(struct work_struct *work);
int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 31/35] drm/i915: Scheduler state dump via debugfs
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (29 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 30/35] drm/i915: Add scheduler support functions for TDR John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 32/35] drm/i915: Enable GPU scheduler by default John.C.Harrison
` (5 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Added a facility for triggering the scheduler state dump via a debugfs
entry.
v2: New patch in series.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_debugfs.c | 33 +++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.c | 9 +++++----
drivers/gpu/drm/i915/i915_scheduler.h | 6 ++++++
3 files changed, 44 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 2c8b00f..e0dc06d77 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -1285,6 +1285,38 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
i915_scheduler_file_queue_max_set,
"%llu\n");
+static int
+i915_scheduler_dump_flags_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = scheduler->dump_flags;
+
+ return 0;
+}
+
+static int
+i915_scheduler_dump_flags_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ scheduler->dump_flags = lower_32_bits(val) & i915_sf_dump_mask;
+
+ if (val & 1)
+ i915_scheduler_dump_all(dev, "DebugFS");
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_dump_flags_fops,
+ i915_scheduler_dump_flags_get,
+ i915_scheduler_dump_flags_set,
+ "0x%llx\n");
+
static int i915_frequency_info(struct seq_file *m, void *unused)
{
struct drm_info_node *node = m->private;
@@ -5666,6 +5698,7 @@ static const struct i915_debugfs_files {
{"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
{"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
{"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
+ {"i915_scheduler_dump_flags", &i915_scheduler_dump_flags_fops},
{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index c69e2b8..b738e0b 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -184,6 +184,10 @@ int i915_scheduler_init(struct drm_device *dev)
scheduler->priority_level_preempt = 900;
scheduler->min_flying = 2;
scheduler->file_queue_max = 64;
+ scheduler->dump_flags = i915_sf_dump_force |
+ i915_sf_dump_details |
+ i915_sf_dump_seqno |
+ i915_sf_dump_dependencies;
dev_priv->scheduler = scheduler;
@@ -1311,10 +1315,7 @@ static int i915_scheduler_dump_all_locked(struct drm_device *dev,
int i, r, ret = 0;
for_each_ring(ring, dev_priv, i) {
- scheduler->flags[ring->id] |= i915_sf_dump_force |
- i915_sf_dump_details |
- i915_sf_dump_seqno |
- i915_sf_dump_dependencies;
+ scheduler->flags[ring->id] |= scheduler->dump_flags & i915_sf_dump_mask;
r = i915_scheduler_dump_locked(ring, msg);
if (ret == 0)
ret = r;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index dcf1f05..47c7951 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -108,6 +108,7 @@ struct i915_scheduler {
int32_t priority_level_preempt;
uint32_t min_flying;
uint32_t file_queue_max;
+ uint32_t dump_flags;
/* Statistics: */
struct i915_scheduler_stats stats[I915_NUM_RINGS];
@@ -124,6 +125,11 @@ enum {
i915_sf_dump_details = (1 << 9),
i915_sf_dump_dependencies = (1 << 10),
i915_sf_dump_seqno = (1 << 11),
+
+ i915_sf_dump_mask = i915_sf_dump_force |
+ i915_sf_dump_details |
+ i915_sf_dump_dependencies |
+ i915_sf_dump_seqno,
};
const char *i915_scheduler_flag_str(uint32_t flags);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 32/35] drm/i915: Enable GPU scheduler by default
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (30 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 31/35] drm/i915: Scheduler state dump via debugfs John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 33/35] drm/i915: Add scheduling priority to per-context parameters John.C.Harrison
` (4 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Now that all the scheduler patches have been applied, it is safe to enable.
v5: Updated for new module parameter.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_params.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c
index 0ef3159..9be486f 100644
--- a/drivers/gpu/drm/i915/i915_params.c
+++ b/drivers/gpu/drm/i915/i915_params.c
@@ -57,7 +57,7 @@ struct i915_params i915 __read_mostly = {
.edp_vswing = 0,
.enable_guc_submission = true,
.guc_log_level = -1,
- .enable_scheduler = 0,
+ .enable_scheduler = 1,
};
module_param_named(modeset, i915.modeset, int, 0400);
@@ -206,4 +206,4 @@ MODULE_PARM_DESC(guc_log_level,
"GuC firmware logging level (-1:disabled (default), 0-3:enabled)");
module_param_named_unsafe(enable_scheduler, i915.enable_scheduler, int, 0600);
-MODULE_PARM_DESC(enable_scheduler, "Enable scheduler (0 = disable [default], 1 = enable)");
+MODULE_PARM_DESC(enable_scheduler, "Enable scheduler (0 = disable, 1 = enable [default])");
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 33/35] drm/i915: Add scheduling priority to per-context parameters
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (31 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 32/35] drm/i915: Enable GPU scheduler by default John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 34/35] drm/i915: Add support for retro-actively banning batch buffers John.C.Harrison
` (3 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: Dave Gordon <david.s.gordon@intel.com>
Added an interface for user land applications/libraries/services to
set their GPU scheduler priority. This extends the existing context
parameter IOCTL interface to add a scheduler priority parameter. The
range is +/-1023 with +ve numbers meaning higher priority. Only
system processes may set a higher priority than the default (zero),
normal applications may only lower theirs.
v2: New patch in series.
For: VIZ-1587
Signed-off-by: Dave Gordon <David.S.Gordon@Intel.com>
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 14 ++++++++++++++
drivers/gpu/drm/i915/i915_gem_context.c | 24 ++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 3 +++
include/uapi/drm/i915_drm.h | 1 +
4 files changed, 42 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 3f4c4f0..5d02f44 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -847,6 +847,19 @@ struct i915_ctx_hang_stats {
bool banned;
};
+/*
+ * User-settable GFX scheduler priorities are on a scale of -1023 (I don't
+ * care about running) to +1023 (I'm the most important thing in existence)
+ * with zero being the default. Any process may decrease its scheduling
+ * priority, but only a sufficiently privileged process may increase it
+ * beyond zero.
+ */
+
+struct i915_ctx_sched_info {
+ /* Scheduling priority */
+ int32_t priority;
+};
+
struct i915_fence_timeline {
char name[32];
unsigned fence_context;
@@ -887,6 +900,7 @@ struct intel_context {
int flags;
struct drm_i915_file_private *file_priv;
struct i915_ctx_hang_stats hang_stats;
+ struct i915_ctx_sched_info sched_info;
struct i915_hw_ppgtt *ppgtt;
/* Legacy ring buffer submission */
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 3dcb2f4..6ac03e8 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -956,6 +956,9 @@ int i915_gem_context_getparam_ioctl(struct drm_device *dev, void *data,
else
args->value = to_i915(dev)->gtt.base.total;
break;
+ case I915_CONTEXT_PARAM_PRIORITY:
+ args->value = (__u64) ctx->sched_info.priority;
+ break;
default:
ret = -EINVAL;
break;
@@ -993,6 +996,7 @@ int i915_gem_context_setparam_ioctl(struct drm_device *dev, void *data,
else
ctx->hang_stats.ban_period_seconds = args->value;
break;
+
case I915_CONTEXT_PARAM_NO_ZEROMAP:
if (args->size) {
ret = -EINVAL;
@@ -1001,6 +1005,26 @@ int i915_gem_context_setparam_ioctl(struct drm_device *dev, void *data,
ctx->flags |= args->value ? CONTEXT_NO_ZEROMAP : 0;
}
break;
+
+ case I915_CONTEXT_PARAM_PRIORITY:
+ {
+ int32_t priority = (int32_t) args->value;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (args->size)
+ ret = -EINVAL;
+ else if ((priority > scheduler->priority_level_max) ||
+ (priority < scheduler->priority_level_min))
+ ret = -EINVAL;
+ else if ((priority > 0) &&
+ !capable(CAP_SYS_ADMIN))
+ ret = -EPERM;
+ else
+ ctx->sched_info.priority = priority;
+ break;
+ }
+
default:
ret = -EINVAL;
break;
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index a42a13e..793fbce 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1707,6 +1707,9 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
params->args_DR4 = args->DR4;
params->batch_obj = batch_obj;
+ /* Start with the context's priority level */
+ qe.priority = ctx->sched_info.priority;
+
/*
* Save away the list of objects used by this batch buffer for the
* purpose of tracking inter-buffer dependencies.
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index acf2102..8a01a47 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -1140,6 +1140,7 @@ struct drm_i915_gem_context_param {
#define I915_CONTEXT_PARAM_BAN_PERIOD 0x1
#define I915_CONTEXT_PARAM_NO_ZEROMAP 0x2
#define I915_CONTEXT_PARAM_GTT_SIZE 0x3
+#define I915_CONTEXT_PARAM_PRIORITY 0x4
__u64 value;
};
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 34/35] drm/i915: Add support for retro-actively banning batch buffers
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (32 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 33/35] drm/i915: Add scheduling priority to per-context parameters John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 35/35] drm/i915: Allow scheduler to manage inter-ring object synchronisation John.C.Harrison
` (2 subsequent siblings)
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
If a given context submits too many hanging batch buffers then it will
be banned and no further batch buffers will be accepted for it.
However, it is possible that a large number of buffers may already
have been accepted and are sat in the scheduler waiting to be
executed. This patch adds a late ban check to ensure that these will
also be discarded.
v4: New patch in series.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 6 ++++++
drivers/gpu/drm/i915/intel_lrc.c | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 793fbce..0b8c61e 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1292,6 +1292,12 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ /* Check the context wasn't banned between submission and execution: */
+ if (params->ctx->hang_stats.banned) {
+ DRM_DEBUG("Trying to execute for banned context!\n");
+ return -ENOENT;
+ }
+
/* Make sure the request's seqno is the latest and greatest: */
if (req->reserved_seqno != dev_priv->last_seqno) {
ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index e124443..5fbeb0e 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1002,6 +1002,12 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ /* Check the context wasn't banned between submission and execution: */
+ if (params->ctx->hang_stats.banned) {
+ DRM_DEBUG("Trying to execute for banned context!\n");
+ return -ENOENT;
+ }
+
/* Make sure the request's seqno is the latest and greatest: */
if (req->reserved_seqno != dev_priv->last_seqno) {
ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH v5 35/35] drm/i915: Allow scheduler to manage inter-ring object synchronisation
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (33 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 34/35] drm/i915: Add support for retro-actively banning batch buffers John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-18 14:27 ` [PATCH 01/20] igt/gem_ctx_param_basic: Updated to support scheduler priority interface John.C.Harrison
2016-02-18 15:30 ` ✗ Fi.CI.BAT: failure for GPU scheduler for i915 driver Patchwork
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler has always tracked batch buffer dependencies based on
DRM object usage. This means that it will not submit a batch on one
ring that has outstanding dependencies still executing on other rings.
This is exactly the same synchronisation performed by
i915_gem_object_sync() using hardware semaphores where available and
CPU stalls where not (e.g. in execlist mode and/or on Gen8 hardware).
Unfortunately, when a batch buffer is submitted to the driver the
_object_sync() call happens first. Thus in case where hardware
semaphores are disabled, the driver has already stalled until the
dependency has been resolved.
This patch adds an optimisation to _object_sync() to ignore the
synchronisation in the case where it will subsequently be handled by
the scheduler. This removes the driver stall and (in the single
application case) provides near hardware semaphore performance even
when hardware semaphores are disabled. In a busy system where there is
other work that can be executed on the stalling ring, it provides
better than hardware semaphore performance as it removes the stall
from both the driver and from the hardware. There is also a theory
that this method should improve power usage as hardware semaphores are
apparently not very power efficient - the stalled ring does not go
into as low a power a state as when it is genuinely idle.
The optimisation is to check whether both ends of the synchronisation
are batch buffer requests. If they are, then the scheduler will have
the inter-dependency tracked and managed. If one or other end is not a
batch buffer request (e.g. a page flip) then the code falls back to
the CPU stall or hardware semaphore as appropriate.
To check whether the existing usage is a batch buffer, the code simply
calls the 'are you tracking this request' function of the scheduler on
the object's last_read_req member. To check whether the new usage is a
batch buffer, a flag is passed in from the caller.
Issue: VIZ-5566
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 2 +-
drivers/gpu/drm/i915/i915_gem.c | 17 ++++++++++++++---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 +-
drivers/gpu/drm/i915/intel_display.c | 2 +-
drivers/gpu/drm/i915/intel_lrc.c | 2 +-
5 files changed, 18 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 5d02f44..207ac16 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3011,7 +3011,7 @@ int __must_check i915_mutex_lock_interruptible(struct drm_device *dev);
#endif
int i915_gem_object_sync(struct drm_i915_gem_object *obj,
struct intel_engine_cs *to,
- struct drm_i915_gem_request **to_req);
+ struct drm_i915_gem_request **to_req, bool to_batch);
void i915_vma_move_to_active(struct i915_vma *vma,
struct drm_i915_gem_request *req);
int i915_gem_dumb_create(struct drm_file *file_priv,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index a2c136d..b14e384 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3577,7 +3577,7 @@ static int
__i915_gem_object_sync(struct drm_i915_gem_object *obj,
struct intel_engine_cs *to,
struct drm_i915_gem_request *from_req,
- struct drm_i915_gem_request **to_req)
+ struct drm_i915_gem_request **to_req, bool to_batch)
{
struct intel_engine_cs *from;
int ret;
@@ -3589,6 +3589,15 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
if (i915_gem_request_completed(from_req))
return 0;
+ /*
+ * The scheduler will manage inter-ring object dependencies
+ * as long as both to and from requests are scheduler managed
+ * (i.e. batch buffers).
+ */
+ if (to_batch &&
+ i915_scheduler_is_request_tracked(from_req, NULL, NULL))
+ return 0;
+
if (!i915_semaphore_is_enabled(obj->base.dev)) {
struct drm_i915_private *i915 = to_i915(obj->base.dev);
ret = __i915_wait_request(from_req,
@@ -3639,6 +3648,8 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
* @to_req: request we wish to use the object for. See below.
* This will be allocated and returned if a request is
* required but not passed in.
+ * @to_batch: is the sync request on behalf of batch buffer submission?
+ * If so then the scheduler can (potentially) manage the synchronisation.
*
* This code is meant to abstract object synchronization with the GPU.
* Calling with NULL implies synchronizing the object with the CPU
@@ -3669,7 +3680,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
int
i915_gem_object_sync(struct drm_i915_gem_object *obj,
struct intel_engine_cs *to,
- struct drm_i915_gem_request **to_req)
+ struct drm_i915_gem_request **to_req, bool to_batch)
{
const bool readonly = obj->base.pending_write_domain == 0;
struct drm_i915_gem_request *req[I915_NUM_RINGS];
@@ -3691,7 +3702,7 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
req[n++] = obj->last_read_req[i];
}
for (i = 0; i < n; i++) {
- ret = __i915_gem_object_sync(obj, to, req[i], to_req);
+ ret = __i915_gem_object_sync(obj, to, req[i], to_req, to_batch);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 0b8c61e..2923dcd 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -950,7 +950,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
struct drm_i915_gem_object *obj = vma->obj;
if (obj->active & other_rings) {
- ret = i915_gem_object_sync(obj, req->ring, &req);
+ ret = i915_gem_object_sync(obj, req->ring, &req, true);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 5953590..e768026 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11693,7 +11693,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
* into the display plane and skip any waits.
*/
if (!mmio_flip) {
- ret = i915_gem_object_sync(obj, ring, &request);
+ ret = i915_gem_object_sync(obj, ring, &request, false);
if (ret)
goto cleanup_pending;
}
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 5fbeb0e..0a97261 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -686,7 +686,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
struct drm_i915_gem_object *obj = vma->obj;
if (obj->active & other_rings) {
- ret = i915_gem_object_sync(obj, req->ring, &req);
+ ret = i915_gem_object_sync(obj, req->ring, &req, true);
if (ret)
return ret;
}
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* [PATCH 01/20] igt/gem_ctx_param_basic: Updated to support scheduler priority interface
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (34 preceding siblings ...)
2016-02-18 14:27 ` [PATCH v5 35/35] drm/i915: Allow scheduler to manage inter-ring object synchronisation John.C.Harrison
@ 2016-02-18 14:27 ` John.C.Harrison
2016-02-18 15:30 ` ✗ Fi.CI.BAT: failure for GPU scheduler for i915 driver Patchwork
36 siblings, 0 replies; 82+ messages in thread
From: John.C.Harrison @ 2016-02-18 14:27 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The GPU scheduler has added an execution priority level to the context
object. There is an IOCTL interface to allow user apps/libraries to
set this priority. This patch updates the context paramter IOCTL test
to include the new interface.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
lib/ioctl_wrappers.h | 1 +
tests/gem_ctx_param_basic.c | 34 +++++++++++++++++++++++++++++++++-
2 files changed, 34 insertions(+), 1 deletion(-)
diff --git a/lib/ioctl_wrappers.h b/lib/ioctl_wrappers.h
index 214ec78..e650b8f 100644
--- a/lib/ioctl_wrappers.h
+++ b/lib/ioctl_wrappers.h
@@ -105,6 +105,7 @@ struct local_i915_gem_context_param {
#define LOCAL_CONTEXT_PARAM_BAN_PERIOD 0x1
#define LOCAL_CONTEXT_PARAM_NO_ZEROMAP 0x2
#define LOCAL_CONTEXT_PARAM_GTT_SIZE 0x3
+#define LOCAL_CONTEXT_PARAM_PRIORITY 0x4
uint64_t value;
};
void gem_context_require_ban_period(int fd);
diff --git a/tests/gem_ctx_param_basic.c b/tests/gem_ctx_param_basic.c
index b75800c..585a1a8 100644
--- a/tests/gem_ctx_param_basic.c
+++ b/tests/gem_ctx_param_basic.c
@@ -147,10 +147,42 @@ igt_main
TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
}
+ ctx_param.param = LOCAL_CONTEXT_PARAM_PRIORITY;
+
+ igt_subtest("priority-root-set") {
+ ctx_param.context = ctx;
+ ctx_param.value = 2048;
+ TEST_FAIL(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM, EINVAL);
+ ctx_param.value = -2048;
+ TEST_FAIL(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM, EINVAL);
+ ctx_param.value = 512;
+ TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
+ ctx_param.value = -512;
+ TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
+ ctx_param.value = 0;
+ TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
+ }
+
+ igt_subtest("priority-non-root-set") {
+ igt_fork(child, 1) {
+ igt_drop_root();
+
+ ctx_param.context = ctx;
+ ctx_param.value = 512;
+ TEST_FAIL(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM, EPERM);
+ ctx_param.value = -512;
+ TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
+ ctx_param.value = 0;
+ TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
+ }
+
+ igt_waitchildren();
+ }
+
/* NOTE: This testcase intentionally tests for the next free parameter
* to catch ABI extensions. Don't "fix" this testcase without adding all
* the tests for the new param first. */
- ctx_param.param = LOCAL_CONTEXT_PARAM_GTT_SIZE + 1;
+ ctx_param.param = LOCAL_CONTEXT_PARAM_PRIORITY + 1;
igt_subtest("invalid-param-get") {
ctx_param.context = ctx;
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 82+ messages in thread
* ✗ Fi.CI.BAT: failure for GPU scheduler for i915 driver
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
` (35 preceding siblings ...)
2016-02-18 14:27 ` [PATCH 01/20] igt/gem_ctx_param_basic: Updated to support scheduler priority interface John.C.Harrison
@ 2016-02-18 15:30 ` Patchwork
36 siblings, 0 replies; 82+ messages in thread
From: Patchwork @ 2016-02-18 15:30 UTC (permalink / raw)
To: John.C.Harrison; +Cc: intel-gfx
== Summary ==
Series 3585v1 GPU scheduler for i915 driver
2016-02-18T14:27:54.467402 http://patchwork.freedesktop.org/api/1.0/series/3585/revisions/1/mbox/
Applying: drm/i915: Add total count to context status debugfs output
Repository lacks necessary blobs to fall back on 3-way merge.
Cannot fall back to three-way merge.
Patch failed at 0001 drm/i915: Add total count to context status debugfs output
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 06/35] drm/i915: Start of GPU scheduler
2016-02-18 14:26 ` [PATCH v5 06/35] drm/i915: Start of GPU scheduler John.C.Harrison
@ 2016-02-19 13:03 ` Joonas Lahtinen
2016-02-19 17:03 ` John Harrison
0 siblings, 1 reply; 82+ messages in thread
From: Joonas Lahtinen @ 2016-02-19 13:03 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
Hi,
Now the code is in reviewable chunks, excellent!
I've added my comments below. A few repeats from last round, but now
with more questions about the logic itself.
On to, 2016-02-18 at 14:26 +0000, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> Initial creation of scheduler source files. Note that this patch
> implements most of the scheduler functionality but does not hook it in
> to the driver yet. It also leaves the scheduler code in 'pass through'
> mode so that even when it is hooked in, it will not actually do very
> much. This allows the hooks to be added one at a time in bite size
> chunks and only when the scheduler is finally enabled at the end does
> anything start happening.
>
> The general theory of operation is that when batch buffers are
> submitted to the driver, the execbuffer() code packages up all the
> information required to execute the batch buffer at a later time. This
> package is given over to the scheduler which adds it to an internal
> node list. The scheduler also scans the list of objects associated
> with the batch buffer and compares them against the objects already in
> use by other buffers in the node list. If matches are found then the
> new batch buffer node is marked as being dependent upon the matching
> node. The same is done for the context object. The scheduler also
> bumps up the priority of such matching nodes on the grounds that the
> more dependencies a given batch buffer has the more important it is
> likely to be.
>
> The scheduler aims to have a given (tuneable) number of batch buffers
> in flight on the hardware at any given time. If fewer than this are
> currently executing when a new node is queued, then the node is passed
> straight through to the submit function. Otherwise it is simply added
> to the queue and the driver returns back to user land.
>
> The scheduler is notified when each batch buffer completes and updates
> its internal tracking accordingly. At the end of the completion
> interrupt processing, if any scheduler tracked batches were processed,
> the scheduler's deferred worker thread is woken up. This can do more
> involved processing such as actually removing completed nodes from the
> queue and freeing up the resources associated with them (internal
> memory allocations, DRM object references, context reference, etc.).
> The work handler also checks the in flight count and calls the
> submission code if a new slot has appeared.
>
> When the scheduler's submit code is called, it scans the queued node
> list for the highest priority node that has no unmet dependencies.
> Note that the dependency calculation is complex as it must take
> inter-ring dependencies and potential preemptions into account. Note
> also that in the future this will be extended to include external
> dependencies such as the Android Native Sync file descriptors and/or
> the linux dma-buff synchronisation scheme.
>
> If a suitable node is found then it is sent to execbuff_final() for
> submission to the hardware. The in flight count is then re-checked and
> a new node popped from the list if appropriate. All nodes that are not
> submitted have their priority bumped. This ensures that low priority
> tasks do not get starved out by busy higher priority ones - everything
> will eventually get its turn to run.
>
> Note that this patch does not implement pre-emptive scheduling. Only
> basic scheduling by re-ordering batch buffer submission is currently
> implemented. Pre-emption of actively executing batch buffers comes in
> the next patch series.
>
> v2: Changed priority levels to +/-1023 due to feedback from Chris
> Wilson.
>
> Removed redundant index from scheduler node.
>
> Changed time stamps to use jiffies instead of raw monotonic. This
> provides lower resolution but improved compatibility with other i915
> code.
>
> Major re-write of completion tracking code due to struct fence
> conversion. The scheduler no longer has it's own private IRQ handler
> but just lets the existing request code handle completion events.
> Instead, the scheduler now hooks into the request notify code to be
> told when a request has completed.
>
> Reduced driver mutex locking scope. Removal of scheduler nodes no
> longer grabs the mutex lock.
>
> v3: Refactor of dependency generation to make the code more readable.
> Also added in read-read optimisation support - i.e., don't treat a
> shared read-only buffer as being a dependency.
>
> Allowed the killing of queued nodes rather than only flying ones.
>
> v4: Updated the commit message to better reflect the current state of
> the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array
> memory allocator function (kmalloc_array instead of kmalloc).
> Corrected the format of some comments. Wrapped some lines differently
> to keep the style checker happy.
>
> Fixed a WARN_ON when killing nodes. The dependency removal code checks
> that nodes being destroyed do not have any oustanding dependencies
> (which would imply they should not have been executed yet). In the
> case of nodes being destroyed, e.g. due to context banning, then this
> might well be the case - they have not been executed and do indeed
> have outstanding dependencies.
>
> Re-instated the code to disble interrupts when not in use. The
> underlying problem causing broken IRQ reference counts seems to have
> been fixed now.
>
> v5: Shuffled various functions around to remove forward declarations
> as apparently these are frowned upon. Removed lots of white space as
> apparently having easy to read code is also frowned upon. Split the
> direct submission scheduler bypass code out into a separate function.
> Squashed down the i915_scheduler.c sections of various patches into
> this patch. Thus the later patches simply hook in existing code into
> various parts of the driver rather than adding the code as well. Added
> documentation to various functions. Re-worked the submit function in
> terms of mutex locking, error handling and exit paths. Split the
> delayed work handler function in half. Made use of the kernel 'clamp'
> macro. [Joonas Lahtinen]
>
> Added runtime PM calls as these must be done at the top level before
> acquiring the driver mutex lock. [Chris Wilson]
>
> Removed some obsolete debug code that had been forgotten about.
>
> Moved more clean up code into the 'i915_gem_scheduler_clean_node()'
> function rather than replicating it in mutliple places.
>
> Used lighter weight spinlocks.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/Makefile | 1 +
> drivers/gpu/drm/i915/i915_drv.h | 6 +
> drivers/gpu/drm/i915/i915_gem.c | 5 +
> drivers/gpu/drm/i915/i915_scheduler.c | 874 ++++++++++++++++++++++++++++++++++
> drivers/gpu/drm/i915/i915_scheduler.h | 95 ++++
> 5 files changed, 981 insertions(+)
> create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
> create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
>
> diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
> index 15398c5..79cb38b 100644
> --- a/drivers/gpu/drm/i915/Makefile
> +++ b/drivers/gpu/drm/i915/Makefile
> @@ -10,6 +10,7 @@ ccflags-y := -Werror
> i915-y := i915_drv.o \
> i915_irq.o \
> i915_params.o \
> + i915_scheduler.o \
> i915_suspend.o \
> i915_sysfs.o \
> intel_csr.o \
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index f4487b9..03add1a 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -1703,6 +1703,8 @@ struct i915_execbuffer_params {
> struct drm_i915_gem_request *request;
> };
>
> +struct i915_scheduler;
> +
> /* used in computing the new watermarks state */
> struct intel_wm_config {
> unsigned int num_pipes_active;
> @@ -1968,6 +1970,8 @@ struct drm_i915_private {
>
> struct i915_runtime_pm pm;
>
> + struct i915_scheduler *scheduler;
> +
I think we should have i915.enable_scheduler parameter (just like
i915.enable_execlists) and make this a data member, not pointer.
Then we can go forward and have i915.enable_scheduler_preempt and so on
to control things at boot time.
> /* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
> struct {
> int (*execbuf_submit)(struct i915_execbuffer_params *params,
> @@ -2290,6 +2294,8 @@ struct drm_i915_gem_request {
> /** process identifier submitting this request */
> struct pid *pid;
>
> + struct i915_scheduler_queue_entry *scheduler_qe;
> +
Funny whitespace.
> /**
> * The ELSP only accepts two elements at a time, so we queue
> * context/tail pairs on a given queue (ring->execlist_queue) until the
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index dfe43ea..7d9aa24 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -32,6 +32,7 @@
> #include "i915_vgpu.h"
> #include "i915_trace.h"
> #include "intel_drv.h"
> +#include "i915_scheduler.h"
> #include
> #include
> #include
> @@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)
> */
> intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
>
> + ret = i915_scheduler_init(dev);
> + if (ret)
> + goto out_unlock;
> +
> ret = i915_gem_init_userptr(dev);
> if (ret)
> goto out_unlock;
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> new file mode 100644
> index 0000000..fc23ee7
> --- /dev/null
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -0,0 +1,874 @@
> +/*
> + * Copyright (c) 2014 Intel Corporation
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the next
> + * paragraph) shall be included in all copies or substantial portions of the
> + * Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> + * IN THE SOFTWARE.
> + *
> + */
> +
> +#include "i915_drv.h"
> +#include "intel_drv.h"
> +#include "i915_scheduler.h"
> +
> +/**
> + * i915_scheduler_is_enabled - Returns true if the scheduler is enabled.
> + * @dev: DRM device
> + */
> +bool i915_scheduler_is_enabled(struct drm_device *dev)
> +{
> + struct drm_i915_private *dev_priv = dev->dev_private;
> +
> + return dev_priv->scheduler != NULL;
> +}
i915.enable_scheduler would be used instead.
> +
> +/**
> + * i915_scheduler_init - Initialise the scheduler.
> + * @dev: DRM device
> + * Returns zero on success or -ENOMEM if memory allocations fail.
> + */
> +int i915_scheduler_init(struct drm_device *dev)
> +{
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + int r;
> +
This protection -->
> + if (scheduler)
> + return 0;
> +
> + scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
> + if (!scheduler)
> + return -ENOMEM;
> +
Is not needed if we have data member and i915.enable_scheduler.
> + spin_lock_init(&scheduler->lock);
> +
> + for (r = 0; r < I915_NUM_RINGS; r++)
> + INIT_LIST_HEAD(&scheduler->node_queue[r]);
> +
> + /* Default tuning values: */
> + scheduler->priority_level_min = -1023;
> + scheduler->priority_level_max = 1023;
> + scheduler->priority_level_bump = 50;
> + scheduler->priority_level_preempt = 900;
> + scheduler->min_flying = 2;
> +
> + dev_priv->scheduler = scheduler;
> +
> + return 0;
> +}
> +
> +/*
> + * Add a popped node back in to the queue. For example, because the ring was
> + * hung when execfinal() was called and thus the ring submission needs to be
> + * retried later.
> + */
> +static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
> +{
> + WARN_ON(!node);
Rather remove this completely as it is an internal function, or use
BUG_ON because next line will straight lead to OOPS after warning. I'll
pass mentioning rest of the obvious WARN_ON vs BUG_ON's.
> + WARN_ON(!I915_SQS_IS_FLYING(node));
> +
> + /* Seqno will be reassigned on relaunch */
> + node->params.request->seqno = 0;
> + node->status = i915_sqs_queued;
> +}
> +
> +/*
> + * Give up on a node completely. For example, because it is causing the
> + * ring to hang or is using some resource that no longer exists.
> + */
> +static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
> +{
> + WARN_ON(!node);
> + WARN_ON(I915_SQS_IS_COMPLETE(node));
> +
> + node->status = i915_sqs_dead;
> +}
> +
> +/* Mark a node as in flight on the hardware. */
As for documentation, from below, I only assume scheduler lock must be
held and node's can be only manipulated when scheduler lock is held?
It's good to add a comment describing the expected locking. Or maybe
add assert_scheduler_lock_held() helper and sprinkle it around the
functions similar to assert_rpm_wakelock_held(), which is self-
documenting?
> +static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
> +{
> + struct drm_i915_private *dev_priv = node->params.dev->dev_private;
If node is NULL, this would already OOPS, I do not think it's
unreasonable to expect node not to be NULL. The below WARN_ON check is
never reached.
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct intel_engine_cs *ring;
> +
> + WARN_ON(!scheduler);
> + WARN_ON(!node);
> + WARN_ON(node->status != i915_sqs_popped);
> +
> + ring = node->params.ring;
> +
This and similar assignments can be squashed to declaration line, as
this is basically an alias.
> + /*
> + * Add the node (which should currently be in state popped) to the
> + * front of the queue. This ensure that flying nodes are always held
> + * in hardware submission order.
> + */
> + list_add(&node->link, &scheduler->node_queue[ring->id]);
> +
> + node->status = i915_sqs_flying;
> +
> + if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
> + bool success = true;
> +
> + success = ring->irq_get(ring);
> + if (success)
> + scheduler->flags[ring->id] |= i915_sf_interrupts_enabled;
> + else
> + return -EINVAL;
Shouldn't the node be cleaned up from node_queue in case of an error?
Also, I think above could be written much more compact form (because it
looks like something where more logic will be added later). It makes it
easier to write and visualize the error paths when reading if there are
no nested if's.
I won't mention about the error paths of functions below, I expect the
following guidline to be adhered;
if (scheduler->flags[ring->id] & I915_SF_INT_ENABLED)
goto out;
if (!ring->irq_get(ring))
goto err_irq;
if (!...)
goto err_foo;
scheduler->flags[ring->id] |= I915_SF_INT_ENABLED;
out:
return 0;
err_foo:
foobar();
err_irq:
list_remove()
return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static uint32_t i915_scheduler_count_flying(struct i915_scheduler *scheduler,
> + struct intel_engine_cs *ring)
> +{
> + struct i915_scheduler_queue_entry *node;
> + uint32_t flying = 0;
> +
> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link)
+1 for the for_each_scheduler_node(...)
> + if (I915_SQS_IS_FLYING(node))
> + flying++;
> +
> + return flying;
> +}
> +
> +static void i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler)
> +{
> + struct i915_scheduler_queue_entry *node;
> + int i;
> +
> + /*
> + * Ensure circular dependencies don't cause problems and that a bump
> + * by object usage only bumps each using buffer once:
> + */
> + for (i = 0; i < I915_NUM_RINGS; i++) {
> + list_for_each_entry(node, &scheduler->node_queue[i], link)
> + node->bumped = false;
> + }
> +}
> +
> +static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
> + struct i915_scheduler_queue_entry *target,
Is there specific reason why struct i915_scheduler_queue_entry are not
referred to just as "node" but "qe" and here something else, do "node"
and "qe" have a special semantic meaning?
> + uint32_t bump)
> +{
> + uint32_t new_priority;
> + int i, count;
> +
> + if (target->priority >= scheduler->priority_level_max)
> + return 1;
So if one node reaches maximum priority the dependencies are expected
to be maxed out already?
> +
> + if (target->bumped)
> + return 0;
> +
> + new_priority = target->priority + bump;
> + if ((new_priority <= target->priority) ||
> + (new_priority > scheduler->priority_level_max))
> + target->priority = scheduler->priority_level_max;
> + else
> + target->priority = new_priority;
> +
> + count = 1;
> + target->bumped = true;
> +
> + for (i = 0; i < target->num_deps; i++) {
> + if (!target->dep_list[i])
> + continue;
> +
> + if (target->dep_list[i]->bumped)
> + continue;
> +
> + count += i915_scheduler_priority_bump(scheduler,
> + target->dep_list[i],
> + bump);
> + }
> +
> + return count;
> +}
> +
> +/*
> + * Nodes are considered valid dependencies if they are queued on any ring or
> + * if they are in flight on a different ring. In flight on the same ring is no
> + * longer interesting for non-premptive nodes as the ring serialises execution.
> + * For pre-empting nodes, all in flight dependencies are valid as they must not
> + * be jumped by the act of pre-empting.
> + *
> + * Anything that is neither queued nor flying is uninteresting.
> + */
> +static inline bool i915_scheduler_is_dependency_valid(
> + struct i915_scheduler_queue_entry *node, uint32_t idx)
> +{
> + struct i915_scheduler_queue_entry *dep;
> +
> + dep = node->dep_list[idx];
> + if (!dep)
> + return false;
> +
> + if (I915_SQS_IS_QUEUED(dep))
> + return true;
> +
> + if (I915_SQS_IS_FLYING(dep)) {
> + if (node->params.ring != dep->params.ring)
> + return true;
> + }
> +
> + return false;
> +}
> +
> +static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry **pop_node)
> +{
> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct i915_scheduler_queue_entry *best = NULL;
> + struct i915_scheduler_queue_entry *node;
> + int ret;
> + int i;
> + bool any_queued = false;
> + bool has_local, has_remote, only_remote = false;
> +
> + *pop_node = NULL;
> + ret = -ENODATA;
> +
> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> + if (!I915_SQS_IS_QUEUED(node))
> + continue;
> + any_queued = true;
> +
> + has_local = false;
> + has_remote = false;
> + for (i = 0; i < node->num_deps; i++) {
> + if (!i915_scheduler_is_dependency_valid(node, i))
> + continue;
> +
> + if (node->dep_list[i]->params.ring == node->params.ring)
> + has_local = true;
> + else
> + has_remote = true;
> + }
> +
> + if (has_remote && !has_local)
> + only_remote = true;
> +
> + if (!has_local && !has_remote) {
> + if (!best ||
> + (node->priority > best->priority))
> + best = node;
> + }
> + }
> +
> + if (best) {
> + list_del(&best->link);
> +
> + INIT_LIST_HEAD(&best->link);
> + best->status = i915_sqs_popped;
> +
> + ret = 0;
> + } else {
> + /* Can only get here if:
> + * (a) there are no buffers in the queue
> + * (b) all queued buffers are dependent on other buffers
> + * e.g. on a buffer that is in flight on a different ring
> + */
> + if (only_remote) {
> + /* The only dependent buffers are on another ring. */
> + ret = -EAGAIN;
> + } else if (any_queued) {
> + /* It seems that something has gone horribly wrong! */
> + DRM_ERROR("Broken dependency tracking on ring %d!\n",
> + (int) ring->id);
Would this condition be worthy WARN_ONCE(), because it will keep
repeating as the situation will persist?
> + }
> + }
> +
> + *pop_node = best;
> + return ret;
> +}
> +
> +/*
> + * NB: The driver mutex lock must be held before calling this function. It is
> + * only really required during the actual back end submission call. However,
> + * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
> + * And releasing the one before acquiring the other leads to other code
> + * being run and interfering.
> + */
> +static int i915_scheduler_submit(struct intel_engine_cs *ring)
> +{
> + struct drm_device *dev = ring->dev;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct i915_scheduler_queue_entry *node;
> + int ret, count = 0, flying;
> +
> + WARN_ON(!mutex_is_locked(&dev->struct_mutex));
> +
> + spin_lock_irq(&scheduler->lock);
> +
> + WARN_ON(scheduler->flags[ring->id] & i915_sf_submitting);
> + scheduler->flags[ring->id] |= i915_sf_submitting;
> +
> + /* First time around, complain if anything unexpected occurs: */
> + ret = i915_scheduler_pop_from_queue_locked(ring, &node);
> + if (ret)
> + goto error;
> +
> + do {
> + WARN_ON(!node);
> + WARN_ON(node->params.ring != ring);
> + WARN_ON(node->status != i915_sqs_popped);
> + count++;
> +
> + /*
> + * The call to pop above will have removed the node from the
> + * list. So add it back in and mark it as in flight.
> + */
> + i915_scheduler_node_fly(node);
> +
> + spin_unlock_irq(&scheduler->lock);
> + ret = dev_priv->gt.execbuf_final(&node->params);
> + spin_lock_irq(&scheduler->lock);
> +
> + /*
> + * Handle failed submission but first check that the
> + * watchdog/reset code has not nuked the node while we
> + * weren't looking:
> + */
> + if (ret && (node->status != i915_sqs_dead)) {
> + bool requeue = true;
> +
> + /*
> + * Oh dear! Either the node is broken or the ring is
> + * busy. So need to kill the node or requeue it and try
> + * again later as appropriate.
> + */
> +
> + switch (-ret) {
> + case ENODEV:
> + case ENOENT:
> + /* Fatal errors. Kill the node. */
> + requeue = false;
> + i915_scheduler_node_kill(node);
> + break;
These break indents still.
> +
> + case EAGAIN:
> + case EBUSY:
> + case EIO:
> + case ENOMEM:
> + case ERESTARTSYS:
> + case EINTR:
> + /* Supposedly recoverable errors. */
> + break;
> +
> + default:
> + /*
> + * Assume the error is recoverable and hope
> + * for the best.
> + */
> + MISSING_CASE(-ret);
> + break;
> + }
> +
> + if (requeue) {
> + i915_scheduler_node_requeue(node);
> + /*
> + * No point spinning if the ring is currently
> + * unavailable so just give up and come back
> + * later.
> + */
> + break;
> + }
> + }
> +
> + /* Keep launching until the sky is sufficiently full. */
> + flying = i915_scheduler_count_flying(scheduler, ring);
> + if (flying >= scheduler->min_flying)
> + break;
> +
> + /* Grab another node and go round again... */
> + ret = i915_scheduler_pop_from_queue_locked(ring, &node);
> + } while (ret == 0);
> +
> + /* Don't complain about not being able to submit extra entries */
> + if (ret == -ENODATA)
> + ret = 0;
> +
> + /*
> + * Bump the priority of everything that was not submitted to prevent
> + * starvation of low priority tasks by a spamming high priority task.
> + */
This, this calls for an I-G-T test to make sure we're effective.
> + i915_scheduler_priority_bump_clear(scheduler);
> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> + if (!I915_SQS_IS_QUEUED(node))
> + continue;
> +
> + i915_scheduler_priority_bump(scheduler, node,
> + scheduler->priority_level_bump);
> + }
> +
> + /* On success, return the number of buffers submitted. */
> + if (ret == 0)
> + ret = count;
> +
> +error:
> + scheduler->flags[ring->id] &= ~i915_sf_submitting;
> + spin_unlock_irq(&scheduler->lock);
> + return ret;
> +}
> +
> +static void i915_generate_dependencies(struct i915_scheduler *scheduler,
> + struct i915_scheduler_queue_entry *node,
> + uint32_t ring)
> +{
> + struct i915_scheduler_obj_entry *this, *that;
> + struct i915_scheduler_queue_entry *test;
> + int i, j;
> + bool found;
> +
> + list_for_each_entry(test, &scheduler->node_queue[ring], link) {
> + if (I915_SQS_IS_COMPLETE(test))
> + continue;
> +
> + /*
> + * Batches on the same ring for the same
> + * context must be kept in order.
> + */
> + found = (node->params.ctx == test->params.ctx) &&
> + (node->params.ring == test->params.ring);
> +
> + /*
> + * Batches working on the same objects must
> + * be kept in order.
> + */
> + for (i = 0; (i < node->num_objs) && !found; i++) {
> + this = node->saved_objects + i;
node->objs and node->num_objs would be a better naming match. num_objs
and saved_objects makes me think we have a bug here, indexing wrong
array.
> +
> + for (j = 0; j < test->num_objs; j++) {
> + that = test->saved_objects + j;
> +
> + if (this->obj != that->obj)
> + continue;
> +
> + /* Only need to worry about writes */
> + if (this->read_only && that->read_only)
> + continue;
> +
> + found = true;
> + break;
> + }
> + }
> +
> + if (found) {
> + node->dep_list[node->num_deps] = test;
> + node->num_deps++;
> + }
> + }
> +}
> +
> +static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_entry *qe)
> +{
> + struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + int ret;
> +
> + scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
> + ret = dev_priv->gt.execbuf_final(&qe->params);
> + scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
> +
The above code makes me think of locking again, maybe document at top
of this function.
> + /*
> + * Don't do any clean up on failure because the caller will
> + * do it all anyway.
> + */
> + if (ret)
> + return ret;
> +
> + /* Need to release any resources held by the node: */
> + i915_scheduler_clean_node(qe);
> +
> + return 0;
> +}
> +
> +/**
> + * i915_scheduler_queue_execbuffer - Submit a batch buffer request to the
> + * scheduler.
> + * @qe: The batch buffer request to be queued.
> + * The expectation is the qe passed in is a local stack variable. This
> + * function will copy its contents into a freshly allocated list node. The
> + * new node takes ownership of said contents so the original qe should simply
> + * be discarded and not cleaned up (i.e. don't free memory it points to or
> + * dereference objects it holds). The node is added to the scheduler's queue
> + * and the batch buffer will be submitted to the hardware at some future
> + * point in time (which may be immediately, before returning or may be quite
> + * a lot later).
> + */
> +int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
> +{
> + struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct intel_engine_cs *ring = qe->params.ring;
> + struct i915_scheduler_queue_entry *node;
> + struct i915_scheduler_queue_entry *test;
> + bool not_flying;
> + int i, r;
> + int incomplete = 0;
> +
> + WARN_ON(!scheduler);
> +
> + if (1/*!i915.enable_scheduler*/)
> + return i915_scheduler_queue_execbuffer_bypass(qe);
> +
I'm thinking, should we branch already before calling a function named
i915_scheduler_queue_execbuffer if scheduler is disabled?
> + node = kmalloc(sizeof(*node), GFP_KERNEL);
> + if (!node)
> + return -ENOMEM;
> +
> + *node = *qe;
Now I read the added documentation for the function, maybe we should at
least zero out qe to avoid future problems?
> + INIT_LIST_HEAD(&node->link);
> + node->status = i915_sqs_queued;
> + node->stamp = jiffies;
> + i915_gem_request_reference(node->params.request);
> +
> + WARN_ON(node->params.request->scheduler_qe);
> + node->params.request->scheduler_qe = node;
> +
> + /* Need to determine the number of incomplete entries in the list as
> + * that will be the maximum size of the dependency list.
> + *
> + * Note that the allocation must not be made with the spinlock acquired
> + * as kmalloc can sleep. However, the unlock/relock is safe because no
> + * new entries can be queued up during the unlock as the i915 driver
> + * mutex is still held. Entries could be removed from the list but that
> + * just means the dep_list will be over-allocated which is fine.
> + */
> + spin_lock_irq(&scheduler->lock);
-->
> + for (r = 0; r < I915_NUM_RINGS; r++) {
> + list_for_each_entry(test, &scheduler->node_queue[r], link) {
> + if (I915_SQS_IS_COMPLETE(test))
> + continue;
> +
> + incomplete++;
> + }
> + }
> +
<-- This counting structure, I still think it would be good idea to
make it a static helper.
> + /* Temporarily unlock to allocate memory: */
> + spin_unlock_irq(&scheduler->lock);
> + if (incomplete) {
> + node->dep_list = kmalloc_array(incomplete,
> + sizeof(*node->dep_list),
> + GFP_KERNEL);
> + if (!node->dep_list) {
> + kfree(node);
> + return -ENOMEM;
> + }
> + } else
> + node->dep_list = NULL;
> +
> + spin_lock_irq(&scheduler->lock);
> + node->num_deps = 0;
> +
> + if (node->dep_list) {
> + for (r = 0; r < I915_NUM_RINGS; r++)
> + i915_generate_dependencies(scheduler, node, r);
> +
> + WARN_ON(node->num_deps > incomplete);
> + }
> +
> + node->priority = clamp(node->priority,
> + scheduler->priority_level_min,
> + scheduler->priority_level_max);
> +
> + if ((node->priority > 0) && node->num_deps) {
> + i915_scheduler_priority_bump_clear(scheduler);
> +
> + for (i = 0; i < node->num_deps; i++)
> + i915_scheduler_priority_bump(scheduler,
> + node->dep_list[i], node->priority);
If node is posted with maximum priority to begin with, wouldn't the
priority propagation stop after first level dependencies due to the
check in the beginning of priority_bump function?
> +
> + list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
> +
> + not_flying = i915_scheduler_count_flying(scheduler, ring) <
> + scheduler->min_flying;
> +
> + spin_unlock_irq(&scheduler->lock);
> +
> + if (not_flying)
> + i915_scheduler_submit(ring);
> +
> + return 0;
> +}
> +
> +/**
> + * i915_scheduler_notify_request - Notify the scheduler that the given
> + * request has completed on the hardware.
> + * @req: Request structure which has completed
> + * @preempt: Did it complete pre-emptively?
> + * A sequence number has popped out of the hardware and the request handling
> + * code has mapped it back to a request and will mark that request complete.
> + * It also calls this function to notify the scheduler about the completion
> + * so the scheduler's node can be updated appropriately.
> + * Returns true if the request is scheduler managed, false if not. The return
> + * value is combined for all freshly completed requests and if any were true
> + * then i915_scheduler_wakeup() is called so the scheduler can do further
> + * processing (submit more work) at the end.
> + */
> +bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
> +{
> + struct drm_i915_private *dev_priv = to_i915(req->ring->dev);
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct i915_scheduler_queue_entry *node = req->scheduler_qe;
> + unsigned long flags;
> +
> + if (!node)
> + return false;
> +
> + spin_lock_irqsave(&scheduler->lock, flags);
> +
> + WARN_ON(!I915_SQS_IS_FLYING(node));
> +
> + /* Node was in flight so mark it as complete. */
> + if (req->cancelled)
> + node->status = i915_sqs_dead;
> + else
> + node->status = i915_sqs_complete;
> +
> + spin_unlock_irqrestore(&scheduler->lock, flags);
> +
> + return true;
> +}
> +
> +static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
> + struct i915_scheduler_queue_entry *remove)
> +{
> + struct i915_scheduler_queue_entry *node;
> + int i, r;
> + int count = 0;
> +
> + /*
> + * Ensure that a node is not being removed which is still dependent
> + * upon other (not completed) work. If that happens, it implies
> + * something has gone very wrong with the dependency tracking! Note
> + * that there is no need to worry if this node has been explicitly
> + * killed for some reason - it might be being killed before it got
> + * sent to the hardware.
> + */
> + if (remove->status != i915_sqs_dead) {
> + for (i = 0; i < remove->num_deps; i++)
> + if ((remove->dep_list[i]) &&
> + (!I915_SQS_IS_COMPLETE(remove->dep_list[i])))
> + count++;
> + WARN_ON(count);
> + }
> +
> + /*
> + * Remove this node from the dependency lists of any other node which
> + * might be waiting on it.
> + */
> + for (r = 0; r < I915_NUM_RINGS; r++) {
> + list_for_each_entry(node, &scheduler->node_queue[r], link) {
> + for (i = 0; i < node->num_deps; i++) {
> + if (node->dep_list[i] != remove)
> + continue;
> +
> + node->dep_list[i] = NULL;
> + }
> + }
> + }
> +
> + return 0;
> +}
> +
> +/**
> + * i915_scheduler_wakeup - wake the scheduler's worker thread
> + * @dev: DRM device
> + * Called at the end of seqno interrupt processing if any request has
> + * completed that corresponds to a scheduler node.
> + */
> +void i915_scheduler_wakeup(struct drm_device *dev)
> +{
> + /* XXX: Need to call i915_scheduler_remove() via work handler. */
> +}
> +
> +/**
> + * i915_scheduler_clean_node - free up any allocations/references
> + * associated with the given scheduler queue entry.
> + * @node: Queue entry structure which is complete
> + * After a give batch buffer completes on the hardware, all the information
> + * required to resubmit it is no longer required. However, the node entry
> + * itself might still be required for tracking purposes for a while longer.
> + * This function should be called as soon as the node is known to be complete
> + * so that these resources may be freed even though the node itself might
> + * hang around.
> + */
> +void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
> +{
> + if (!I915_SQS_IS_COMPLETE(node)) {
> + WARN(!node->params.request->cancelled,
> + "Cleaning active node: %d!\n", node->status);
> + return;
> + }
> +
> + if (node->params.batch_obj) {
> + /*
> + * The batch buffer must be unpinned before it is unreferenced
> + * otherwise the unpin fails with a missing vma!?
> + */
> + if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
> + i915_gem_execbuff_release_batch_obj(node->params.batch_obj);
> +
This code seems little bit out of place.
> + node->params.batch_obj = NULL;
> + }
> +
> + /* And anything else owned by the node: */
> + if (node->params.cliprects) {
> + kfree(node->params.cliprects);
> + node->params.cliprects = NULL;
> + }
> +}
> +
The below function is quite hacky, I understood this will be mitigated
by per-ctx sequence numbering in the future?
> +static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
> + struct intel_engine_cs *ring,
> + struct list_head *remove)
> +{
> + struct i915_scheduler_queue_entry *node, *node_next;
> + int flying = 0, queued = 0;
> + bool do_submit;
> + uint32_t min_seqno;
> +
> + spin_lock_irq(&scheduler->lock);
> +
> + /*
> + * In the case where the system is idle, starting 'min_seqno' from a big
> + * number will cause all nodes to be removed as they are now back to
> + * being in-order. However, this will be a problem if the last one to
> + * complete was actually out-of-order as the ring seqno value will be
> + * lower than one or more completed buffers. Thus code looking for the
> + * completion of said buffers will wait forever.
> + * Instead, use the hardware seqno as the starting point. This means
> + * that some buffers might be kept around even in a completely idle
> + * system but it should guarantee that no-one ever gets confused when
> + * waiting for buffer completion.
> + */
> + min_seqno = ring->get_seqno(ring, true);
> +
> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> + if (I915_SQS_IS_QUEUED(node))
> + queued++;
> + else if (I915_SQS_IS_FLYING(node))
> + flying++;
> + else if (I915_SQS_IS_COMPLETE(node))
> + continue;
> +
> + if (node->params.request->seqno == 0)
> + continue;
> +
> + if (!i915_seqno_passed(node->params.request->seqno, min_seqno))
> + min_seqno = node->params.request->seqno;
> + }
> +
> + INIT_LIST_HEAD(remove);
> + list_for_each_entry_safe(node, node_next, &scheduler->node_queue[ring->id], link) {
> + /*
> + * Only remove completed nodes which have a lower seqno than
> + * all pending nodes. While there is the possibility of the
> + * ring's seqno counting backwards, all higher buffers must
> + * be remembered so that the 'i915_seqno_passed()' test can
> + * report that they have in fact passed.
> + *
> + * NB: This is not true for 'dead' nodes. The GPU reset causes
> + * the software seqno to restart from its initial value. Thus
> + * the dead nodes must be removed even though their seqno values
> + * are potentially vastly greater than the current ring seqno.
> + */
> + if (!I915_SQS_IS_COMPLETE(node))
> + continue;
> +
> + if (node->status != i915_sqs_dead) {
> + if (i915_seqno_passed(node->params.request->seqno, min_seqno) &&
> + (node->params.request->seqno != min_seqno))
> + continue;
> + }
> +
> + list_del(&node->link);
> + list_add(&node->link, remove);
> +
> + /* Strip the dependency info while the mutex is still locked */
> + i915_scheduler_remove_dependent(scheduler, node);
> +
> + continue;
> + }
> +
> + /*
> + * Release the interrupt reference count if there are no longer any
> + * nodes to worry about.
> + */
> + if (!flying && !queued &&
> + (scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
> + ring->irq_put(ring);
> + scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;
Maybe have this piece of code as a helper? To enable interrupts and
disable them, maybe even a reference count?
> + }
> +
> + /* Launch more packets now? */
> + do_submit = (queued > 0) && (flying < scheduler->min_flying);
> +
> + spin_unlock_irq(&scheduler->lock);
> +
> + return do_submit;
> +}
> +
> +void i915_scheduler_process_work(struct intel_engine_cs *ring)
> +{
> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct i915_scheduler_queue_entry *node;
> + bool do_submit;
> + struct list_head remove;
> +
> + if (list_empty(&scheduler->node_queue[ring->id]))
> + return;
> +
> + /* Remove completed nodes. */
> + do_submit = i915_scheduler_remove(scheduler, ring, &remove);
> +
> + if (!do_submit && list_empty(&remove))
> + return;
> +
> + /* Need to grab the pm lock outside of the mutex lock */
> + if (do_submit)
> + intel_runtime_pm_get(dev_priv);
> +
> + mutex_lock(&ring->dev->struct_mutex);
> +
> + if (do_submit)
> + i915_scheduler_submit(ring);
> +
> + while (!list_empty(&remove)) {
> + node = list_first_entry(&remove, typeof(*node), link);
> + list_del(&node->link);
> +
> + /* Free up all the DRM references */
> + i915_scheduler_clean_node(node);
> +
> + /* And anything else owned by the node: */
> + node->params.request->scheduler_qe = NULL;
> + i915_gem_request_unreference(node->params.request);
> + kfree(node->dep_list);
> + kfree(node);
Maybe have a separate helper for freeing a node, would make this much
cleaner.
> + }
> +
> + mutex_unlock(&ring->dev->struct_mutex);
> +
> + if (do_submit)
> + intel_runtime_pm_put(dev_priv);
> +}
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> new file mode 100644
> index 0000000..415fec8
> --- /dev/null
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -0,0 +1,95 @@
> +/*
> + * Copyright (c) 2014 Intel Corporation
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the next
> + * paragraph) shall be included in all copies or substantial portions of the
> + * Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> + * IN THE SOFTWARE.
> + *
> + */
> +
> +#ifndef _I915_SCHEDULER_H_
> +#define _I915_SCHEDULER_H_
> +
> +enum i915_scheduler_queue_status {
> + /* Limbo: */
> + i915_sqs_none = 0,
> + /* Not yet submitted to hardware: */
> + i915_sqs_queued,
> + /* Popped from queue, ready to fly: */
> + i915_sqs_popped,
> + /* Sent to hardware for processing: */
> + i915_sqs_flying,
> + /* Finished processing on the hardware: */
> + i915_sqs_complete,
> + /* Killed by watchdog or catastrophic submission failure: */
> + i915_sqs_dead,
> + /* Limit value for use with arrays/loops */
> + i915_sqs_MAX
> +};
> +
To uppercase.
> +#define I915_SQS_IS_QUEUED(node) (((node)->status == i915_sqs_queued))
> +#define I915_SQS_IS_FLYING(node) (((node)->status == i915_sqs_flying))
> +#define I915_SQS_IS_COMPLETE(node) (((node)->status == i915_sqs_complete) || \
> + ((node)->status == i915_sqs_dead))
> +
> +struct i915_scheduler_obj_entry {
> + struct drm_i915_gem_object *obj;
> + bool read_only;
> +};
> +
> +struct i915_scheduler_queue_entry {
> + struct i915_execbuffer_params params;
> + /* -1023 = lowest priority, 0 = default, 1023 = highest */
> + int32_t priority;
> + struct i915_scheduler_obj_entry *saved_objects;
> + int num_objs;
> + bool bumped;
> + struct i915_scheduler_queue_entry **dep_list;
> + int num_deps;
> + enum i915_scheduler_queue_status status;
> + unsigned long stamp;
> + struct list_head link;
Above whitespace is still incorrect and I could really use comments.
> +};
> +
> +struct i915_scheduler {
> + struct list_head node_queue[I915_NUM_RINGS];
> + uint32_t flags[I915_NUM_RINGS];
> + spinlock_t lock;
> +
> + /* Tuning parameters: */
> + int32_t priority_level_min;
> + int32_t priority_level_max;
> + int32_t priority_level_bump;
> + int32_t priority_level_preempt;
> + uint32_t min_flying;
> +};
> +
> +/* Flag bits for i915_scheduler::flags */
> +enum {
> + i915_sf_interrupts_enabled = (1 << 0),
> + i915_sf_submitting = (1 << 1),
> +};
These must be I915_UPPERCASE_SOMETHING by convention.
> +
> +bool i915_scheduler_is_enabled(struct drm_device *dev);
> +int i915_scheduler_init(struct drm_device *dev);
> +void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
> +int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
> +bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
> +void i915_scheduler_wakeup(struct drm_device *dev);
> +
> +#endif /* _I915_SCHEDULER_H_ */
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 06/35] drm/i915: Start of GPU scheduler
2016-02-19 13:03 ` Joonas Lahtinen
@ 2016-02-19 17:03 ` John Harrison
2016-02-26 9:13 ` Joonas Lahtinen
0 siblings, 1 reply; 82+ messages in thread
From: John Harrison @ 2016-02-19 17:03 UTC (permalink / raw)
To: Joonas Lahtinen, Intel-GFX
On 19/02/2016 13:03, Joonas Lahtinen wrote:
> Hi,
>
> Now the code is in reviewable chunks, excellent!
>
> I've added my comments below. A few repeats from last round, but now
> with more questions about the logic itself.
>
> On to, 2016-02-18 at 14:26 +0000, John.C.Harrison@Intel.com wrote:
>> From: John Harrison <John.C.Harrison@Intel.com>
>>
>> Initial creation of scheduler source files. Note that this patch
>> implements most of the scheduler functionality but does not hook it in
>> to the driver yet. It also leaves the scheduler code in 'pass through'
>> mode so that even when it is hooked in, it will not actually do very
>> much. This allows the hooks to be added one at a time in bite size
>> chunks and only when the scheduler is finally enabled at the end does
>> anything start happening.
>>
>> The general theory of operation is that when batch buffers are
>> submitted to the driver, the execbuffer() code packages up all the
>> information required to execute the batch buffer at a later time. This
>> package is given over to the scheduler which adds it to an internal
>> node list. The scheduler also scans the list of objects associated
>> with the batch buffer and compares them against the objects already in
>> use by other buffers in the node list. If matches are found then the
>> new batch buffer node is marked as being dependent upon the matching
>> node. The same is done for the context object. The scheduler also
>> bumps up the priority of such matching nodes on the grounds that the
>> more dependencies a given batch buffer has the more important it is
>> likely to be.
>>
>> The scheduler aims to have a given (tuneable) number of batch buffers
>> in flight on the hardware at any given time. If fewer than this are
>> currently executing when a new node is queued, then the node is passed
>> straight through to the submit function. Otherwise it is simply added
>> to the queue and the driver returns back to user land.
>>
>> The scheduler is notified when each batch buffer completes and updates
>> its internal tracking accordingly. At the end of the completion
>> interrupt processing, if any scheduler tracked batches were processed,
>> the scheduler's deferred worker thread is woken up. This can do more
>> involved processing such as actually removing completed nodes from the
>> queue and freeing up the resources associated with them (internal
>> memory allocations, DRM object references, context reference, etc.).
>> The work handler also checks the in flight count and calls the
>> submission code if a new slot has appeared.
>>
>> When the scheduler's submit code is called, it scans the queued node
>> list for the highest priority node that has no unmet dependencies.
>> Note that the dependency calculation is complex as it must take
>> inter-ring dependencies and potential preemptions into account. Note
>> also that in the future this will be extended to include external
>> dependencies such as the Android Native Sync file descriptors and/or
>> the linux dma-buff synchronisation scheme.
>>
>> If a suitable node is found then it is sent to execbuff_final() for
>> submission to the hardware. The in flight count is then re-checked and
>> a new node popped from the list if appropriate. All nodes that are not
>> submitted have their priority bumped. This ensures that low priority
>> tasks do not get starved out by busy higher priority ones - everything
>> will eventually get its turn to run.
>>
>> Note that this patch does not implement pre-emptive scheduling. Only
>> basic scheduling by re-ordering batch buffer submission is currently
>> implemented. Pre-emption of actively executing batch buffers comes in
>> the next patch series.
>>
>> v2: Changed priority levels to +/-1023 due to feedback from Chris
>> Wilson.
>>
>> Removed redundant index from scheduler node.
>>
>> Changed time stamps to use jiffies instead of raw monotonic. This
>> provides lower resolution but improved compatibility with other i915
>> code.
>>
>> Major re-write of completion tracking code due to struct fence
>> conversion. The scheduler no longer has it's own private IRQ handler
>> but just lets the existing request code handle completion events.
>> Instead, the scheduler now hooks into the request notify code to be
>> told when a request has completed.
>>
>> Reduced driver mutex locking scope. Removal of scheduler nodes no
>> longer grabs the mutex lock.
>>
>> v3: Refactor of dependency generation to make the code more readable.
>> Also added in read-read optimisation support - i.e., don't treat a
>> shared read-only buffer as being a dependency.
>>
>> Allowed the killing of queued nodes rather than only flying ones.
>>
>> v4: Updated the commit message to better reflect the current state of
>> the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array
>> memory allocator function (kmalloc_array instead of kmalloc).
>> Corrected the format of some comments. Wrapped some lines differently
>> to keep the style checker happy.
>>
>> Fixed a WARN_ON when killing nodes. The dependency removal code checks
>> that nodes being destroyed do not have any oustanding dependencies
>> (which would imply they should not have been executed yet). In the
>> case of nodes being destroyed, e.g. due to context banning, then this
>> might well be the case - they have not been executed and do indeed
>> have outstanding dependencies.
>>
>> Re-instated the code to disble interrupts when not in use. The
>> underlying problem causing broken IRQ reference counts seems to have
>> been fixed now.
>>
>> v5: Shuffled various functions around to remove forward declarations
>> as apparently these are frowned upon. Removed lots of white space as
>> apparently having easy to read code is also frowned upon. Split the
>> direct submission scheduler bypass code out into a separate function.
>> Squashed down the i915_scheduler.c sections of various patches into
>> this patch. Thus the later patches simply hook in existing code into
>> various parts of the driver rather than adding the code as well. Added
>> documentation to various functions. Re-worked the submit function in
>> terms of mutex locking, error handling and exit paths. Split the
>> delayed work handler function in half. Made use of the kernel 'clamp'
>> macro. [Joonas Lahtinen]
>>
>> Added runtime PM calls as these must be done at the top level before
>> acquiring the driver mutex lock. [Chris Wilson]
>>
>> Removed some obsolete debug code that had been forgotten about.
>>
>> Moved more clean up code into the 'i915_gem_scheduler_clean_node()'
>> function rather than replicating it in mutliple places.
>>
>> Used lighter weight spinlocks.
>>
>> For: VIZ-1587
>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
>> ---
>> drivers/gpu/drm/i915/Makefile | 1 +
>> drivers/gpu/drm/i915/i915_drv.h | 6 +
>> drivers/gpu/drm/i915/i915_gem.c | 5 +
>> drivers/gpu/drm/i915/i915_scheduler.c | 874 ++++++++++++++++++++++++++++++++++
>> drivers/gpu/drm/i915/i915_scheduler.h | 95 ++++
>> 5 files changed, 981 insertions(+)
>> create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
>> create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
>>
>> diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
>> index 15398c5..79cb38b 100644
>> --- a/drivers/gpu/drm/i915/Makefile
>> +++ b/drivers/gpu/drm/i915/Makefile
>> @@ -10,6 +10,7 @@ ccflags-y := -Werror
>> i915-y := i915_drv.o \
>> i915_irq.o \
>> i915_params.o \
>> + i915_scheduler.o \
>> i915_suspend.o \
>> i915_sysfs.o \
>> intel_csr.o \
>> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
>> index f4487b9..03add1a 100644
>> --- a/drivers/gpu/drm/i915/i915_drv.h
>> +++ b/drivers/gpu/drm/i915/i915_drv.h
>> @@ -1703,6 +1703,8 @@ struct i915_execbuffer_params {
>> struct drm_i915_gem_request *request;
>> };
>>
>> +struct i915_scheduler;
>> +
>> /* used in computing the new watermarks state */
>> struct intel_wm_config {
>> unsigned int num_pipes_active;
>> @@ -1968,6 +1970,8 @@ struct drm_i915_private {
>>
>> struct i915_runtime_pm pm;
>>
>> + struct i915_scheduler *scheduler;
>> +
> I think we should have i915.enable_scheduler parameter (just like
> i915.enable_execlists) and make this a data member, not pointer.
>
> Then we can go forward and have i915.enable_scheduler_preempt and so on
> to control things at boot time.
There are. The enable_scheduler comes in as patch #21. Including it from
the start means modifying i915_params.[ch] in this patch as well. And
then you are getting into one big patch that contains everything and is
unreviewable due to its sheer size. The enable_preemption parameter
comes in with the pre-emption code. No point in adding that until there
is something for it to control.
>
>> /* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
>> struct {
>> int (*execbuf_submit)(struct i915_execbuffer_params *params,
>> @@ -2290,6 +2294,8 @@ struct drm_i915_gem_request {
>> /** process identifier submitting this request */
>> struct pid *pid;
>>
>> + struct i915_scheduler_queue_entry *scheduler_qe;
>> +
> Funny whitespace.
>
>> /**
>> * The ELSP only accepts two elements at a time, so we queue
>> * context/tail pairs on a given queue (ring->execlist_queue) until the
>> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
>> index dfe43ea..7d9aa24 100644
>> --- a/drivers/gpu/drm/i915/i915_gem.c
>> +++ b/drivers/gpu/drm/i915/i915_gem.c
>> @@ -32,6 +32,7 @@
>> #include "i915_vgpu.h"
>> #include "i915_trace.h"
>> #include "intel_drv.h"
>> +#include "i915_scheduler.h"
>> #include
>> #include
>> #include
>> @@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)
>> */
>> intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
>>
>> + ret = i915_scheduler_init(dev);
>> + if (ret)
>> + goto out_unlock;
>> +
>> ret = i915_gem_init_userptr(dev);
>> if (ret)
>> goto out_unlock;
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
>> new file mode 100644
>> index 0000000..fc23ee7
>> --- /dev/null
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
>> @@ -0,0 +1,874 @@
>> +/*
>> + * Copyright (c) 2014 Intel Corporation
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a
>> + * copy of this software and associated documentation files (the "Software"),
>> + * to deal in the Software without restriction, including without limitation
>> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
>> + * and/or sell copies of the Software, and to permit persons to whom the
>> + * Software is furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice (including the next
>> + * paragraph) shall be included in all copies or substantial portions of the
>> + * Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
>> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
>> + * IN THE SOFTWARE.
>> + *
>> + */
>> +
>> +#include "i915_drv.h"
>> +#include "intel_drv.h"
>> +#include "i915_scheduler.h"
>> +
>> +/**
>> + * i915_scheduler_is_enabled - Returns true if the scheduler is enabled.
>> + * @dev: DRM device
>> + */
>> +bool i915_scheduler_is_enabled(struct drm_device *dev)
>> +{
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> +
>> + return dev_priv->scheduler != NULL;
>> +}
> i915.enable_scheduler would be used instead.
It is when it arrives.
>
>> +
>> +/**
>> + * i915_scheduler_init - Initialise the scheduler.
>> + * @dev: DRM device
>> + * Returns zero on success or -ENOMEM if memory allocations fail.
>> + */
>> +int i915_scheduler_init(struct drm_device *dev)
>> +{
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + int r;
>> +
> This protection -->
>
>> + if (scheduler)
>> + return 0;
>> +
>> + scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
>> + if (!scheduler)
>> + return -ENOMEM;
>> +
> Is not needed if we have data member and i915.enable_scheduler.
Making the scheduler structure a object rather than a pointer basically
means moving the whole of i915_scheduler.h into i915_drv.h. There is too
much interconnectedness between the various structures. I would say that
keeping the code nicely partitioned and readable is worth the one
dynamic allocation at driver load time.
>
>> + spin_lock_init(&scheduler->lock);
>> +
>> + for (r = 0; r < I915_NUM_RINGS; r++)
>> + INIT_LIST_HEAD(&scheduler->node_queue[r]);
>> +
>> + /* Default tuning values: */
>> + scheduler->priority_level_min = -1023;
>> + scheduler->priority_level_max = 1023;
>> + scheduler->priority_level_bump = 50;
>> + scheduler->priority_level_preempt = 900;
>> + scheduler->min_flying = 2;
>> +
>> + dev_priv->scheduler = scheduler;
>> +
>> + return 0;
>> +}
>> +
>> +/*
>> + * Add a popped node back in to the queue. For example, because the ring was
>> + * hung when execfinal() was called and thus the ring submission needs to be
>> + * retried later.
>> + */
>> +static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
>> +{
>> + WARN_ON(!node);
> Rather remove this completely as it is an internal function, or use
> BUG_ON because next line will straight lead to OOPS after warning. I'll
> pass mentioning rest of the obvious WARN_ON vs BUG_ON's.
Yeah, most of the WARN/BUG_ONs are probably unnecessary now. They were
scattered liberally around during initial development when strange and
unexpected things could happen. Now the the code paths are pretty well
established they aren't really much use. I'll reduce them down to just
the ones that still make sense.
>
>> + WARN_ON(!I915_SQS_IS_FLYING(node));
>> +
>> + /* Seqno will be reassigned on relaunch */
>> + node->params.request->seqno = 0;
>> + node->status = i915_sqs_queued;
>> +}
>> +
>> +/*
>> + * Give up on a node completely. For example, because it is causing the
>> + * ring to hang or is using some resource that no longer exists.
>> + */
>> +static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
>> +{
>> + WARN_ON(!node);
>> + WARN_ON(I915_SQS_IS_COMPLETE(node));
>> +
>> + node->status = i915_sqs_dead;
>> +}
>> +
>> +/* Mark a node as in flight on the hardware. */
> As for documentation, from below, I only assume scheduler lock must be
> held and node's can be only manipulated when scheduler lock is held?
> It's good to add a comment describing the expected locking. Or maybe
> add assert_scheduler_lock_held() helper and sprinkle it around the
> functions similar to assert_rpm_wakelock_held(), which is self-
> documenting?
>
>> +static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
>> +{
>> + struct drm_i915_private *dev_priv = node->params.dev->dev_private;
> If node is NULL, this would already OOPS, I do not think it's
> unreasonable to expect node not to be NULL. The below WARN_ON check is
> never reached.
>
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + struct intel_engine_cs *ring;
>> +
>> + WARN_ON(!scheduler);
>> + WARN_ON(!node);
>> + WARN_ON(node->status != i915_sqs_popped);
>> +
>> + ring = node->params.ring;
>> +
> This and similar assignments can be squashed to declaration line, as
> this is basically an alias.
>
>> + /*
>> + * Add the node (which should currently be in state popped) to the
>> + * front of the queue. This ensure that flying nodes are always held
>> + * in hardware submission order.
>> + */
>> + list_add(&node->link, &scheduler->node_queue[ring->id]);
>> +
>> + node->status = i915_sqs_flying;
>> +
>> + if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
>> + bool success = true;
>> +
>> + success = ring->irq_get(ring);
>> + if (success)
>> + scheduler->flags[ring->id] |= i915_sf_interrupts_enabled;
>> + else
>> + return -EINVAL;
> Shouldn't the node be cleaned up from node_queue in case of an error?
Not really. The batch buffer will still be executed regardless of
whether interrupts could be enabled or not. The return code is actually
ignored so this should really be a void function, I guess. Without
interrupts then completion might take a while to be noticed but stuff
should still basically work.
>
> Also, I think above could be written much more compact form (because it
> looks like something where more logic will be added later). It makes it
> easier to write and visualize the error paths when reading if there are
> no nested if's.
>
> I won't mention about the error paths of functions below, I expect the
> following guidline to be adhered;
>
> if (scheduler->flags[ring->id] & I915_SF_INT_ENABLED)
> goto out;
>
> if (!ring->irq_get(ring))
> goto err_irq;
>
> if (!...)
> goto err_foo;
>
> scheduler->flags[ring->id] |= I915_SF_INT_ENABLED;
> out:
> return 0;
> err_foo:
> foobar();
> err_irq:
> list_remove()
> return -EINVAL;
>
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static uint32_t i915_scheduler_count_flying(struct i915_scheduler *scheduler,
>> + struct intel_engine_cs *ring)
>> +{
>> + struct i915_scheduler_queue_entry *node;
>> + uint32_t flying = 0;
>> +
>> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link)
> +1 for the for_each_scheduler_node(...)
>
>> + if (I915_SQS_IS_FLYING(node))
>> + flying++;
>> +
>> + return flying;
>> +}
>> +
>> +static void i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler)
>> +{
>> + struct i915_scheduler_queue_entry *node;
>> + int i;
>> +
>> + /*
>> + * Ensure circular dependencies don't cause problems and that a bump
>> + * by object usage only bumps each using buffer once:
>> + */
>> + for (i = 0; i < I915_NUM_RINGS; i++) {
>> + list_for_each_entry(node, &scheduler->node_queue[i], link)
>> + node->bumped = false;
>> + }
>> +}
>> +
>> +static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
>> + struct i915_scheduler_queue_entry *target,
> Is there specific reason why struct i915_scheduler_queue_entry are not
> referred to just as "node" but "qe" and here something else, do "node"
> and "qe" have a special semantic meaning?
Not sure what you mean. The use of 'node' is because almost everything
an i915_scheduler_queue_entry object is being used it is as a node in a
list that is being iterated over. The only 'qe' variable is in
'i915_scheduler_queue_execbuffer[_bypass]' and there it is because the
qe is being passed in from outside and is copied to a local 'node' that
is then added to the list. I think the use of 'target' here is largely
historical. The subsequent array search was originally a list search and
hence had a 'node' that was used as the iterator. Thus the qe passed in
was called something else to avoid conflicts and target seemed appropriate.
>> + uint32_t bump)
>> +{
>> + uint32_t new_priority;
>> + int i, count;
>> +
>> + if (target->priority >= scheduler->priority_level_max)
>> + return 1;
> So if one node reaches maximum priority the dependencies are expected
> to be maxed out already?
Yes. Dependencies should always be of equal or higher priority to the
dependee. When a new dependency is created, the priority of the dependee
is added on to ensure that something of high priority can never be stuck
waiting on something of low priority. Also, it means that a batch buffer
that lots of other buffers are waiting on will be bumped to higher and
higher priority the more waiters there are and thus is more likely to be
executed sooner and free them all up.
>> +
>> + if (target->bumped)
>> + return 0;
>> +
>> + new_priority = target->priority + bump;
>> + if ((new_priority <= target->priority) ||
>> + (new_priority > scheduler->priority_level_max))
>> + target->priority = scheduler->priority_level_max;
>> + else
>> + target->priority = new_priority;
>> +
>> + count = 1;
>> + target->bumped = true;
>> +
>> + for (i = 0; i < target->num_deps; i++) {
>> + if (!target->dep_list[i])
>> + continue;
>> +
>> + if (target->dep_list[i]->bumped)
>> + continue;
>> +
>> + count += i915_scheduler_priority_bump(scheduler,
>> + target->dep_list[i],
>> + bump);
>> + }
>> +
>> + return count;
>> +}
>> +
>> +/*
>> + * Nodes are considered valid dependencies if they are queued on any ring or
>> + * if they are in flight on a different ring. In flight on the same ring is no
>> + * longer interesting for non-premptive nodes as the ring serialises execution.
>> + * For pre-empting nodes, all in flight dependencies are valid as they must not
>> + * be jumped by the act of pre-empting.
>> + *
>> + * Anything that is neither queued nor flying is uninteresting.
>> + */
>> +static inline bool i915_scheduler_is_dependency_valid(
>> + struct i915_scheduler_queue_entry *node, uint32_t idx)
>> +{
>> + struct i915_scheduler_queue_entry *dep;
>> +
>> + dep = node->dep_list[idx];
>> + if (!dep)
>> + return false;
>> +
>> + if (I915_SQS_IS_QUEUED(dep))
>> + return true;
>> +
>> + if (I915_SQS_IS_FLYING(dep)) {
>> + if (node->params.ring != dep->params.ring)
>> + return true;
>> + }
>> +
>> + return false;
>> +}
>> +
>> +static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
>> + struct i915_scheduler_queue_entry **pop_node)
>> +{
>> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + struct i915_scheduler_queue_entry *best = NULL;
>> + struct i915_scheduler_queue_entry *node;
>> + int ret;
>> + int i;
>> + bool any_queued = false;
>> + bool has_local, has_remote, only_remote = false;
>> +
>> + *pop_node = NULL;
>> + ret = -ENODATA;
>> +
>> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
>> + if (!I915_SQS_IS_QUEUED(node))
>> + continue;
>> + any_queued = true;
>> +
>> + has_local = false;
>> + has_remote = false;
>> + for (i = 0; i < node->num_deps; i++) {
>> + if (!i915_scheduler_is_dependency_valid(node, i))
>> + continue;
>> +
>> + if (node->dep_list[i]->params.ring == node->params.ring)
>> + has_local = true;
>> + else
>> + has_remote = true;
>> + }
>> +
>> + if (has_remote && !has_local)
>> + only_remote = true;
>> +
>> + if (!has_local && !has_remote) {
>> + if (!best ||
>> + (node->priority > best->priority))
>> + best = node;
>> + }
>> + }
>> +
>> + if (best) {
>> + list_del(&best->link);
>> +
>> + INIT_LIST_HEAD(&best->link);
>> + best->status = i915_sqs_popped;
>> +
>> + ret = 0;
>> + } else {
>> + /* Can only get here if:
>> + * (a) there are no buffers in the queue
>> + * (b) all queued buffers are dependent on other buffers
>> + * e.g. on a buffer that is in flight on a different ring
>> + */
>> + if (only_remote) {
>> + /* The only dependent buffers are on another ring. */
>> + ret = -EAGAIN;
>> + } else if (any_queued) {
>> + /* It seems that something has gone horribly wrong! */
>> + DRM_ERROR("Broken dependency tracking on ring %d!\n",
>> + (int) ring->id);
> Would this condition be worthy WARN_ONCE(), because it will keep
> repeating as the situation will persist?
>
>> + }
>> + }
>> +
>> + *pop_node = best;
>> + return ret;
>> +}
>> +
>> +/*
>> + * NB: The driver mutex lock must be held before calling this function. It is
>> + * only really required during the actual back end submission call. However,
>> + * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
>> + * And releasing the one before acquiring the other leads to other code
>> + * being run and interfering.
>> + */
>> +static int i915_scheduler_submit(struct intel_engine_cs *ring)
>> +{
>> + struct drm_device *dev = ring->dev;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + struct i915_scheduler_queue_entry *node;
>> + int ret, count = 0, flying;
>> +
>> + WARN_ON(!mutex_is_locked(&dev->struct_mutex));
>> +
>> + spin_lock_irq(&scheduler->lock);
>> +
>> + WARN_ON(scheduler->flags[ring->id] & i915_sf_submitting);
>> + scheduler->flags[ring->id] |= i915_sf_submitting;
>> +
>> + /* First time around, complain if anything unexpected occurs: */
>> + ret = i915_scheduler_pop_from_queue_locked(ring, &node);
>> + if (ret)
>> + goto error;
>> +
>> + do {
>> + WARN_ON(!node);
>> + WARN_ON(node->params.ring != ring);
>> + WARN_ON(node->status != i915_sqs_popped);
>> + count++;
>> +
>> + /*
>> + * The call to pop above will have removed the node from the
>> + * list. So add it back in and mark it as in flight.
>> + */
>> + i915_scheduler_node_fly(node);
>> +
>> + spin_unlock_irq(&scheduler->lock);
>> + ret = dev_priv->gt.execbuf_final(&node->params);
>> + spin_lock_irq(&scheduler->lock);
>> +
>> + /*
>> + * Handle failed submission but first check that the
>> + * watchdog/reset code has not nuked the node while we
>> + * weren't looking:
>> + */
>> + if (ret && (node->status != i915_sqs_dead)) {
>> + bool requeue = true;
>> +
>> + /*
>> + * Oh dear! Either the node is broken or the ring is
>> + * busy. So need to kill the node or requeue it and try
>> + * again later as appropriate.
>> + */
>> +
>> + switch (-ret) {
>> + case ENODEV:
>> + case ENOENT:
>> + /* Fatal errors. Kill the node. */
>> + requeue = false;
>> + i915_scheduler_node_kill(node);
>> + break;
> These break indents still.
>
>> +
>> + case EAGAIN:
>> + case EBUSY:
>> + case EIO:
>> + case ENOMEM:
>> + case ERESTARTSYS:
>> + case EINTR:
>> + /* Supposedly recoverable errors. */
>> + break;
>> +
>> + default:
>> + /*
>> + * Assume the error is recoverable and hope
>> + * for the best.
>> + */
>> + MISSING_CASE(-ret);
>> + break;
>> + }
>> +
>> + if (requeue) {
>> + i915_scheduler_node_requeue(node);
>> + /*
>> + * No point spinning if the ring is currently
>> + * unavailable so just give up and come back
>> + * later.
>> + */
>> + break;
>> + }
>> + }
>> +
>> + /* Keep launching until the sky is sufficiently full. */
>> + flying = i915_scheduler_count_flying(scheduler, ring);
>> + if (flying >= scheduler->min_flying)
>> + break;
>> +
>> + /* Grab another node and go round again... */
>> + ret = i915_scheduler_pop_from_queue_locked(ring, &node);
>> + } while (ret == 0);
>> +
>> + /* Don't complain about not being able to submit extra entries */
>> + if (ret == -ENODATA)
>> + ret = 0;
>> +
>> + /*
>> + * Bump the priority of everything that was not submitted to prevent
>> + * starvation of low priority tasks by a spamming high priority task.
>> + */
> This, this calls for an I-G-T test to make sure we're effective.
Yeah, IGTs are under development.
>> + i915_scheduler_priority_bump_clear(scheduler);
>> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
>> + if (!I915_SQS_IS_QUEUED(node))
>> + continue;
>> +
>> + i915_scheduler_priority_bump(scheduler, node,
>> + scheduler->priority_level_bump);
>> + }
>> +
>> + /* On success, return the number of buffers submitted. */
>> + if (ret == 0)
>> + ret = count;
>> +
>> +error:
>> + scheduler->flags[ring->id] &= ~i915_sf_submitting;
>> + spin_unlock_irq(&scheduler->lock);
>> + return ret;
>> +}
>> +
>> +static void i915_generate_dependencies(struct i915_scheduler *scheduler,
>> + struct i915_scheduler_queue_entry *node,
>> + uint32_t ring)
>> +{
>> + struct i915_scheduler_obj_entry *this, *that;
>> + struct i915_scheduler_queue_entry *test;
>> + int i, j;
>> + bool found;
>> +
>> + list_for_each_entry(test, &scheduler->node_queue[ring], link) {
>> + if (I915_SQS_IS_COMPLETE(test))
>> + continue;
>> +
>> + /*
>> + * Batches on the same ring for the same
>> + * context must be kept in order.
>> + */
>> + found = (node->params.ctx == test->params.ctx) &&
>> + (node->params.ring == test->params.ring);
>> +
>> + /*
>> + * Batches working on the same objects must
>> + * be kept in order.
>> + */
>> + for (i = 0; (i < node->num_objs) && !found; i++) {
>> + this = node->saved_objects + i;
> node->objs and node->num_objs would be a better naming match. num_objs
> and saved_objects makes me think we have a bug here, indexing wrong
> array.
>
>> +
>> + for (j = 0; j < test->num_objs; j++) {
>> + that = test->saved_objects + j;
>> +
>> + if (this->obj != that->obj)
>> + continue;
>> +
>> + /* Only need to worry about writes */
>> + if (this->read_only && that->read_only)
>> + continue;
>> +
>> + found = true;
>> + break;
>> + }
>> + }
>> +
>> + if (found) {
>> + node->dep_list[node->num_deps] = test;
>> + node->num_deps++;
>> + }
>> + }
>> +}
>> +
>> +static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_entry *qe)
>> +{
>> + struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + int ret;
>> +
>> + scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
>> + ret = dev_priv->gt.execbuf_final(&qe->params);
>> + scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
>> +
> The above code makes me think of locking again, maybe document at top
> of this function.
No need to acquire the spinlock at this point. If the scheduler is in
bypass mode then there is no internal state to protect. The driver mutex
lock will be held because there is only one code path to get here which
is the execbuff IOCTL. The whole point of bypass mode is that it is a
single contiguous execution path from IOCTL entry all the way through to
hardware submission. A mutex check could be added but it seems
unnecessary given the limited calling scope.
>
>> + /*
>> + * Don't do any clean up on failure because the caller will
>> + * do it all anyway.
>> + */
>> + if (ret)
>> + return ret;
>> +
>> + /* Need to release any resources held by the node: */
>> + i915_scheduler_clean_node(qe);
>> +
>> + return 0;
>> +}
>> +
>> +/**
>> + * i915_scheduler_queue_execbuffer - Submit a batch buffer request to the
>> + * scheduler.
>> + * @qe: The batch buffer request to be queued.
>> + * The expectation is the qe passed in is a local stack variable. This
>> + * function will copy its contents into a freshly allocated list node. The
>> + * new node takes ownership of said contents so the original qe should simply
>> + * be discarded and not cleaned up (i.e. don't free memory it points to or
>> + * dereference objects it holds). The node is added to the scheduler's queue
>> + * and the batch buffer will be submitted to the hardware at some future
>> + * point in time (which may be immediately, before returning or may be quite
>> + * a lot later).
>> + */
>> +int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
>> +{
>> + struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + struct intel_engine_cs *ring = qe->params.ring;
>> + struct i915_scheduler_queue_entry *node;
>> + struct i915_scheduler_queue_entry *test;
>> + bool not_flying;
>> + int i, r;
>> + int incomplete = 0;
>> +
>> + WARN_ON(!scheduler);
>> +
>> + if (1/*!i915.enable_scheduler*/)
>> + return i915_scheduler_queue_execbuffer_bypass(qe);
>> +
> I'm thinking, should we branch already before calling a function named
> i915_scheduler_queue_execbuffer if scheduler is disabled?
That would require putting if(scheduler) logic in random points
throughout the driver. Doing it this way, the driver as a whole has a
single execution path which is via the scheduler. It is then up to the
scheduler itself to decide whether to run in bypass mode, simple
re-ordering mode, full pre-emption mode or any other mode we feel a need
to add.
>> + node = kmalloc(sizeof(*node), GFP_KERNEL);
>> + if (!node)
>> + return -ENOMEM;
>> +
>> + *node = *qe;
> Now I read the added documentation for the function, maybe we should at
> least zero out qe to avoid future problems?
The execbuff success path hands the qe contents over and pretty much
immediately drops all the way back out of the IOCTL. The failure path
does the full cleanup of de-allocating all the qe stuff it had
previously allocated. IMO there doesn't seem to be much need to zero out
the structure at this point.
>
>> + INIT_LIST_HEAD(&node->link);
>> + node->status = i915_sqs_queued;
>> + node->stamp = jiffies;
>> + i915_gem_request_reference(node->params.request);
>> +
>> + WARN_ON(node->params.request->scheduler_qe);
>> + node->params.request->scheduler_qe = node;
>> +
>> + /* Need to determine the number of incomplete entries in the list as
>> + * that will be the maximum size of the dependency list.
>> + *
>> + * Note that the allocation must not be made with the spinlock acquired
>> + * as kmalloc can sleep. However, the unlock/relock is safe because no
>> + * new entries can be queued up during the unlock as the i915 driver
>> + * mutex is still held. Entries could be removed from the list but that
>> + * just means the dep_list will be over-allocated which is fine.
>> + */
>> + spin_lock_irq(&scheduler->lock);
> -->
>
>> + for (r = 0; r < I915_NUM_RINGS; r++) {
>> + list_for_each_entry(test, &scheduler->node_queue[r], link) {
>> + if (I915_SQS_IS_COMPLETE(test))
>> + continue;
>> +
>> + incomplete++;
>> + }
>> + }
>> +
> <-- This counting structure, I still think it would be good idea to
> make it a static helper.
>
>> + /* Temporarily unlock to allocate memory: */
>> + spin_unlock_irq(&scheduler->lock);
>> + if (incomplete) {
>> + node->dep_list = kmalloc_array(incomplete,
>> + sizeof(*node->dep_list),
>> + GFP_KERNEL);
>> + if (!node->dep_list) {
>> + kfree(node);
>> + return -ENOMEM;
>> + }
>> + } else
>> + node->dep_list = NULL;
>> +
>> + spin_lock_irq(&scheduler->lock);
>> + node->num_deps = 0;
>> +
>> + if (node->dep_list) {
>> + for (r = 0; r < I915_NUM_RINGS; r++)
>> + i915_generate_dependencies(scheduler, node, r);
>> +
>> + WARN_ON(node->num_deps > incomplete);
>> + }
>> +
>> + node->priority = clamp(node->priority,
>> + scheduler->priority_level_min,
>> + scheduler->priority_level_max);
>> +
>> + if ((node->priority > 0) && node->num_deps) {
>> + i915_scheduler_priority_bump_clear(scheduler);
>> +
>> + for (i = 0; i < node->num_deps; i++)
>> + i915_scheduler_priority_bump(scheduler,
>> + node->dep_list[i], node->priority);
> If node is posted with maximum priority to begin with, wouldn't the
> priority propagation stop after first level dependencies due to the
> check in the beginning of priority_bump function?
The recursion will stop whenever it encounters a node already at maximum
priority. However, it is not starting from the newly submitted node but
from that node's dependencies. If they are already at max priority then
their dependencies must also be and so on down the chain and no further
processing is required. On the other hand, if they are not then they
will be bumped and their dependencies checked and so on down the chain.
>> +
>> + list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
>> +
>> + not_flying = i915_scheduler_count_flying(scheduler, ring) <
>> + scheduler->min_flying;
>> +
>> + spin_unlock_irq(&scheduler->lock);
>> +
>> + if (not_flying)
>> + i915_scheduler_submit(ring);
>> +
>> + return 0;
>> +}
>> +
>> +/**
>> + * i915_scheduler_notify_request - Notify the scheduler that the given
>> + * request has completed on the hardware.
>> + * @req: Request structure which has completed
>> + * @preempt: Did it complete pre-emptively?
>> + * A sequence number has popped out of the hardware and the request handling
>> + * code has mapped it back to a request and will mark that request complete.
>> + * It also calls this function to notify the scheduler about the completion
>> + * so the scheduler's node can be updated appropriately.
>> + * Returns true if the request is scheduler managed, false if not. The return
>> + * value is combined for all freshly completed requests and if any were true
>> + * then i915_scheduler_wakeup() is called so the scheduler can do further
>> + * processing (submit more work) at the end.
>> + */
>> +bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
>> +{
>> + struct drm_i915_private *dev_priv = to_i915(req->ring->dev);
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + struct i915_scheduler_queue_entry *node = req->scheduler_qe;
>> + unsigned long flags;
>> +
>> + if (!node)
>> + return false;
>> +
>> + spin_lock_irqsave(&scheduler->lock, flags);
>> +
>> + WARN_ON(!I915_SQS_IS_FLYING(node));
>> +
>> + /* Node was in flight so mark it as complete. */
>> + if (req->cancelled)
>> + node->status = i915_sqs_dead;
>> + else
>> + node->status = i915_sqs_complete;
>> +
>> + spin_unlock_irqrestore(&scheduler->lock, flags);
>> +
>> + return true;
>> +}
>> +
>> +static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
>> + struct i915_scheduler_queue_entry *remove)
>> +{
>> + struct i915_scheduler_queue_entry *node;
>> + int i, r;
>> + int count = 0;
>> +
>> + /*
>> + * Ensure that a node is not being removed which is still dependent
>> + * upon other (not completed) work. If that happens, it implies
>> + * something has gone very wrong with the dependency tracking! Note
>> + * that there is no need to worry if this node has been explicitly
>> + * killed for some reason - it might be being killed before it got
>> + * sent to the hardware.
>> + */
>> + if (remove->status != i915_sqs_dead) {
>> + for (i = 0; i < remove->num_deps; i++)
>> + if ((remove->dep_list[i]) &&
>> + (!I915_SQS_IS_COMPLETE(remove->dep_list[i])))
>> + count++;
>> + WARN_ON(count);
>> + }
>> +
>> + /*
>> + * Remove this node from the dependency lists of any other node which
>> + * might be waiting on it.
>> + */
>> + for (r = 0; r < I915_NUM_RINGS; r++) {
>> + list_for_each_entry(node, &scheduler->node_queue[r], link) {
>> + for (i = 0; i < node->num_deps; i++) {
>> + if (node->dep_list[i] != remove)
>> + continue;
>> +
>> + node->dep_list[i] = NULL;
>> + }
>> + }
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +/**
>> + * i915_scheduler_wakeup - wake the scheduler's worker thread
>> + * @dev: DRM device
>> + * Called at the end of seqno interrupt processing if any request has
>> + * completed that corresponds to a scheduler node.
>> + */
>> +void i915_scheduler_wakeup(struct drm_device *dev)
>> +{
>> + /* XXX: Need to call i915_scheduler_remove() via work handler. */
>> +}
>> +
>> +/**
>> + * i915_scheduler_clean_node - free up any allocations/references
>> + * associated with the given scheduler queue entry.
>> + * @node: Queue entry structure which is complete
>> + * After a give batch buffer completes on the hardware, all the information
>> + * required to resubmit it is no longer required. However, the node entry
>> + * itself might still be required for tracking purposes for a while longer.
>> + * This function should be called as soon as the node is known to be complete
>> + * so that these resources may be freed even though the node itself might
>> + * hang around.
>> + */
>> +void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
>> +{
>> + if (!I915_SQS_IS_COMPLETE(node)) {
>> + WARN(!node->params.request->cancelled,
>> + "Cleaning active node: %d!\n", node->status);
>> + return;
>> + }
>> +
>> + if (node->params.batch_obj) {
>> + /*
>> + * The batch buffer must be unpinned before it is unreferenced
>> + * otherwise the unpin fails with a missing vma!?
>> + */
>> + if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
>> + i915_gem_execbuff_release_batch_obj(node->params.batch_obj);
>> +
> This code seems little bit out of place.
Not sure where else you could put it. The code itself is just a copy of
the existing clean up code that was already in the execbuff code path.
The cleanup can no longer be done within the execbuff call due to the
batch buffer execution being arbitrarily delayed by the scheduler. Thus
it is down to the scheduler to do all the necessary cleanup when it
knows that the batch buffer is no longer required.
>
>> + node->params.batch_obj = NULL;
>> + }
>> +
>> + /* And anything else owned by the node: */
>> + if (node->params.cliprects) {
>> + kfree(node->params.cliprects);
>> + node->params.cliprects = NULL;
>> + }
>> +}
>> +
> The below function is quite hacky, I understood this will be mitigated
> by per-ctx sequence numbering in the future?
To be honest, I think it is already obsolete due to the previous work of
converting from seqno tests everywhere to testing request structures.
The point of the code below was to cope with seqno values popping out of
the hardware in random order due to the re-ordering of batch buffers
prior to execution. Whereas, the struct request conversion allowed the
seqno to be late allocated and thus kept in order.
Unfortunately, pre-emption confuses matters again. Although the
intention is that seqno values will still be kept in order. It's just
that a given request might have multiple different seqno values assigned
to it throughout its life. The pre-emption code hasn't quite settled
down yet though, especially with interesting new hardware planned for
future chips. Thus I don't really want to rip out all of this code quite
yet just in case we do still need it.
As you say, per-ctx seqnos would definitely have an effect here as well.
So yes, long term the intention is to remove all this nastyness. Short
term, it seems best to keep it around until proven unnecessary.
>
>> +static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
>> + struct intel_engine_cs *ring,
>> + struct list_head *remove)
>> +{
>> + struct i915_scheduler_queue_entry *node, *node_next;
>> + int flying = 0, queued = 0;
>> + bool do_submit;
>> + uint32_t min_seqno;
>> +
>> + spin_lock_irq(&scheduler->lock);
>> +
>> + /*
>> + * In the case where the system is idle, starting 'min_seqno' from a big
>> + * number will cause all nodes to be removed as they are now back to
>> + * being in-order. However, this will be a problem if the last one to
>> + * complete was actually out-of-order as the ring seqno value will be
>> + * lower than one or more completed buffers. Thus code looking for the
>> + * completion of said buffers will wait forever.
>> + * Instead, use the hardware seqno as the starting point. This means
>> + * that some buffers might be kept around even in a completely idle
>> + * system but it should guarantee that no-one ever gets confused when
>> + * waiting for buffer completion.
>> + */
>> + min_seqno = ring->get_seqno(ring, true);
>> +
>> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
>> + if (I915_SQS_IS_QUEUED(node))
>> + queued++;
>> + else if (I915_SQS_IS_FLYING(node))
>> + flying++;
>> + else if (I915_SQS_IS_COMPLETE(node))
>> + continue;
>> +
>> + if (node->params.request->seqno == 0)
>> + continue;
>> +
>> + if (!i915_seqno_passed(node->params.request->seqno, min_seqno))
>> + min_seqno = node->params.request->seqno;
>> + }
>> +
>> + INIT_LIST_HEAD(remove);
>> + list_for_each_entry_safe(node, node_next, &scheduler->node_queue[ring->id], link) {
>> + /*
>> + * Only remove completed nodes which have a lower seqno than
>> + * all pending nodes. While there is the possibility of the
>> + * ring's seqno counting backwards, all higher buffers must
>> + * be remembered so that the 'i915_seqno_passed()' test can
>> + * report that they have in fact passed.
>> + *
>> + * NB: This is not true for 'dead' nodes. The GPU reset causes
>> + * the software seqno to restart from its initial value. Thus
>> + * the dead nodes must be removed even though their seqno values
>> + * are potentially vastly greater than the current ring seqno.
>> + */
>> + if (!I915_SQS_IS_COMPLETE(node))
>> + continue;
>> +
>> + if (node->status != i915_sqs_dead) {
>> + if (i915_seqno_passed(node->params.request->seqno, min_seqno) &&
>> + (node->params.request->seqno != min_seqno))
>> + continue;
>> + }
>> +
>> + list_del(&node->link);
>> + list_add(&node->link, remove);
>> +
>> + /* Strip the dependency info while the mutex is still locked */
>> + i915_scheduler_remove_dependent(scheduler, node);
>> +
>> + continue;
>> + }
>> +
>> + /*
>> + * Release the interrupt reference count if there are no longer any
>> + * nodes to worry about.
>> + */
>> + if (!flying && !queued &&
>> + (scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
>> + ring->irq_put(ring);
>> + scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;
> Maybe have this piece of code as a helper? To enable interrupts and
> disable them, maybe even a reference count?
Not sure how much you could abstract out into a helper other than the
two lines of irq_get|put + flag set|clear. The flying|queued test is
particular to this one instance. Also, there is no need to reference
count as the irq code does that already.
>
>> + }
>> +
>> + /* Launch more packets now? */
>> + do_submit = (queued > 0) && (flying < scheduler->min_flying);
>> +
>> + spin_unlock_irq(&scheduler->lock);
>> +
>> + return do_submit;
>> +}
>> +
>> +void i915_scheduler_process_work(struct intel_engine_cs *ring)
>> +{
>> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + struct i915_scheduler_queue_entry *node;
>> + bool do_submit;
>> + struct list_head remove;
>> +
>> + if (list_empty(&scheduler->node_queue[ring->id]))
>> + return;
>> +
>> + /* Remove completed nodes. */
>> + do_submit = i915_scheduler_remove(scheduler, ring, &remove);
>> +
>> + if (!do_submit && list_empty(&remove))
>> + return;
>> +
>> + /* Need to grab the pm lock outside of the mutex lock */
>> + if (do_submit)
>> + intel_runtime_pm_get(dev_priv);
>> +
>> + mutex_lock(&ring->dev->struct_mutex);
>> +
>> + if (do_submit)
>> + i915_scheduler_submit(ring);
>> +
>> + while (!list_empty(&remove)) {
>> + node = list_first_entry(&remove, typeof(*node), link);
>> + list_del(&node->link);
>> +
>> + /* Free up all the DRM references */
>> + i915_scheduler_clean_node(node);
>> +
>> + /* And anything else owned by the node: */
>> + node->params.request->scheduler_qe = NULL;
>> + i915_gem_request_unreference(node->params.request);
>> + kfree(node->dep_list);
>> + kfree(node);
> Maybe have a separate helper for freeing a node, would make this much
> cleaner.
There is a helper for freeing up the contents of the node - see
i915_scheduler_clean_node() above. However, that function is called from
multiple places and this is the only instance where the node itself is
actually freed. The other callers either need to keep the node around
for a while longer (i.e. until this piece of code can run) or don't own
the node because it is stack allocated (in the bypass case). So the
'anything else' section cannot be moved into the existing clean function
and any secondary helper would be single usage and only four lines long.
It doesn't seem worth it.
>
>> + }
>> +
>> + mutex_unlock(&ring->dev->struct_mutex);
>> +
>> + if (do_submit)
>> + intel_runtime_pm_put(dev_priv);
>> +}
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
>> new file mode 100644
>> index 0000000..415fec8
>> --- /dev/null
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
>> @@ -0,0 +1,95 @@
>> +/*
>> + * Copyright (c) 2014 Intel Corporation
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a
>> + * copy of this software and associated documentation files (the "Software"),
>> + * to deal in the Software without restriction, including without limitation
>> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
>> + * and/or sell copies of the Software, and to permit persons to whom the
>> + * Software is furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice (including the next
>> + * paragraph) shall be included in all copies or substantial portions of the
>> + * Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
>> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
>> + * IN THE SOFTWARE.
>> + *
>> + */
>> +
>> +#ifndef _I915_SCHEDULER_H_
>> +#define _I915_SCHEDULER_H_
>> +
>> +enum i915_scheduler_queue_status {
>> + /* Limbo: */
>> + i915_sqs_none = 0,
>> + /* Not yet submitted to hardware: */
>> + i915_sqs_queued,
>> + /* Popped from queue, ready to fly: */
>> + i915_sqs_popped,
>> + /* Sent to hardware for processing: */
>> + i915_sqs_flying,
>> + /* Finished processing on the hardware: */
>> + i915_sqs_complete,
>> + /* Killed by watchdog or catastrophic submission failure: */
>> + i915_sqs_dead,
>> + /* Limit value for use with arrays/loops */
>> + i915_sqs_MAX
>> +};
>> +
> To uppercase.
Grrr. Linux kernel style guide is daft.
>
>> +#define I915_SQS_IS_QUEUED(node) (((node)->status == i915_sqs_queued))
>> +#define I915_SQS_IS_FLYING(node) (((node)->status == i915_sqs_flying))
>> +#define I915_SQS_IS_COMPLETE(node) (((node)->status == i915_sqs_complete) || \
>> + ((node)->status == i915_sqs_dead))
>> +
>> +struct i915_scheduler_obj_entry {
>> + struct drm_i915_gem_object *obj;
>> + bool read_only;
>> +};
>> +
>> +struct i915_scheduler_queue_entry {
>> + struct i915_execbuffer_params params;
>> + /* -1023 = lowest priority, 0 = default, 1023 = highest */
>> + int32_t priority;
>> + struct i915_scheduler_obj_entry *saved_objects;
>> + int num_objs;
>> + bool bumped;
>> + struct i915_scheduler_queue_entry **dep_list;
>> + int num_deps;
>> + enum i915_scheduler_queue_status status;
>> + unsigned long stamp;
>> + struct list_head link;
> Above whitespace is still incorrect and I could really use comments.
>
>> +};
>> +
>> +struct i915_scheduler {
>> + struct list_head node_queue[I915_NUM_RINGS];
>> + uint32_t flags[I915_NUM_RINGS];
>> + spinlock_t lock;
>> +
>> + /* Tuning parameters: */
>> + int32_t priority_level_min;
>> + int32_t priority_level_max;
>> + int32_t priority_level_bump;
>> + int32_t priority_level_preempt;
>> + uint32_t min_flying;
>> +};
>> +
>> +/* Flag bits for i915_scheduler::flags */
>> +enum {
>> + i915_sf_interrupts_enabled = (1 << 0),
>> + i915_sf_submitting = (1 << 1),
>> +};
> These must be I915_UPPERCASE_SOMETHING by convention.
>
>> +
>> +bool i915_scheduler_is_enabled(struct drm_device *dev);
>> +int i915_scheduler_init(struct drm_device *dev);
>> +void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
>> +int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
>> +bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
>> +void i915_scheduler_wakeup(struct drm_device *dev);
>> +
>> +#endif /* _I915_SCHEDULER_H_ */
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 07/35] drm/i915: Prepare retire_requests to handle out-of-order seqnos
2016-02-18 14:26 ` [PATCH v5 07/35] drm/i915: Prepare retire_requests to handle out-of-order seqnos John.C.Harrison
@ 2016-02-19 19:23 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-19 19:23 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:26 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> A major point of the GPU scheduler is that it re-orders batch buffers
> after they have been submitted to the driver. This leads to requests
> completing out of order. In turn, this means that the retire
> processing can no longer assume that all completed entries are at the
> front of the list. Rather than attempting to re-order the request list
> on a regular basis, it is better to simply scan the entire list.
>
> v2: Removed deferred free code as no longer necessary due to request
> handling updates.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem.c | 31 +++++++++++++------------------
> 1 file changed, 13 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 7d9aa24..0003cfc 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -3233,6 +3233,7 @@ void i915_gem_reset(struct drm_device *dev)
> void
> i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
> {
> + struct drm_i915_gem_object *obj, *obj_next;
> struct drm_i915_gem_request *req, *req_next;
> LIST_HEAD(list_head);
>
> @@ -3245,37 +3246,31 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
> */
> i915_gem_request_notify(ring, false);
>
> + /*
> + * Note that request entries might be out of order due to rescheduling
> + * and pre-emption. Thus both lists must be processed in their entirety
> + * rather than stopping at the first non-complete entry.
> + */
> +
> /* Retire requests first as we use it above for the early return.
> * If we retire requests last, we may use a later seqno and so clear
> * the requests lists without clearing the active list, leading to
> * confusion.
> */
> - while (!list_empty(&ring->request_list)) {
> - struct drm_i915_gem_request *request;
> -
> - request = list_first_entry(&ring->request_list,
> - struct drm_i915_gem_request,
> - list);
> -
> - if (!i915_gem_request_completed(request))
> - break;
> + list_for_each_entry_safe(req, req_next, &ring->request_list, list) {
> + if (!i915_gem_request_completed(req))
> + continue;
>
> - i915_gem_request_retire(request);
> + i915_gem_request_retire(req);
> }
>
> /* Move any buffers on the active list that are no longer referenced
> * by the ringbuffer to the flushing/inactive lists as appropriate,
> * before we free the context associated with the requests.
> */
> - while (!list_empty(&ring->active_list)) {
> - struct drm_i915_gem_object *obj;
> -
> - obj = list_first_entry(&ring->active_list,
> - struct drm_i915_gem_object,
> - ring_list[ring->id]);
> -
> + list_for_each_entry_safe(obj, obj_next, &ring->active_list, ring_list[ring->id]) {
> if (!list_empty(&obj->last_read_req[ring->id]->list))
> - break;
> + continue;
>
> i915_gem_object_retire__read(obj, ring->id);
> }
>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 08/35] drm/i915: Disable hardware semaphores when GPU scheduler is enabled
2016-02-18 14:26 ` [PATCH v5 08/35] drm/i915: Disable hardware semaphores when GPU scheduler is enabled John.C.Harrison
@ 2016-02-19 19:27 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-19 19:27 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:26 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> Hardware sempahores require seqno values to be continuously
> incrementing. However, the scheduler's reordering of batch buffers
> means that the seqno values going through the hardware could be out of
> order. Thus semaphores can not be used.
>
> On the other hand, the scheduler superceeds the need for hardware
> semaphores anyway. Having one ring stall waiting for something to
> complete on another ring is inefficient if that ring could be working
> on some other, independent task. This is what the scheduler is meant
> to do - keep the hardware as busy as possible by reordering batch
> buffers to avoid dependency stalls.
>
> v4: Downgraded a BUG_ON to WARN_ON as the latter is preferred.
>
> v5: Squashed the i915_scheduler.c portions down into the 'start of
> scheduler' patch. [Joonas Lahtinen]
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/i915_drv.c | 9 +++++++++
> drivers/gpu/drm/i915/intel_ringbuffer.c | 4 ++++
> 2 files changed, 13 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
> index 975af35..5760a17 100644
> --- a/drivers/gpu/drm/i915/i915_drv.c
> +++ b/drivers/gpu/drm/i915/i915_drv.c
> @@ -34,6 +34,7 @@
> #include "i915_drv.h"
> #include "i915_trace.h"
> #include "intel_drv.h"
> +#include "i915_scheduler.h"
>
> #include <linux/console.h>
> #include <linux/module.h>
> @@ -517,6 +518,14 @@ void intel_detect_pch(struct drm_device *dev)
>
> bool i915_semaphore_is_enabled(struct drm_device *dev)
> {
> + /* Hardware semaphores are not compatible with the scheduler due to the
> + * seqno values being potentially out of order. However, semaphores are
> + * also not required as the scheduler will handle interring dependencies
> + * and try do so in a way that does not cause dead time on the hardware.
> + */
> + if (i915_scheduler_is_enabled(dev))
> + return false;
> +
> if (INTEL_INFO(dev)->gen < 6)
> return false;
>
> diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
> index 9d4f19d..ca7b8af 100644
> --- a/drivers/gpu/drm/i915/intel_ringbuffer.c
> +++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
> @@ -33,6 +33,7 @@
> #include <drm/i915_drm.h>
> #include "i915_trace.h"
> #include "intel_drv.h"
> +#include "i915_scheduler.h"
>
> int __intel_ring_space(int head, int tail, int size)
> {
> @@ -1400,6 +1401,9 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,
> u32 wait_mbox = signaller->semaphore.mbox.wait[waiter->id];
> int ret;
>
> + /* Arithmetic on sequence numbers is unreliable with a scheduler. */
> + WARN_ON(i915_scheduler_is_enabled(signaller->dev));
> +
> /* Throughout all of the GEM code, seqno passed implies our current
> * seqno is >= the last seqno executed. However for hardware the
> * comparison is strictly greater than.
>
I'd rather get rid of this altogether, but I guess we'll need it for the older gens. Another option would be to make the sync_to hook NULL in the scheduler case, though I guess the failure mode is less desirable there.
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
2016-02-18 14:26 ` [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled John.C.Harrison
@ 2016-02-19 19:28 ` Jesse Barnes
2016-02-19 19:53 ` Ville Syrjälä
2016-02-20 9:22 ` Chris Wilson
0 siblings, 2 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-19 19:28 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX, Lankhorst, Maarten, Syrjala, Ville
On 02/18/2016 06:26 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> MMIO flips are the preferred mechanism now but more importantly, pipe
> based flips cause issues for the scheduler. Specifically, submitting
> work to the rings around the side of the scheduler could cause that
> work to be lost if the scheduler generates a pre-emption event on that
> ring.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/intel_display.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
> index 6e12ed7..731d20a 100644
> --- a/drivers/gpu/drm/i915/intel_display.c
> +++ b/drivers/gpu/drm/i915/intel_display.c
> @@ -46,6 +46,7 @@
> #include <linux/dma_remapping.h>
> #include <linux/reservation.h>
> #include <linux/dma-buf.h>
> +#include "i915_scheduler.h"
>
> /* Primary plane formats for gen <= 3 */
> static const uint32_t i8xx_primary_formats[] = {
> @@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
> return true;
> else if (i915.enable_execlists)
> return true;
> + else if (i915_scheduler_is_enabled(ring->dev))
> + return true;
> else if (obj->base.dma_buf &&
> !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
> false))
>
Why can't we use mmio flips unconditionally? Maarten or Ville?
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 13/35] drm/i915: Redirect execbuffer_final() via scheduler
2016-02-18 14:27 ` [PATCH v5 13/35] drm/i915: Redirect execbuffer_final() via scheduler John.C.Harrison
@ 2016-02-19 19:33 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-19 19:33 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> Updated the execbuffer() code to pass the packaged up batch buffer
> information to the scheduler rather than calling execbuffer_final()
> directly. The scheduler queue() code is currently a stub which simply
> chains on to _final() immediately.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 18 +++++++-----------
> drivers/gpu/drm/i915/intel_lrc.c | 12 ++++--------
> 2 files changed, 11 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> index 7978dae..09c5ce9 100644
> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> @@ -33,6 +33,7 @@
> #include "intel_drv.h"
> #include <linux/dma_remapping.h>
> #include <linux/uaccess.h>
> +#include "i915_scheduler.h"
>
> #define __EXEC_OBJECT_HAS_PIN (1<<31)
> #define __EXEC_OBJECT_HAS_FENCE (1<<30)
> @@ -1226,6 +1227,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
> struct drm_i915_gem_execbuffer2 *args,
> struct list_head *vmas)
> {
> + struct i915_scheduler_queue_entry *qe;
> struct drm_device *dev = params->dev;
> struct intel_engine_cs *ring = params->ring;
> struct drm_i915_private *dev_priv = dev->dev_private;
> @@ -1270,17 +1272,11 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
>
> i915_gem_execbuffer_move_to_active(vmas, params->request);
>
> - ret = dev_priv->gt.execbuf_final(params);
> + qe = container_of(params, typeof(*qe), params);
> + ret = i915_scheduler_queue_execbuffer(qe);
> if (ret)
> return ret;
>
> - /*
> - * Free everything that was stored in the QE structure (until the
> - * scheduler arrives and does it instead):
> - */
> - if (params->dispatch_flags & I915_DISPATCH_SECURE)
> - i915_gem_execbuff_release_batch_obj(params->batch_obj);
> -
> return 0;
> }
>
> @@ -1420,8 +1416,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
> struct intel_engine_cs *ring;
> struct intel_context *ctx;
> struct i915_address_space *vm;
> - struct i915_execbuffer_params params_master; /* XXX: will be removed later */
> - struct i915_execbuffer_params *params = ¶ms_master;
> + struct i915_scheduler_queue_entry qe;
> + struct i915_execbuffer_params *params = &qe.params;
> const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
> u32 dispatch_flags;
> int ret;
> @@ -1529,7 +1525,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
> else
> vm = &dev_priv->gtt.base;
>
> - memset(¶ms_master, 0x00, sizeof(params_master));
> + memset(&qe, 0x00, sizeof(qe));
>
> eb = eb_create(args);
> if (eb == NULL) {
> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> index 12e8949..ff4565f 100644
> --- a/drivers/gpu/drm/i915/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/intel_lrc.c
> @@ -136,6 +136,7 @@
> #include <drm/i915_drm.h>
> #include "i915_drv.h"
> #include "intel_mocs.h"
> +#include "i915_scheduler.h"
>
> #define GEN9_LR_CONTEXT_RENDER_SIZE (22 * PAGE_SIZE)
> #define GEN8_LR_CONTEXT_RENDER_SIZE (20 * PAGE_SIZE)
> @@ -910,6 +911,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
> struct drm_i915_gem_execbuffer2 *args,
> struct list_head *vmas)
> {
> + struct i915_scheduler_queue_entry *qe;
> struct drm_device *dev = params->dev;
> struct intel_engine_cs *ring = params->ring;
> struct drm_i915_private *dev_priv = dev->dev_private;
> @@ -952,17 +954,11 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
>
> i915_gem_execbuffer_move_to_active(vmas, params->request);
>
> - ret = dev_priv->gt.execbuf_final(params);
> + qe = container_of(params, typeof(*qe), params);
> + ret = i915_scheduler_queue_execbuffer(qe);
> if (ret)
> return ret;
>
> - /*
> - * Free everything that was stored in the QE structure (until the
> - * scheduler arrives and does it instead):
> - */
> - if (params->dispatch_flags & I915_DISPATCH_SECURE)
> - i915_gem_execbuff_release_batch_obj(params->batch_obj);
> -
> return 0;
> }
>
>
I think this is ok, but might need changes if some of the earlier patches see changes due to Joonas's reviews.
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 14/35] drm/i915: Keep the reserved space mechanism happy
2016-02-18 14:27 ` [PATCH v5 14/35] drm/i915: Keep the reserved space mechanism happy John.C.Harrison
@ 2016-02-19 19:36 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-19 19:36 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> Ring space is reserved when constructing a request to ensure that the
> subsequent 'add_request()' call cannot fail due to waiting for space
> on a busy or broken GPU. However, the scheduler jumps in to the middle
> of the execbuffer process between request creation and request
> submission. Thus it needs to cancel the reserved space when the
> request is simply added to the scheduler's queue and not yet
> submitted. Similarly, it needs to re-reserve the space when it finally
> does want to send the batch buffer to the hardware.
>
> v3: Updated to use locally cached request pointer.
>
> v5: Updated due to changes to earlier patches in series - for runtime
> PM calls and splitting bypass mode into a separate function.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 20 ++++++++++++++------
> drivers/gpu/drm/i915/i915_scheduler.c | 4 ++++
> drivers/gpu/drm/i915/intel_lrc.c | 13 +++++++++++--
> 3 files changed, 29 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> index 09c5ce9..11bea8d 100644
> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> @@ -1295,18 +1295,22 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
> /* The mutex must be acquired before calling this function */
> WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
>
> + ret = intel_ring_reserve_space(req);
> + if (ret)
> + goto error;
> +
> /*
> * Unconditionally invalidate gpu caches and ensure that we do flush
> * any residual writes from the previous batch.
> */
> ret = intel_ring_invalidate_all_caches(req);
> if (ret)
> - return ret;
> + goto error;
>
> /* Switch to the correct context for the batch */
> ret = i915_switch_context(req);
> if (ret)
> - return ret;
> + goto error;
>
> WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
> "%s didn't clear reload\n", ring->name);
> @@ -1315,7 +1319,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
> params->instp_mode != dev_priv->relative_constants_mode) {
> ret = intel_ring_begin(req, 4);
> if (ret)
> - return ret;
> + goto error;
>
> intel_ring_emit(ring, MI_NOOP);
> intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
> @@ -1329,7 +1333,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
> if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
> ret = i915_reset_gen7_sol_offsets(params->dev, req);
> if (ret)
> - return ret;
> + goto error;
> }
>
> exec_len = params->args_batch_len;
> @@ -1343,13 +1347,17 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
> exec_start, exec_len,
> params->dispatch_flags);
> if (ret)
> - return ret;
> + goto error;
>
> trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
>
> i915_gem_execbuffer_retire_commands(params);
>
> - return 0;
> +error:
> + if (ret)
> + intel_ring_reserved_space_cancel(req->ringbuf);
> +
> + return ret;
> }
>
> /**
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 3986890..a3ffd04 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -483,6 +483,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
> struct i915_scheduler *scheduler = dev_priv->scheduler;
> int ret;
>
> + intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
> +
> scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
> ret = dev_priv->gt.execbuf_final(&qe->params);
> scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
> @@ -539,6 +541,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
> node->stamp = jiffies;
> i915_gem_request_reference(node->params.request);
>
> + intel_ring_reserved_space_cancel(node->params.request->ringbuf);
> +
> WARN_ON(node->params.request->scheduler_qe);
> node->params.request->scheduler_qe = node;
>
> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> index ff4565f..f4bab82 100644
> --- a/drivers/gpu/drm/i915/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/intel_lrc.c
> @@ -978,13 +978,17 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
> /* The mutex must be acquired before calling this function */
> WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
>
> + ret = intel_logical_ring_reserve_space(req);
> + if (ret)
> + goto err;
> +
> /*
> * Unconditionally invalidate gpu caches and ensure that we do flush
> * any residual writes from the previous batch.
> */
> ret = logical_ring_invalidate_all_caches(req);
> if (ret)
> - return ret;
> + goto err;
>
> if (ring == &dev_priv->ring[RCS] &&
> params->instp_mode != dev_priv->relative_constants_mode) {
> @@ -1006,13 +1010,18 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
>
> ret = ring->emit_bb_start(req, exec_start, params->dispatch_flags);
> if (ret)
> - return ret;
> + goto err;
>
> trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
>
> i915_gem_execbuffer_retire_commands(params);
>
> return 0;
> +
> +err:
> + intel_ring_reserved_space_cancel(params->request->ringbuf);
> +
> + return ret;
> }
>
> void intel_execlists_retire_requests(struct intel_engine_cs *ring)
>
I had to double check that it was ok to cancel space if the reserve failed (I guess it is), so seems ok.
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 15/35] drm/i915: Added tracking/locking of batch buffer objects
2016-02-18 14:27 ` [PATCH v5 15/35] drm/i915: Added tracking/locking of batch buffer objects John.C.Harrison
@ 2016-02-19 19:42 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-19 19:42 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> The scheduler needs to track interdependencies between batch buffers.
> These are calculated by analysing the object lists of the buffers and
> looking for commonality. The scheduler also needs to keep those
> buffers locked long after the initial IOCTL call has returned to user
> land.
>
> v3: Updated to support read-read optimisation.
>
> v5: Updated due to changes to earlier patches in series for splitting
> bypass mode into a separate function and consoliding the clean up code.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 48 ++++++++++++++++++++++++++++--
> drivers/gpu/drm/i915/i915_scheduler.c | 15 ++++++++++
> 2 files changed, 61 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> index 11bea8d..f45f4dc 100644
> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> @@ -1428,7 +1428,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
> struct i915_execbuffer_params *params = &qe.params;
> const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
> u32 dispatch_flags;
> - int ret;
> + int ret, i;
> bool need_relocs;
>
> if (!i915_gem_check_execbuffer(args))
> @@ -1543,6 +1543,14 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
> goto pre_mutex_err;
> }
>
> + qe.saved_objects = kzalloc(
> + sizeof(*qe.saved_objects) * args->buffer_count,
> + GFP_KERNEL);
> + if (!qe.saved_objects) {
> + ret = -ENOMEM;
> + goto err;
> + }
> +
> /* Look up object handles */
> ret = eb_lookup_vmas(eb, exec, args, vm, file);
> if (ret)
> @@ -1663,7 +1671,30 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
> params->args_DR1 = args->DR1;
> params->args_DR4 = args->DR4;
> params->batch_obj = batch_obj;
> - params->ctx = ctx;
> +
> + /*
> + * Save away the list of objects used by this batch buffer for the
> + * purpose of tracking inter-buffer dependencies.
> + */
> + for (i = 0; i < args->buffer_count; i++) {
> + struct drm_i915_gem_object *obj;
> +
> + /*
> + * NB: 'drm_gem_object_lookup()' increments the object's
> + * reference count and so must be matched by a
> + * 'drm_gem_object_unreference' call.
> + */
> + obj = to_intel_bo(drm_gem_object_lookup(dev, file,
> + exec[i].handle));
> + qe.saved_objects[i].obj = obj;
> + qe.saved_objects[i].read_only = obj->base.pending_write_domain == 0;
> +
> + }
> + qe.num_objs = i;
> +
> + /* Lock and save the context object as well. */
> + i915_gem_context_reference(ctx);
> + params->ctx = ctx;
>
> ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
> if (ret)
> @@ -1696,6 +1727,19 @@ err:
> i915_gem_context_unreference(ctx);
> eb_destroy(eb);
>
> + /* Need to release the objects: */
> + if (qe.saved_objects) {
> + for (i = 0; i < qe.num_objs; i++)
> + drm_gem_object_unreference(
> + &qe.saved_objects[i].obj->base);
> +
> + kfree(qe.saved_objects);
> + }
> +
> + /* Context too */
> + if (params->ctx)
> + i915_gem_context_unreference(params->ctx);
> +
> /*
> * If the request was created but not successfully submitted then it
> * must be freed again. If it was submitted then it is being tracked
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index a3ffd04..60a59d3 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -719,6 +719,8 @@ void i915_scheduler_wakeup(struct drm_device *dev)
> */
> void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
> {
> + int i;
> +
> if (!I915_SQS_IS_COMPLETE(node)) {
> WARN(!node->params.request->cancelled,
> "Cleaning active node: %d!\n", node->status);
> @@ -736,6 +738,19 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
> node->params.batch_obj = NULL;
> }
>
> + /* Release the locked buffers: */
> + for (i = 0; i < node->num_objs; i++)
> + drm_gem_object_unreference(&node->saved_objects[i].obj->base);
> + kfree(node->saved_objects);
> + node->saved_objects = NULL;
> + node->num_objs = 0;
> +
> + /* Context too: */
> + if (node->params.ctx) {
> + i915_gem_context_unreference(node->params.ctx);
> + node->params.ctx = NULL;
> + }
> +
> /* And anything else owned by the node: */
> if (node->params.cliprects) {
> kfree(node->params.cliprects);
>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 16/35] drm/i915: Hook scheduler node clean up into retire requests
2016-02-18 14:27 ` [PATCH v5 16/35] drm/i915: Hook scheduler node clean up into retire requests John.C.Harrison
@ 2016-02-19 19:44 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-19 19:44 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> The scheduler keeps its own lock on various DRM objects in order to
> guarantee safe access long after the original execbuff IOCTL has
execbuf is getting bigger, but I'm not sure if it qualifies as "buff" yet. Intentional misspelling? :)
> completed. This is especially important when pre-emption is enabled as
> the batch buffer might need to be submitted to the hardware multiple
> times. This patch hooks the clean up of these locks into the request
> retire function. The request can only be retired after it has
> completed on the hardware and thus is no longer eligible for
> re-submission. Thus there is no point holding on to the locks beyond
> that time.
>
> v3: Updated to not WARN when cleaning a node that is being cancelled.
> The clean will happen later so skipping it at the point of
> cancellation is fine.
>
> v5: Squashed the i915_scheduler.c portions down into the 'start of
> scheduler' patch. [Joonas Lahtinen]
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 1ab7256..2dd9b55 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -1489,6 +1489,9 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
> fence_signal_locked(&request->fence);
> }
>
> + if (request->scheduler_qe)
> + i915_scheduler_clean_node(request->scheduler_qe);
> +
> i915_gem_request_unreference(request);
> }
>
>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 18/35] drm/i915: Added scheduler support to page fault handler
2016-02-18 14:27 ` [PATCH v5 18/35] drm/i915: Added scheduler support to page fault handler John.C.Harrison
@ 2016-02-19 19:45 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-19 19:45 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> GPU page faults can now require scheduler operation in order to
> complete. For example, in order to free up sufficient memory to handle
> the fault the handler must wait for a batch buffer to complete that
> has not even been sent to the hardware yet. Thus EAGAIN no longer
> means a GPU hang, it can occur under normal operation.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem.c | 9 +++++++--
> 1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 17b44b3..a47a495 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -2003,10 +2003,15 @@ out:
> }
> case -EAGAIN:
> /*
> - * EAGAIN means the gpu is hung and we'll wait for the error
> - * handler to reset everything when re-faulting in
> + * EAGAIN can mean the gpu is hung and we'll have to wait for
> + * the error handler to reset everything when re-faulting in
> * i915_mutex_lock_interruptible.
> + *
> + * It can also indicate various other nonfatal errors for which
> + * the best response is to give other threads a chance to run,
> + * and then retry the failing operation in its entirety.
> */
> + /*FALLTHRU*/
> case 0:
> case -ERESTARTSYS:
> case -EINTR:
>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
2016-02-19 19:28 ` Jesse Barnes
@ 2016-02-19 19:53 ` Ville Syrjälä
2016-02-19 20:01 ` Jesse Barnes
2016-02-20 9:22 ` Chris Wilson
1 sibling, 1 reply; 82+ messages in thread
From: Ville Syrjälä @ 2016-02-19 19:53 UTC (permalink / raw)
To: Jesse Barnes; +Cc: Intel-GFX, Syrjala, Ville, Lankhorst, Maarten
On Fri, Feb 19, 2016 at 11:28:05AM -0800, Jesse Barnes wrote:
> On 02/18/2016 06:26 AM, John.C.Harrison@Intel.com wrote:
> > From: John Harrison <John.C.Harrison@Intel.com>
> >
> > MMIO flips are the preferred mechanism now but more importantly, pipe
> > based flips cause issues for the scheduler. Specifically, submitting
> > work to the rings around the side of the scheduler could cause that
> > work to be lost if the scheduler generates a pre-emption event on that
> > ring.
> >
> > For: VIZ-1587
> > Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> > ---
> > drivers/gpu/drm/i915/intel_display.c | 3 +++
> > 1 file changed, 3 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
> > index 6e12ed7..731d20a 100644
> > --- a/drivers/gpu/drm/i915/intel_display.c
> > +++ b/drivers/gpu/drm/i915/intel_display.c
> > @@ -46,6 +46,7 @@
> > #include <linux/dma_remapping.h>
> > #include <linux/reservation.h>
> > #include <linux/dma-buf.h>
> > +#include "i915_scheduler.h"
> >
> > /* Primary plane formats for gen <= 3 */
> > static const uint32_t i8xx_primary_formats[] = {
> > @@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
> > return true;
> > else if (i915.enable_execlists)
> > return true;
> > + else if (i915_scheduler_is_enabled(ring->dev))
> > + return true;
> > else if (obj->base.dma_buf &&
> > !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
> > false))
> >
>
> Why can't we use mmio flips unconditionally? Maarten or Ville?
We do when execlists are used, which is always on gen9+. So I guess I'm
missing the point of this patch. For gen5+ we could also do it trivially.
For older platforms it'd require a bit of work since we'd need to
complete the flips from the vblank interrupt. Well, we actually do
that already even with CS flips on those platforms, but we do look
at the flip pending interrupt to figure out if CS already issued
the flip or not. So that part would need changing.
I also think we should switch to using the vblank interrupt for this
stuff on all platforms, mainly since the flip done interrupt is somewhat
broken on at least BDW (no idea if it got fixed in SKL or later), and
doing things in more than one way certainly won't decrease our bug count.
--
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
2016-02-19 19:53 ` Ville Syrjälä
@ 2016-02-19 20:01 ` Jesse Barnes
2016-02-22 9:41 ` Lankhorst, Maarten
0 siblings, 1 reply; 82+ messages in thread
From: Jesse Barnes @ 2016-02-19 20:01 UTC (permalink / raw)
To: Ville Syrjälä; +Cc: Intel-GFX, Syrjala, Ville, Lankhorst, Maarten
On 02/19/2016 11:53 AM, Ville Syrjälä wrote:
> On Fri, Feb 19, 2016 at 11:28:05AM -0800, Jesse Barnes wrote:
>> On 02/18/2016 06:26 AM, John.C.Harrison@Intel.com wrote:
>>> From: John Harrison <John.C.Harrison@Intel.com>
>>>
>>> MMIO flips are the preferred mechanism now but more importantly, pipe
>>> based flips cause issues for the scheduler. Specifically, submitting
>>> work to the rings around the side of the scheduler could cause that
>>> work to be lost if the scheduler generates a pre-emption event on that
>>> ring.
>>>
>>> For: VIZ-1587
>>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>>> ---
>>> drivers/gpu/drm/i915/intel_display.c | 3 +++
>>> 1 file changed, 3 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
>>> index 6e12ed7..731d20a 100644
>>> --- a/drivers/gpu/drm/i915/intel_display.c
>>> +++ b/drivers/gpu/drm/i915/intel_display.c
>>> @@ -46,6 +46,7 @@
>>> #include <linux/dma_remapping.h>
>>> #include <linux/reservation.h>
>>> #include <linux/dma-buf.h>
>>> +#include "i915_scheduler.h"
>>>
>>> /* Primary plane formats for gen <= 3 */
>>> static const uint32_t i8xx_primary_formats[] = {
>>> @@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
>>> return true;
>>> else if (i915.enable_execlists)
>>> return true;
>>> + else if (i915_scheduler_is_enabled(ring->dev))
>>> + return true;
>>> else if (obj->base.dma_buf &&
>>> !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
>>> false))
>>>
>>
>> Why can't we use mmio flips unconditionally? Maarten or Ville?
>
> We do when execlists are used, which is always on gen9+. So I guess I'm
> missing the point of this patch. For gen5+ we could also do it trivially.
didn't check if the scheduler is also enabled for gen8 (I guess it would be nice, that would cover BDW and BSW).
>
> For older platforms it'd require a bit of work since we'd need to
> complete the flips from the vblank interrupt. Well, we actually do
> that already even with CS flips on those platforms, but we do look
> at the flip pending interrupt to figure out if CS already issued
> the flip or not. So that part would need changing.
>
> I also think we should switch to using the vblank interrupt for this
> stuff on all platforms, mainly since the flip done interrupt is somewhat
> broken on at least BDW (no idea if it got fixed in SKL or later), and
> doing things in more than one way certainly won't decrease our bug count.
Yeah that's probably the way to go; I haven't checked the behavior on SKL either.
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
2016-02-19 19:28 ` Jesse Barnes
2016-02-19 19:53 ` Ville Syrjälä
@ 2016-02-20 9:22 ` Chris Wilson
2016-02-22 20:42 ` Jesse Barnes
1 sibling, 1 reply; 82+ messages in thread
From: Chris Wilson @ 2016-02-20 9:22 UTC (permalink / raw)
To: Jesse Barnes; +Cc: Intel-GFX, Syrjala, Ville, Lankhorst, Maarten
On Fri, Feb 19, 2016 at 11:28:05AM -0800, Jesse Barnes wrote:
> On 02/18/2016 06:26 AM, John.C.Harrison@Intel.com wrote:
> > From: John Harrison <John.C.Harrison@Intel.com>
> >
> > MMIO flips are the preferred mechanism now
Because introducing variable latency in waking up a big core is a good
idea?
> but more importantly, pipe
> > based flips cause issues for the scheduler. Specifically, submitting
> > work to the rings around the side of the scheduler could cause that
> > work to be lost if the scheduler generates a pre-emption event on that
> > ring.
No. That is just incredibily bad design.
> Why can't we use mmio flips unconditionally? Maarten or Ville?
Why would we want to? CS flips work just fine in execlists and no reason
was ever given as to why they were not enabled, just laziness.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
2016-02-19 20:01 ` Jesse Barnes
@ 2016-02-22 9:41 ` Lankhorst, Maarten
2016-02-22 12:53 ` John Harrison
0 siblings, 1 reply; 82+ messages in thread
From: Lankhorst, Maarten @ 2016-02-22 9:41 UTC (permalink / raw)
To: jbarnes; +Cc: Intel-GFX, Syrjala, Ville
Hey,
Jesse Barnes schreef op vr 19-02-2016 om 12:01 [-0800]:
> On 02/19/2016 11:53 AM, Ville Syrjälä wrote:
> > On Fri, Feb 19, 2016 at 11:28:05AM -0800, Jesse Barnes wrote:
> >> On 02/18/2016 06:26 AM, John.C.Harrison@Intel.com wrote:
> >>> From: John Harrison <John.C.Harrison@Intel.com>
> >>>
> >>> MMIO flips are the preferred mechanism now but more importantly, pipe
> >>> based flips cause issues for the scheduler. Specifically, submitting
> >>> work to the rings around the side of the scheduler could cause that
> >>> work to be lost if the scheduler generates a pre-emption event on that
> >>> ring.
> >>>
> >>> For: VIZ-1587
> >>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> >>> ---
> >>> drivers/gpu/drm/i915/intel_display.c | 3 +++
> >>> 1 file changed, 3 insertions(+)
> >>>
> >>> diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
> >>> index 6e12ed7..731d20a 100644
> >>> --- a/drivers/gpu/drm/i915/intel_display.c
> >>> +++ b/drivers/gpu/drm/i915/intel_display.c
> >>> @@ -46,6 +46,7 @@
> >>> #include <linux/dma_remapping.h>
> >>> #include <linux/reservation.h>
> >>> #include <linux/dma-buf.h>
> >>> +#include "i915_scheduler.h"
> >>>
> >>> /* Primary plane formats for gen <= 3 */
> >>> static const uint32_t i8xx_primary_formats[] = {
> >>> @@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
> >>> return true;
> >>> else if (i915.enable_execlists)
> >>> return true;
> >>> + else if (i915_scheduler_is_enabled(ring->dev))
> >>> + return true;
> >>> else if (obj->base.dma_buf &&
> >>> !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
> >>> false))
> >>>
> >>
> >> Why can't we use mmio flips unconditionally? Maarten or Ville?
> >
> > We do when execlists are used, which is always on gen9+. So I guess I'm
> > missing the point of this patch. For gen5+ we could also do it trivially.
>
> didn't check if the scheduler is also enabled for gen8 (I guess it would be nice, that would cover BDW and BSW).
I have a patch that would enable support for mmio flips on all
platforms, but because of Chris Wilson's objections I still didn't force
mmio flips by default.
> >
> > For older platforms it'd require a bit of work since we'd need to
> > complete the flips from the vblank interrupt. Well, we actually do
> > that already even with CS flips on those platforms, but we do look
> > at the flip pending interrupt to figure out if CS already issued
> > the flip or not. So that part would need changing.
It was easy to fix in a way similar to that.
> > I also think we should switch to using the vblank interrupt for this
> > stuff on all platforms, mainly since the flip done interrupt is somewhat
> > broken on at least BDW (no idea if it got fixed in SKL or later), and
> > doing things in more than one way certainly won't decrease our bug count.
>
> Yeah that's probably the way to go; I haven't checked the behavior on SKL either.
---------------------------------------------------------------------
Intel International B.V.
Registered in The Netherlands under number 34098535
Statutory seat: Haarlemmermeer
Registered address: Capronilaan 37, 1119NG Schiphol-Rijk
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
2016-02-22 9:41 ` Lankhorst, Maarten
@ 2016-02-22 12:53 ` John Harrison
0 siblings, 0 replies; 82+ messages in thread
From: John Harrison @ 2016-02-22 12:53 UTC (permalink / raw)
To: Lankhorst, Maarten, jbarnes; +Cc: Intel-GFX, Syrjala, Ville
So is the summary that currently MMIO flips do not work on some platforms?
That could be a problem because the scheduler is intended to be enabled
for everything and thus will be forcing MMIO flips on everything. So
should I reverse the logic here - only enable the scheduler if MMIO
flips are already enabled?
Thanks,
John.
On 22/02/2016 09:41, Lankhorst, Maarten wrote:
> Hey,
>
>
> Jesse Barnes schreef op vr 19-02-2016 om 12:01 [-0800]:
>> On 02/19/2016 11:53 AM, Ville Syrjälä wrote:
>>> On Fri, Feb 19, 2016 at 11:28:05AM -0800, Jesse Barnes wrote:
>>>> On 02/18/2016 06:26 AM, John.C.Harrison@Intel.com wrote:
>>>>> From: John Harrison <John.C.Harrison@Intel.com>
>>>>>
>>>>> MMIO flips are the preferred mechanism now but more importantly, pipe
>>>>> based flips cause issues for the scheduler. Specifically, submitting
>>>>> work to the rings around the side of the scheduler could cause that
>>>>> work to be lost if the scheduler generates a pre-emption event on that
>>>>> ring.
>>>>>
>>>>> For: VIZ-1587
>>>>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>>>>> ---
>>>>> drivers/gpu/drm/i915/intel_display.c | 3 +++
>>>>> 1 file changed, 3 insertions(+)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
>>>>> index 6e12ed7..731d20a 100644
>>>>> --- a/drivers/gpu/drm/i915/intel_display.c
>>>>> +++ b/drivers/gpu/drm/i915/intel_display.c
>>>>> @@ -46,6 +46,7 @@
>>>>> #include <linux/dma_remapping.h>
>>>>> #include <linux/reservation.h>
>>>>> #include <linux/dma-buf.h>
>>>>> +#include "i915_scheduler.h"
>>>>>
>>>>> /* Primary plane formats for gen <= 3 */
>>>>> static const uint32_t i8xx_primary_formats[] = {
>>>>> @@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
>>>>> return true;
>>>>> else if (i915.enable_execlists)
>>>>> return true;
>>>>> + else if (i915_scheduler_is_enabled(ring->dev))
>>>>> + return true;
>>>>> else if (obj->base.dma_buf &&
>>>>> !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
>>>>> false))
>>>>>
>>>> Why can't we use mmio flips unconditionally? Maarten or Ville?
>>> We do when execlists are used, which is always on gen9+. So I guess I'm
>>> missing the point of this patch. For gen5+ we could also do it trivially.
>> didn't check if the scheduler is also enabled for gen8 (I guess it would be nice, that would cover BDW and BSW).
> I have a patch that would enable support for mmio flips on all
> platforms, but because of Chris Wilson's objections I still didn't force
> mmio flips by default.
>
>>> For older platforms it'd require a bit of work since we'd need to
>>> complete the flips from the vblank interrupt. Well, we actually do
>>> that already even with CS flips on those platforms, but we do look
>>> at the flip pending interrupt to figure out if CS already issued
>>> the flip or not. So that part would need changing.
> It was easy to fix in a way similar to that.
>
>>> I also think we should switch to using the vblank interrupt for this
>>> stuff on all platforms, mainly since the flip done interrupt is somewhat
>>> broken on at least BDW (no idea if it got fixed in SKL or later), and
>>> doing things in more than one way certainly won't decrease our bug count.
>> Yeah that's probably the way to go; I haven't checked the behavior on SKL either.
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
2016-02-20 9:22 ` Chris Wilson
@ 2016-02-22 20:42 ` Jesse Barnes
2016-02-23 11:16 ` Chris Wilson
0 siblings, 1 reply; 82+ messages in thread
From: Jesse Barnes @ 2016-02-22 20:42 UTC (permalink / raw)
To: Chris Wilson, John.C.Harrison, Intel-GFX, Lankhorst, Maarten,
Syrjala, Ville
On 02/20/2016 01:22 AM, Chris Wilson wrote:
> On Fri, Feb 19, 2016 at 11:28:05AM -0800, Jesse Barnes wrote:
>> On 02/18/2016 06:26 AM, John.C.Harrison@Intel.com wrote:
>>> From: John Harrison <John.C.Harrison@Intel.com>
>>>
>>> MMIO flips are the preferred mechanism now
>
> Because introducing variable latency in waking up a big core is a good
> idea?
Is the latency variability really that bad? I'm not too concerned about waking up the core either, I think it's going to be hit up quite a bit in these situations anyway, and vblanks should be driving the rendering as well.
>> but more importantly, pipe
>>> based flips cause issues for the scheduler. Specifically, submitting
>>> work to the rings around the side of the scheduler could cause that
>>> work to be lost if the scheduler generates a pre-emption event on that
>>> ring.
>
> No. That is just incredibily bad design.
>
>> Why can't we use mmio flips unconditionally? Maarten or Ville?
>
> Why would we want to? CS flips work just fine in execlists and no reason
> was ever given as to why they were not enabled, just laziness.
I'm not saying it can't be done, but I thought with atomic we decided to go with mmio because it makes things a lot simpler (iirc Ville's design did that way back?).
I'm fine with either, but it seems like we should settle on one rather than trying to maintain two different ways of flipping. We'll have work to do either way.
Jesse
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled
2016-02-22 20:42 ` Jesse Barnes
@ 2016-02-23 11:16 ` Chris Wilson
0 siblings, 0 replies; 82+ messages in thread
From: Chris Wilson @ 2016-02-23 11:16 UTC (permalink / raw)
To: Jesse Barnes; +Cc: Intel-GFX, Syrjala, Ville, Lankhorst, Maarten
On Mon, Feb 22, 2016 at 12:42:48PM -0800, Jesse Barnes wrote:
> On 02/20/2016 01:22 AM, Chris Wilson wrote:
> > On Fri, Feb 19, 2016 at 11:28:05AM -0800, Jesse Barnes wrote:
> >> On 02/18/2016 06:26 AM, John.C.Harrison@Intel.com wrote:
> >>> From: John Harrison <John.C.Harrison@Intel.com>
> >>>
> >>> MMIO flips are the preferred mechanism now
> >
> > Because introducing variable latency in waking up a big core is a good
> > idea?
>
> Is the latency variability really that bad? I'm not too concerned about waking up the core either, I think it's going to be hit up quite a bit in these situations anyway, and vblanks should be driving the rendering as well.
The workqueue we use is subject to being preempted by the user, so the
latency can be made arbitrarily large. Also, we don't control what tasks
are ahead of the flip.
> >> but more importantly, pipe
> >>> based flips cause issues for the scheduler. Specifically, submitting
> >>> work to the rings around the side of the scheduler could cause that
> >>> work to be lost if the scheduler generates a pre-emption event on that
> >>> ring.
> >
> > No. That is just incredibily bad design.
> >
> >> Why can't we use mmio flips unconditionally? Maarten or Ville?
> >
> > Why would we want to? CS flips work just fine in execlists and no reason
> > was ever given as to why they were not enabled, just laziness.
>
> I'm not saying it can't be done, but I thought with atomic we decided to go with mmio because it makes things a lot simpler (iirc Ville's design did that way back?).
My point is that the interface to the scheduler excludes even the
possibility of doing so because it is not cleanly segrated into a tool
for determining the next request to execute but instead subsumes half of
GEM and half of the engine backend becoming another unwanted midlayer
incorporating its own policy. The scheduler doesn't even deal in
requests, which is what prevents us from constructing a request here
with the known dependency and passing it to the scheduler for execution.
As I saw it, CS flips would be one of the best demonstrations for a
scheduler - because to execute the flip within the deadline should
cause a priority boost for all the prerequisites, and then ideally
reduce the number of missed target msc under load. (Though we can
equally promote the last_write_request through another scheduler
interface.)
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 20/35] drm/i915: Add scheduler hook to GPU reset
2016-02-18 14:27 ` [PATCH v5 20/35] drm/i915: Add scheduler hook to GPU reset John.C.Harrison
@ 2016-02-23 20:27 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-23 20:27 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> When the watchdog resets the GPU, all interrupts get disabled despite
> the reference count remaining. As the scheduler probably had
> interrupts enabled during the reset (it would have been waiting for
> the bad batch to complete), it must be poked to tell it that the
> interrupt has been disabled.
>
> v5: New patch in series.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem.c | 2 ++
> drivers/gpu/drm/i915/i915_scheduler.c | 11 +++++++++++
> drivers/gpu/drm/i915/i915_scheduler.h | 1 +
> 3 files changed, 14 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index d946f53..d7f7f7a 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -3248,6 +3248,8 @@ static void i915_gem_reset_ring_cleanup(struct drm_i915_private *dev_priv,
> buffer->last_retired_head = buffer->tail;
> intel_ring_update_space(buffer);
> }
> +
> + i915_scheduler_reset_cleanup(ring);
> }
>
> void i915_gem_reset(struct drm_device *dev)
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 8130a9c..4f25bf2 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -778,6 +778,17 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
> }
> }
>
> +void i915_scheduler_reset_cleanup(struct intel_engine_cs *ring)
> +{
> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + if (scheduler->flags[ring->id] & i915_sf_interrupts_enabled) {
> + ring->irq_put(ring);
> + scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;
> + }
> +}
> +
So I guess these flags are also protected by the struct_mutex? If so, I guess it looks ok.
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 21/35] drm/i915: Added a module parameter to allow the scheduler to be disabled
2016-02-18 14:27 ` [PATCH v5 21/35] drm/i915: Added a module parameter to allow the scheduler to be disabled John.C.Harrison
@ 2016-02-23 20:29 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-23 20:29 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> It can be useful to be able to disable the GPU scheduler via a module
> parameter for debugging purposes.
>
> v5: Converted from a multi-feature 'overrides' mask to a single
> 'enable' boolean. Further features (e.g. pre-emption) will now be
> separate 'enable' booleans added later. [Chris Wilson]
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_params.c | 4 ++++
> drivers/gpu/drm/i915/i915_params.h | 1 +
> drivers/gpu/drm/i915/i915_scheduler.c | 5 ++++-
> 3 files changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c
> index d0eba58..0ef3159 100644
> --- a/drivers/gpu/drm/i915/i915_params.c
> +++ b/drivers/gpu/drm/i915/i915_params.c
> @@ -57,6 +57,7 @@ struct i915_params i915 __read_mostly = {
> .edp_vswing = 0,
> .enable_guc_submission = true,
> .guc_log_level = -1,
> + .enable_scheduler = 0,
> };
>
> module_param_named(modeset, i915.modeset, int, 0400);
> @@ -203,3 +204,6 @@ MODULE_PARM_DESC(enable_guc_submission, "Enable GuC submission (default:false)")
> module_param_named(guc_log_level, i915.guc_log_level, int, 0400);
> MODULE_PARM_DESC(guc_log_level,
> "GuC firmware logging level (-1:disabled (default), 0-3:enabled)");
> +
> +module_param_named_unsafe(enable_scheduler, i915.enable_scheduler, int, 0600);
> +MODULE_PARM_DESC(enable_scheduler, "Enable scheduler (0 = disable [default], 1 = enable)");
> diff --git a/drivers/gpu/drm/i915/i915_params.h b/drivers/gpu/drm/i915/i915_params.h
> index 5299290..f855c86 100644
> --- a/drivers/gpu/drm/i915/i915_params.h
> +++ b/drivers/gpu/drm/i915/i915_params.h
> @@ -60,6 +60,7 @@ struct i915_params {
> bool enable_guc_submission;
> bool verbose_state_checks;
> bool nuclear_pageflip;
> + int enable_scheduler;
> };
>
> extern struct i915_params i915 __read_mostly;
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 4f25bf2..47d7de4 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -34,6 +34,9 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
> {
> struct drm_i915_private *dev_priv = dev->dev_private;
>
> + if (!i915.enable_scheduler)
> + return false;
> +
> return dev_priv->scheduler != NULL;
> }
>
> @@ -548,7 +551,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
>
> WARN_ON(!scheduler);
>
> - if (1/*!i915.enable_scheduler*/)
> + if (!i915.enable_scheduler)
> return i915_scheduler_queue_execbuffer_bypass(qe);
>
> node = kmalloc(sizeof(*node), GFP_KERNEL);
>
I did a double take here; maybe a comment along the lines of "if the scheduler is disabled, queue the buffer immediately" would help, and something similar for where the if (1) is added temporarily.
Doesn't matter too much though.
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Thanks,
Jesse
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 22/35] drm/i915: Support for 'unflushed' ring idle
2016-02-18 14:27 ` [PATCH v5 22/35] drm/i915: Support for 'unflushed' ring idle John.C.Harrison
@ 2016-02-23 20:35 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-23 20:35 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> When the seqno wraps around zero, the entire GPU is forced to be idle
> for some reason (possibly only to work around issues with hardware
> semaphores but no-one seems too sure!). This causes a problem if the
> force idle occurs at an inopportune moment such as in the middle of
> submitting a batch buffer. Specifically, it would lead to recursive
> submits - submitting work requires a new seqno, the new seqno requires
> idling the ring, idling the ring requires submitting work, submitting
> work requires a new seqno...
>
> This change adds a 'flush' parameter to the idle function call which
> specifies whether the scheduler queues should be flushed out. I.e. is
> the call intended to just idle the ring as it is right now (no flush)
> or is it intended to force all outstanding work out of the system
> (with flush).
>
> In the seqno wrap case, pending work is not an issue because the next
> operation will be to submit it. However, in other cases, the intention
> is to make sure everything that could be done has been done.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem.c | 4 ++--
> drivers/gpu/drm/i915/intel_lrc.c | 2 +-
> drivers/gpu/drm/i915/intel_ringbuffer.c | 17 +++++++++++++++--
> drivers/gpu/drm/i915/intel_ringbuffer.h | 2 +-
> 4 files changed, 19 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index d7f7f7a..a249e52 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -2564,7 +2564,7 @@ i915_gem_init_seqno(struct drm_device *dev, u32 seqno)
>
> /* Carefully retire all requests without writing to the rings */
> for_each_ring(ring, dev_priv, i) {
> - ret = intel_ring_idle(ring);
> + ret = intel_ring_idle(ring, false);
> if (ret)
> return ret;
> }
> @@ -3808,7 +3808,7 @@ int i915_gpu_idle(struct drm_device *dev)
> i915_add_request_no_flush(req);
> }
>
> - ret = intel_ring_idle(ring);
> + ret = intel_ring_idle(ring, true);
> if (ret)
> return ret;
> }
> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> index f4bab82..e056875 100644
> --- a/drivers/gpu/drm/i915/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/intel_lrc.c
> @@ -1058,7 +1058,7 @@ void intel_logical_ring_stop(struct intel_engine_cs *ring)
> if (!intel_ring_initialized(ring))
> return;
>
> - ret = intel_ring_idle(ring);
> + ret = intel_ring_idle(ring, true);
> if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
> DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
> ring->name, ret);
> diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
> index a2093f5..70ef9f0 100644
> --- a/drivers/gpu/drm/i915/intel_ringbuffer.c
> +++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
> @@ -2288,9 +2288,22 @@ static void __wrap_ring_buffer(struct intel_ringbuffer *ringbuf)
> intel_ring_update_space(ringbuf);
> }
>
> -int intel_ring_idle(struct intel_engine_cs *ring)
> +int intel_ring_idle(struct intel_engine_cs *ring, bool flush)
> {
> struct drm_i915_gem_request *req;
> + int ret;
> +
> + /*
> + * NB: Must not flush the scheduler if this idle request is from
> + * within an execbuff submission (i.e. due to 'get_seqno' calling
> + * 'wrap_seqno' calling 'idle'). As that would lead to recursive
> + * flushes!
> + */
> + if (flush) {
> + ret = i915_scheduler_flush(ring, true);
> + if (ret)
> + return ret;
> + }
>
> /* Wait upon the last request to be completed */
> if (list_empty(&ring->request_list))
> @@ -3095,7 +3108,7 @@ intel_stop_ring_buffer(struct intel_engine_cs *ring)
> if (!intel_ring_initialized(ring))
> return;
>
> - ret = intel_ring_idle(ring);
> + ret = intel_ring_idle(ring, true);
> if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
> DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
> ring->name, ret);
> diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
> index ada93a9..cca476f 100644
> --- a/drivers/gpu/drm/i915/intel_ringbuffer.h
> +++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
> @@ -478,7 +478,7 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf);
> int intel_ring_space(struct intel_ringbuffer *ringbuf);
> bool intel_ring_stopped(struct intel_engine_cs *ring);
>
> -int __must_check intel_ring_idle(struct intel_engine_cs *ring);
> +int __must_check intel_ring_idle(struct intel_engine_cs *ring, bool flush);
> void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno);
> int intel_ring_flush_all_caches(struct drm_i915_gem_request *req);
> int intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req);
>
Maybe Chris remembers the history here; I think the wraparound idle goes all the way back to Eric's original work with wrapping (something we had a lot of trouble with early on iirc).
My only suggestion here is to add wrappers for a new __intel_ring_idle(ring, bool), one for the existing usage of intel_ring_idle(ring) (which would pass false) and a new intel_ring_flush(ring) that passes true, along with some kdoc explaining the difference. Otherwise everyone will have to look up the param all the time. :)
With that change (because I'm a bool param hater, at least in exposed APIs):
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler
2016-02-18 14:27 ` [PATCH v5 24/35] drm/i915: Added trace points to scheduler John.C.Harrison
@ 2016-02-23 20:42 ` Jesse Barnes
2016-02-23 20:42 ` Jesse Barnes
1 sibling, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-23 20:42 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> Added trace points to the scheduler to track all the various events,
> node state transitions and other interesting things that occur.
>
> v2: Updated for new request completion tracking implementation.
>
> v3: Updated for changes to node kill code.
>
> v4: Wrapped some long lines to keep the style checker happy.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 +
> drivers/gpu/drm/i915/i915_scheduler.c | 26 ++++
> drivers/gpu/drm/i915/i915_trace.h | 196 +++++++++++++++++++++++++++++
> drivers/gpu/drm/i915/intel_lrc.c | 2 +
> 4 files changed, 226 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> index b9ad0fd..d4de8c7 100644
> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> @@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
>
> i915_gem_execbuffer_move_to_active(vmas, params->request);
>
> + trace_i915_gem_ring_queue(ring, params);
> +
> qe = container_of(params, typeof(*qe), params);
> ret = i915_scheduler_queue_execbuffer(qe);
> if (ret)
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 47d7de4..e56ce08 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
> /* Seqno will be reassigned on relaunch */
> node->params.request->seqno = 0;
> node->status = i915_sqs_queued;
> + trace_i915_scheduler_unfly(node->params.ring, node);
> + trace_i915_scheduler_node_state_change(node->params.ring, node);
> }
>
> /*
> @@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
> WARN_ON(!node);
> WARN_ON(I915_SQS_IS_COMPLETE(node));
>
> + if (I915_SQS_IS_FLYING(node))
> + trace_i915_scheduler_unfly(node->params.ring, node);
> +
> node->status = i915_sqs_dead;
> + trace_i915_scheduler_node_state_change(node->params.ring, node);
> }
>
> /* Mark a node as in flight on the hardware. */
> @@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
>
> node->status = i915_sqs_flying;
>
> + trace_i915_scheduler_fly(ring, node);
> + trace_i915_scheduler_node_state_change(ring, node);
> +
> if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
> bool success = true;
>
> @@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
> INIT_LIST_HEAD(&best->link);
> best->status = i915_sqs_popped;
>
> + trace_i915_scheduler_node_state_change(ring, best);
> +
> ret = 0;
> } else {
> /* Can only get here if:
> @@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
> }
> }
>
> + trace_i915_scheduler_pop_from_queue(ring, best);
> +
> *pop_node = best;
> return ret;
> }
> @@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
> struct i915_scheduler *scheduler = dev_priv->scheduler;
> int ret;
>
> + trace_i915_scheduler_queue(qe->params.ring, qe);
> +
> intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
>
> scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
> @@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
> not_flying = i915_scheduler_count_flying(scheduler, ring) <
> scheduler->min_flying;
>
> + trace_i915_scheduler_queue(ring, node);
> + trace_i915_scheduler_node_state_change(ring, node);
> +
> spin_unlock_irq(&scheduler->lock);
>
> if (not_flying)
> @@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
> struct i915_scheduler_queue_entry *node = req->scheduler_qe;
> unsigned long flags;
>
> + trace_i915_scheduler_landing(req);
> +
> if (!node)
> return false;
>
> @@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
> else
> node->status = i915_sqs_complete;
>
> + trace_i915_scheduler_node_state_change(req->ring, node);
> +
> spin_unlock_irqrestore(&scheduler->lock, flags);
>
> return true;
> @@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
> /* Launch more packets now? */
> do_submit = (queued > 0) && (flying < scheduler->min_flying);
>
> + trace_i915_scheduler_remove(ring, min_seqno, do_submit);
> +
> spin_unlock_irq(&scheduler->lock);
>
> return do_submit;
> @@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring)
> node = list_first_entry(&remove, typeof(*node), link);
> list_del(&node->link);
>
> + trace_i915_scheduler_destroy(ring, node);
> +
> /* Free up all the DRM references */
> i915_scheduler_clean_node(node);
>
> diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
> index 455c215..c3c4e58 100644
> --- a/drivers/gpu/drm/i915/i915_trace.h
> +++ b/drivers/gpu/drm/i915/i915_trace.h
> @@ -9,6 +9,7 @@
> #include "i915_drv.h"
> #include "intel_drv.h"
> #include "intel_ringbuffer.h"
> +#include "i915_scheduler.h"
>
> #undef TRACE_SYSTEM
> #define TRACE_SYSTEM i915
> @@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm,
> __entry->dev, __entry->ring, __entry->to, __entry->vm)
> );
>
> +TRACE_EVENT(i915_scheduler_queue,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node ? node->params.request->seqno : 0;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d",
> + __entry->ring, __entry->uniq, __entry->seqno)
> +);
> +
> +TRACE_EVENT(i915_scheduler_fly,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node ? node->params.request->seqno : 0;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d",
> + __entry->ring, __entry->uniq, __entry->seqno)
> +);
> +
> +TRACE_EVENT(i915_scheduler_unfly,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node ? node->params.request->seqno : 0;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d",
> + __entry->ring, __entry->uniq, __entry->seqno)
> +);
> +
> +TRACE_EVENT(i915_scheduler_landing,
> + TP_PROTO(struct drm_i915_gem_request *req),
> + TP_ARGS(req),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + __field(u32, status)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = req->ring->id;
> + __entry->uniq = req->uniq;
> + __entry->seqno = req->seqno;
> + __entry->status = req->scheduler_qe ?
> + req->scheduler_qe->status : ~0U;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
> + __entry->ring, __entry->uniq, __entry->seqno,
> + __entry->status)
> +);
> +
> +TRACE_EVENT(i915_scheduler_remove,
> + TP_PROTO(struct intel_engine_cs *ring,
> + u32 min_seqno, bool do_submit),
> + TP_ARGS(ring, min_seqno, do_submit),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, min_seqno)
> + __field(bool, do_submit)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->min_seqno = min_seqno;
> + __entry->do_submit = do_submit;
> + ),
> +
> + TP_printk("ring=%d, min_seqno = %d, do_submit=%d",
> + __entry->ring, __entry->min_seqno, __entry->do_submit)
> +);
> +
> +TRACE_EVENT(i915_scheduler_destroy,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node ? node->params.request->seqno : 0;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d",
> + __entry->ring, __entry->uniq, __entry->seqno)
> +);
> +
> +TRACE_EVENT(i915_scheduler_pop_from_queue,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node ? node->params.request->seqno : 0;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d",
> + __entry->ring, __entry->uniq, __entry->seqno)
> +);
> +
> +TRACE_EVENT(i915_scheduler_node_state_change,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + __field(u32, status)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node->params.request->seqno;
> + __entry->status = node->status;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
> + __entry->ring, __entry->uniq, __entry->seqno,
> + __entry->status)
> +);
> +
> +TRACE_EVENT(i915_gem_ring_queue,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_execbuffer_params *params),
> + TP_ARGS(ring, params),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = params->request->uniq;
> + __entry->seqno = params->request->seqno;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d", __entry->ring,
> + __entry->uniq, __entry->seqno)
> +);
> +
> #endif /* _I915_TRACE_H_ */
>
> /* This part must be outside protection */
> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> index 9c7a79a..2b9f49c 100644
> --- a/drivers/gpu/drm/i915/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/intel_lrc.c
> @@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
>
> i915_gem_execbuffer_move_to_active(vmas, params->request);
>
> + trace_i915_gem_ring_queue(ring, params);
> +
> qe = container_of(params, typeof(*qe), params);
> ret = i915_scheduler_queue_execbuffer(qe);
> if (ret)
>
Looks fine.
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler
2016-02-18 14:27 ` [PATCH v5 24/35] drm/i915: Added trace points to scheduler John.C.Harrison
2016-02-23 20:42 ` Jesse Barnes
@ 2016-02-23 20:42 ` Jesse Barnes
2016-02-26 15:55 ` John Harrison
1 sibling, 1 reply; 82+ messages in thread
From: Jesse Barnes @ 2016-02-23 20:42 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> Added trace points to the scheduler to track all the various events,
> node state transitions and other interesting things that occur.
>
> v2: Updated for new request completion tracking implementation.
>
> v3: Updated for changes to node kill code.
>
> v4: Wrapped some long lines to keep the style checker happy.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 +
> drivers/gpu/drm/i915/i915_scheduler.c | 26 ++++
> drivers/gpu/drm/i915/i915_trace.h | 196 +++++++++++++++++++++++++++++
> drivers/gpu/drm/i915/intel_lrc.c | 2 +
> 4 files changed, 226 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> index b9ad0fd..d4de8c7 100644
> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> @@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
>
> i915_gem_execbuffer_move_to_active(vmas, params->request);
>
> + trace_i915_gem_ring_queue(ring, params);
> +
> qe = container_of(params, typeof(*qe), params);
> ret = i915_scheduler_queue_execbuffer(qe);
> if (ret)
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 47d7de4..e56ce08 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
> /* Seqno will be reassigned on relaunch */
> node->params.request->seqno = 0;
> node->status = i915_sqs_queued;
> + trace_i915_scheduler_unfly(node->params.ring, node);
> + trace_i915_scheduler_node_state_change(node->params.ring, node);
> }
>
> /*
> @@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
> WARN_ON(!node);
> WARN_ON(I915_SQS_IS_COMPLETE(node));
>
> + if (I915_SQS_IS_FLYING(node))
> + trace_i915_scheduler_unfly(node->params.ring, node);
> +
> node->status = i915_sqs_dead;
> + trace_i915_scheduler_node_state_change(node->params.ring, node);
> }
>
> /* Mark a node as in flight on the hardware. */
> @@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
>
> node->status = i915_sqs_flying;
>
> + trace_i915_scheduler_fly(ring, node);
> + trace_i915_scheduler_node_state_change(ring, node);
> +
> if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
> bool success = true;
>
> @@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
> INIT_LIST_HEAD(&best->link);
> best->status = i915_sqs_popped;
>
> + trace_i915_scheduler_node_state_change(ring, best);
> +
> ret = 0;
> } else {
> /* Can only get here if:
> @@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
> }
> }
>
> + trace_i915_scheduler_pop_from_queue(ring, best);
> +
> *pop_node = best;
> return ret;
> }
> @@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
> struct i915_scheduler *scheduler = dev_priv->scheduler;
> int ret;
>
> + trace_i915_scheduler_queue(qe->params.ring, qe);
> +
> intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
>
> scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
> @@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
> not_flying = i915_scheduler_count_flying(scheduler, ring) <
> scheduler->min_flying;
>
> + trace_i915_scheduler_queue(ring, node);
> + trace_i915_scheduler_node_state_change(ring, node);
> +
> spin_unlock_irq(&scheduler->lock);
>
> if (not_flying)
> @@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
> struct i915_scheduler_queue_entry *node = req->scheduler_qe;
> unsigned long flags;
>
> + trace_i915_scheduler_landing(req);
> +
> if (!node)
> return false;
>
> @@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
> else
> node->status = i915_sqs_complete;
>
> + trace_i915_scheduler_node_state_change(req->ring, node);
> +
> spin_unlock_irqrestore(&scheduler->lock, flags);
>
> return true;
> @@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
> /* Launch more packets now? */
> do_submit = (queued > 0) && (flying < scheduler->min_flying);
>
> + trace_i915_scheduler_remove(ring, min_seqno, do_submit);
> +
> spin_unlock_irq(&scheduler->lock);
>
> return do_submit;
> @@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring)
> node = list_first_entry(&remove, typeof(*node), link);
> list_del(&node->link);
>
> + trace_i915_scheduler_destroy(ring, node);
> +
> /* Free up all the DRM references */
> i915_scheduler_clean_node(node);
>
> diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
> index 455c215..c3c4e58 100644
> --- a/drivers/gpu/drm/i915/i915_trace.h
> +++ b/drivers/gpu/drm/i915/i915_trace.h
> @@ -9,6 +9,7 @@
> #include "i915_drv.h"
> #include "intel_drv.h"
> #include "intel_ringbuffer.h"
> +#include "i915_scheduler.h"
>
> #undef TRACE_SYSTEM
> #define TRACE_SYSTEM i915
> @@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm,
> __entry->dev, __entry->ring, __entry->to, __entry->vm)
> );
>
> +TRACE_EVENT(i915_scheduler_queue,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node ? node->params.request->seqno : 0;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d",
> + __entry->ring, __entry->uniq, __entry->seqno)
> +);
> +
> +TRACE_EVENT(i915_scheduler_fly,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node ? node->params.request->seqno : 0;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d",
> + __entry->ring, __entry->uniq, __entry->seqno)
> +);
> +
> +TRACE_EVENT(i915_scheduler_unfly,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node ? node->params.request->seqno : 0;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d",
> + __entry->ring, __entry->uniq, __entry->seqno)
> +);
> +
> +TRACE_EVENT(i915_scheduler_landing,
> + TP_PROTO(struct drm_i915_gem_request *req),
> + TP_ARGS(req),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + __field(u32, status)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = req->ring->id;
> + __entry->uniq = req->uniq;
> + __entry->seqno = req->seqno;
> + __entry->status = req->scheduler_qe ?
> + req->scheduler_qe->status : ~0U;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
> + __entry->ring, __entry->uniq, __entry->seqno,
> + __entry->status)
> +);
> +
> +TRACE_EVENT(i915_scheduler_remove,
> + TP_PROTO(struct intel_engine_cs *ring,
> + u32 min_seqno, bool do_submit),
> + TP_ARGS(ring, min_seqno, do_submit),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, min_seqno)
> + __field(bool, do_submit)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->min_seqno = min_seqno;
> + __entry->do_submit = do_submit;
> + ),
> +
> + TP_printk("ring=%d, min_seqno = %d, do_submit=%d",
> + __entry->ring, __entry->min_seqno, __entry->do_submit)
> +);
> +
> +TRACE_EVENT(i915_scheduler_destroy,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node ? node->params.request->seqno : 0;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d",
> + __entry->ring, __entry->uniq, __entry->seqno)
> +);
> +
> +TRACE_EVENT(i915_scheduler_pop_from_queue,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node ? node->params.request->seqno : 0;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d",
> + __entry->ring, __entry->uniq, __entry->seqno)
> +);
> +
> +TRACE_EVENT(i915_scheduler_node_state_change,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_scheduler_queue_entry *node),
> + TP_ARGS(ring, node),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + __field(u32, status)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = node ? node->params.request->uniq : 0;
> + __entry->seqno = node->params.request->seqno;
> + __entry->status = node->status;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
> + __entry->ring, __entry->uniq, __entry->seqno,
> + __entry->status)
> +);
> +
> +TRACE_EVENT(i915_gem_ring_queue,
> + TP_PROTO(struct intel_engine_cs *ring,
> + struct i915_execbuffer_params *params),
> + TP_ARGS(ring, params),
> +
> + TP_STRUCT__entry(
> + __field(u32, ring)
> + __field(u32, uniq)
> + __field(u32, seqno)
> + ),
> +
> + TP_fast_assign(
> + __entry->ring = ring->id;
> + __entry->uniq = params->request->uniq;
> + __entry->seqno = params->request->seqno;
> + ),
> +
> + TP_printk("ring=%d, uniq=%d, seqno=%d", __entry->ring,
> + __entry->uniq, __entry->seqno)
> +);
> +
> #endif /* _I915_TRACE_H_ */
>
> /* This part must be outside protection */
> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> index 9c7a79a..2b9f49c 100644
> --- a/drivers/gpu/drm/i915/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/intel_lrc.c
> @@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
>
> i915_gem_execbuffer_move_to_active(vmas, params->request);
>
> + trace_i915_gem_ring_queue(ring, params);
> +
> qe = container_of(params, typeof(*qe), params);
> ret = i915_scheduler_queue_execbuffer(qe);
> if (ret)
>
Oops, forgot to ask if there are any tools for igt that make use of these, e.g. some kind of scheduler top util that looks for bubbles or starved batches or something.
Jesse
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle
2016-02-18 14:27 ` [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle John.C.Harrison
@ 2016-02-23 21:02 ` Jesse Barnes
2016-03-01 15:52 ` John Harrison
0 siblings, 1 reply; 82+ messages in thread
From: Jesse Barnes @ 2016-02-23 21:02 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> The scheduler decouples the submission of batch buffers to the driver
> from their subsequent submission to the hardware. This means that an
> application which is continuously submitting buffers as fast as it can
> could potentialy flood the driver. To prevent this, the driver now
> tracks how many buffers are in progress (queued in software or
> executing in hardware) and limits this to a given (tunable) number. If
> this number is exceeded then the queue to the driver will return
> EAGAIN and thus prevent the scheduler's queue becoming arbitrarily
> large.
>
> v3: Added a missing decrement of the file queue counter.
>
> v4: Updated a comment.
>
> v5: Updated due to changes to earlier patches in series - removing
> forward declarations and white space. Also added some documentation.
> [Joonas Lahtinen]
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_drv.h | 2 ++
> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 8 +++++
> drivers/gpu/drm/i915/i915_scheduler.c | 48 ++++++++++++++++++++++++++++++
> drivers/gpu/drm/i915/i915_scheduler.h | 2 ++
> 4 files changed, 60 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index 071a27b..3f4c4f0 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -336,6 +336,8 @@ struct drm_i915_file_private {
> } rps;
>
> struct intel_engine_cs *bsd_ring;
> +
> + u32 scheduler_queue_length;
> };
>
> enum intel_dpll_id {
> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> index d4de8c7..dff120c 100644
> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> @@ -1803,6 +1803,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
> return -EINVAL;
> }
>
> + /* Throttle batch requests per device file */
> + if (i915_scheduler_file_queue_is_full(file))
> + return -EAGAIN;
> +
> /* Copy in the exec list from userland */
> exec_list = drm_malloc_ab(sizeof(*exec_list), args->buffer_count);
> exec2_list = drm_malloc_ab(sizeof(*exec2_list), args->buffer_count);
> @@ -1893,6 +1897,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
> return -EINVAL;
> }
>
> + /* Throttle batch requests per device file */
> + if (i915_scheduler_file_queue_is_full(file))
> + return -EAGAIN;
> +
> exec2_list = kmalloc(sizeof(*exec2_list)*args->buffer_count,
> GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
> if (exec2_list == NULL)
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index e56ce08..f7f29d5 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -69,6 +69,7 @@ int i915_scheduler_init(struct drm_device *dev)
> scheduler->priority_level_bump = 50;
> scheduler->priority_level_preempt = 900;
> scheduler->min_flying = 2;
> + scheduler->file_queue_max = 64;
>
> dev_priv->scheduler = scheduler;
>
> @@ -464,6 +465,44 @@ static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)
> return ret;
> }
>
> +/**
> + * i915_scheduler_file_queue_is_full - Returns true if the queue is full.
> + * @file: File object to query.
> + * This allows throttling of applications by limiting the total number of
> + * outstanding requests to a specified level. Once that limit is reached,
> + * this call will return true and no more requests should be accepted.
> + */
> +bool i915_scheduler_file_queue_is_full(struct drm_file *file)
> +{
> + struct drm_i915_file_private *file_priv = file->driver_priv;
> + struct drm_i915_private *dev_priv = file_priv->dev_priv;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + return file_priv->scheduler_queue_length >= scheduler->file_queue_max;
> +}
> +
> +/**
> + * i915_scheduler_file_queue_inc - Increment the file's request queue count.
> + * @file: File object to process.
> + */
> +static void i915_scheduler_file_queue_inc(struct drm_file *file)
> +{
> + struct drm_i915_file_private *file_priv = file->driver_priv;
> +
> + file_priv->scheduler_queue_length++;
> +}
> +
> +/**
> + * i915_scheduler_file_queue_dec - Decrement the file's request queue count.
> + * @file: File object to process.
> + */
> +static void i915_scheduler_file_queue_dec(struct drm_file *file)
> +{
> + struct drm_i915_file_private *file_priv = file->driver_priv;
> +
> + file_priv->scheduler_queue_length--;
> +}
> +
> static void i915_generate_dependencies(struct i915_scheduler *scheduler,
> struct i915_scheduler_queue_entry *node,
> uint32_t ring)
> @@ -640,6 +679,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
>
> list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
>
> + i915_scheduler_file_queue_inc(node->params.file);
> +
> not_flying = i915_scheduler_count_flying(scheduler, ring) <
> scheduler->min_flying;
>
> @@ -883,6 +924,12 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
> /* Strip the dependency info while the mutex is still locked */
> i915_scheduler_remove_dependent(scheduler, node);
>
> + /* Likewise clean up the file pointer. */
> + if (node->params.file) {
> + i915_scheduler_file_queue_dec(node->params.file);
> + node->params.file = NULL;
> + }
> +
> continue;
> }
>
> @@ -1205,6 +1252,7 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
> node->status,
> ring->name);
>
> + i915_scheduler_file_queue_dec(node->params.file);
> node->params.file = NULL;
> }
> }
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 075befb..b78de12 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -77,6 +77,7 @@ struct i915_scheduler {
> int32_t priority_level_bump;
> int32_t priority_level_preempt;
> uint32_t min_flying;
> + uint32_t file_queue_max;
> };
>
> /* Flag bits for i915_scheduler::flags */
> @@ -100,5 +101,6 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
> unsigned long stamp, bool is_locked);
> bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
> bool *completed, bool *busy);
> +bool i915_scheduler_file_queue_is_full(struct drm_file *file);
>
> #endif /* _I915_SCHEDULER_H_ */
>
Just to clarify and make sure I understood the previous stuff: a queued execbuf that has not yet been dispatched does not reserve and pin pages right? That occurs at actual dispatch time? If so, I guess clients will hit this 64 queued item limit pretty regularly... How much metadata overhead does that involve? Has it been derived from some performance work with a bunch of workloads? It's fine if not, I can imagine that different mixes of workloads would be affected by lower or higher queue depths (e.g. small batch tests).
If this is tunable, I guess it should be clamped like a nice or rlimit value, with values outside that range requiring CAP_SYS_ADMIN.
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Jesse
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters
2016-02-18 14:27 ` [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters John.C.Harrison
@ 2016-02-23 21:06 ` Jesse Barnes
2016-03-11 16:28 ` John Harrison
0 siblings, 1 reply; 82+ messages in thread
From: Jesse Barnes @ 2016-02-23 21:06 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> There are various parameters within the scheduler which can be tuned
> to improve performance, reduce memory footprint, etc. This change adds
> support for altering these via debugfs.
>
> v2: Updated for priorities now being signed values.
>
> v5: Squashed priority bumping entries into this patch rather than a
> separate patch all of their own.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_debugfs.c | 169 ++++++++++++++++++++++++++++++++++++
> 1 file changed, 169 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
> index b923949..7d01c07 100644
> --- a/drivers/gpu/drm/i915/i915_debugfs.c
> +++ b/drivers/gpu/drm/i915/i915_debugfs.c
> @@ -39,6 +39,7 @@
> #include "intel_ringbuffer.h"
> #include <drm/i915_drm.h>
> #include "i915_drv.h"
> +#include "i915_scheduler.h"
>
> enum {
> ACTIVE_LIST,
> @@ -1122,6 +1123,168 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops,
> i915_next_seqno_get, i915_next_seqno_set,
> "0x%llx\n");
>
> +static int
> +i915_scheduler_priority_min_get(void *data, u64 *val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + *val = (u64) scheduler->priority_level_min;
> + return 0;
> +}
> +
> +static int
> +i915_scheduler_priority_min_set(void *data, u64 val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + scheduler->priority_level_min = (int32_t) val;
> + return 0;
> +}
> +
> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_min_fops,
> + i915_scheduler_priority_min_get,
> + i915_scheduler_priority_min_set,
> + "%lld\n");
> +
> +static int
> +i915_scheduler_priority_max_get(void *data, u64 *val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + *val = (u64) scheduler->priority_level_max;
> + return 0;
> +}
> +
> +static int
> +i915_scheduler_priority_max_set(void *data, u64 val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + scheduler->priority_level_max = (int32_t) val;
> + return 0;
> +}
> +
> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_max_fops,
> + i915_scheduler_priority_max_get,
> + i915_scheduler_priority_max_set,
> + "%lld\n");
> +
> +static int
> +i915_scheduler_priority_bump_get(void *data, u64 *val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + *val = (u64) scheduler->priority_level_bump;
> + return 0;
> +}
> +
> +static int
> +i915_scheduler_priority_bump_set(void *data, u64 val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + scheduler->priority_level_bump = (u32) val;
> + return 0;
> +}
> +
> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_bump_fops,
> + i915_scheduler_priority_bump_get,
> + i915_scheduler_priority_bump_set,
> + "%lld\n");
> +
> +static int
> +i915_scheduler_priority_preempt_get(void *data, u64 *val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + *val = (u64) scheduler->priority_level_preempt;
> + return 0;
> +}
> +
> +static int
> +i915_scheduler_priority_preempt_set(void *data, u64 val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + scheduler->priority_level_preempt = (u32) val;
> + return 0;
> +}
> +
> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_preempt_fops,
> + i915_scheduler_priority_preempt_get,
> + i915_scheduler_priority_preempt_set,
> + "%lld\n");
> +
> +static int
> +i915_scheduler_min_flying_get(void *data, u64 *val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + *val = (u64) scheduler->min_flying;
> + return 0;
> +}
> +
> +static int
> +i915_scheduler_min_flying_set(void *data, u64 val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + scheduler->min_flying = (u32) val;
> + return 0;
> +}
> +
> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_min_flying_fops,
> + i915_scheduler_min_flying_get,
> + i915_scheduler_min_flying_set,
> + "%llu\n");
> +
> +static int
> +i915_scheduler_file_queue_max_get(void *data, u64 *val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + *val = (u64) scheduler->file_queue_max;
> + return 0;
> +}
> +
> +static int
> +i915_scheduler_file_queue_max_set(void *data, u64 val)
> +{
> + struct drm_device *dev = data;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + scheduler->file_queue_max = (u32) val;
> + return 0;
> +}
> +
> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
> + i915_scheduler_file_queue_max_get,
> + i915_scheduler_file_queue_max_set,
> + "%llu\n");
> +
> static int i915_frequency_info(struct seq_file *m, void *unused)
> {
> struct drm_info_node *node = m->private;
> @@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files {
> {"i915_gem_drop_caches", &i915_drop_caches_fops},
> {"i915_error_state", &i915_error_state_fops},
> {"i915_next_seqno", &i915_next_seqno_fops},
> + {"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
> + {"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
> + {"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops},
> + {"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
> + {"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
> + {"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
> {"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
> {"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
> {"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
>
Do these need to be serialized at all? I guess some raciness doesn't hurt too much for these guys, unless somehow an inconsistent set of values would cause a livelock in the scheduler or something.
If not,
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 06/35] drm/i915: Start of GPU scheduler
2016-02-19 17:03 ` John Harrison
@ 2016-02-26 9:13 ` Joonas Lahtinen
2016-02-26 14:18 ` John Harrison
0 siblings, 1 reply; 82+ messages in thread
From: Joonas Lahtinen @ 2016-02-26 9:13 UTC (permalink / raw)
To: John Harrison, Intel-GFX
Hi,
The below answers are reasonable. So v6 should be the version.
Regards, Joonas
On pe, 2016-02-19 at 17:03 +0000, John Harrison wrote:
> On 19/02/2016 13:03, Joonas Lahtinen wrote:
> >
> > Hi,
> >
> > Now the code is in reviewable chunks, excellent!
> >
> > I've added my comments below. A few repeats from last round, but
> > now
> > with more questions about the logic itself.
> >
> > On to, 2016-02-18 at 14:26 +0000, John.C.Harrison@Intel.com wrote:
> > >
> > > From: John Harrison <John.C.Harrison@Intel.com>
> > >
> > > Initial creation of scheduler source files. Note that this patch
> > > implements most of the scheduler functionality but does not hook
> > > it in
> > > to the driver yet. It also leaves the scheduler code in 'pass
> > > through'
> > > mode so that even when it is hooked in, it will not actually do
> > > very
> > > much. This allows the hooks to be added one at a time in bite
> > > size
> > > chunks and only when the scheduler is finally enabled at the end
> > > does
> > > anything start happening.
> > >
> > > The general theory of operation is that when batch buffers are
> > > submitted to the driver, the execbuffer() code packages up all
> > > the
> > > information required to execute the batch buffer at a later time.
> > > This
> > > package is given over to the scheduler which adds it to an
> > > internal
> > > node list. The scheduler also scans the list of objects
> > > associated
> > > with the batch buffer and compares them against the objects
> > > already in
> > > use by other buffers in the node list. If matches are found then
> > > the
> > > new batch buffer node is marked as being dependent upon the
> > > matching
> > > node. The same is done for the context object. The scheduler also
> > > bumps up the priority of such matching nodes on the grounds that
> > > the
> > > more dependencies a given batch buffer has the more important it
> > > is
> > > likely to be.
> > >
> > > The scheduler aims to have a given (tuneable) number of batch
> > > buffers
> > > in flight on the hardware at any given time. If fewer than this
> > > are
> > > currently executing when a new node is queued, then the node is
> > > passed
> > > straight through to the submit function. Otherwise it is simply
> > > added
> > > to the queue and the driver returns back to user land.
> > >
> > > The scheduler is notified when each batch buffer completes and
> > > updates
> > > its internal tracking accordingly. At the end of the completion
> > > interrupt processing, if any scheduler tracked batches were
> > > processed,
> > > the scheduler's deferred worker thread is woken up. This can do
> > > more
> > > involved processing such as actually removing completed nodes
> > > from the
> > > queue and freeing up the resources associated with them (internal
> > > memory allocations, DRM object references, context reference,
> > > etc.).
> > > The work handler also checks the in flight count and calls the
> > > submission code if a new slot has appeared.
> > >
> > > When the scheduler's submit code is called, it scans the queued
> > > node
> > > list for the highest priority node that has no unmet
> > > dependencies.
> > > Note that the dependency calculation is complex as it must take
> > > inter-ring dependencies and potential preemptions into account.
> > > Note
> > > also that in the future this will be extended to include external
> > > dependencies such as the Android Native Sync file descriptors
> > > and/or
> > > the linux dma-buff synchronisation scheme.
> > >
> > > If a suitable node is found then it is sent to execbuff_final()
> > > for
> > > submission to the hardware. The in flight count is then re-
> > > checked and
> > > a new node popped from the list if appropriate. All nodes that
> > > are not
> > > submitted have their priority bumped. This ensures that low
> > > priority
> > > tasks do not get starved out by busy higher priority ones -
> > > everything
> > > will eventually get its turn to run.
> > >
> > > Note that this patch does not implement pre-emptive scheduling.
> > > Only
> > > basic scheduling by re-ordering batch buffer submission is
> > > currently
> > > implemented. Pre-emption of actively executing batch buffers
> > > comes in
> > > the next patch series.
> > >
> > > v2: Changed priority levels to +/-1023 due to feedback from Chris
> > > Wilson.
> > >
> > > Removed redundant index from scheduler node.
> > >
> > > Changed time stamps to use jiffies instead of raw monotonic. This
> > > provides lower resolution but improved compatibility with other
> > > i915
> > > code.
> > >
> > > Major re-write of completion tracking code due to struct fence
> > > conversion. The scheduler no longer has it's own private IRQ
> > > handler
> > > but just lets the existing request code handle completion events.
> > > Instead, the scheduler now hooks into the request notify code to
> > > be
> > > told when a request has completed.
> > >
> > > Reduced driver mutex locking scope. Removal of scheduler nodes no
> > > longer grabs the mutex lock.
> > >
> > > v3: Refactor of dependency generation to make the code more
> > > readable.
> > > Also added in read-read optimisation support - i.e., don't treat
> > > a
> > > shared read-only buffer as being a dependency.
> > >
> > > Allowed the killing of queued nodes rather than only flying ones.
> > >
> > > v4: Updated the commit message to better reflect the current
> > > state of
> > > the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct
> > > array
> > > memory allocator function (kmalloc_array instead of kmalloc).
> > > Corrected the format of some comments. Wrapped some lines
> > > differently
> > > to keep the style checker happy.
> > >
> > > Fixed a WARN_ON when killing nodes. The dependency removal code
> > > checks
> > > that nodes being destroyed do not have any oustanding
> > > dependencies
> > > (which would imply they should not have been executed yet). In
> > > the
> > > case of nodes being destroyed, e.g. due to context banning, then
> > > this
> > > might well be the case - they have not been executed and do
> > > indeed
> > > have outstanding dependencies.
> > >
> > > Re-instated the code to disble interrupts when not in use. The
> > > underlying problem causing broken IRQ reference counts seems to
> > > have
> > > been fixed now.
> > >
> > > v5: Shuffled various functions around to remove forward
> > > declarations
> > > as apparently these are frowned upon. Removed lots of white space
> > > as
> > > apparently having easy to read code is also frowned upon. Split
> > > the
> > > direct submission scheduler bypass code out into a separate
> > > function.
> > > Squashed down the i915_scheduler.c sections of various patches
> > > into
> > > this patch. Thus the later patches simply hook in existing code
> > > into
> > > various parts of the driver rather than adding the code as well.
> > > Added
> > > documentation to various functions. Re-worked the submit function
> > > in
> > > terms of mutex locking, error handling and exit paths. Split the
> > > delayed work handler function in half. Made use of the kernel
> > > 'clamp'
> > > macro. [Joonas Lahtinen]
> > >
> > > Added runtime PM calls as these must be done at the top level
> > > before
> > > acquiring the driver mutex lock. [Chris Wilson]
> > >
> > > Removed some obsolete debug code that had been forgotten about.
> > >
> > > Moved more clean up code into the
> > > 'i915_gem_scheduler_clean_node()'
> > > function rather than replicating it in mutliple places.
> > >
> > > Used lighter weight spinlocks.
> > >
> > > For: VIZ-1587
> > > Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> > > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > > ---
> > > drivers/gpu/drm/i915/Makefile | 1 +
> > > drivers/gpu/drm/i915/i915_drv.h | 6 +
> > > drivers/gpu/drm/i915/i915_gem.c | 5 +
> > > drivers/gpu/drm/i915/i915_scheduler.c | 874
> > > ++++++++++++++++++++++++++++++++++
> > > drivers/gpu/drm/i915/i915_scheduler.h | 95 ++++
> > > 5 files changed, 981 insertions(+)
> > > create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
> > > create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
> > >
> > > diff --git a/drivers/gpu/drm/i915/Makefile
> > > b/drivers/gpu/drm/i915/Makefile
> > > index 15398c5..79cb38b 100644
> > > --- a/drivers/gpu/drm/i915/Makefile
> > > +++ b/drivers/gpu/drm/i915/Makefile
> > > @@ -10,6 +10,7 @@ ccflags-y := -Werror
> > > i915-y := i915_drv.o \
> > > i915_irq.o \
> > > i915_params.o \
> > > + i915_scheduler.o \
> > > i915_suspend.o \
> > > i915_sysfs.o \
> > > intel_csr.o \
> > > diff --git a/drivers/gpu/drm/i915/i915_drv.h
> > > b/drivers/gpu/drm/i915/i915_drv.h
> > > index f4487b9..03add1a 100644
> > > --- a/drivers/gpu/drm/i915/i915_drv.h
> > > +++ b/drivers/gpu/drm/i915/i915_drv.h
> > > @@ -1703,6 +1703,8 @@ struct i915_execbuffer_params {
> > > struct drm_i915_gem_request *request;
> > > };
> > >
> > > +struct i915_scheduler;
> > > +
> > > /* used in computing the new watermarks state */
> > > struct intel_wm_config {
> > > unsigned int num_pipes_active;
> > > @@ -1968,6 +1970,8 @@ struct drm_i915_private {
> > >
> > > struct i915_runtime_pm pm;
> > >
> > > + struct i915_scheduler *scheduler;
> > > +
> > I think we should have i915.enable_scheduler parameter (just like
> > i915.enable_execlists) and make this a data member, not pointer.
> >
> > Then we can go forward and have i915.enable_scheduler_preempt and
> > so on
> > to control things at boot time.
> There are. The enable_scheduler comes in as patch #21. Including it
> from
> the start means modifying i915_params.[ch] in this patch as well.
> And
> then you are getting into one big patch that contains everything and
> is
> unreviewable due to its sheer size. The enable_preemption parameter
> comes in with the pre-emption code. No point in adding that until
> there
> is something for it to control.
>
>
> >
> >
> > >
> > > /* Abstract the submission mechanism (legacy ringbuffer
> > > or execlists) away */
> > > struct {
> > > int (*execbuf_submit)(struct
> > > i915_execbuffer_params *params,
> > > @@ -2290,6 +2294,8 @@ struct drm_i915_gem_request {
> > > /** process identifier submitting this request */
> > > struct pid *pid;
> > >
> > > + struct i915_scheduler_queue_entry *scheduler_qe;
> > > +
> > Funny whitespace.
> >
> > >
> > > /**
> > > * The ELSP only accepts two elements at a time, so we
> > > queue
> > > * context/tail pairs on a given queue (ring-
> > > >execlist_queue) until the
> > > diff --git a/drivers/gpu/drm/i915/i915_gem.c
> > > b/drivers/gpu/drm/i915/i915_gem.c
> > > index dfe43ea..7d9aa24 100644
> > > --- a/drivers/gpu/drm/i915/i915_gem.c
> > > +++ b/drivers/gpu/drm/i915/i915_gem.c
> > > @@ -32,6 +32,7 @@
> > > #include "i915_vgpu.h"
> > > #include "i915_trace.h"
> > > #include "intel_drv.h"
> > > +#include "i915_scheduler.h"
> > > #include
> > > #include
> > > #include
> > > @@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)
> > > */
> > > intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
> > >
> > > + ret = i915_scheduler_init(dev);
> > > + if (ret)
> > > + goto out_unlock;
> > > +
> > > ret = i915_gem_init_userptr(dev);
> > > if (ret)
> > > goto out_unlock;
> > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c
> > > b/drivers/gpu/drm/i915/i915_scheduler.c
> > > new file mode 100644
> > > index 0000000..fc23ee7
> > > --- /dev/null
> > > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > > @@ -0,0 +1,874 @@
> > > +/*
> > > + * Copyright (c) 2014 Intel Corporation
> > > + *
> > > + * Permission is hereby granted, free of charge, to any person
> > > obtaining a
> > > + * copy of this software and associated documentation files (the
> > > "Software"),
> > > + * to deal in the Software without restriction, including
> > > without limitation
> > > + * the rights to use, copy, modify, merge, publish, distribute,
> > > sublicense,
> > > + * and/or sell copies of the Software, and to permit persons to
> > > whom the
> > > + * Software is furnished to do so, subject to the following
> > > conditions:
> > > + *
> > > + * The above copyright notice and this permission notice
> > > (including the next
> > > + * paragraph) shall be included in all copies or substantial
> > > portions of the
> > > + * Software.
> > > + *
> > > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
> > > KIND, EXPRESS OR
> > > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> > > MERCHANTABILITY,
> > > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
> > > EVENT SHALL
> > > + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
> > > DAMAGES OR OTHER
> > > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
> > > OTHERWISE, ARISING
> > > + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> > > OTHER DEALINGS
> > > + * IN THE SOFTWARE.
> > > + *
> > > + */
> > > +
> > > +#include "i915_drv.h"
> > > +#include "intel_drv.h"
> > > +#include "i915_scheduler.h"
> > > +
> > > +/**
> > > + * i915_scheduler_is_enabled - Returns true if the scheduler is
> > > enabled.
> > > + * @dev: DRM device
> > > + */
> > > +bool i915_scheduler_is_enabled(struct drm_device *dev)
> > > +{
> > > + struct drm_i915_private *dev_priv = dev->dev_private;
> > > +
> > > + return dev_priv->scheduler != NULL;
> > > +}
> > i915.enable_scheduler would be used instead.
> It is when it arrives.
>
> >
> >
> > >
> > > +
> > > +/**
> > > + * i915_scheduler_init - Initialise the scheduler.
> > > + * @dev: DRM device
> > > + * Returns zero on success or -ENOMEM if memory allocations
> > > fail.
> > > + */
> > > +int i915_scheduler_init(struct drm_device *dev)
> > > +{
> > > + struct drm_i915_private *dev_priv = dev->dev_private;
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + int r;
> > > +
> > This protection -->
> >
> > >
> > > + if (scheduler)
> > > + return 0;
> > > +
> > > + scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
> > > + if (!scheduler)
> > > + return -ENOMEM;
> > > +
> > Is not needed if we have data member and i915.enable_scheduler.
> Making the scheduler structure a object rather than a pointer
> basically
> means moving the whole of i915_scheduler.h into i915_drv.h. There is
> too
> much interconnectedness between the various structures. I would say
> that
> keeping the code nicely partitioned and readable is worth the one
> dynamic allocation at driver load time.
>
> >
> >
> > >
> > > + spin_lock_init(&scheduler->lock);
> > > +
> > > + for (r = 0; r < I915_NUM_RINGS; r++)
> > > + INIT_LIST_HEAD(&scheduler->node_queue[r]);
> > > +
> > > + /* Default tuning values: */
> > > + scheduler->priority_level_min = -1023;
> > > + scheduler->priority_level_max = 1023;
> > > + scheduler->priority_level_bump = 50;
> > > + scheduler->priority_level_preempt = 900;
> > > + scheduler->min_flying = 2;
> > > +
> > > + dev_priv->scheduler = scheduler;
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +/*
> > > + * Add a popped node back in to the queue. For example, because
> > > the ring was
> > > + * hung when execfinal() was called and thus the ring submission
> > > needs to be
> > > + * retried later.
> > > + */
> > > +static void i915_scheduler_node_requeue(struct
> > > i915_scheduler_queue_entry *node)
> > > +{
> > > + WARN_ON(!node);
> > Rather remove this completely as it is an internal function, or use
> > BUG_ON because next line will straight lead to OOPS after warning.
> > I'll
> > pass mentioning rest of the obvious WARN_ON vs BUG_ON's.
> Yeah, most of the WARN/BUG_ONs are probably unnecessary now. They
> were
> scattered liberally around during initial development when strange
> and
> unexpected things could happen. Now the the code paths are pretty
> well
> established they aren't really much use. I'll reduce them down to
> just
> the ones that still make sense.
>
>
> >
> >
> > >
> > > + WARN_ON(!I915_SQS_IS_FLYING(node));
> > > +
> > > + /* Seqno will be reassigned on relaunch */
> > > + node->params.request->seqno = 0;
> > > + node->status = i915_sqs_queued;
> > > +}
> > > +
> > > +/*
> > > + * Give up on a node completely. For example, because it is
> > > causing the
> > > + * ring to hang or is using some resource that no longer exists.
> > > + */
> > > +static void i915_scheduler_node_kill(struct
> > > i915_scheduler_queue_entry *node)
> > > +{
> > > + WARN_ON(!node);
> > > + WARN_ON(I915_SQS_IS_COMPLETE(node));
> > > +
> > > + node->status = i915_sqs_dead;
> > > +}
> > > +
> > > +/* Mark a node as in flight on the hardware. */
> > As for documentation, from below, I only assume scheduler lock must
> > be
> > held and node's can be only manipulated when scheduler lock is
> > held?
> > It's good to add a comment describing the expected locking. Or
> > maybe
> > add assert_scheduler_lock_held() helper and sprinkle it around the
> > functions similar to assert_rpm_wakelock_held(), which is self-
> > documenting?
> >
> > >
> > > +static int i915_scheduler_node_fly(struct
> > > i915_scheduler_queue_entry *node)
> > > +{
> > > + struct drm_i915_private *dev_priv = node->params.dev-
> > > >dev_private;
> > If node is NULL, this would already OOPS, I do not think it's
> > unreasonable to expect node not to be NULL. The below WARN_ON check
> > is
> > never reached.
> >
> > >
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + struct intel_engine_cs *ring;
> > > +
> > > + WARN_ON(!scheduler);
> > > + WARN_ON(!node);
> > > + WARN_ON(node->status != i915_sqs_popped);
> > > +
> > > + ring = node->params.ring;
> > > +
> > This and similar assignments can be squashed to declaration line,
> > as
> > this is basically an alias.
> >
> > >
> > > + /*
> > > + * Add the node (which should currently be in state
> > > popped) to the
> > > + * front of the queue. This ensure that flying nodes are
> > > always held
> > > + * in hardware submission order.
> > > + */
> > > + list_add(&node->link, &scheduler->node_queue[ring->id]);
> > > +
> > > + node->status = i915_sqs_flying;
> > > +
> > > + if (!(scheduler->flags[ring->id] &
> > > i915_sf_interrupts_enabled)) {
> > > + bool success = true;
> > > +
> > > + success = ring->irq_get(ring);
> > > + if (success)
> > > + scheduler->flags[ring->id] |=
> > > i915_sf_interrupts_enabled;
> > > + else
> > > + return -EINVAL;
> > Shouldn't the node be cleaned up from node_queue in case of an
> > error?
> Not really. The batch buffer will still be executed regardless of
> whether interrupts could be enabled or not. The return code is
> actually
> ignored so this should really be a void function, I guess. Without
> interrupts then completion might take a while to be noticed but
> stuff
> should still basically work.
>
>
> >
> >
> > Also, I think above could be written much more compact form
> > (because it
> > looks like something where more logic will be added later). It
> > makes it
> > easier to write and visualize the error paths when reading if there
> > are
> > no nested if's.
> >
> > I won't mention about the error paths of functions below, I expect
> > the
> > following guidline to be adhered;
> >
> > if (scheduler->flags[ring->id] & I915_SF_INT_ENABLED)
> > goto out;
> >
> > if (!ring->irq_get(ring))
> > goto err_irq;
> >
> > if (!...)
> > goto err_foo;
> >
> > scheduler->flags[ring->id] |= I915_SF_INT_ENABLED;
> > out:
> > return 0;
> > err_foo:
> > foobar();
> > err_irq:
> > list_remove()
> > return -EINVAL;
> >
> > >
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static uint32_t i915_scheduler_count_flying(struct
> > > i915_scheduler *scheduler,
> > > + struct
> > > intel_engine_cs *ring)
> > > +{
> > > + struct i915_scheduler_queue_entry *node;
> > > + uint32_t flying = 0;
> > > +
> > > + list_for_each_entry(node, &scheduler->node_queue[ring-
> > > >id], link)
> > +1 for the for_each_scheduler_node(...)
> >
> > >
> > > + if (I915_SQS_IS_FLYING(node))
> > > + flying++;
> > > +
> > > + return flying;
> > > +}
> > > +
> > > +static void i915_scheduler_priority_bump_clear(struct
> > > i915_scheduler *scheduler)
> > > +{
> > > + struct i915_scheduler_queue_entry *node;
> > > + int i;
> > > +
> > > + /*
> > > + * Ensure circular dependencies don't cause problems and
> > > that a bump
> > > + * by object usage only bumps each using buffer once:
> > > + */
> > > + for (i = 0; i < I915_NUM_RINGS; i++) {
> > > + list_for_each_entry(node, &scheduler-
> > > >node_queue[i], link)
> > > + node->bumped = false;
> > > + }
> > > +}
> > > +
> > > +static int i915_scheduler_priority_bump(struct i915_scheduler
> > > *scheduler,
> > > + struct
> > > i915_scheduler_queue_entry *target,
> > Is there specific reason why struct i915_scheduler_queue_entry are
> > not
> > referred to just as "node" but "qe" and here something else, do
> > "node"
> > and "qe" have a special semantic meaning?
> Not sure what you mean. The use of 'node' is because almost
> everything
> an i915_scheduler_queue_entry object is being used it is as a node in
> a
> list that is being iterated over. The only 'qe' variable is in
> 'i915_scheduler_queue_execbuffer[_bypass]' and there it is because
> the
> qe is being passed in from outside and is copied to a local 'node'
> that
> is then added to the list. I think the use of 'target' here is
> largely
> historical. The subsequent array search was originally a list search
> and
> hence had a 'node' that was used as the iterator. Thus the qe passed
> in
> was called something else to avoid conflicts and target seemed
> appropriate.
>
> >
> > >
> > > + uint32_t bump)
> > > +{
> > > + uint32_t new_priority;
> > > + int i, count;
> > > +
> > > + if (target->priority >= scheduler->priority_level_max)
> > > + return 1;
> > So if one node reaches maximum priority the dependencies are
> > expected
> > to be maxed out already?
> Yes. Dependencies should always be of equal or higher priority to
> the
> dependee. When a new dependency is created, the priority of the
> dependee
> is added on to ensure that something of high priority can never be
> stuck
> waiting on something of low priority. Also, it means that a batch
> buffer
> that lots of other buffers are waiting on will be bumped to higher
> and
> higher priority the more waiters there are and thus is more likely to
> be
> executed sooner and free them all up.
>
>
> >
> > >
> > > +
> > > + if (target->bumped)
> > > + return 0;
> > > +
> > > + new_priority = target->priority + bump;
> > > + if ((new_priority <= target->priority) ||
> > > + (new_priority > scheduler->priority_level_max))
> > > + target->priority = scheduler-
> > > >priority_level_max;
> > > + else
> > > + target->priority = new_priority;
> > > +
> > > + count = 1;
> > > + target->bumped = true;
> > > +
> > > + for (i = 0; i < target->num_deps; i++) {
> > > + if (!target->dep_list[i])
> > > + continue;
> > > +
> > > + if (target->dep_list[i]->bumped)
> > > + continue;
> > > +
> > > + count += i915_scheduler_priority_bump(scheduler,
> > > + target-
> > > >dep_list[i],
> > > + bump);
> > > + }
> > > +
> > > + return count;
> > > +}
> > > +
> > > +/*
> > > + * Nodes are considered valid dependencies if they are queued on
> > > any ring or
> > > + * if they are in flight on a different ring. In flight on the
> > > same ring is no
> > > + * longer interesting for non-premptive nodes as the ring
> > > serialises execution.
> > > + * For pre-empting nodes, all in flight dependencies are valid
> > > as they must not
> > > + * be jumped by the act of pre-empting.
> > > + *
> > > + * Anything that is neither queued nor flying is uninteresting.
> > > + */
> > > +static inline bool i915_scheduler_is_dependency_valid(
> > > + struct i915_scheduler_queue_entry *node,
> > > uint32_t idx)
> > > +{
> > > + struct i915_scheduler_queue_entry *dep;
> > > +
> > > + dep = node->dep_list[idx];
> > > + if (!dep)
> > > + return false;
> > > +
> > > + if (I915_SQS_IS_QUEUED(dep))
> > > + return true;
> > > +
> > > + if (I915_SQS_IS_FLYING(dep)) {
> > > + if (node->params.ring != dep->params.ring)
> > > + return true;
> > > + }
> > > +
> > > + return false;
> > > +}
> > > +
> > > +static int i915_scheduler_pop_from_queue_locked(struct
> > > intel_engine_cs *ring,
> > > + struct
> > > i915_scheduler_queue_entry **pop_node)
> > > +{
> > > + struct drm_i915_private *dev_priv = ring->dev-
> > > >dev_private;
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + struct i915_scheduler_queue_entry *best = NULL;
> > > + struct i915_scheduler_queue_entry *node;
> > > + int ret;
> > > + int i;
> > > + bool any_queued = false;
> > > + bool has_local, has_remote, only_remote = false;
> > > +
> > > + *pop_node = NULL;
> > > + ret = -ENODATA;
> > > +
> > > + list_for_each_entry(node, &scheduler->node_queue[ring-
> > > >id], link) {
> > > + if (!I915_SQS_IS_QUEUED(node))
> > > + continue;
> > > + any_queued = true;
> > > +
> > > + has_local = false;
> > > + has_remote = false;
> > > + for (i = 0; i < node->num_deps; i++) {
> > > + if
> > > (!i915_scheduler_is_dependency_valid(node, i))
> > > + continue;
> > > +
> > > + if (node->dep_list[i]->params.ring ==
> > > node->params.ring)
> > > + has_local = true;
> > > + else
> > > + has_remote = true;
> > > + }
> > > +
> > > + if (has_remote && !has_local)
> > > + only_remote = true;
> > > +
> > > + if (!has_local && !has_remote) {
> > > + if (!best ||
> > > + (node->priority > best->priority))
> > > + best = node;
> > > + }
> > > + }
> > > +
> > > + if (best) {
> > > + list_del(&best->link);
> > > +
> > > + INIT_LIST_HEAD(&best->link);
> > > + best->status = i915_sqs_popped;
> > > +
> > > + ret = 0;
> > > + } else {
> > > + /* Can only get here if:
> > > + * (a) there are no buffers in the queue
> > > + * (b) all queued buffers are dependent on other
> > > buffers
> > > + * e.g. on a buffer that is in flight on a
> > > different ring
> > > + */
> > > + if (only_remote) {
> > > + /* The only dependent buffers are on
> > > another ring. */
> > > + ret = -EAGAIN;
> > > + } else if (any_queued) {
> > > + /* It seems that something has gone
> > > horribly wrong! */
> > > + DRM_ERROR("Broken dependency tracking on
> > > ring %d!\n",
> > > + (int) ring->id);
> > Would this condition be worthy WARN_ONCE(), because it will keep
> > repeating as the situation will persist?
> >
> > >
> > > + }
> > > + }
> > > +
> > > + *pop_node = best;
> > > + return ret;
> > > +}
> > > +
> > > +/*
> > > + * NB: The driver mutex lock must be held before calling this
> > > function. It is
> > > + * only really required during the actual back end submission
> > > call. However,
> > > + * attempting to acquire a mutex while holding a spin lock is a
> > > Bad Idea.
> > > + * And releasing the one before acquiring the other leads to
> > > other code
> > > + * being run and interfering.
> > > + */
> > > +static int i915_scheduler_submit(struct intel_engine_cs *ring)
> > > +{
> > > + struct drm_device *dev = ring->dev;
> > > + struct drm_i915_private *dev_priv = dev->dev_private;
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + struct i915_scheduler_queue_entry *node;
> > > + int ret, count = 0, flying;
> > > +
> > > + WARN_ON(!mutex_is_locked(&dev->struct_mutex));
> > > +
> > > + spin_lock_irq(&scheduler->lock);
> > > +
> > > + WARN_ON(scheduler->flags[ring->id] &
> > > i915_sf_submitting);
> > > + scheduler->flags[ring->id] |= i915_sf_submitting;
> > > +
> > > + /* First time around, complain if anything unexpected
> > > occurs: */
> > > + ret = i915_scheduler_pop_from_queue_locked(ring, &node);
> > > + if (ret)
> > > + goto error;
> > > +
> > > + do {
> > > + WARN_ON(!node);
> > > + WARN_ON(node->params.ring != ring);
> > > + WARN_ON(node->status != i915_sqs_popped);
> > > + count++;
> > > +
> > > + /*
> > > + * The call to pop above will have removed the
> > > node from the
> > > + * list. So add it back in and mark it as in
> > > flight.
> > > + */
> > > + i915_scheduler_node_fly(node);
> > > +
> > > + spin_unlock_irq(&scheduler->lock);
> > > + ret = dev_priv->gt.execbuf_final(&node->params);
> > > + spin_lock_irq(&scheduler->lock);
> > > +
> > > + /*
> > > + * Handle failed submission but first check that
> > > the
> > > + * watchdog/reset code has not nuked the node
> > > while we
> > > + * weren't looking:
> > > + */
> > > + if (ret && (node->status != i915_sqs_dead)) {
> > > + bool requeue = true;
> > > +
> > > + /*
> > > + * Oh dear! Either the node is broken or
> > > the ring is
> > > + * busy. So need to kill the node or
> > > requeue it and try
> > > + * again later as appropriate.
> > > + */
> > > +
> > > + switch (-ret) {
> > > + case ENODEV:
> > > + case ENOENT:
> > > + /* Fatal errors. Kill the node.
> > > */
> > > + requeue = false;
> > > + i915_scheduler_node_kill(node);
> > > + break;
> > These break indents still.
> >
> > >
> > > +
> > > + case EAGAIN:
> > > + case EBUSY:
> > > + case EIO:
> > > + case ENOMEM:
> > > + case ERESTARTSYS:
> > > + case EINTR:
> > > + /* Supposedly recoverable
> > > errors. */
> > > + break;
> > > +
> > > + default:
> > > + /*
> > > + * Assume the error is
> > > recoverable and hope
> > > + * for the best.
> > > + */
> > > + MISSING_CASE(-ret);
> > > + break;
> > > + }
> > > +
> > > + if (requeue) {
> > > + i915_scheduler_node_requeue(node
> > > );
> > > + /*
> > > + * No point spinning if the ring
> > > is currently
> > > + * unavailable so just give up
> > > and come back
> > > + * later.
> > > + */
> > > + break;
> > > + }
> > > + }
> > > +
> > > + /* Keep launching until the sky is sufficiently
> > > full. */
> > > + flying = i915_scheduler_count_flying(scheduler,
> > > ring);
> > > + if (flying >= scheduler->min_flying)
> > > + break;
> > > +
> > > + /* Grab another node and go round again... */
> > > + ret = i915_scheduler_pop_from_queue_locked(ring,
> > > &node);
> > > + } while (ret == 0);
> > > +
> > > + /* Don't complain about not being able to submit extra
> > > entries */
> > > + if (ret == -ENODATA)
> > > + ret = 0;
> > > +
> > > + /*
> > > + * Bump the priority of everything that was not
> > > submitted to prevent
> > > + * starvation of low priority tasks by a spamming high
> > > priority task.
> > > + */
> > This, this calls for an I-G-T test to make sure we're effective.
> Yeah, IGTs are under development.
>
> >
> > >
> > > + i915_scheduler_priority_bump_clear(scheduler);
> > > + list_for_each_entry(node, &scheduler->node_queue[ring-
> > > >id], link) {
> > > + if (!I915_SQS_IS_QUEUED(node))
> > > + continue;
> > > +
> > > + i915_scheduler_priority_bump(scheduler, node,
> > > + scheduler-
> > > >priority_level_bump);
> > > + }
> > > +
> > > + /* On success, return the number of buffers submitted.
> > > */
> > > + if (ret == 0)
> > > + ret = count;
> > > +
> > > +error:
> > > + scheduler->flags[ring->id] &= ~i915_sf_submitting;
> > > + spin_unlock_irq(&scheduler->lock);
> > > + return ret;
> > > +}
> > > +
> > > +static void i915_generate_dependencies(struct i915_scheduler
> > > *scheduler,
> > > + struct
> > > i915_scheduler_queue_entry *node,
> > > + uint32_t ring)
> > > +{
> > > + struct i915_scheduler_obj_entry *this, *that;
> > > + struct i915_scheduler_queue_entry *test;
> > > + int i, j;
> > > + bool found;
> > > +
> > > + list_for_each_entry(test, &scheduler->node_queue[ring],
> > > link) {
> > > + if (I915_SQS_IS_COMPLETE(test))
> > > + continue;
> > > +
> > > + /*
> > > + * Batches on the same ring for the same
> > > + * context must be kept in order.
> > > + */
> > > + found = (node->params.ctx == test->params.ctx)
> > > &&
> > > + (node->params.ring == test-
> > > >params.ring);
> > > +
> > > + /*
> > > + * Batches working on the same objects must
> > > + * be kept in order.
> > > + */
> > > + for (i = 0; (i < node->num_objs) && !found; i++)
> > > {
> > > + this = node->saved_objects + i;
> > node->objs and node->num_objs would be a better naming match.
> > num_objs
> > and saved_objects makes me think we have a bug here, indexing wrong
> > array.
> >
> > >
> > > +
> > > + for (j = 0; j < test->num_objs; j++) {
> > > + that = test->saved_objects + j;
> > > +
> > > + if (this->obj != that->obj)
> > > + continue;
> > > +
> > > + /* Only need to worry about
> > > writes */
> > > + if (this->read_only && that-
> > > >read_only)
> > > + continue;
> > > +
> > > + found = true;
> > > + break;
> > > + }
> > > + }
> > > +
> > > + if (found) {
> > > + node->dep_list[node->num_deps] = test;
> > > + node->num_deps++;
> > > + }
> > > + }
> > > +}
> > > +
> > > +static int i915_scheduler_queue_execbuffer_bypass(struct
> > > i915_scheduler_queue_entry *qe)
> > > +{
> > > + struct drm_i915_private *dev_priv = qe->params.dev-
> > > >dev_private;
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + int ret;
> > > +
> > > + scheduler->flags[qe->params.ring->id] |=
> > > i915_sf_submitting;
> > > + ret = dev_priv->gt.execbuf_final(&qe->params);
> > > + scheduler->flags[qe->params.ring->id] &=
> > > ~i915_sf_submitting;
> > > +
> > The above code makes me think of locking again, maybe document at
> > top
> > of this function.
> No need to acquire the spinlock at this point. If the scheduler is
> in
> bypass mode then there is no internal state to protect. The driver
> mutex
> lock will be held because there is only one code path to get here
> which
> is the execbuff IOCTL. The whole point of bypass mode is that it is
> a
> single contiguous execution path from IOCTL entry all the way through
> to
> hardware submission. A mutex check could be added but it seems
> unnecessary given the limited calling scope.
>
> >
> >
> > >
> > > + /*
> > > + * Don't do any clean up on failure because the caller
> > > will
> > > + * do it all anyway.
> > > + */
> > > + if (ret)
> > > + return ret;
> > > +
> > > + /* Need to release any resources held by the node: */
> > > + i915_scheduler_clean_node(qe);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +/**
> > > + * i915_scheduler_queue_execbuffer - Submit a batch buffer
> > > request to the
> > > + * scheduler.
> > > + * @qe: The batch buffer request to be queued.
> > > + * The expectation is the qe passed in is a local stack
> > > variable. This
> > > + * function will copy its contents into a freshly allocated list
> > > node. The
> > > + * new node takes ownership of said contents so the original qe
> > > should simply
> > > + * be discarded and not cleaned up (i.e. don't free memory it
> > > points to or
> > > + * dereference objects it holds). The node is added to the
> > > scheduler's queue
> > > + * and the batch buffer will be submitted to the hardware at
> > > some future
> > > + * point in time (which may be immediately, before returning or
> > > may be quite
> > > + * a lot later).
> > > + */
> > > +int i915_scheduler_queue_execbuffer(struct
> > > i915_scheduler_queue_entry *qe)
> > > +{
> > > + struct drm_i915_private *dev_priv = qe->params.dev-
> > > >dev_private;
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + struct intel_engine_cs *ring = qe->params.ring;
> > > + struct i915_scheduler_queue_entry *node;
> > > + struct i915_scheduler_queue_entry *test;
> > > + bool not_flying;
> > > + int i, r;
> > > + int incomplete = 0;
> > > +
> > > + WARN_ON(!scheduler);
> > > +
> > > + if (1/*!i915.enable_scheduler*/)
> > > + return
> > > i915_scheduler_queue_execbuffer_bypass(qe);
> > > +
> > I'm thinking, should we branch already before calling a function
> > named
> > i915_scheduler_queue_execbuffer if scheduler is disabled?
> That would require putting if(scheduler) logic in random points
> throughout the driver. Doing it this way, the driver as a whole has
> a
> single execution path which is via the scheduler. It is then up to
> the
> scheduler itself to decide whether to run in bypass mode, simple
> re-ordering mode, full pre-emption mode or any other mode we feel a
> need
> to add.
>
>
> >
> > >
> > > + node = kmalloc(sizeof(*node), GFP_KERNEL);
> > > + if (!node)
> > > + return -ENOMEM;
> > > +
> > > + *node = *qe;
> > Now I read the added documentation for the function, maybe we
> > should at
> > least zero out qe to avoid future problems?
> The execbuff success path hands the qe contents over and pretty much
> immediately drops all the way back out of the IOCTL. The failure
> path
> does the full cleanup of de-allocating all the qe stuff it had
> previously allocated. IMO there doesn't seem to be much need to zero
> out
> the structure at this point.
>
> >
> >
> > >
> > > + INIT_LIST_HEAD(&node->link);
> > > + node->status = i915_sqs_queued;
> > > + node->stamp = jiffies;
> > > + i915_gem_request_reference(node->params.request);
> > > +
> > > + WARN_ON(node->params.request->scheduler_qe);
> > > + node->params.request->scheduler_qe = node;
> > > +
> > > + /* Need to determine the number of incomplete entries in
> > > the list as
> > > + * that will be the maximum size of the dependency list.
> > > + *
> > > + * Note that the allocation must not be made with the
> > > spinlock acquired
> > > + * as kmalloc can sleep. However, the unlock/relock is
> > > safe because no
> > > + * new entries can be queued up during the unlock as the
> > > i915 driver
> > > + * mutex is still held. Entries could be removed from
> > > the list but that
> > > + * just means the dep_list will be over-allocated which
> > > is fine.
> > > + */
> > > + spin_lock_irq(&scheduler->lock);
> > -->
> >
> > >
> > > + for (r = 0; r < I915_NUM_RINGS; r++) {
> > > + list_for_each_entry(test, &scheduler-
> > > >node_queue[r], link) {
> > > + if (I915_SQS_IS_COMPLETE(test))
> > > + continue;
> > > +
> > > + incomplete++;
> > > + }
> > > + }
> > > +
> > <-- This counting structure, I still think it would be good idea to
> > make it a static helper.
> >
> > >
> > > + /* Temporarily unlock to allocate memory: */
> > > + spin_unlock_irq(&scheduler->lock);
> > > + if (incomplete) {
> > > + node->dep_list = kmalloc_array(incomplete,
> > > + sizeof(*node-
> > > >dep_list),
> > > + GFP_KERNEL);
> > > + if (!node->dep_list) {
> > > + kfree(node);
> > > + return -ENOMEM;
> > > + }
> > > + } else
> > > + node->dep_list = NULL;
> > > +
> > > + spin_lock_irq(&scheduler->lock);
> > > + node->num_deps = 0;
> > > +
> > > + if (node->dep_list) {
> > > + for (r = 0; r < I915_NUM_RINGS; r++)
> > > + i915_generate_dependencies(scheduler,
> > > node, r);
> > > +
> > > + WARN_ON(node->num_deps > incomplete);
> > > + }
> > > +
> > > + node->priority = clamp(node->priority,
> > > + scheduler->priority_level_min,
> > > + scheduler->priority_level_max);
> > > +
> > > + if ((node->priority > 0) && node->num_deps) {
> > > + i915_scheduler_priority_bump_clear(scheduler);
> > > +
> > > + for (i = 0; i < node->num_deps; i++)
> > > + i915_scheduler_priority_bump(scheduler,
> > > + node->dep_list[i], node-
> > > >priority);
> > If node is posted with maximum priority to begin with, wouldn't the
> > priority propagation stop after first level dependencies due to the
> > check in the beginning of priority_bump function?
> The recursion will stop whenever it encounters a node already at
> maximum
> priority. However, it is not starting from the newly submitted node
> but
> from that node's dependencies. If they are already at max priority
> then
> their dependencies must also be and so on down the chain and no
> further
> processing is required. On the other hand, if they are not then they
> will be bumped and their dependencies checked and so on down the
> chain.
>
> >
> > >
> > > +
> > > + list_add_tail(&node->link, &scheduler->node_queue[ring-
> > > >id]);
> > > +
> > > + not_flying = i915_scheduler_count_flying(scheduler,
> > > ring) <
> > > + scheduler-
> > > >min_flying;
> > > +
> > > + spin_unlock_irq(&scheduler->lock);
> > > +
> > > + if (not_flying)
> > > + i915_scheduler_submit(ring);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +/**
> > > + * i915_scheduler_notify_request - Notify the scheduler that the
> > > given
> > > + * request has completed on the hardware.
> > > + * @req: Request structure which has completed
> > > + * @preempt: Did it complete pre-emptively?
> > > + * A sequence number has popped out of the hardware and the
> > > request handling
> > > + * code has mapped it back to a request and will mark that
> > > request complete.
> > > + * It also calls this function to notify the scheduler about the
> > > completion
> > > + * so the scheduler's node can be updated appropriately.
> > > + * Returns true if the request is scheduler managed, false if
> > > not. The return
> > > + * value is combined for all freshly completed requests and if
> > > any were true
> > > + * then i915_scheduler_wakeup() is called so the scheduler can
> > > do further
> > > + * processing (submit more work) at the end.
> > > + */
> > > +bool i915_scheduler_notify_request(struct drm_i915_gem_request
> > > *req)
> > > +{
> > > + struct drm_i915_private *dev_priv = to_i915(req->ring-
> > > >dev);
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + struct i915_scheduler_queue_entry *node = req-
> > > >scheduler_qe;
> > > + unsigned long flags;
> > > +
> > > + if (!node)
> > > + return false;
> > > +
> > > + spin_lock_irqsave(&scheduler->lock, flags);
> > > +
> > > + WARN_ON(!I915_SQS_IS_FLYING(node));
> > > +
> > > + /* Node was in flight so mark it as complete. */
> > > + if (req->cancelled)
> > > + node->status = i915_sqs_dead;
> > > + else
> > > + node->status = i915_sqs_complete;
> > > +
> > > + spin_unlock_irqrestore(&scheduler->lock, flags);
> > > +
> > > + return true;
> > > +}
> > > +
> > > +static int i915_scheduler_remove_dependent(struct i915_scheduler
> > > *scheduler,
> > > + struct
> > > i915_scheduler_queue_entry *remove)
> > > +{
> > > + struct i915_scheduler_queue_entry *node;
> > > + int i, r;
> > > + int count = 0;
> > > +
> > > + /*
> > > + * Ensure that a node is not being removed which is
> > > still dependent
> > > + * upon other (not completed) work. If that happens, it
> > > implies
> > > + * something has gone very wrong with the dependency
> > > tracking! Note
> > > + * that there is no need to worry if this node has been
> > > explicitly
> > > + * killed for some reason - it might be being killed
> > > before it got
> > > + * sent to the hardware.
> > > + */
> > > + if (remove->status != i915_sqs_dead) {
> > > + for (i = 0; i < remove->num_deps; i++)
> > > + if ((remove->dep_list[i]) &&
> > > + (!I915_SQS_IS_COMPLETE(remove-
> > > >dep_list[i])))
> > > + count++;
> > > + WARN_ON(count);
> > > + }
> > > +
> > > + /*
> > > + * Remove this node from the dependency lists of any
> > > other node which
> > > + * might be waiting on it.
> > > + */
> > > + for (r = 0; r < I915_NUM_RINGS; r++) {
> > > + list_for_each_entry(node, &scheduler-
> > > >node_queue[r], link) {
> > > + for (i = 0; i < node->num_deps; i++) {
> > > + if (node->dep_list[i] != remove)
> > > + continue;
> > > +
> > > + node->dep_list[i] = NULL;
> > > + }
> > > + }
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +/**
> > > + * i915_scheduler_wakeup - wake the scheduler's worker thread
> > > + * @dev: DRM device
> > > + * Called at the end of seqno interrupt processing if any
> > > request has
> > > + * completed that corresponds to a scheduler node.
> > > + */
> > > +void i915_scheduler_wakeup(struct drm_device *dev)
> > > +{
> > > + /* XXX: Need to call i915_scheduler_remove() via work
> > > handler. */
> > > +}
> > > +
> > > +/**
> > > + * i915_scheduler_clean_node - free up any
> > > allocations/references
> > > + * associated with the given scheduler queue entry.
> > > + * @node: Queue entry structure which is complete
> > > + * After a give batch buffer completes on the hardware, all the
> > > information
> > > + * required to resubmit it is no longer required. However, the
> > > node entry
> > > + * itself might still be required for tracking purposes for a
> > > while longer.
> > > + * This function should be called as soon as the node is known
> > > to be complete
> > > + * so that these resources may be freed even though the node
> > > itself might
> > > + * hang around.
> > > + */
> > > +void i915_scheduler_clean_node(struct i915_scheduler_queue_entry
> > > *node)
> > > +{
> > > + if (!I915_SQS_IS_COMPLETE(node)) {
> > > + WARN(!node->params.request->cancelled,
> > > + "Cleaning active node: %d!\n", node-
> > > >status);
> > > + return;
> > > + }
> > > +
> > > + if (node->params.batch_obj) {
> > > + /*
> > > + * The batch buffer must be unpinned before it
> > > is unreferenced
> > > + * otherwise the unpin fails with a missing
> > > vma!?
> > > + */
> > > + if (node->params.dispatch_flags &
> > > I915_DISPATCH_SECURE)
> > > + i915_gem_execbuff_release_batch_obj(node
> > > ->params.batch_obj);
> > > +
> > This code seems little bit out of place.
> Not sure where else you could put it. The code itself is just a copy
> of
> the existing clean up code that was already in the execbuff code
> path.
> The cleanup can no longer be done within the execbuff call due to
> the
> batch buffer execution being arbitrarily delayed by the scheduler.
> Thus
> it is down to the scheduler to do all the necessary cleanup when it
> knows that the batch buffer is no longer required.
>
> >
> >
> > >
> > > + node->params.batch_obj = NULL;
> > > + }
> > > +
> > > + /* And anything else owned by the node: */
> > > + if (node->params.cliprects) {
> > > + kfree(node->params.cliprects);
> > > + node->params.cliprects = NULL;
> > > + }
> > > +}
> > > +
> > The below function is quite hacky, I understood this will be
> > mitigated
> > by per-ctx sequence numbering in the future?
> To be honest, I think it is already obsolete due to the previous work
> of
> converting from seqno tests everywhere to testing request
> structures.
> The point of the code below was to cope with seqno values popping out
> of
> the hardware in random order due to the re-ordering of batch buffers
> prior to execution. Whereas, the struct request conversion allowed
> the
> seqno to be late allocated and thus kept in order.
>
> Unfortunately, pre-emption confuses matters again. Although the
> intention is that seqno values will still be kept in order. It's
> just
> that a given request might have multiple different seqno values
> assigned
> to it throughout its life. The pre-emption code hasn't quite settled
> down yet though, especially with interesting new hardware planned
> for
> future chips. Thus I don't really want to rip out all of this code
> quite
> yet just in case we do still need it.
>
> As you say, per-ctx seqnos would definitely have an effect here as
> well.
> So yes, long term the intention is to remove all this nastyness.
> Short
> term, it seems best to keep it around until proven unnecessary.
>
> >
> >
> > >
> > > +static bool i915_scheduler_remove(struct i915_scheduler
> > > *scheduler,
> > > + struct intel_engine_cs *ring,
> > > + struct list_head *remove)
> > > +{
> > > + struct i915_scheduler_queue_entry *node, *node_next;
> > > + int flying = 0, queued = 0;
> > > + bool do_submit;
> > > + uint32_t min_seqno;
> > > +
> > > + spin_lock_irq(&scheduler->lock);
> > > +
> > > + /*
> > > + * In the case where the system is idle, starting
> > > 'min_seqno' from a big
> > > + * number will cause all nodes to be removed as they are
> > > now back to
> > > + * being in-order. However, this will be a problem if
> > > the last one to
> > > + * complete was actually out-of-order as the ring seqno
> > > value will be
> > > + * lower than one or more completed buffers. Thus code
> > > looking for the
> > > + * completion of said buffers will wait forever.
> > > + * Instead, use the hardware seqno as the starting
> > > point. This means
> > > + * that some buffers might be kept around even in a
> > > completely idle
> > > + * system but it should guarantee that no-one ever gets
> > > confused when
> > > + * waiting for buffer completion.
> > > + */
> > > + min_seqno = ring->get_seqno(ring, true);
> > > +
> > > + list_for_each_entry(node, &scheduler->node_queue[ring-
> > > >id], link) {
> > > + if (I915_SQS_IS_QUEUED(node))
> > > + queued++;
> > > + else if (I915_SQS_IS_FLYING(node))
> > > + flying++;
> > > + else if (I915_SQS_IS_COMPLETE(node))
> > > + continue;
> > > +
> > > + if (node->params.request->seqno == 0)
> > > + continue;
> > > +
> > > + if (!i915_seqno_passed(node->params.request-
> > > >seqno, min_seqno))
> > > + min_seqno = node->params.request->seqno;
> > > + }
> > > +
> > > + INIT_LIST_HEAD(remove);
> > > + list_for_each_entry_safe(node, node_next, &scheduler-
> > > >node_queue[ring->id], link) {
> > > + /*
> > > + * Only remove completed nodes which have a
> > > lower seqno than
> > > + * all pending nodes. While there is the
> > > possibility of the
> > > + * ring's seqno counting backwards, all higher
> > > buffers must
> > > + * be remembered so that the
> > > 'i915_seqno_passed()' test can
> > > + * report that they have in fact passed.
> > > + *
> > > + * NB: This is not true for 'dead' nodes. The
> > > GPU reset causes
> > > + * the software seqno to restart from its
> > > initial value. Thus
> > > + * the dead nodes must be removed even though
> > > their seqno values
> > > + * are potentially vastly greater than the
> > > current ring seqno.
> > > + */
> > > + if (!I915_SQS_IS_COMPLETE(node))
> > > + continue;
> > > +
> > > + if (node->status != i915_sqs_dead) {
> > > + if (i915_seqno_passed(node-
> > > >params.request->seqno, min_seqno) &&
> > > + (node->params.request->seqno !=
> > > min_seqno))
> > > + continue;
> > > + }
> > > +
> > > + list_del(&node->link);
> > > + list_add(&node->link, remove);
> > > +
> > > + /* Strip the dependency info while the mutex is
> > > still locked */
> > > + i915_scheduler_remove_dependent(scheduler,
> > > node);
> > > +
> > > + continue;
> > > + }
> > > +
> > > + /*
> > > + * Release the interrupt reference count if there are no
> > > longer any
> > > + * nodes to worry about.
> > > + */
> > > + if (!flying && !queued &&
> > > + (scheduler->flags[ring->id] &
> > > i915_sf_interrupts_enabled)) {
> > > + ring->irq_put(ring);
> > > + scheduler->flags[ring->id] &=
> > > ~i915_sf_interrupts_enabled;
> > Maybe have this piece of code as a helper? To enable interrupts and
> > disable them, maybe even a reference count?
> Not sure how much you could abstract out into a helper other than
> the
> two lines of irq_get|put + flag set|clear. The flying|queued test is
> particular to this one instance. Also, there is no need to reference
> count as the irq code does that already.
>
> >
> >
> > >
> > > + }
> > > +
> > > + /* Launch more packets now? */
> > > + do_submit = (queued > 0) && (flying < scheduler-
> > > >min_flying);
> > > +
> > > + spin_unlock_irq(&scheduler->lock);
> > > +
> > > + return do_submit;
> > > +}
> > > +
> > > +void i915_scheduler_process_work(struct intel_engine_cs *ring)
> > > +{
> > > + struct drm_i915_private *dev_priv = ring->dev-
> > > >dev_private;
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + struct i915_scheduler_queue_entry *node;
> > > + bool do_submit;
> > > + struct list_head remove;
> > > +
> > > + if (list_empty(&scheduler->node_queue[ring->id]))
> > > + return;
> > > +
> > > + /* Remove completed nodes. */
> > > + do_submit = i915_scheduler_remove(scheduler, ring,
> > > &remove);
> > > +
> > > + if (!do_submit && list_empty(&remove))
> > > + return;
> > > +
> > > + /* Need to grab the pm lock outside of the mutex lock */
> > > + if (do_submit)
> > > + intel_runtime_pm_get(dev_priv);
> > > +
> > > + mutex_lock(&ring->dev->struct_mutex);
> > > +
> > > + if (do_submit)
> > > + i915_scheduler_submit(ring);
> > > +
> > > + while (!list_empty(&remove)) {
> > > + node = list_first_entry(&remove, typeof(*node),
> > > link);
> > > + list_del(&node->link);
> > > +
> > > + /* Free up all the DRM references */
> > > + i915_scheduler_clean_node(node);
> > > +
> > > + /* And anything else owned by the node: */
> > > + node->params.request->scheduler_qe = NULL;
> > > + i915_gem_request_unreference(node-
> > > >params.request);
> > > + kfree(node->dep_list);
> > > + kfree(node);
> > Maybe have a separate helper for freeing a node, would make this
> > much
> > cleaner.
> There is a helper for freeing up the contents of the node - see
> i915_scheduler_clean_node() above. However, that function is called
> from
> multiple places and this is the only instance where the node itself
> is
> actually freed. The other callers either need to keep the node
> around
> for a while longer (i.e. until this piece of code can run) or don't
> own
> the node because it is stack allocated (in the bypass case). So the
> 'anything else' section cannot be moved into the existing clean
> function
> and any secondary helper would be single usage and only four lines
> long.
> It doesn't seem worth it.
>
> >
> >
> > >
> > > + }
> > > +
> > > + mutex_unlock(&ring->dev->struct_mutex);
> > > +
> > > + if (do_submit)
> > > + intel_runtime_pm_put(dev_priv);
> > > +}
> > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h
> > > b/drivers/gpu/drm/i915/i915_scheduler.h
> > > new file mode 100644
> > > index 0000000..415fec8
> > > --- /dev/null
> > > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > > @@ -0,0 +1,95 @@
> > > +/*
> > > + * Copyright (c) 2014 Intel Corporation
> > > + *
> > > + * Permission is hereby granted, free of charge, to any person
> > > obtaining a
> > > + * copy of this software and associated documentation files (the
> > > "Software"),
> > > + * to deal in the Software without restriction, including
> > > without limitation
> > > + * the rights to use, copy, modify, merge, publish, distribute,
> > > sublicense,
> > > + * and/or sell copies of the Software, and to permit persons to
> > > whom the
> > > + * Software is furnished to do so, subject to the following
> > > conditions:
> > > + *
> > > + * The above copyright notice and this permission notice
> > > (including the next
> > > + * paragraph) shall be included in all copies or substantial
> > > portions of the
> > > + * Software.
> > > + *
> > > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
> > > KIND, EXPRESS OR
> > > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> > > MERCHANTABILITY,
> > > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
> > > EVENT SHALL
> > > + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
> > > DAMAGES OR OTHER
> > > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
> > > OTHERWISE, ARISING
> > > + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> > > OTHER DEALINGS
> > > + * IN THE SOFTWARE.
> > > + *
> > > + */
> > > +
> > > +#ifndef _I915_SCHEDULER_H_
> > > +#define _I915_SCHEDULER_H_
> > > +
> > > +enum i915_scheduler_queue_status {
> > > + /* Limbo: */
> > > + i915_sqs_none = 0,
> > > + /* Not yet submitted to hardware: */
> > > + i915_sqs_queued,
> > > + /* Popped from queue, ready to fly: */
> > > + i915_sqs_popped,
> > > + /* Sent to hardware for processing: */
> > > + i915_sqs_flying,
> > > + /* Finished processing on the hardware: */
> > > + i915_sqs_complete,
> > > + /* Killed by watchdog or catastrophic submission
> > > failure: */
> > > + i915_sqs_dead,
> > > + /* Limit value for use with arrays/loops */
> > > + i915_sqs_MAX
> > > +};
> > > +
> > To uppercase.
> Grrr. Linux kernel style guide is daft.
>
> >
> >
> > >
> > > +#define I915_SQS_IS_QUEUED(node) (((node)->status ==
> > > i915_sqs_queued))
> > > +#define I915_SQS_IS_FLYING(node) (((node)->status ==
> > > i915_sqs_flying))
> > > +#define I915_SQS_IS_COMPLETE(node) (((node)->status ==
> > > i915_sqs_complete) || \
> > > + ((node)->status ==
> > > i915_sqs_dead))
> > > +
> > > +struct i915_scheduler_obj_entry {
> > > + struct drm_i915_gem_object *obj;
> > > + bool read_only;
> > > +};
> > > +
> > > +struct i915_scheduler_queue_entry {
> > > + struct i915_execbuffer_params params;
> > > + /* -1023 = lowest priority, 0 = default, 1023 = highest
> > > */
> > > + int32_t priority;
> > > + struct i915_scheduler_obj_entry *saved_objects;
> > > + int num_objs;
> > > + bool bumped;
> > > + struct i915_scheduler_queue_entry **dep_list;
> > > + int num_deps;
> > > + enum i915_scheduler_queue_status status;
> > > + unsigned long stamp;
> > > + struct list_head link;
> > Above whitespace is still incorrect and I could really use
> > comments.
> >
> > >
> > > +};
> > > +
> > > +struct i915_scheduler {
> > > + struct list_head node_queue[I915_NUM_RINGS];
> > > + uint32_t flags[I915_NUM_RINGS];
> > > + spinlock_t lock;
> > > +
> > > + /* Tuning parameters: */
> > > + int32_t priority_level_min;
> > > + int32_t priority_level_max;
> > > + int32_t priority_level_bump;
> > > + int32_t priority_level_preempt;
> > > + uint32_t min_flying;
> > > +};
> > > +
> > > +/* Flag bits for i915_scheduler::flags */
> > > +enum {
> > > + i915_sf_interrupts_enabled = (1 << 0),
> > > + i915_sf_submitting = (1 << 1),
> > > +};
> > These must be I915_UPPERCASE_SOMETHING by convention.
> >
> > >
> > > +
> > > +bool i915_scheduler_is_enabled(struct drm_device *dev);
> > > +int i915_scheduler_init(struct drm_device *dev);
> > > +void i915_scheduler_clean_node(struct i915_scheduler_queue_entry
> > > *node);
> > > +int i915_scheduler_queue_execbuffer(struct
> > > i915_scheduler_queue_entry *qe);
> > > +bool i915_scheduler_notify_request(struct drm_i915_gem_request
> > > *req);
> > > +void i915_scheduler_wakeup(struct drm_device *dev);
> > > +
> > > +#endif /* _I915_SCHEDULER_H_ */
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 06/35] drm/i915: Start of GPU scheduler
2016-02-26 9:13 ` Joonas Lahtinen
@ 2016-02-26 14:18 ` John Harrison
0 siblings, 0 replies; 82+ messages in thread
From: John Harrison @ 2016-02-26 14:18 UTC (permalink / raw)
To: Joonas Lahtinen, Intel-GFX
On 26/02/2016 09:13, Joonas Lahtinen wrote:
> Hi,
>
> The below answers are reasonable. So v6 should be the version.
Are you planning on looking at the other patches first or are you going
to wait until v6 is posted?
Thanks,
John.
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler
2016-02-23 20:42 ` Jesse Barnes
@ 2016-02-26 15:55 ` John Harrison
2016-02-26 17:12 ` Jesse Barnes
0 siblings, 1 reply; 82+ messages in thread
From: John Harrison @ 2016-02-26 15:55 UTC (permalink / raw)
To: Jesse Barnes, Intel-GFX
On 23/02/2016 20:42, Jesse Barnes wrote:
> On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
>> From: John Harrison <John.C.Harrison@Intel.com>
>>
>> Added trace points to the scheduler to track all the various events,
>> node state transitions and other interesting things that occur.
>>
>> v2: Updated for new request completion tracking implementation.
>>
>> v3: Updated for changes to node kill code.
>>
>> v4: Wrapped some long lines to keep the style checker happy.
>>
>> For: VIZ-1587
>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>> ---
>> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 +
>> drivers/gpu/drm/i915/i915_scheduler.c | 26 ++++
>> drivers/gpu/drm/i915/i915_trace.h | 196 +++++++++++++++++++++++++++++
>> drivers/gpu/drm/i915/intel_lrc.c | 2 +
>> 4 files changed, 226 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>> index b9ad0fd..d4de8c7 100644
>> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>> @@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
>>
>> i915_gem_execbuffer_move_to_active(vmas, params->request);
>>
>> + trace_i915_gem_ring_queue(ring, params);
>> +
>> qe = container_of(params, typeof(*qe), params);
>> ret = i915_scheduler_queue_execbuffer(qe);
>> if (ret)
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
>> index 47d7de4..e56ce08 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.c
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
>> @@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
>> /* Seqno will be reassigned on relaunch */
>> node->params.request->seqno = 0;
>> node->status = i915_sqs_queued;
>> + trace_i915_scheduler_unfly(node->params.ring, node);
>> + trace_i915_scheduler_node_state_change(node->params.ring, node);
>> }
>>
>> /*
>> @@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
>> WARN_ON(!node);
>> WARN_ON(I915_SQS_IS_COMPLETE(node));
>>
>> + if (I915_SQS_IS_FLYING(node))
>> + trace_i915_scheduler_unfly(node->params.ring, node);
>> +
>> node->status = i915_sqs_dead;
>> + trace_i915_scheduler_node_state_change(node->params.ring, node);
>> }
>>
>> /* Mark a node as in flight on the hardware. */
>> @@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
>>
>> node->status = i915_sqs_flying;
>>
>> + trace_i915_scheduler_fly(ring, node);
>> + trace_i915_scheduler_node_state_change(ring, node);
>> +
>> if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
>> bool success = true;
>>
>> @@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
>> INIT_LIST_HEAD(&best->link);
>> best->status = i915_sqs_popped;
>>
>> + trace_i915_scheduler_node_state_change(ring, best);
>> +
>> ret = 0;
>> } else {
>> /* Can only get here if:
>> @@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
>> }
>> }
>>
>> + trace_i915_scheduler_pop_from_queue(ring, best);
>> +
>> *pop_node = best;
>> return ret;
>> }
>> @@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
>> struct i915_scheduler *scheduler = dev_priv->scheduler;
>> int ret;
>>
>> + trace_i915_scheduler_queue(qe->params.ring, qe);
>> +
>> intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
>>
>> scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
>> @@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
>> not_flying = i915_scheduler_count_flying(scheduler, ring) <
>> scheduler->min_flying;
>>
>> + trace_i915_scheduler_queue(ring, node);
>> + trace_i915_scheduler_node_state_change(ring, node);
>> +
>> spin_unlock_irq(&scheduler->lock);
>>
>> if (not_flying)
>> @@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
>> struct i915_scheduler_queue_entry *node = req->scheduler_qe;
>> unsigned long flags;
>>
>> + trace_i915_scheduler_landing(req);
>> +
>> if (!node)
>> return false;
>>
>> @@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
>> else
>> node->status = i915_sqs_complete;
>>
>> + trace_i915_scheduler_node_state_change(req->ring, node);
>> +
>> spin_unlock_irqrestore(&scheduler->lock, flags);
>>
>> return true;
>> @@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
>> /* Launch more packets now? */
>> do_submit = (queued > 0) && (flying < scheduler->min_flying);
>>
>> + trace_i915_scheduler_remove(ring, min_seqno, do_submit);
>> +
>> spin_unlock_irq(&scheduler->lock);
>>
>> return do_submit;
>> @@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring)
>> node = list_first_entry(&remove, typeof(*node), link);
>> list_del(&node->link);
>>
>> + trace_i915_scheduler_destroy(ring, node);
>> +
>> /* Free up all the DRM references */
>> i915_scheduler_clean_node(node);
>>
>> diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
>> index 455c215..c3c4e58 100644
>> --- a/drivers/gpu/drm/i915/i915_trace.h
>> +++ b/drivers/gpu/drm/i915/i915_trace.h
>> @@ -9,6 +9,7 @@
>> #include "i915_drv.h"
>> #include "intel_drv.h"
>> #include "intel_ringbuffer.h"
>> +#include "i915_scheduler.h"
>>
>> #undef TRACE_SYSTEM
>> #define TRACE_SYSTEM i915
>> @@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm,
>> __entry->dev, __entry->ring, __entry->to, __entry->vm)
>> );
>>
>> +TRACE_EVENT(i915_scheduler_queue,
>> + TP_PROTO(struct intel_engine_cs *ring,
>> + struct i915_scheduler_queue_entry *node),
>> + TP_ARGS(ring, node),
>> +
>> + TP_STRUCT__entry(
>> + __field(u32, ring)
>> + __field(u32, uniq)
>> + __field(u32, seqno)
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->ring = ring->id;
>> + __entry->uniq = node ? node->params.request->uniq : 0;
>> + __entry->seqno = node ? node->params.request->seqno : 0;
>> + ),
>> +
>> + TP_printk("ring=%d, uniq=%d, seqno=%d",
>> + __entry->ring, __entry->uniq, __entry->seqno)
>> +);
>> +
>> +TRACE_EVENT(i915_scheduler_fly,
>> + TP_PROTO(struct intel_engine_cs *ring,
>> + struct i915_scheduler_queue_entry *node),
>> + TP_ARGS(ring, node),
>> +
>> + TP_STRUCT__entry(
>> + __field(u32, ring)
>> + __field(u32, uniq)
>> + __field(u32, seqno)
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->ring = ring->id;
>> + __entry->uniq = node ? node->params.request->uniq : 0;
>> + __entry->seqno = node ? node->params.request->seqno : 0;
>> + ),
>> +
>> + TP_printk("ring=%d, uniq=%d, seqno=%d",
>> + __entry->ring, __entry->uniq, __entry->seqno)
>> +);
>> +
>> +TRACE_EVENT(i915_scheduler_unfly,
>> + TP_PROTO(struct intel_engine_cs *ring,
>> + struct i915_scheduler_queue_entry *node),
>> + TP_ARGS(ring, node),
>> +
>> + TP_STRUCT__entry(
>> + __field(u32, ring)
>> + __field(u32, uniq)
>> + __field(u32, seqno)
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->ring = ring->id;
>> + __entry->uniq = node ? node->params.request->uniq : 0;
>> + __entry->seqno = node ? node->params.request->seqno : 0;
>> + ),
>> +
>> + TP_printk("ring=%d, uniq=%d, seqno=%d",
>> + __entry->ring, __entry->uniq, __entry->seqno)
>> +);
>> +
>> +TRACE_EVENT(i915_scheduler_landing,
>> + TP_PROTO(struct drm_i915_gem_request *req),
>> + TP_ARGS(req),
>> +
>> + TP_STRUCT__entry(
>> + __field(u32, ring)
>> + __field(u32, uniq)
>> + __field(u32, seqno)
>> + __field(u32, status)
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->ring = req->ring->id;
>> + __entry->uniq = req->uniq;
>> + __entry->seqno = req->seqno;
>> + __entry->status = req->scheduler_qe ?
>> + req->scheduler_qe->status : ~0U;
>> + ),
>> +
>> + TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
>> + __entry->ring, __entry->uniq, __entry->seqno,
>> + __entry->status)
>> +);
>> +
>> +TRACE_EVENT(i915_scheduler_remove,
>> + TP_PROTO(struct intel_engine_cs *ring,
>> + u32 min_seqno, bool do_submit),
>> + TP_ARGS(ring, min_seqno, do_submit),
>> +
>> + TP_STRUCT__entry(
>> + __field(u32, ring)
>> + __field(u32, min_seqno)
>> + __field(bool, do_submit)
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->ring = ring->id;
>> + __entry->min_seqno = min_seqno;
>> + __entry->do_submit = do_submit;
>> + ),
>> +
>> + TP_printk("ring=%d, min_seqno = %d, do_submit=%d",
>> + __entry->ring, __entry->min_seqno, __entry->do_submit)
>> +);
>> +
>> +TRACE_EVENT(i915_scheduler_destroy,
>> + TP_PROTO(struct intel_engine_cs *ring,
>> + struct i915_scheduler_queue_entry *node),
>> + TP_ARGS(ring, node),
>> +
>> + TP_STRUCT__entry(
>> + __field(u32, ring)
>> + __field(u32, uniq)
>> + __field(u32, seqno)
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->ring = ring->id;
>> + __entry->uniq = node ? node->params.request->uniq : 0;
>> + __entry->seqno = node ? node->params.request->seqno : 0;
>> + ),
>> +
>> + TP_printk("ring=%d, uniq=%d, seqno=%d",
>> + __entry->ring, __entry->uniq, __entry->seqno)
>> +);
>> +
>> +TRACE_EVENT(i915_scheduler_pop_from_queue,
>> + TP_PROTO(struct intel_engine_cs *ring,
>> + struct i915_scheduler_queue_entry *node),
>> + TP_ARGS(ring, node),
>> +
>> + TP_STRUCT__entry(
>> + __field(u32, ring)
>> + __field(u32, uniq)
>> + __field(u32, seqno)
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->ring = ring->id;
>> + __entry->uniq = node ? node->params.request->uniq : 0;
>> + __entry->seqno = node ? node->params.request->seqno : 0;
>> + ),
>> +
>> + TP_printk("ring=%d, uniq=%d, seqno=%d",
>> + __entry->ring, __entry->uniq, __entry->seqno)
>> +);
>> +
>> +TRACE_EVENT(i915_scheduler_node_state_change,
>> + TP_PROTO(struct intel_engine_cs *ring,
>> + struct i915_scheduler_queue_entry *node),
>> + TP_ARGS(ring, node),
>> +
>> + TP_STRUCT__entry(
>> + __field(u32, ring)
>> + __field(u32, uniq)
>> + __field(u32, seqno)
>> + __field(u32, status)
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->ring = ring->id;
>> + __entry->uniq = node ? node->params.request->uniq : 0;
>> + __entry->seqno = node->params.request->seqno;
>> + __entry->status = node->status;
>> + ),
>> +
>> + TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
>> + __entry->ring, __entry->uniq, __entry->seqno,
>> + __entry->status)
>> +);
>> +
>> +TRACE_EVENT(i915_gem_ring_queue,
>> + TP_PROTO(struct intel_engine_cs *ring,
>> + struct i915_execbuffer_params *params),
>> + TP_ARGS(ring, params),
>> +
>> + TP_STRUCT__entry(
>> + __field(u32, ring)
>> + __field(u32, uniq)
>> + __field(u32, seqno)
>> + ),
>> +
>> + TP_fast_assign(
>> + __entry->ring = ring->id;
>> + __entry->uniq = params->request->uniq;
>> + __entry->seqno = params->request->seqno;
>> + ),
>> +
>> + TP_printk("ring=%d, uniq=%d, seqno=%d", __entry->ring,
>> + __entry->uniq, __entry->seqno)
>> +);
>> +
>> #endif /* _I915_TRACE_H_ */
>>
>> /* This part must be outside protection */
>> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
>> index 9c7a79a..2b9f49c 100644
>> --- a/drivers/gpu/drm/i915/intel_lrc.c
>> +++ b/drivers/gpu/drm/i915/intel_lrc.c
>> @@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
>>
>> i915_gem_execbuffer_move_to_active(vmas, params->request);
>>
>> + trace_i915_gem_ring_queue(ring, params);
>> +
>> qe = container_of(params, typeof(*qe), params);
>> ret = i915_scheduler_queue_execbuffer(qe);
>> if (ret)
>>
> Oops, forgot to ask if there are any tools for igt that make use of these, e.g. some kind of scheduler top util that looks for bubbles or starved batches or something.
Nothing at the moment. They have been used to debug issues by looking
through trace logs. Also the validation team have tests that hook in to
the various trace points to verify the internal operation of the
scheduler and other parts of the i915 driver.
>
> Jesse
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler
2016-02-26 15:55 ` John Harrison
@ 2016-02-26 17:12 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-02-26 17:12 UTC (permalink / raw)
To: John Harrison, Intel-GFX
On 02/26/2016 07:55 AM, John Harrison wrote:
> On 23/02/2016 20:42, Jesse Barnes wrote:
>> On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
>>> From: John Harrison <John.C.Harrison@Intel.com>
>>>
>>> Added trace points to the scheduler to track all the various events,
>>> node state transitions and other interesting things that occur.
>>>
>>> v2: Updated for new request completion tracking implementation.
>>>
>>> v3: Updated for changes to node kill code.
>>>
>>> v4: Wrapped some long lines to keep the style checker happy.
>>>
>>> For: VIZ-1587
>>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>>> ---
>>> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 +
>>> drivers/gpu/drm/i915/i915_scheduler.c | 26 ++++
>>> drivers/gpu/drm/i915/i915_trace.h | 196 +++++++++++++++++++++++++++++
>>> drivers/gpu/drm/i915/intel_lrc.c | 2 +
>>> 4 files changed, 226 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>>> index b9ad0fd..d4de8c7 100644
>>> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>>> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>>> @@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
>>> i915_gem_execbuffer_move_to_active(vmas, params->request);
>>> + trace_i915_gem_ring_queue(ring, params);
>>> +
>>> qe = container_of(params, typeof(*qe), params);
>>> ret = i915_scheduler_queue_execbuffer(qe);
>>> if (ret)
>>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
>>> index 47d7de4..e56ce08 100644
>>> --- a/drivers/gpu/drm/i915/i915_scheduler.c
>>> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
>>> @@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
>>> /* Seqno will be reassigned on relaunch */
>>> node->params.request->seqno = 0;
>>> node->status = i915_sqs_queued;
>>> + trace_i915_scheduler_unfly(node->params.ring, node);
>>> + trace_i915_scheduler_node_state_change(node->params.ring, node);
>>> }
>>> /*
>>> @@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
>>> WARN_ON(!node);
>>> WARN_ON(I915_SQS_IS_COMPLETE(node));
>>> + if (I915_SQS_IS_FLYING(node))
>>> + trace_i915_scheduler_unfly(node->params.ring, node);
>>> +
>>> node->status = i915_sqs_dead;
>>> + trace_i915_scheduler_node_state_change(node->params.ring, node);
>>> }
>>> /* Mark a node as in flight on the hardware. */
>>> @@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
>>> node->status = i915_sqs_flying;
>>> + trace_i915_scheduler_fly(ring, node);
>>> + trace_i915_scheduler_node_state_change(ring, node);
>>> +
>>> if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
>>> bool success = true;
>>> @@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
>>> INIT_LIST_HEAD(&best->link);
>>> best->status = i915_sqs_popped;
>>> + trace_i915_scheduler_node_state_change(ring, best);
>>> +
>>> ret = 0;
>>> } else {
>>> /* Can only get here if:
>>> @@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
>>> }
>>> }
>>> + trace_i915_scheduler_pop_from_queue(ring, best);
>>> +
>>> *pop_node = best;
>>> return ret;
>>> }
>>> @@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
>>> struct i915_scheduler *scheduler = dev_priv->scheduler;
>>> int ret;
>>> + trace_i915_scheduler_queue(qe->params.ring, qe);
>>> +
>>> intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
>>> scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
>>> @@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
>>> not_flying = i915_scheduler_count_flying(scheduler, ring) <
>>> scheduler->min_flying;
>>> + trace_i915_scheduler_queue(ring, node);
>>> + trace_i915_scheduler_node_state_change(ring, node);
>>> +
>>> spin_unlock_irq(&scheduler->lock);
>>> if (not_flying)
>>> @@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
>>> struct i915_scheduler_queue_entry *node = req->scheduler_qe;
>>> unsigned long flags;
>>> + trace_i915_scheduler_landing(req);
>>> +
>>> if (!node)
>>> return false;
>>> @@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
>>> else
>>> node->status = i915_sqs_complete;
>>> + trace_i915_scheduler_node_state_change(req->ring, node);
>>> +
>>> spin_unlock_irqrestore(&scheduler->lock, flags);
>>> return true;
>>> @@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
>>> /* Launch more packets now? */
>>> do_submit = (queued > 0) && (flying < scheduler->min_flying);
>>> + trace_i915_scheduler_remove(ring, min_seqno, do_submit);
>>> +
>>> spin_unlock_irq(&scheduler->lock);
>>> return do_submit;
>>> @@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring)
>>> node = list_first_entry(&remove, typeof(*node), link);
>>> list_del(&node->link);
>>> + trace_i915_scheduler_destroy(ring, node);
>>> +
>>> /* Free up all the DRM references */
>>> i915_scheduler_clean_node(node);
>>> diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
>>> index 455c215..c3c4e58 100644
>>> --- a/drivers/gpu/drm/i915/i915_trace.h
>>> +++ b/drivers/gpu/drm/i915/i915_trace.h
>>> @@ -9,6 +9,7 @@
>>> #include "i915_drv.h"
>>> #include "intel_drv.h"
>>> #include "intel_ringbuffer.h"
>>> +#include "i915_scheduler.h"
>>> #undef TRACE_SYSTEM
>>> #define TRACE_SYSTEM i915
>>> @@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm,
>>> __entry->dev, __entry->ring, __entry->to, __entry->vm)
>>> );
>>> +TRACE_EVENT(i915_scheduler_queue,
>>> + TP_PROTO(struct intel_engine_cs *ring,
>>> + struct i915_scheduler_queue_entry *node),
>>> + TP_ARGS(ring, node),
>>> +
>>> + TP_STRUCT__entry(
>>> + __field(u32, ring)
>>> + __field(u32, uniq)
>>> + __field(u32, seqno)
>>> + ),
>>> +
>>> + TP_fast_assign(
>>> + __entry->ring = ring->id;
>>> + __entry->uniq = node ? node->params.request->uniq : 0;
>>> + __entry->seqno = node ? node->params.request->seqno : 0;
>>> + ),
>>> +
>>> + TP_printk("ring=%d, uniq=%d, seqno=%d",
>>> + __entry->ring, __entry->uniq, __entry->seqno)
>>> +);
>>> +
>>> +TRACE_EVENT(i915_scheduler_fly,
>>> + TP_PROTO(struct intel_engine_cs *ring,
>>> + struct i915_scheduler_queue_entry *node),
>>> + TP_ARGS(ring, node),
>>> +
>>> + TP_STRUCT__entry(
>>> + __field(u32, ring)
>>> + __field(u32, uniq)
>>> + __field(u32, seqno)
>>> + ),
>>> +
>>> + TP_fast_assign(
>>> + __entry->ring = ring->id;
>>> + __entry->uniq = node ? node->params.request->uniq : 0;
>>> + __entry->seqno = node ? node->params.request->seqno : 0;
>>> + ),
>>> +
>>> + TP_printk("ring=%d, uniq=%d, seqno=%d",
>>> + __entry->ring, __entry->uniq, __entry->seqno)
>>> +);
>>> +
>>> +TRACE_EVENT(i915_scheduler_unfly,
>>> + TP_PROTO(struct intel_engine_cs *ring,
>>> + struct i915_scheduler_queue_entry *node),
>>> + TP_ARGS(ring, node),
>>> +
>>> + TP_STRUCT__entry(
>>> + __field(u32, ring)
>>> + __field(u32, uniq)
>>> + __field(u32, seqno)
>>> + ),
>>> +
>>> + TP_fast_assign(
>>> + __entry->ring = ring->id;
>>> + __entry->uniq = node ? node->params.request->uniq : 0;
>>> + __entry->seqno = node ? node->params.request->seqno : 0;
>>> + ),
>>> +
>>> + TP_printk("ring=%d, uniq=%d, seqno=%d",
>>> + __entry->ring, __entry->uniq, __entry->seqno)
>>> +);
>>> +
>>> +TRACE_EVENT(i915_scheduler_landing,
>>> + TP_PROTO(struct drm_i915_gem_request *req),
>>> + TP_ARGS(req),
>>> +
>>> + TP_STRUCT__entry(
>>> + __field(u32, ring)
>>> + __field(u32, uniq)
>>> + __field(u32, seqno)
>>> + __field(u32, status)
>>> + ),
>>> +
>>> + TP_fast_assign(
>>> + __entry->ring = req->ring->id;
>>> + __entry->uniq = req->uniq;
>>> + __entry->seqno = req->seqno;
>>> + __entry->status = req->scheduler_qe ?
>>> + req->scheduler_qe->status : ~0U;
>>> + ),
>>> +
>>> + TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
>>> + __entry->ring, __entry->uniq, __entry->seqno,
>>> + __entry->status)
>>> +);
>>> +
>>> +TRACE_EVENT(i915_scheduler_remove,
>>> + TP_PROTO(struct intel_engine_cs *ring,
>>> + u32 min_seqno, bool do_submit),
>>> + TP_ARGS(ring, min_seqno, do_submit),
>>> +
>>> + TP_STRUCT__entry(
>>> + __field(u32, ring)
>>> + __field(u32, min_seqno)
>>> + __field(bool, do_submit)
>>> + ),
>>> +
>>> + TP_fast_assign(
>>> + __entry->ring = ring->id;
>>> + __entry->min_seqno = min_seqno;
>>> + __entry->do_submit = do_submit;
>>> + ),
>>> +
>>> + TP_printk("ring=%d, min_seqno = %d, do_submit=%d",
>>> + __entry->ring, __entry->min_seqno, __entry->do_submit)
>>> +);
>>> +
>>> +TRACE_EVENT(i915_scheduler_destroy,
>>> + TP_PROTO(struct intel_engine_cs *ring,
>>> + struct i915_scheduler_queue_entry *node),
>>> + TP_ARGS(ring, node),
>>> +
>>> + TP_STRUCT__entry(
>>> + __field(u32, ring)
>>> + __field(u32, uniq)
>>> + __field(u32, seqno)
>>> + ),
>>> +
>>> + TP_fast_assign(
>>> + __entry->ring = ring->id;
>>> + __entry->uniq = node ? node->params.request->uniq : 0;
>>> + __entry->seqno = node ? node->params.request->seqno : 0;
>>> + ),
>>> +
>>> + TP_printk("ring=%d, uniq=%d, seqno=%d",
>>> + __entry->ring, __entry->uniq, __entry->seqno)
>>> +);
>>> +
>>> +TRACE_EVENT(i915_scheduler_pop_from_queue,
>>> + TP_PROTO(struct intel_engine_cs *ring,
>>> + struct i915_scheduler_queue_entry *node),
>>> + TP_ARGS(ring, node),
>>> +
>>> + TP_STRUCT__entry(
>>> + __field(u32, ring)
>>> + __field(u32, uniq)
>>> + __field(u32, seqno)
>>> + ),
>>> +
>>> + TP_fast_assign(
>>> + __entry->ring = ring->id;
>>> + __entry->uniq = node ? node->params.request->uniq : 0;
>>> + __entry->seqno = node ? node->params.request->seqno : 0;
>>> + ),
>>> +
>>> + TP_printk("ring=%d, uniq=%d, seqno=%d",
>>> + __entry->ring, __entry->uniq, __entry->seqno)
>>> +);
>>> +
>>> +TRACE_EVENT(i915_scheduler_node_state_change,
>>> + TP_PROTO(struct intel_engine_cs *ring,
>>> + struct i915_scheduler_queue_entry *node),
>>> + TP_ARGS(ring, node),
>>> +
>>> + TP_STRUCT__entry(
>>> + __field(u32, ring)
>>> + __field(u32, uniq)
>>> + __field(u32, seqno)
>>> + __field(u32, status)
>>> + ),
>>> +
>>> + TP_fast_assign(
>>> + __entry->ring = ring->id;
>>> + __entry->uniq = node ? node->params.request->uniq : 0;
>>> + __entry->seqno = node->params.request->seqno;
>>> + __entry->status = node->status;
>>> + ),
>>> +
>>> + TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
>>> + __entry->ring, __entry->uniq, __entry->seqno,
>>> + __entry->status)
>>> +);
>>> +
>>> +TRACE_EVENT(i915_gem_ring_queue,
>>> + TP_PROTO(struct intel_engine_cs *ring,
>>> + struct i915_execbuffer_params *params),
>>> + TP_ARGS(ring, params),
>>> +
>>> + TP_STRUCT__entry(
>>> + __field(u32, ring)
>>> + __field(u32, uniq)
>>> + __field(u32, seqno)
>>> + ),
>>> +
>>> + TP_fast_assign(
>>> + __entry->ring = ring->id;
>>> + __entry->uniq = params->request->uniq;
>>> + __entry->seqno = params->request->seqno;
>>> + ),
>>> +
>>> + TP_printk("ring=%d, uniq=%d, seqno=%d", __entry->ring,
>>> + __entry->uniq, __entry->seqno)
>>> +);
>>> +
>>> #endif /* _I915_TRACE_H_ */
>>> /* This part must be outside protection */
>>> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
>>> index 9c7a79a..2b9f49c 100644
>>> --- a/drivers/gpu/drm/i915/intel_lrc.c
>>> +++ b/drivers/gpu/drm/i915/intel_lrc.c
>>> @@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
>>> i915_gem_execbuffer_move_to_active(vmas, params->request);
>>> + trace_i915_gem_ring_queue(ring, params);
>>> +
>>> qe = container_of(params, typeof(*qe), params);
>>> ret = i915_scheduler_queue_execbuffer(qe);
>>> if (ret)
>>>
>> Oops, forgot to ask if there are any tools for igt that make use of these, e.g. some kind of scheduler top util that looks for bubbles or starved batches or something.
> Nothing at the moment. They have been used to debug issues by looking through trace logs. Also the validation team have tests that hook in to the various trace points to verify the internal operation of the scheduler and other parts of the i915 driver.
Nice. I wasn't asking because it's a blocker or anything, just curious.
Thanks,
Jesse
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles
2016-02-18 14:26 ` [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles John.C.Harrison
@ 2016-03-01 8:59 ` Joonas Lahtinen
2016-03-01 14:52 ` John Harrison
0 siblings, 1 reply; 82+ messages in thread
From: Joonas Lahtinen @ 2016-03-01 8:59 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On to, 2016-02-18 at 14:26 +0000, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> The scheduler decouples the submission of batch buffers to the driver
> with submission of batch buffers to the hardware. Thus it is possible
> for an application to close its DRM file handle while there is still
> work outstanding. That means the scheduler needs to know about file
> close events so it can remove the file pointer from such orphaned
> batch buffers and not attempt to dereference it later.
>
> v3: Updated to not wait for outstanding work to complete but merely
> remove the file handle reference. The wait was getting excessively
> complicated with inter-ring dependencies, pre-emption, and other such
> issues.
>
> v4: Changed some white space to keep the style checker happy.
>
> v5: Added function documentation and removed apparently objectionable
> white space. [Joonas Lahtinen]
>
> Used lighter weight spinlocks.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/i915_dma.c | 3 +++
> drivers/gpu/drm/i915/i915_scheduler.c | 48 +++++++++++++++++++++++++++++++++++
> drivers/gpu/drm/i915/i915_scheduler.h | 2 ++
> 3 files changed, 53 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
> index a0f5659..678adc7 100644
> --- a/drivers/gpu/drm/i915/i915_dma.c
> +++ b/drivers/gpu/drm/i915/i915_dma.c
> @@ -46,6 +46,7 @@
> #include
> #include
> #include
> +#include "i915_scheduler.h"
> #include
> #include
> #include
> @@ -1258,6 +1259,8 @@ void i915_driver_lastclose(struct drm_device *dev)
>
> void i915_driver_preclose(struct drm_device *dev, struct drm_file *file)
> {
> + i915_scheduler_closefile(dev, file);
Any reason not to put this inside the struct_mutex lock?
> +
> mutex_lock(&dev->struct_mutex);
> i915_gem_context_close(dev, file);
> i915_gem_release(dev, file);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index fc23ee7..ab5007a 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -872,3 +872,51 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
> if (do_submit)
> intel_runtime_pm_put(dev_priv);
> }
> +
> +/**
> + * i915_scheduler_closefile - notify the scheduler that a DRM file handle
> + * has been closed.
> + * @dev: DRM device
> + * @file: file being closed
> + *
> + * Goes through the scheduler's queues and removes all connections to the
> + * disappearing file handle that still exist. There is an argument to say
> + * that this should also flush such outstanding work through the hardware.
> + * However, with pre-emption, TDR and other such complications doing so
> + * becomes a locking nightmare. So instead, just warn with a debug message
> + * if the application is leaking uncompleted work and make sure a null
> + * pointer dereference will not follow.
> + */
> +int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
Return value not used, should be void if this can not fail.
Other than that,
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> +{
> + struct i915_scheduler_queue_entry *node;
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct intel_engine_cs *ring;
> + int i;
> +
> + if (!scheduler)
> + return 0;
> +
> + spin_lock_irq(&scheduler->lock);
> +
> + for_each_ring(ring, dev_priv, i) {
> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> + if (node->params.file != file)
> + continue;
> +
> + if (!I915_SQS_IS_COMPLETE(node))
> + DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
> + node->params.request->uniq,
> + node->params.request->seqno,
> + node->status,
> + ring->name);
> +
> + node->params.file = NULL;
> + }
> + }
> +
> + spin_unlock_irq(&scheduler->lock);
> +
> + return 0;
> +}
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 415fec8..0e8b6a9 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -87,6 +87,8 @@ enum {
>
> bool i915_scheduler_is_enabled(struct drm_device *dev);
> int i915_scheduler_init(struct drm_device *dev);
> +int i915_scheduler_closefile(struct drm_device *dev,
> + struct drm_file *file);
> void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
> int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
> bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 11/35] drm/i915: Added scheduler hook into i915_gem_request_notify()
2016-02-18 14:26 ` [PATCH v5 11/35] drm/i915: Added scheduler hook into i915_gem_request_notify() John.C.Harrison
@ 2016-03-01 9:10 ` Joonas Lahtinen
0 siblings, 0 replies; 82+ messages in thread
From: Joonas Lahtinen @ 2016-03-01 9:10 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On to, 2016-02-18 at 14:26 +0000, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> The scheduler needs to know when requests have completed so that it
> can keep its own internal state up to date and can submit new requests
> to the hardware from its queue.
>
> v2: Updated due to changes in request handling. The operation is now
> reversed from before. Rather than the scheduler being in control of
> completion events, it is now the request code itself. The scheduler
> merely receives a notification event. It can then optionally request
> it's worker thread be woken up after all completion processing is
> complete.
>
> v4: Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
>
> v5: Squashed the i915_scheduler.c portions down into the 'start of
> scheduler' patch. [Joonas Lahtinen]
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem.c | 16 ++++++++++++++++
> 1 file changed, 16 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 0003cfc..c3b7def 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -2872,6 +2872,7 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
> {
> struct drm_i915_gem_request *req, *req_next;
> unsigned long flags;
> + bool wake_sched = false;
> u32 seqno;
>
> if (list_empty(&ring->fence_signal_list)) {
> @@ -2908,6 +2909,14 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
> */
> list_del_init(&req->signal_link);
>
> + /*
> + * NB: Must notify the scheduler before signalling
> + * the node. Otherwise the node can get retired first
> + * and call scheduler_clean() while the scheduler
> + * thinks it is still active.
> + */
> + wake_sched |= i915_scheduler_notify_request(req);
if (i915_scheduler_notify_request(req))
wake_sched = true;
> +
> if (!req->cancelled) {
> fence_signal_locked(&req->fence);
> trace_i915_gem_request_complete(req);
> @@ -2924,6 +2933,13 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
>
> if (!fence_locked)
> spin_unlock_irqrestore(&ring->fence_lock, flags);
> +
> + /* Necessary? Or does the fence_signal() call do an implicit wakeup? */
> + wake_up_all(&ring->irq_queue);
This should probably be figured out after the struct fence series is
merged.
Regards, Joonas
> +
> + /* Final scheduler processing after all individual updates are done. */
> + if (wake_sched)
> + i915_scheduler_wakeup(ring->dev);
> }
>
> static const char *i915_gem_request_get_driver_name(struct fence *req_fence)
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler
2016-02-18 14:27 ` [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler John.C.Harrison
@ 2016-03-01 9:16 ` Joonas Lahtinen
2016-03-01 15:12 ` John Harrison
0 siblings, 1 reply; 82+ messages in thread
From: Joonas Lahtinen @ 2016-03-01 9:16 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On to, 2016-02-18 at 14:27 +0000, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> The scheduler needs to do interrupt triggered work that is too complex
> to do in the interrupt handler. Thus it requires a deferred work
> handler to process such tasks asynchronously.
>
> v2: Updated to reduce mutex lock usage. The lock is now only held for
> the minimum time within the remove function rather than for the whole
> of the worker thread's operation.
>
> v5: Removed objectionable white space and added some documentation.
> [Joonas Lahtinen]
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/i915_dma.c | 3 +++
> drivers/gpu/drm/i915/i915_drv.h | 10 ++++++++++
> drivers/gpu/drm/i915/i915_gem.c | 2 ++
> drivers/gpu/drm/i915/i915_scheduler.c | 29 +++++++++++++++++++++++++++--
> drivers/gpu/drm/i915/i915_scheduler.h | 1 +
> 5 files changed, 43 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
> index 678adc7..c3d382d 100644
> --- a/drivers/gpu/drm/i915/i915_dma.c
> +++ b/drivers/gpu/drm/i915/i915_dma.c
> @@ -1158,6 +1158,9 @@ int i915_driver_unload(struct drm_device *dev)
> WARN_ON(unregister_oom_notifier(&dev_priv->mm.oom_notifier));
> unregister_shrinker(&dev_priv->mm.shrinker);
>
> + /* Cancel the scheduler work handler, which should be idle now. */
> + cancel_work_sync(&dev_priv->mm.scheduler_work);
> +
> io_mapping_free(dev_priv->gtt.mappable);
> arch_phys_wc_del(dev_priv->gtt.mtrr);
>
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index 03add1a..4d544f1 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -1291,6 +1291,16 @@ struct i915_gem_mm {
> struct delayed_work retire_work;
>
> /**
> + * New scheme is to get an interrupt after every work packet
> + * in order to allow the low latency scheduling of pending
> + * packets. The idea behind adding new packets to a pending
> + * queue rather than directly into the hardware ring buffer
> + * is to allow high priority packets to over take low priority
> + * ones.
> + */
> + struct work_struct scheduler_work;
> +
> + /**
> * When we detect an idle GPU, we want to turn on
> * powersaving features. So once we see that there
> * are no more requests outstanding and no more
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index c3b7def..1ab7256 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -5427,6 +5427,8 @@ i915_gem_load(struct drm_device *dev)
> i915_gem_retire_work_handler);
> INIT_DELAYED_WORK(&dev_priv->mm.idle_work,
> i915_gem_idle_work_handler);
> + INIT_WORK(&dev_priv->mm.scheduler_work,
> + i915_scheduler_work_handler);
> init_waitqueue_head(&dev_priv->gpu_error.reset_queue);
>
> dev_priv->relative_constants_mode = I915_EXEC_CONSTANTS_REL_GENERAL;
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index ab5007a..3986890 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -697,7 +697,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
> */
> void i915_scheduler_wakeup(struct drm_device *dev)
> {
> - /* XXX: Need to call i915_scheduler_remove() via work handler. */
> + struct drm_i915_private *dev_priv = to_i915(dev);
> +
> + queue_work(dev_priv->wq, &dev_priv->mm.scheduler_work);
> }
>
> /**
> @@ -827,7 +829,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
> return do_submit;
> }
>
> -void i915_scheduler_process_work(struct intel_engine_cs *ring)
> +static void i915_scheduler_process_work(struct intel_engine_cs *ring)
This is an odd change, maybe should have been static void to begin
with?
> {
> struct drm_i915_private *dev_priv = ring->dev->dev_private;
> struct i915_scheduler *scheduler = dev_priv->scheduler;
> @@ -874,6 +876,29 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
> }
>
> /**
> + * i915_scheduler_work_handler - scheduler's work handler callback.
> + * @work: Work structure
> + * A lot of the scheduler's work must be done asynchronously in response to
> + * an interrupt or other event. However, that work cannot be done at
> + * interrupt time or in the context of the event signaller (which might in
> + * fact be an interrupt). Thus a worker thread is required. This function
> + * will cause the thread to wake up and do its processing.
> + */
> +void i915_scheduler_work_handler(struct work_struct *work)
> +{
> + struct intel_engine_cs *ring;
> + struct drm_i915_private *dev_priv;
> + struct drm_device *dev;
> + int i;
> +
> + dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work);
> + dev = dev_priv->dev;
dev is unused variable?
With those changes,
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> +
> + for_each_ring(ring, dev_priv, i)
> + i915_scheduler_process_work(ring);
> +}
> +
> +/**
> * i915_scheduler_closefile - notify the scheduler that a DRM file handle
> * has been closed.
> * @dev: DRM device
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 0e8b6a9..180d75f 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -93,5 +93,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
> int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
> bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
> void i915_scheduler_wakeup(struct drm_device *dev);
> +void i915_scheduler_work_handler(struct work_struct *work);
>
> #endif /* _I915_SCHEDULER_H_ */
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls
2016-02-18 14:27 ` [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls John.C.Harrison
@ 2016-03-01 10:02 ` Joonas Lahtinen
2016-03-11 11:47 ` John Harrison
0 siblings, 1 reply; 82+ messages in thread
From: Joonas Lahtinen @ 2016-03-01 10:02 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On to, 2016-02-18 at 14:27 +0000, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> The scheduler can cause batch buffers, and hence requests, to be
> submitted to the ring out of order and asynchronously to their
> submission to the driver. Thus at the point of waiting for the
> completion of a given request, it is not even guaranteed that the
> request has actually been sent to the hardware yet. Even it is has
> been sent, it is possible that it could be pre-empted and thus
> 'unsent'.
>
> This means that it is necessary to be able to submit requests to the
> hardware during the wait call itself. Unfortunately, while some
> callers of __wait_request() release the mutex lock first, others do
> not (and apparently can not). Hence there is the ability to deadlock
> as the wait stalls for submission but the asynchronous submission is
> stalled for the mutex lock.
>
> This change hooks the scheduler in to the __wait_request() code to
> ensure correct behaviour. That is, flush the target batch buffer
> through to the hardware and do not deadlock waiting for something that
> cannot currently be submitted. Instead, the wait call must return
> EAGAIN at least as far back as necessary to release the mutex lock and
> allow the scheduler's asynchronous processing to get in and handle the
> pre-emption operation and eventually (re-)submit the work.
>
> v3: Removed the explicit scheduler flush from i915_wait_request().
> This is no longer necessary and was causing unintended changes to the
> scheduler priority level which broke a validation team test.
>
> v4: Corrected the format of some comments to keep the style checker
> happy.
>
> v5: Added function description. [Joonas Lahtinen]
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/i915_drv.h | 3 ++-
> drivers/gpu/drm/i915/i915_gem.c | 37 ++++++++++++++++++++++++++-------
> drivers/gpu/drm/i915/i915_scheduler.c | 31 +++++++++++++++++++++++++++
> drivers/gpu/drm/i915/i915_scheduler.h | 2 ++
> drivers/gpu/drm/i915/intel_display.c | 5 +++--
> drivers/gpu/drm/i915/intel_ringbuffer.c | 2 +-
> 6 files changed, 69 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index 4d544f1..5eeeced 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -3071,7 +3071,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
> unsigned reset_counter,
> bool interruptible,
> s64 *timeout,
> - struct intel_rps_client *rps);
> + struct intel_rps_client *rps,
> + bool is_locked);
Multiple bool parameters are better converted to int flags, it makes
the callsite code more readable if you have enum I915_WAIT_REQ_* flags.
> int __must_check i915_wait_request(struct drm_i915_gem_request *req);
> int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);
> int __must_check
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 2dd9b55..17b44b3 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -1258,7 +1258,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
> unsigned reset_counter,
> bool interruptible,
> s64 *timeout,
> - struct intel_rps_client *rps)
> + struct intel_rps_client *rps,
> + bool is_locked)
> {
> struct intel_engine_cs *ring = i915_gem_request_get_ring(req);
> struct drm_device *dev = ring->dev;
> @@ -1268,8 +1269,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
> DEFINE_WAIT(wait);
> unsigned long timeout_expire;
> s64 before = 0; /* Only to silence a compiler warning. */
> - int ret;
> + int ret = 0;
> + bool busy;
Strange whitespace.
>
> + might_sleep();
This will splat if is_locked is true? So maybe it should be protected
by if (!is_locked)?
> WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");
>
> if (i915_gem_request_completed(req))
> @@ -1324,6 +1327,26 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
> break;
> }
>
> + if (is_locked) {
> + /*
> + * If this request is being processed by the scheduler
> + * then it is unsafe to sleep with the mutex lock held
> + * as the scheduler may require the lock in order to
> + * progress the request.
> + */
This is becoming a monstrous function, I think it could be split down
to two functions, __i915_wait_request and
__i915_wait_request_locked/nonblocking, because if you take all the
timeout/sleeping related stuff away, not much is left.
> + if (i915_scheduler_is_request_tracked(req, NULL, &busy)) {
> + if (busy) {
> + ret = -EAGAIN;
> + break;
> + }
> + }
If this is kept in single function, would not it be enough to execute
this before the loop?
> +
> + /*
> + * If the request is not tracked by the scheduler
> + * then the regular test can be done.
> + */
> + }
> +
> if (i915_gem_request_completed(req)) {
> ret = 0;
> break;
> @@ -1542,7 +1565,7 @@ i915_wait_request(struct drm_i915_gem_request *req)
>
> ret = __i915_wait_request(req,
> atomic_read(&dev_priv->gpu_error.reset_counter),
> - interruptible, NULL, NULL);
> + interruptible, NULL, NULL, true);
> if (ret)
> return ret;
>
> @@ -1655,7 +1678,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
> mutex_unlock(&dev->struct_mutex);
> for (i = 0; ret == 0 && i < n; i++)
> ret = __i915_wait_request(requests[i], reset_counter, true,
> - NULL, rps);
> + NULL, rps, false);
> mutex_lock(&dev->struct_mutex);
>
> for (i = 0; i < n; i++) {
> @@ -3511,7 +3534,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
> if (ret == 0)
> ret = __i915_wait_request(req[i], reset_counter, true,
> args->timeout_ns > 0 ? &args->timeout_ns : NULL,
> - to_rps_client(file));
> + to_rps_client(file), false);
> i915_gem_request_unreference(req[i]);
> }
> return ret;
> @@ -3544,7 +3567,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
> atomic_read(&i915->gpu_error.reset_counter),
> i915->mm.interruptible,
> NULL,
> - &i915->rps.semaphores);
> + &i915->rps.semaphores, true);
> if (ret)
> return ret;
>
> @@ -4523,7 +4546,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
> if (target == NULL)
> return 0;
>
> - ret = __i915_wait_request(target, reset_counter, true, NULL, NULL);
> + ret = __i915_wait_request(target, reset_counter, true, NULL, NULL, false);
> if (ret == 0)
> queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 60a59d3..edab63d 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -918,6 +918,37 @@ void i915_scheduler_work_handler(struct work_struct *work)
> }
>
> /**
> + * i915_scheduler_is_request_tracked - return info to say what the scheduler's
> + * connection to this request is (if any).
> + * @req: request to be queried
> + * @compeleted: if non-null, set to completion status
> + * @busy: if non-null set to busy status
> + *
> + * Looks up the given request in the scheduler's internal queue and reports
> + * on whether the request has completed or is still pending.
> + * Returns true if the request was found or false if it was not.
> + */
> +bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
> + bool *completed, bool *busy)
> +{
> + struct drm_i915_private *dev_priv = req->ring->dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> +
> + if (!scheduler)
> + return false;
> +
> + if (req->scheduler_qe == NULL)
> + return false;
> +
These better be their own functions, making the code more readable.
_is_request_completed()
_is_request_busy()
> + if (completed)
> + *completed = I915_SQS_IS_COMPLETE(req->scheduler_qe);
> + if (busy)
> + *busy = I915_SQS_IS_QUEUED(req->scheduler_qe);
> +
> + return true;
> +}
> +
> +/**
> * i915_scheduler_closefile - notify the scheduler that a DRM file handle
> * has been closed.
> * @dev: DRM device
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 180d75f..a88adce 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -94,5 +94,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
> bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
> void i915_scheduler_wakeup(struct drm_device *dev);
> void i915_scheduler_work_handler(struct work_struct *work);
> +bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
> + bool *completed, bool *busy);
>
> #endif /* _I915_SCHEDULER_H_ */
> diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
> index 731d20a..5953590 100644
> --- a/drivers/gpu/drm/i915/intel_display.c
> +++ b/drivers/gpu/drm/i915/intel_display.c
> @@ -11458,7 +11458,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work)
> WARN_ON(__i915_wait_request(mmio_flip->req,
> mmio_flip->crtc->reset_counter,
> false, NULL,
> - &mmio_flip->i915->rps.mmioflips));
> + &mmio_flip->i915->rps.mmioflips,
> + false));
> i915_gem_request_unreference(mmio_flip->req);
> }
>
> @@ -13523,7 +13524,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev,
>
> ret = __i915_wait_request(intel_plane_state->wait_req,
> reset_counter, true,
> - NULL, NULL);
> + NULL, NULL, false);
>
> /* Swallow -EIO errors to allow updates during hw lockup. */
> if (ret == -EIO)
> diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
> index ca7b8af..a2093f5 100644
> --- a/drivers/gpu/drm/i915/intel_ringbuffer.c
> +++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
> @@ -2304,7 +2304,7 @@ int intel_ring_idle(struct intel_engine_cs *ring)
> return __i915_wait_request(req,
> atomic_read(&to_i915(ring->dev)->gpu_error.reset_counter),
> to_i915(ring->dev)->mm.interruptible,
> - NULL, NULL);
> + NULL, NULL, true);
> }
>
> int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles
2016-03-01 8:59 ` Joonas Lahtinen
@ 2016-03-01 14:52 ` John Harrison
0 siblings, 0 replies; 82+ messages in thread
From: John Harrison @ 2016-03-01 14:52 UTC (permalink / raw)
To: Joonas Lahtinen, Intel-GFX
On 01/03/2016 08:59, Joonas Lahtinen wrote:
> On to, 2016-02-18 at 14:26 +0000, John.C.Harrison@Intel.com wrote:
>> From: John Harrison <John.C.Harrison@Intel.com>
>>
>> The scheduler decouples the submission of batch buffers to the driver
>> with submission of batch buffers to the hardware. Thus it is possible
>> for an application to close its DRM file handle while there is still
>> work outstanding. That means the scheduler needs to know about file
>> close events so it can remove the file pointer from such orphaned
>> batch buffers and not attempt to dereference it later.
>>
>> v3: Updated to not wait for outstanding work to complete but merely
>> remove the file handle reference. The wait was getting excessively
>> complicated with inter-ring dependencies, pre-emption, and other such
>> issues.
>>
>> v4: Changed some white space to keep the style checker happy.
>>
>> v5: Added function documentation and removed apparently objectionable
>> white space. [Joonas Lahtinen]
>>
>> Used lighter weight spinlocks.
>>
>> For: VIZ-1587
>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
>> ---
>> drivers/gpu/drm/i915/i915_dma.c | 3 +++
>> drivers/gpu/drm/i915/i915_scheduler.c | 48 +++++++++++++++++++++++++++++++++++
>> drivers/gpu/drm/i915/i915_scheduler.h | 2 ++
>> 3 files changed, 53 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
>> index a0f5659..678adc7 100644
>> --- a/drivers/gpu/drm/i915/i915_dma.c
>> +++ b/drivers/gpu/drm/i915/i915_dma.c
>> @@ -46,6 +46,7 @@
>> #include
>> #include
>> #include
>> +#include "i915_scheduler.h"
>> #include
>> #include
>> #include
>> @@ -1258,6 +1259,8 @@ void i915_driver_lastclose(struct drm_device *dev)
>>
>> void i915_driver_preclose(struct drm_device *dev, struct drm_file *file)
>> {
>> + i915_scheduler_closefile(dev, file);
> Any reason not to put this inside the struct_mutex lock?
Any reason to put it inside? The smaller the processing done while
holding any given lock the better. The implementation only works on
scheduler internal data structures so while it does require the
scheduler's lock, it does not require the driver lock.
>
>> +
>> mutex_lock(&dev->struct_mutex);
>> i915_gem_context_close(dev, file);
>> i915_gem_release(dev, file);
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
>> index fc23ee7..ab5007a 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.c
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
>> @@ -872,3 +872,51 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
>> if (do_submit)
>> intel_runtime_pm_put(dev_priv);
>> }
>> +
>> +/**
>> + * i915_scheduler_closefile - notify the scheduler that a DRM file handle
>> + * has been closed.
>> + * @dev: DRM device
>> + * @file: file being closed
>> + *
>> + * Goes through the scheduler's queues and removes all connections to the
>> + * disappearing file handle that still exist. There is an argument to say
>> + * that this should also flush such outstanding work through the hardware.
>> + * However, with pre-emption, TDR and other such complications doing so
>> + * becomes a locking nightmare. So instead, just warn with a debug message
>> + * if the application is leaking uncompleted work and make sure a null
>> + * pointer dereference will not follow.
>> + */
>> +int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
> Return value not used, should be void if this can not fail.
Left over from a previous implementation that was much more complicated.
> Other than that,
>
> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
>
>> +{
>> + struct i915_scheduler_queue_entry *node;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + struct intel_engine_cs *ring;
>> + int i;
>> +
>> + if (!scheduler)
>> + return 0;
>> +
>> + spin_lock_irq(&scheduler->lock);
>> +
>> + for_each_ring(ring, dev_priv, i) {
>> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
>> + if (node->params.file != file)
>> + continue;
>> +
>> + if (!I915_SQS_IS_COMPLETE(node))
>> + DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
>> + node->params.request->uniq,
>> + node->params.request->seqno,
>> + node->status,
>> + ring->name);
>> +
>> + node->params.file = NULL;
>> + }
>> + }
>> +
>> + spin_unlock_irq(&scheduler->lock);
>> +
>> + return 0;
>> +}
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
>> index 415fec8..0e8b6a9 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.h
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
>> @@ -87,6 +87,8 @@ enum {
>>
>> bool i915_scheduler_is_enabled(struct drm_device *dev);
>> int i915_scheduler_init(struct drm_device *dev);
>> +int i915_scheduler_closefile(struct drm_device *dev,
>> + struct drm_file *file);
>> void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
>> int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
>> bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler
2016-03-01 9:16 ` Joonas Lahtinen
@ 2016-03-01 15:12 ` John Harrison
0 siblings, 0 replies; 82+ messages in thread
From: John Harrison @ 2016-03-01 15:12 UTC (permalink / raw)
To: Joonas Lahtinen, Intel-GFX
On 01/03/2016 09:16, Joonas Lahtinen wrote:
> On to, 2016-02-18 at 14:27 +0000, John.C.Harrison@Intel.com wrote:
>> From: John Harrison <John.C.Harrison@Intel.com>
>>
>> The scheduler needs to do interrupt triggered work that is too complex
>> to do in the interrupt handler. Thus it requires a deferred work
>> handler to process such tasks asynchronously.
>>
>> v2: Updated to reduce mutex lock usage. The lock is now only held for
>> the minimum time within the remove function rather than for the whole
>> of the worker thread's operation.
>>
>> v5: Removed objectionable white space and added some documentation.
>> [Joonas Lahtinen]
>>
>> For: VIZ-1587
>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
>> ---
>> drivers/gpu/drm/i915/i915_dma.c | 3 +++
>> drivers/gpu/drm/i915/i915_drv.h | 10 ++++++++++
>> drivers/gpu/drm/i915/i915_gem.c | 2 ++
>> drivers/gpu/drm/i915/i915_scheduler.c | 29 +++++++++++++++++++++++++++--
>> drivers/gpu/drm/i915/i915_scheduler.h | 1 +
>> 5 files changed, 43 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
>> index 678adc7..c3d382d 100644
>> --- a/drivers/gpu/drm/i915/i915_dma.c
>> +++ b/drivers/gpu/drm/i915/i915_dma.c
>> @@ -1158,6 +1158,9 @@ int i915_driver_unload(struct drm_device *dev)
>> WARN_ON(unregister_oom_notifier(&dev_priv->mm.oom_notifier));
>> unregister_shrinker(&dev_priv->mm.shrinker);
>>
>> + /* Cancel the scheduler work handler, which should be idle now. */
>> + cancel_work_sync(&dev_priv->mm.scheduler_work);
>> +
>> io_mapping_free(dev_priv->gtt.mappable);
>> arch_phys_wc_del(dev_priv->gtt.mtrr);
>>
>> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
>> index 03add1a..4d544f1 100644
>> --- a/drivers/gpu/drm/i915/i915_drv.h
>> +++ b/drivers/gpu/drm/i915/i915_drv.h
>> @@ -1291,6 +1291,16 @@ struct i915_gem_mm {
>> struct delayed_work retire_work;
>>
>> /**
>> + * New scheme is to get an interrupt after every work packet
>> + * in order to allow the low latency scheduling of pending
>> + * packets. The idea behind adding new packets to a pending
>> + * queue rather than directly into the hardware ring buffer
>> + * is to allow high priority packets to over take low priority
>> + * ones.
>> + */
>> + struct work_struct scheduler_work;
>> +
>> + /**
>> * When we detect an idle GPU, we want to turn on
>> * powersaving features. So once we see that there
>> * are no more requests outstanding and no more
>> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
>> index c3b7def..1ab7256 100644
>> --- a/drivers/gpu/drm/i915/i915_gem.c
>> +++ b/drivers/gpu/drm/i915/i915_gem.c
>> @@ -5427,6 +5427,8 @@ i915_gem_load(struct drm_device *dev)
>> i915_gem_retire_work_handler);
>> INIT_DELAYED_WORK(&dev_priv->mm.idle_work,
>> i915_gem_idle_work_handler);
>> + INIT_WORK(&dev_priv->mm.scheduler_work,
>> + i915_scheduler_work_handler);
>> init_waitqueue_head(&dev_priv->gpu_error.reset_queue);
>>
>> dev_priv->relative_constants_mode = I915_EXEC_CONSTANTS_REL_GENERAL;
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
>> index ab5007a..3986890 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.c
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
>> @@ -697,7 +697,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
>> */
>> void i915_scheduler_wakeup(struct drm_device *dev)
>> {
>> - /* XXX: Need to call i915_scheduler_remove() via work handler. */
>> + struct drm_i915_private *dev_priv = to_i915(dev);
>> +
>> + queue_work(dev_priv->wq, &dev_priv->mm.scheduler_work);
>> }
>>
>> /**
>> @@ -827,7 +829,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
>> return do_submit;
>> }
>>
>> -void i915_scheduler_process_work(struct intel_engine_cs *ring)
>> +static void i915_scheduler_process_work(struct intel_engine_cs *ring)
> This is an odd change, maybe should have been static void to begin
> with?
The problems of introducing code in patch A and plumbing it to the
driver in patch B. Making it static from the start gives an unused
function compiler warning.
>> {
>> struct drm_i915_private *dev_priv = ring->dev->dev_private;
>> struct i915_scheduler *scheduler = dev_priv->scheduler;
>> @@ -874,6 +876,29 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
>> }
>>
>> /**
>> + * i915_scheduler_work_handler - scheduler's work handler callback.
>> + * @work: Work structure
>> + * A lot of the scheduler's work must be done asynchronously in response to
>> + * an interrupt or other event. However, that work cannot be done at
>> + * interrupt time or in the context of the event signaller (which might in
>> + * fact be an interrupt). Thus a worker thread is required. This function
>> + * will cause the thread to wake up and do its processing.
>> + */
>> +void i915_scheduler_work_handler(struct work_struct *work)
>> +{
>> + struct intel_engine_cs *ring;
>> + struct drm_i915_private *dev_priv;
>> + struct drm_device *dev;
>> + int i;
>> +
>> + dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work);
>> + dev = dev_priv->dev;
> dev is unused variable?
>
> With those changes,
>
> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
>
>> +
>> + for_each_ring(ring, dev_priv, i)
>> + i915_scheduler_process_work(ring);
>> +}
>> +
>> +/**
>> * i915_scheduler_closefile - notify the scheduler that a DRM file handle
>> * has been closed.
>> * @dev: DRM device
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
>> index 0e8b6a9..180d75f 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.h
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
>> @@ -93,5 +93,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
>> int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
>> bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
>> void i915_scheduler_wakeup(struct drm_device *dev);
>> +void i915_scheduler_work_handler(struct work_struct *work);
>>
>> #endif /* _I915_SCHEDULER_H_ */
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle
2016-02-23 21:02 ` Jesse Barnes
@ 2016-03-01 15:52 ` John Harrison
0 siblings, 0 replies; 82+ messages in thread
From: John Harrison @ 2016-03-01 15:52 UTC (permalink / raw)
To: Jesse Barnes, Intel-GFX
On 23/02/2016 21:02, Jesse Barnes wrote:
> On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
>> From: John Harrison <John.C.Harrison@Intel.com>
>>
>> The scheduler decouples the submission of batch buffers to the driver
>> from their subsequent submission to the hardware. This means that an
>> application which is continuously submitting buffers as fast as it can
>> could potentialy flood the driver. To prevent this, the driver now
>> tracks how many buffers are in progress (queued in software or
>> executing in hardware) and limits this to a given (tunable) number. If
>> this number is exceeded then the queue to the driver will return
>> EAGAIN and thus prevent the scheduler's queue becoming arbitrarily
>> large.
>>
>> v3: Added a missing decrement of the file queue counter.
>>
>> v4: Updated a comment.
>>
>> v5: Updated due to changes to earlier patches in series - removing
>> forward declarations and white space. Also added some documentation.
>> [Joonas Lahtinen]
>>
>> For: VIZ-1587
>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>> ---
>> drivers/gpu/drm/i915/i915_drv.h | 2 ++
>> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 8 +++++
>> drivers/gpu/drm/i915/i915_scheduler.c | 48 ++++++++++++++++++++++++++++++
>> drivers/gpu/drm/i915/i915_scheduler.h | 2 ++
>> 4 files changed, 60 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
>> index 071a27b..3f4c4f0 100644
>> --- a/drivers/gpu/drm/i915/i915_drv.h
>> +++ b/drivers/gpu/drm/i915/i915_drv.h
>> @@ -336,6 +336,8 @@ struct drm_i915_file_private {
>> } rps;
>>
>> struct intel_engine_cs *bsd_ring;
>> +
>> + u32 scheduler_queue_length;
>> };
>>
>> enum intel_dpll_id {
>> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>> index d4de8c7..dff120c 100644
>> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>> @@ -1803,6 +1803,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
>> return -EINVAL;
>> }
>>
>> + /* Throttle batch requests per device file */
>> + if (i915_scheduler_file_queue_is_full(file))
>> + return -EAGAIN;
>> +
>> /* Copy in the exec list from userland */
>> exec_list = drm_malloc_ab(sizeof(*exec_list), args->buffer_count);
>> exec2_list = drm_malloc_ab(sizeof(*exec2_list), args->buffer_count);
>> @@ -1893,6 +1897,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
>> return -EINVAL;
>> }
>>
>> + /* Throttle batch requests per device file */
>> + if (i915_scheduler_file_queue_is_full(file))
>> + return -EAGAIN;
>> +
>> exec2_list = kmalloc(sizeof(*exec2_list)*args->buffer_count,
>> GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
>> if (exec2_list == NULL)
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
>> index e56ce08..f7f29d5 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.c
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
>> @@ -69,6 +69,7 @@ int i915_scheduler_init(struct drm_device *dev)
>> scheduler->priority_level_bump = 50;
>> scheduler->priority_level_preempt = 900;
>> scheduler->min_flying = 2;
>> + scheduler->file_queue_max = 64;
>>
>> dev_priv->scheduler = scheduler;
>>
>> @@ -464,6 +465,44 @@ static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)
>> return ret;
>> }
>>
>> +/**
>> + * i915_scheduler_file_queue_is_full - Returns true if the queue is full.
>> + * @file: File object to query.
>> + * This allows throttling of applications by limiting the total number of
>> + * outstanding requests to a specified level. Once that limit is reached,
>> + * this call will return true and no more requests should be accepted.
>> + */
>> +bool i915_scheduler_file_queue_is_full(struct drm_file *file)
>> +{
>> + struct drm_i915_file_private *file_priv = file->driver_priv;
>> + struct drm_i915_private *dev_priv = file_priv->dev_priv;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + return file_priv->scheduler_queue_length >= scheduler->file_queue_max;
>> +}
>> +
>> +/**
>> + * i915_scheduler_file_queue_inc - Increment the file's request queue count.
>> + * @file: File object to process.
>> + */
>> +static void i915_scheduler_file_queue_inc(struct drm_file *file)
>> +{
>> + struct drm_i915_file_private *file_priv = file->driver_priv;
>> +
>> + file_priv->scheduler_queue_length++;
>> +}
>> +
>> +/**
>> + * i915_scheduler_file_queue_dec - Decrement the file's request queue count.
>> + * @file: File object to process.
>> + */
>> +static void i915_scheduler_file_queue_dec(struct drm_file *file)
>> +{
>> + struct drm_i915_file_private *file_priv = file->driver_priv;
>> +
>> + file_priv->scheduler_queue_length--;
>> +}
>> +
>> static void i915_generate_dependencies(struct i915_scheduler *scheduler,
>> struct i915_scheduler_queue_entry *node,
>> uint32_t ring)
>> @@ -640,6 +679,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
>>
>> list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
>>
>> + i915_scheduler_file_queue_inc(node->params.file);
>> +
>> not_flying = i915_scheduler_count_flying(scheduler, ring) <
>> scheduler->min_flying;
>>
>> @@ -883,6 +924,12 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
>> /* Strip the dependency info while the mutex is still locked */
>> i915_scheduler_remove_dependent(scheduler, node);
>>
>> + /* Likewise clean up the file pointer. */
>> + if (node->params.file) {
>> + i915_scheduler_file_queue_dec(node->params.file);
>> + node->params.file = NULL;
>> + }
>> +
>> continue;
>> }
>>
>> @@ -1205,6 +1252,7 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
>> node->status,
>> ring->name);
>>
>> + i915_scheduler_file_queue_dec(node->params.file);
>> node->params.file = NULL;
>> }
>> }
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
>> index 075befb..b78de12 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.h
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
>> @@ -77,6 +77,7 @@ struct i915_scheduler {
>> int32_t priority_level_bump;
>> int32_t priority_level_preempt;
>> uint32_t min_flying;
>> + uint32_t file_queue_max;
>> };
>>
>> /* Flag bits for i915_scheduler::flags */
>> @@ -100,5 +101,6 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
>> unsigned long stamp, bool is_locked);
>> bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
>> bool *completed, bool *busy);
>> +bool i915_scheduler_file_queue_is_full(struct drm_file *file);
>>
>> #endif /* _I915_SCHEDULER_H_ */
>>
> Just to clarify and make sure I understood the previous stuff: a queued execbuf that has not yet been dispatched does not reserve and pin pages right? That occurs at actual dispatch time? If so, I guess clients will hit this 64 queued item limit pretty regularly... How much metadata overhead does that involve? Has it been derived from some performance work with a bunch of workloads? It's fine if not, I can imagine that different mixes of workloads would be affected by lower or higher queue depths (e.g. small batch tests).
Whether a client will hit the limit is entirely down to how that client
is written. A GL program that renders double buffered with one batch per
frame will only ever have two batch buffer requests outstanding.
Whereas, the gem_exec_nop IGT test submits a thousand batches before
waiting for anything to complete.
> If this is tunable, I guess it should be clamped like a nice or rlimit value, with values outside that range requiring CAP_SYS_ADMIN.
What is considered a valid range? Previously the limit would have been
the size of the ring buffer. In times past, they were large and you
could have thousands of batches in flight. With the GuC we are down to
4K words. At, for example, 32 words per batch buffer request that makes
for 128 outstanding batches. So set the maximum permitted to 256?
Larger? Smaller?
>
> Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
>
> Jesse
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions
2016-02-18 14:27 ` [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions John.C.Harrison
@ 2016-03-07 11:31 ` Joonas Lahtinen
2016-03-11 16:22 ` John Harrison
0 siblings, 1 reply; 82+ messages in thread
From: Joonas Lahtinen @ 2016-03-07 11:31 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On to, 2016-02-18 at 14:27 +0000, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> When requesting that all GPU work is completed, it is now necessary to
> get the scheduler involved in order to flush out work that queued and
> not yet submitted.
>
> v2: Updated to add support for flushing the scheduler queue by time
> stamp rather than just doing a blanket flush.
>
> v3: Moved submit_max_priority() to this patch from an earlier patch
> is it is no longer required in the other.
>
> v4: Corrected the format of a comment to keep the style checker happy.
> Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
>
> v5: Shuffled functions around to remove forward prototypes, removed
> similarly offensive white space and added documentation. Re-worked the
> mutex locking around the submit function. [Joonas Lahtinen]
>
> Used lighter weight spinlocks.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem.c | 24 ++++-
> drivers/gpu/drm/i915/i915_scheduler.c | 178 ++++++++++++++++++++++++++++++++++
> drivers/gpu/drm/i915/i915_scheduler.h | 3 +
> 3 files changed, 204 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index a47a495..d946f53 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -3786,6 +3786,10 @@ int i915_gpu_idle(struct drm_device *dev)
>
> /* Flush everything onto the inactive list. */
> for_each_ring(ring, dev_priv, i) {
> + ret = i915_scheduler_flush(ring, true);
> + if (ret < 0)
> + return ret;
> +
> if (!i915.enable_execlists) {
> struct drm_i915_gem_request *req;
>
> @@ -4519,7 +4523,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
> unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES;
> struct drm_i915_gem_request *request, *target = NULL;
> unsigned reset_counter;
> - int ret;
> + int i, ret;
> + struct intel_engine_cs *ring;
>
> ret = i915_gem_wait_for_error(&dev_priv->gpu_error);
> if (ret)
> @@ -4529,6 +4534,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
> if (ret)
> return ret;
>
> + for_each_ring(ring, dev_priv, i) {
> + /*
> + * Flush out scheduler entries that are getting 'stale'. Note
> + * that the following recent_enough test will only check
> + * against the time at which the request was submitted to the
> + * hardware (i.e. when it left the scheduler) not the time it
> + * was submitted to the driver.
> + *
> + * Also, there is not much point worring about busy return
> + * codes from the scheduler flush call. Even if more work
> + * cannot be submitted right now for whatever reason, we
> + * still want to throttle against stale work that has already
> + * been submitted.
> + */
> + i915_scheduler_flush_stamp(ring, recent_enough, false);
> + }
> +
> spin_lock(&file_priv->mm.lock);
> list_for_each_entry(request, &file_priv->mm.request_list, client_list) {
> if (time_after_eq(request->emitted_jiffies, recent_enough))
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index edab63d..8130a9c 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -304,6 +304,10 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
> * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
> * And releasing the one before acquiring the other leads to other code
> * being run and interfering.
> + *
> + * Hence any caller that does not already have the mutex lock for other
> + * reasons should call i915_scheduler_submit_unlocked() instead in order to
> + * obtain the lock first.
> */
> static int i915_scheduler_submit(struct intel_engine_cs *ring)
> {
> @@ -428,6 +432,22 @@ error:
> return ret;
> }
>
> +static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)
> +{
> + struct drm_device *dev = ring->dev;
> + int ret;
> +
> + ret = i915_mutex_lock_interruptible(dev);
> + if (ret)
> + return ret;
> +
> + ret = i915_scheduler_submit(ring);
> +
> + mutex_unlock(&dev->struct_mutex);
> +
> + return ret;
> +}
> +
This kind of code should really be factored to upper level functions.
> static void i915_generate_dependencies(struct i915_scheduler *scheduler,
> struct i915_scheduler_queue_entry *node,
> uint32_t ring)
> @@ -917,6 +937,164 @@ void i915_scheduler_work_handler(struct work_struct *work)
> i915_scheduler_process_work(ring);
> }
>
> +static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
> + bool is_locked)
> +{
> + struct i915_scheduler_queue_entry *node;
> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + int ret, count = 0;
> + bool found;
> +
> + do {
> + found = false;
> + spin_lock_irq(&scheduler->lock);
> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> + if (!I915_SQS_IS_QUEUED(node))
> + continue;
> +
> + if (node->priority < scheduler->priority_level_max)
> + continue;
> +
> + found = true;
> + break;
> + }
> + spin_unlock_irq(&scheduler->lock);
> +
> + if (!found)
> + break;
> +
> + if (is_locked)
> + ret = i915_scheduler_submit(ring);
> + else
> + ret = i915_scheduler_submit_unlocked(ring);
The widespread use of 'is_locked' notation is really bothering me, I
think the difference in code path should be handled in the upper level,
instead of being passed down, for example this will cause repetitive
lock/unlock in loop.
> + if (ret < 0)
> + return ret;
> +
> + count += ret;
> + } while (found);
> +
> + return count;
> +}
> +
> +/**
> + * i915_scheduler_flush_stamp - force requests of a given age through the
> + * scheduler.
> + * @ring: Ring to be flushed
> + * @target: Jiffy based time stamp to flush up to
> + * @is_locked: Is the driver mutex lock held?
> + * DRM has a throttle by age of request facility. This requires waiting for
> + * outstanding work over a given age. This function helps that by forcing
> + * queued batch buffers over said age through the system.
> + * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
> + * for a pre-emption event to complete) but the mutex lock is held which
> + * would prevent the scheduler's asynchronous processing from completing.
> + */
> +int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
> + unsigned long target,
> + bool is_locked)
> +{
> + struct i915_scheduler_queue_entry *node;
> + struct drm_i915_private *dev_priv;
> + struct i915_scheduler *scheduler;
> + int flush_count = 0;
> +
> + if (!ring)
> + return -EINVAL;
> +
> + dev_priv = ring->dev->dev_private;
> + scheduler = dev_priv->scheduler;
> +
> + if (!scheduler)
> + return 0;
> +
> + if (is_locked && (scheduler->flags[ring->id] & i915_sf_submitting)) {
> + /*
> + * Scheduler is busy already submitting another batch,
> + * come back later rather than going recursive...
> + */
> + return -EAGAIN;
> + }
> +
Again, this would be handled better at higher level.
So I would like to see those 'is_locked' and other similar constructs
factored out from the series. It should make the code much more robust
to modify at later point.
Regards, Joonas
> + spin_lock_irq(&scheduler->lock);
> + i915_scheduler_priority_bump_clear(scheduler);
> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> + if (!I915_SQS_IS_QUEUED(node))
> + continue;
> +
> + if (node->stamp > target)
> + continue;
> +
> + flush_count = i915_scheduler_priority_bump(scheduler,
> + node, scheduler->priority_level_max);
> + }
> + spin_unlock_irq(&scheduler->lock);
> +
> + if (flush_count) {
> + DRM_DEBUG_DRIVER("<%s> Bumped %d entries\n", ring->name, flush_count);
> + flush_count = i915_scheduler_submit_max_priority(ring, is_locked);
> + }
> +
> + return flush_count;
> +}
> +
> +/**
> + * i915_scheduler_flush - force all requests through the scheduler.
> + * @ring: Ring to be flushed
> + * @is_locked: Is the driver mutex lock held?
> + * For various reasons it is sometimes necessary to the scheduler out, e.g.
> + * due to ring reset.
> + * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
> + * for a pre-emption event to complete) but the mutex lock is held which
> + * would prevent the scheduler's asynchronous processing from completing.
> + */
> +int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked)
> +{
> + struct i915_scheduler_queue_entry *node;
> + struct drm_i915_private *dev_priv;
> + struct i915_scheduler *scheduler;
> + bool found;
> + int ret;
> + uint32_t count = 0;
> +
> + if (!ring)
> + return -EINVAL;
> +
> + dev_priv = ring->dev->dev_private;
> + scheduler = dev_priv->scheduler;
> +
> + if (!scheduler)
> + return 0;
> +
> + WARN_ON(is_locked && (scheduler->flags[ring->id] & i915_sf_submitting));
> +
> + do {
> + found = false;
> + spin_lock_irq(&scheduler->lock);
> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> + if (!I915_SQS_IS_QUEUED(node))
> + continue;
> +
> + found = true;
> + break;
> + }
> + spin_unlock_irq(&scheduler->lock);
> +
> + if (found) {
> + if (is_locked)
> + ret = i915_scheduler_submit(ring);
> + else
> + ret = i915_scheduler_submit_unlocked(ring);
> + if (ret < 0)
> + return ret;
> +
> + count += ret;
> + }
> + } while (found);
> +
> + return count;
> +}
> +
> /**
> * i915_scheduler_is_request_tracked - return info to say what the scheduler's
> * connection to this request is (if any).
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index a88adce..839b048 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -94,6 +94,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
> bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
> void i915_scheduler_wakeup(struct drm_device *dev);
> void i915_scheduler_work_handler(struct work_struct *work);
> +int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
> +int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
> + unsigned long stamp, bool is_locked);
> bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
> bool *completed, bool *busy);
>
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 23/35] drm/i915: Defer seqno allocation until actual hardware submission time
2016-02-18 14:27 ` [PATCH v5 23/35] drm/i915: Defer seqno allocation until actual hardware submission time John.C.Harrison
@ 2016-03-07 12:16 ` Joonas Lahtinen
0 siblings, 0 replies; 82+ messages in thread
From: Joonas Lahtinen @ 2016-03-07 12:16 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
On to, 2016-02-18 at 14:27 +0000, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> The seqno value is now only used for the final test for completion of
> a request. It is no longer used to track the request through the
> software stack. Thus it is no longer necessary to allocate the seqno
> immediately with the request. Instead, it can be done lazily and left
> until the request is actually sent to the hardware. This is particular
> advantageous with a GPU scheduler as the requests can then be
> re-ordered between their creation and their hardware submission
> without having out of order seqnos.
>
> v2: i915_add_request() can't fail!
>
> Combine with 'drm/i915: Assign seqno at start of exec_final()'
> Various bits of code during the execbuf code path need a seqno value
> to be assigned to the request. This change makes this assignment
> explicit at the start of submission_final() rather than relying on an
> auto-generated seqno to have happened already. This is in preparation
> for a future patch which changes seqno values to be assigned lazily
> (during add_request).
>
> v3: Updated to use locally cached request pointer.
>
> v4: Changed some white space and comment formatting to keep the style
> checker happy.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/i915_drv.h | 1 +
> drivers/gpu/drm/i915/i915_gem.c | 23 ++++++++++++++++++++++-
> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 14 ++++++++++++++
> drivers/gpu/drm/i915/intel_lrc.c | 14 ++++++++++++++
> 4 files changed, 51 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index 5eeeced..071a27b 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -2257,6 +2257,7 @@ struct drm_i915_gem_request {
> * has finished processing this request.
> */
> u32 seqno;
> + u32 reserved_seqno;
>
> /* Unique identifier which can be used for trace points & debug */
> uint32_t uniq;
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index a249e52..a2c136d 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -2616,6 +2616,11 @@ i915_gem_get_seqno(struct drm_device *dev, u32 *seqno)
>
> /* reserve 0 for non-seqno */
> if (dev_priv->next_seqno == 0) {
> + /*
> + * Why is the full re-initialisation required? Is it only for
> + * hardware semaphores? If so, could skip it in the case where
> + * semaphores are disabled?
> + */
> int ret = i915_gem_init_seqno(dev, 0);
> if (ret)
> return ret;
> @@ -2673,6 +2678,12 @@ void __i915_add_request(struct drm_i915_gem_request *request,
> WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
> }
>
> + /* Make the request's seqno 'live': */
> + if (!request->seqno) {
> + request->seqno = request->reserved_seqno;
> + WARN_ON(request->seqno != dev_priv->last_seqno);
> + }
> +
> /* Record the position of the start of the request so that
> * should we detect the updated seqno part-way through the
> * GPU processing the request, we never over-estimate the
> @@ -2930,6 +2941,9 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
>
> list_for_each_entry_safe(req, req_next, &ring->fence_signal_list, signal_link) {
> if (!req->cancelled) {
> + /* How can this happen? */
> + WARN_ON(req->seqno == 0);
> +
> if (!i915_seqno_passed(seqno, req->seqno))
> break;
> }
> @@ -3079,7 +3093,14 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,
> if (req == NULL)
> return -ENOMEM;
>
> - ret = i915_gem_get_seqno(ring->dev, &req->seqno);
> + /*
> + * Assign an identifier to track this request through the hardware
> + * but don't make it live yet. It could change in the future if this
> + * request gets overtaken. However, it still needs to be allocated
> + * in advance because the point of submission must not fail and seqno
> + * allocation can fail.
> + */
> + ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
> if (ret)
> goto err;
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> index f45f4dc..b9ad0fd 100644
> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> @@ -1295,6 +1295,20 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
> /* The mutex must be acquired before calling this function */
> WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
>
> + /* Make sure the request's seqno is the latest and greatest: */
> + if (req->reserved_seqno != dev_priv->last_seqno) {
> + ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
> + if (ret)
> + return ret;
> + }
Would this situation be caused by reordering? Kinda strange to
pregenerate and then re-generate if some request got prioritized over
us. I'd assume that to happen constantly with compositor running.
What exactly stops from generating the seqno always at submission time,
if we're able to re-generate it anyway?
Regards, Joonas
> + /*
> + * And make it live because some of the execbuff submission code
> + * requires the seqno to be available up front.
> + */
> + WARN_ON(req->seqno);
> + req->seqno = req->reserved_seqno;
> + WARN_ON(req->seqno != dev_priv->last_seqno);
> +
> ret = intel_ring_reserve_space(req);
> if (ret)
> goto error;
> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> index e056875..9c7a79a 100644
> --- a/drivers/gpu/drm/i915/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/intel_lrc.c
> @@ -978,6 +978,20 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
> /* The mutex must be acquired before calling this function */
> WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
>
> + /* Make sure the request's seqno is the latest and greatest: */
> + if (req->reserved_seqno != dev_priv->last_seqno) {
> + ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
> + if (ret)
> + return ret;
> + }
> + /*
> + * And make it live because some of the execbuff submission code
> + * requires the seqno to be available up front.
> + */
> + WARN_ON(req->seqno);
> + req->seqno = req->reserved_seqno;
> + WARN_ON(req->seqno != dev_priv->last_seqno);
> +
> ret = intel_logical_ring_reserve_space(req);
> if (ret)
> goto err;
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 27/35] drm/i915: Added debug state dump facilities to scheduler
2016-02-18 14:27 ` [PATCH v5 27/35] drm/i915: Added debug state dump facilities to scheduler John.C.Harrison
@ 2016-03-07 12:31 ` Joonas Lahtinen
2016-03-11 16:38 ` John Harrison
0 siblings, 1 reply; 82+ messages in thread
From: Joonas Lahtinen @ 2016-03-07 12:31 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
Hi,
On to, 2016-02-18 at 14:27 +0000, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> When debugging batch buffer submission issues, it is useful to be able
> to see what the current state of the scheduler is. This change adds
> functions for decoding the internal scheduler state and reporting it.
>
> v3: Updated a debug message with the new state_str() function.
>
> v4: Wrapped some long lines to keep the style checker happy. Removed
> the fence/sync code as that will now be part of a separate patch series.
>
> v5: Removed forward declarations and white space. Added documentation.
> [Joonas Lahtinen]
>
> Also squashed in later patch to add seqno information from the start.
> It was only being added in a separate patch due to historical reasons
> which have since gone away.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/i915_scheduler.c | 302 +++++++++++++++++++++++++++++++++-
> drivers/gpu/drm/i915/i915_scheduler.h | 15 ++
> 2 files changed, 315 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index f7f29d5..d0eed52 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -40,6 +40,117 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
> return dev_priv->scheduler != NULL;
> }
>
These sort of functions should be in i915_debugfs.c, so that nobody
even thinks of using them for other purposes.
>
> +const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node)
> +{
> + static char str[50];
> + char *ptr = str;
> +
> + *(ptr++) = node->bumped ? 'B' : '-',
> + *(ptr++) = i915_gem_request_completed(node->params.request) ? 'C' : '-';
> +
Especially this kind of code needs to be in i915_debugfs.c. I'd
implement new code more along lines of i915_sseu_status(), but I see
this kind of code was previously merged.
> + *ptr = 0;
> +
> + return str;
> +}
> +
> +char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status)
> +{
> + switch (status) {
> + case i915_sqs_none:
> + return 'N';
> +
> + case i915_sqs_queued:
> + return 'Q';
> +
> + case i915_sqs_popped:
> + return 'X';
> +
> + case i915_sqs_flying:
> + return 'F';
> +
> + case i915_sqs_complete:
> + return 'C';
> +
> + case i915_sqs_dead:
> + return 'D';
> +
> + default:
> + break;
> + }
> +
Bad indent.
> + return '?';
> +}
> +
> +const char *i915_scheduler_queue_status_str(
> + enum i915_scheduler_queue_status status)
> +{
> + static char str[50];
> +
> + switch (status) {
> + case i915_sqs_none:
> + return "None";
> +
> + case i915_sqs_queued:
> + return "Queued";
> +
> + case i915_sqs_popped:
> + return "Popped";
> +
> + case i915_sqs_flying:
> + return "Flying";
> +
> + case i915_sqs_complete:
> + return "Complete";
> +
> + case i915_sqs_dead:
> + return "Dead";
> +
> + default:
> + break;
> + }
> +
> + sprintf(str, "[Unknown_%d!]", status);
> + return str;
> +}
> +
> +const char *i915_scheduler_flag_str(uint32_t flags)
> +{
> + static char str[100];
> + char *ptr = str;
> +
> + *ptr = 0;
> +
> +#define TEST_FLAG(flag, msg) \
> + do { \
> + if (flags & (flag)) { \
> + strcpy(ptr, msg); \
> + ptr += strlen(ptr); \
> + flags &= ~(flag); \
> + } \
> + } while (0)
> +
> + TEST_FLAG(i915_sf_interrupts_enabled, "IntOn|");
> + TEST_FLAG(i915_sf_submitting, "Submitting|");
> + TEST_FLAG(i915_sf_dump_force, "DumpForce|");
> + TEST_FLAG(i915_sf_dump_details, "DumpDetails|");
> + TEST_FLAG(i915_sf_dump_dependencies, "DumpDeps|");
> + TEST_FLAG(i915_sf_dump_seqno, "DumpSeqno|");
> +
> +#undef TEST_FLAG
> +
> + if (flags) {
> + sprintf(ptr, "Unknown_0x%X!", flags);
> + ptr += strlen(ptr);
> + }
> +
> + if (ptr == str)
> + strcpy(str, "-");
> + else
> + ptr[-1] = 0;
> +
> + return str;
> +};
> +
> /**
> * i915_scheduler_init - Initialise the scheduler.
> * @dev: DRM device
> @@ -1024,6 +1135,193 @@ void i915_scheduler_work_handler(struct work_struct *work)
> i915_scheduler_process_work(ring);
> }
>
> +static int i915_scheduler_dump_locked(struct intel_engine_cs *ring,
> + const char *msg)
> +{
> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct i915_scheduler_queue_entry *node;
> + int flying = 0, queued = 0, complete = 0, other = 0;
> + static int old_flying = -1, old_queued = -1, old_complete = -1;
> + bool b_dump;
> + char brkt[2] = { '<', '>' };
> +
> + if (!ring)
> + return -EINVAL;
> +
> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> + if (I915_SQS_IS_QUEUED(node))
> + queued++;
> + else if (I915_SQS_IS_FLYING(node))
> + flying++;
> + else if (I915_SQS_IS_COMPLETE(node))
> + complete++;
> + else
> + other++;
> + }
> +
> + b_dump = (flying != old_flying) ||
> + (queued != old_queued) ||
> + (complete != old_complete);
> + if (scheduler->flags[ring->id] & i915_sf_dump_force) {
> + if (!b_dump) {
> + b_dump = true;
> + brkt[0] = '{';
> + brkt[1] = '}';
> + }
> +
> + scheduler->flags[ring->id] &= ~i915_sf_dump_force;
> + }
> +
> + if (b_dump) {
> + old_flying = flying;
> + old_queued = queued;
> + old_complete = complete;
> + DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d, "
> + "Flags = %s, Next = %d:%d %c%s%c\n",
> + ring->name, queued, flying, complete, other,
> + i915_scheduler_flag_str(scheduler->flags[ring->id]),
> + dev_priv->request_uniq, dev_priv->next_seqno,
> + brkt[0], msg, brkt[1]);
Convert to debugfs, it's the de-facto.
> + } else {
> + /*DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d"
> + ", Flags = %s, Next = %d:%d [%s]\n",
> + ring->name,
> + queued, flying, complete, other,
> + i915_scheduler_flag_str(scheduler->flags[ring->id]),
> + dev_priv->request_uniq, dev_priv->next_seqno, msg); */
> +
Dead code to be removed from patches.
> + return 0;
> + }
> +
> + if (scheduler->flags[ring->id] & i915_sf_dump_seqno) {
> + uint32_t seqno;
> +
> + seqno = ring->get_seqno(ring, true);
> +
> + DRM_DEBUG_DRIVER("<%s> Seqno = %d\n", ring->name, seqno);
> + }
> +
> + if (scheduler->flags[ring->id] & i915_sf_dump_details) {
> + int i, deps;
> + uint32_t count, counts[i915_sqs_MAX];
> +
> + memset(counts, 0x00, sizeof(counts));
> +
> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> + if (node->status < i915_sqs_MAX) {
> + count = counts[node->status]++;
> + } else {
> + DRM_DEBUG_DRIVER("<%s> Unknown status: %d!\n",
> + ring->name, node->status);
> + count = -1;
> + }
> +
> + deps = 0;
> + for (i = 0; i < node->num_deps; i++)
> + if (i915_scheduler_is_dependency_valid(node, i))
> + deps++;
> +
> + DRM_DEBUG_DRIVER("<%s> %c:%02d> uniq = %d, seqno"
> + " = %d/%s, deps = %d / %d, %s [pri = "
> + "%4d]\n", ring->name,
> + i915_scheduler_queue_status_chr(node->status),
> + count,
> + node->params.request->uniq,
> + node->params.request->seqno,
> + node->params.ring->name,
> + deps, node->num_deps,
> + i915_qe_state_str(node),
> + node->priority);
> +
> + if ((scheduler->flags[ring->id] & i915_sf_dump_dependencies)
> + == 0)
> + continue;
> +
> + for (i = 0; i < node->num_deps; i++)
> + if (node->dep_list[i])
> + DRM_DEBUG_DRIVER("<%s> |-%c:"
> + "%02d%c uniq = %d, seqno = %d/%s, %s [pri = %4d]\n",
> + ring->name,
> + i915_scheduler_queue_status_chr(node->dep_list[i]->status),
> + i,
> + i915_scheduler_is_dependency_valid(node, i)
> + ? '>' : '#',
> + node->dep_list[i]->params.request->uniq,
> + node->dep_list[i]->params.request->seqno,
> + node->dep_list[i]->params.ring->name,
> + i915_qe_state_str(node->dep_list[i]),
> + node->dep_list[i]->priority);
> + }
> + }
> +
> + return 0;
> +}
> +
> +/**
> + * i915_scheduler_dump - dump the scheduler's internal state to the debug log.
> + * @ring: Ring to dump info for
> + * @msg: A reason why it is being dumped
> + * For debugging purposes, it can be very useful to see the internal state of
> + * the scheduler for a given ring.
> + */
> +int i915_scheduler_dump(struct intel_engine_cs *ring, const char *msg)
> +{
> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + unsigned long flags;
> + int ret;
> +
> + spin_lock_irqsave(&scheduler->lock, flags);
> + ret = i915_scheduler_dump_locked(ring, msg);
> + spin_unlock_irqrestore(&scheduler->lock, flags);
> +
> + return ret;
> +}
> +
This kind of stuff especially should be static functions in
i915_debugfs.c. And then they don't need kerneldoc either, because we
expect no sane invidual to call them.
> +static int i915_scheduler_dump_all_locked(struct drm_device *dev,
> + const char *msg)
> +{
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct intel_engine_cs *ring;
> + int i, r, ret = 0;
> +
> + for_each_ring(ring, dev_priv, i) {
> + scheduler->flags[ring->id] |= i915_sf_dump_force |
> + i915_sf_dump_details |
> + i915_sf_dump_seqno |
> + i915_sf_dump_dependencies;
This looks really bad to me, invasive debugging. Rather to be carried
as function parameters in the debug functions, and they should be
uppercase enum in i915_debugfs.c too.
> + r = i915_scheduler_dump_locked(ring, msg);
> + if (ret == 0)
> + ret = r;
> + }
> +
> + return ret;
> +}
> +
> +/**
> + * i915_scheduler_dump_all - dump the scheduler's internal state to the debug
> + * log.
> + * @dev: DRM device
> + * @msg: A reason why it is being dumped
> + * For debugging purposes, it can be very useful to see the internal state of
> + * the scheduler.
> + */
> +int i915_scheduler_dump_all(struct drm_device *dev, const char *msg)
> +{
> + struct drm_i915_private *dev_priv = dev->dev_private;
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + unsigned long flags;
> + int ret;
> +
> + spin_lock_irqsave(&scheduler->lock, flags);
> + ret = i915_scheduler_dump_all_locked(dev, msg);
> + spin_unlock_irqrestore(&scheduler->lock, flags);
> +
> + return ret;
> +}
> +
> static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
> bool is_locked)
> {
> @@ -1246,10 +1544,10 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
> continue;
>
> if (!I915_SQS_IS_COMPLETE(node))
> - DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
> + DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%s on %s\n",
> node->params.request->uniq,
> node->params.request->seqno,
> - node->status,
> + i915_qe_state_str(node),
If the code is to be called from outside of i915_debugfs.c, it should
really be written in a more safe manner than static char, especially as
it is not much of an effort to make a table to index.
Regards, Joonas
> ring->name);
>
> i915_scheduler_file_queue_dec(node->params.file);
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index b78de12..a071ebb 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -41,6 +41,9 @@ enum i915_scheduler_queue_status {
> /* Limit value for use with arrays/loops */
> i915_sqs_MAX
> };
> +char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status);
> +const char *i915_scheduler_queue_status_str(
> + enum i915_scheduler_queue_status status);
>
> #define I915_SQS_IS_QUEUED(node) (((node)->status == i915_sqs_queued))
> #define I915_SQS_IS_FLYING(node) (((node)->status == i915_sqs_flying))
> @@ -65,6 +68,7 @@ struct i915_scheduler_queue_entry {
> unsigned long stamp;
> struct list_head link;
> };
> +const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
>
> struct i915_scheduler {
> struct list_head node_queue[I915_NUM_RINGS];
> @@ -82,9 +86,17 @@ struct i915_scheduler {
>
> /* Flag bits for i915_scheduler::flags */
> enum {
> + /* Internal state */
> i915_sf_interrupts_enabled = (1 << 0),
> i915_sf_submitting = (1 << 1),
> +
> + /* Dump/debug flags */
> + i915_sf_dump_force = (1 << 8),
> + i915_sf_dump_details = (1 << 9),
> + i915_sf_dump_dependencies = (1 << 10),
> + i915_sf_dump_seqno = (1 << 11),
> };
> +const char *i915_scheduler_flag_str(uint32_t flags);
>
> bool i915_scheduler_is_enabled(struct drm_device *dev);
> int i915_scheduler_init(struct drm_device *dev);
> @@ -99,6 +111,9 @@ void i915_scheduler_work_handler(struct work_struct *work);
> int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
> int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
> unsigned long stamp, bool is_locked);
> +int i915_scheduler_dump(struct intel_engine_cs *ring,
> + const char *msg);
> +int i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
> bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
> bool *completed, bool *busy);
> bool i915_scheduler_file_queue_is_full(struct drm_file *file);
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls
2016-03-01 10:02 ` Joonas Lahtinen
@ 2016-03-11 11:47 ` John Harrison
0 siblings, 0 replies; 82+ messages in thread
From: John Harrison @ 2016-03-11 11:47 UTC (permalink / raw)
To: Joonas Lahtinen, Intel-GFX
On 01/03/2016 10:02, Joonas Lahtinen wrote:
> On to, 2016-02-18 at 14:27 +0000, John.C.Harrison@Intel.com wrote:
>> From: John Harrison <John.C.Harrison@Intel.com>
>>
>> The scheduler can cause batch buffers, and hence requests, to be
>> submitted to the ring out of order and asynchronously to their
>> submission to the driver. Thus at the point of waiting for the
>> completion of a given request, it is not even guaranteed that the
>> request has actually been sent to the hardware yet. Even it is has
>> been sent, it is possible that it could be pre-empted and thus
>> 'unsent'.
>>
>> This means that it is necessary to be able to submit requests to the
>> hardware during the wait call itself. Unfortunately, while some
>> callers of __wait_request() release the mutex lock first, others do
>> not (and apparently can not). Hence there is the ability to deadlock
>> as the wait stalls for submission but the asynchronous submission is
>> stalled for the mutex lock.
>>
>> This change hooks the scheduler in to the __wait_request() code to
>> ensure correct behaviour. That is, flush the target batch buffer
>> through to the hardware and do not deadlock waiting for something that
>> cannot currently be submitted. Instead, the wait call must return
>> EAGAIN at least as far back as necessary to release the mutex lock and
>> allow the scheduler's asynchronous processing to get in and handle the
>> pre-emption operation and eventually (re-)submit the work.
>>
>> v3: Removed the explicit scheduler flush from i915_wait_request().
>> This is no longer necessary and was causing unintended changes to the
>> scheduler priority level which broke a validation team test.
>>
>> v4: Corrected the format of some comments to keep the style checker
>> happy.
>>
>> v5: Added function description. [Joonas Lahtinen]
>>
>> For: VIZ-1587
>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
>> ---
>> drivers/gpu/drm/i915/i915_drv.h | 3 ++-
>> drivers/gpu/drm/i915/i915_gem.c | 37 ++++++++++++++++++++++++++-------
>> drivers/gpu/drm/i915/i915_scheduler.c | 31 +++++++++++++++++++++++++++
>> drivers/gpu/drm/i915/i915_scheduler.h | 2 ++
>> drivers/gpu/drm/i915/intel_display.c | 5 +++--
>> drivers/gpu/drm/i915/intel_ringbuffer.c | 2 +-
>> 6 files changed, 69 insertions(+), 11 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
>> index 4d544f1..5eeeced 100644
>> --- a/drivers/gpu/drm/i915/i915_drv.h
>> +++ b/drivers/gpu/drm/i915/i915_drv.h
>> @@ -3071,7 +3071,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
>> unsigned reset_counter,
>> bool interruptible,
>> s64 *timeout,
>> - struct intel_rps_client *rps);
>> + struct intel_rps_client *rps,
>> + bool is_locked);
> Multiple bool parameters are better converted to int flags, it makes
> the callsite code more readable if you have enum I915_WAIT_REQ_* flags.
>
>> int __must_check i915_wait_request(struct drm_i915_gem_request *req);
>> int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);
>> int __must_check
>> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
>> index 2dd9b55..17b44b3 100644
>> --- a/drivers/gpu/drm/i915/i915_gem.c
>> +++ b/drivers/gpu/drm/i915/i915_gem.c
>> @@ -1258,7 +1258,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
>> unsigned reset_counter,
>> bool interruptible,
>> s64 *timeout,
>> - struct intel_rps_client *rps)
>> + struct intel_rps_client *rps,
>> + bool is_locked)
>> {
>> struct intel_engine_cs *ring = i915_gem_request_get_ring(req);
>> struct drm_device *dev = ring->dev;
>> @@ -1268,8 +1269,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
>> DEFINE_WAIT(wait);
>> unsigned long timeout_expire;
>> s64 before = 0; /* Only to silence a compiler warning. */
>> - int ret;
>> + int ret = 0;
>> + bool busy;
> Strange whitespace.
>
>>
>> + might_sleep();
> This will splat if is_locked is true? So maybe it should be protected
> by if (!is_locked)?
The locking here is just the driver mutex lock. Whereas the test will
only go bang if in atomic context (spinlock, IRQ, etc.). So it should be
safe. If it isn't then the driver is already broken because the whole
point of _wait_request() is that it will sleep! Although one could argue
it is already broken because it quite likes sleeping with the mutex lock
held.
>> WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");
>>
>> if (i915_gem_request_completed(req))
>> @@ -1324,6 +1327,26 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
>> break;
>> }
>>
>> + if (is_locked) {
>> + /*
>> + * If this request is being processed by the scheduler
>> + * then it is unsafe to sleep with the mutex lock held
>> + * as the scheduler may require the lock in order to
>> + * progress the request.
>> + */
> This is becoming a monstrous function, I think it could be split down
> to two functions, __i915_wait_request and
> __i915_wait_request_locked/nonblocking, because if you take all the
> timeout/sleeping related stuff away, not much is left.
I don't see what you mean. You can't take away all the timeout/sleeping
related stuff, that is the whole point of the function! Everything that
is inside the loop needs to be inside the loop. It is a hideously
complicated wait function but as far as I can tell, it needs to be such.
I recall at some point someone did a refactor to move the contents of
the loop into a separate function but that
doesn't seem to have made it upstream. Or maybe it did and it has since
been reverted?
The scheduler addition is now much simpler due to the re-implementing of
the _is_tracked() function - see comments below.
>> + if (i915_scheduler_is_request_tracked(req, NULL, &busy)) {
>> + if (busy) {
>> + ret = -EAGAIN;
>> + break;
>> + }
>> + }
> If this is kept in single function, would not it be enough to execute
> this before the loop?
It might have changed. The sleep might have been aborted due to
something that means the scheduler is now busy and sleeping with the
mutex lock held will cause a deadlock.
>> +
>> + /*
>> + * If the request is not tracked by the scheduler
>> + * then the regular test can be done.
>> + */
>> + }
>> +
>> if (i915_gem_request_completed(req)) {
>> ret = 0;
>> break;
>> @@ -1542,7 +1565,7 @@ i915_wait_request(struct drm_i915_gem_request *req)
>>
>> ret = __i915_wait_request(req,
>> atomic_read(&dev_priv->gpu_error.reset_counter),
>> - interruptible, NULL, NULL);
>> + interruptible, NULL, NULL, true);
>> if (ret)
>> return ret;
>>
>> @@ -1655,7 +1678,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
>> mutex_unlock(&dev->struct_mutex);
>> for (i = 0; ret == 0 && i < n; i++)
>> ret = __i915_wait_request(requests[i], reset_counter, true,
>> - NULL, rps);
>> + NULL, rps, false);
>> mutex_lock(&dev->struct_mutex);
>>
>> for (i = 0; i < n; i++) {
>> @@ -3511,7 +3534,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
>> if (ret == 0)
>> ret = __i915_wait_request(req[i], reset_counter, true,
>> args->timeout_ns > 0 ? &args->timeout_ns : NULL,
>> - to_rps_client(file));
>> + to_rps_client(file), false);
>> i915_gem_request_unreference(req[i]);
>> }
>> return ret;
>> @@ -3544,7 +3567,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
>> atomic_read(&i915->gpu_error.reset_counter),
>> i915->mm.interruptible,
>> NULL,
>> - &i915->rps.semaphores);
>> + &i915->rps.semaphores, true);
>> if (ret)
>> return ret;
>>
>> @@ -4523,7 +4546,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
>> if (target == NULL)
>> return 0;
>>
>> - ret = __i915_wait_request(target, reset_counter, true, NULL, NULL);
>> + ret = __i915_wait_request(target, reset_counter, true, NULL, NULL, false);
>> if (ret == 0)
>> queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);
>>
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
>> index 60a59d3..edab63d 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.c
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
>> @@ -918,6 +918,37 @@ void i915_scheduler_work_handler(struct work_struct *work)
>> }
>>
>> /**
>> + * i915_scheduler_is_request_tracked - return info to say what the scheduler's
>> + * connection to this request is (if any).
>> + * @req: request to be queried
>> + * @compeleted: if non-null, set to completion status
>> + * @busy: if non-null set to busy status
>> + *
>> + * Looks up the given request in the scheduler's internal queue and reports
>> + * on whether the request has completed or is still pending.
>> + * Returns true if the request was found or false if it was not.
>> + */
>> +bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
>> + bool *completed, bool *busy)
>> +{
>> + struct drm_i915_private *dev_priv = req->ring->dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + if (!scheduler)
>> + return false;
>> +
>> + if (req->scheduler_qe == NULL)
>> + return false;
>> +
> These better be their own functions, making the code more readable.
>
> _is_request_completed()
> _is_request_busy()
Yeah, the original need has actually changed somewhat since this was
written. There is no longer any code which requires all three items
together. So I've split it into two simpler functions which are more
appropriate to their use cases.
>> + if (completed)
>> + *completed = I915_SQS_IS_COMPLETE(req->scheduler_qe);
>> + if (busy)
>> + *busy = I915_SQS_IS_QUEUED(req->scheduler_qe);
>> +
>> + return true;
>> +}
>> +
>> +/**
>> * i915_scheduler_closefile - notify the scheduler that a DRM file handle
>> * has been closed.
>> * @dev: DRM device
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
>> index 180d75f..a88adce 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.h
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
>> @@ -94,5 +94,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
>> bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
>> void i915_scheduler_wakeup(struct drm_device *dev);
>> void i915_scheduler_work_handler(struct work_struct *work);
>> +bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
>> + bool *completed, bool *busy);
>>
>> #endif /* _I915_SCHEDULER_H_ */
>> diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
>> index 731d20a..5953590 100644
>> --- a/drivers/gpu/drm/i915/intel_display.c
>> +++ b/drivers/gpu/drm/i915/intel_display.c
>> @@ -11458,7 +11458,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work)
>> WARN_ON(__i915_wait_request(mmio_flip->req,
>> mmio_flip->crtc->reset_counter,
>> false, NULL,
>> - &mmio_flip->i915->rps.mmioflips));
>> + &mmio_flip->i915->rps.mmioflips,
>> + false));
>> i915_gem_request_unreference(mmio_flip->req);
>> }
>>
>> @@ -13523,7 +13524,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev,
>>
>> ret = __i915_wait_request(intel_plane_state->wait_req,
>> reset_counter, true,
>> - NULL, NULL);
>> + NULL, NULL, false);
>>
>> /* Swallow -EIO errors to allow updates during hw lockup. */
>> if (ret == -EIO)
>> diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
>> index ca7b8af..a2093f5 100644
>> --- a/drivers/gpu/drm/i915/intel_ringbuffer.c
>> +++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
>> @@ -2304,7 +2304,7 @@ int intel_ring_idle(struct intel_engine_cs *ring)
>> return __i915_wait_request(req,
>> atomic_read(&to_i915(ring->dev)->gpu_error.reset_counter),
>> to_i915(ring->dev)->mm.interruptible,
>> - NULL, NULL);
>> + NULL, NULL, true);
>> }
>>
>> int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions
2016-03-07 11:31 ` Joonas Lahtinen
@ 2016-03-11 16:22 ` John Harrison
0 siblings, 0 replies; 82+ messages in thread
From: John Harrison @ 2016-03-11 16:22 UTC (permalink / raw)
To: Joonas Lahtinen, Intel-GFX
On 07/03/2016 11:31, Joonas Lahtinen wrote:
> On to, 2016-02-18 at 14:27 +0000, John.C.Harrison@Intel.com wrote:
>> From: John Harrison <John.C.Harrison@Intel.com>
>>
>> When requesting that all GPU work is completed, it is now necessary to
>> get the scheduler involved in order to flush out work that queued and
>> not yet submitted.
>>
>> v2: Updated to add support for flushing the scheduler queue by time
>> stamp rather than just doing a blanket flush.
>>
>> v3: Moved submit_max_priority() to this patch from an earlier patch
>> is it is no longer required in the other.
>>
>> v4: Corrected the format of a comment to keep the style checker happy.
>> Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
>>
>> v5: Shuffled functions around to remove forward prototypes, removed
>> similarly offensive white space and added documentation. Re-worked the
>> mutex locking around the submit function. [Joonas Lahtinen]
>>
>> Used lighter weight spinlocks.
>>
>> For: VIZ-1587
>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
>> ---
>> drivers/gpu/drm/i915/i915_gem.c | 24 ++++-
>> drivers/gpu/drm/i915/i915_scheduler.c | 178 ++++++++++++++++++++++++++++++++++
>> drivers/gpu/drm/i915/i915_scheduler.h | 3 +
>> 3 files changed, 204 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
>> index a47a495..d946f53 100644
>> --- a/drivers/gpu/drm/i915/i915_gem.c
>> +++ b/drivers/gpu/drm/i915/i915_gem.c
>> @@ -3786,6 +3786,10 @@ int i915_gpu_idle(struct drm_device *dev)
>>
>> /* Flush everything onto the inactive list. */
>> for_each_ring(ring, dev_priv, i) {
>> + ret = i915_scheduler_flush(ring, true);
>> + if (ret < 0)
>> + return ret;
>> +
>> if (!i915.enable_execlists) {
>> struct drm_i915_gem_request *req;
>>
>> @@ -4519,7 +4523,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
>> unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES;
>> struct drm_i915_gem_request *request, *target = NULL;
>> unsigned reset_counter;
>> - int ret;
>> + int i, ret;
>> + struct intel_engine_cs *ring;
>>
>> ret = i915_gem_wait_for_error(&dev_priv->gpu_error);
>> if (ret)
>> @@ -4529,6 +4534,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
>> if (ret)
>> return ret;
>>
>> + for_each_ring(ring, dev_priv, i) {
>> + /*
>> + * Flush out scheduler entries that are getting 'stale'. Note
>> + * that the following recent_enough test will only check
>> + * against the time at which the request was submitted to the
>> + * hardware (i.e. when it left the scheduler) not the time it
>> + * was submitted to the driver.
>> + *
>> + * Also, there is not much point worring about busy return
>> + * codes from the scheduler flush call. Even if more work
>> + * cannot be submitted right now for whatever reason, we
>> + * still want to throttle against stale work that has already
>> + * been submitted.
>> + */
>> + i915_scheduler_flush_stamp(ring, recent_enough, false);
>> + }
>> +
>> spin_lock(&file_priv->mm.lock);
>> list_for_each_entry(request, &file_priv->mm.request_list, client_list) {
>> if (time_after_eq(request->emitted_jiffies, recent_enough))
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
>> index edab63d..8130a9c 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.c
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
>> @@ -304,6 +304,10 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
>> * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
>> * And releasing the one before acquiring the other leads to other code
>> * being run and interfering.
>> + *
>> + * Hence any caller that does not already have the mutex lock for other
>> + * reasons should call i915_scheduler_submit_unlocked() instead in order to
>> + * obtain the lock first.
>> */
>> static int i915_scheduler_submit(struct intel_engine_cs *ring)
>> {
>> @@ -428,6 +432,22 @@ error:
>> return ret;
>> }
>>
>> +static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)
>> +{
>> + struct drm_device *dev = ring->dev;
>> + int ret;
>> +
>> + ret = i915_mutex_lock_interruptible(dev);
>> + if (ret)
>> + return ret;
>> +
>> + ret = i915_scheduler_submit(ring);
>> +
>> + mutex_unlock(&dev->struct_mutex);
>> +
>> + return ret;
>> +}
>> +
> This kind of code should really be factored to upper level functions.
See below.
>> static void i915_generate_dependencies(struct i915_scheduler *scheduler,
>> struct i915_scheduler_queue_entry *node,
>> uint32_t ring)
>> @@ -917,6 +937,164 @@ void i915_scheduler_work_handler(struct work_struct *work)
>> i915_scheduler_process_work(ring);
>> }
>>
>> +static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
>> + bool is_locked)
>> +{
>> + struct i915_scheduler_queue_entry *node;
>> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + int ret, count = 0;
>> + bool found;
>> +
>> + do {
>> + found = false;
>> + spin_lock_irq(&scheduler->lock);
>> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
>> + if (!I915_SQS_IS_QUEUED(node))
>> + continue;
>> +
>> + if (node->priority < scheduler->priority_level_max)
>> + continue;
>> +
>> + found = true;
>> + break;
>> + }
>> + spin_unlock_irq(&scheduler->lock);
>> +
>> + if (!found)
>> + break;
>> +
>> + if (is_locked)
>> + ret = i915_scheduler_submit(ring);
>> + else
>> + ret = i915_scheduler_submit_unlocked(ring);
> The widespread use of 'is_locked' notation is really bothering me, I
> think the difference in code path should be handled in the upper level,
> instead of being passed down, for example this will cause repetitive
> lock/unlock in loop.
My aim has been to try to keep the amount of code inside any given lock
to the absolute minimum. One could just push the mutex lock acquire all
the way up to the top level before any of the scheduler code is called.
To me, that just seems an excessive and unnecessary hog of the global
driver lock.
>
>> + if (ret < 0)
>> + return ret;
>> +
>> + count += ret;
>> + } while (found);
>> +
>> + return count;
>> +}
>> +
>> +/**
>> + * i915_scheduler_flush_stamp - force requests of a given age through the
>> + * scheduler.
>> + * @ring: Ring to be flushed
>> + * @target: Jiffy based time stamp to flush up to
>> + * @is_locked: Is the driver mutex lock held?
>> + * DRM has a throttle by age of request facility. This requires waiting for
>> + * outstanding work over a given age. This function helps that by forcing
>> + * queued batch buffers over said age through the system.
>> + * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
>> + * for a pre-emption event to complete) but the mutex lock is held which
>> + * would prevent the scheduler's asynchronous processing from completing.
>> + */
>> +int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
>> + unsigned long target,
>> + bool is_locked)
>> +{
>> + struct i915_scheduler_queue_entry *node;
>> + struct drm_i915_private *dev_priv;
>> + struct i915_scheduler *scheduler;
>> + int flush_count = 0;
>> +
>> + if (!ring)
>> + return -EINVAL;
>> +
>> + dev_priv = ring->dev->dev_private;
>> + scheduler = dev_priv->scheduler;
>> +
>> + if (!scheduler)
>> + return 0;
>> +
>> + if (is_locked && (scheduler->flags[ring->id] & i915_sf_submitting)) {
>> + /*
>> + * Scheduler is busy already submitting another batch,
>> + * come back later rather than going recursive...
>> + */
>> + return -EAGAIN;
>> + }
>> +
> Again, this would be handled better at higher level.
>
> So I would like to see those 'is_locked' and other similar constructs
> factored out from the series. It should make the code much more robust
> to modify at later point.
As above. There is no need to lock the global driver mutex while doing
internal scheduler processing. If you think the trade off of unnecessary
locking for code simplification is best as just lock and be simple then
sure, all the 'is_locked' stuff can disappear. Otherwise, I don't see
any easy way to delay the mutex lock until it is actually required.
>
> Regards, Joonas
>
>> + spin_lock_irq(&scheduler->lock);
>> + i915_scheduler_priority_bump_clear(scheduler);
>> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
>> + if (!I915_SQS_IS_QUEUED(node))
>> + continue;
>> +
>> + if (node->stamp > target)
>> + continue;
>> +
>> + flush_count = i915_scheduler_priority_bump(scheduler,
>> + node, scheduler->priority_level_max);
>> + }
>> + spin_unlock_irq(&scheduler->lock);
>> +
>> + if (flush_count) {
>> + DRM_DEBUG_DRIVER("<%s> Bumped %d entries\n", ring->name, flush_count);
>> + flush_count = i915_scheduler_submit_max_priority(ring, is_locked);
>> + }
>> +
>> + return flush_count;
>> +}
>> +
>> +/**
>> + * i915_scheduler_flush - force all requests through the scheduler.
>> + * @ring: Ring to be flushed
>> + * @is_locked: Is the driver mutex lock held?
>> + * For various reasons it is sometimes necessary to the scheduler out, e.g.
>> + * due to ring reset.
>> + * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
>> + * for a pre-emption event to complete) but the mutex lock is held which
>> + * would prevent the scheduler's asynchronous processing from completing.
>> + */
>> +int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked)
>> +{
>> + struct i915_scheduler_queue_entry *node;
>> + struct drm_i915_private *dev_priv;
>> + struct i915_scheduler *scheduler;
>> + bool found;
>> + int ret;
>> + uint32_t count = 0;
>> +
>> + if (!ring)
>> + return -EINVAL;
>> +
>> + dev_priv = ring->dev->dev_private;
>> + scheduler = dev_priv->scheduler;
>> +
>> + if (!scheduler)
>> + return 0;
>> +
>> + WARN_ON(is_locked && (scheduler->flags[ring->id] & i915_sf_submitting));
>> +
>> + do {
>> + found = false;
>> + spin_lock_irq(&scheduler->lock);
>> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
>> + if (!I915_SQS_IS_QUEUED(node))
>> + continue;
>> +
>> + found = true;
>> + break;
>> + }
>> + spin_unlock_irq(&scheduler->lock);
>> +
>> + if (found) {
>> + if (is_locked)
>> + ret = i915_scheduler_submit(ring);
>> + else
>> + ret = i915_scheduler_submit_unlocked(ring);
>> + if (ret < 0)
>> + return ret;
>> +
>> + count += ret;
>> + }
>> + } while (found);
>> +
>> + return count;
>> +}
>> +
>> /**
>> * i915_scheduler_is_request_tracked - return info to say what the scheduler's
>> * connection to this request is (if any).
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
>> index a88adce..839b048 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.h
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
>> @@ -94,6 +94,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
>> bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
>> void i915_scheduler_wakeup(struct drm_device *dev);
>> void i915_scheduler_work_handler(struct work_struct *work);
>> +int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
>> +int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
>> + unsigned long stamp, bool is_locked);
>> bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
>> bool *completed, bool *busy);
>>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters
2016-02-23 21:06 ` Jesse Barnes
@ 2016-03-11 16:28 ` John Harrison
2016-03-11 17:25 ` Jesse Barnes
0 siblings, 1 reply; 82+ messages in thread
From: John Harrison @ 2016-03-11 16:28 UTC (permalink / raw)
To: Jesse Barnes, Intel-GFX
On 23/02/2016 21:06, Jesse Barnes wrote:
> On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
>> From: John Harrison <John.C.Harrison@Intel.com>
>>
>> There are various parameters within the scheduler which can be tuned
>> to improve performance, reduce memory footprint, etc. This change adds
>> support for altering these via debugfs.
>>
>> v2: Updated for priorities now being signed values.
>>
>> v5: Squashed priority bumping entries into this patch rather than a
>> separate patch all of their own.
>>
>> For: VIZ-1587
>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>> ---
>> drivers/gpu/drm/i915/i915_debugfs.c | 169 ++++++++++++++++++++++++++++++++++++
>> 1 file changed, 169 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
>> index b923949..7d01c07 100644
>> --- a/drivers/gpu/drm/i915/i915_debugfs.c
>> +++ b/drivers/gpu/drm/i915/i915_debugfs.c
>> @@ -39,6 +39,7 @@
>> #include "intel_ringbuffer.h"
>> #include <drm/i915_drm.h>
>> #include "i915_drv.h"
>> +#include "i915_scheduler.h"
>>
>> enum {
>> ACTIVE_LIST,
>> @@ -1122,6 +1123,168 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops,
>> i915_next_seqno_get, i915_next_seqno_set,
>> "0x%llx\n");
>>
>> +static int
>> +i915_scheduler_priority_min_get(void *data, u64 *val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + *val = (u64) scheduler->priority_level_min;
>> + return 0;
>> +}
>> +
>> +static int
>> +i915_scheduler_priority_min_set(void *data, u64 val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + scheduler->priority_level_min = (int32_t) val;
>> + return 0;
>> +}
>> +
>> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_min_fops,
>> + i915_scheduler_priority_min_get,
>> + i915_scheduler_priority_min_set,
>> + "%lld\n");
>> +
>> +static int
>> +i915_scheduler_priority_max_get(void *data, u64 *val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + *val = (u64) scheduler->priority_level_max;
>> + return 0;
>> +}
>> +
>> +static int
>> +i915_scheduler_priority_max_set(void *data, u64 val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + scheduler->priority_level_max = (int32_t) val;
>> + return 0;
>> +}
>> +
>> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_max_fops,
>> + i915_scheduler_priority_max_get,
>> + i915_scheduler_priority_max_set,
>> + "%lld\n");
>> +
>> +static int
>> +i915_scheduler_priority_bump_get(void *data, u64 *val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + *val = (u64) scheduler->priority_level_bump;
>> + return 0;
>> +}
>> +
>> +static int
>> +i915_scheduler_priority_bump_set(void *data, u64 val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + scheduler->priority_level_bump = (u32) val;
>> + return 0;
>> +}
>> +
>> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_bump_fops,
>> + i915_scheduler_priority_bump_get,
>> + i915_scheduler_priority_bump_set,
>> + "%lld\n");
>> +
>> +static int
>> +i915_scheduler_priority_preempt_get(void *data, u64 *val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + *val = (u64) scheduler->priority_level_preempt;
>> + return 0;
>> +}
>> +
>> +static int
>> +i915_scheduler_priority_preempt_set(void *data, u64 val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + scheduler->priority_level_preempt = (u32) val;
>> + return 0;
>> +}
>> +
>> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_preempt_fops,
>> + i915_scheduler_priority_preempt_get,
>> + i915_scheduler_priority_preempt_set,
>> + "%lld\n");
>> +
>> +static int
>> +i915_scheduler_min_flying_get(void *data, u64 *val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + *val = (u64) scheduler->min_flying;
>> + return 0;
>> +}
>> +
>> +static int
>> +i915_scheduler_min_flying_set(void *data, u64 val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + scheduler->min_flying = (u32) val;
>> + return 0;
>> +}
>> +
>> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_min_flying_fops,
>> + i915_scheduler_min_flying_get,
>> + i915_scheduler_min_flying_set,
>> + "%llu\n");
>> +
>> +static int
>> +i915_scheduler_file_queue_max_get(void *data, u64 *val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + *val = (u64) scheduler->file_queue_max;
>> + return 0;
>> +}
>> +
>> +static int
>> +i915_scheduler_file_queue_max_set(void *data, u64 val)
>> +{
>> + struct drm_device *dev = data;
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> +
>> + scheduler->file_queue_max = (u32) val;
>> + return 0;
>> +}
>> +
>> +DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
>> + i915_scheduler_file_queue_max_get,
>> + i915_scheduler_file_queue_max_set,
>> + "%llu\n");
>> +
>> static int i915_frequency_info(struct seq_file *m, void *unused)
>> {
>> struct drm_info_node *node = m->private;
>> @@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files {
>> {"i915_gem_drop_caches", &i915_drop_caches_fops},
>> {"i915_error_state", &i915_error_state_fops},
>> {"i915_next_seqno", &i915_next_seqno_fops},
>> + {"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
>> + {"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
>> + {"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops},
>> + {"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
>> + {"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
>> + {"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
>> {"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
>> {"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
>> {"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
>>
> Do these need to be serialized at all? I guess some raciness doesn't hurt too much for these guys, unless somehow an inconsistent set of values would cause a livelock in the scheduler or something.
Serialised with what? Each other or the scheduler operation? Neither
should be necessary. The scheduler will read the current values whenever
it tests against one of these limits. If multiple are being changed
while the system is busy, it doesn't really matter. They are just tuning
values and best guesses type of numbers not array indices or other
things that would cause kernel panics if you got them wrong. E.g. if you
set the max file queue depth to smaller than the current queue contents
that just means you won't be able to submit more stuff until the queue
has drained - which is presumably the intended result of lowering the
max queue value anyway. The queue won't leak the extra entries or get
into an inconsistent state.
>
> If not,
> Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 27/35] drm/i915: Added debug state dump facilities to scheduler
2016-03-07 12:31 ` Joonas Lahtinen
@ 2016-03-11 16:38 ` John Harrison
2016-03-15 10:53 ` Joonas Lahtinen
0 siblings, 1 reply; 82+ messages in thread
From: John Harrison @ 2016-03-11 16:38 UTC (permalink / raw)
To: Joonas Lahtinen, Intel-GFX
The intention of the state dump code is not really for it to be a user
accessible debugfs entry (although one of the later patches does add a
debugfs interface). It is more intended for debugging lock ups and
unexpected behaviour by adding a dump function call to whatever WARN_ON
or similar is being hit. So in theory, the code should not be necessary
and there is no point upstreaming it. On the other hand, it does come in
very handy when debugging scheduler related issues and it is complicated
enough that you can't just knock it up in five minutes when a bug report
is logged.
The code could certainly be moved into a separate file, e.g.
i915_scheduler_debug.c. I don't think it would be good to move it to
debugfs and make it all 'seq_printf' style output only. It is much more
use as ordinary printk style output so you can call it directly at the
point of detecting an inconsistent state and get the dmesg output prior
to the kernel panic.
On 07/03/2016 12:31, Joonas Lahtinen wrote:
> Hi,
>
> On to, 2016-02-18 at 14:27 +0000, John.C.Harrison@Intel.com wrote:
>> From: John Harrison <John.C.Harrison@Intel.com>
>>
>> When debugging batch buffer submission issues, it is useful to be able
>> to see what the current state of the scheduler is. This change adds
>> functions for decoding the internal scheduler state and reporting it.
>>
>> v3: Updated a debug message with the new state_str() function.
>>
>> v4: Wrapped some long lines to keep the style checker happy. Removed
>> the fence/sync code as that will now be part of a separate patch series.
>>
>> v5: Removed forward declarations and white space. Added documentation.
>> [Joonas Lahtinen]
>>
>> Also squashed in later patch to add seqno information from the start.
>> It was only being added in a separate patch due to historical reasons
>> which have since gone away.
>>
>> For: VIZ-1587
>> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
>> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
>> ---
>> drivers/gpu/drm/i915/i915_scheduler.c | 302 +++++++++++++++++++++++++++++++++-
>> drivers/gpu/drm/i915/i915_scheduler.h | 15 ++
>> 2 files changed, 315 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
>> index f7f29d5..d0eed52 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.c
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
>> @@ -40,6 +40,117 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
>> return dev_priv->scheduler != NULL;
>> }
>>
> These sort of functions should be in i915_debugfs.c, so that nobody
> even thinks of using them for other purposes.
>
>>
>> +const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node)
>> +{
>> + static char str[50];
>> + char *ptr = str;
>> +
>> + *(ptr++) = node->bumped ? 'B' : '-',
>> + *(ptr++) = i915_gem_request_completed(node->params.request) ? 'C' : '-';
>> +
> Especially this kind of code needs to be in i915_debugfs.c. I'd
> implement new code more along lines of i915_sseu_status(), but I see
> this kind of code was previously merged.
>
>> + *ptr = 0;
>> +
>> + return str;
>> +}
>> +
>> +char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status)
>> +{
>> + switch (status) {
>> + case i915_sqs_none:
>> + return 'N';
>> +
>> + case i915_sqs_queued:
>> + return 'Q';
>> +
>> + case i915_sqs_popped:
>> + return 'X';
>> +
>> + case i915_sqs_flying:
>> + return 'F';
>> +
>> + case i915_sqs_complete:
>> + return 'C';
>> +
>> + case i915_sqs_dead:
>> + return 'D';
>> +
>> + default:
>> + break;
>> + }
>> +
> Bad indent.
>
>> + return '?';
>> +}
>> +
>> +const char *i915_scheduler_queue_status_str(
>> + enum i915_scheduler_queue_status status)
>> +{
>> + static char str[50];
>> +
>> + switch (status) {
>> + case i915_sqs_none:
>> + return "None";
>> +
>> + case i915_sqs_queued:
>> + return "Queued";
>> +
>> + case i915_sqs_popped:
>> + return "Popped";
>> +
>> + case i915_sqs_flying:
>> + return "Flying";
>> +
>> + case i915_sqs_complete:
>> + return "Complete";
>> +
>> + case i915_sqs_dead:
>> + return "Dead";
>> +
>> + default:
>> + break;
>> + }
>> +
>> + sprintf(str, "[Unknown_%d!]", status);
>> + return str;
>> +}
>> +
>> +const char *i915_scheduler_flag_str(uint32_t flags)
>> +{
>> + static char str[100];
>> + char *ptr = str;
>> +
>> + *ptr = 0;
>> +
>> +#define TEST_FLAG(flag, msg) \
>> + do { \
>> + if (flags & (flag)) { \
>> + strcpy(ptr, msg); \
>> + ptr += strlen(ptr); \
>> + flags &= ~(flag); \
>> + } \
>> + } while (0)
>> +
>> + TEST_FLAG(i915_sf_interrupts_enabled, "IntOn|");
>> + TEST_FLAG(i915_sf_submitting, "Submitting|");
>> + TEST_FLAG(i915_sf_dump_force, "DumpForce|");
>> + TEST_FLAG(i915_sf_dump_details, "DumpDetails|");
>> + TEST_FLAG(i915_sf_dump_dependencies, "DumpDeps|");
>> + TEST_FLAG(i915_sf_dump_seqno, "DumpSeqno|");
>> +
>> +#undef TEST_FLAG
>> +
>> + if (flags) {
>> + sprintf(ptr, "Unknown_0x%X!", flags);
>> + ptr += strlen(ptr);
>> + }
>> +
>> + if (ptr == str)
>> + strcpy(str, "-");
>> + else
>> + ptr[-1] = 0;
>> +
>> + return str;
>> +};
>> +
>> /**
>> * i915_scheduler_init - Initialise the scheduler.
>> * @dev: DRM device
>> @@ -1024,6 +1135,193 @@ void i915_scheduler_work_handler(struct work_struct *work)
>> i915_scheduler_process_work(ring);
>> }
>>
>> +static int i915_scheduler_dump_locked(struct intel_engine_cs *ring,
>> + const char *msg)
>> +{
>> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + struct i915_scheduler_queue_entry *node;
>> + int flying = 0, queued = 0, complete = 0, other = 0;
>> + static int old_flying = -1, old_queued = -1, old_complete = -1;
>> + bool b_dump;
>> + char brkt[2] = { '<', '>' };
>> +
>> + if (!ring)
>> + return -EINVAL;
>> +
>> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
>> + if (I915_SQS_IS_QUEUED(node))
>> + queued++;
>> + else if (I915_SQS_IS_FLYING(node))
>> + flying++;
>> + else if (I915_SQS_IS_COMPLETE(node))
>> + complete++;
>> + else
>> + other++;
>> + }
>> +
>> + b_dump = (flying != old_flying) ||
>> + (queued != old_queued) ||
>> + (complete != old_complete);
>> + if (scheduler->flags[ring->id] & i915_sf_dump_force) {
>> + if (!b_dump) {
>> + b_dump = true;
>> + brkt[0] = '{';
>> + brkt[1] = '}';
>> + }
>> +
>> + scheduler->flags[ring->id] &= ~i915_sf_dump_force;
>> + }
>> +
>> + if (b_dump) {
>> + old_flying = flying;
>> + old_queued = queued;
>> + old_complete = complete;
>> + DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d, "
>> + "Flags = %s, Next = %d:%d %c%s%c\n",
>> + ring->name, queued, flying, complete, other,
>> + i915_scheduler_flag_str(scheduler->flags[ring->id]),
>> + dev_priv->request_uniq, dev_priv->next_seqno,
>> + brkt[0], msg, brkt[1]);
> Convert to debugfs, it's the de-facto.
>
>> + } else {
>> + /*DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d"
>> + ", Flags = %s, Next = %d:%d [%s]\n",
>> + ring->name,
>> + queued, flying, complete, other,
>> + i915_scheduler_flag_str(scheduler->flags[ring->id]),
>> + dev_priv->request_uniq, dev_priv->next_seqno, msg); */
>> +
> Dead code to be removed from patches.
>
>> + return 0;
>> + }
>> +
>> + if (scheduler->flags[ring->id] & i915_sf_dump_seqno) {
>> + uint32_t seqno;
>> +
>> + seqno = ring->get_seqno(ring, true);
>> +
>> + DRM_DEBUG_DRIVER("<%s> Seqno = %d\n", ring->name, seqno);
>> + }
>> +
>> + if (scheduler->flags[ring->id] & i915_sf_dump_details) {
>> + int i, deps;
>> + uint32_t count, counts[i915_sqs_MAX];
>> +
>> + memset(counts, 0x00, sizeof(counts));
>> +
>> + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
>> + if (node->status < i915_sqs_MAX) {
>> + count = counts[node->status]++;
>> + } else {
>> + DRM_DEBUG_DRIVER("<%s> Unknown status: %d!\n",
>> + ring->name, node->status);
>> + count = -1;
>> + }
>> +
>> + deps = 0;
>> + for (i = 0; i < node->num_deps; i++)
>> + if (i915_scheduler_is_dependency_valid(node, i))
>> + deps++;
>> +
>> + DRM_DEBUG_DRIVER("<%s> %c:%02d> uniq = %d, seqno"
>> + " = %d/%s, deps = %d / %d, %s [pri = "
>> + "%4d]\n", ring->name,
>> + i915_scheduler_queue_status_chr(node->status),
>> + count,
>> + node->params.request->uniq,
>> + node->params.request->seqno,
>> + node->params.ring->name,
>> + deps, node->num_deps,
>> + i915_qe_state_str(node),
>> + node->priority);
>> +
>> + if ((scheduler->flags[ring->id] & i915_sf_dump_dependencies)
>> + == 0)
>> + continue;
>> +
>> + for (i = 0; i < node->num_deps; i++)
>> + if (node->dep_list[i])
>> + DRM_DEBUG_DRIVER("<%s> |-%c:"
>> + "%02d%c uniq = %d, seqno = %d/%s, %s [pri = %4d]\n",
>> + ring->name,
>> + i915_scheduler_queue_status_chr(node->dep_list[i]->status),
>> + i,
>> + i915_scheduler_is_dependency_valid(node, i)
>> + ? '>' : '#',
>> + node->dep_list[i]->params.request->uniq,
>> + node->dep_list[i]->params.request->seqno,
>> + node->dep_list[i]->params.ring->name,
>> + i915_qe_state_str(node->dep_list[i]),
>> + node->dep_list[i]->priority);
>> + }
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +/**
>> + * i915_scheduler_dump - dump the scheduler's internal state to the debug log.
>> + * @ring: Ring to dump info for
>> + * @msg: A reason why it is being dumped
>> + * For debugging purposes, it can be very useful to see the internal state of
>> + * the scheduler for a given ring.
>> + */
>> +int i915_scheduler_dump(struct intel_engine_cs *ring, const char *msg)
>> +{
>> + struct drm_i915_private *dev_priv = ring->dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + unsigned long flags;
>> + int ret;
>> +
>> + spin_lock_irqsave(&scheduler->lock, flags);
>> + ret = i915_scheduler_dump_locked(ring, msg);
>> + spin_unlock_irqrestore(&scheduler->lock, flags);
>> +
>> + return ret;
>> +}
>> +
> This kind of stuff especially should be static functions in
> i915_debugfs.c. And then they don't need kerneldoc either, because we
> expect no sane invidual to call them.
>
>> +static int i915_scheduler_dump_all_locked(struct drm_device *dev,
>> + const char *msg)
>> +{
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + struct intel_engine_cs *ring;
>> + int i, r, ret = 0;
>> +
>> + for_each_ring(ring, dev_priv, i) {
>> + scheduler->flags[ring->id] |= i915_sf_dump_force |
>> + i915_sf_dump_details |
>> + i915_sf_dump_seqno |
>> + i915_sf_dump_dependencies;
> This looks really bad to me, invasive debugging. Rather to be carried
> as function parameters in the debug functions, and they should be
> uppercase enum in i915_debugfs.c too.
>
>> + r = i915_scheduler_dump_locked(ring, msg);
>> + if (ret == 0)
>> + ret = r;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +/**
>> + * i915_scheduler_dump_all - dump the scheduler's internal state to the debug
>> + * log.
>> + * @dev: DRM device
>> + * @msg: A reason why it is being dumped
>> + * For debugging purposes, it can be very useful to see the internal state of
>> + * the scheduler.
>> + */
>> +int i915_scheduler_dump_all(struct drm_device *dev, const char *msg)
>> +{
>> + struct drm_i915_private *dev_priv = dev->dev_private;
>> + struct i915_scheduler *scheduler = dev_priv->scheduler;
>> + unsigned long flags;
>> + int ret;
>> +
>> + spin_lock_irqsave(&scheduler->lock, flags);
>> + ret = i915_scheduler_dump_all_locked(dev, msg);
>> + spin_unlock_irqrestore(&scheduler->lock, flags);
>> +
>> + return ret;
>> +}
>> +
>> static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
>> bool is_locked)
>> {
>> @@ -1246,10 +1544,10 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
>> continue;
>>
>> if (!I915_SQS_IS_COMPLETE(node))
>> - DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
>> + DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%s on %s\n",
>> node->params.request->uniq,
>> node->params.request->seqno,
>> - node->status,
>> + i915_qe_state_str(node),
> If the code is to be called from outside of i915_debugfs.c, it should
> really be written in a more safe manner than static char, especially as
> it is not much of an effort to make a table to index.
>
> Regards, Joonas
>
>> ring->name);
>>
>> i915_scheduler_file_queue_dec(node->params.file);
>> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
>> index b78de12..a071ebb 100644
>> --- a/drivers/gpu/drm/i915/i915_scheduler.h
>> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
>> @@ -41,6 +41,9 @@ enum i915_scheduler_queue_status {
>> /* Limit value for use with arrays/loops */
>> i915_sqs_MAX
>> };
>> +char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status);
>> +const char *i915_scheduler_queue_status_str(
>> + enum i915_scheduler_queue_status status);
>>
>> #define I915_SQS_IS_QUEUED(node) (((node)->status == i915_sqs_queued))
>> #define I915_SQS_IS_FLYING(node) (((node)->status == i915_sqs_flying))
>> @@ -65,6 +68,7 @@ struct i915_scheduler_queue_entry {
>> unsigned long stamp;
>> struct list_head link;
>> };
>> +const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
>>
>> struct i915_scheduler {
>> struct list_head node_queue[I915_NUM_RINGS];
>> @@ -82,9 +86,17 @@ struct i915_scheduler {
>>
>> /* Flag bits for i915_scheduler::flags */
>> enum {
>> + /* Internal state */
>> i915_sf_interrupts_enabled = (1 << 0),
>> i915_sf_submitting = (1 << 1),
>> +
>> + /* Dump/debug flags */
>> + i915_sf_dump_force = (1 << 8),
>> + i915_sf_dump_details = (1 << 9),
>> + i915_sf_dump_dependencies = (1 << 10),
>> + i915_sf_dump_seqno = (1 << 11),
>> };
>> +const char *i915_scheduler_flag_str(uint32_t flags);
>>
>> bool i915_scheduler_is_enabled(struct drm_device *dev);
>> int i915_scheduler_init(struct drm_device *dev);
>> @@ -99,6 +111,9 @@ void i915_scheduler_work_handler(struct work_struct *work);
>> int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
>> int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
>> unsigned long stamp, bool is_locked);
>> +int i915_scheduler_dump(struct intel_engine_cs *ring,
>> + const char *msg);
>> +int i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
>> bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
>> bool *completed, bool *busy);
>> bool i915_scheduler_file_queue_is_full(struct drm_file *file);
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters
2016-03-11 16:28 ` John Harrison
@ 2016-03-11 17:25 ` Jesse Barnes
0 siblings, 0 replies; 82+ messages in thread
From: Jesse Barnes @ 2016-03-11 17:25 UTC (permalink / raw)
To: John Harrison, Intel-GFX
On 03/11/2016 08:28 AM, John Harrison wrote:
> On 23/02/2016 21:06, Jesse Barnes wrote:
>> On 02/18/2016 06:27 AM, John.C.Harrison@Intel.com wrote:
>>> From: John Harrison <John.C.Harrison@Intel.com>
>>> struct drm_info_node *node = m->private;
>>> @@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files {
>>> {"i915_gem_drop_caches", &i915_drop_caches_fops},
>>> {"i915_error_state", &i915_error_state_fops},
>>> {"i915_next_seqno", &i915_next_seqno_fops},
>>> + {"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
>>> + {"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
>>> + {"i915_scheduler_priority_bump",
>>> &i915_scheduler_priority_bump_fops},
>>> + {"i915_scheduler_priority_preempt",
>>> &i915_scheduler_priority_preempt_fops},
>>> + {"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
>>> + {"i915_scheduler_file_queue_max",
>>> &i915_scheduler_file_queue_max_fops},
>>> {"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
>>> {"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
>>> {"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
>>>
>> Do these need to be serialized at all? I guess some raciness doesn't
>> hurt too much for these guys, unless somehow an inconsistent set of
>> values would cause a livelock in the scheduler or something.
> Serialised with what? Each other or the scheduler operation? Neither
> should be necessary. The scheduler will read the current values whenever
> it tests against one of these limits. If multiple are being changed
> while the system is busy, it doesn't really matter. They are just tuning
> values and best guesses type of numbers not array indices or other
> things that would cause kernel panics if you got them wrong. E.g. if you
> set the max file queue depth to smaller than the current queue contents
> that just means you won't be able to submit more stuff until the queue
> has drained - which is presumably the intended result of lowering the
> max queue value anyway. The queue won't leak the extra entries or get
> into an inconsistent state.
Yeah I meant both serialized against scheduler accesses/usage and
atomicity with respect to one another. Sounds like it doesn't matter
too much though, so it's fine with me.
Jesse
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [PATCH v5 27/35] drm/i915: Added debug state dump facilities to scheduler
2016-03-11 16:38 ` John Harrison
@ 2016-03-15 10:53 ` Joonas Lahtinen
0 siblings, 0 replies; 82+ messages in thread
From: Joonas Lahtinen @ 2016-03-15 10:53 UTC (permalink / raw)
To: John Harrison, Intel-GFX
Hi,
On pe, 2016-03-11 at 16:38 +0000, John Harrison wrote:
> The intention of the state dump code is not really for it to be a user
> accessible debugfs entry (although one of the later patches does add a
> debugfs interface). It is more intended for debugging lock ups and
> unexpected behaviour by adding a dump function call to whatever WARN_ON
> or similar is being hit. So in theory, the code should not be necessary
> and there is no point upstreaming it. On the other hand, it does come in
> very handy when debugging scheduler related issues and it is complicated
> enough that you can't just knock it up in five minutes when a bug report
> is logged.
>
> The code could certainly be moved into a separate file, e.g.
> i915_scheduler_debug.c. I don't think it would be good to move it to
> debugfs and make it all 'seq_printf' style output only. It is much more
> use as ordinary printk style output so you can call it directly at the
> point of detecting an inconsistent state and get the dmesg output prior
> to the kernel panic.
>
In that case, it could be moved to it's own file and enabled
conditionally through a config parameter?
If we want to keep feature debugging code, there should be a configure switch within i915 configuration to enable the bits.
CC'd Daniel and Chris as this relates to the fault injection code too.
Something along CONFIG_I915_DEBUG_SCHEDULER and CONFIG_I915_DEBUG_FAULT_INJECT could be added?
Regards, Joonas
>
> On 07/03/2016 12:31, Joonas Lahtinen wrote:
> >
> > Hi,
> >
> > On to, 2016-02-18 at 14:27 +0000, John.C.Harrison@Intel.com wrote:
> > >
> > > From: John Harrison <John.C.Harrison@Intel.com>
> > >
> > > When debugging batch buffer submission issues, it is useful to be able
> > > to see what the current state of the scheduler is. This change adds
> > > functions for decoding the internal scheduler state and reporting it.
> > >
> > > v3: Updated a debug message with the new state_str() function.
> > >
> > > v4: Wrapped some long lines to keep the style checker happy. Removed
> > > the fence/sync code as that will now be part of a separate patch series.
> > >
> > > v5: Removed forward declarations and white space. Added documentation.
> > > [Joonas Lahtinen]
> > >
> > > Also squashed in later patch to add seqno information from the start.
> > > It was only being added in a separate patch due to historical reasons
> > > which have since gone away.
> > >
> > > For: VIZ-1587
> > > Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> > > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > > ---
> > > drivers/gpu/drm/i915/i915_scheduler.c | 302 +++++++++++++++++++++++++++++++++-
> > > drivers/gpu/drm/i915/i915_scheduler.h | 15 ++
> > > 2 files changed, 315 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> > > index f7f29d5..d0eed52 100644
> > > --- a/drivers/gpu/drm/i915/i915_scheduler.c
> > > +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> > > @@ -40,6 +40,117 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
> > > return dev_priv->scheduler != NULL;
> > > }
> > >
> > These sort of functions should be in i915_debugfs.c, so that nobody
> > even thinks of using them for other purposes.
> >
> > >
> > >
> > > +const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node)
> > > +{
> > > + static char str[50];
> > > + char *ptr = str;
> > > +
> > > + *(ptr++) = node->bumped ? 'B' : '-',
> > > + *(ptr++) = i915_gem_request_completed(node->params.request) ? 'C' : '-';
> > > +
> > Especially this kind of code needs to be in i915_debugfs.c. I'd
> > implement new code more along lines of i915_sseu_status(), but I see
> > this kind of code was previously merged.
> >
> > >
> > > + *ptr = 0;
> > > +
> > > + return str;
> > > +}
> > > +
> > > +char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status)
> > > +{
> > > + switch (status) {
> > > + case i915_sqs_none:
> > > + return 'N';
> > > +
> > > + case i915_sqs_queued:
> > > + return 'Q';
> > > +
> > > + case i915_sqs_popped:
> > > + return 'X';
> > > +
> > > + case i915_sqs_flying:
> > > + return 'F';
> > > +
> > > + case i915_sqs_complete:
> > > + return 'C';
> > > +
> > > + case i915_sqs_dead:
> > > + return 'D';
> > > +
> > > + default:
> > > + break;
> > > + }
> > > +
> > Bad indent.
> >
> > >
> > > + return '?';
> > > +}
> > > +
> > > +const char *i915_scheduler_queue_status_str(
> > > + enum i915_scheduler_queue_status status)
> > > +{
> > > + static char str[50];
> > > +
> > > + switch (status) {
> > > + case i915_sqs_none:
> > > + return "None";
> > > +
> > > + case i915_sqs_queued:
> > > + return "Queued";
> > > +
> > > + case i915_sqs_popped:
> > > + return "Popped";
> > > +
> > > + case i915_sqs_flying:
> > > + return "Flying";
> > > +
> > > + case i915_sqs_complete:
> > > + return "Complete";
> > > +
> > > + case i915_sqs_dead:
> > > + return "Dead";
> > > +
> > > + default:
> > > + break;
> > > + }
> > > +
> > > + sprintf(str, "[Unknown_%d!]", status);
> > > + return str;
> > > +}
> > > +
> > > +const char *i915_scheduler_flag_str(uint32_t flags)
> > > +{
> > > + static char str[100];
> > > + char *ptr = str;
> > > +
> > > + *ptr = 0;
> > > +
> > > +#define TEST_FLAG(flag, msg) \
> > > + do { \
> > > + if (flags & (flag)) { \
> > > + strcpy(ptr, msg); \
> > > + ptr += strlen(ptr); \
> > > + flags &= ~(flag); \
> > > + } \
> > > + } while (0)
> > > +
> > > + TEST_FLAG(i915_sf_interrupts_enabled, "IntOn|");
> > > + TEST_FLAG(i915_sf_submitting, "Submitting|");
> > > + TEST_FLAG(i915_sf_dump_force, "DumpForce|");
> > > + TEST_FLAG(i915_sf_dump_details, "DumpDetails|");
> > > + TEST_FLAG(i915_sf_dump_dependencies, "DumpDeps|");
> > > + TEST_FLAG(i915_sf_dump_seqno, "DumpSeqno|");
> > > +
> > > +#undef TEST_FLAG
> > > +
> > > + if (flags) {
> > > + sprintf(ptr, "Unknown_0x%X!", flags);
> > > + ptr += strlen(ptr);
> > > + }
> > > +
> > > + if (ptr == str)
> > > + strcpy(str, "-");
> > > + else
> > > + ptr[-1] = 0;
> > > +
> > > + return str;
> > > +};
> > > +
> > > /**
> > > * i915_scheduler_init - Initialise the scheduler.
> > > * @dev: DRM device
> > > @@ -1024,6 +1135,193 @@ void i915_scheduler_work_handler(struct work_struct *work)
> > > i915_scheduler_process_work(ring);
> > > }
> > >
> > > +static int i915_scheduler_dump_locked(struct intel_engine_cs *ring,
> > > + const char *msg)
> > > +{
> > > + struct drm_i915_private *dev_priv = ring->dev->dev_private;
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + struct i915_scheduler_queue_entry *node;
> > > + int flying = 0, queued = 0, complete = 0, other = 0;
> > > + static int old_flying = -1, old_queued = -1, old_complete = -1;
> > > + bool b_dump;
> > > + char brkt[2] = { '<', '>' };
> > > +
> > > + if (!ring)
> > > + return -EINVAL;
> > > +
> > > + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> > > + if (I915_SQS_IS_QUEUED(node))
> > > + queued++;
> > > + else if (I915_SQS_IS_FLYING(node))
> > > + flying++;
> > > + else if (I915_SQS_IS_COMPLETE(node))
> > > + complete++;
> > > + else
> > > + other++;
> > > + }
> > > +
> > > + b_dump = (flying != old_flying) ||
> > > + (queued != old_queued) ||
> > > + (complete != old_complete);
> > > + if (scheduler->flags[ring->id] & i915_sf_dump_force) {
> > > + if (!b_dump) {
> > > + b_dump = true;
> > > + brkt[0] = '{';
> > > + brkt[1] = '}';
> > > + }
> > > +
> > > + scheduler->flags[ring->id] &= ~i915_sf_dump_force;
> > > + }
> > > +
> > > + if (b_dump) {
> > > + old_flying = flying;
> > > + old_queued = queued;
> > > + old_complete = complete;
> > > + DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d, "
> > > + "Flags = %s, Next = %d:%d %c%s%c\n",
> > > + ring->name, queued, flying, complete, other,
> > > + i915_scheduler_flag_str(scheduler->flags[ring->id]),
> > > + dev_priv->request_uniq, dev_priv->next_seqno,
> > > + brkt[0], msg, brkt[1]);
> > Convert to debugfs, it's the de-facto.
> >
> > >
> > > + } else {
> > > + /*DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d"
> > > + ", Flags = %s, Next = %d:%d [%s]\n",
> > > + ring->name,
> > > + queued, flying, complete, other,
> > > + i915_scheduler_flag_str(scheduler->flags[ring->id]),
> > > + dev_priv->request_uniq, dev_priv->next_seqno, msg); */
> > > +
> > Dead code to be removed from patches.
> >
> > >
> > > + return 0;
> > > + }
> > > +
> > > + if (scheduler->flags[ring->id] & i915_sf_dump_seqno) {
> > > + uint32_t seqno;
> > > +
> > > + seqno = ring->get_seqno(ring, true);
> > > +
> > > + DRM_DEBUG_DRIVER("<%s> Seqno = %d\n", ring->name, seqno);
> > > + }
> > > +
> > > + if (scheduler->flags[ring->id] & i915_sf_dump_details) {
> > > + int i, deps;
> > > + uint32_t count, counts[i915_sqs_MAX];
> > > +
> > > + memset(counts, 0x00, sizeof(counts));
> > > +
> > > + list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
> > > + if (node->status < i915_sqs_MAX) {
> > > + count = counts[node->status]++;
> > > + } else {
> > > + DRM_DEBUG_DRIVER("<%s> Unknown status: %d!\n",
> > > + ring->name, node->status);
> > > + count = -1;
> > > + }
> > > +
> > > + deps = 0;
> > > + for (i = 0; i < node->num_deps; i++)
> > > + if (i915_scheduler_is_dependency_valid(node, i))
> > > + deps++;
> > > +
> > > + DRM_DEBUG_DRIVER("<%s> %c:%02d> uniq = %d, seqno"
> > > + " = %d/%s, deps = %d / %d, %s [pri = "
> > > + "%4d]\n", ring->name,
> > > + i915_scheduler_queue_status_chr(node->status),
> > > + count,
> > > + node->params.request->uniq,
> > > + node->params.request->seqno,
> > > + node->params.ring->name,
> > > + deps, node->num_deps,
> > > + i915_qe_state_str(node),
> > > + node->priority);
> > > +
> > > + if ((scheduler->flags[ring->id] & i915_sf_dump_dependencies)
> > > + == 0)
> > > + continue;
> > > +
> > > + for (i = 0; i < node->num_deps; i++)
> > > + if (node->dep_list[i])
> > > + DRM_DEBUG_DRIVER("<%s> |-%c:"
> > > + "%02d%c uniq = %d, seqno = %d/%s, %s [pri = %4d]\n",
> > > + ring->name,
> > > + i915_scheduler_queue_status_chr(node->dep_list[i]->status),
> > > + i,
> > > + i915_scheduler_is_dependency_valid(node, i)
> > > + ? '>' : '#',
> > > + node->dep_list[i]->params.request->uniq,
> > > + node->dep_list[i]->params.request->seqno,
> > > + node->dep_list[i]->params.ring->name,
> > > + i915_qe_state_str(node->dep_list[i]),
> > > + node->dep_list[i]->priority);
> > > + }
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +/**
> > > + * i915_scheduler_dump - dump the scheduler's internal state to the debug log.
> > > + * @ring: Ring to dump info for
> > > + * @msg: A reason why it is being dumped
> > > + * For debugging purposes, it can be very useful to see the internal state of
> > > + * the scheduler for a given ring.
> > > + */
> > > +int i915_scheduler_dump(struct intel_engine_cs *ring, const char *msg)
> > > +{
> > > + struct drm_i915_private *dev_priv = ring->dev->dev_private;
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + unsigned long flags;
> > > + int ret;
> > > +
> > > + spin_lock_irqsave(&scheduler->lock, flags);
> > > + ret = i915_scheduler_dump_locked(ring, msg);
> > > + spin_unlock_irqrestore(&scheduler->lock, flags);
> > > +
> > > + return ret;
> > > +}
> > > +
> > This kind of stuff especially should be static functions in
> > i915_debugfs.c. And then they don't need kerneldoc either, because we
> > expect no sane invidual to call them.
> >
> > >
> > > +static int i915_scheduler_dump_all_locked(struct drm_device *dev,
> > > + const char *msg)
> > > +{
> > > + struct drm_i915_private *dev_priv = dev->dev_private;
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + struct intel_engine_cs *ring;
> > > + int i, r, ret = 0;
> > > +
> > > + for_each_ring(ring, dev_priv, i) {
> > > + scheduler->flags[ring->id] |= i915_sf_dump_force |
> > > + i915_sf_dump_details |
> > > + i915_sf_dump_seqno |
> > > + i915_sf_dump_dependencies;
> > This looks really bad to me, invasive debugging. Rather to be carried
> > as function parameters in the debug functions, and they should be
> > uppercase enum in i915_debugfs.c too.
> >
> > >
> > > + r = i915_scheduler_dump_locked(ring, msg);
> > > + if (ret == 0)
> > > + ret = r;
> > > + }
> > > +
> > > + return ret;
> > > +}
> > > +
> > > +/**
> > > + * i915_scheduler_dump_all - dump the scheduler's internal state to the debug
> > > + * log.
> > > + * @dev: DRM device
> > > + * @msg: A reason why it is being dumped
> > > + * For debugging purposes, it can be very useful to see the internal state of
> > > + * the scheduler.
> > > + */
> > > +int i915_scheduler_dump_all(struct drm_device *dev, const char *msg)
> > > +{
> > > + struct drm_i915_private *dev_priv = dev->dev_private;
> > > + struct i915_scheduler *scheduler = dev_priv->scheduler;
> > > + unsigned long flags;
> > > + int ret;
> > > +
> > > + spin_lock_irqsave(&scheduler->lock, flags);
> > > + ret = i915_scheduler_dump_all_locked(dev, msg);
> > > + spin_unlock_irqrestore(&scheduler->lock, flags);
> > > +
> > > + return ret;
> > > +}
> > > +
> > > static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
> > > bool is_locked)
> > > {
> > > @@ -1246,10 +1544,10 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
> > > continue;
> > >
> > > if (!I915_SQS_IS_COMPLETE(node))
> > > - DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
> > > + DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%s on %s\n",
> > > node->params.request->uniq,
> > > node->params.request->seqno,
> > > - node->status,
> > > + i915_qe_state_str(node),
> > If the code is to be called from outside of i915_debugfs.c, it should
> > really be written in a more safe manner than static char, especially as
> > it is not much of an effort to make a table to index.
> >
> > Regards, Joonas
> >
> > >
> > > ring->name);
> > >
> > > i915_scheduler_file_queue_dec(node->params.file);
> > > diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> > > index b78de12..a071ebb 100644
> > > --- a/drivers/gpu/drm/i915/i915_scheduler.h
> > > +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> > > @@ -41,6 +41,9 @@ enum i915_scheduler_queue_status {
> > > /* Limit value for use with arrays/loops */
> > > i915_sqs_MAX
> > > };
> > > +char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status);
> > > +const char *i915_scheduler_queue_status_str(
> > > + enum i915_scheduler_queue_status status);
> > >
> > > #define I915_SQS_IS_QUEUED(node) (((node)->status == i915_sqs_queued))
> > > #define I915_SQS_IS_FLYING(node) (((node)->status == i915_sqs_flying))
> > > @@ -65,6 +68,7 @@ struct i915_scheduler_queue_entry {
> > > unsigned long stamp;
> > > struct list_head link;
> > > };
> > > +const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
> > >
> > > struct i915_scheduler {
> > > struct list_head node_queue[I915_NUM_RINGS];
> > > @@ -82,9 +86,17 @@ struct i915_scheduler {
> > >
> > > /* Flag bits for i915_scheduler::flags */
> > > enum {
> > > + /* Internal state */
> > > i915_sf_interrupts_enabled = (1 << 0),
> > > i915_sf_submitting = (1 << 1),
> > > +
> > > + /* Dump/debug flags */
> > > + i915_sf_dump_force = (1 << 8),
> > > + i915_sf_dump_details = (1 << 9),
> > > + i915_sf_dump_dependencies = (1 << 10),
> > > + i915_sf_dump_seqno = (1 << 11),
> > > };
> > > +const char *i915_scheduler_flag_str(uint32_t flags);
> > >
> > > bool i915_scheduler_is_enabled(struct drm_device *dev);
> > > int i915_scheduler_init(struct drm_device *dev);
> > > @@ -99,6 +111,9 @@ void i915_scheduler_work_handler(struct work_struct *work);
> > > int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
> > > int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
> > > unsigned long stamp, bool is_locked);
> > > +int i915_scheduler_dump(struct intel_engine_cs *ring,
> > > + const char *msg);
> > > +int i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
> > > bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
> > > bool *completed, bool *busy);
> > > bool i915_scheduler_file_queue_is_full(struct drm_file *file);
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 82+ messages in thread
end of thread, other threads:[~2016-03-15 10:53 UTC | newest]
Thread overview: 82+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-18 14:26 [PATCH v5 00/35] GPU scheduler for i915 driver John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 01/35] drm/i915: Add total count to context status debugfs output John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 02/35] drm/i915: Prelude to splitting i915_gem_do_execbuffer in two John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 03/35] drm/i915: Split i915_dem_do_execbuffer() in half John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 04/35] drm/i915: Cache request pointer in *_submission_final() John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 05/35] drm/i915: Re-instate request->uniq because it is extremely useful John.C.Harrison
2016-02-18 14:26 ` [PATCH v5 06/35] drm/i915: Start of GPU scheduler John.C.Harrison
2016-02-19 13:03 ` Joonas Lahtinen
2016-02-19 17:03 ` John Harrison
2016-02-26 9:13 ` Joonas Lahtinen
2016-02-26 14:18 ` John Harrison
2016-02-18 14:26 ` [PATCH v5 07/35] drm/i915: Prepare retire_requests to handle out-of-order seqnos John.C.Harrison
2016-02-19 19:23 ` Jesse Barnes
2016-02-18 14:26 ` [PATCH v5 08/35] drm/i915: Disable hardware semaphores when GPU scheduler is enabled John.C.Harrison
2016-02-19 19:27 ` Jesse Barnes
2016-02-18 14:26 ` [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled John.C.Harrison
2016-02-19 19:28 ` Jesse Barnes
2016-02-19 19:53 ` Ville Syrjälä
2016-02-19 20:01 ` Jesse Barnes
2016-02-22 9:41 ` Lankhorst, Maarten
2016-02-22 12:53 ` John Harrison
2016-02-20 9:22 ` Chris Wilson
2016-02-22 20:42 ` Jesse Barnes
2016-02-23 11:16 ` Chris Wilson
2016-02-18 14:26 ` [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles John.C.Harrison
2016-03-01 8:59 ` Joonas Lahtinen
2016-03-01 14:52 ` John Harrison
2016-02-18 14:26 ` [PATCH v5 11/35] drm/i915: Added scheduler hook into i915_gem_request_notify() John.C.Harrison
2016-03-01 9:10 ` Joonas Lahtinen
2016-02-18 14:27 ` [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler John.C.Harrison
2016-03-01 9:16 ` Joonas Lahtinen
2016-03-01 15:12 ` John Harrison
2016-02-18 14:27 ` [PATCH v5 13/35] drm/i915: Redirect execbuffer_final() via scheduler John.C.Harrison
2016-02-19 19:33 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 14/35] drm/i915: Keep the reserved space mechanism happy John.C.Harrison
2016-02-19 19:36 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 15/35] drm/i915: Added tracking/locking of batch buffer objects John.C.Harrison
2016-02-19 19:42 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 16/35] drm/i915: Hook scheduler node clean up into retire requests John.C.Harrison
2016-02-19 19:44 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls John.C.Harrison
2016-03-01 10:02 ` Joonas Lahtinen
2016-03-11 11:47 ` John Harrison
2016-02-18 14:27 ` [PATCH v5 18/35] drm/i915: Added scheduler support to page fault handler John.C.Harrison
2016-02-19 19:45 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions John.C.Harrison
2016-03-07 11:31 ` Joonas Lahtinen
2016-03-11 16:22 ` John Harrison
2016-02-18 14:27 ` [PATCH v5 20/35] drm/i915: Add scheduler hook to GPU reset John.C.Harrison
2016-02-23 20:27 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 21/35] drm/i915: Added a module parameter to allow the scheduler to be disabled John.C.Harrison
2016-02-23 20:29 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 22/35] drm/i915: Support for 'unflushed' ring idle John.C.Harrison
2016-02-23 20:35 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 23/35] drm/i915: Defer seqno allocation until actual hardware submission time John.C.Harrison
2016-03-07 12:16 ` Joonas Lahtinen
2016-02-18 14:27 ` [PATCH v5 24/35] drm/i915: Added trace points to scheduler John.C.Harrison
2016-02-23 20:42 ` Jesse Barnes
2016-02-23 20:42 ` Jesse Barnes
2016-02-26 15:55 ` John Harrison
2016-02-26 17:12 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle John.C.Harrison
2016-02-23 21:02 ` Jesse Barnes
2016-03-01 15:52 ` John Harrison
2016-02-18 14:27 ` [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters John.C.Harrison
2016-02-23 21:06 ` Jesse Barnes
2016-03-11 16:28 ` John Harrison
2016-03-11 17:25 ` Jesse Barnes
2016-02-18 14:27 ` [PATCH v5 27/35] drm/i915: Added debug state dump facilities to scheduler John.C.Harrison
2016-03-07 12:31 ` Joonas Lahtinen
2016-03-11 16:38 ` John Harrison
2016-03-15 10:53 ` Joonas Lahtinen
2016-02-18 14:27 ` [PATCH v5 28/35] drm/i915: Add early exit to execbuff_final() if insufficient ring space John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 29/35] drm/i915: Added scheduler statistic reporting to debugfs John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 30/35] drm/i915: Add scheduler support functions for TDR John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 31/35] drm/i915: Scheduler state dump via debugfs John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 32/35] drm/i915: Enable GPU scheduler by default John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 33/35] drm/i915: Add scheduling priority to per-context parameters John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 34/35] drm/i915: Add support for retro-actively banning batch buffers John.C.Harrison
2016-02-18 14:27 ` [PATCH v5 35/35] drm/i915: Allow scheduler to manage inter-ring object synchronisation John.C.Harrison
2016-02-18 14:27 ` [PATCH 01/20] igt/gem_ctx_param_basic: Updated to support scheduler priority interface John.C.Harrison
2016-02-18 15:30 ` ✗ Fi.CI.BAT: failure for GPU scheduler for i915 driver Patchwork
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.